Sync in background

ChristianKl · 2024-05-14T02:21:11.399Z

I setup Joplin for Android for a new phone and a lot of data has to be synced, so ideally Joplin would to that in the background. When I click on "Battery Usage" it shows Screen time: 47 minutes for Joplin but no background time while it for example shows for Oura "Screen time: 11 minutes Background 25 min". To me that sounds like Joplin does not work in the background to do it's syncing.

Maybe, the sync work is not be doing in an Android Service (which can work in the background) but an Android Activity (which does nothing when it's not on the screen)?

muzak · 2024-05-14T02:37:08.932Z

Joplin mobile apps don't support background sync yet. If you want more info, I'd suggest finding the many background sync threads already on the forum. As far as workarounds go:

Joplin Sync Issues

Try using a caffeine app or solutions from Don't kill my app! to keep your phone screen on while Joplin syncs.

ChristianKl · 2024-05-14T08:32:27.763Z

That thread you linked suggests that the developers don't know what they are doing.

android - How to keep the application running in background react native - Stack Overflow gives various options of how you can do background tasks in Android in ReactNative.

dpoulton · 2024-05-14T10:39:27.321Z

ChristianKl:

That thread you linked suggests that the developers don't know what they are doing.

I believe that none of those in the thread are Joplin contributors.

ChristianKl · 2024-05-15T09:58:08.580Z

graphit0 seems to suggest that this is a priority issue which seems to be misinformation.

There's currently an open Github ticket which got relatively little attention and is not a high-priority ticket.

laurent · 2024-05-15T10:29:39.058Z

It's high priority, but the solutions we tried so far didn't work out unfortunately.

ChristianKl · 2024-05-15T13:38:52.623Z

background sync on mobile devices · Issue #3872 · laurent22/joplin · GitHub did add background search and could have been integrated even if it's not perfect.

With that solution, you have two remaining issues:
(1) Doing the initial sync that might take hours in the background.
(2) Background tasks that are scheduled at a frequency of every 15 minutes are only allowed to run that often if the user opens the app.
(3) Handle the case of what the app does when a background sync is in progress when the user opens the app.

You can solve the first issue by putting up a notification on Android. As long as an app shows a notification to tell the user that it's doing work in the background it's allowed to run in the background.

On Android the second issue is a mix of informing the user, that Android might reduce the frequency of the syncs, and if the user wants to prevent that they can turn off battery optimization for Joplin.

The third issue is just a matter of writing code. No rocket science.

I'm an Android dev so I don't know about how far you could get on iOS but even if the feature isn't perfect on iOS it's likely still an improvement over the status quo.

As far as people complaining about sync not being perfect, they are already complaining about sync not being perfect and expect it to sync in the background whether or not rudimentary background sync is available.

If it's high-priority, what's stopping it from being implemented on Android?

graphit0 · 2024-05-15T14:04:58.664Z

Rephrasing point stated before

Why not just ship cheap and ~~dirty~~ flaky background sync on ReactNative?

Let me state clearly that I feel your pain and frustration. However, we must consider the open source nature of Joplin. All flakiness, user support and maintenance burden will lie entirely on maintainer and community. All lost data will be permanently tied to Joplin reputation, not the contributor who submitted the PR.

Once again I want to say that we, as a community, see and know the problem -- we respond to user support requests popping up weekly. This is unfortunate situation for both devs and users. However, this is the path Joplin team decided is best. As a fellow developer you, @ChristianKl, might have your own tough decisions made out of necessity which had to sticked to and I hope you could empathize with this one.

Leaving things on more productive note in my mind background sync could be best achieved by completing the prerequisite standalone sync API which was proposed in past GSoC

About the Standalone Sync API category Standalone Sync API

(Replace this first paragraph with a brief description of your new category. This guidance will appear in the category selection area, so try to keep it below 200 characters.) Use the following paragraphs for a longer description, or to establish category guidelines or rules: Why should people use this category? What is it for? How exactly is this different than the other categories we already have? What should topics in this category generally contain? Do we need this category? Can…

Additionally in this post I linked what implementations were already developed which hopefully could be of use to potential contributors

Standalone Sync API Features

Hi, I saw Standalone Sync API on GSoC 2024 ideas list. I see it shouldn't use dependencies from main app, but it needs to interact with local database (SQLite3) for syncing. I want to ask how abstract this library should be? Should It take any arbitrary database interface, or works on joplin sqlite database schema only.

Yet another project of note is Joplin with background sync service by @Chen

Background sync in android Features

Hi, I am a joplin user. Plan to make a patch to let joplin do background sync in android. I plan to use HeadlessJS of React Native to achive this purpose. The code size will be quite big. Per guild in source, I need to start discussion in this forum. So I created this topic. If you have any question about it, please ask it. Regards, Chen Yang

P.S. if there's something in this post that seems misleading or false, please let me know

laurent · 2024-05-15T14:10:56.227Z

ChristianKl:

I'm an Android dev so I don't know about how far you could get on iOS but even if the feature isn't perfect on iOS it's likely still an improvement over the status quo.

If you're an Android dev we would definitely welcome a pull request from you.

In fact I am not an Android developer but I saw the mess that React Native can be in some conditions. What I suspect is that with background sync we might end up with two instances of Joplin running in parallel and trying to sync at the same time which will result in data loss. Or maybe not, I'm not an Android developer after all. It seems there are lots of people who know a lot of things, but the problem is who has the time to write this code and maintain it - provide support, sometimes urgently, when it breaks and we need to release a new version.

ChristianKl · 2024-05-15T15:10:53.493Z

I have done Android and Flutter but I have not worked with ReactNative before.

What I suspect is that with background sync we might end up with two instances of Joplin running in parallel and trying to sync at the same time which will result in data loss.

The word "instance" doesn't translate that well into Android. In Android, you have an app and within an app you have services and activities. Services are background processes and activities are foreground processes. In good old Android whenever the user switches the screen the user switches to a different Activity.

I would expect that if the user open Joplin (an Activity) while the background process (an Service) is going on both run in parallel but are one app.

To prevent conflicts with synchronization, there are two ways to deal with it. Joplin currently does sync-locks.The sync-lock could specify whether it's a background or foreground sync-lock. That way when a background sync process happens, Joplin does not start a foreground sync process in parallel.

The other more clean (and more Android specific) solution would be that the synchronization is always done by the Service and not by the Activity and when the Activity starts the Activity sends an message to the Service to start syncing and the Service just continues syncing when it already syncs.

It seems like @Chen already did some work, so maybe it will lead to a satisfactory solution.

roman_r_m · 2024-05-15T18:39:12.914Z

graphit0:

Leaving things on more productive note in my mind background sync could be best achieved by completing the prerequisite standalone sync API

There's very little relation between the two, so don't get your hopes too high