Sync issues

Operating system

Android

Joplin version

3.5.13

Desktop version info

windows 3.5.13 android latest

Sync target

Dropbox

What issue do you have?

sync not matching, notes missing, items not going back to desktop from mobile, how to diagnose and fix? sync on mobile never seems to finish. my most important note number 000 (so it sorts first) is not synchronized for some reason. what can i check to see if the sync has ever completed 100% on the mobile device - so far it looks like it has not as some old items are hanging around. please advise - urgent.

If the initial sync never finished on mobile, then uploads likely might not work either.

On the mobile app, because background sync is not supported, you must keep the screen on continuously and with the Joplin app in the foreground while it syncs. As the initial sync will be large, you will either need to temporarily disable screen timeout via the development settings on Android, while you leave for as long as it needs to sync, or download a third party app such as 'Caffeine', to toggle off screen timeout with a button.

The sync is complete when clicking the sync button completes within a few seconds and the status text does not mention any local or remote items created / updated / deleted.

Is there a particular reason why background sync is not supported? I don't recall having this issue with OneNote or with Evernote.

Because it's a React Native app, which makes it difficult to support background sync in a good way, without introducing further maintenance overhead (i.e maintaining separate native code for Android and iOS). The Obsidian mobile app has the same issue and even the Evernote mobile app since v10, unless the changed that since then. But bear in mind Evernote does not download all your data locally, so it doesn't need to do massive sync operations anyway

Can you explain why the joplin android app needs to be in the foreground? Is it that difficult to implement a background thread to do the sync? Joplin is still, very very slowly, decrypting my thousands of items; it's been at it for weeks!! Thanks for mentioning the caffeine app. I'm about to download it to hurry the decryption process.

Actually I just saw @mrjo118 's answer above explaining the problem (React implementation). Maybe it's time to change implementation languages? :slightly_smiling_face:

Okay, considering those limitations, there must be a solution. For example, I have settings on my Android to not move an app into the background if it's active. Isn't there a setting in Android that would allow this to happen on this particular case for this particular app? Because obviously they're doing things to manage memory and save battery, but if you need a long synchronization done, it seems like you should be able to enter a certain mode or some kind of avoidance of the background swap-out. Is that feasible? Another perhaps simpler solution would be for it to pick up where it left off on the synchronization, rather than starting over each time?

thanks for the tip - your solution worked for my sync issue.

Isn't there a setting in Android that would allow this to happen on this particular case for this particular app?

It's not as simple as that. The framework used to write the Joplin mobile apps has it's own limitations which can't be bypassed by disabling app optimisations on Android settings. The Android and iOS apps share the same code in order to make maintenance significantly easier, and to make full use of such OS level settings you must write native code (which means separate code for the Android and iOS apps).

The main issue is maintenance overhead. Yes it is technically possible to support background sync on any Android app, but an open source project takes an incredible amount of work to maintain, so can't support every feature, otherwise the project becomes unmanageable. This is indeed a highly requested feature, but adding it in an unreliable way would create more complaints, and adding it in a reliable way (native code) introduces a level of maintenance requirement which might be considered too much. Or alternatively, the core developers don't have the required skill set to implement this functionality natively (native app code uses different programming languages), and no volunteer has contributed a code change implemented this way.

Another perhaps simpler solution would be for it to pick up where it left off on the synchronization, rather than starting over each time?

This is only really a problem for the initial sync when a large number of items needs to be fetched, as this step restarts if the app is killed. For smaller syncs, this step usually does not take very long, and the rest of the sync process will continue where it left off if the app is killed off mid way through

I develop apps as well, and it seems to me what you're describing is simply a fork, meaning that one piece of code is implemented for one operating system, another piece of code is implemented for the other operating system. So if you write it once, then you're finished for both. But if my example of having spent three months waiting for a sync to finish with no notifications given is not a good example of why this should be fixed to produce a better user experience, then I don't know what to tell you. Yes, I did reload my conversion from OneNote several times, but that's because that's what they told me to do. And they said they had improved the import code, but they didn't have any way of merging the changes between the two versions of the import. Regardless, I've seen several apps that have a functionality to keep themselves alive during a long process by simply putting a message on the message bar that they are keeping themselves alive until they're finished. So perhaps that can be part of a future solution. But to me, it seems the simplest way is just to keep track of the last thing you synchronized, then check that after you return from being killed during sync, seems to be as simple as reading a number from storage, and then continue from there. My 0.02 USDC.

But to me, it seems the simplest way is just to keep track of the last thing you synchronized, then check that after you return from being killed during sync, seems to be as simple as reading a number from storage, and then continue from there

You can do this already as far as I'm aware. Just disable battery optimisations for Joplin in Android app settings and never manually kill the app

But if my example of having spent three months waiting for a sync to finish with no notifications given is not a good example of why this should be fixed to produce a better user experience, then I don't know what to tell you

Honestly because of the way the sync in Joplin is designed to work with third party services which you can only make CRUD operations on, which in many cases are throttled, the initial sync for a large number of sync items is always going to be slow (if you were to use Joplin cloud / server instead, they include some optimisations to the sync process). So if the sync was going to take 5 hours, while having a foreground service would indicate to the user that the sync is still ongoing, many users would still wonder why its taking so long unless they read about this on the forum or documentation and expected it. Also bear in mind that on Android 15+, the dataSync type for a foreground service can run for a maximum of 6 hours during a 24 hour period, and this limit may be reduced in future.

I have pushed for making in clearer in the documentation that you need to keep the screen on and the Joplin app in the foreground during the initial sync, though unfortunately that was rejected. There is mention of this limitation in the FAQ though.

Aside from the limitation not being obvious for the initial sync on mobile, I think what is more of an issue is no background sync for incremental syncing, but to some extent this can be mitigated by paying a bit of money and using a faster sync target. If you don't want to do that, you just have to make sure to manually trigger the sync on your phone when you want to switch to another device, if you want to make sure everything is in sync. Even if you don't disable battery optimisations on the Joplin app, these incremental syncs should sync pretty quickly, even when the sync starts over if the app is restarted.

My Joplin app on mobile is already set up for background operations. Point taken on dropbox throttle but it seems very fast in my tests.

Can you please tell me some benefits of having a Joplin server set up? I have no problem setting one up. I just wonder if it's going to be a good investment of time. Dropbox is working well. And my understanding is if the Joplin server goes down, you have no synchronization. Is that correct, or does it fall back to the standard way by switching back to Dropbox if it goes offline? The main reason i like dropbox is it is unlikely to go offline for long. thanks.

The main benefit of Joplin server is faster sync, but it also provides some additional features such as sharing and publishing notes.

If you have everything working on Dropbox now, don't care about the additional features and you're happy with the incremental sync speed, then there's no point changing to Joplin server at this point in time.

Using Joplin server would mean ongoing time investment maintaining your self hosted server, and although Joplin server can be used for free, it would still incur running costs in electricity to run your own server 24/7, or alternatively pay for a PaaS solution.