I have about 4000 notes on Joplin's windows desktop app, which sync to another windows device. But they will not sync to my Android phone. Following advice in this forum, I uninstalled my Android Joplin app, rebooted my phone and reinstalled the app. I have since tried to sync 4 times. Each time the same thing occurs: the android app fetches items, then freezes at various points (e.g,. Fetched items: 1221/1250, Fetched items: 1742/1750, Fetched items: 1942:1950). Each time, then sync process preceeds a bit futher and it shows "create local items" totaling between 40 and 77
Any suggestions for how to get this moving to completion? Thanks for your help!
Do you mean that the sync process ends prematurely with the "Create local items" action? If not, try letting sync run to completion. From my experience, restarting the app after "Fetched items" freezes can unfreeze the counter without erasing sync progress.
Be careful sharing logs if using the OneDrive sync target — see token logging issue.
Qualifications: 1. Joplin didn't log entire authorization tokens (just the start, but still a large portion of the token) and 2. these tokens should expire after a few days. (To be safe, though, I still suggest reviewing the log.)
Questions
Does the UI freeze? (For example, can notes be opened and edited?)
Thanks for your responses. Sorry for the delay, but I was out of town at a family wedding and returned today.
The UI does not freeze, but the fetched items count stops counting. Encryption is enabled and notes have no problem syncing in other devices. I've run the sync over a dozen times. Once it went over 5000 items but then it stopped. Often it stops when the count is far less. Once, I let it sit for ~3 hours without the count increasing, although the sync icon continued to spin (as it always does). It does not recreate all of my notebooks and does not download all notes.
The only notebook that did sync is called "Conflicts (attachments)," which exists on my other devices. It's there because I'm trying to figure out what to do with the entries. See Massive Conflicts
I haven't posted the logs due to warning that was provided.
I appreciate your advice and assistance. I love Joplin and this community but I'm getting close to my limit on the amount of frustration and time I'm willing to endure to make this work. Thanks again for your time. It's a valuable commodity and I'm grateful for the help.
I tried a new Joplin installation on another device, this one an iPad. The exact same thing occurred on my iPad as on my Android phone: the sync began, counted fetched items over 2000, then stopped counting and ended the sync hours later. Of course, little of the data from my other Joplin installations was transferred to my iPad.
I've had this happen enough times on enough devices (and read forum posts from many others with similar problems) that this seems like a much bigger problem. I hope @laurent and other Joplin developers will put fixing sync on their 2024 to-do list. I know this is an easy ask for someone with no coding skills but I think it's critical to Joplin's success.
As things stand, Joplin is unusable on 2 of the 4 devices on which I need it. I play with my devices a little more, and see if I can get this working.
Have you tried changing "attachment download behavior" from "always" to "manual"? If a very large attachment is causing the issue, this change might help.
I think this change would need to be made before syncing.
This option can be found under the "Synchronization" in settings on mobile.
On desktop, you can see the size of attachments by clicking on "Tools", then "Note attachments", then clicking the arrow by "size" to sort by size.
Thanks for that suggestion. Unfortunately, I had already switched the attachment sync setting to manual as part of my own troubleshooting. The problems I've described all occurred in that mode, so it's not the attachments that are causing my problems.
I'm going to run sync on my Android phone and iPad today and see if I get different results. If not, I'll delete my installations and start anew one more time.
One other puzzle piece: my 'main' Joplin install (on a Windows 11 desktop) has a banner warning "Some items cannot be synchronized. View them now." It looks like there are three .png files causing the problem, although I have no idea to which notes those files are located. I've clicked "retry" multiple times, but it does not work. Could these files be causing my syncing problems on Android and iPad?
I've opened my attachments using the Tools menu and can find the three .png files. They appear 10 times each, with each file tied to a separate ID. Any idea how to locate and delete the offending files so I can get rid of the error message?
I also noticed that I have many duplicate (triplicate, quadruplicate, etc.) attachments. I'd love to cull them, too.
On my windows devices that are syncing properly, I've cleared all the conflicts and the items that could not be synchronized. I've uninstalled and reinstalled the Joplin app multiple times on Android and iPad.
Even after all that effort, those 2 devices will not sync. They begin the sync process, then simply stop at some point, leaving me with some notes visible but without any notebooks or the vast majority of my notes.
This can't be how Joplin works. If anyone has suggestions, please share them. I've never been so frustrated with an app!
Just to make sure that you're aware: Joplin doesn't have background sync on mobile devices -- therefore it essential to sync the app on the foreground with screen unlocked. Better to turn off the auto lock screen for first initial sync. First sync may take some time (few hours).
Interrupting, exiting app, locking screen etc is not recommended during first sync. It may result in broken sync that's difficult to reverse, so it's best to do initial sync on the first try or reinstall the app.
Thank you for pointing that out, but I was already aware of that limitation. My problems with syncing my Android and iPad have occurred while the screens are continually awake. In fact, I have watched too many synchronizations in the past few week. So, unfortunately, this is not the problem.
Ok, let's try different cloud. i.e. create different profile with the same data (export your main profile and import it in new profile), sync it to different cloud provider (dropbox for example). Then try to sync one of the problematic devices there.
I did as you suggested with different, but unsuccessful results. After creating the new profile and duplicating my data in it, I moved the sync to Dropbox. I uninstalled Joplin from my Android phone and reinstalled it to start fresh. The sync seemed to run better. It fetched 11,336 items and created 3223 local items. But that's the end of the good news. None of my notebooks were recreated on the phone. All I ended up with was two notebooks of "Attachments (Conflicts)" and not much else. Some of my notes came over but without any tags or notebook assignments.
Bottom line: I don't think the sync target is the source of my problems.
Thank you so much for your suggestions. Your time is greatly appreciated.
I get the sync stopped at 11k elements. Has it froze as always or it ran till "completed" status? Have you tried to start the sync again after that?
Just to confirm, have you duplicated your data into new profile via jex export? Or by some other means?
Also since there's no logs, can you briefly look at them yourself (in the moment of sync freeze) and note maybe there's an obvious error message explaining the behaviour.
I re-ran a sync on both profiles (Original and Duplicate) then checked the logs in both.
I first re-ran the Original Profile, which, shockingly, ran better than it has previously. It has routinely stopped when it got to about 2000 items. This time, before it quit, it got up over 6000 items and was constantly downloading new notes and decrypting them. (A little key icon was shown next to an encrypted note before it disappeared and was replaced with text.) I have never observed this behavior before. When it quit, the log file error said:
Synchronizer: "Could not refresh lock - cancelling sync. Error was: ", "Error: Lock has expired Code: lockExpired
There was more detail, but that was the gist.
Then I ran the Duplicate Profile sync. It ran for 8 secs before showing that it was completed. Then it ran decrypting thousands of items in 100 item chunks. (I repeated this 3 times with the exact same results.) The log error had the same error about not refreshing the lock and that the lock expired.
Do these tea leaves give any clue as to what this means?
Here's an interesting twist on the Duplicate Profile . . . . On my Android Phone, Joplin shows only to "Conflicts (attachments)" notebooks. But on my Windows computer, all the the data has been duplicated. So while I originally had 5 notebooks, I now have 10 (2 of each). Each duplicate notebook contains the same number of notes (one with over 3,900).
To recap . . . the data won't sync to Android but it has duplicated on Windows.
It's already something. I think cloud providers are throttling file updates and eventually disabling the connection due to the amount of calls and size of tranfered resources.
That's still unusual but not unheard of.
Maybe if you repeatedly sync all the items, it will restore the folder structure, tags and decrypt all the notes. But I imagine it won't go through in one go due to size and number of items. So, after you get a freeze, you might wait some time (like few hours or a day) and try again.
On top of that, let's delete contents of folder Apps/Joplin/locks on Dropbox (.json files) -- if it's empty then no need to do anything else.
Then, on mobile device try switching (configuration -- sync --) "attachment download behavior" to "Auto". It would reduce the amount of data needed to syncronize and hopefully get less throttling by cloud provider.
Then let's repeatedly sync mobile device connected to dropbox.
this behavior is quite weird, indeed. Just to confirm, you haven't imported the data more than once, right? Each additional import doesn't merge notes, rather adds brand new ones. In any case, let's delete this unnecessary duplicated data so it wouldn't try to sync to mobile devices and increase the overall sync size.
graphit0, I started all over again, and this time I think we got it licked!
For anyone interested in what it took: I deleted the duplicate data in the desktop Duplicate Profile that you had me create. Then I deleted the entire Joplin installation on my Android Phone and reinstalled it from scratch. I then synced the Android. For unexplained reasons, this time, it ran to completion, creating 19931 local items and fetching 26273 items over 55 minutes. That was insane. But it did sync the notebooks from the Duplicate profile, in addition to two sets of "Conflict (attachments)" notebooks that had over 400 items. I don't understand the conflicts (none of which are resolvable through the Conflict Resolution plugin), so I exported them for safekeeping and then deleted them. After syncing again, both installations seemed accurate, except that the Android version had no Tags. After syncing TWICE more, the tags finally came across.
So now -- drum roll, please -- it looks like I have a fully synced version of my notes on both my desktop and my Android Phone. I now need to move new notes from my Original Profile to the Duplicate Profile, but that should be relatively painless. Then I will hope that this also works on my iPad. (For the record, my Original Profile continues to freeze and stop fetching items and will still not sync on Android.)
I am grateful beyond words for your suggestions and encouragement. I was ready to abandon ship and return to Evernote before getting your suggestion that I create a new profile and import the notes there. Thanks to you and everyone else who was kind enough to help me through this data nightmare, I am back to where I need to be!
