Joplin synchronized on premise through a server on a Debian machine and then connected to a desktop application, an Android smartphone, an Android tablet
Sync target
Joplin Server
What issue do you have?
I have Joplin server installed on a Debian machine - Joplin Server v3.0.1
I have Joplin installed on Ubuntu 24.04LTS - Joplin 3.1.24
I have Joplin installed on other mobile devices, mainly Android
On all the devices I set the synchronization with 5min interval
I started using Joplin since two months
On the desktop application on Ubuntu I set also a backup (this is to add more security in case of fail):
1 keep
6hr interval
I use Joplin to take notes from different devices. It's very rare I delete a note, but it happens to edit.
In the last two weeks or so I get some conflicts about some notes, mainly in the desktop application. Moreover I discovered that the short message, appearing at the bottom left about the sync, reports many times as it deletes some notes even if I never made any deletion.
That made me suspicious and it seems that some notes are uploaded on the server, then deleted but not from the device that created it for the first time.
So that note it is deleted from the server for no reason, and other devices cannot see it anymore, but it still stays on the device used to create it.
When I try to edit that specific note, that note goes in conflict.
Why do I get this behavior? Is it a bug or a configuration error?
I thought that this is due to the fact that I have also a backup on, but why? If it can help, I enabled the note history set at 90days
How to solve it?
I don't still know about which notes this happens
About your second question, I'm not sure about all the notes, but I noted a specific note that it was not on the other device, then I edited it on the desktop app and I discovered the conflict.
Moreover I did a random check for other notes, they are at least on two devices (one of them, the desktop app) but when I edited on the desktop app, I got anyway a conflict.
I cannot compare exactly the list of the notes, because the conflicting ones are moved again on their notebooks creating a new copy of it (one previously existing in the server, one recycled as a new one).
I don't know exactly how to manage the conflicts: I need help also on it
If you're unsure what a note share is, then it can be assumed the answer to my question is "no".
Could you share the log file for the desktop application? It will be in the folder which Joplin stores the local data on your computer. Please zip it before uploading so that it will allow uploading the whole log file.
Also, roughly how many notes have you had conflicts for in the last two weeks? Were there many or just a few?
There were masses of deletions on 10th December. I can see multiple instances of the default 'Welcome' notebook being deleted then, as well as various note names which you should recognise as your own notes.
Are you able to recall and describe what actions you took on that date? It looks like the majority of your notes were deleted from your device at that point, and I'm sure you would have noticed if that was the case in practice
It says in one there are 165notes, in another 169, in another 150
Still it is not clear to me the issue related to the conflict. Why does it happen?
What could be the cause?
Is there something related to the backup on the desktop app? Also today I got another deleting.
There is another deleting operation in thelog but I never did it for sure:
Thanks for this additional info. Annoyingly the deleteRemote lines in the log do not specify the item type. But I can see from earlier in the log that the deleted item in your last comment is a revision:
2024-11-21 14:52:09: Synchronizer: Sync: createRemote: remote does not exist, and local is new and has never been synced: Revision: (Local 3adbcc9af2a1414c861854acb7475fdc)
You can expect you see deletions of note revisions most days, because the note history has a cut off days, for which older entries will get deleted when the maintenance task runs.
I can see there are 12 matches to this string in the log, including some recent ones:
"Sync: noteConflict: remote has been deleted, but local has changes"
So your suspicion of the note conflicting due to being deleted looks to be correct, though I'm still not sure why at this point.
Could you also post the log file from the Android client for which you posted the sync status screen please? You can share the log to yourself via the app via email or something, then zip it and upload here. Then maybe one of suspect note ids can be matched on both clients to determine what might have happened.
Actually I just noticed you have 2 mobile clients on the screenshots. Please share the logs for both, as I don't know which of the clients causes the deletion
Actually they are smaller than the desktop one mobile.log (211.2 KB) tablet.log (107.0 KB)
Here they are attached.
What does "You can expect you see deletions of note revisions most days, because the note history has a cut off days, for which older entries will get deleted when the maintenance task runs." mean? So does note history causes the issue? It is set on 90days
Yes, the note history (revisions) will cause the synchronizer to say 'deleting' objects regularly, so you can ignore that, as it is not an indicator of missing notes.
But obviously when you're actually missing notes on some devices, that is an issue.
Ok I have found something of significance, which is actually in all 3 logs:
"JoplinServerApi: Error: Delta cursor is invalid and the complete data should be resynced"
This error is produced by the Joplin Server instance. I think this is a bug in the Joplin client, that there is incorrect handling of this error, which has code "resyncRequired". This error code has handling implemented exclusively by Joplin Server, Joplin Cloud and OneDrive sync targets
EDIT: Actually I see in the case of OneDrive, the resyncRequired error handling is within a block of code which is no longer used, so actually this issue is exclusive to Joplin Server / Joplin Cloud
I've understood that I should not worry about the deleting messages of notes I read in the update status but I've not understood how to manage the conflicts that happens and how to solve the issue in a definitive way.
Moreover I'll be cautious about possible deleted notes and reporting here if I'll encounter in the next few days.
Therefore I need answers to:
how to manage the conflicts I'll get?
how will I solve the conflict issue in a definitive way?
Is there something I can do about the discovered bug?
At this stage, considering that no notes seem to be deleted, the question 1 is the most important to me.
To manage the conflicts, if you're sure each note was not changed on another device since the last sync, then you can simply move those notes back to the notebook where they originated from without losing any content.
I don't understand what you mean by solving in a definitive way
I don't have an answer to that at this point. It requires more investigation to determine what causes the resyncRequired error to happen and why that could cause deletion of notes if that is actually the cause
I've managed to trace the activity of one of the conflicted note ids across multiple logs. The logs indicate that since the initial creation of the note, at some point both clients which contained that note, thought that the remote file had been deleted, without any indication that any of your clients had deleted it. But prior to that happening, there was a case of the resyncRequired error happening on 11th February.
tablet.log:
Reference to the note id is not present
mobile.log:
02-08T13:42:35: Synchronizer: Sync: createLocal: remote exists but local does not: (Remote 2857db64f58544f19603fd0d1e4811f3.md)
02-11T22:31:24: JoplinServerApi: Error: Delta cursor is invalid and the complete data should be resynced
02-18T08:08:13: DeleteAction: sync: handleConflictAction: note/resource conflict: titles: [""]; Item IDs: ["2857db64f58544f19603fd0d1e4811f3"]
02-18T08:08:13: Synchronizer: Sync: noteConflict: remote has been deleted, but local has changes: Note: (Local 2857db64f58544f19603fd0d1e4811f3)
02-18T08:08:13: Synchronizer: Sync: deleteRemote: local has been deleted: (Remote 6f8eb4d87578435995b426148220a3e8)
log.txt:
2025-02-07 16:28:17: Updating lat/long of note 2857db64f58544f19603fd0d1e4811f3
2025-02-07 16:29:16: Synchronizer: Sync: createRemote: remote does not exist, and local is new and has never been synced: Note: (Local 2857db64f58544f19603fd0d1e4811f3)
2025-02-11 21:26:53: JoplinServerApi: [warn] Error: Delta cursor is invalid and the complete data should be resynced
2025-02-18 09:29:49: Synchronizer: Sync: noteConflict: remote has been deleted, but local has changes: Note: (Local 2857db64f58544f19603fd0d1e4811f3)
2025-02-18 09:29:49: DeleteAction: sync: handleConflictAction: note/resource conflict: titles: ["Necessario avvio Dantherm"]; Item IDs: ["2857db64f58544f19603fd0d1e4811f3"]
This suggests one of two things:
Something external has caused the Joplin server to lose the note file (probably unlikely)
Whatever caused the resyncRequired error to occur has caused Joplin server to delete the note file, without explicit instruction from the Joplin client. So a potential bug in the Joplin server code, rather than the client.
I think there might be a Joplin server log file you can obtain, but I don't know the steps to do so. Do you know of such a log file and if so, could you share it here?
I cannot say anything about this note because I don't know how to refer to a specific note
Moreover I got access to the server web application and among the list of the existing items I cannot find it.
Moreover I found these tasks that I don't know what exactly do, if it can help:
If I deactivate the history note, will change something? Will the issue of continuous conflicts be solved? I noted that if I got a conflict on the desktop app and I put again in the correct notebook, then on the other devices I have two equal notes. I checked this behavior right now and on my tablet there are two equal notes, the same on the samrtphone. Are you sure that this is the correct way to manage it? Both the copies of notes are synchronized with the last updates.
By the way, the history version is visible only on the desktop app, not through the Android apps used by the other devices
The name of the note with id 2857db64f58544f19603fd0d1e4811f3 is "Necessario avvio Dantherm" according to the log.
Have a look in the reports tab of the Admin section, if it might contain something useful.
Changing note history settings is not going to make a difference. I can't suggest any workaround at this point unfortunately, as I haven't been able to determine the root cause. I did some further investigation yesterday and I think the resyncRequired error is a red herring, as I read the documentation about it and I haven't found any issues in the code producing and handling it.
Thank you for confirming that the note id does not currently exist on the server. From all the information I have now, all I can determine is that 'something' has caused a deletion of the said note on Joplin server which was not triggered by any of the Joplin clients (if they did, they would have produced a relevant log entry including the note id).
Ok, so I don't know the reason you ended up with this problem, however from the screenshots of your sync status page it is clear that there is a sync issue with some notes, eg. on the mobile status it says "Note: 158/165" and "Conflicted: 0". So that means there were 7 notes which were not synced at that point and are note current conflicts either (I assume you ran the sync before opening the sync status?) and for desktop 165/169 means there were 4 notes not synced at that point.
In order for the number on the left side of the slash to be lower than the right hand side, it means that items exist with no entry in the sync_items table of the local database for the client. Making a change to a note should trigger creation of an item in the sync_items table if it does not exist already for the current sync target, but somehow some of your notes managed to bypass the initial creation of the sync_items entry on creation / download of the note from another client.
I think your best course of action would be to do a complete clear down of all Joplin data on clients and server, and then re-import your data from a consolidated backup, using these steps:
-Do a full jex export on all 3 devices
-On Joplin desktop, create 3 new profiles (don't setup sync for any of them, but delete the default welcome notebook) and import each jex in each one
-Manually consolidate all of your notes into 1 of the profile (a pain I know, but fortunately you don't have masses of notes) and make sure the tags and resources work correctly as well
-Once you're happy with it, create a new jex export from this profile and keep it somewhere safe, as this will be your golden copy
-Delete all profiles on Joplin desktop, except for the golden copy
-Delete all data from Joplin server, or alternatively set up a new user and use that as your new account. You can then delete the original user and it's data should get deleted by the scheduled maintenance task
-Configure your remaining Joplin desktop profile to sync to Joplin server
-Uninstall and reinstall Joplin on mobile and tablet and re-sync the data from scratch
-Going forward, periodically check the sync status page making sure that all items get synced and the values are consistent across device after syncing up. Report back here if issues re-occur
Please do get the log and post here please, in case it offers any additional insight of the root of the issue.
But I think you will need to reset everything as per my steps regardless of any insight those logs may provide, because frankly your profile has got into a mess somehow, in order for the sync statuses to vary so much between clients
About the full procedure to reset everything I cannot do these days because I'm around and I'll not have access to all the devices. Maybe in some weeks
About getting the log file, I think I can do. I've not understood which container. I have joplin-app-1 and joplin-db-1 About the first command, should I type:
