Hello,
I was surprised when I found out that Joplin looses my data once in a while. From a distributed collaborative notebook I would expect a fault-safe protocol to handle user data, or at least some conflict management. In my case, I lost data over some month, but it took a while to realize this.
My setup:
Synchonization via Nextcloud / WebDAV
Clients on PCs and Android
Observed behavior:
1 Update content on PC
2 Open Android App: Updates are missing (e.g. no WAN connection due to limited WiFi)
3 Changes in the Android App then overwrite the PC content, without notice or warning or conflict handling
Questions:
Is this considered an error or is "normal"?
Is this not possible with the Joplin server?
Proposal:
Implement a protocol that is safe handling data conflicts or at least implement conflict detection and some manual conflict resolution.
I really came to love Joplin, but I cannot work with an App that looses my data.
Hopefully, there is a solution to this problem.
Regards, U. Ma.
Hi, I did a fast test and could immediately reproduce a data loss:
Create an item on the PC client.
Add some data.
Synchronize
Open Joplin on Android
Open the item, and add some more data
Leave the editor and close the App fast
Open the item on the PC client again. Some data (the last letters) is missing
Reopen Android App and PC content is there, previous content is damaged
The problem has to do with timing and poor connectivity. However, the protocol should be safe against such behavior, like git. But maybe this is because of a nextcloud server in the background? Is the Joplin server capable of diff / merge of simultaneously changed items?
Joplin does have a conflict mechanism for when there are changes on a note which is not synced with the latest version, that conflicts with another version of the note. The conflict resolution has to be done manually though.
In terms of the reproduction steps you mention, this is not an issue regarding handling of conflicts. Joplin utilises auto saving which saves as you type, for both desktop and mobile. However, in order to avoid performance issues, the auto save has a 100ms buffer, so that it is not constantly saving if you type really fast. So you need to wait at least this amount of time before exiting the note, otherwise the data will be lost. In practise though, particularly if you paste large amounts of text in one go, the save will also not complete instantly, so you should ideally wait at least 1 second before switching / exiting a note to give it time to save. The problem is by exiting the note and app really quickly, you are not giving the note enough time to save
That makes sense. And I guess if the mobile is a bit slow it will be more than 100ms which may cause data loss.
Not sure if there's a solution to this since saving in real time would be wasteful. Or maybe we can detect when the app is being closed and save at that time, but that won't work if the app is being forced closed.
In the above reproduction only the last letters have been missing. This may well be because of 100ms delay. In real life, this probably isnt a problem.
You are also correct, there is a conflict mechanism in Joplin. I disabled my WiFi and changed the same item on the PC and the Android in parallel. Then, Android hat the same item twice, and nothing was lost, resp. I could recover both PC and Android content.
However, I have lost some data in the last month, but I cannot reproduce it at the moment. I will monitor Joplin more closely in the future, and try to find a way to reproduce it.