Hello everybody!
Troubleshooting
It's been a bit of a journey. I'm still not at the end, but it seems I'm making progress.
TLDR;
A combination of a local issue and an obscure edge case will make your days an annoyance.
The whole process
After trying and doing observations I decided to reinstall my entire OS and migrate only configuration for software that I'm 100% sure I want to migrate. Joplin's data was backed twofold:
- the
~/.config/joplin-desktop
folder *.jex
archive of the entire Joplin installation
The first item was kept only for reference. I then proceeded to wipe my drive and install Ubuntu 18.04 on it (KDE Neon).
Nextcloud configuration
I created a folder ~/joplin/rel-test-2
for Joplin to use. This was to be synced with other nodes via Nextcloud.
I downloaded the Joplin appimage (1.0.197
) and ran it, setting ~/joplin/rel-test-2
as a back-end (file-system back-end). I then proceeded to set encryption and prepare the second node (laptop) the same way. After testing that both nodes have the same data and that E2EE is working I loaded the *.jex
archive onto one of the nodes (PC) and waited for the sync to complete (Joplin to disk, Nextcloud to cloud provider). I then powered on the laptop again and waited for the sync to complete here. I don't know why, but Joplin on the laptop couldn't detect the data to load. I recreated the .config/joplin-desktop
folder and all went well.
Drift observation
I decided to keep track of potential file drift. I would every few hours create a debug report and also copy the contents of the Tools > Synchronisation Status
page. I would put that on a separate drive for safekeeping.
After switching machines I'd use sort
to make sure the entries are in alphabetical order (to help with using diff
):
I would then use vimdiff file-a.sorted file-b.sorted
and look for differences. Here is an example:
Apart from an occasional white-space and something incrementing by one (or a normal difference in
downloaded and decrypted
) all would be the same.
This meant that whatever was causing the drift, died when when I reinstalled the OS. Come to think of it, sometimes after waking the PC from sleep, I'd hear two BIOS beeps. I read that this may mean memory issues, but there was nothing mentioned about waking the PC up from sleep vs. a cold boot. It's been a week since I reinstalled and despite using sleep all the time, there were no beeps.
After becoming more optimistic I sadly noticed once again that notes are getting sometimes reverted to a previous state. But there was no drift.
This suggests that sync drift and sudden reverting of data are separate issues with separate causes.
So I decided to switch from file-system sync to WebDAV sync using a method outlined here:
I'm keeping track of the data I input into Joplin, by keeping a copy in a separate place so I can compare and look if anything is missing. So far I haven't noticed anything vanish. I'm giving myself some more days to observe the issue.
I remain optimistic.