Sometimes latest changes get reverted (during sync?)

Hello everybody!

Troubleshooting

It's been a bit of a journey. I'm still not at the end, but it seems I'm making progress. :slight_smile:

TLDR;

A combination of a local issue and an obscure edge case will make your days an annoyance.

The whole process

After trying and doing observations I decided to reinstall my entire OS and migrate only configuration for software that I'm 100% sure I want to migrate. Joplin's data was backed twofold:

  1. the ~/.config/joplin-desktop folder
  2. *.jex archive of the entire Joplin installation

The first item was kept only for reference. I then proceeded to wipe my drive and install Ubuntu 18.04 on it (KDE Neon).

Nextcloud configuration

I created a folder ~/joplin/rel-test-2 for Joplin to use. This was to be synced with other nodes via Nextcloud.

I downloaded the Joplin appimage (1.0.197) and ran it, setting ~/joplin/rel-test-2 as a back-end (file-system back-end). I then proceeded to set encryption and prepare the second node (laptop) the same way. After testing that both nodes have the same data and that E2EE is working I loaded the *.jex archive onto one of the nodes (PC) and waited for the sync to complete (Joplin to disk, Nextcloud to cloud provider). I then powered on the laptop again and waited for the sync to complete here. I don't know why, but Joplin on the laptop couldn't detect the data to load. I recreated the .config/joplin-desktop folder and all went well.

Drift observation

I decided to keep track of potential file drift. I would every few hours create a debug report and also copy the contents of the Tools > Synchronisation Status page. I would put that on a separate drive for safekeeping.

After switching machines I'd use sort to make sure the entries are in alphabetical order (to help with using diff):

image

I would then use vimdiff file-a.sorted file-b.sorted and look for differences. Here is an example:


Apart from an occasional white-space and something incrementing by one (or a normal difference in downloaded and decrypted) all would be the same.

This meant that whatever was causing the drift, died when when I reinstalled the OS. Come to think of it, sometimes after waking the PC from sleep, I'd hear two BIOS beeps. I read that this may mean memory issues, but there was nothing mentioned about waking the PC up from sleep vs. a cold boot. It's been a week since I reinstalled and despite using sleep all the time, there were no beeps.

After becoming more optimistic I sadly noticed once again that notes are getting sometimes reverted to a previous state. But there was no drift.

This suggests that sync drift and sudden reverting of data are separate issues with separate causes.

So I decided to switch from file-system sync to WebDAV sync using a method outlined here:

I'm keeping track of the data I input into Joplin, by keeping a copy in a separate place so I can compare and look if anything is missing. So far I haven't noticed anything vanish. I'm giving myself some more days to observe the issue.

I remain optimistic.

1 Like