File system sync issue

shikuz · 2024-01-31T01:33:24.886Z

Description: Some files are not synced (new or updated) during file system sync. I currently assume that this is related to 2.14 because I never noticed this in the past, and was not able to reproduce on a 2.13 mobile app. I cannot downgrade to 2.13 on the desktop due to the new profile version.

EDIT: as @tomasz86 pointed out, this is probably not 2.14-specific.

Reproduction steps:

Disconnect a 2.14 desktop client A from the internet.
Create a new note on another Joplin client B (desktop or mobile).
Sync client B.
Sync client A while it's still offline (to an outdated sync target folder).
Connect client A to the internet and wait for the folder to sync.
Sync client A.

Result: The new note will not be synced - even though it exists in the sync target folder - as long as its update time is older than the lock file / info.json time. Once you modify its update time to be newer than the last lock, it will be appear on the next sync.

Environment:

Joplin 2.14.11 (prod, darwin)

Client ID: 962f7dafcef24edbb82e0fca7f255ac4
Sync Version: 3
Profile Version: 45
Keychain Supported: No

Revision: 0f9d6a1

Calendar: 1.0.0-rc2
Conflict Resolution: 1.2.3
Inline tags: 1.2.1
Jarvis: 0.7.0
Markdown Table: Colorize: 1.1.4
Markmap: 1.6.0
Menu items, Shortcuts, Toolbar icons: 1.1.0
Note Link System: 0.8.0
Note Tabs: 1.4.0
Plugin Bundle: 0.5.5
Rich Markdown: 0.13.3
Simple Backup: 1.3.6
Space Indenter: 0.2.2
Suitcase: 0.3.2
Table Formatter Plugin: 1.2.1
Templates: 2.4.0

ajay · 2024-01-31T05:03:01.670Z

In step 4)

... do you actually mean you do manually start a full sync while offline ?
(If so, why would anybody do this ?).
... or do you mean Joplin on desktop A starts an automatic sync of one or more notes ?

tomasz86 · 2024-01-31T07:59:57.643Z

shikuz:

Result: The new note will not be synced - even though it exists in the sync target folder - as long as its update time is older than the lock file / info.json time. Once you modify its update time to be newer than the last lock, it will be appear on the next sync.

Sounds like the same problem as https://github.com/laurent22/joplin/issues/6517. It has likely been always present, i.e. it's not related to v2.14.

shikuz · 2024-01-31T08:28:42.205Z

It can be either automatic or manually triggered. I mentioned this explicitly just for the reproduction.

shikuz · 2024-01-31T08:41:02.722Z

Oh no, this is terrible.

I tried various sync methods in the past. For the past 6 months or so I've been using file system / Syncthing, but never noticed it until now (I suppose that my usage patterns also changed recently, revealing this flaw).

I guess I'll reconsider file sync.

tomasz86 · 2024-01-31T12:42:46.264Z

Yeah, what I do is to have a computer that is on 24/7 and (among other things) runs a script that every minute updates Joplin sync target files to a future date (exactly 1 year from now). This way, Joplin is always forced to synchronise all files.

To be more specific, the script only updates files that have their modified date at least 5 minutes old (so that it doesn't mess up notes that are being updated at the very moment). It also ignores files which have a future date set already.