How to sync? Options and considerations, advice needed

Support
1

Dear Joplinners,
Early 2021 I made the switch from evernote to Joplin, and I really like Joplin. Joplin is almost the only software where I make donations to express my liking.

So far I used syncing via the cloud (a Nextcloud variant for Dutch academics: surfdrive) to my satisfaction. However, since a few weeks syncing has become problematic. It turns out I'm affected by the notorious FileLock bug in Nextcloud (HELP needed: Sync Target upgrade error). Since I have no admin access to the cloud provider, I am out of luck with patching, tweaking or otherwise dealing with this.

So: I'm looking for another way to sync. Essentially I need synchronisation between Joplin on my laptop (Win11) and phone (Android). Ideally also to other clients (mostly running linux). In Joplin I have E2EE set, of course.

What is a robust way of doing this? Alternatives I consider are:
1/ SyncThing directly between phone and laptop as I have already set up basic syncing of two folders on laptop and phone
2/ SyncThing to a "central node" for instance a self administrated raspberry pi.
3/ Surfdrive (based on nextcloud), as I used to have.
4/ Set up and manage my own Nextcloud and patch it sufficiently
5/ Set up and manage own Joplin Server.
6/ Subscribe to Joplin Cloud.
Others like onedrive and google drive maybe possible, but I love to keep things in Europe and under my own control.

Thoughts and considerations I have for the various options:
For 1/: Syncthing seems to be fast and easy to manage.
Disadvantage is that only two clients are syncing. Also, I just read this post about upcoming limitations for SyncThing in Android (Android 10/Q and Syncthing Future)

For 2/: Theoretically that would allow syncing from the various clients to this central node, and hence behave similar to the set-up I used to have via the surfdrive cloud.
I'm a little worried about exposing this R-Pi to the world, and what happens if this central node for one reason or another goes down or becomes corrupt.

For 3/: In fact this is my preferred scenario. But what to do to avoid problems with the FileLock bug? This has already cost me quite some time re-setting things up etc. It sometimes worked, but adding another client or other actions result in the problem showing up again. At this moment it looks unreliable to me.

For 4/: While I have some computer experience and linux skills, I am unsure about doing this in a safe and secure way. On the other hand, when I manage to do this, this private cloud can be used by several people in my household, of course.
The tutorial on Build Your Own Raspberry Pi Cloud Server With Nextcloud looks doable.
Will the FileLock bug affect me nevertheless?

For 5/: Similar considerations as with 4/. But it seems a bit more difficult even and less versatile in use. I have no experience with docker and so on.

For 6/: that would cost a few euros a year. Basic version is not enough, the Pro option is a bit more expensive than what I used to spend on my evernote subscription.

Please let me hear your advices and additional considerations.
Thanks,
Maarten

2

That information is outdated. Syncthing actually works better in Android 11 and newer because it now can finally access and sync files on the SD card as well, which wasn't the case before. If you do go with Syncthing, I'd suggest to simply connect all devices with one another (i.e. phone, laptop, "central node") instead of connecting the other two just with the server. It's more robust and faster this way (and there is no real reason not to do this unless you've got a very large number of devices).

4

Thank you @tomasz86. So SyncThing should do the job.
A question I have is about the amount of storage needed (a concern for the phone, not so for the laptops). I fear the data will in fact be three times on the phone:
1-unencrypted in the joplin folder
2-encrypted in the joplin folder
3-encrypted in the syncthing folder.

With ~2GB of notes and attachments that is a significant part of the available storage capacity on my (old) phone... As it is on Android 10 I cannot put the sync folder on the SD card, neither does the Android version allows to store Joplin and its data on the SD card :frowning:

5

From looking at the encryption on the desktop version it will be more than three times in total. From what I have seen the encrypted version of a file is almost double the size of the unencrypted version. It's the attachments that can really start grabbing disk space when encrypted. But with 2GB of data and two encrypted copies you are possibly looking at somewhere approaching a total of 9GB of data.

image

6

Please also keep in mind that Syncthing reuses blocks (and doesn't download any data at all if the same ones are already present on the disk), however with encrypted files it won't be able to do so, so the synchronisation itself will probably also be slower and require more bandwidth.

7

Thanks for the feedback and inputs. I have SyncThing set-up, and first sync between laptop and phone is running.
Now let's see how reliable this is, and how much space remains on my phone. Not much, I fear (although I've set attachment download to auto, so not all attachments will be decrypted and copied to the Joplin working dir - I assume)

8

Just to update you all. SyncThing works quite nicely. Setup is fairly easy and it works through firewalls and the like.
I do get occasional sync conflicts in SyncThing (not in Joplin), but these are easily resolved.

The only strange thing that happens on Joplin side is that some newly created notes do not show up in my phone (Android). If I edit the note on any other computer it does get created on the phone...

9

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.