I'm loving Joplin but I'm finding that sync is slow, and not always reliable with thousands of notes (I have around 7000) . I am using a self-hosted Nextcloud running on Ubuntu and I'm syncing via a 1GB wired LAN. There aren't any network-related issues.
I was wondering how difficult it would be to implement a remote database as the sync target rather than using files or file systems, and whether it would be quicker. For example, could a remote MySql database be a sync target?
I don't have any need to see the "files" on the target system (be they on Nextcloud, Dropbox or any file system). All I really need from the remote system is that it's able to allow various clients to synchronize.
As far as I can see, all Joplin clients store their notes in a local SqlLite database. I'm thinking that not having to convert the SqlLite client data objects to files for every note would be more efficient. It would be just a question of updating or inserting records in a remote database.
(Another advantage would be that I could backup the remote database easily as one unit of time-stamped data, without having to think about having to potentially restore thousands of individual files.)
Any ideas?
I have searched for this topic to see if it was already suggested by someone else, but couldn't find it. Apologies if I'm wrong.
Nextcloud is notoriously slow when faced with loads of small files like the Joplin sync target (something about a bad webDAV implementation). I still use my self hosted Nextcloud instance because other than on initial sync of a new client the speed really doesn't matter to much to me.
Along with other problematic hosting options this was part of the reason why Joplin Server/Cloud was made, to massively increase the performance of the sync target.
If you are already self hosting Nextcloud then it might be worth having a look at setting up Joplin Server instead as I suspect it will alleviate many of your concerns.
Thanks, but one of the reasons for using Joplin was to host everything myself. I don't want to rely on third parties (partly due to my experience with Evernote).
From my understanding it is not easy/impossible to reliably sync a database via OneDrive, Dropbox, Nextcloud etc. These services are build for file-sharing not for a database. It can lead to sync errors and inconsistent data. I dont understand the technical details but there is a bunch of information if you google it, twoexamples.
Here is a little speed comparison of Joplin Server vs. Nextcloud:
Do take a look at Joplin Server, it’s lightning fast and can be conveniently self-hosted (I use it that way, too). It’s truly a blessing that it’s available.
Thanks, but I didn't mean having the database as a file hosted on OneDrive or NextCloud, etc (I agree this would be a nightmare to manage). I meant to access a database running on a server somewhere. This would mean Joplin becoming a client of a database, such as MySql or Postgres, etc, rather than the client of a file-hosting system.
Essentially that is what Server kind of is, its postgres on a docker image.
Otherwise wouldn't this be a bit of a nightmare to maintain both in development and for the user? The different database architectures to be supported alone would be one thing and the db maintenence is then entirely up to the user.
Thanks, I deployed the Joplin server with a Postgres database and the Windows client is now significantly faster - I'd even say "very" fast. However, the initial sync on Android has taken ages.
Yes, initial sync was painfully slow for ~3500 notes on Dropbox.
I had to unlock my iPhone and iPad DOZENS of times for it to finish the initial sync from the desktop app.
But once you get over the initial pain, it works quite well.