It would be very useful to be able to configure several synchronisation targets (e.g. multiple Webdav backing storage areas) each containing non overlapping notebooks / folders, and each with a separate master password / own encryption configuration.
Additionally it should be possible to move a note or notebook from one encryption/storage bucket to another. Or in other words chose which bucket a given notebook or note should be stored in - including a local bucket not synchronised anywhere.
This would be akin to managing several logins/profiles within one screen and being able to move things between them easily.
I feel like I don't quite get the difference between multiple profiles feature suggested in gsoc'20 and multiple sync target feature you're describing.
Would you kindly illustrate the unique points of it?
Does it serve different use case as multiple profiles?
Regarding this comment, I'm sorry for being so daft, but I don't quite understand it either.
How does it relate to suggested feature? Do you imply Joplin to split its database on multiple remote sync targets, sync them at the same time and have those databases communicate with each other (without a server mind you)?
I mean if so, that's a peer2peer architecture looking more like a messenger (or the internet itself) than a note taking application.
If you have notebook A syncing to say webdav server A and notebook B syncing to webdav B, you can easily move a note between notebooks and expect it to be synced to a different target. With multiple profiles you're looking at export from one profile and import into another or copy & paste (won't work with attachments!)
Continuing with my example above you could sync a set of notebooks you wanted to share to your webdav B and give credentials to someone else. Your friend can then add it as another sync target and receive your notebooks.
I've always been quite keen on the way that OneNote handles it.
Essentially OneNote itself is more of a container and Notebooks within act as a combination profile/target.
Allows you to:
Open or close local (or remote) notebooks at will
Have different synchronisation accounts per notebook (or none)
Have certain notebooks shared between two or more people
Now with Joplin the name Notebook has already been taken and is synonymous with "folder" when it comes to the tree heirachy.
So in order to reduce ambiguity I'll refer to the concept of what would be a OneNote "Notebook" as a "Portfolio".
One way I can see this working (with an absolute ton of effort) would be to redo the entire backend somewhat by splitting the databases.
A "master" sqlite database would control overall application settings (plugins, appearance, general settings, open "portfolios", available "portfolios" etc.) but no longer stores the main data.
For the "portfolios" you would have further, individual, sqlite databases within the main profile directory that contains all the information surrounding note data, sync targets, encryption, note revisions and references its own associated resources folder.
If one was simply working with markdown files from a text editor like Atom or VSCode then this approach is analagous to simply opening a closing a project folder which may or may not be situated in a folder set up for cloud syncing.
Roman has succinctly explained my thoughts better than I had.
The reason my feature suggestion is so much like gsoc'20 is because I had not found gsoc'20 as I was not looking with the same vocabulary. But basically my feature suggestion is very similar to gsoc'20, with the nuance that there must be an easy way to move notes/notebooks between what you name portfolios.
You are right that I have no idea of the implementation complexity, my aim was to try and suggest a feature with the least amount of complexity. The portfolio idea is right on.
I'd like to see this also - would mean I can have a notebook for work related and a seperate one for personal stuff all in the same app/UI but are actually saved to different locations
Having (partially) re-implemented Joplin's sync algorithm twice, I can see why it can be challenging to support multiple sync targets of the same type, e.g. 2 different WebDAV servers/accounts. Laurent can correct me if I'm wrong.
Essentially, Joplin stores some sync state per sync target, and currently it just assumes that at most it will have 1 WebDAV, 1 Dropbox, 1 OneDrive, etc.
This way it can store this state using the target type as key, and the number of those keys is fixed and known in advance.
Allowing to have multiple targets of the same type could cause subtle issues.
Consider the following scenario:
Suppose Joplin runs sync and sees that a note that does not exist locally is present on the server.
There are 2 possibilities:
a. It's a new note that has been added from another client. In this case it needs to be downloaded from the server.
b. It's been deleted locally and must be deleted on the server.
Normally Joplin stores deleted notes in the database using sync target type as a key.
Now, suppose you add a new WebDAV after deleting a note. And then you run sync.
Since there's no state stored for the new WebDAV server so there's no way to resolve the above ambiguity.
For the record, I don't know the internals of the code here, so I may be off with my suggestions, but I would also very interested in multiple sync targets (or selectively syncing notebooks on different devices, etc.).
Essentially, Joplin stores some sync state per sync target, and currently it just assumes that at most it will have 1 WebDAV, 1 Dropbox, 1 OneDrive, etc.
This way it can store this state using the target type as key, and the number of those keys is fixed and known in advance.
Normally Joplin stores deleted notes in the database using sync target type as a key.
From a database standpoint, this seems like this would be a bad key because it is so inflexible and does not handle some of the use cases. If I change the url for the WebDAV sync target, wouldn't that invalidate all of that sync information? Again, without knowing anything about the internals, it seems like a generated key for each sync target would do the job with more flexibility.
I can imagine an internal list of sync targets with a state associated with them (active, inactive, deleted?, etc.) that you could associate with a notebook (or maybe set as the default for all notebooks). It does open up possibilities for sync errors, but I don't think there are any new types of errors.
Not quite. The sync information will remain but the actual data on the server may be different. Hence this FAQ entry:
When changing the WebDAV URL, make sure that the new location has the same exact content as the old location (i.e. copy all the Joplin data over to the new location). Otherwise, if there's nothing on the new location, Joplin is going to think that you have deleted all your data and will proceed to delete it locally too.
Maybe but there's also a disadvantage here - you'd have to keep storing sync stage for every sync target you've ever set up because there's no way for Joplin to know when you no longer need it. It'd have to be done manually. It could be done sure, just not as simple as it may seem.
Oh, I can see quite a few new types of errors.
E.g. you have a note in a notebook that you sync to target A. You move the note to another notebook you sync to target B. What do you do with the note on A now? Delete or keep? Now on another device you edit the note and then sync with A - what happens then?
E.g. you have a note in a notebook that you sync to target A. You move the note to another notebook you sync to target B. What do you do with the note on A now? Delete or keep? Now on another device you edit the note and then sync with A - what happens then?
Well, since it is a move, which is typically a copy then delete, I would say delete and then on the second device, that, when it syncs, should have it deleted too (before it can be edited).
But, I see your point; it is a complicated situation. Maybe part of the issue is that Joplin is trying to be too smart in these situations. How often people switch sync targets back and forth? In my personal use, that kind of switch would be a massive, once in a blue moon kind of undertaking. Are there good reasons to do it frequently? If not, then you don't need the previous sync histories, just the current one.
I can also see Joplin punting and asking the user what to do. For instance, when you switch sync targets, the user is asked if it is a push to or a pull from the sync target or some kind of sync (pull and push with conflict resolution). Throwing in a backup at that point to keep your data from being lost if you screw up would be useful too.
My goal is to be able to selectively sync my notes and keep personal data and work data separate but still be able to see and move data between them as needed. Multiple sync targets may not be the right approach for a number of reasons, but I wanted to keep the discussion ongoing. The swapping of profiles sort of meets this goal, but it fragments my workflow too much, so I'm still looking for something better.
Remember that Joplin is "offline first" so there is no current way to do this, it might take a long time if the client hasn't been synced in a while before it even gets to that notebook - what if you don't even have an internet connection at that time?
Remember that Joplin is "offline first" so there is no current way to do this, it might take a long time if the client hasn't been synced in a while before it even gets to that notebook - what if you don't even have an internet connection at that time?
True, but that is a sync error regardless of whether I move a note between sync targets or not, so it isn't really a "new" problem. It is a new way to create an existing problem.
Wouldn't this delete it from A? I mean, it sounds a little simple, but when I mv a directory in Linux, it's no longer in the original source directory. This, to me, sounds like a straightforward CRUD problem, and this operation in particular is a "copy and delete".
This, to me, sounds like a straightforward CRUD problem, and this operation in particular is a "copy and delete".