Version controlled notes: WebDAV, SVN and Joplin

[davor]

I am new here, and stumbled on Joplin in my pursuit to replace OneNote. I like the idea of decoupling the backend service and using open standards, but I find it a pity that GIT or SVN haven't been explored any further, although according to this post it seems to have been suggested many times, even in 2020…

One thing that is keeping me from ditching OneNote is the fact that it stores an update-date for every written paragraph, not just the file. (You can see who wrote a line of text by simply right-clicking on it, which then gives you some more context about it.) This functionality would essentially be the “blame” function in SVN/GIT.

I think a version control system behind a notebook is the ideal solution and if I am to go with limitations of markdown, then at least I should properly version-control it. Before I gave up on Joplin, it occurred to me that the supported WebDAV protocol is also supported by SVN through Apache HTTP server. And since I already had everything running, I gave it a quick try, and it works perfectly. One needs to set SVNAutoversioning On so every write is considered a commit as long as DeltaV protocol extensions are not used.

And here some of my thoughts on the apparent bottlenecks in the old post.

@laurent

I expect bundling a git client with desktop and especially mobile would not be an easy task, assuming it can be done with React Native.

For SVN, you can use WebDAV and extend it with specific SVN-DeltaV functions. For GIT it is a bit more complicated. If libgit2 can't be used, then plain reading over HTTP and commits through a custom-made CGI backend should suffice as a first working iteration.

@laurent

Not sure about performance either. The unit of operation for Joplin is a file read or write, and the unit for git is the commit. So does that mean there should be millions of commits, one per file? Or if not, then the way Joplin sync works would need to be changed completely.

The commit operation is normally independent of the number of commits in the repo. (i.e. its complexity is constant: O(1) or at most O(log(n)). So yes, there should be millions of commits.

@laurent

Then there’s the need to manage ssh keys, passwords, maybe 2FA, etc.

Same as with WebDAV I use now in Joplin: I guess you store the passwords as plain files?

@laurent

I also wonder what will happen when someone rebase or rollback commits, since Joplin is not designed to handle this either.

There should be some rules. The repo should always precede. You store the hash of the last commit. If it doesn't exist in the repo the next time you sync, then someone has tampered with it, and a full restore would be one possible solution. Another is to ask the user to resolve conflicts…

@tessus

I don’t even want to know how users will handle a simple git conflict (due to 2 clients using the same repo).

The way conflicts are handled in Joplin currently (by simply stating that the file is a conflict) is the most basic solution. This would work as well. But surely there are better tools available.

[mrjo118]

@davor

Before I gave up on Joplin, it occurred to me that the supported WebDAV protocol is also supported by SVN through Apache HTTP server. And since I already had everything running, I gave it a quick try, and it works perfectly.

I’m not quite sure what you’re saying. Do you mean this already works with version control used in the backend only, or Joplin would need to send something additional in the requests, in order to support this solution?

[davor]

Yes, Joplin works with SVN backend repository though Apache mod_dav_svn. The backend SVN repository is logging any write as a commit automatically.

But Joplin doesn’t take advantage of this. One could further work around this already-working solution and implement version control aspects exposed by DeltaV extensions:

• No need to implement revisions on file level if they are already perfectly captured by SVN.
• No need to place meta-data in MD files if one can use SVN properties.
• No need to keep trashed items if they are already in the repository.
• No need to implement locking if SVN already offers bullet-proof check-in/out operations.
• ...

Obviously, some current aspects of Joplin would need to be recoded to fully take advantage of WebDAV+DeltaV, but that is the reality when making frontends supporting different technologies.

[mrjo118]

WebDAV + DeltaV sounds like a cool idea, although it doesn’t seem to have widespread adoption at this time according to Google / AI, which could be an issue because it would probably have quite a big maintenance overhead to integrate it properly

[davor]

Its a specification that hasn’t changed in the past two decades, just like WebDAV. Which means that if one gets it right, its zero maintenance. Just like SVN: its dead tech, but more stable and unchanging than anything new.

I am not saying that Joplin should implement this. Just pointing that it exists and that building a notebook around it could provide certain advantages that would reduce reinventing the wheel and make solutions more robust.