Sharing notes or notebooks

I think it is possible to have shared notebooks. Here is how it can work

First off, the file structure will have to be notebook->note. For example,

PersonalNotebook
—Note1
—Note2
—Note3
SharedNotebook
—Note1
—Note2

And the settings will have to be divided for each notebook, not for all notebooks. This way you can choose a specific notebook to sync to dropbox and different notebook to sync to nextcloud. Same for encryption. One notebook uses one password to encrypt/decrypt the notes, the other notebook will use a different password.

To share a notebook you will have to do that with the cloud storage provider by sharing the folder with others users.

When you drag a note to go from one notebook to another it will have to be recreated instead of moved due to encryption since each notebook will have a different password.

This is how I will see Joplin implementing sharing notes in a user friendly way.

4 Likes

Short of a few caveats that I think would meet my full-needs with Joplin, I have been impressed enough to permanently move to it as the main note-taking app. I also have been moving a couple of my tech staff to this solution as well and we’ve got a sync process worked out with some minor problems and a few workarounds.

I stood up NextCloud and have our organization’s external shares set up for our network (smb network shares to home/department/collaboration folders). One of the network shares is our department files. Creating a note folder there will allow the entire department to access and leverage the NextCloud/webdav sync capabilities.

When we have staff meetings, I now have one member of the team taking minutes versus multiple people interacting on the same note. This prevents the overwriting of data since person A’s edits may not have sync’d before person B’s edits started to sync, thus overwriting person A’s initial notes. This works as a viable alternative if there’s only a few people interacting with the notes. You start to have issues if a lot more people are connected to the same shares and interacting with any shared notebooks/notes because there is not a queue system in place between Joplin and NC to coordinate the order of synchronization.

As far as the structure of the notebooks/notes, we did something similar to what @trymeout suggested for a while now. We created a notebook for each position, e.g. IT Director, IT Administrator, IT Technician. Each role has their own section they edit/sync but all of us have capabilities to search in all the notebooks for things we need. We have a shared notebook consisting of checklists, projects, and a running-knowledgebase area where how-tos and documentation are revised and formatted before it is finalized and permanently stored in the Documize system. The neat part is that the knowledge base is ready to go since Documize supports markdown sections as well. Using the custom-stylesheet for joplin-wide and taking portions of the stylesheets from other members here (thanks to @tessus), I’ve been able to format the preview window so now that exported-to-PDF print-outs look structurally sound. It’s also easy to copy/paste the customizations to other staff members so we all have the same look, feel, and operation.

The only things I really could use and look forward to, if it ever comes, are:

  • In-app updates versus downloading and manually launching
  • Multiple notebooks (or a better process with password-protection or encryption) to cater to my own personal-work notes - accessible only to me, say performance reviews of my staff and such
  • Multiple sync connections (or a switching between sync accounts option) - since I have my own NC server at home that I have my personal notes for school, shopping lists and kid’s checklists (that my wife and I use in Joplin), and various other documentations

I look forward to every update and the new features everyone has been contributing. Thanks to many of you for dedicating the time/energy to this project, especially @laurent and @tessus for your active responses and contributions. I only wish my brain was wired to code/program to further the product some more.

3 Likes

This will make Joplin the best self hosted note sharing app out there. Joplin + Nextcloud = best open source note app!

2 Likes

Sharing notes works fine on my setup. But, is there any way to manage shares? I only can see a list of ID’s, but I cannot to find a way to recover shared notes URL or remove shares.

Are those features supported?

Thanx

At this point the note sharing capability is more or less only a proof of concept implementation and is lacking essential features. You’ll have to wait until Laurent or somebody else puts more time into this.

Hello

I’ve created a prototype for an encrypted real time (per line) collaboration feature for joplin desktop, based on websockets and sjcl. I didn’t pitch the idea here before i started as my main purpouse was to learn the basics on encryption, websockets and js in general. However i thought i might as well document my progress and bring the idea up for discussion here.

I’ll attach a demo to this post that show some of the features i’ve implemented so far in action. Feel free to comment on pros, cons and whatnot regarding design choices, assumptions, thoughts and the potential of this collaboration model as a future joplin feature.


The goal for this prototype was to see if i could get the following to work

  • Connect two joplin instances to a websocket server
  • Track each clients cursor moves (per line) and display it to the other clients
  • Track changes in the note from each client (per line) and push changes to the other clients without exposing the contents to the server.

Prototype demo
I recorded a short demo of how the prototype works in current state. Both clients and the server run on the same laptop from ca 2015 (i5 3GHz, 8gb ram on arch linux)

rt-collab-demo.mkv (3.8 MB)

Dependencies in current state
Client side

  • ace editor (already in codebase, though i remember reading in some post it’s planned to be replaced?)
  • i guess we are limited to standard keybindings
  • sjcl (already in codebase)
  • ws

Server side

  • node.js
  • ws

Prototype design choices

Node.js based websocket server
As i mentioned, mostly because i was aiming to learn about websockets. However i do think this feature would be quite easy to set up for anyone with access to a server running node. As the current share feature is based on nextcloud, i guess this collab model could be an additional alternative for self hosters.

Transfering note changes between clients (lineUpdate).
I chose transfering whole lines as i figured the encryption part might cause performance issues if i were to encrypt and send updates letter by letter or word by word. Not neccesarily true, just an assumption on my part.

Per line updates also made for an easy way to build a server side model of the note in ciphertext, and keeping it synced between clients.

Client side symmetric encryption.
The encryption is based on a key that will somehow have to be shared between collaborators (in prototype it’s just a set variable). This is of course a drawback from a security standpoint.

One reason for not going asym on lineUpdates is performance, each client would have to encrypt each sent line with a separate pubkey from each of the other connected clients. With symmetric encryption and a shared key this only has to be done once per lineUpdate. independent of the number of connected clients.

Server, current state

  • Stores the notes as line number and ciphertext in an array tied to a note object (stores nothing on restart right now).
  • Pushes complete note to client on connect
  • Recieves per line updates and cursor movements from clients
  • Pushes updates to connected clients
  • Pushes new clients to connected clients and vice versa

Client, current state

  • Encrypts each line before sending it to server
  • Decrypts each line recieved from server
  • Uses a 256 bit shared key stored in variable
  • Does not encrypt line numbers or cursor movements
  • Is tied to a bunch of Ace methods to locally execute remote note changes. That part may or may not be a problem to solve depending on what new editor is implemented.

API

I’ve defined a simple API for client-server communication with messages sent over websockets in json-strings. Some of the key functions are listed below.

Client-side

message ‘type’ direction contains description
newCollaborator in uid Creates a marker for new collaborator
getNote in Complete stored note from server Pushed from server on client connect, right now overwrites anything local.
cursorMove out uid, line number Sends all vertical cursor movement to server (letting the server know which line each client is at)
cursorMove in uid, line number Updates a marker object that highlights the line each collaborator is on.
updateLine out uid, line number, line ciphertext Sent when leaving a line. The client encrypts the contents of the line using sjcl.encrypt and a 256bit shared key, then sends line number and encrypted data to server
updateLine in uid, line number, line ciphertext Simply decrypts the received line and inserts it at line number, overwriting anything present on that line.

TODO

There’s a lot to be done still for this to become a usable feature. Below is a list of what my prototype currently lacks, in combination with hypothetical discussions on key sharing and gui implementations. There are many different ways a feature like could be implemented. Feel free to comment.

Editor

  • Passing on the following to server and connected clients
    • Deletion of lines (i’ve had a hard time catching del and backspace onkeydown, does anyone know if there’s a specific reason to that? Basically all other keys are catched and i have it working outside of joplin)
    • Selections, cut, paste, deleting or replacing text by seleting and typing
    • Images and other attachments

Thoughts on key sharing

The shared secret key should of course not be exposed to the server in plaintext. I guess it could be sent encrypted over the server together with salt to connecting clients, requirig only a password on the recieving end. Maybe it could also be done by a file shared by e.g email, that opens in joplin and contains an encrypted version of the shared key. A drawback for both of above alternatives is that they leave the shared key open to dictionary attacks if somehow intercepted.
Another alternative might be to encrypt the shared key asymmetrically. Each client would then need a public/private keypair in addition to the shared key.

Server

  • User management
  • Authentication/Authorization
  • Note management.
  • Note storage. Sqlite? Mariadb?
  • Look over security

Client and thoughts on GUI-integration

  • Settings, add a “Shares” tab for managing server credentials and keys
  • Note, show names for online collaborators somewhere
  • Note, give user markers different colors
  • Notebooklist, possibly a new section for shares, first level being either user or server.
  • on right click note -> share, an alternative could be added to share to a server

Mobile

  • I have not looked into this, but i believe that for this to become a usable feature it must also exist on mobile.
5 Likes

Looks cool!

A few thought on this.

  1. Maybe you can avoid the need for a sever and just connect 2 Joplin instances directly. I think this fits well with overall Joplin architecture.
  2. Crazy idea but what if you could extend this and use for synchronization. Say instead of sending these updates in realtime you serialize them and sync via Dropbox or any other way. Then all other instances can read them later and apply. I think it can help avoiding conflictsp plus you can synchronize undo history this way.
  3. Another interesing thing to explore is operational transformations or CRDTs for collaborative editing.

Thanks for your comments.

  1. I agree it would be nice not having to set up a server at all, but with a p2p setup both clients would need to be online at the same time to sync, which to me seems to make it less useable.

  2. That sounds like a good feature, but may be more related to the current sync implementation or the nextcloud share feature than to what this prototype does. It would be nice to be able to use the regular sync to the same node server though. Not sure how much work that would be.

  3. Maybe, i havent looked into that as my aim was to learn about the basics. On a side note i also saw there is a project for making real time collab apps with Ace editor which might make stuff easier. I’m not sure if that supports any type of encryption though.

I think Joplin should have shared notes and allow users to choose their storage provider for sharing notes. However if joplin can afford this, they could host a nextcloud server with like free 50mb of storage and even have paid plans for easy cloud deployment.

Collaboration and sharing, together with already existing privacy could be very interesting for business. I also believe, that there would be demand for paid services (e.g. hosting private instance), but the question is whether it is not against the Joplin’s philosophy. In different words, the users that won’t pay or won’t host (which is pretty close to paying) won’t have the access to this feature.

What about sharing a key through URL with pseudo protocol? For example: joplin://sync-my-joplin.com/178489dd-bc0b-4ebb-8053-e252bd4afca0. The link would contain the key or any other information. Joplin desktop app would be registered to handle the protocol. I am not aware how difficult is it to implement on different platform thought. The second option would be to simple drop the URL somewhere in the app.

I’m not familiar with pseudo protocol, but that sounds worth looking into if we get to that point.

I’ve kept working on the client and server in spare time for the past weeks, and realized it’s a quite complex project. However i’ve also grown more confident that it could turn out a nice feature.

I might put up a repo with the current state of things once i’ve cleaned up the code. Still long way to go though.

1 Like

@laurent Why not get and filter out some files through web api, and then write it to the specified directory instantly, after that, you can choose your favorite way to sync to any place and share it.I think that this does not conflict with the need to generate a static blog site - it should be possible to reuse part of the process. If no one realizes it, after I use and improve the vscode plugin, I will start to implement it (I have a need to generate a static blog)

I couldn’t read all that. But I just thought I’d share this … The matrix guys are doing interesting things where everything is end-to-end encrypted and you can self host and yet be a part of the larger network. In the end, the only way to share encrypted data between parties is through public key cryptography which include sharing, verification, and revocation processes. It’s not trivial.

Anyway. This is a neat read:
https://matrix.org/blog/2020/06/16/matrix-decomposition-an-independent-academic-analysis-of-matrix-state-resolution

2 Likes

Matrix has crossed my mind. Imagine a bunch of self hosted joplin collab/share servers connecting all users on each server together. That would indeed make the feature more easily accessible and more competitive to proprietary note collab solutions, presumably with e2ee and the fundamental server functionality already taken care of. I suppose it might also work quite well with the per line update model i’m prototyping.

2 Likes

I think Nextcloud or Owncloud is the way to go since these services are federated like Matrix and are designed for cloud storage.

Hi There,

first of all thx for the nice work you put in.

At the moment I do use 2 profiles (profile Alice and profile Bob) in Joplin mainly on the Win 10 desktop app.

  1. How I can find the Joplin notes I want to share in Nextcloud?

I do have an empty Joplin folder in Nextcloud.
In my Nextcloud main folder there are a lots of .md-files e.g. 0a9b646940274dbfb59032075d4ef58b also a file called info.json and one .owncloudsync.
As I can remember these did appear a year ago when I installed Joplin for the first time.

  1. If these are the files how can I distinguish them from each other?
  2. And is it possible to get them into this Joplin folder in Nextcloud anyhow, preferably the note from profile A separated from profile B?

Unless I misunderstood your intentions, you can't do that with Nextcloud.

I think what Tessus meant is, you can code something running on your local machine (or in a Joplin plugin). And you can code it to upload to Nextcloud.

Please don't mess with your sync folder in Nextcloud, or you're gonna have a bad time.

(And happy new year, everyone!)

1 Like

Thx Mate, Any other ideas of sharing notes?

I got this one!
Other post with - crating a sync taget folder

Is here someone who have coded something like that - for sharing Joplin notes though Nextcloud?