This project aims to develop a plugin that generates a link to collaborate for a selected note, along with a web app consisting of a real-time collaborative editor. The web app has 2 main routes containing the owner app(for the one who has the note) and a peer app (for other users). the owner app should run on the desktop where the note resides. The owner app fetches the note and also saves the changes using the Joplin Data API.
Implementation Details:
The key features that are added to Joplin through this project:
A web application using ReactJS and YJS. Where these collaborative notes can be handled or viewed. The Editors implemented in Joplin (using CodeMirror) will be the base for the UI. The changes and Conflicts in the note are handled by YJS. YJS perfectly fits Joplin, as YJS provides libraries that handle editor libraries like CodeMirror. YJS also handles the server-less model for clients to edit the note and sync with other users(peer-to-peer connection through WebRTC). This web app will be served by Joplin Server.Reference Link: Yjs · GitHubReference Blog: [Tutorial]How to build a real-time collaborative app using CRDT in angular? | by Alen George | BlockSurvey | Medium
A Signalling Server. This is a simple WebSocket Server for signaling. Signaling is basically the procedure to establish a WebRTC connection(peer-to-peer connection) between the clients.
Read more on Signalling here
Plugin: Collaborate on a Note. The plugin adds options to collaborate on a note. When the option is clicked, The plugin generates a link for the owner - attached with the auth token for the Joplin Data API and the selected note's id. The owner gets a share option in the web app to get the peer URL for the collaborative session. Which can be shared and used by other users to join the particular session.
Flow Diagram:
Project Timeline:
Week 1 - 3
Creating a react app with a basic structure setup.
Setup the basic structure of the web app
Implementation:
Initial redux store.
Routes in the web app.
A basic UI with CodeMirror.
Utils related to Data API, YJS WebRTC connections, and Editor bindings.
Write tests and Document the code developed.
Week 4 - 5
Creating a Signalling Server and a Plugin.
Implementing a basic signaling server.
Implementing Collaboration on a Note Plugin which:
Adds UI components for the Plugin.
Adds Handlers for the added UI Components.
Test and Document the code developed.
Week 6
Testing, debugging and handling edge cases for the current stack.
Week 7 - 8
Adding toolbar for the editor.
Implementing toolbar related code:
Utils
UI component
Handlers for the toolbar functions
Write tests and Document the code developed.
Week 9 - 10
Adding persistence (to store offline changes in browser), testing, and cleanup.
note: In our use-case, Implementing persistence is a bit tricky and I'm researching more on it.
Could you describe what kind of end point you have a in mind? The server can already read and write notes so maybe you don't need to do anything for this. As long as you have the note ID you can update a note or read it back.
Also I think what's missing from your plan is what you plan to release and when. I think this project can be split into smaller parts that can be released independently, which will allow users to test your project earlier.
yes, but I think there should be a server route that handles the WebSocket communication which is used by the client web app to collaborate in real-time. like changes in one client application are sent to the server and the server should emit those changes to other client applications that are connected to the current collaborative session.
Thanks for pointing it out! I will add them.
Like developing a basic functional feature and adding more functionality over it?
Isn't Yjs and CRDT in general use direct p2p communication? In this case introducing a server might be additional overhead both in terms of dev time and at runtime.
also, as I said earlier it needs a signaling server for a new client to connect to other clients in a room. It is a basic server that doesn't have too much load as it is only pointing a new client to clients who are already in the room.
There are public signaling servers available for this. But there are issues reported by users of y-webrtc that the signaling isn't working over different networks.
Ok that makes sense. If this server really is necessary, I think your best option is to create a separate one so that it can be installed and managed separately from Joplin Server.
I'm actually not convinced that a server is needed and if it is, it should be separate from the Joplin server.
Consider this - to initiate a shared editing session you probably won't just press a button and expect the other person to accept; instead you will most likely agree on specific time and so you might as well send them a direct link to connect to your Joplin via some other channel, e.g. email or whatsapp. In this scenario a server is not needed thus reducing the scope, which I think it very important for this project as it's already huge.
Now, if we are going to have a server maybe it should be part of Joplin server to fully leverage its sharing functions. Suppose you and I are editing a document together and you insert a picture, I am going to see just the id which isn't very helpful.
With the Joplin server it can be shared automatically so that we both can see the picture.
This may be out of scope for GSoC but it's something to consider.
Btw, if we decide to have a server, we'll need an API for a plugin to show some kind of popup notification, I don't think it's possible currently?
activity on the repository (project alive, issue closed within a reasonable time etc.)
basic implementation into Joplin
This shall happen rather soon
If we come to the conclusion that a server is required, it shall be easily deployable.
It may be merged into the Joplin Server, if all runs stable and implementation is not much of a big deal.
For sure but I can imagine if you edit something that is deleted by someone else you are going to have a conflict that cannot be avoided.
I could imaging that in that case, the user, who changed something in the changed section, should be asked if the removal should be applied and sync of this part is frozen until this is resolved.
Anything else should happen without a conflict. The only challenge is how to keep the overview if a lot of changes are done, avoiding that your cursor jumps up and down the note etc.
do we really need an additional web app, as that requires some development work.
Should we rather focus to have the features well integrated in the existing apps.
Check with Joplin Web - Web application companion for Joplin - #111 by foxmask on the support of plug-ins at the web-companion.
As you have done a check on the libraries.
could you describe #### Week 1 - 2 in more detail?
I would like to see milestones / interaction loops how to add feature by feature
I think the GSoC idea wanted the student to develop a web app:
Expected Outcome: A web application that allows users to collaborate on a note
However this is true.
I didn't see this, it's Great!. (accessing local Joplin notes from wherever we want...). I'll look into it.
week 1-2
I thought of creating similar end-points to share notes. where this end-point enables real-time collaboration on a note and which lets the web application save the changes to the note which has collaboration enabled.
As we have now decided to go with a P2P (WebRTC) connection instead of WebSockets, I need to create a signaling server. (which establishes a peer to peer connection)
milestone
Creating the basic backend server for the web app to use.
