Conflict Resolution Plugin

GSoC Projects Conflict Resolution
1

Hey All!
I'm Ahmed, an undergraduate student from Egypt. I'm excited to work through the summer on a plugin that improves the process of conflict resolution on Joplin.

Summary

A plugin will be created that adds a context menu option on conflict notes to open a merge menu. Once clicked a diff viewer will be shown to aid in merging the notes together. I'll potentially also add a way to auto-merge the two conflicting notes using revision data.

Implementation

Since the project doesn't have a set-in-stone spec yet, most of the implementation details is still under discussion and could change.

Basic Plugin

  • The plugin will add a context menu button to conflict notes in order to open the diff window. The plugin system will be extended to support this if it doesn't already.
  • A notification will be shown to the user to open the diff window when a conflict occurs.
  • CodeMirrors addon merge/merge.js. It'll hook to the event of clicking the context menu button and a dialog will be shown with the local and remote files visible in a 3-way diff. The user will be able to input the merged file in the middle editor. Arrow buttons will exist to aid the process.

  • User will be able to either Save or Cancel.

In the basic plugin, the middle note will initially be clear and user have to manually merge both notes part by part copying over the interesting part.

Suggested Note Feature

An option will be added in Settings to either enable or disable this feature. Since I've been told that this feature may be a bit resource intensive. (Unless we find a better idea)

Creating revisions on each sync.

The file packages\lib\Synchronizer.ts will be modified to make sure a patch is created for every
note before sync. This way every sync push is guaranteed to include patches for all local changes.

The ID of the latest revision of each note before sync will be saved, and then later on in the plugin, the 2 "latest revisions" could be used to reconstruct a suggested note.

Data API revisions endpoint

A new endpoint revisions will be added over at the folder packages\lib\services\rest\routes
with modifications to packages\lib\services\rest\Api.ts . The endpoint will provide access to
Revision data via id of revision and also ability to query all revisions of a certain note id.

Getting it all together

The basic plugin discussed above will be modified to do the following:

  • Using the latest local revision and the latest revision received from remote, an algorithm could be done to find the first common ancestor between the two revisions which will be used as a base. Then revisions from local will be applied first and then revisions from remote will be applied next. (All using google's match-patch-diff plugin).

image
image681×681 24.6 KB

  • The suggested note will simply be placed into the diff editor as a default for the merged notes. User can still go over it, make changes and then save!

Project Timeline

May 17th - June 7th (Community Bonding)

  • Discuss implementation plan with mentor.
  • Setup blog where I'll be reporting progress.
  • Possibly start work in order to be ahead of schedule. (It's better if the project is done early than late! Nothing will be rushed however.)

June 7 - June 21 (2 weeks)

  • Start work on plugin user interface.
  • Add conflict menu event and button to open the plugin window.
  • Create a notification that shows to the user when a conflict occurs in order to open the plugin window directly.

June 21 - July 5 (2 weeks)

  • Create plugin initial behavior for loading the notes.
  • Add Save and Cancel behaviors.
  • Write tests to make sure basic plugin works as intended.

July 5 - July 12 (1 week) - Phase 1 Eval

  • Submission of the first evaluation.
  • Create revisions for changed files on sync saving its ID for later use in case of conflicts.

July 12 - July 26 (2 weeks)

  • University final exams period
  • Data API revisions endpoint will be added for plugins to access.

July 26 - August 9 (2 weeks)

  • Unit tests for revision data API.
  • Implement algorithm to merge notes together using revision data.
  • Heavy tests on the algorithms.
  • Modify the plugin to show the merged note text by default

August 9 - August 16 (1 week)

  • Do any leftover work. Extra week just in case.

August 16 - August 23 (1 week) - Final Eval

  • Review all code and cleanup.
  • Code delivery.
  • Mentor Evaluation.

Finally

Please don't hesitate to give me feedback, whether technical or not! As I said earlier, there isn't any spec that's ready and such a feature was almost never discussed before GSoC, so the more feedback the better this feature will be! Maybe I've missed something or there's something not consistent. Please do let me know.

8 Likes
2

reserved

3

reserved 2

4

reserved 3

5

Will there also be the possibility to directly compare two notes (no conflict note and revision)?

2 Likes
6

You mean like selecting any two notes and open a diff viewer just between the two ?
Not really tbh, I've not planned this but I think we can squeeze that in..?

However, I'm not sure if this fits with a plugin for "conflict resolution" or even if it'll fit within the time of GSoC? Maybe a separate plugin I can do for sure later on.

2 Likes
7

@Ahmad45123
I'm having a little trouble understanding why so many changes are necessary in the core of Joplin (particularly around sync). Shouldn't it be possible to detect conflict notes without having to get so deep in to the guts of Joplin?
It seems to me that the same thing can be achieved using a simple conflict detection algorithm along with a facility for merging any two files. In that case the plugin would suggest a file to merge with, but the user will be able to select any note that they would prefer merging with (or any 2 non-conflict notes).

8

The basic plugin that I mention above is exactly what you describe: a simple dialog facility that will provide a diff viewer to aid in merging the 2 conflicting notes. (Like you mentiones, Should be simple to generalize it to work with any 2 notes, not necessarily conflicting)

Why all the other stuff then?
My proposal was actually initially just about this basic plugin but then I had a hard time distributing the work over the 10 weeks because there's simply not much to do :confused: As you can see I now have it where the basic plugin will be done in only 4-5 weeks before first evaluation.

So I had to come up with some idea that I can work on after evaluation one until the end of GSoC.

I thought why not make a system that attempts to automatically merge conflicting notes via revision data. (Kinda how git works).
The problem with this idea is that Joplin's revisions were meant for backups and they're not consistent, like there's no guarantees that every sync will have revisions for the changes made and also there's no Revisions API I can access from the plugin so all that will need to be sorted first.

Why this specifically?
I honestly have no idea, I'm not even sure if such an "auromatic merge" feature would be useful or needed by the community. It's simply the only idea I could come up with to fill in the time.

1 Like
9

Indeed we should avoid making changes to the synchronizer as there would be performance issues. And the current revision system wouldn't help you in this particular case.

Isn't it possible to get some auto-merging with two way diff? It doesn't have to be perfect, as the result would have to be reviewed and approved by the user anyway, but I think you could get something working.

In any case, a plugin that shows the two versions of the notes side by side with a way to manually merge them would already be very useful, and if it's well done with good test units and documentation it seems like it makes a complete GSoC project.

2 Likes
10

just wondering if #gsoc-projects:real-time-collaboration could be any help here, as there could be some conflict resolution necessary as well.

1 Like
11

I'd say not, the whole point of real-time collaboration is to avoid conflicts while editing.

12

I've been trying to plan the UI/UX, and this is the initial design I've got in mind:

image
image1306×679 220 KB

(Ignore the html code, just a placeholder...)

Pretty basic design for now.. However I'm still not sure how would it "open" best ? I was thinking at first to add a button to the screen which appears when you select multiple notes in one notebook:

image
image972×369 11.7 KB

However, this idea turns out pretty broken as we can't select notes across different notebooks. And ofcourse conflicts, our main goal of the project, show in a different notebook.

There's the obvious option of just adding Comboboxes into the dialog and the user selects the note from a list but I think this can get very messy with a huge amount of notes.

Do you guys have any suggestions on what would be the option that is best for UX ?

Do you think a live preview of the notes being compared would be beneficial at all ? Or just markdown would do the job ?

13

I find the option via the context menu or the tools good, because you can also search notes for comparison.
But it should also be possible to select the note in a selection dialog.
In a kind of autocomplete list (Which is dynamically loaded) that shows you the note and the path to it (Notebook1\Notebook2\Note).
Unbenannt

I think as a first target only a markdown comparison.
A preview would be nice, but have to be identical to the one in Joplin and if then the users still use css, this must be taken into account also the options from Joplin (Tools>Options>markdown).

14

Maybe the plugin could detect that a conflict has occurred and show a popup notification. This may require a change to the plugin API

1 Like
15

There is already a very mature UX for reference, please check the merge function of each Git visualization tool. . . Including VSCode, JetBrians IDE

1 Like
16

In your proposal you had this:

A plugin will be created that adds a context menu option on conflict notes to open a merge menu. Once clicked a diff viewer will be shown to aid in merging the notes together.

So why are you discussing selecting multiple notes, adding combo boxes, etc. as that seems unrelated? The best UX is what you had in the first line of your proposal - you have a conflict, you right click on the note and there's a diff viewer. If something is missing to get that working, we can add the relevant plugin APIs.

1 Like
17

My initial plan in the proposal indeed was that users would only click Compare on the conflict note and then the plugin will automatically find the other note and then show a diff tool.

JackGruber suggested, we can "generalize" the plugin a little to be a diff tool rather then just for Conflicts.. So you would be able to select 2 notes and compare them even if they were not conflicting.

So I was thinking about Jacks idea, but I'm not sure what's the best way to make the users select two notes at once.

2 Likes
18

So maybe users would right click on the note they want to edit and Click "compare with.." and then a dialog would show up to select which note do they want to compare with. The dialog will have a search box. I think this will work.

2 Likes
19

Right, sorry I missed @JackGruber's comment. What would be the use case for comparing two unrelated notes? Although I guess it's more like an additional goal and you should make sure it doesn't make the basic case (checking for conflicts on a note) more complex in terms of UX.

2 Likes
20

Two similar notes, which you may have from a clipped website, import, ...
This was just a question from me, since a compare of these notes is similar to a conflict note.
But this should not be a focus of the plugin, at most a strech goal and was only a thought of mine.

That would be good, I haven't had many conflicts yet, but it's quite easy to overlook the Conflicts notebook when you have many notebooks.

Since all conflict notes should have a 1-to-1 assignment, you could select the appropriate note for the conflict note directly after clicking on Resolve conflict in the context menue or a menue shows all conflict notes and the user can select from this windows which he wants to compare.

Unfortunately I didn't see if the orginal noteId is assigned to the conflict note somewhere, because the field is_conflict only contains 1 or 0, you might have to adjust something for this when the noteId is currently not available.

1 Like