Spent some time researching which function from the node-diff3 library to use for the diff engine, most applications use diff3Merge for auto merging but our case is different because we never merge anything automatically, every decision is left to the user. So, after some research I found that diffIndices is the better fit here since diff3Merge silently folds onesided changes like additions and deletions into "ok" blocks and never surfaces them, which is exactly what we need to show the user.
The main challenge this week was figuring out the right approach for the diff engine, diff3Merge is the standard choice in most merge tools but it does not work for a conflict resolution UI where every change needs to be shown explicitly to the user rather than auto-accepted
Maybe we should challenge this? Are there situations where it might be acceptable to auto-accept edits? An example is where a user has made 2 unrelated simple inserts. My preference would be for auto-resolution. What situations would diff3Merge handle automatically?
We dropped silent auto-resolution for three reasons:
First is the ongoing encrypted notes GSoC project , there is another GSoC project running in parallel that adds notelevel local encryption inside joplin itself. The problem is that merge engine runs inside the sync pipeline and reads note bodies as plain text so it has no way to distinguish cipher text from regular content, so if a locally encrypted note hits a conflict, auto merging two cipher text bodies would produce a silently corrupted result, and if the merge falls back to writing conflict markers into the body, the note becomes permanently unrecoverable since decryption fails on a structurally broken body
Second, external tool encryption. even outside that project, nothing stops a user from pasting PGP-encrypted or any other externally encrypted content into a note body. The merge engine has no way to detect this, so auto-merging two versions of such a note line by line could silently corrupt the content with no indication to the user
Third, and more fundamentally, mrjo118's position was that no change to a note should happen without the user being aware of it. so even when a merge is technically safe, silently applying it means the note has been modified without the user's knowledge.The UI exists to keap the user in control of every change
to answer your question about what diff3Merge handles automatically, in simple terms when both sides make changes to different sections of the note (like device A inserting a line at the top and device B inserting a line at the bottom), diff3Merge quietly accepts both changes and includes them in the final result without telling anyone and also same with when only one side makes a change to a section.This mean it treats it as a non issue and moves on. That sounds fine, but the problem is the user never sees it happened their note changed silently. That is exactly what we are trying to avoid, so even though diff3Merge is technically capable of handling these cases, we are intentionally not using that silent behaviour
relevant link: https://github.com/bhousel/node-diff3?tab=readme-ov-file#diff3merge
So, this is also why diffIndices was chosen over diff3Merge because diffIndices gives raw regions that we classify and surface to the user ourselves, whereas diff3Merge silently absorbs one sided changes into the result without the user ever seeing them
The main problem is that if a user has binary content in a note (even if encoded), it is possible that auto merge may corrupt the contained data. Using git as an example:
Can git auto merge corrupt binary content which has been base64 encoded
ChatGPT:
...yes...
In Git, this is not so much of an issue, because it contains a full history which is usually retained forever. But the note history mechanism in Joplin to be honest is not reliable enough to fix such issues, which may have been silently broken for an indefinate period (broken code changes in Git are usually more obvious, because it would normally result in a compile error or bug), regardless of how rare it might be that this would happen.
For this reason, I'd argue preserving data integrity is priority over automating the conflict resolution where possible.