GSoC idea: Sync Accountability

GSoC
1

Hi everyone,

This is not a proposal draft. I've been thinking about an idea I have for my proposal draft, and I wanted to discuss about it briefly before getting into the full proposal draft - particularly to validate the problem and the evaluate the feasiblity of the solution.

Would greatly appreciate any feedback from the maintainers about the validity of the problem I'm looking to solve! If it's OK, I'll begin writing the full proposal draft as soon as possible.

Problem

While Joplin has always been an offline-first note-taking app, it does have extensive cloud synchronization capabilities. Users are able to configure the synchronization target and behaviour to their liking. However, it can be challenging to work on the same Joplin data across multiple devices in one sync environment [not at the same time].

The first challenge is lack of sync transparency or accountability. There is no way to track which devices are synced to the same target. This may be confusing to new users, but more importantly, without the concept of accounts, there is no “guarantee” that gives the user confidence that all their devices are now connected to the same target and will synchronize with each other.

If I have more than two devices synced to my target, it can be unclear whenever I’m fetching remote changes which device made the newest changes. It’s also unclear whether the other devices are up-to-date or not, and when they last synchronized. In my experience using Joplin in remote areas with poor internet connection, I always felt that having this feature would have helped me immensely to be aware that my last session didn’t get saved to the sync target.

At the moment, while every device can be configured to sync to any target, they’re oblivious to the other devices synced to the same target. I believe Joplin would benefit from more accountability / transparency for the synchronization process.

Solution

  • A “Device List” section on the Synchronization settings menu. Each device would list:
    • A device name or type (e.g. phone model name?)
    • When they last synchronized
    • A summary of whether their last sync was a fetch, an update, or both
  • The above would be powered by some sort of manifest or sync log - one file for each device, updated in the sync target every time a device fetches or updates an item during a synchronization.
  • It would also only be displayed for devices with synchronization enabled.
  • I was also thinking of a “complete purge” feature for both the sync target and the local device, but I believe that’s outside the scope of the problems I mentioned above.
2

Thank you for proposing this, but it's unclear what real problem is being addressed. Just because the software cannot do something, it doesn't mean it's a problem.

1 Like
3

The key problem that I see is that the lack of transparency described can be a pain point when it leads to a desync or a lack of a single-source-of-truth in a multi device environment. My goal is to enable an auditable sync model that is less "blind". I have three main points:

  • Having a more context-aware sync model, specifically manifest/sync log system, gives users more context to fix unexpected sync conflicts. Part of the issue I encountered was similar to this, and while the solution I proposed doesn't address the root cause of sync conflicts, I believe this sync model can be extended to solve it because the key idea here is to make each sync operation logged and traceable, attaching a lot of context and information to each synchronization. This 'context' can be used by any Joplin device in the environment. At the moment, I think needing to remember the changes we made can be a pain point for some users.
  • Lack of visibility into the devices linked to my sync target can cause a lot of confusion, even if there's no sync conflict at all. I encountered these two threads - one, two, three where syncs do unexpected things (from the user POV). I'm sure they have their own root causes, and this new sync model won't magically fix them if it is the users' fault. But we are empowering them with the telemetry so theyn can see the origins of the changes, e.g. where and when, to gives them clues for what happened and why.
  • Perhaps I poorly worded the solution and made it seem like a "Device List" was the primary outcome. I think the Device List is a side effect of the newer sync model I proposed, which in addition to the device list would allow us to: 1) to display the users more information when there's a conflict, 2) show more context on every synchronization operation (e.g. not only fetching changes from remote, but fetching changes from where? from my work laptop, personal laptop, or from my phone?)
4

Hello, sorry for now we won't add this feature, so I would suggest focusing on a different potential project