Good point on the upgrade migration, I traced the scenario: if an old client already synced body=JOPLIN_CIPHER:xxx before upgrading, local_cipher_text would be empty and we'd need fallback detection in Note.load too, duplicating sync-side logic.
keeping cipher in body with the prefix avoids that. i traced the code paths that need explicit handling with this approach:
SearchEngine.ts: both doInitialNoteIndexing_ (L140) and the incremental sync query (L238) currently select all non-E2EE, non-conflict notes. both need an is_locally_encrypted filter to exclude locked notes from FTS
RevisionService.ts: collectRevisions (L151) has the same pattern, needs a matching filter to skip locally encrypted notes
Note.ts: beforeChangeItemJson (L849) would capture cipher values that change per-save due to IV rotation, so it likely needs explicit suppression when is_locally_encrypted is true
the tradeoff: with body="" and a separate column, some of this suppression is implicit. with cipher in body, it becomes explicit. but migration simplicity and old-client compatibility are worth it since the suppression points are well-locatable and testable.
i'll rework the first post to drop local_cipher_text. one thing to confirm: on gated load, plaintext is placed on the in-memory model for editor/viewer use, while for locally encrypted notes the persisted DB body remains JOPLIN_CIPHER:... . is that the intended behavior?
one thing to confirm: on gated load, plaintext is placed on the in-memory model for editor/viewer use, while for locally encrypted notes the persisted DB body remains JOPLIN_CIPHER:... . is that the intended behavior?
Yes, on load the body value is decrypted on to the same field if useLocalEncryption = true, and when not gated the encrypted cipher (including the prefix) will be set to the body in the model. And the same for save, but the other way round. Therefore in the db, the cipher is stored with the prefix still intact
@mrjo118, it looks like the Secure Notes plugin is relatively popular and well maintained. Is there something missing from it that requires the feature to be part of the core application? I'm wondering if instead of having an encrypted note project, we could have a project that focuses on adding plugin API to support Secure Notes or other similar plugins?
Expanding on resource encryption, thereās not necessarily a limitation for encrypting resources, but there may be limitations performing clean up of encrypted resources. It looks like plugins can have startup tasks, but would they allow access to delete files, particularly on mobile? Also on the desktop app, because the application data would not be in a secure sandbox like on iOS on Android, so you would want to have some kind of onExit task to clean up resources as well, which I donāt think you can do with plugins.
Expanding on revision control, because changes to encrypted text would result in the whole cipher being completely different, the diffs used to create revisions effectively mean each revision contains the whole note contents every time. From the perspective of storage space, this is maybe an acceptable caveat of encrypted notes. However, revisions are evaluated by merging diffs, and so an encrypted note with a long revision history could present a performance issue to evaluate revisions from much large diffs. Originally I propsed to opt out if revisions completely for encrypted notes, but thatās not ideal to be honest. One potential solution would be for notes marked as encrypted, to have a special case in the revision service to always create a revision with no parent id, so that the revision will always contain the full contents and wont need to merge with any other revision. For a plugin, there is neither a way to opt out of the revision service or to mark a note in a special way that it will be treated differently by the revision service.
Additionally for revisions, I realised yesterday that there is an edge case whereby if there are unsynced revisions on one device and you enable encryption on another device, due to the distributed nature of revisions, if you delete the local revisions upon enabling encryption, that wont stop the unencrypted data being uploaded from a revision on the other device later. I was thinking to deal with this, you could store a new flag on the revisions table which marks whether the note was encrypted at the time, and then have some kind of check when the encrypted flag on a note changes from null / false to true on save (including save triggered by the sync), that all revisions without the encrypted flag would be deleted. But this again would be a core code change and is not something that can be done in a plugin
@keshav0479, rather than listing the reasons why it can't be done as a plugin, maybe it would be worth listing solutions on how the core app could be changed to address these issues?
Thanks, that's a good reframe. the core changes i see:
RevisionService.collectRevisions (L151) needs encrypted-note handling. cipher changes completely between saves, so diff-based revisions store the full content every time. standalone revisions (no parent_id, skipping the merge path at L105-109) could fix this. there's also a cross-device edge case where unsynced plaintext revisions could upload after encryption is enabled on another device, a flag on the revisions table could handle this
SearchEngine.ts has two indexing queries (L140, L238) that include all non-E2EE notes. adding an is_locally_encrypted filter to both would exclude encrypted notes from FTS
for editing, a plugin has to build its own editor panel since it can't gate Note.load across all paths. option-gated Note.load lets the existing editor work transparently
Note.save's beforeChangeItemJson (L849) captures full cipher diffs on every save. needs suppression for encrypted notes
decrypted temp files need cleanup on app quit, but there's no plugin onExit hook
search filter and an onExit hook could be plugin APIs, but revision handling, Note.load gating, and item_changes suppression are internal to core services. i'll update section 3.7 to frame these as core changes needed.
@keshav0479 To finish off my analysis about the conflict resolution considerations. My concern was that storing the encrypted note content in a separate field would mean that the value would be hidden from any kind of conflict resolution, making it impossible to resolve it in any meaningful way. This however is a non issue for the explored approach of not using a separate field. While the content is still in encrypted form, it wont be legible to compare differences, but as long as the user is able to choose one version, or both, then the contents can at least be verified by unlocking the notes from the original notebook later. Other considerations I think are out of scope for this project, but instead are factors to consider for the automatic conflict resolution project instead.
Regarding your last post about core changes for a plugin implementation, some additional considerations:
Within a plugin, encryption keys cannot be added to the sync, so without core changes, the key must be embedded in the note (as per the secure note plugin). This means to change a password, you must decrypt an individual note and re-encrypt it with a new password. Can a UI with the ability to add and manage named encryption keys be added, which are then reusable and included in the sync? Then a key can be created with a name matching the name of the relevant plugin, and this key can be used for all notes encrypted by this plugin
How could locking / unlocking all notes in a notebook be implemented? Presumably it should be easy enough to add a new plugin screen to allow doing this using the named key relating to the plugin. While the previous point could allow encrypting different notes with different encryption keys, this would be a stumbling block for batch note locking, as it would require scanning all note bodies in the notebook to determine which key to use, and then potentially require entering multiple passwords to unlock for multiple different decryption keys. Regardless, this feature would require validation to prevent double encryption / decryption
For your point about the revisions service, the issue is a bit bigger than just adding a flag on the revisions table to handle the cross device edge case. You need a means to mark a note in the db as encrypted, so the flag can transfer to the revision. This presents all the same issues with catering backwards compatibility when syncing with old clients and when upgrading after syncing encrypted notes from new clients on an old client. You also need a means to determine an item is encrypted when an is_locally_encrypted flag on the note is lost due to syncing with an old Joplin version - this canāt be done on save as when E2EE is enabled, the body will remain encrypted until after it is saved in the db. A possible compromise might be to allow revisions which are either unencrypted or uploaded by the sync using an old Joplin client (is_locally_encrypted is missing on the sync item) to be downloaded on the new client. However, whenever a note is opened which is detected as a locked note (on every load), it will perform both a migration and a clean up task - this task will update the is_locally_encrypted column on the note to true if it is undefined (and save it immediately) and it will delete all revisions for the note where is_locally_encrypted is not set to true for the revision. While the unencrypted revision may be persisted for some time, it is an edge case and it would prevent the user ever seeing the unencrypted revision in that client, as if would be deleted upon opening the note. The delete revisions api will need to be extended with a flag to enable this, and there would need to be an extention to the note update api to allow setting the is_encrypted flag
āfor editing, a plugin has to build its own editor panel since it can't gate Note.load across all paths. option-gated Note.load lets the existing editor work transparentlyā - how were you expecting option gating to work for a plugin implementation? I guess the above approach of performing a migration which will ensure the is_locally_encrypted flag is set upon opening a locked note, should allow for an option gated approach. In order to make this extensible and not tied directly to a single plugin though, maybe a local_encryption_key_id column is needed on the note table too (which is also updated on migration on note load), and the update note api needs to be updated to allow setting this (by key name, or provide a whole bunch of apis to manage these named encryption keys). If using this approach though, it would be important to implement some kind of safeguard to prevent alternate encrypted note plugins from corrupting notes encrypted by your plugin though. You could potentially handle that by including the plugin name in the encrypted body though (and validating against it for all operations), like the secure notes plugin does
Regarding resources, does the plugin api allow replacing resources and retaining the same id? Plus a new consideration I hadnāt thought of earlier which affects all approaches: how do you handle when a note with a resource is encrypted, and that resource is used in other notes which are not encrypted?
Note that all 3 columns mentioned for adding should be added onto the server item, and if they are lost during sync using an old Joplin client, they will get repopulated by the migration logic upon opening a locked note (with the exception of is_locally_encrypted on the revisions table which gets copied over from the note). As a compromise, all revisions created on an old client get deleted upon opening the note (because they will not be standalone revisions and are undetermiend if encrypted content or not), and encrypted ciphers can end up unnecessarily in the search indexing. But once the migration has completed, they will be wiped from notes_normalized on the next restart of the app
on the revision migration: doing migration + cleanup on every note open makes sense. if a locked note is opened, set is_locally_encrypted if undefined, then delete revisions for that note without the encrypted flag. simpler than tracking cross-device sync state and handles the edge case within a bounded window.
on shared resources: if a resource is referenced by both an encrypted and a non-encrypted note (note_resources is many-to-many), encrypting the resource would break the other note. resource replacement with same id is possible, but shared links are still the hard constraint. safest approach: only encrypt resources exclusively used by encrypted notes. if shared, leave unencrypted and surface a UI warning when locking.
on scope: i'd keep this project to a single vault master key. named key architecture and multi-plugin support is interesting but better as a follow-up.
I'll update the first post to frame section 3.7 as core changes needed and add shared resource handling to 3.9.
safest approach: only encrypt resources exclusively used by encrypted notes. if shared, leave unencrypted and surface a UI warning when locking.
I think that is a fair approach. But do you have an idea how to check if revisions are used in unencrypted notes though? I know that either the revision or resource service does some kind of scan to check for resources not used anymore in notes or revisions, but Iām not sure how efficient it is, as it may need to scan the the body of all notes and revisions? Can you ensure this validation could be quick if the user has many notes and / or revisions?
on scope: i'd keep this project to a single vault master key. named key architecture and multi-plugin support is interesting but better as a follow-up.
Fair enough. If the encrypted note key is a specific key for that purpose, then it would be weird to add a UI in the core app to manage the key for a feature which does not exist in the core app. So the implementation must be in the core app to enable it to be synced, but the UI to manage the key should be implemented in the plugin instead, and appropriate apis must be implemented to enable this
for resource sharing checks - note_resources already indexes which notes use which resources, so checking against current notes is just a fast join query against that table. no body scanning needed there.
revisions are trickier since resource refs in revisions aren't indexed the same way. my plan would be to only validate against current note associations (fast path), and skip resource encryption if we can't confirm exclusive usage without an expensive revision scan. basically: be conservative, warn the user, and don't block the lock action on a slow scan.
on key management - that split makes sense. core handles key storage + sync + encrypt/decrypt APIs, plugin provides the UI for vault setup and lock/unlock. i'll restructure the proposal around that.
revisions are trickier since resource refs in revisions aren't indexed the same way. my plan would be to only validate against current note associations (fast path), and skip resource encryption if we can't confirm exclusive usage without an expensive revision scan. basically: be conservative, warn the user, and don't block the lock action on a slow scan.
Sounds sensible. You would need to make sure an unencrypted revision containing an encrypted resouse can open though. It could have the icon for a resource not found for embedded resources such as images. However for non rendered resources and if simply opening an encrypted resource from the manage attachments screen (currently in desktop, a pr is in review for adding to mobile too), either it should kpen a password prompt to allow opening it, or if too much impact, just display an error saying this resource is locked.
Also note that when having encrypted revisions support instead of full opt out, the unlock note ui would also need to be added for encrypted revisions. The secure notes plugin currently has this actually
for resources in old revisions- since the revision viewer renders markdown, encrypted images would hit the normal "resource not available" fallback the viewer already has. non-rendered attachments opened from the attachment manager can just return an error. adding a password prompt there feels like scope creep for the stretch goal, so i'd defer that unless you think it's needed.
on the revision viewer unlock - yeah that's a real deliverable i missed. NoteRevisionViewer would need a gate similar to the editor lock screen. i'll scope that into weeks 7-8 with the rest of the revision work.
On the resource management though there is a catch. It feels like there needs to be an is_locally_encrypted flag on resources as well. Among other validations which are needed, then you could also provide a proper āResource is lockedā error on the attachment management screen instead of just an unable to read file error.
If you remove the reference to the encrypted resource in the encrypted note, how would you manage detecting the need for decryption of that resource efficiently? What if the resource is used in other encrypted notes still? Is it really an option to not have dedicated management of encrypted resources with situations like this?
yeah, an is_locally_encrypted flag on resources makes sense. without it there's no clean way to tell "encrypted file" from "unreadable file" in the attachment manager.
i also traced a related issue in ResourceService.indexNoteResources() (L82-91) - it parses note.body to extract resource IDs via Note.linkedResourceIds. with a locally encrypted body (JOPLIN_CIPHER:...), that parse returns nothing, which would cause setAssociatedResources to mark all the note's resources as is_associated = 0, eventually orphaning and deleting them. it already skips E2EE-encrypted notes at L82 - locally encrypted notes need the same skip.
for the decrypt lifecycle when a resource reference is removed: NoteResource.associatedNoteIds(resourceId) (L75) returns all currently associated note IDs. joining that with notes.is_locally_encrypted tells us if any remaining note is still encrypted. if none are, decrypt the resource and clear the flag. same fast indexed query, no body scanning.
i'll update section 3.9 to cover the resource flag, the indexNoteResources skip, and the decrypt-on-disassociate lifecycle.
shared resource constraint (from discussion with mrjo118, post #26): resources can be referenced by multiple notes (note_resources is many-to-many). if a resource is used in both an encrypted note and a non-encrypted note, encrypting the resource file would break the non-encrypted note. approach: only encrypt resources exclusively referenced by encrypted notes. if a resource is shared with non-encrypted notes, leave it unencrypted and surface a UI warning when locking ("this note has resources shared with other notes - resources will not be encrypted"). resource replacement with the same ID is possible, but shared note_resources links are the hard constraint.
What if the resource is used in other encrypted notes still? [upon note decryption]
Section 3.9 does not address the latter question. Additionally, I think there needs to be some kind of indicator within the encrypted note, when a resource is not encrypted. Ideally there should also be an indicator for this on the attachment management screen on both desktop and mobile too (now that it is added on mobile in the latest unreleased code)
on the decrypt lifecycle when unlocking- same query in reverse, when a note is unlocked, for each of its resources, check note_resources joined with notes.is_locally_encrypted to see if any other encrypted note still references it. if yes, keep the resource encrypted. if no encrypted note references it anymore, decrypt the file and clear the flag.
on indicators -agreed. when a resource inside an encrypted note isn't encrypted (because it's shared with a plaintext note), there should be a visible indicator. inside the note, something like a badge on the resource link showing it's unprotected. on the attachment manager, a per-resource lock/unlock status, on both desktop and mobile.
on the decrypt lifecycle when unlocking- same query in reverse
Did I misunderstand this part?:
Is this logic used just for cleaning unused resources, or is it used for populating the note_resources table as well?
Iām wondering how would the note_resources records actually get populated for encrypted notes? Are they populated on save or is it done via a background task? If the latter, then you would not actually have a record of which resources are in use for encrypted notes, as it canāt be determined from the encrypted content
you're right, note_resources is populated by the background indexer parsing note.body, not on save. so for locally encrypted notes, relying only on indexer parsing isn't enough.
fix is to call NoteResource.setAssociatedResources from the gated save path before encrypting body, while plaintext is still available. and in ResourceService.indexNoteResources, skip locally encrypted notes so ciphertext bodies don't wipe associations. (to be clear - skip and continue processing, not break the queue like the current encryption_applied guard does, otherwise the indexer stalls on locked notes.)
for fresh clients where notes arrive already encrypted via sync - note_resources may be empty initially, but synced resources aren't immediately orphan-deleted because addOrphanedResources sets last_seen_time=0 for synced items, and orphanResources skips those. then when the user unlocks the note, we hydrate note_resources from the decrypted body.
read through the solution in #87. the architecture makes sense - option-gated Resource save/load mirroring the note pattern, is_locally_encrypted on the resources table and server item, JOPLIN_CIPHER: prefix with migration fallback.
one thing worth noting from the code: Resource.readyStatus (L164) already returns 'encrypted' for encryption_blob_encrypted (E2EE blobs). for local encryption we'd want a distinct status like 'locally_encrypted' so the UI can distinguish between E2EE-pending-decrypt and vault-locked resources, since the user action is different for each.
i'll align section 3.9 with the full lifecycle: skip shared resources on encrypt with warning, decrypt prompt on unlock, locked placeholder with tap-to-decrypt for encrypted resources appearing in unencrypted notes, and confirmation prompts for bulk notebook operations.