Adding this for reference, but superceded by the latter part of my reply
and we will use regex to make it performative.
I think the MANIFEST part would need to be mandatory even if there are no ids, to ensure efficiency for scanning purposes, but it is going to mean you can’t remove the prefix by a fixed length anymore for every Note load, although the fallback detection logic of encrypted notes can still check by a fixed length substring, which is where the performance matters more. So those cases are probably ok.
What I’m more concerned about though is scanning the resources contained in the note on every save. As the note editors auto save as you type, this could slow down the save operation on large notes. You should make sure the call to scan and update the note_resources is fire and forget, rather than waiting for it. I don’t think its necessary to queue those changes as the calls could build up due to the slowness of it, and you’ve already proposed a contingency if encrypted attachments end up being used in unencrypted notes, and proposed a means to view encrypted attachments from the attachment management screen.
JOPLIN_CIPHER:MANIFEST(new_id_1 : original_id_1):[AES_PAYLOAD]
Best to use a different separator than a colon within the manifest, so you can use the ending colon as an anchor for the regex.
Further thoughs
Taking step back, if you make a copy of a resource and encrypt the copy, isn’t that defeating the purpose of encrypting a resource, as the encrypted resource is still publically available? I think it would be simpler and make more sense to simply add a visual indicator on embedded resources and resource links which are not encrypted within an encrypted note (in a similar way to showing a vault locked placeholder image for an encrypted resource in an unencrypted note).
So when you encrypt a note, scan the resources in the body and for any which are found on note_resources, do not encrypt those resources and display a warning that some resources were not encrypted because they are used in other notes, please upload a copy of this resources if you wish to encrypt them. Lets say we ditch the MANIFEST. In terms of if an encrypted resource is used in other encrypted notes, those references wont be in note_resources, but we don’t care, as the is_locally_encrypted flag will prevent against double encryption. On decryption of a note, if we decrypt a resource used in other encrypted notes, the same concept applies. If it is publically available in any note, there is no reason to care about it being public in other encrypted notes. In those notes, you would just see the indicator that the resource is not encrypted when you open the note. To prevent confusion for users though, when decrypting any note which contains at least 1 resource, you could present prompt (with the option to cancel) which warns that all contained resources will be decrypted, and if shared in other notes, they will be decrypted in those notes also.
The other situation to consider is when you add or remove resource references to notes without toggling encryption on the note, but where the resources have various encryption states. Just having a indicator for the encrypted state of the attachments when it doesn’t match the state of the note should be sufficient. The only area of concern here is if you remove the only reference of an encrypted resource in an encrypted note (or other edge cases where note_resources has not yet been populated for uses of a resource), then there isn’t a specific UI available to decrypt it. I guess in this case though, the user could just download the resource from attachment management and reupload it, or more likely they don’t need to resource anymore and want to delete it, which they can do from attachment management anyway.
The only thing you need to be careful with is to make sure that you do not double encrypt or double decrypt resources with the local encryption. This will be controlled by the is_locally_encrypted flag on the resource. However you also need to include fallback logic to detect if the resource is encrypted upon selecting the resource for encryption / decryption in the encrypt / decrypt note flow and for rendering / loading resources in all cases, so use an option gated Resource load / save for view and edit of notes on desktop, mobile, cli and api. This would involve using a JOPLIN_CIPHER prefix when encrypted, in the same way as the note body. Even though I don’t think you can update a resource within Joplin, functions which allow reuploading data could still strip the is_locally_encrypted flag from the server item, if used on an old version of the client. So where an option gated Resource load / save is used, if is_locally_encrypted is undefined, check for the prefix update the is_locally_encrypted flag accordingly. For better optimisation, if the resource is not encrypted, update the is_locally_encrypted flag to false in the same save call used to save the encrypted content.
In summary, the proposed solution for resource management
- Encrypted resources should be viewable with a password prompt on resource management on desktop and mobile, and there should be a visual indicator shown on the list item for encrypted attachments
- An is_locally_encrypted column should be added to the resources table and the server item
- Option gated Resource save / load should be used to encrypt / decrypt embedded and linked resources via the firewall. This logic should include adding and removing of a JOPLIN_CIPHER: prefix on encrypted content, and include migration logic where is_locally_encrypted is undefined
- Resources embedded or linked in notes should have a visual indicator when the encryption state does not match that of the note. Encrypted resources in an unencrypted note do not need to be readable, but should indicate that they are locked. Additionally, consider if encrypted resources should always have an encrypted indicator for user peace of mind, due to the possible mixed states
- When encrypting a note, scan the resources in the body, and where a resource is included in other notes, do not encrypt it and present a warning in this scenario
- When decrypting a note containing at least 1 resource, present a warning prompt about the resources being decrypted in other notes if shared, with the option to cancel
Additionally, it is worth considering adding a confirm prompt (which is always shown) for the encrypt / decrypt all in notebook functions, as the cascading encrypt / decrypt would have an affect on mixed resource encryption states as well
UPDATE: In order to cater for possible edge cases where resource usages have not yet been scanned, for a resource in a note which is about to be encrypted, the following can be done to allow decrypting encrypted resources appearing in unencrypted notes:
For encrypted resource in an unencrypted note, show a locked placeholder in place of the embedded resource or resource link. When you tap this placeholder, it will prompt ‘This resource is locked, do you want to permanently remove encryption for this resource in all locations?’. If choosing yes, upon entering the password it will decrypt the note and save it in its unencrypted state.
UPDATE 2: While the resource management solution no longer involves calling setAssociatedResources directly, you do need to ensure the service which cleans unused resources will exclude clean up of resources where is_locally_encrypted is true. This means that orphaned encrypted resources will have to be deleted manually, but I think that is an ok compromise to avoid the complexity required to associate them.
An optimal way to do this could be to alter the NoteResources.orphanResources function to join to the resources table and exclude encrypted resources there. Please include details about this in the “Resource Handling, Mixed-State UI, & Option-Gated Resources” section.