DecryptionWorker error, Unknown decryption method 5 - decryption failing in joplin linux terminal client

Support
1

Question: When should we anticipate the updated linux terminal client (v1.0.197 or recent) to be updated and released via npm, or how can I download and install the latest release/pre-release of the linux terminal client today, other than building and installing from source?

Currently, the recent encryption change is causing decryption to fail in the joplin Linux terminal client, for any items that have been edited in the updated iOS app (or updated desktop client), and synced to other clients.

Now that I have updated the Joplin desktop app, and re-encrypted all notes, the desktop app is working. However this breaking change has crippled my joplin linux terminal client (my primary client) and disabled my workflow.

Joplin linux terminal client error message:

DecryptionWorker error, Unknown decryption method 5

Environment

Joplin version: joplin 1.0.161 (prod)
Platform: terminal client, linux
OS specifics: Ubuntu 18.04.4 LTS, bionic, 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

&

Joplin version: 1.0.197
Platform: Joplin desktop app, linux
OS specifics: Ubuntu 18.04.4 LTS, bionic, 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

&

Joplin version: 10.0.46 (updated with the past week)
Platform: iOS

Describe what you expected to happen

The Joplin terminal client should be released concurrently with other platform updates, especially when potentially breaking changes such as new encryption algorithms are introduced.

Alternately, perhaps an improved migration path would be to introduce the new encryption method as optional (recommended) in the iOS app (which updated automatically, and without notice started using the new encryption algo), until such a time as all user's devices may be updated simultaneously and only then should the data be re-encrypted.

Logfile

Previously, prior to updating joplin terminal to joplin@1.0.161, the error message was like...

2020-03-31 14:01:10: "DecryptionWorker: error for: 1a52... (revisions)", "Error: Unknown decryption method: 5
Error: Unknown decryption method: 5
    at EncryptionService.decrypt (/home/craig/.joplin-bin/lib/node_modules/joplin/lib/services/EncryptionService.js:363:52)
    at EncryptionService.decryptAbstract_ (/home/craig/.joplin-bin/lib/node_modules/joplin/lib/services/EncryptionService.js:431:33)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async EncryptionService.decryptString (/home/craig/.joplin-bin/lib/node_modules/joplin/lib/services/EncryptionService.js:496:3)
    at async Function.decrypt (/home/craig/.joplin-bin/lib/node_modules/joplin/lib/models/BaseItem.js:366:21)
    at async DecryptionWorker.start (/home/craig/.joplin-bin/lib/node_modules/joplin/lib/services/DecryptionWorker.js:169:29)", "{"id":"1a52<...>","parent_id":"","item_type":0,"item_id":"","item_updated_time":0,"title_diff":"","body_diff":"","metadata_diff":"","encryption_cipher_text":"<...>{\"iv\":\"z/btRab9upFo01ynC9Z1Gg==\",\"v\":1,\"iter\":101,\"ks\":128,\"ts\":64,\"mode\":\"ccm\",\"adata\":\"\",\"cipher\":\"aes\",\"salt\":\"<...>=\",\"ct\":\"UY1V<...>\"}","encryption_applied":1,"updated_time":1585676636500,"created_time":1585676730747,"type_":13}"

Now, after updating joplin terminal to joplin@1.0.161 the error message is `DecryptionWorker: ... decryption has failed more than 2 times - skipping it``

2020-03-31 14:07:52: "DecryptionWorker: starting decryption..."
2020-03-31 14:07:52: "DecryptionWorker: 48e6c6c338774cc1a793c129bf6014fc decryption has failed more than 2 times - skipping it"
2020-03-31 14:07:52: "DecryptionWorker: e0bdbc62a0c4459eb159ea49996d41bc decryption has failed more than 2 times - skipping it"
2020-03-31 14:07:52: "DecryptionWorker: c60bd2fbc24645aa83dbc417e49c7e06 decryption has failed more than 2 times - skipping it"
2020-03-31 14:07:53: "DecryptionWorker: b48b554f91e64828b31cbaae51e0ace0 decryption has failed more than 2 times - skipping it"
2020-03-31 14:07:53: "DecryptionWorker: 32bb980258fa46dea29456b455191494 decryption has failed more than 2 times - skipping it"
2020-03-31 14:07:53: "DecryptionWorker: 533d2d24c71843daaf43c488a2074a19 decryption has failed more than 2 times - skipping it"
2020-03-31 14:07:53: "DecryptionWorker: 3be871a53d3a4257acd929d41aa7fdda decryption has failed more than 2 times - skipping it"
2020-03-31 14:07:53: "DecryptionWorker: 6d628afd8d6a4bb2ad54c9f72b987b10 decryption has failed more than 2 times - skipping it"
2020-03-31 14:07:53: "DecryptionWorker: de19cdbf924e4662b5c59ad333a33e17 decryption has failed more than 2 times - skipping it"
2020-03-31 14:07:53: "DecryptionWorker: 2dc9e22003014d0bb20278842d5e3afc decryption has failed more than 2 times - skipping it"
2020-03-31 14:07:53: "DecryptionWorker: 36b30b7b55104822a6edd7149e0c3309 decryption has failed more than 2 times - skipping it"
2020-03-31 14:07:53: "DecryptionWorker: d87313bfd5b94b499d2c746fa8909842 decryption has failed more than 2 times - skipping it"
2020-03-31 14:07:53: "DecryptionWorker: 679e75972a6142f7bd81391e21cbecba decryption has failed more than 2 times - skipping it"
2020-03-31 14:07:53: "DecryptionWorker: efd0faa4976248229b8070f67e061678 decryption has failed more than 2 times - skipping it"
2020-03-31 14:07:53: "DecryptionWorker: b1b10eca70e843349ce5aa4022a43853 decryption has failed more than 2 times - skipping it"
2020-03-31 14:07:53: "DecryptionWorker: b941007e1c214189b3daaceb73ffcbb2 decryption has failed more than 2 times - skipping it"
2020-03-31 14:07:53: "DecryptionWorker: 1a52d53a68414fdbbb0829a49762509d decryption has failed more than 2 times - skipping it"
2020-03-31 14:07:53: "DecryptionWorker: completed decryption."

Thanks!

2

I usually released CLI versions when major changes were implemented. However, Laurent changed the release script, which failed for me the last time. I think it should work now (since I changed it locally), but Laurent suggested that I should hold off creating releases for now.

Thus you will have to wait until Laurent creates a new cli release.

3

Thanks for the reply, @tessus. Meanwhile, I suppose I’ll just build and run from the source tree.

Something for the core team to consider for the iOS implementation (even now), as perhaps an improved migration path for major changes such as a new default encryption method would be to ensure that the feature is opt-in (recommended) on all clients?

e.g., As my iOS device is configured to auto-update, the new iOS app installed and without notice started using the new encryption algorithm for any items edited on the device, which meant the items were unavailable once synced to other clients. It would be more appropriate for the iOS app to prompt the user to confirm that all clients have been upgraded, and only then should the new algorithm be used for any documents.

Similarly, the “wizard” in the Joplin desktop app to initiate re-encryption of all data should be updated to prompt the user to confirm that the terminal client (and all other clients) are updated before proceeding. Thanks.

4

@tessus, I think the cli app is good to go actually, so if it works for you now feel free to release it.

1 Like
5

@qcu, doing e2ee upgrade in a distributed setting is tricky.

Normally for encryption method updates, I’ll first create the new method, without enabling it for encryption, and make it available to all clients. That way all clients can decrypt the data. Then a few weeks later, when there’s a good chance everybody has updated their clients, I enable the new method for encryption. Now, even if some clients haven’t been updated yet, they’ll still be able to decrypt the data.

The problem here is that I forgot to update the cli client, so the above method doesn’t work. I can’t make the release now but maybe tessus can, otherwise it will have to wait till tomorrow.

6

Well, I could, but there's debug output again in the cli client. As soon as I start the client, I get the following in the middle of the screen:

18:33:54 DecryptionWorker: cannot start because no master key is currently loaded.             ||
18:33:57 RevisionService::maintenance: Starting...                                             ||
18:33:57 RevisionService::maintenance: Service is enabled                                      ||
18:33:57 RevisionService::collectRevisions: Created revisions for 0 notes                      ||
18:33:57 RevisionService::maintenance: Done in 108ms                                           ||
18:34:03 SearchEngine: Updating FTS table...   │|                                              ||
18:34:03 SearchEngine: Updated FTS table in 2ms. Inserted: 0. Deleted: 0

Therefore I won't create a release. We really need a wrapper function so that this stuff is not printed in the cli app.

I suggest a complete re-write of the logging system. We should have created a GSoC project for that. :wink:

7

Yep, we should have increased the minor instead of the patch for that release so that we know which versions work with each other, but we didn't.

8

Thanks for everything gents. I’m good with the desktop client for now.

9

In case it's not obvious, it's a bug as we indeed normally don't pipe the log to the cli gui...

10

Yes, I know. The notion of re-wriing the logging system was rather kidding than anything else. (The smiley should have been one sentence before where I set it.)
Although it really could need some improvements. I wanted to work on it a while back, but never got around to it.

11

I don't know but @laurent had suggested a hack that works:

What if you comment out line 630 of lib/BaseApplication.js? That was recently changed to add console logging to the desktop app, but I guess it should be disabled when running the CLI app.

In that case, it might be possible to check which client is running and enable logging selectively.

1 Like
12

@RedDocMD, I’m simply going to comment out that line for now. I think it’s probably too much logging in the Electron app too, and of course it doesn’t make sense in the cli app.

1 Like
13

Right, but the current line already included a restriction for the the cli app, but for some reason it didn't work. I rather look into why that is.

14

There was no restriction for cli app, so if it was commented out, as it was on master, it would be active in both cli and desktop.

It wouldn’t do anything on prod though, so actually it would have been ok to release it.

15

Ah, my bad.

I misread console in this.logger_.addTarget('console', { level: Logger.LEVEL_DEBUG }) as terminal/cli and thought it would only output the line when in terminal/cli.

16

I saw that 1.0.162 was pushed a few days ago. Many thanks. The terminal client is working for the most part, and synced almost all re-encrypted notes from the other devices.

However, just thought it should be noted that:

1/ Although joplin terminal client is now syncing for any new/updated entries, there are three notes that were previously updated (on iOS) with a new key, while the terminal client was still running the old release. Even though all new and updated notes on any device are now syncing properly, these three notes are still failing to decrypt for some reason, and still showing up as encrypted in the updated terminal client...

Title: :key: Encrypted
Content: One or more items are currently encrypted and you may need to supply a master password. To do so please type e2ee decrypt...

If I try forcing e2ee to run, it reports "e2ee: Completed decryption." Even though, the terminal client continues to fail to decrypt these three problem notes.

2/ Unfortunately, no details appear in the log file now, as apparently all encryption/decryption related logging has been disabled with the latest npm release?

IMHO, there should be a log file entry, as before, if a there is a decryption failure after several attempts.

Question: short of recreating these three notes, on desktop or mobile, and copying the contents over is there a way to ensure that the terminal client keys are update properly (I recall a prompt to "update an outdated key" within e2ee settings on the desktop app, and perhaps this step is missing in the terminal client)?

P.S. Thank you for providing the terminal client! It is the most efficient client for me, especially with the custom keymaps.

17

Could you add an issue any this in GitHub please? I think I just need to enable the same Retry logic as on the other clients.

18

Done. #2981.

Thanks!

19

Thank you, @laurent!

To close this out for anyone else finding this issue, with cli-v1.0.163 or later run joplin e2ee --retry-failed-items decrypt from the command line.

$ joplin e2ee --retry-failed-items decrypt
Starting decryption... Please wait as it may take several minutes depending on how much there is to decrypt.
1 Like