Mobile: failed decryption and sync

@t0dd Are you still experiencing the error on your Android device and still have the revisions present on the sync target? If so, would you be willing to test if that change works if I create a custom build using it?

You would need to uninstall the main Joplin app to install it, as the dev build won't be compatible to upgrade or install in parallel

Thank you for looking into this!

Be aware that switching to the bundled SQLite might cause database issues on certain Android devices:

@personalizedrefriger I think it is worth verifying if using the bundled sqlite fixes the issue regardless of whether we go with that change.

@t0dd I have modified the build pipeline on my Joplin fork, so that it publishes the built apk for my code. I have used it to build a custom apk including the required code change, available under the assets dropdown at this link: Release 1736437986 ยท mrjo118/joplin ยท GitHub

If you are still experiencing the issue on your device, have not yet attempted to remove your note revisions from the sync target and are willing to sideload this custom build, please could you do the following:
-Uninstall your existing Joplin app on your phone
-Sideload the app-release.apk file (google how to sideload an apk on Android, if you don't know how to do it)
-Connect your sync target and sync all your notes
-Respond back with the result once done (if it is successful, if you could screenshot the 'Configuration > Tools > Sync status' screen on the mobile app as well, that would be great)

I will take a look this weekend. Been busy.
UPDATE: Installing now. But I notice your version is 3.2.8. That is not an altered supported version. The version I had installed was 3.1.8. Therefore, if this fixes the issue, we don't know if it is because there was some fix in the 3.2 series of releases or your changes.

And done. Yup. Works. Whether it was your changes or the update to the client from 3.1.8 to 3.2.8, we now know a configuration that may work with certain older Android platforms. And since I don't know you from Adam, I am hoping you didn't just backdoor my cell phone and send my info to some offsite system somewhere. :wink:

@t0dd Thank you for doing this test for me. You can now uninstall the custom apk.

The reason I asked for a screenshot of the sync status screen was to verify that you still have the revisions present. Unfortunately the revisions are no longer present (there is only 5 instead of thousands), so the test result is not conclusive. I'm unsure if there is still a problem item present on your sync target or not at this point.

However, if you wouldn't mind to test one more thing, it's possible we can still get a conclusive result. If you could install the official 3.2.5 prelease found here Release android-v3.2.5 ยท laurent22/joplin-android ยท GitHub (that is the version my custom build was based on) and resync your profile, then if the error comes back, it will conclude that using the bundled version of sqlite resolves the issue.

Even if the result would not be conclusive, it is still of benefit to you, because you will have the official apk installed, which can later be upgraded via the play store (or sideload future prereleases without needing to uninstall first) once the next main release is out

1 Like

Ok. Now, I am annoyed. Your version works like a champ. The prerelease works like a champ. And the supported version now works like a champ.

I don't understand why the supported version works. I have uninstalled and reinstalled it 6000 times well after any revision history should have been retained. I just don't get it.

I don't know how to help you test your idea now. And that also ticks me off. I suppose just keep it in your back pocket and test it with the next guy who reports this problem. I don't know. Annoying. So annoying that I can't replicate this now. Grr!

Thank you for your help with this. On the bright side you can now use Joplin on your phone again without issues.

There is another way that the test can be done again from a slightly different angle, if you're happy to install some other custom apks. I won't send you uninstalling and reinstalling Joplin again as I know that is a pain. I've created 2 modified apks from Joplin 3.2.6, both of which I have given different package names (so they can be installed in parallel with official Joplin) and different app names, so they can be distinguished from each other in the app drawer.

JoplinHack (uses device sqlite): Release 1736794968 ยท mrjo118/joplin ยท GitHub
JoplinHackBundled (uses bundled sqlite): Release 1736795176 ยท mrjo118/joplin ยท GitHub
The code changes deviating from the dev branch can be viewed by clicking the hash next to the green tick on each release

If you install both of those on your device, then open the top note from the default notebook, the result will determine whether the cursor size is / is not set correctly. Basically both apps set the CursorWindow size to 1kb via the 'workaround' code, so if the limit is being enforced, it should hit an error. When this is the case, opening the note will open a white screen only, without the note contents. And if you go to Settings, Tools, Logs, you will see the CursorWindow error in the logs.

Additionally, can you attempt to import the following jex file on both apps and then attempt to open the 'Over 4mb note', and report back on the results (note that as it is a very big note, please allow up to 5 minutes for the note to render - if it still does not render by then, then just check the log if there are any errors):
big.jex (4.7 MB)
I limited the note to 4.8mb because it is already very slow to open on Android, and importing a 50mb note literally crashes Joplin, even on desktop

Further analysis:
So what I discovered is that when using the bundled sqlite binary of the react-native-sqlite-storage library, the 'workaround' which was implemented by the Joplin devs does not actually work. I think the reason for that is because the CursorWindow limit is something specific to the Android implementation of sqlite, so the react native implementation I suspect does not enforce any limit at all, hence why it appeared to work previously when I switched to the bundled version and tested syncing a 4.8mb note with this implementation, and there was no error. That would make sense because there is no such limitation with sqlite on Windows, Linux, Mac or IOS.

The workaround which was implemented is targeted to change the CursorWindow limit for the Android implementation only, and I still suspect the cause of the issue is that certain phones do not make use of the overridden class, to determine the limit to use.

I made these conclusions from the following results when testing on an Android 15 emulator:
JoplinHack - note fails to open due to CursorWindow error. When importing big.jex, opening the 'Over 4mb note' fails
JoplinHackBundled - note opens successfully with no error. When importing big.jex, opening the 'Over 4mb note' works successfully

@t0dd To confirm my theory is correct, the default welcome notes will open without error in both apps, while opening the big note imported via big.jex will fail on JoplinHack (due to override being ignored, but system limit being under 4mb) and succeed on JoplinHackBundled

@t0dd I've now built a 3rd apk as well:

JoplinHackVersion: Release 1736809592 ยท mrjo118/joplin ยท GitHub

This one is the same as JoplinHack, but additionally it will write the device sqlite version on startup. You can use this instead of JoplinHack, or just install this in addition. All I need specifically from this one is a full copy of the log file after opening the app.

Using this on the Android 15 emulator, it shows version 3.44.3 in use. I built another version locally using the bundled sqlite to see what version that uses, and it shows version 3.28 in use.

I don't have time to look at any of this this week. I will try to carve out time next week.

1 Like

@t0dd I've since found a way to get some output from the Kotlin code of the native Android app code, which I didn't think I was able to do before without asking you to enable some dev features on your phone, but I now am. So I've created a 4th apk:
JoplingHackErrorMsg: Release 1737661306 ยท mrjo118/joplin ยท GitHub

This one supercedes JoplinHackVersion, so you can don't need to bother installing the 3rd apk. When you open this app, it will have a popup message which can be dismissed by pressing the back button on your phone. Please screenshot the popup (if it is scrollable, screenshot all parts of the message) and send that as part of your investigations, and after dismissing the popup also export the full log and send that over.

@t0dd thank you so much for your offerings to do tests for me. In an unexpected turn of events, I discovered that the workaround for the CursorWindow size extension actually does not work on my own Android phone (Samsung Galaxy S9+ on Android 10). I had not attempted diagnosis on a real device previously as I was using Android emulator for investigations.

So I was able to do the tests myself and they have confirmed my theory. I'll do a write up of my findings another day, but I may have some good news, depending on results of a couple more tests

I have created PR Mobile: Fixes #11571: Use alternative fix to set the sqlite CursorWindow size to 50mb by mrjo118 ยท Pull Request #11726 ยท laurent22/joplin ยท GitHub with my analysis and a code fix

1 Like