Auto Sync Block: 1.0.2
Backup: 1.5.1
Combine notes: 1.2.3
Easy Backlinks: 1.3.1
Favorites: 1.3.2
Freehand Drawing: 4.3.0
Menu items, Shortcuts, Toolbar icons: 1.1.0
Note list (Preview): 1.1.0
Test Plugin: 1.0.0
Sync target
Dropbox
Editor
Rich Text Editor
What issue do you have?
Turning on Encryption seems like a no-brainer. But I've spent the last week enabling Encryption on my Joplin Dropbox sync configuration, and have experienced some tremendous pain points, significant enough that I would actually considerably warn others with large database/resources before casually turning this on.
The primary pain points:
Encryption increases resources size by 2x. This unfortunately doesn't scale well. My note database went from 20gb to 40gb
Additionally I suspect this might be bugged?? Looking in my resources folder, I can still see the original files left behind, free to open, right beside their encrypted variation. Correct me if I'm wrong, but shouldn't the original unencrypted version be removed? Otherwise I'm not sure what the point is
Joplin always installs to C:\Users\yourname\.config?? This is an extremely strange location. But it also means I basically just walloped my poor C drive by enabling encryption.
Additionally moving your database OUT of this folder looks complicated, in that there is no built in means to move it it safely. From what I can tell you basically have to symlink and hope??
The backup plugin greatly compounds this problem, because now instead of making sure I have to keep 20gb free for a backup to be successful, I now need to keep 40gb free. This is extra frustrating because there is a flaw with the backup plugin in that it doesn't even use the Temporary folder specified to create the backup, it only uses that folder after making the backup locally on the C drive, kind of defeating the point (I've reported this as a bug, twice) . As a result everyday when the Joplin backup tries to kick off, it drops my C drive to zero, doesn't even make a successful backup but stores that incomplete backup anyways. This is quite serious, because not only will you have an incomplete backup to work from, it will also discard previous backups that WERE successful and complete.
I'm at my wits end here because I have tried scrapping and optimizing as much as I can with my nvme C drive, but squeezing another 20-30gb out just so a backup can successfully complete is ridiculous, especially when I have other drives (one dedicated purely for backup) with tons of space.
I've tried symlinking the tmp folder itself, but that doesn't work because it gets destroyed (I assume because Joplin always creates a new tmp folder on launch?)
After Encryption was completed on the desktop (took about 6-8 hours leaving it open in the background), it took me leaving my iPhone ON and Joplin open, for 4-5 days before the re-sync was complete on the Joplin mobile app (attachment downloads set to Auto). I'm really not sure what happened here, it's possible I launched Joplin too early while the desktop encryption was kicking off and it got into a confused state. Either way, this considerably sucked.
I'm looking for any advice or help on the areas underlined. Searching the forum just brought me to a lot of previous threads talking about this that were auto closed. As it stands I either have to turn Encryption off, or Joplin's backup off (and create my own manual scheduled backup)
I think it would have helped to read up on what the encryption in Joplin does before enabling it. It encrypts the data during transport from client to server and on the server (in your case Dropbox), but Joplin does not currently support what is called encryption at rest (keeping the data encrypted while on your device). If you want to encrypt the data on your device, you should can enable encryption at OS level on your devices (generally considered a good practise).
It is a current limitation that E2EE uses about twice the disk space after doing a full sync, which you can read about on the forum. The majority of users don't have massive profiles so it's not really an issue. The main reason for this is when you download encrypted resources, they are decrypted for local access, but the encrypted versions are not deleted locally. So actually the encrypted versions in your resources folder are 'junk' data, and the decrypted versions are what is expected to remain. There is an open issue for implementing crypted file cleanup, with a few attempts made to address it Delete .crypted files when the app starts · Issue #9093 · laurent22/joplin · GitHub
This is extra frustrating because there is a flaw with the backup plugin in that it doesn't even use the Temporary folder specified to create the backup, it only uses that folder after making the backup locally on the C drive,
This is quite serious, because not only will you have an incomplete backup to work from, it will also discard previous backups that WERE successful and complete.
The ideal solution would be to get a larger system drive, though I'd understand push back given how expensive hardware is these days. An alternative could be to point Joplin data to an secondary drive if your rig supports it, or an external drive. The official method for this is to create a symlink to an external drive path at C:\Users\yourname\.config\joplin-desktop. just be careful not to disconnect the drive while Joplin is running if you do this, as ithat would risk data corruption
4-5 days before the re-sync was complete on the Joplin mobile app
How much longer did that take compared to when you setup your iphone originally? You'll kind of just have to live with for new device setups unless you want to spend the time to disable E2EE again and resync all your devices. Incremental sync should be pretty fast for note data, but I did observe large attachments can be very slow to decrypt on mobile (tested on Android). So you'll have to see what your experience with attachment downloads is like, and make a judgement call whether it's actually better in the longrun to just turn E2EE off again.
Also consider that I did warn that things would be much easier for you if you cut down your primary data set in Joplin and kept a separate archive. And also that using Dropbox for such a large data set is not recommended.
The original un-encrypted sync I would say was about 1-2 days. The encrypted re-sync was noticeably slower in its progress due to the encryption.
The primary pain point of this, is that the Joplin app on iOS MUST be running and focused with the iPhone ON for the sync to make any progress (there is no background app sync for Joplin iOS, which I imagine is its own set of headaches in iOS). The major brunt of the sync was made with turning off auto-lock and leaving Joplin running with the phone on and plugged in overnight, though I attempted to leave it also running for most of the day as well.
Additionally during the sync, any new notes made on the iOS Joplin app wouldn't appear until it is completed.
Based on my testing whether the dataset is 20gb or 20mb, the performance in Joplin is fairly the same with the same performance hiccups (you and I have been over the continuous sync UX lag). Outside of this Encryption bloat and the Backup plugin having some extremely questionable logic, using Joplin with a large dataset through Dropbox hasn't really been bad at all once it all was finally setup and running.
These issues mentioned here are largely issues with what felt like red-flags towards odd or un-optimized disc usage, while semi-forced to be on a MAIN drive, that clearly compounds considerably the larger the dataset goes. I also do kinda feel scale is something Joplin should be at least glancing at once in awhile, if it's meant to be a long-running note taking app still going in 10-20 years.