Operating system
Windows
Joplin version
3.3.10
Desktop version info
Joplin 3.3.10 (prod, win32)
Device: win32, AMD PRO A10-8700B R6, 10 Compute Cores 4C+6G
Client-ID: ebb3c247ba844893ac337478ded450d1
Sync-Version: 3
Profil-Version: 47
Unterstützter Schlüsselbund: Ja
Alternative instance ID: -
Revision: d9ba532
Backup: 1.4.3
Freehand Drawing: 3.0.1
VS-Code Style Note Search: 0.2.2
Sync target
OneDrive
What issue do you have?
Hello Team,
I just migrated from OneNote to Joplin and run my first backups on Joplin.
The backup size is round about 1GB, the Joplin Desktop Data folders round about 3.5GB. I learned, that's because there is no revision of the notes and attachemets.
But how can it be, that a new migrated Joplin instance has so many revisions of notes and attachments?
Especially as the size of the OneNote Data Files and Joplin Data Files are similar. Did the OneNote Import include "OneNote Revisions" as well?
Best Regards and greetings from Germany
Andreas
I found something similar a long, long time ago. It turned out the data contained a great many orphaned images, images where the notes had been deleted. The backups contained only the images referenced by existing notes so it was much smaller. A backup and then a restore was the best way to eliminate the unused images. Newer versions delete these orphans automatically according to the history setting. Your OneNote import may have brought in a ton of attachments that are no longer used anywhere.
1 Like
I don't use the backup plugin. I don't know if the backup use the .jex technology.
Just in case... I you want to export your datas into a .jex file, do it 2 or 3 times in order to be sure of the same size of the .jex file. Sometimes sizes are different and it means datas are not exported correctly...
I've been using the backup plugin ever since the day it was invented and I've never, ever had a problem. I do a lot of playing around with different versions on 3 different Windows Pro machines (7, 10 and 11) and do backups nightly and restore to one machine or another every few days just to try new approaches. I guess I've been lucky that way, I've never lost anything. Well, a few times maybe through my own carelessness, but the backup to a JEX file has performed perfectly for years.
1 Like
Hello HarSel,
thanks for your feedback. Yes, I was thinking along similar lines. Just import a backup into a new Joplin and check whether everything is still there. ..... but how can I check that, other than randomly?
But the issue with the images (and attachments) is a good starting point for further checks, as I use a lot of images and attachments in Joplin (and previously OneNote).
If I use “Tool - Notes - Attachments”, would I see the orphaned images? How can I find out where these images are used? Is there a way to jump from the ID to the note where it is used?
Bye
Andreas
There's a plugin called "Joplin Batch" that I use frequently to check (and delete) orphan attachments. Install that and you will find it under the Tools menu. It will look through everything and tell you which attachments are not referenced by any notes. You can then tell it to delete these one at a time or delete them all.
1 Like
The backup is just a compressed "7Z" file. Nothing special about it. "Extract All" and you will see all the MD files and a folder of resources.
I don't know the conclusion you have from that but facts are there : the issue with .jex was a big deal for me last year... At the begining I didn't identify the origine of the loss of datas after a restauration (from 3Go to 1 Go) until I look at .jex size... When I was doing for example 3 jex exportations, one after the other, no of them was identiical. Since some months it works good. But I do at least 2 .jex everytime in order to be sure the size of the .jex file is identical.
Hello HarSel,
today I tried “Joplin Batch” and it deleted a lot of orphaned files, but the problem still persists.
Live Joplin instance - 5.19 GB
Full export of the live Joplin instance - 1.06 GB
Import of the full export into a new test instance - 1.1 GB
I could delete the live Joplin instance and create a new one based on the export ... but I'm not brave enough. What if important content is deleted? There must be a reason why the current live instance is so big? Is there a way to compare the current live instance with a newly created instance based on the export?
Any thoughts are welcome
Andreas
Are we measuring size differently to the way the computer would measure size.
Could you try finding a way to repack the sqlite database, programs like Bleachbit can do this for Firefox and Chrome for example.
Is it possible that this is similar to where you might use a virtual machine with an dynamically expanding hard drive to defer allocation to when it's needed, as you add files you'd see the VM get bigger, but when you delete them, it wouldn't get smaller.
For Joplin I imagine this might be behaviour on a schedule and the dev console might be able to trigger it; if not, it sounds like a neat idea, were it possible.
So summarised, the allocation on the filesystem might be 5.1GB, but there might be 3-4GB of data waiting to be pruned, or just the tables reallocated on disk at a later point to reclaim it.
I imagine when you reimport this back in, the relevant tables are being rebuilt from scratch, so wouldn't have what feels like file level fragmentation in close enough terms.
(maybe)
Hello everyone, hello @james-carroll,
thank you very much for your feedback. The problem is solved! I was brave! I deleted Joplin, reinstalled from scratch and then imported the backup. It worked great, no data loss and the size is now 2.8 GB. All good for me.
Now Joplin works flawlessly on Windows. The next challenge is my Chromebook running Joplin on Linux. Unfortunately my Chromebook has an ARM processor so I have to use Joplin from Snap.
Bye
Andreas
Could have been worth trying the VACUUM command on the SQLite database to see if it helps:
2 Likes