Orphaned revisions

Features
1

Hello,

when installing Joplin on a new terminal, by default, a notebook and a few introductory notes are created. I always start by deleting the “Welcome” notebook, but items are synced to my Nextcloud server anyway. Basically it consists in 3 orphan resources and 5 orphan revisions. Notes are properly deleted.

I’ve removed the orphan resources using Joplin’s Web Clipper API after reading this post Is there a way to force Joplin to run the unused resources cleanup? and checking @tessus great script source code on github.

However there is no API to delete orphan revisions. Checking the DB I find traces of these items in the revisions and sync_items tables.

I was wondering if the orphan revisions are cleaned up properly as well after the configured delay in Joplin’s options ? Is there a way to clean up these orphan resources without waiting for the history cleanup ?

I could erase the files and delete the corresponding items from DB manually (in the revisions and sync_items tables) but it doesn’t feel right.

Any recommendation on removing these orphan revisions ?

Thanks in advance.

2

Yes they are deleted after the set limit (90 days by default).

3

Extra ! Both the files and the DB are cleaned up I assume ? Thanks

4

Hello again,

so here is the deal, I’m using Joplin 1.0.224 on an Ubuntu machine (20.04 focal fossa). I’m currently downloading 1.0.226. I sync to a home hosted Nextcloud instance installed on a raspberry pi. The revision history is set to 1 day. Obviously I’ve been waiting for a few days before writing this post. Orphaned revisions aren’t removed from the remote (i.e. Nextcloud).

Below you’ll see my sync status as shown in Joplin:

Attachments

Not downloaded: 0
Downloading: 0
Downloaded and decrypted: 3
Downloaded and encrypted: 0
Error: 0

Sync status (synced items / total items)

Note: 25/25
Folder: 5/5
Resource: 7/7
Tag: 0/0
NoteTag: 0/0
MasterKey: 1/1
Revision: 0/0
Total: 38/38
Conflicted: 0
To delete: 0

Folders

misc: 3 notes
science: 1 notes
computer science: 17 notes
music: 3 notes
Recipes: 1 notes

When looking at my Joplin sync folder on Nextcloud I see the following:

$ ~/Nextcloud/Joplin$ ls -A | wc -l
152

ls returns a total line and 3 dotted folder, thus I have 148 .md files actually representing Joplin’s content versus 38 expected according to Joplin’s sync status.

When I open some of the files on Nextcloud a lot of them have a parent_id. I assume many of them are old revisions which haven’t been purged on the sync remote.

Then my question is: is this normal ? I would have expected revisions to be purged ?!?

Am I correct to assume I should only have 38 .md files (as shown by Joplin’s sync status report) in Joplin’s Nextcloud folder ?

I’m thinking about writing a bash script to get read of these orphaned revision files as long as I haven’t too much yet. I intend to track the revisions (as they are chained back to the notes) to make sure that I don’t remove a wrong file by mistake.

What do you think ?

Thanks in advance.

5

So … after changing my revision history to 1 day and making sure that there was no revision anymore in the DB (as shown by Joplin’s synchronization status and highlighted in my previous comment), I’ve created a bash script to remove all the orphaned revision files.

This works for me because I’m running Ubuntu on my computer (i.e. I have bash), I sync Joplin to Nextcloud and Nextcloud is also synchronized locally on my computer. In other words everything’s performed locally on my computer. Once the cleanup’s done, it’s automatically synchronized back to Nextcloud (I assume this would work similarly for Dropbox for instance).

So far so good, Joplin sync folder now shows the expected number of files (38 in my case) and Joplin is synchronizing without error.

If you’d like I can share the script but it’s tailored for my needs and my configuration. Thus you’d be using it at your own risks and you’d better tweak it to your needs. Still it could help you start quickly if you have a similar issue.

Anyway @laurent there seem to be a bug with the revision cleaning in the sync directory (at least for me).

6

Pour ceux qui veulent aller vite, si le dashboard de status de synchronisation de Joplin indique bien 0 révision, dans votre répertoire de syncrhonisation, la commande ci-dessous vous donnera la liste des fichiers de révisions orphelins (puisque plus référencés en db):

grep 'type_: 13' *.md | awk -F ':' '{print $1}'
7

Before I open a bug report, i wanted to ask if i might have missed something?

I checked my synctarget and found 2175 orphaned files.
All 2175 files are of the revision type (type_: 13) and I can not find the ids in the database table revisions .

My note history is set to 90 days, but the files go back to September 2020 when I set up a new synctarget and all files except 5 are older than 90 days.

This might be a bug, or do these ids exist in another table?

Joplin version: 1.7.11 / 1.8.2

Sync status (synced items / total items)

Note: 5327/5327
Folder: 54/54
Resource: 8974/8974
Tag: 1714/1714
NoteTag: 18639/18639
MasterKey: 0/0
Revision: 388/388
Total: 35096/35096
8

@JackGruber as far as I know, after the 90 days (in your case) revision ids are purged from DB. Thus if you have orphaned files on your remote (or its local sync folder), there is no reference to then anymore in DB.