How can I re-import Deleted Files?

Support
1
  • version 2.8.8
  • operating system Windows 10

Hi, I started Joplin by creating notes into a single profile. I set up OneDrive as the sync target.

Then I realized I wanted my notes to be separated by concern (home life, business 1, business 2, etc).

I then created 2 more profiles (Default being the current one, plus 2 more for the 2 businesses). I exported notes accordingly so I started out basically with 3 copies of the same profile... or so I thought.

I then went back and started deleting irrelevant notes from the respective profiles, intending to have just the relevant notes in each profile. I unfortunately started with deleting notes for the meatiest profile away from the less meaty profile. I thought I was just deleting notes from a specific profile but when I looked at the other profiles I discovered that those notes were being deleted throughout all profiles! All the profiles had the same data!! So I deleted a ton of very valuable notes!

I did some digging and found that (thank goodness) there appears to be a Note History feature and many notes are still in the ether located at "C:\Users\USERNAME\OneDrive\Apps\Joplin". There are a lot of md files in there, and when I opened up a few, I can see content from notes that were deleted.

However, these md files are not standard, and have odd formatting as follows:

id: 0a4a56e02bae4c9a3510ab23413afac0
parent_id: 859bd8dc7ef14c0db2e677735a34e3f6
item_type: 1
item_id: 55d805eef1ff41dd98ecfc2f3570a062
item_updated_time: 1663211544309
title_diff: ""
body_diff: "[{"diffs":[[0,"lHost>\\n\\n"],[1,"...CONTENT OF MD FILE..."]],"start1":375,"start2":375,"length1":8,"length2":424}]"
metadata_diff: {"new":{},"deleted":}
encryption_cipher_text:
encryption_applied: 0
updated_time: 2022-09-15T03:15:00.764Z
created_time: 2022-09-15T03:15:00.764Z
type_: 13

I've read that Joplin modifies the md files to accept additional functionality so I believe I understand why it's formatted this way, however, I've tried to import the individual md files and md directories, both with and without frontmatter, and Joplin imports this odd formatting verbatim, and the note title is the hash "id" parameter above.

At this point I just need all of my original deleted data back and I can reorganize my profiles properly. How can I get the saved copies of the md files back in my Joplin so I can save properly? Or is there a tool out there that can take these modified MD files and give me back a non-Joplin-modified MD file so I can manually add the MD content back in?

Of course, I will be backing up properly and using profiles in the right way from here out-- but I just need to fix this before proceeding.

Any direction please help!! I've invested a ton of time into creating beautiful and helpful notes that are now POOF gone.

THANK YOU

2

I wrote a guide on how to use the revision service to restore deleted content. It is a very manual process but it does work.

3

This was IMMENSELY helpful! I was able to recover every note by the instructions you've provided. I can't thank you enough!!!

This manual behavior makes me feel there is a way to execute this with a script. Is the restoreNoteRevision command available in the command prompt by chance? Then a user could have a text file of item_id's on separate lines and have a script iterate over them and execute restoreNoteRevision on each id in the text file.

I'll add a few comments that I think would help the next unfortunate soul that has to learn this hard lesson.

Since item_id is not a unique field, scanning the list of item_id's will result in potentially multiple versions (revisions) of the same item_id. Trying to keep track of which item_id's you've already imported can be a pain the longer the list is. SQL can trim this list down so only unique item_id's are shown.

While using SQLiteStudio, you can click Tools -> Open SQL Editor and run the following SQL:

SELECT item_id,title_diff,body_diff,created_time FROM revisions r1 WHERE r1.created_time = (SELECT max(created_time) FROM revisions r2 WHERE r2.item_id=r1.item_id) ORDER BY created_time DESC

This will show only unique item_id's instead of multiple sets of them. Please note this SQL returns only the latest revision per item_id, though it really doesn't matter when importing. It does matter though when you are trying to visually scan the database for specific text in the title_diff or body_diff fields. Then you may want to not run the above SQL.

Anyway, thank you so much again!

4

Now that my data is back, this brings up an issue. I just opened a different profile and noticed that because I am connected to OneDrive sync, the OTHER profile just finished updating the recent changes too! I thought profiles were supposed to segregate concerns (like work vs personal) in different databases? Is there a way to allow Joplin to use OneDrive for sync of both profiles, but not sync with each other? Seems any change in one profile overwrites the other profile.

5

Glad to see the restoration worked, I intentionally gave instructions manually via the UI to not scare people off because it involved scripts or complicated sql statements, I'll have a look at adding that query somewhere (it is a wiki post so in theory anyone can edit it but I suspect as you are new it might not let you yet, if it does let you then have at it (if you want)).

It doesn't create a new "sub-profile" on the sync target, each profile is essentially an entire profile in its own right.
So for example I could create have my main personal profile on desktop and make a child work profile but each has its own sync target.
That means I can have another client that only syncs my work profile and another that only syncs my personal profile - they are all independent.

Unfortunately this is where it falls down on OneDrive and Dropbox in particular. It is easy to create your own subfolders on WebDav, NextCloud, Joplin Server etc. to hold your extra profiles (e.g. www.mynextcloud.com/nextcloud/files/Joplin/Personal) but on Dropbox/OneDrive it just puts it into a single folder that can't be changed (as it does it via the API).

I actually made a post about this here - Joplin 2.8 is available! - #10 by Daeraxa

6

Thanks for the clarity. I see-- perhaps that info is shown somewhere in the Joplin guides but it is not obvious when using the Sync Wizard. Might be worth adding a warning message if the user selects one of those methods to let them know they will overwrite other profiles if these methods are used. I'd imagine this is a somewhat common case (keeping work and personal separated) so adding an extra cautionary note may be helpful. Anyway-- thank you again!

2 Likes
7

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.