Data lost after re-install and sync

Hi,

first of all, thank you for your work on Joplin, and for making such a great software product for free. After a bit of experimentation Joplin strikes me as an excellent note taking tool (I could not manage to make it an effective planning / organisation tool a la org-mode, but it is not designed to be one).

My data loss case

I am certainly not the first candidate who managed to erase most (but maybe not all) of his data after a re-install of Joplin on both his notebook (Linux) and phone (Android 5, Note 3). While there seems to be a mechanism to prevent any data loss, it did not prevent my notes from being deleted. Against better judgement I did not make a backup of my Dropbox folder before initializing the sync after re-install. Et voila...

Strangely, while I am angry at experimenting without a backup, I feel a bit relieved after this data loss.

  • version: 1.7.11
  • sync target: Dropbox

Some questions:

  1. I have some remaining encrypted .md files. Is there any way to recover the data? I have the remains of the @joplin and joplin-desktop folders (including a 12 MB slite database). Is it possible to do that with the Joplin application? Please note that the latter does not show me any Notebooks or notes.
  2. Is there a guide or best practices how to re-sync after a re-install?

Cheers!

If you have a paid account you can revert your data in Dropbox to a previous state: How to use Dropbox Rewind | Dropbox Help

You can also try replacing your Joplin database with that .sqlite file, open Joplin, and see if you can see your notes there.
If you installed Joplin via AppImage the database should be in ~/.config/joplin-desktop/

Hello Roman,

thank you for your swift reply. Also, thanks for the hint with Dropbox. I do not have a paid account, unfortunately.

Folders deleted

Unfortunatly it seems that somehow I must have deleted most of the folders.

SELECT * FROM folders ORDER BY ROWID ASC LIMIT 10

yields
4a56d1c43aaa4d4fa074c4023b1fc702|OLD|1616214285058|1616214300233|1616214285058|1616214300233||0||0

with the OLD folder being created after I meddled with the last update process. Curiously, the notes table in the database file is empty.

Storage place of notes

Why is the database file 12 MB in size? Where and how are the notes stored?

Deleted Items table

Interestingly, the deleted_items table has around 2300 entries:

2322|4|fc96c2e8dfe64be69a66e91fceea0342|1618346899702|9

Any ideas how to extract those entries? I believe that the last number is the actual note ID. Where is the note content in once items are deleted?

Markdown files
Just had a look at the markdown files. It seems that a lot of them are not really encoded, but not really useful for human use - they look to me like diff files. Any idea how to un-garble them?

Example:

id: f01beffb4d70478e87483853fa4ca0c1
parent_id: 791003797815453a8f0e505c8aa62556
item_type: 1
item_id: e2f3c9c4e91b48369e8084d67bcbc645
item_updated_time: 1610605264954
title_diff: 
body_diff: "@@ -414,36 +414,8 @@\\n  %5Bx%5D\\n- Narayanan, H. et al. (2020)\\n  Bio\\n@@ -436,75 +436,207 @@\\n the \\n-d\\n+D\\n igital \\n-age: the r\\n+Age The R\\n ole of \\n-p\\n+P\\n rocess \\n-m\\n+M\\n odels\\n-. Biotechnol. J. 15, 1900172\\n+ (2019) %5Bpaper%5D - Narayanan%0A%0A#### Towards a Digital Bioprocess Replica Computational Approaches in Biopharmaceutical Development and Manufacturing (2020) %5Bpaper%5D\\n %0A%0A- \\n@@ -1604,16 +1604,461 @@\\n nable.%0A%0A\\n+#### Bioprocessing in the Digital Age The Role of Process Models (2019) %5Bpaper%5D%0A%0A- Pharma very complex, industry 3.0 standards not fullfilled, yet alone industry 4.0. %0A%09- high cost to produce a single data point%0A%09- complexitities of the processes and procedures involved%0A%09- %5B %5D are there companies (e.g. Ginkgo) that are doing better in that regard?%0A%09- tranform current operations into formalized information structures called *process models*%0A%0A\\n ## quest\\n"
metadata_diff: {"new":{},"deleted":[]}
encryption_cipher_text: 
encryption_applied: 0
updated_time: 2021-01-14T06:24:38.038Z
created_time: 2021-01-14T06:24:38.038Z
type_: 13

Did you have a fix on this issue? I have the same problem with it.

Did you have a fix on this issue? I have the same problem with it.

Unfortunately no. Looking through the database it seems to me that most content is gone - although I do not understand the size (~ 12 MB), and whether notes can be stored in the blob entry fields.

There has been at least another thread like this before: thread regarding sync errors and data loss in May 2020.

I have to say that these issues with syncing when making a re-install do make me waver whether I should invest into Joplin as a tool. Data integrity is key for a note taking tool.

Perhaps a good feature would be to ask the user whether he would want to finalize a sync (e.g. 300 files are going to be deleted. YES/NO?)

Where and how is note data stored in Joplin? Is there perhaps a spec?

Cheers!

Local
The notes, tages, ... are stored in the database. Only the resourses are stored in the .resource folder

Sync target
Everything is stored as *.md files

Thank you, Jack.

This works really well for us, thank you!

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.