How to recover corrupted database?

Hello,

I have troubles with the search function of Joplin on the windows portable version). It won’t find any notes anymore. The mains screen remains grey. (More information about this first “symptom” here)
I tried to use a backup profile (several days old) with the same Joplin version. And the search function worked well.
The search function is also working well on my Android Phone.

Therefore, I assume something may be corrupted in the database of the portable instance on windows.

I have more than 21000 objects synced with Dropbox, windows, android. I made quite a lot of changes / updates in my notes since this previous backup which worked well. I would like to avoid having to reinstall everything.
Is there a way to clean-up the database (on the Windows portable instance) so that the search function can work again? Do you have any other idea where my problem could come from?

FYI, here is the Joplin version I actually use:

Joplin 1.0.218 (prod, win32)
ID client : 40d3272809504f35b540692fa1f6b9ff
Version de Synchro : 1
Version du profil : 30
Keychain Supported: Non
Révision : 5be8c2cd (master)

Thanks in advance for your support.

When you open a support request, it asks you to post a log. Why not start here?

Thanks Laurent for your reply. Sorry for not having read the instructions of the support requests till the end :pensive:

So, here is the log I get when repeating the search action:
-1592063173605.log (36.8 KB)
In case it is helpful, you can find here also the log.txt located in Joplin's directory:
log.txt (663.8 KB)

Thanks

Looks like it’s due to an invalid note in your data set, but not sure how it’s possible. Could you try again what you’ve just done with this release? https://github.com/laurent22/joplin/releases/tag/v1.0.220

I’ve added some logging statement to detect invalid notes, which hopefully should tell us where it happens and how to fix it.

Thank you Laurent for your investigations.

Here is the equivalent log, with the newest v1.0.220 release:
-1592085820661.log (40.5 KB)

Could you also provide the log.txt file? As this is where the error will be logged

Here is the corresponding log.txt:
log.txt (720.9 KB)

Hmm, somehow the log entry didn’t get triggered. At this point, the only option would be to try with your Joplin profile directly, but I guess that’s not an option?

Hello Laurent,

I just tried again, just in case something went wrong the first time:
-1592152717761.log (68.7 KB)
log.txt (812.3 KB)

At this point, the only option would be to try with your Joplin profile directly

Do you mean that you would need to access to my complete profile directory [JoplinProfile] located next to the .exe file? This would be indeed complicated because its size is quite big: approx. 1Go.

I made a copy of my joplin directory for test purposes (easy since I am working with the portable version :wink: ). On this copy, I deleted most of my older notes to keep only the most recent ones. The problem is still visible. But unfortunately, the profile is still big (approx 1Go) because the corresponding ressources were not deleted. Is there any way to reduce the size of this profile to share it with you?

I think it should be fairly simple to create a script that would remove all personal data from the database: note bodies, folder & tag names, geolocation, resource names, sync settings.

Then the database can be safely shared for analysis.

If you can upload your profile without the "resources" folder that should be enough to replicate the bug. Feel free to send it for example at this address: joplin/AdresseSupport.png at master · laurent22/joplin · GitHub

Sure I will do that, Thank you.
How can I safely remove the sync config from my profile to make sure there will be no unexpected behavior on my DropBox?

Hello @laurent,

Since your previous message, when trying to recover "properly" my content from Joplin, and jumping from one instance to the other, I think I made it even worse :grimacing:
So I had anyway to start everything from scratch, including sync with DropBox etc. I exported my content as *.jex files from my corrupted profile(s), and imported again in a very new one. My database seems clean now , I can search it again.

However If you think that an analysis of my corrupted profile might help to detect possible bugs of Joplin, I will send you as private message a link where you can download it.

Thanks for your help ! I appreciate the support I got on this forum :slight_smile:

Del

Yes if you have a version of your profile that can be used to replicate the bug it would be very useful to fix the issue. Feel free to send the link directly to me by PM.

Thanks @Del, I’ve been able to replicate and fix the bug thanks to your profile. The fix is there and will be part of the next release: https://github.com/laurent22/joplin/commit/2377f7bc071bfe54352ced00fd8b150bf36ba515

Your search engine index is somehow corrupted but you won’t need to fix it, the app will just handle it better.

Wow this was a quick fix !! Thanks.