It seems my backups have somehow been deleted from Onedrive. I am not sure if the issue was with onedrive or Joplin. I haven't changed anything and have been using Joplin for about a year now with no issues. Suddenly in December my notebooks on my Android device app disappeared and now only have a "conflicts" folder with about 20 of my recently added items- meaning my entire collection over the past year is gone. I have a backup of my original save from evernote, when i exported the files to joplin originally 1 year ago, but I don't know where all the backups would have gone that have been going on since.
In my OneDrive, it seems I have 2 folders that were related to Joplin backing them up. One is called "Graph" and the other "Graph 1".
They both contain a .lock, .resource, .sync, locks, and temp folder, and then in the base directory of each contains many iterations of a .md file. The graph folder has a very large amount maybe over 100 .md files and a info.json file at the end. The Graph 1 folder has baout 40 .md files with an info.json file at the end.
I am looking if anyone can help or tell me that there is anything I can do with those .md files or the .json files to help recover my data.
Did you sign into OneDrive on the Joplin app just before this happened, or were you already signed in since the past year?
I have a suspicion that the recent OneDrive issues people have been having where they lose all their notes is because OneDrive created a new app authorisation for Joplin by itself (which you can confirm by answering my first question - noone else provided sufficient information to prove this). It happened to me recently, that I created a brand new Joplin profile on pc, synced to OneDrive for the first time, then connected my phone to the same account, but it had access to a brand new note collection (the default notebook only). When I checked on OneDrive it actually had created a Graph 1 folder when I connected the second device, instead of using the original Graph folder.
Do you have a computer? What you can do is verify whether the original Graph folder contains your note collection, as I suspect the Graph 1 folder is what your mobile is connected to now. Try doing the following:
-On your computer download the Joplin for desktop app
-On OneDrive, right click the 'Graph' folder and choose the download a zip option. Do the same for the 'Graph 1' folder as well just in case it has any data not in the other folder, but the one to use for verification is the 'Graph' folder
-Extract the zip file to a known location on your computer
-Set up file system sync, pointing to the extracted directory
-Let the sync complete
-Post here whether you have all your notes available on Joplin desktop. If you do have them all, then make a jex export of all your notes on the desktop app, and you'll be able to use that jex export to then create a clean profile and resolve your sync issues (I can provide some guidance with that if required, if this is indeed successful)
Also would you mind sharing the log file from your Joplin Android app (In configuration, tools, log, three dots menu and choose share) and also the deletion log (also under tools), zipping them up and posting on this thread please?
Sure thing, thanks so much for your response. Here are the logs as requested. I will see if I can follow the steps listed on my computer and get them back. mobile-log (deletions).log (393 Bytes) mobile-log.log (22.5 KB)
Sorry just missed that- I hadn't to my knowledge. I had been signed in for the past year using the same two devices (my same phone and PC) and never touched anything as far as adding any new accounts, signing in/out of one drive or joplin, or linking any other devices or changing sync locations.
I did not notice you mentioned "Suddenly in December". So it has been like this for some time then. The logs don't go back as far as December, however they are revealing. Because you say that you have not logged in recently, the log indicates that OneDrive re-authenticated by itself today. So that does strongly suggest your loss of notes was caused by Joplin requesting re-authentication with OneDrive and OneDrive creating a new data folder instead of reusing the existing one. As you say this happened back in the December, I guess re-authentication may have happened multiple times, but it is just random if it re-uses the existing data folder or not
I am about to cry (tears of joy) - I was able to successfully try out the local sync method you mentioned sourcing from the extracted zip to my pc to the original "Graph" Folder and it has revealed that much of my data is still there. I haven't yet tried adding the graph 1 folder contents, but this means the content that has gone missing is still stored in those two folders at least.
Just a tip for you. If you want to go merging the data from both zips of the Graph folders, I suggest to create a new profile to sync to the 2nd directory and do a jex backup of any specific notes you want to copy over (don't try switching the file system sync dir between directories on the same profile). Then switch back to the original profile and import those.
There is a switch profile option in the menu on Joplin desktop
I actually did already create a new profile to test the sync as i was unsure what would happen when i changed directories, since my desktop still contained my original export from evernote last year.
So I made a troulbeshooting profile and that is where i restored the graph folder from with success. could I just add the contents from graph 1 and not overwrite the files but just add the ones from each directory?
Also, do you have a venmo/paypal I can send buy you a pint or coffee for if that's allowed lol. You have no idea how much relief you have given me I owe you huge thanks.
And I guess lastly, once I get this data recovered, is the solution to simply use something other than onedrive? I'm not sure what happened for this to occur to begin with and not sure if I trust onedrive any longer to house my data. And I'm not too sure how I would go about storing my data locally for backup and syncing.
and yes sorry I should have clarified but this issue did start in December and i figured it was a onedrive issue directly so i reached out to them via email since December but they haven't been very helpful in getting any information other than telling me my data was deleted from their servers and unrecoverable. I had nearly given up thinking everything was lost before finally posting here.
You could try in a 4th profile doing a file system sync with the merged files of both directories (except for info.json, keep the one from the folder with the most notes) but I'm not sure that would necessarily work, so you should definitely look on the 'sync status screen' to check the total number of notes, resources etc before and after merging. But the safest way is using the jex import / export mechanism provided, which will allow you to select multiple notes and export just the selected ones to a jex.
At the moment yes, unless the Joplin devs / community implement a code change to prevent this scenario happening, then the solution for now is to simply use another sync target if you want to be sure this wont happen again. I personally moved to Dropbox since a number of these threads were raised about people losing all their notes on OneDrive sync.
The problem looks to be caused by Microsoft making changes to their API, as Joplin has not changed anything fundamental around sync recently, but it does seem to be recent that the app directory for Joplin created on OneDrive is called 'Graph' instead of 'Joplin' as it wasn't like that a year or so ago when I started using Joplin, and it's only in the last few months that there's been an influx of similar issues like this (at least 5 or 6 I recall) all with OneDrive.
While it's not really the fault of the Joplin devs that this happened, I do think there might be a flaw with the failsafe mechanism, as it's supposed to protect you from situations like this, and other's have mentioned they get an error about the failsafe when it happens, but then on the next sync it continues to delete the notes anyway without mention of them explicitly turning it off.
Do you have failsafe turned on for both your devices and did you ever turn it off at any point?
(edited post to include screenshots of both app and desktop sync messages)
So my failsafe is still enabled, so I guess I should have used better terminology considering I still had my files from my original evernote import. But just all of my joplin files were deleted from the past year from desktop, leaving only my original evernote imports. But on my app, everything was deleted except for the conflict folder. It seems that the "Conflict" notebook from the app actually contained the data that was located in that duplicate "Graph 1" folder now that I have imported that folder as well.
I had not modified the failsafe settings in anyway from either device.
Thanks for that- unfortunately the installation of the backup plugin was done I think only after I had already lost my data (hindsight is 20/20 ). I tried following those steps and restoring the data from that profile folder but there wasn't much there.
It sounds like desktop was still syncing with the original Graph folder then, while mobile started syncing to a new Graph 1 folder and pulled along some but not all of the notes with it. It might be that failsafe did not kick in or was bypassed because it requires 90% of notes being about to be deleted, and somehow the change of the app authentication managed split the notes across the 2 OneDrive Graph folders so the original profile did not reach the 90% threshold.