Running Joplin 1.0.216 on (X)Ubuntu 18.04 and (K)Ubuntu 20.04. Also Joplin 1.0.329 on Android 9.
Because of a user error while trying to import my notes to the newer desktop, I destroyed my Dropbox sync directory.
All my notes were still present in Xubuntu, so I created a new Dropbox API and started syncing. The result was that Joplin started deleting the notes on my computer, even if fail-safe in the sync options is ‘on’.
That was …unexpected. But the answers to this question, WebDAV use failsafe sync almost make it look like that is the expected behaviour.
If I have to expect that connecting to an empty sync target will delete my notes, then how do I re-establish a working sync?
I also tried to re-establish sync from the Android app by connecting through a new API. When I hit sync, Joplin says it is completed, however no files are uploaded to Dropbox. Only the directories .lock, .resource and .sync show up and Dropbox tells me they are empty directories.
The directory Apps and its subdirectory Joplin also reports as empty. Well, at least on my Android device the notes are not deleted. I’ll take that as progress.
By restoring the directory /Joplin from a backup to /Dropbox/Apps/ I got the notes back and sync is working again.
However, if you lose data in your sync area, or for some reason need to change it and don’t have a backup, there seems to be no easy way to restore sync. Even if your local database contains all notes, changing the sync directory to an empty location will delete them. A kind of ‘restart sync’ would be a nice feature for such cases.
This just happened to me. I was trying to change sync targets to switch from a malfunctioning Nextcloud server to OneDrive. I know, I should have a backup of...something...? (It's not obvious what that something is, but still.) I exported all my notes (4+ years worth) to individual markup files, so that's something. First sync and Joplin proceeds to delete all my notes. $%@%&
This "empty-sync-source-so-delete-all-local-notes-without-warning" behavior borders on a bug in my opinion. Others will disagree, of course. In my case, it's enough to convince me to stop using Joplin completely. I'm currently looking for an alternative workflow. Such a shame.
In the next app you use, don't forget this time to create regular backups. When you are entirely in control of your data, client and server side, it also means you are responsible for backup and recovery, and that's true of any app you use. Even if you don't accidentally delete all your data yourself, you could get hacked and lose everything all the same.
Or if you don't want to deal with all this, Evernote, OneNote, etc. are there to make things easier and safer.
To be fair I have always found this design choice rather odd. I can hardly imagine a case where I would want to delete all my notes while keeping sync config in place.
It's not a design choice though - the app just does what it's being asked. If you delete two notes from the server, you want the two notes to be deleted from the client. If you delete all your notes, you want all them to be deleted from the client. Or maybe the user made a mistake, but the app can't know that.
There's a fail-safe in place to prevent all data to be lost though, and since it's been put in place there have been a lot less reports about this. I'm not sure why it didn't work in this particular case.
Here is my take on what happens, please correct me if I am wrong;
During sync, any Joplin app on any platform does the following:
download notes which do only exist on the sync target, but not locally
upload notes which do only exist locally, but not on the sync target
deletes local notes, which existed on the sync target BUT have been deleted there
replace older notes with newer notes (on sync target or locally) e.g. after edits
The fail-safe switch prevents
(all) local notes from being deleted in one and only one case: when the sync target is completely empty. It has no impact at all in cases where there are some notes in the local profile AND some notes in the cloud.
My point here is not that I shouldn't be responsible for backing up my data. I have lost enough data over the years to know I need to make backups.
My point is that software generally has developed to the point where I do not expect an application to irrevocably delete my data without some kind of confirmation. I had not expected a simple synchronization to delete data in this way. Certainly, a sync can and should delete data. I think Joplin defaults to deleting data when the user's intention is not clear. For me, that's a problem.
I guess this is the root of the issue. There is a warning for situations like this, the error prompts you to disable the fail safe if you truly want to wipe your notes.
If that warning did not appear (and your failsafe was enabled), then this is likely a bug. Can you provide some system details, particularly any configuration that might be unique? The fail safe has saved a number of people from deleting data in the past, so it seems this is an edge case bug which is especially hard to troubleshoot.
I was going to write there's no way for Joplin to decide, upon seeing an empty sync target, whether everything should be deleted or it is a new target. But then I realized Joplin keeps track of last sync attempts, so it can tell it has synced before. But I guess what it can't tell is whether it has synced with this particular sync target before.
I will try to reconstruct what I remember as the sequence of actions. I suspect my memory will not be precise enough to identify or reproduce the effect (sorry), but here goes!
Platforms: Linux Mint 20, and Android
I was having unrelated problems with the Nextcloud instance I had been using for several years as a sync target. That prompted me to change to OneDrive as an alternative while I sorted things out.
I remembered the warning about syncing with an empty target, so I exported my notes to .MD files. I also had the fail safe turned on.
The sync seemed to work fine on the Linux box. Notes stayed, and a folder structure appeared in OneDrive.
The sync even appeared to work on the Android tablet at first. After a while, perhaps 24 hours (?) the note I was looking at on the Android tablet disappeared. I eventually found it in the "Conflicts" area along with most of my other notes. I couldn't figure out how to get them all back where they were on the Linux box, so I deleted the Joplin Android app and its data, then reinstalled it. I think I did that a couple of times before it was stable.
The first sync of the Android app seemed to only get a few of the notes from OneDrive. I couldn't see any errors, but most of the notes were missing. I tinkered about, renaming folders to see if that was the problem. In hindsight, this was probably a bad idea because...
When Joplin on the Linux box synced the next time it started deleting notes. By the time I stopped it, it had deleted all of the most recent notes, about 150 in total.
I suspect there is quite a bit of "user error" in the problems I experienced. I was following my typical trial-and-error troubleshooting method, and depending too much on Joplin to preserve my data in case of error. I never did anything that seemed to me to be "deleting" notes, but somewhere along the line Joplin decided that I had done that.
What I would like to have happened in this case is for Joplin to have popped up a dialog on the Linux box as soon as it detected that it was going to delete any notes. It's easy enough to say something to the effect, "Synchronization will result in X deleted remote notes and Y deleted local notes, OK-Cancel" any time X or Y is not zero.
Fail-safe doesn’t work with OneDrive because we can’t know in advance how many items will be deleted. Normally it’s also less of an issue on OneDrive because there’s only one folder to sync with.
I guess it comes down to using the app in an undocumented way - you aren’t supposed to fiddle with the sync target (renaming folders, deleting them, etc). Everything should be done from the client. As soon as you touch the sync target, you need to have a very good understanding of how two-way sync works because the app won’t guess what you’re trying to do.
Everything I did was inside either the Android client or the Linux desktop client. The actual file structure of the sync target was and still is a mystery to me (as it should be!).
Is there some design reason that precludes a query-before-delete dialog? Even as an option in the way fail safe is implemented?
Sync is a background process and currently there's no mechanism to ask for user input while it happens. Also I assume you don't mean we should ask confirmation for every single item that gets updated or deleted? So it would be for specific cases, and it's often very hard to determine these cases. For example with OneDrive, we get the changes in batches - so we might get 50 deletion operations, then 50 update operations, then 50 more updates, etc. If it's 50 deletion, then 50 deletion, then only deletions after that I guess we could trigger the fail-safe, but we can't know that in advance.
But I don't know what happened in your case if you say you didn't change something on the sync target, as I don't remember it happened to anybody else with OneDrive. Maybe it's a bug, but honestly I doubt it because every single time I looked more closely at a sync issue like this, the conclusion was that it was some user error and not a bug.
I just wanted to add to this discussion. I am just getting started with Joplin, and my main interface is Android version 1.8.5. I had entered several notes and then setup Dropbox sync. Everything worked as expected. However, I still had the initial "getting started" notebook (can't remember the actual name) on my Android. I didn't really want those notes taking up Dropbox space. I deleted that notebook via Android and synced, but it didn't appear to remove anything from Dropbox. Maybe I should have waited longer, but I knew that I had the Fail-Safe turned on, so I figured I could just delete everything from the Joplin folder in Dropbox, re-sync and be all good.
However, when I synced (after deleting all contents from the Dropbox Joplin folder), my Android client deleted every note I had. This was NOT what I expected since it sounds EXACTLY like what Fail-safe is supposed to prevent. "Do not wipe out local data when sync target is empty."
I restored all the files using Dropbox restore, synced again, and the notes were restored to my Android. At this point, it seems like there is indeed a bug in the Fail-safe code. I don't know that this will have much of an impact on my regular use, but just wanted to let you know.
I have actually experienced this exact thing once when doing some testing, but never investigated. I have a suspicion this happens when you have only a few notes.