I am currently using Joplin desktop v1.0.216 on my Ubuntu OS and v1.0.329 on my Android phone.
I chose the Filesystem sync in combination with Syncthing app running on both Linux and Android.
In specific, the folder that I share between my phone and my PC with Syncthing is /mnt/MyDrive/Joplin/ on the PC, but I just recently discovered that the real attachment files are stored into ~/.config/joplin-desktop/resources and this is something that I dont like.
Would there be a way to make Joplin store everything inside /mnt/MyDrive/Joplin/ instead of using ~/.config/joplin-desktop/ folder?
If yes, do you think it is a good approch for synchronizing between my 2 devices?
Also, in this case, how may I proceed in order not make a mess with my existing notes inside /mnt/MyDrive/Joplin/?
I am not a dev just a user like yourself, but my advice would be Don't do it based on my understanding of what Joplin is doing.
If you sync with a folder or an online service you are making a copy of your notes and resources in that location that other clients can import. If you look in the target sync folder you will see all your notes (and other information Joplin needs) as .md files and all your attachments in a .resource sub-folder.
Now if you look in your ~/.config/joplin-desktop/ folder you will see a resources folder with files in it but no notes. That is because the notes are stored in the database.sqlite file. This database also has lots of other information in it that Joplin needs including the status of your notes and resources.
It's not that the "real" attachment files (resources) are stored in ~/.config/joplin-desktop/resources. That folder contains your local copy whilst the selected sync folder contains the sync copy. This makes more sense when you imagine the sync target NOT being on your computer but somewhere like Nextcloud or Dropbox. The Joplin clients connect to the sync folder and import any new notes and resources as well as any modifications or deletions. As they import the new / changed notes they update their own databases.
If you do this and sync using the config folder your notes are probably going to end up in a complete mess. You will be syncing the database.sqlite file which, as stated previously, should be unique to each device otherwise Joplin cannot maintain a record of what needs to be added, amended, deleted etc.
hi and thanks for the response!
Yes I was imagining it would have been a mess.
So whatâs the good approch to backup my data on desktop in case I need to format my system without including the ~/.config/joplin-desktop/resources folder in the backup but just including the /mnt/MyDrive/Joplin/ with all my notes and resource references?
@dpoulton, I have been wondering why for this situation (need to format system or new PC) the advice isnât to have an image of Joplinâs folder structure. My nightly backup procedures include just this to my NAS drive. Is there any reason why this isnât a good idea?
Having a backup of Joplin's profile folder is already good and in case of a problem it's possible to recover from it. The drawback of backing up the complete profile is that you also backup the sync state, so when you restore it, it might cause problems depending on the state of your sync target, which is why we recommend in general to export a JEX archive.
For example, let's say some files are accidentally deleted on Nextcloud, which you then sync on Joplin. As a result you'll also get files deleted locally. Now if you restore a profile backup, you might recover your notes - but when you sync, they'll get deleted again. You can fix this in various ways, but it can be complicated or tedious. However if you recover from a JEX archive, it's like importing new notes so they aren't affected by the existing sync state. You can sync, and they'll simply be uploaded to your sync target, which allows you to start clean directly.
Another issue is that if you backup the profile directory, you are backing up a live database, which might be in the middle of a transaction, so there's a small chance you will lose data that was just being written or about to be written. You can avoid this by closing Joplin before doing the backup.
So a backup of the profile is still ok, and you can definitely recover from it, but just no ideal.
So my simple understanding is - different backup strategies for different recovery scenarios?
Profile backup (when Joplin closed) for PC recovery if sync target is ok, Full JEX backup if sync target not ok?
I actually do both. My profile directory is in a one-way sync (not using Joplin) with a NAS folder but I also take a JEX copy semi-regularly.
With JEX files you can date them and save them as a âsnapshotâ of your notes at a particular time. That means if you realise you must have somehow deleted an important note, but it may have been some time back, you can select an archive from, say, a month or two back, expand it using 7-Zip and search the files for a keyword that would identify your note. The contents of a JEX file are easy to search as all the notes are just Markdown text files, not in a database. My JEX files are only about 20MB each on average so I also currenly have about 50 JEX files on my NAS dating back to April 2019.
And both those NAS folders are also backed up to iDrive every night and external storage every two weeks! Itâs not just Joplin that gets this treatment, all my more important stuff does.
@dpoulton Thanks for the full reply. Whatâs your thinking on taking â JEX copy semi-regularly? I would worry that I might not have backed up some notes and I wouldnât like the arbitrary decision making of when to do it. I like things automatic.
Ideally I would want to do a full JEX backup automatically in my nightly procedure. The downside to this is you end up with tons of files with multiple duplicates. The solution would be to do one full backup and then differential backups, preferably from within Joplin. If not one could clear out the duplicates if the note timestamps were set to their last update time instead of ânowâ as is done at the moment.
For me personally the loss of a couple of notes would likely be an inconvenience rather than a disaster. So I suppose what I am doing suits me but admittedly may not be so good for others. I have not had a data loss that I was not expecting as a possibility (in that I was trying something âcleverâ with my system which turned out not to be so âcleverâ after all). Therefore I had made a JEX export just before tinkering. If something went really bad just after I had created lots of notes, but before I saved a JEX export, then I would used the synced profile folder copy.
I suppose this fits in with @laurent 's comments above. A profile folder copy is a good idea but in certain circumstances may be problematic. So the JEX export can act as a âbackupâ to the profile copy as a âlast known goodâ.
I agree with you that having an automated JEX export would be a useful feature and it has been discussed here before. I also believe that when it was discussed the need for any export to be incremental was mentioned. This was because there will be those with huge amount of data in their Resources folder and repeatedly exporting the lot would be cumbersome.
I am sorry I have not had the time to search for this but I am sure that somewhere on this forum someone came up with a script that used the Joplin CLI version to automate a JEX export. I never looked into this myself because for me it did not cross the risk vs effort & laziness threshold!
I rolled my own in AutoIt (gets called before I shutdown my PC)
; File: JoplinNotes&ResourcesBackup.au3
$raw = False
If $CmdLine[0] = 1 And $CmdLine[1] = "--Raw" Then $raw = True
Run(@Scriptdir & "\JoplinPortable.exe") ; doesn't matter if already running
WinWaitActive("Joplin")
WaitThenSend ("!f")
WaitThenSend ("e")
WaitThenSend ("{Right}") ; JEX format
If $raw Then WaitThenSend ("{Down}") ; Raw format
WaitThenSend ("{Enter}")
$timeStamp = @YEAR &'-'& @MON &'-'& @MDAY &'_'& @HOUR &'.'& @MIN
WaitThenSend ("D:\Roger\Backups\Joplin\" & $timeStamp & " JoplinBackup{Enter}")
Func WaitThenSend($text)
Sleep(1000)
Send($text)
EndFunc
I would prefer differential rather than incremental as it would mean having just 2 files, the full backup and a file with everything in thatâs changed since then.
Fair enough, Iâve not read that whole thread (this time). I personally use raspberry pi for backup so this is not an issue for me.
As for incremental backups, JEX is a tar archive in disguise. It contains all your notes in .md files so you can unpack it and use any backup tool that support incremental backups.
It was only a quick reponse to the post above it where it was suggested some sort of backup could be achieved by creating a JEX, expanding it, and doing a backup on it. This was just to mention that the RAW format obviates the need to expand the file. Nothing insightful !
And not that it makes any difference but it is a tar file. 7-Zip handles tar files as well as zip files and many, many others...
Would you mind indulging an old timer (who developed in a Motorola Unix environment for over 20 years and has probably forgotten more than he learnt!) what saying the .jex file is a tar file means to Windows only users?
