Imported notes to new computer, and Joplin started to sync them as duplicates

[filwi]

Operating system

Windows

Joplin version

3.5.13

Desktop version info

Joplin v3.5.13 (prod, win32)

Sync target

Dropbox

What issue do you have?

I got a new computer, and, figuring to be smart, I thought to export my notes as a .jex from my old computer, import them on my new one, and just sync whatever had changed in the last two days.

Worked like a charm - except that instead of downloading the items it needed to sync, Joplin treated all the notes as new ones, and started uploading them. So now I've got about a thousand duplicates in my Dropbox folder, including likely some that I started working on.

Is there any way to see which ones are duplicates? I don't have any way to roll-back Dropbox, and my old computer synced the uploaded files, making that database polluted as well...

Any advice?

[mrjo118]

My advice would be the following:

• Make another jex backup (with all the duplicates included), in case something goes wrong
• In the all notes notebook, sort by updated time
• Export all notes updated within the last 2 days
• Then delete ALL notebooks and empty the trash
• Import you previous jex backup (without the duplicates)
• Import the exported notes from the last 2 days and manually deduplicate them
• Sync
• Then sync all your other devices

[TheFronos]

I think it should be possible to download the files locally then use software like czkawka (FOSS) to remove the duplicate files. Czkawka uses hashes to determine duplicates, so only files that are exactly the same will be marked as duplicates. It groups the duplicate files allowing you to do stuff like choosing one file from each duplicate group in bulk (you can choose based on file age, for example), and then deleting the other duplicate files.

Edit: Maybe you should ask the program to not look for duplicates inside the .resource folder. I did on mine and for some reason some pictures are duplicate. Removing them might cause some notes could lose their link to the images even though the images themselves wouldn’t be lost, but I’m not sure

Here is my suggestion.

1. Download all of the files from Dropbox
2. Make a copy of them just in case and keep it safe
3. Check for duplicate files locally with Czkawka.
4. Remove the files from the cloud (again, please make sure to have copies)
5. Upload the now duplicate free pile of files.

Edit 2: Forgot to say that in czkawka, in order to find any duplicate files no matter how small, you have go to the Items Configuration tab and set Size (bytes) > Min to 0.

[filwi]

Thanks for the suggestions! I did a bit of both, filtering and downloading the Dropbox stuff, then re-upping it. So far, it looks like it worked!