Storage requirements, sync limitations, and attachment download behavior

I understand that "attachment download behavior", if set to "Auto", will download attachments when each note is opened.

When are the attachments deleted? When I close the note? Some time later? Never?

I ask because it seems that depending on the answer, storage on Android may get out of control. If attachments stay for a long time, or perhaps forever, is there a way to manually delete them to clear up some space?

What if I change the option to "Always", then later set it to "Auto". Does any attachment deletion take place?

I am an Evernote user with 10,000+ notes. Since I can't selectively sync certain notebooks, migrating to Joplin has been quite an ordeal. It took me a while to discover that sync doesn't occur in the background, for example. The initial sync took several days to complete, and now decryption is looking like it will take another day. So, I don't want to make any decisions that will require me to start over at some point in the near future.

It seems to me that if I set the sync behavior to "Always", the 20GB my sync folder already takes up on the phone will then also cause another 20GB or so to get added to the Joplin database, which will cause my 20GB database to end up consuming 40GB on the phone!

Between this doubling of storage space requirements, lack of selective notebook sync, and the slow initial sync without background syncing, I'm concerned that this will all become unmanageable on my phone. How do other Joplin users with large notebooks handle these sorts of concerns on their phones?

Thanks.

1 Like

There's not much more to it than what you described. You should use the "auto" download mode, and currently the file will stay on your phone after having been downloaded (they are not auto-deleted).

If some of your notes are old and you don't currently need them, one option can also be to export them as JEX and then delete them from Joplin. You can always re-import them if needed.

I changed my workflow several years ago when it became evident that Evernote chose a strange strategy. I stopped creating big attachments into the Evernote database using links into OneDrive instead. Also, I relocated step by step all the big attachments I had in Evernote to OneDrive. At the beginning of this year, I had 10000 notes with size no more than 4MB (by Evernote app estimation).

I also realized that from these 10000 notes, I have only 4000-5000 I used/updated the last 5 years. So I separated and deleted after quickly checking all "old and unuseful" notes.

That I want to say that there is no way to use Joplin as file storage, Joplin has other strong features, but not this. Anyway, I was surprised you never experienced issues with Evernote. Because when I had many big files attached to notes, my Evernote app worked with a turtle speed.

1 Like

Joplin is a note taking application (and was designed as such) and not a content management system.

Thus the main purpose of the application is to take notes. If you want a content management system you will have to look at other SW.
You don't expect your car to be used as a boat, do you?

@tessus I agree with you on all parameters :slight_smile:

But I can understand another side too. For 12 years, Evernote was a reliable app for handling big amounts of documents. They even encouraged you to do so 6-7 years ago by advertising putting everything into Evernote. So it is reasonable that we have many users who get used to putting all your data into a note-taking app. I was such a user, too, until Evernote stopped working properly with my data.

Joplin appears everywhere as "the closest replacement for Evernote." And this is the reason people expected that the Joplin boat would be a boat as Evernote, not a car :slight_smile: I can understand these users. It is a very useful feature when you can store and find in one app not only your notes but your large documents too. But unfortunately, there is no possibility to use Joplin as document storage.

I am a little wordy today, sorry.

2 Likes

To be clear, most of my notes are web clips. Given how much Joplin has invested in web clipping, I expect my use case and concerns will not be unique. There are at least a few hundred notes with photos and PDFs, but I've already offloaded the vast majority.

But yes, I'm hoping to replace Evernote as much as possible. Many features of Joplin are not geared towards text-only notes. And Joplin imports Enex files.

So it's not just "a car". It's a car that advertises the ability to wade in water of significant depth.

Bottom line, I'm just trying to get some system to work, hopefully without abandoning a lot of media in the process. If there's something better than Joplin, I'd love to hear about it.

No, I certainly do understand where you are coming from.

It's just hard to make Joplin into something that it isn't. I also understand that people have a lot of notes, but 10,000 - 100,.000 I just don't get. Not really my place to judge, but who the heck reads 100,000 notes?

Even if people saved a lot of web pages, after a few years they'd most likely be obsolete anyway.

If you are looking for a knowledge base system, a wiki would probably be a better suited architecture. We all probably agree that putting 25 TB of movies into Joplin is not a good idea. Why? Because it is not a content management system. A content management system has features like FTS for BLOBs, but as mentioned before, Joplin was not designed for this.
People on this forum have 20 GB or more of attachments in Joplin. Well, it's very interesting but yet again, Joplin was not designed for this.

It was certainly not designed for storing 20+ years of collected stuff (from your great-great-grandma's baby pictures to black hat hacking videos) - maybe you can do that, but don't complain that it results in chaos and madness.

Seriously, you want to sync GBs of media onto X number of devices? Why? Get a NAS and put media on it.
People seem to think that Joplin is a storage solution. It is not. It is a note taking application.

PS.: I actually wrote this as a reply to @tnwn's comment, but some of it applies to @mzguy's comment as well.

1 Like

These are a lot of great points, but here's my challenge. Fundamentally I clip websites because I'd like to retain the information for the future. Over 10 years I've accumulated something like 8,000. That's 800 a year, or about 65 a month. On average. I don't think that's crazy use.
I do it to save time.

What would not save me time would be to go through and curate those things.... because technology and Moore's law should ensure that's a waste of my time. I could just delete all things over 5 years old if they are web clips, but that seems silly.

Similarly, I haven't deleted any Gmail in over a decade, and I don't organize into folders. Because if scalability and search, it takes me very little time to find anything I need. I've saved countless hours avoiding organization like I used to. And I don't need to try to design an organization system that predicts my needs a year or two from now.

No, I don't want all these clips on my phone though. Joplin is laggy when syncing now, and I expect that's due to the size of my notebooks. Your point matches mine: I'd love to be able to selectively sync. That's what I did with Evernote.

I'm desperate to be able to limit what gets synced and what gets downloaded (or retained as a download, really). That capability would render this entire discussion moot and make the system much more accommodating and flexible.

And if not, then why have a web clipper? Why import Evernote databases? The best systems deserve to be used, and not crippled if they are "overused".

Joplin seems great in many respects. I'm hoping to find away I can use it without giving up too much.

I came here from this Reddit post, it contains many other alternatives. As I remember, Nimbus Note, Notion, Obsidian and Amplenote were very promising. But not everything so bad in Joplin. It is a robust solution.

I do not have 100000 notes, I have a little more than 4000 now. For 12 years. Not very much for such a time. Why I need all of these? Because it is mostly my projects notes and documentation, articles and courses about something I learn. I use it in my new projects, when I need similar ideas, documents, practices. I replaced with it my paper workflow by photocopying everything in it. So it is my use case. I think it is exactly how Joplin was designed to work.

Many other features from old Evernote are useful in life too. I can imagine a writer or reporter who wants to clip 100000 interesting web pages to use them in the time they need it. Is not note-taking app their system? Yeap, there were many good features in Evernote. They even had feature "other notes similar to this one", or "additional search results to the Google query". So there we have another use case.

So many good cases for note-taking app. But you know many of them I am sure. I saw your positive comments in many feature requests.

I'm not finding anything at the moment more suitable than Joplin. Disregarding storage requirements for a moment, can you guide me regarding performance?

Currently, I have about 10,000 notes on mobile, and 55,000 resource files, the vast majority of which are image files associated with web clips. When I launch the Android app, there is a delay of approximately 15 seconds where the UI is either unresponsive or doesn't refresh properly. During this time the app is beginning to sync.

How can I best reduce this unresponsive time? Reduce the number of notes, or reduce the number of resources? Something else? I don't mind if sync takes a minute or two while I use the app, but having the app be unusable for a time completely kills daily usability.

Have you finished the first full sync on Android? I ask because in your case, it might take 3-4 nights. I finished my first Android sync in 2 nights. I had to switch the display timer to 10 hours to prevent the app from closing at night.

Now Android app responsive and syncs new notes for seconds. But the first sync was a terrible ordeal.

Oh yes, the first sync was quite bad. However, with every sync I can't use the UI effectively for about 15 seconds while the sync begins.

What do you use for sync? There's a big bottleneck with WebDAV at the moment as it doesn't handle delta sync (unlike OneDrive or Dropbox), so on each sync the app needs to download the list of files, then parse it, and compare it to what you have locally. That seems a bit crazy when you have thousands of files but it works well enough.

However the problem is when parsing the data - because the WebDAV format is so bloated, it takes a long time to parse and while it does this the app is frozen. And it gets worse and worse the more you have notes. There's currently no simple solutions to this, except maybe use Dropbox (but it has its drawbacks too).

I use local file system sync. Background sync is done by Syncthing. I assume this circumvents the issue you're describing?

There appears to be an initial portion of the sync where no updates appear in the hamburger menu (I know it's syncing because "Cancel" appears). This seems to be where the slowdown takes place. If the sync has any items that need adding / deleting, the UI updates and is responsive. Could this be fixed? Again, the issue isn't that the sync is slow, it's that the syncing activity blocks the app from being usable for some amount of time.

@tnwn
Do you have a lot of resource files in Joplin on Android? Roughly how many and what total size? Are you seeing an initial lag in the UI when syncing? Thanks.

I do not know because I do not have root; I can not see the Joplin folder on Android.

I had around 4000 notes, 30000 attachments, 4GB raw or 8GB encrypted data in all. Everything is smooth on Android.

So I can not help you with the localization of this issue.

You can go into Joplin settings on Android, check the sync status, and see how many notes and resources are synchronized.

Which sync method are you using?

To confirm: if you open the app or trigger a sync in some other way, then immediately start opening different notes, you don't get delays or freezes at all? I can get about one note open, then my UI freezes for 15 seconds and I can't open another note or notebook.

I use OneDrive and have problems with it on Windows, but it is another story we discuss in another thread.

I do not experience any delays on Android.