Why I'm leaving Joplin (for now)

Lounge
21

I don't know about these programs, but images in general can store any kind of metadata so it means it's easy to index them and map them to some extra metadata in a database

Your solution with plain text files means you would have to store metadata in the file or at least an ID, which looks ugly and is fragile too, if the user changes it to something the app does not understand.

If you don't store an ID in the text file, you have no way to map it reliably to metadata in a database. For example if the user moves the file or rename it while the app wasn't running, you have no way to know which file this is, or that it was changed. You have no way to map any metadata, tags, revisions, note links and so on back to the file that was moved. You could use heuristics but that can't be 100% reliable.

Believe me I thought about all this. Joplin was even using text files at first! But this is not a sensible solution and I'm very glad I didn't stay with it.

5 Likes
22

See the first test data for Joplin: a bunch of folders with Markdown files inside. Been there done that :grin:

1 Like
23

MultiMarkdown does all (minus revisions) of that. Hybrid means you could store things like revisions in a database linked to an ID in the metadata. It’s not ugly since you can hide all that if the user doesn’t want to juggle with it (you do something like this in your md files!) nor is it fragile. And it’s performant. Maybe it wasn’t available at Joplins beginning.

http://fletcher.github.io/MultiMarkdown-5/metadata.html

Von unterwegs gesendet.

1 Like
24

How the metadata is stored in the file is not too important, but having an editable ID in the file and build the whole software around this is a receipt for disaster. You might say "well users shouldn't change it, their fault if they do" but often users don't know why something's not working, they just know it's broken and that's how the app is perceived. The metadata might be accidentally changed or deleted, either by the user, or automatically by some formatting tool or prettifier, by a bad copy and paste, etc.

It's also not user friendly to have a long string of random characters on top of each file. Funnily enough, you're the one suggesting I should take off my programmer glasses but at the same time you're suggesting turning the notes into code.

7 Likes
25

But the notes in Nextcloud are presented as files. What happens to them on Joplins end when you rename them in Nextcloud? You lose all tags?

Both parties are right though. It's a trade off. Metadata in a database is fast. Metadata in note files is portable. I think where both parties might be misunderstanding each other is what they mean by portability. The criticism is that the database - while the data can be seen and extracted - locks the notes into Joplin, as in they cannot be edited by multiple apps including Joplin at the same time.

Database Note file
fast slow
difficult to mess up easy to mess up
locked away freely editable by any app (!)

I liked Boostnote because all data is stored in the notes. But they missed some low hanging fruit. The note format is slightly off which defeats the purpose, but it proofs you can have some extra information that the app strips away.

And I always hoped they would go to a system where the metadata is in a database as a cache, but the cache is based on the notes, refreshed in the background. That way you have the best of both worlds. But the team went a different way, and moved to Boostnote Next (where you need an account) so I've been looking around for replacement markdown note taking apps and see what the main criticism about them is.

2 Likes
26

Indeed in BoostNote they store the notes as JSON files with an ID as name, so not more editable than if it was in a database. That's a note:

{"_id":"note:0Rxbbhzib","title":"dfqdsfqdsfds","content":"one line\ntwo line","tags":[],"folderPathname":"/aaaaaaaaa","data":{},"createdAt":"2021-05-07T12:12:49.050Z","updatedAt":"2021-05-07T12:17:28.062Z","trashed":false,"_rev":"YII8y4KkE"}

Probably they've started with plain text files but realised it's a bad idea and went for a database in the end. It seems that's what most note taking apps do, start with plain text files, then when they realise all the problems it causes and how difficult it is to add any new feature, they switch to a proper database.

In fact I feel it's a bit of a waste of time to build an app around plain text files. Because why bother? If they really are plain text files just use VSCode or some editor to manage them, then there's no need for a specialised app.

4 Likes
27

Have you tried Tagspaces

1 Like
28

This is only partially true. For Boostnote (not Boostnote Next) they store notes as cson with a randomly generated name. I just tested this and added test.cson. The note is immediately visible, editable and retains it's filename. This is a note:

content: '''
  # THIS IS A TEST
  
  This is _only a test._
'''
createdAt: "2021-05-07T01:06:15.582Z"
updatedAt: "2021-05-07T17:02:15.314Z"
title: "THIS IS A TEST"
type: "MARKDOWN_NOTE"
tags: []
isStarred: false
isTrashed: false
linesHighlighted: []

This is infinitely more editable than a database, because you can literally use any text editor without any technical database knowledge or software. You can sync the notes to some private cloud. You can find yourself in a place without the "right" editor, and just quickly add something important.

Granted, the note format is off which defeats the purpose, but it proves you can have some extra information that the app strips away, while it can be edited with other editors for users that wish to.

Frontmatter is a portable format for embedding metadata with markdown. For example the note could be like this:

---
comment: This content is parsed and hidden by APP_NAME. \
You can only see it with an external editor. \
If you mess this up, it will be replaced with defaults.
createdAt: "2021-05-07T01:06:15.582Z"
updatedAt: "2021-05-07T17:02:15.314Z"
title: "THIS IS A TEST"
type: "MARKDOWN_NOTE"
isStarred: false
isTrashed: false
---
# THIS IS A TEST
  
This is _only a test._

When Boostnote went in the database direction with Boostnote Next, there was a rift in the community. A non-trivial minority is simply opposed to a database dependency that diminishes note editor versatility.

There are simply two preferences. Not a good and a bad one, but different ones. Like those that prefer inverted y-axis in gaming and those that don't. What I personally think would be absolutely ideal is an app where the user can choose per notebook: Where is it stored, and how is it stored. How: With a database (faster), or as self-contained markdown files, where a database is only used as a cache, but the note is always the source of truth. Where: You can store one notebook with encryption somewhere, and another notebook without encryption to be shared with others in Nextcloud for example.

Because those people really really really love and prefer those dedicated editors like Joplin. They just dislike that the notes cannot be editor agnostic, and wish that while others may disagree, they could accept that it makes sense for some, and continue from that premise.

Then the question becomes: Do we want to build an optional storage engine for editor agnostic notes backed by a database cache for a speedy startup?

  1. Yes - we strife for options that make all users happy; or
  2. No - Joplin is not and will not be that. For editor agnostic storage options, stick with something like QOwnNotes.

"I like the concept of having notes accessible in plain text files to gain a maximum of freedom. Out of this need QOwnNotes was born."

- QOwnNotes developer

Of course QOwnNotes has the opposite problem. It's no Joplin.

For the sake of completeness, there's a third feature/tradeoff that some find important, and that's the ability to handle multiple notebooks with different storage parameters in a single app. Neither Joplin nor QOwnNotes have that as far as I can tell. It is for that reason that I stay with Boostnote for now, although for looks and functionality I find Joplin really attractive. That's why I registered to weigh in on this.

2 Likes
29

I'd like to share what I've observed the Joplin teams's priorities to be. NOTE: I write this as a user, not as a developer on the team, so it is just a guess, and I might be totally wrong!

From most important to least important:

  • sync - it just works! (any sync related issue is quickly prioritized)
  • open source (code and roadmap are completely open, developers are active in the community)
  • platform diversity (works on mobile and most desktops including obscure linux versions)
  • maintainable (team will ignore an extreme edge cases to keep things simpler)
  • extensible (let others build plugins to increase features vs trying to build everything on their own, see: maintainable)
  • customizable (users can customize with CSS and preferences, within limits, not guaranteed to be stable, but way more custom than most desktop apps)
  • portable (while very easy to get note text out of joplin, it is more difficult to get metadata out)
  • interoperable (use joplin as an editor for a standard app format, doesnt seem to be on the roadmap, but does support one-way editing in other apps)

These priorities make sense to me, and so I have been happy with it. I think "sharing" is something else that is on the list, but I don't know where it falls in priority. If you disagree with the priorities, you'd probably be happier with a different solution.

I too really wanted a folder based notes system when I came to Joplin, and for a little while I managed my own notes in Git. I realized that syncing without thinking is so important to me that everything else was secondary. Plus, I wanted some creature comforts like search and interlinking. I still would like to have a way to export the metadata of my notes as yml, but that seems like an opportunity for a plugin.

Just my 2 cents, will be interesting to see if I have any of the priorities misrepresented!

17 Likes
30

You can ...

  1. use things like https://sqlitestudio.pl/ to edit ~/.config/joplin-desktop/database.sqlite for whatever purpose you pleasure,
  2. or just export all to RAW -> mangle with those plain .md files with whatever editor you like, then import RAW
  3. or sync with Dropbox, WebDAV, nextcloud, filesystem and mangle with those .md files on sync target.

Note that you shouldn't change filename or broke those metadata at end of .md files.

Using SQLite gives us tag management and full-text search functionality. If using the so-called "human-friendly filesystem layout" then to provide full-text search and/or tag management it would be need to run the equivalent of "full raw import" each time when starting app and handle those situations when metadata got modified and become inconsistent. (read: you will lost note, attachment and/or inter-note links if you are not careful)

I really don't see any format lock-in problems and appreciate how Joplin handle these data now.

2 Likes
31

As a recent émigré from Evernote I feel compelled to start off by thanking laurent & co. for all their work in creating such a great project. Often I'm pleasantly reminded that the computer issues I experience have already been solved by people in the open source community who are much smarter than me. I do have decades of experience in software development, however, and will stipulate that IMHO there are no flaws in Joplin's design decisions.

I disagree with this assessment. The issue has never been about which editor to use as Joplin allows you to choose whichever external editor you wish. (I happen to be a Neovim kind of guy but I can appreciate that others might prefer Typora or iA Writer. De gustibus non disputandum est, and all that.)

No it seems to me that the issues stem from people's differing opinions of what the nature of notes and an Evernote replacement should be. Evernote's original raison d'être was to allow us to quickly jot down notes, make searchable copies of images, documents, and web pages and later quickly find those things. Joplin does that admirably. (Well, the OCR options have a ways to go.) None of the feature/architecture requests discussed here would improve Joplin's core functionality.

Indeed, despite mentioning alternative ways of storing notes in other products no one has yet provided examples of such applications that are superior to Joplin as a note-taking app. I have noticed elsewhere that some who are comfortable staying within the Apple ecosystem have moved on from Evernote to DEVONthink, but I think it's important to consider DEVONthink is never described as a note-taking app but rather a document management solution. The two categories are focused on solving different problems.

1 Like
32

It's interesting to hear someone else qualify your personal preference as invalid. :thinking:

Let me put it this way. Using QOwnNotes, I can sync my to-do note with Nextcloud Notes. From Nextcloud, it is synced to my Android smartphone using CardDAV. On my Android smartphone, I then use the Notes widget to put the to-do note on my home screen. When I edit the note from my Android home screen while on the train, I can get home, open QOwnNotes, and see my to-do list updated. How do I do that with Joplin, while keeping other notes encrypted and away from Nextcloud?

I've mentioned two note-taking apps that have superior storage flexibility previously, and I proposed that while some couldn't care less, it is important for others. On computers, there simply isn't 100% agreement on what is and isn't an important feature. You can choose humor the group that prefers it, or reject supporting it. Both are fine. But refusing to acknowledge a desire for functionality as valid is not the most constructive approach here.

3 Likes
33

Looks like the only thing that Joplin lacks to achieve all this is widget.

34

I understand what you mean, and it's obvious that all things being equal it would be nice if the notes were simple text files that can be edited directly. In practice, as we've discussed it's neither easy nor a feature that's worth it, considering the massive drawbacks. And it's not small issues that can be fixed over time, it's fundamentally wrong from the start and can't be improved.

Also you mention a few apps that you think have a better storage mechanism, but it seems you're using Joplin regardless? Perhaps that "better" storage prevent them from implementing several of the features that are present in Joplin?

5 Likes
35

sorry but doesn't encryption make it so that you only worry about the device and not about the uploaded files?

36

Joplin users have a lot of different usage patterns. And no one app gives all features expected by any user. That’s why each month all note taking apps lose and win new users.

Also, most Joplin users have one point in common, they are concerned how to deal with their own data :

  • They prefer markdown because it’s easily readable in plain text without (or with) a lot of apps

  • They need to organize their notes

  • They prefer an open source app

  • They hope to not lose any piece of data (contents, links, resources and metadata) now and in 2, 5, 10 years...

  • They desire to be free to stay using one app or change

The problem is, there is no standard:

  1. Markdown, it’s just a piece of standard, its shortcomings are always amended by app developers

  2. There isn’t note taking app standard

I think Joplin team is strongly experienced now working for a lot of years in the note taking app. And for sure with glasses or not, they are known all most users request from this forum.

Two ways for now, just expecting JEX become the new standard thanks to the increasing number of users, but maybe that never become.

Or push a standard together Joplin team and any note taking app team. There is a Google summer code : Why not a note taking summer event ? @laurent

Probably the teams from open source apps would be the most open to that.

The two standards needed :

  • export file JEX or like JEX

  • export markdown lossless

The ability to have one database or Note file “base”, shared between different note app taking sound like a dream for some users, but it’s too early to expect it before to have strong standards.

7 Likes
37

I understand. Fair enough.

That's an opinion based on framing your use cases. Not a fact. You started your reply so promising. :wink:

The specified drawbacks of raw markdown file support are valid, and your rejection of this feature is noted. But for the sake of argument, I remind you that I mentioned frontmatter for optionally embedding metadata with markdown files (which is compatible with QOwnNotes) and a database as a cache, refreshing in the background, to have the speed as if you were using a database exclusively. Two improvements. Technically you could even hash the first so-and-so characters of paragraphs to get a confidence rating for matching renamed notes with pre-existing metadata in a database. Or simpler, git has moved file detection and it isn't slow at all.

Like you said, it's neither easy nor worth it to any existing user, because users that disagree are using something else. But it can always be improved. All it takes is a can-do attitude and some creative thinking.

No, I'm mostly but not exclusively a Boostnote user. Boostnote is switching to a private cloud. So I am searching for a sustainable alternative. And while I knew all along, it is only now that I am finding out how annoying it is that the notes are in a slightly customized format rather than plain editor agnostic md format.

Now I want to merge the multiple apps I use (personal I-dont-care-about-plaintext and shared plaintext-markdown), so I am looking in alternative markdown note app communities for criticism from actual users to learn. The criticism posted here resonated with me, and the replies were dismissive, so I just had to respond in a more agreeing tone.

Let me put it this way (2): If you want multiple storage locations and a slick app, you go with Boostnote. If you want a well maintained clean app, you go with Joplin. If you want to easily share and collaborate on groups of markdown notes without imposing a means of editing on your collaborators, you'll go with a selfless app like Obsidian, QOwnNotes, nota.md, Mindforger, or one of the dozens single file editors for the casual editor. They all work on a collection of raw MD files.

I guess @sprague is saying (or if not, then I am saying it): Hey Joplin. Join the club. Make an optional file mode. Your app will reach a larger audience.

(Or rather was saying, because the answer is quoted in the top of this reply.)

3 Likes
38

I sorry but I missed why using Nextcloud's encrypted folders won't work for you. Or how that's a Joplin issue for that matter. The native text file issue has already been answered.

39

I think this thread has very informative. I have a much clearer notion of the differences and consequences of choosing a between a note organizer or a document organizer. In the ideal world there would be need to choose; they would be the same - but it is obvious that it is not realistic given the design choices in Joplin

However, given that I use Joplin for note taking but still has a folder structure outside of Joplin, either organized by Finder/Windows pathfinder/devonthink/eaglefiler etc: What would be the best way of making them work together? Is it possible to reference external folders/files in a note? Or is something like Hook necessary?

2 Likes
40

Drag and drop the file with pressed Alt key, than a reference is created instead of a copy.

10 Likes