Why I'm leaving Joplin (for now)

You are missing the point. We are talking openness and independence. I would rather not depend on what Joplin has to offer in terms of portability, I would rather have my data be open by nature so I can back it up using my backup tool of choice and other software can easily import it for whatever purpose.

Currently it is just not very inviting for third parties to interface with notes created with Joplin and it is not trivial to just slap the note files in an editor and copy-paste them to a destination of choice. If that were the case, I would feel a lot more comfortable.

Yes, I would still use your editor but when it or you would start to annoy me for some reason I would want an easy way out without needing the app, which may be pulled or impose restrictions on what I can do with my data.

In my view SQLite is only needlessly complicating things. So you chose to have at least two different storage mechanisms for a fricking notes application. Performance should not be an issue. An RDBMS should not be needed at all.

2 Likes

Ok, there's so much wrong in what you write about that it's hard to know where to start. I guess it would help if you had a clue about what you're talking about.

12 Likes

If you want to backup an SQLite database it's trivial - I backup several every night, together with 2 MariaDB databases.

As for a database needlessly complicating things - occasionally, yes, but not in this case. It's a very efficient way of storing, manipulating and indexing small files. It's also one of the best ways of handling versioning, particularly if previous versions are to be kept.

In reality, unless you store everything as text (preferably in something like UTF-8), you're dependent on manipulation from a software layer at some point. Images, video and sound are prime examples of this. Without software these files are just chunks of useless binary data. You can store them in something like Base-64 so that they change to text, but they're not useful text. Joplin does a pretty good job of helping you manipulate notes whilst making it easy to export if that's what you want. Tags are metadata and there are few standards for handling it without heading towards Dublin Core which doesn't really fit with any of the MarkDown "standards". Personally I'm not wild about the way Joplin handles images but I do understand that a choice had to be made, and I also appreciate why this was the selected solution. I've taken a different approach with software I've designed and found that it's usually less flexible in specific areas.

If you don't like Joplin that's fine, don't use it. Try other note taking apps and maybe you'll find one that works better for you. For me Evernote was as disaster so I moved away - Joplin is the third application that I've tried after that and so far it's working for me.

6 Likes

For me Joplin is the best taking notes app. I have used Evernote for years, but sometime is overcomplicated. On top of that, it doesn't have encryption, so there is no doubt. The only problem of joplin is that it doesn't support linux-32bit machines, so I had to move to Turtle. That was a sad decision (and when I left a post about it, my message has been cancelled, that was odd)

2 Likes

I understand the original authors dilemma:

I too wish that tags would be embedded inside the markdown file. Perhaps an easier way to achieve it is to make a plugin for it that parses these kind of entries e.g. in the way Obsidian does it:

tags: #supertag #tag #anothertag

# Here is the first header
bla bla bla. I am happy that my plugin reads in the tags.

Joplin is an amazing piece of software, and it is free. Thanks authors for doing the work!

4 Likes

while it's not a parser (it will not process all your notes retroactively, yet), for future notes the "inline tags" plugin does exactly that. adopting the style you proposed above will add the corresponding Joplin tags to the note, while preserving them as plain text.

in addition, the "go to #tags..." plugin can be used for parsing the inline tags upon request. it requires selecting the text and hitting a keyboard shortcut, so it's not as quick as clicking a link, but it's still very nice.

1 Like

I see what mamaat is talking about. Joplin uses a proprietary database format and it's not easily browsable/manipulable that I'm aware of.

3 Likes

It's not proprietary - sqlite is open source

5 Likes

As a past Evernote "junkie" I can understand where this request from OP is coming from. In fact, I first switched to Joplin, then went back to Evernote once more (manually migrating the notes I had taken in Joplin during those weeks...). Only a few weeks later again did I decide that Evernote (10...) is indeed no longer usable and too laggy for about any real usage, coming back to Joplin for good.

Second, given the vast number of note-taking apps that can read ENEX, it is a de facto standard. However, I assume, it is more of a strategic question if you want to support ENEX export or not. Maybe it would be better to ensure Joplin and JEX become the de facto standard? Go tell Evernote if they want their users back (I am by far not the only one making the Evernote 10 exodus...), they will need to write a JEX importer?

I.e., leaving Joplin for Evernote because you cannot go back to Evernote is, well, let me say. peculiar, all the more considering how unusable Evernote has become and how deep it has sunk (after being considered by me and many others the most important app to have on your computer!)

Whatever, and in any case, for those in the want, there is a feature request for exactly this: [Feature request] Export to ENEX

2 Likes

That's a pretty rough way to answer, isn't it?

What @sprague, MrObvious and @mamaat mean is pretty easy to understand if you take off your programmer glasses for a moment.

They want a folder based solution.

The advantage would be that you have a simple folder based structure of files that you can edit with different programs.

Good examples are iA Writer or even Lightroom from Adobe. Both do not touch the LOCATION or the NAME of the files. This way, for example, in the case of iA Writer, you can edit the Markdown files with any text editor by simply clicking on them in the file system (without going through Joplin exports). Or you can put Markdown files from other programs into this directory structure. In the case of images and Lightroom, it's the same. For example, I can use XnView to edit the same images and folders to remove duplicates, for example, and I can drop images from any source with meaningful file names into a meaningful directory structure. I can rename files even several at once and I can find them via Spotlight. And when I search for content, Spotlight doesn't show me cryptic file names that I have to open all in a row until I find the right one.

Joplin does use Markdown, but it squanders the advantage of that openness by giving Joplin's files cryptic names and packing away images completely.

This is the same behavior as e.g. "Photos" or iTunes / Music does on macOS: all pictures (or music) get cryptic names and are stored in cryptic folders. They are still pictures, but without the detour via "Photos" they are not accessible. In the case of iTunes, it destroyed one's painstakingly created folder and file name structure if one wasn't careful. Without an export you can't get out again. And the result does not have the same structure as before.

The end result is that Joplin creates a silo. While not a completely closed one, it is an added complication for daily work when working with multiple devices, programs (not just text) and operating systems.

The examples of Lightroom (also Darkroom), XnView and iA Writer show how a hybrid solution with a database, folder and file name structure can work very well. With the gift advantage that folders and files are even syncronized and backed up via iCloud without having to do anything.

Just by the way: Tags, Dates and other Meta are solved in iA also (Multi)Markdown compatible (#tag1 #tag2) and very performant.

For me, unfortunately, also a reason not to use Joplin, Ulysses, Bear, Spaces, etc., even if everything else is really great. In principle, I'd be better off with Pages or Word (argh) if I didn't love Markdown so much.

13 Likes

I suppose Laurent chose what was easier to implement at the time. Not sure about iA Writer but Lightroom probably has dozens if not hundreds of people working on it full time so it's not fair to compare it with Joplin.

2 Likes

It is not easier. In fact, it's most likely the opposite. But Kudos to @laurent for everything else. Joplin is a great peace of software.

1 Like

I don't know about these programs, but images in general can store any kind of metadata so it means it's easy to index them and map them to some extra metadata in a database

Your solution with plain text files means you would have to store metadata in the file or at least an ID, which looks ugly and is fragile too, if the user changes it to something the app does not understand.

If you don't store an ID in the text file, you have no way to map it reliably to metadata in a database. For example if the user moves the file or rename it while the app wasn't running, you have no way to know which file this is, or that it was changed. You have no way to map any metadata, tags, revisions, note links and so on back to the file that was moved. You could use heuristics but that can't be 100% reliable.

Believe me I thought about all this. Joplin was even using text files at first! But this is not a sensible solution and I'm very glad I didn't stay with it.

4 Likes

See the first test data for Joplin: a bunch of folders with Markdown files inside. Been there done that :grin:

1 Like

MultiMarkdown does all (minus revisions) of that. Hybrid means you could store things like revisions in a database linked to an ID in the metadata. It’s not ugly since you can hide all that if the user doesn’t want to juggle with it (you do something like this in your md files!) nor is it fragile. And it’s performant. Maybe it wasn’t available at Joplins beginning.

http://fletcher.github.io/MultiMarkdown-5/metadata.html

Von unterwegs gesendet.

1 Like

How the metadata is stored in the file is not too important, but having an editable ID in the file and build the whole software around this is a receipt for disaster. You might say "well users shouldn't change it, their fault if they do" but often users don't know why something's not working, they just know it's broken and that's how the app is perceived. The metadata might be accidentally changed or deleted, either by the user, or automatically by some formatting tool or prettifier, by a bad copy and paste, etc.

It's also not user friendly to have a long string of random characters on top of each file. Funnily enough, you're the one suggesting I should take off my programmer glasses but at the same time you're suggesting turning the notes into code.

6 Likes

But the notes in Nextcloud are presented as files. What happens to them on Joplins end when you rename them in Nextcloud? You lose all tags?

Both parties are right though. It's a trade off. Metadata in a database is fast. Metadata in note files is portable. I think where both parties might be misunderstanding each other is what they mean by portability. The criticism is that the database - while the data can be seen and extracted - locks the notes into Joplin, as in they cannot be edited by multiple apps including Joplin at the same time.

Database Note file
fast slow
difficult to mess up easy to mess up
locked away freely editable by any app (!)

I liked Boostnote because all data is stored in the notes. But they missed some low hanging fruit. The note format is slightly off which defeats the purpose, but it proofs you can have some extra information that the app strips away.

And I always hoped they would go to a system where the metadata is in a database as a cache, but the cache is based on the notes, refreshed in the background. That way you have the best of both worlds. But the team went a different way, and moved to Boostnote Next (where you need an account) so I've been looking around for replacement markdown note taking apps and see what the main criticism about them is.

2 Likes

Indeed in BoostNote they store the notes as JSON files with an ID as name, so not more editable than if it was in a database. That's a note:

{"_id":"note:0Rxbbhzib","title":"dfqdsfqdsfds","content":"one line\ntwo line","tags":[],"folderPathname":"/aaaaaaaaa","data":{},"createdAt":"2021-05-07T12:12:49.050Z","updatedAt":"2021-05-07T12:17:28.062Z","trashed":false,"_rev":"YII8y4KkE"}

Probably they've started with plain text files but realised it's a bad idea and went for a database in the end. It seems that's what most note taking apps do, start with plain text files, then when they realise all the problems it causes and how difficult it is to add any new feature, they switch to a proper database.

In fact I feel it's a bit of a waste of time to build an app around plain text files. Because why bother? If they really are plain text files just use VSCode or some editor to manage them, then there's no need for a specialised app.

4 Likes

Have you tried Tagspaces

1 Like

This is only partially true. For Boostnote (not Boostnote Next) they store notes as cson with a randomly generated name. I just tested this and added test.cson. The note is immediately visible, editable and retains it's filename. This is a note:

content: '''
  # THIS IS A TEST
  
  This is _only a test._
'''
createdAt: "2021-05-07T01:06:15.582Z"
updatedAt: "2021-05-07T17:02:15.314Z"
title: "THIS IS A TEST"
type: "MARKDOWN_NOTE"
tags: []
isStarred: false
isTrashed: false
linesHighlighted: []

This is infinitely more editable than a database, because you can literally use any text editor without any technical database knowledge or software. You can sync the notes to some private cloud. You can find yourself in a place without the "right" editor, and just quickly add something important.

Granted, the note format is off which defeats the purpose, but it proves you can have some extra information that the app strips away, while it can be edited with other editors for users that wish to.

Frontmatter is a portable format for embedding metadata with markdown. For example the note could be like this:

---
comment: This content is parsed and hidden by APP_NAME. \
You can only see it with an external editor. \
If you mess this up, it will be replaced with defaults.
createdAt: "2021-05-07T01:06:15.582Z"
updatedAt: "2021-05-07T17:02:15.314Z"
title: "THIS IS A TEST"
type: "MARKDOWN_NOTE"
isStarred: false
isTrashed: false
---
# THIS IS A TEST
  
This is _only a test._

When Boostnote went in the database direction with Boostnote Next, there was a rift in the community. A non-trivial minority is simply opposed to a database dependency that diminishes note editor versatility.

There are simply two preferences. Not a good and a bad one, but different ones. Like those that prefer inverted y-axis in gaming and those that don't. What I personally think would be absolutely ideal is an app where the user can choose per notebook: Where is it stored, and how is it stored. How: With a database (faster), or as self-contained markdown files, where a database is only used as a cache, but the note is always the source of truth. Where: You can store one notebook with encryption somewhere, and another notebook without encryption to be shared with others in Nextcloud for example.

Because those people really really really love and prefer those dedicated editors like Joplin. They just dislike that the notes cannot be editor agnostic, and wish that while others may disagree, they could accept that it makes sense for some, and continue from that premise.

Then the question becomes: Do we want to build an optional storage engine for editor agnostic notes backed by a database cache for a speedy startup?

  1. Yes - we strife for options that make all users happy; or
  2. No - Joplin is not and will not be that. For editor agnostic storage options, stick with something like QOwnNotes.

"I like the concept of having notes accessible in plain text files to gain a maximum of freedom. Out of this need QOwnNotes was born."

- QOwnNotes developer

Of course QOwnNotes has the opposite problem. It's no Joplin.

For the sake of completeness, there's a third feature/tradeoff that some find important, and that's the ability to handle multiple notebooks with different storage parameters in a single app. Neither Joplin nor QOwnNotes have that as far as I can tell. It is for that reason that I stay with Boostnote for now, although for looks and functionality I find Joplin really attractive. That's why I registered to weigh in on this.

2 Likes