Created a new version of Joplin that keeps local md files

How are you planning to handle resources linked to several notes?

Suppose there are note A and note B located in different notebooks
both of them linked to resource Z

If I edit resource Z (let's call updated resource to be Z-1) from note A, will resource Z be updated in note B?


I tested the apps a bit and created couple of issues in your repo

1 Like

@LeoK987, thanks for experimenting with this feature, it's indeed something that's been asked a few times.

I'm wonderig though - what's your short to medium term plan with this? Do you plan to create a pull request eventually? Because I looked a bit at the commits and there's a lot of formatting-related changes, such as adding brackets, newlines, spaces, etc. and that would make it unmergeable as it is.

So if you do plan to create a pull request, I would suggest paying attention to this - only modify what needs to be modified and don't change the coding style. Also if we add this it would be an option that can be toggled on or off - it would not be the default option so maybe something to keep in mind.

On the other hand if you plan to release this as a separate application, as @dpoulton mentioned it would indeed have to be under a different name and icon ("Joplin" is a trademark in EU and UK), and we won't be able to provide support for it here as I hope you can understand.

In any case this is indeed an interesting project and hopefully something we can integrate to the application!

1 Like

I would also add that some filesystems don't support symbolic links at all (e.g. FAT), so relying on them isn't really safe cross-platform. Also, on Windows, creating symbolic links by default requires administrative privileges.

1 Like

@laurent Thanks for the interests in my revision work.

I plan to sort out the functionalities for the md files to be sync'ed back to Joplin in the near term, likely within a month's time. Then I believe it will be in a better shape for you to integrate into Joplin. What do you think?

Regarding formatting, I think it's due to the formatter (I use ESLint for linting and formatting) in VSCode upon saving a file. I didn't even customize it. How is the setup on your end?

@graphit0 I didn't do anything specific for linked resources. Here is how it works:

When a note is saved to an md file, the program finds related resources from the DB and then copies the related files to the related notebook folder and renames them with their title. When the note is deleted, the related resource files are deleted (the resource files in Joplin's config folder stay at the moment).

So you can imagine, if two notes from different notebooks share the same resource, the resource are saved into the two notebooks separately; if two notes in the same notebook share the same resource, only one resource file is stored in with the notebook. When one note is deleted, then the related resource files are deleted. If it shares a resources with a note in a different notebook, then the other note has no problem; if it shares with a note in the same notebook, then the other note ends up missing that resource.

I didn't think of any use cases of sharing resources among notes. Tell me about its significance. Someone like to always include his photo in his notes?

Hi,

First, let me say that, as a heavy VSCode extension user, this could be a game changer!
Being able to simply and directly add Joplin's notebook folder(s) to my IDE workspaces would surely be something fantastic...

Did you remove those seemingly smarter triggers like block focus change etc?

These names indeed look more filesystem friendly/portable.
One thing though, in alphabetical sorting, this makes TODOs pop before DONE. Also, this me come in the way of user naming conventions/sorting.
My 0,2cts

TY

While that looks like a perfectly sane choice (as this is the one I've used for years before getting to know Joplin :wink: ), I'm wondering if/how this could get in the way of file deduplication, central assets/media management etc?

I tend to like the symlink idea of @graphit0 but there may be portability issues !?

I can see a bunch of them!
First that come to mind are logos, icons and other graphical components that may be used in many/all notes.

I'm in the cloudy devopsy crowd, can't tell you how many times the Kubernetes logo or AWS icons pop into my notes!

I see. Wonder how Joplin takes them in. Through WebClipper or user attachment, I guess Joplin would assign a different id to any incoming resource, even though they are the same image to our eyes, and store them as different files with the id's as their names.

Use cases

The main idea is multiple notes centered around a single changing entity.

I will list a few examples but of course the list cannot be complete because number of use cases is potentially limitless

Management projects

Project is always detailed with a lot of files that are continuously refined as more information is obtained

Consider construction project
Project details change based on market environment . Those project details (prices, contractors, headcount, inventory&supplies, CAD schematics, timelines, stages of completion, etc) are usually stored in spreadsheets, presentations, pdfs, CADs, pictures and many other proprietary formats.

If there's no way to link to a single file (for example representing headcount), one has to add latest version across all notes containing such file in past manually.

Suffice to say that managing projects in this way gets very tiring - because every change requires to update manually almost every note describing the project.

Programming project

Consider documentation about binary blob. Same as before, you don't want to copy latest version to every documentation page.

Design projects

Same as before, but instead of binary blob you have animation, video or interface file.

Corporate documentation

Organizational charts, descriptions of databases, minutes of meetings etc -- all of it is constantly changing

Zettlekasten notetaking

This popular notetaking method relies on links between knowledge points. Breaking link functionality would make it impossible to use it.

Templates

Linked resources are neccessary for reused templates

Linking to large files

Do we really want to keep a copy every time we link to 2GB video (lecture recording for example)?

Inconsistency in UX

From user standpoint it's confusing when resource could be updated only if notes are in the same folder.

Given the necessity to copy resources with links, linking to resource outside of its folder is not actually "linking" but "downloading a resource copy". Therefore, link is no longer functioning as an actual link but as copying interface.

That behavior breaks the expectations of users who is used to linking 2 things together. For example, you don't expect browser to spawn local copy for a website if you refer to it. Therefore, in my personal estimation 80%+ of computer users would report this behavior as a bug.

How others handle cross directory resource linking

I noticed that obsidian manages to link files outside of note folder by custom handling links via config file in vault/.obsidian/workspace.json. In other words, obsidian stores file in the note directory where it was originally added and saves the location in the config file for future reference.

When the resource is linked again, obsidian points to the place where it was initially stored.

In my opinion, this is only marginally better than Joplin's way because user cannot know the original location where file is located. Obsidian doesn't expose this path to the user -- you can only see it in (.json) config file. Therefore, to open the file without obsidian, user has to search the whole tree for via file name.

In this situation, you'd rather have one global .resources folder.

Perhaps I'm just missing something obvious - what's the advantage of using this compared to, say, Obsidian? Joplin Server?

1 Like

@graphit0 If you mean the user add a link in his notes to point to a resource either on his device or online, something like <img width="64" src="https://raw.githubusercontent.com/laurent22/joplin/dev/Assets/LinuxIcons/256x256.png" align="left" />, then the link stays untouched and no resources files are saved/copied.

Only those resources that were embedded in the notes and imported to Joplin are copied to the notebook folder.

I presume that those embedded resources were imported to Joplin as separate entities, i.e. even if there are of the same image, Joplin would still assign unique ids and store them as files with the id as the name.

Am I misunderstanding something?

As far as I understand the point of this project is to keep continuously updated .md export of one's notes at all times

Joplin Server doesn't keep the export. Obsidian does that and many users like this feature -- hence the attempt to bring it to Joplin.

Assuming that is the point, is there a simpler way of doing this? Can the jex backups be parsed into notes? If so, then adjusting frequency of backups could get close to continuously updated export of md notes.

I'm guessing there are other use cases for this project as well.

At the beginning of the thread I linked similar initiatives.

In previous post I omitted implied feature to import changes in .md files back to Joplin. I cannot say if continuous export/import would work at scale/reasonable performance (since noone has tried it before) but volunteers to try this approach are always welcome

Thanks for the update @LeoK987, of course feel free to create a PR whenever you think it's ready.

We use ESLint too, have a look at .eslintrc.js at the root for our rules. It should normally run automatically on commit but you may have changed something or it just doesn't work. Instead you can manually run it using yarn linter ./ from the root.

How are you going to deal with characters which are disallowed in some operating systems and filesystems? You can use almost whatever you want in note names inside Joplin, however Windows is known for a lot of so-called illegal characters that aren't supported by Windows Explorer (see https://learn.microsoft.com/windows/win32/fileio/naming-a-file) and there is also at least the FAT filesystem which is still used e.g. on Android that also has quite a lot of restrictions (see https://averstak.tripod.com/fatdox/names.htm).

1 Like

In Joplin there's the friendlySafeFilename function that does this, and also ensures that Windows reserved names such as NUL or COM1 are excluded.

1 Like

Good to know! Seeing that the list is hardcoded, I would kindly suggest adding COM0 and LPT0 to it. These two are undocumented by MS, however they are also unsupported by Windows Explorer (which can easily be verified by trying to create such a file/folder yourself).

1 Like