Jimmy: A Joplin Import Tool

Marph · 2024-06-08T10:03:09.890Z

github.com

GitHub - marph91/jimmy: Convert your notes to Markdown

Convert your notes to Markdown

marph91.github.io

jimmy

A tool to import your notes from various formats to Markdown.

jimmy is a tool to import notes in various formats to Markdown.

Which formats are supported?

Every format that is supported by pandoc. Additionally txt, eml, fountain, asciidoc and Textbundle.
Apps:
- Anki
- Bear
- Cacher
- CherryTree
- Clipto
- ColorNote
- Day One
- Dynalist
- Facebook
- Fusebase / Nimbus Note
- Google Docs
- Google Keep
- jrnl
- Notion
- Obsidian
- QOwnNotes
- RedNotebook
- Simplenote
- Standard Notes
- Synology Note Station
- TiddlyWiki
- Tomboy-ng / Gnote
- vCard
- Zettelkasten (Zkn3)
- Zim Wiki
- Zoho Notebook

For specific import instructions, visit the documentation.

For formats like Evernote or Onenote, Joplins built-in import should be used.

Features

Many input formats
Markdown + Frontmatter output
- Compatible with any text editor
- Can be imported to Joplin/Obsidian/...
- Preserves resources, tags and note links when possible
Offline
Open Source
Cross-platform
Standalone (no python or node installation required)

What is converted (in most cases)?

Note content
Tags / Labels
Images / Resources / Attachments
External links and internal note links

Reference: Importing notes from other notebook applications

bush911 · 2024-09-20T01:32:08.028Z

This seems a good way to me, but I can't figure it out how to convert Keep to markdown files.
I am on the windows 10 x64, I tried run the app from githu on cmd, it says:


D:\>jimmy-cli-windows takeout-20240920T005824Z-001.zip --format google_keep --frontmatter joplin
[09/20/24 09:48:59] INFO     Importing notes from "takeout-20240920T005824Z-001.zip"
[09/20/24 09:49:00] INFO     Start parsing
Traceback (most recent call last):
  File "jimmy_cli.py", line 103, in <module>
  File "jimmy_cli.py", line 99, in main
  File "jimmy.py", line 94, in jimmy
  File "jimmy.py", line 71, in convert_all_inputs
  File "converter.py", line 53, in convert_multiple
  File "formats\google_keep.py", line 35, in convert
KeyError: 'textContent'
[PYI-12092:ERROR] Failed to execute script 'jimmy_cli' due to unhandled exception!

What am I supposed to do

Marph · 2024-09-20T06:13:04.818Z

That's an error in the script (likely the note is empty, but the script doesn't handle it correctly). I can provide a fixed version later today or tomorrow. Thanks for reporting!

Edit: I couldn't reproduce this with an empty note, but the script is now more robust. I hope this fixes the issue. Here is the new release: v0.0.13

bush911 · 2024-09-21T08:20:30.505Z

I have tried and successfully converted 600+ notes. Everything is fine. It works perfectly as expected.
I am so happy.
I previously tried similar app, which needs Google account tokens, I couldn't do that.

Thank YOU for this helpful tool

Fulltimehuman · 2024-10-04T10:05:09.566Z

I have tried Jimmy several times but keep running into the same "Unhandled exception" error. (5,5GB, ca. 1500 notes Synology Note Station NSX file, conversion on Win 10). My DEBUG-level export looks like this:

                    INFO     Finished parsing: 31 notebooks, 1032 notes, 1581 resources, 1065 tags
                    INFO     Start filtering
                    INFO     Finished filtering: 31 notebooks, 1032 notes, 1581 resources, 1065 tags
                    INFO     Start writing to file system
Traceback (most recent call last):
  File "jimmy_cli.py", line 108, in <module>
  File "jimmy_cli.py", line 104, in main
  File "jimmy.py", line 113, in jimmy
  File "importer.py", line 274, in import_notebook
  File "importer.py", line 271, in import_notebook
  File "importer.py", line 243, in import_note
  File "importer.py", line 160, in import_resources
  File "puremagic\main.py", line 271, in from_file
  File "puremagic\main.py", line 202, in _magic
ValueError: Input was empty
[PYI-8384:ERROR] Failed to execute script 'jimmy_cli' due to unhandled exception!

Notebooks   19%|████████████████ |  6/31 [00:03<00:12]
Notes       17%|█████████████▌|  174/1032 [00:03<00:14]
Resources   29%|███████████████████████  |  458/1581 [00:03<00:07]
Tags        17%|█████████████ |  181/1065 [00:03<00:13]

The conversion itself seems to complete fine, but the script runs into problems when writing the data. I was unable to pinpoint why. Anything I can do here?

Marph · 2024-10-04T11:28:39.761Z

Fulltimehuman:

Anything I can do here?

Thanks for the report! I can provide a fix later today. You can try it.

Background: The binary files (like images, PDFs, ...) in the export archive don't have an extension. That's why the script tries to guess the extension. In this case, the file seems to be empty and the corresponding error isn't catched properly.

Fulltimehuman · 2024-10-04T11:46:46.490Z

Many thanks, that would be very helpful! It's an excellent tool!

Marph · 2024-10-04T13:53:38.968Z

The error should be fixed now: v0.0.21.

Feel free to post again if something doesn't work. I never tried such a big export

Fulltimehuman · 2024-10-04T15:21:17.967Z

Many thanks! It's better -- but not quite there yet. This happens:

[10/04/24 17:17:04] DEBUG    Converting note "Root Causes vs. Facebook Causes"
                    DEBUG    Found too less or too many resources: 0 (note: "Root Causes vs. Facebook Causes", original
                             link: "![](webman/3rdparty/NoteStation/images/transparent.gif)")
                    DEBUG    Converting note "FIU-Verlag"
                    DEBUG    Converting note "beuys, Erweiterter Kunstbegriff"
[10/04/24 17:17:05] INFO     Finished parsing: 31 notebooks, 1032 notes, 1581 resources, 1065 tags
                    INFO     Start filtering
                    INFO     Finished filtering: 31 notebooks, 1032 notes, 1581 resources, 1065 tags
                    INFO     Start writing to file system
Traceback (most recent call last):
  File "jimmy_cli.py", line 125, in <module>
  File "jimmy_cli.py", line 121, in main
  File "jimmy.py", line 113, in jimmy
  File "importer.py", line 290, in import_notebook
  File "importer.py", line 287, in import_notebook
  File "importer.py", line 270, in import_note
  File "importer.py", line 238, in write_note
  File "frontmatter\__init__.py", line 238, in dump
  File "<frozen codecs>", line 918, in open
FileNotFoundError: [Errno 2] No such file or directory: '20241004T151103Z - Jimmy Import from synology_note_station\\Kleidung\\1926 c. Green chiffon web-pattern cocktail dress over it’s original silk slip. The dress is beaded in crystal silver glass seed beads, tiny white glass beads and crystal rhinestones. From 1st Dibs_.md'
[PYI-8820:ERROR] Failed to execute script 'jimmy_cli' due to unhandled exception!

Notebooks   42%|███████████████████████████████████ | 13/31 [00:04<00:06]
Notes       30%|███████████████████████▌ |  307/1032 [00:04<00:11]
Resources   38%|█████████████████████████████  |  598/1581 [00:04<00:07]
Tags        33%|██████████████████████████  |  351/1065 [00:04<00:09]
C:\notes_to_joplin\src>

In this case, it seems to be stuck on a very long file name, maybe including problematic characters?

And because you asked: I noticed that I see plenty of DEBUG messages (not WARNING) saying:

Found too less or too many resources: 0 (note: "Root Causes vs. Facebook Causes", original
                             link: "![](webman/3rdparty/NoteStation/images/transparent.gif)")

In each and every case, the same "transparent.gif" reappears. This affects at least some hundred notes.

Marph · 2024-10-04T16:32:02.417Z

Fulltimehuman:

In this case, it seems to be stuck on a very long file name, maybe including problematic characters?

The only potentially problematic character is ’, but I'm not sure if it's really causing the error. It could be the long path (270 characters), too. I think windows has a limit at 260 (?) characters. I will investigate.

Fulltimehuman:

And because you asked: I noticed that I see plenty of DEBUG messages (not WARNING) saying:
Found too less or too many resources: 0 (note: "Root Causes vs. Facebook Causes", original
                             link: "![](webman/3rdparty/NoteStation/images/transparent.gif)")
In each and every case, the same "transparent.gif" reappears. This affects at least some hundred notes.

It should be ok to ignore this. I guess it's an image added by Synology and not contained in the export. And probably it's transparent anyway

Marph · 2024-10-04T21:21:35.068Z

Marph:

The only potentially problematic character is ’, but I'm not sure if it's really causing the error. It could be the long path (270 characters), too. I think windows has a limit at 260 (?) characters. I will investigate.

It's most likely the path length. For now, I hardcoded a limit of 50 characters for the filename: v0.0.22

Fulltimehuman · 2024-10-04T22:03:57.562Z

I went ahead and gave it a shot by renaming some notes in Note Station and it worked. The issue is the long filenames. I had to rename about 20 notes or so and the script went through fine.

However, that wasn't the end of the story. Upon importing using "MD+Frontmatter (Folder)" I ran into tons of other issues.

First off, the Joplin importer seems to have issues with tags that contain umlauts, e.g. "Ökonomie". It gives me an error message reading "The MD file could not be imported. The tag 'Ökonomie' already exists." (which it doesn't). If I edit the MD file to read "Oekonomie", the file imports fine.
Then, the import process eventually broke off with the error message "Malformed URI" – which is unfortunate as it does not give me a pointer to where the source of the issue lies. It is clear that most of the import is missing.
And then, finally, in the parts successfully imported, there are no images. As far as I can say, all notes that should contain images either show only

"![](webman/3rdparty/NoteStation/images/transparent.gif)"

if I look at them in markdown view and show as empty in the rich-text editor view. Or they do contain some sort of link to a file, but the file does not seem to be there and rich-text view shows only a "broken image icon" with the name of the image file:

![IMAGE-0.PNG](file_45edad5ca262e1217c8aa53294d954dd.PNG)

Any idea if these issues are related to your script or Joplin-specific?

Marph · 2024-10-04T23:01:43.138Z

Thanks for the detailed feedback!

Fulltimehuman:

First off, the Joplin importer seems to have issues with tags that contain umlauts, e.g. "Ökonomie". It gives me an error message reading "The MD file could not be imported. The tag 'Ökonomie' already exists." (which it doesn't). If I edit the MD file to read "Oekonomie", the file imports fine.

I was able to reproduce your issue: Ökonomie doesn't work, while Oekonomie and ökonomie are working. This is an issue in Joplin, but a workaround in the script can be implemented easily. I. e. converting all tags to lower case.

Edit: I opened a github issue: Tags starting with upper case special characters are not imported properly · Issue #11179 · laurent22/joplin · GitHub

Fulltimehuman:

Then, the import process eventually broke off with the error message "Malformed URI" – which is unfortunate as it does not give me a pointer to where the source of the issue lies. It is clear that most of the import is missing.

I guess there is a problem on both sides: The script creates some odd URI and the Joplin import is not robust enough to handle it. You could try to catch the related error at the developer console ("Help > Toggle Development Tools"). Maybe it shows the name of the affected note.

Fulltimehuman:

And then, finally, in the parts successfully imported, there are no images.

That's strange. I couldn't observe this in the test files. Could you provide a example export here or send it to my email (linked at the end of the script output)? The more data it contains, the better for debugging. I guess that's the only way to find out if it's a bug in the script.

Fulltimehuman · 2024-10-05T09:08:29.701Z

So, I figured I could dig a bit deeper. I thought that I might try importing the individual notebooks instead of the full import folder. This is what I found:

Regarding the tags, this affected only a handful on my end, so I quickly edited the MDs manually. All went fine then, but it would surely be nice to have this checked by the script.
Two out of 31 notebooks produced a "Malformed URI" error (but seemed to import anyway). I'll have to look deeper and see what I can find out. (The dev console was no help.)
When importing the individual notebooks, all images were there. HOWEVER, several hundred notes still lack some images and are instead peppered with the "transparent.gif" error message. This looks like this. This would be an issue I would be eager to see resolved.
All the notes that I had in the "Mein Notizbuch" (My Notebook) folder in Note Station got dumped into a notebook folder named "unnamed_4918c825fde249e4babf8ecc0eeed147". Not sure why. Import was no problem.

Also, I include a longer part of the DEBUG-level log here.

Marph · 2024-10-05T10:47:26.810Z

Fulltimehuman:

Regarding the tags, this affected only a handful on my end, so I quickly edited the MDs manually. All went fine then, but it would surely be nice to have this checked by the script.

This is fixed, but not released yet.

Fulltimehuman:

Two out of 31 notebooks produced a "Malformed URI" error (but seemed to import anyway). I'll have to look deeper and see what I can find out. (The dev console was no help.)

Ok, then let's ignore this for now.

Fulltimehuman:

When importing the individual notebooks, all images were there. HOWEVER, several hundred notes still lack some images and are instead peppered with the "transparent.gif" error message. This looks like this. This would be an issue I would be eager to see resolved.

Unfortunately, I can't reproduce this and I would need an example export for debugging. I found another resolvable bug with images, but it's most likely not related.

Edit: Some image links and the tag issue are fixed in the latest release (v0.0.23).

Fulltimehuman:

All the notes that I had in the "Mein Notizbuch" (My Notebook) folder in Note Station got dumped into a notebook folder named "unnamed_4918c825fde249e4babf8ecc0eeed147". Not sure why. Import was no problem.

This is because the export has an empty title ("title":"") and the script assigns a default title. Not sure why, but since it's only this single folder, I wouldn't implement a custom workaround.

kartoo · 2024-10-21T15:12:09.275Z

Thanks for the tool. Does it support individual .tid (TiddlyWiki server) files?

Marph · 2024-10-21T15:44:45.517Z

kartoo:

Does it support individual .tid (TiddlyWiki server) files?

For now it supports only the complete backup in .json. From what I see, .tid files have the same markup, so it's a matter of extracting the (meta-)data. I will add it to the todo list.

kartoo · 2024-10-21T15:52:14.655Z

Awesome, thank you.

Marph · 2024-10-21T20:55:15.132Z

Individual .tid files are supported in v0.0.31. There are some limitations compared to the json export though, like note links and resources aren't preserved. Also Tiddlywikis WikiText can contain plenty of non-convertible stuff. But for mainly text notes with markup, the converter should be feasible.