Importing notes from Zim - missing inserted images

Ari · 12 August 2021 00:40

Testing Joplin 2.2.7 on a mac.
Long-time Zim user here, 10+years - my work notebooks has around a thousand notes.

Many of my notes include locally pasted images.
Trying to import my zim workbook to joplin = massive failure.

In Zim, I exported my notes as "markdown", every note in a separate file, and I see Zim also exported the images, creating a folder with the same name as the note to which it belongs.

Notes that have no images, got imported OK into Zim. Notes that have images got imported as empty notes. In NO CASE was any image imported in any note.

Is there another way to import complex zim notebooks into joplin? some tool or helper script?

CalebJohn · 12 August 2021 01:50

I don't know anything about zim. But there is currently work underway to improve the markdown importer, if you're comfortable with doing stuff yourself then you can try running the import from that branch. Otherwise the importer should be improved in the next few weeks.

JackGruber · 12 August 2021 04:31

Could you please share one export example.

Ari · 12 August 2021 14:43

Here's a detailed example.
In Zim, I created a sample page and a subpage, both with images, with this structure:

The first page looks like this:

The subpage looks like this:

I export this "tree" using Zim > Export > Single page (with sub-pages), Markdown format

Zim creates a folder structure, I attach a zip file where with the folder that Zim creates, you can see both the ".md" and the images are all there, the images are in sub-folders that have the same name as the .md files. They seem to be properly "linked" from the .md files.

Importing with Joplin (importing the entire folder), I see Joplin picks up the structure:

It also picks up the first page, but misses all the images:

And the sub-page is completely empty:

Here's the Zim export folder:

Zim Export Test - for Joplin.zip (258.2 KB)

If I import in Joplin the .md files one by one (instead of the folder), I can see the content in both pages (i.e. no empty pages), but the images are missing in both cases.

So two problems I am facing to import my zim workbook: a) all inserted images are lost, and b) subpages are coming up with empty content (no content at all). Importing pages one by one at least gets the content of the page all right, but is not scalable (as a long-time zim user, I have around 1K notes in the workbook)

Thanks @joplin_team for looking into this

JackGruber · 13 August 2021 04:56

The imports works fine.
The problem is that the TIFF images are not displayed inline and have not been given a name by the export. I'm not sure if this is a bug or if the lib used for the preview just can't handle TIFFs.

tessus · 13 August 2021 08:43

The issue is two-fold:

TIFF files are not recognized as images in the Joplin code:

github.com

laurent22/joplin/blob/dev/packages/lib/models/Resource.ts#L43

    
      
          	static modelType() {
          		return BaseModel.TYPE_RESOURCE;
          	}
          
          
	static encryptionService() {
          		if (!this.encryptionService_) throw new Error('Resource.encryptionService_ is not set!!');
          		return this.encryptionService_;
          	}
          
          
	static isSupportedImageMimeType(type: string) {
          		const imageMimeTypes = ['image/jpg', 'image/jpeg', 'image/png', 'image/gif', 'image/svg+xml', 'image/webp'];
          		return imageMimeTypes.indexOf(type.toLowerCase()) >= 0;
          	}
          
          
	static fetchStatuses(resourceIds: string[]): Promise<any[]> {
          		if (!resourceIds.length) return Promise.resolve([]);
          		return this.db().selectAll(`SELECT resource_id, fetch_status FROM resource_local_states WHERE resource_id IN ("${resourceIds.join('","')}")`);
          	}
          
          
	public static sharedResourceIds(): Promise<string[]> {
          		return this.db().selectAllFields('SELECT id FROM resources WHERE is_shared = 1', [], 'id');

Adding the mime type won't help though. Here's the 2nd problem. TIFF images are not supported by Chromium. Thus they cannot be rendered.

There are TIFF image decoder libraries available, but I doubt that this will make it into Joplin core.

JackGruber · 13 August 2021 09:11

@tessus thanks, haven't found anything this morning

@Ari Then I think the best option is to convert all tiff files to jpeg an replace it in your exported md and then import it into Joplin.

Ari · 13 August 2021 14:50

Thanks @all for the follow up.

I wasn't fully aware that my images are tiffs. These are all snippets of the screen captured with MacOS official snippet tool. Doing a little research on my side, I just found that in a mac, when capturing a screen snippet, it's saved in the clipboard as a tiff file (linux and windows use png for this). As a long time mac user, there are probably a few thousand tiffs in my notes, I use this all the time while taking notes on presentations or random findings in documentation.

Question: any chance that joplin could do the TIFF > PNG conversion during import, using something like this:

or some online service?

That would be uber useful for mac people migrating from random note-taking programs to Joplin

(maybe as an optional import plugin, if joplin spports plugins)

tessus · 14 August 2021 12:03

Are you talking about the Cmd+Shift+4 shortcut? The default format is png.

https://themacbeginner.com/change-screen-capture-file-format-mac-os/

Ari · 14 August 2021 14:51

Yes, as tested in Zim. When capturing the snippet to clipboard and then just pasting it in Zim, the images get saved as tiffs, as in the example I provided. I checked notes captured prior to migrating to mac from a few years back (Linux, windows) > pngs.

The link you found is the file format used by the snipping tool by default when saving to a file. The normal flow for note taking doesnt include that step - I capture the snippet to clipboard and paste it directly in the note taking app

tessus · 14 August 2021 15:10

I think it is Zim which saves that image data as TIFF. e.g. when you paste the image into Preview it is as expected a png.

Ari · 14 August 2021 17:12

@tessus I can run a couple one-line shell scripts to do the .tiff to .PNG text replacement inside the .MD files, and to do in-place lossless tiff to PNG conversion. I will post them here once I have figured that out. Question for you would be what happened with the .Md files that got imported but show in joplin as empty pages. Happens with the sample i attached if you import it as a directory. The subpages show up in joplin navigation tree but show as empty pages. Actually joplin seems to see the subfolder in the source and creates sub folder for it but it fails to import the .Md file there (let alone the images).

If the Md content doesn't get imported if it's in a subfolder then thats a showstopper for me, as it requires me to import pages one by one and manually recreate the hierarchy

JackGruber · 14 August 2021 17:28

@Ari

On my test all MD files (two) where importet and I can also see this on your screenshot.
I don't see any empty notes and all images where imported to the coresponding note.

Test_Page_with_Images with 3 images
This_a_Zim_Sub-page with 1 image

The This_a_Zim_Sub-page on your export has only a image for the note This_a_Zim_Sub-page

Which images and notes were not imported?

system · 13 September 2021 17:29

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
New to Joplin - import question Support	12	2872	11 January 2022
Issue pasting images into Joplin Portable Support	2	959	19 September 2019
Import Markdown with inline images Support	2	1475	5 February 2021
Restoring Attachments Support	5	687	21 November 2022
How to use import folders to import PDF's Support	10	1047	5 January 2024

Importing notes from Zim - missing inserted images

Related topics