Using Joplin's turndown package, size issue

Development Plugins
1

Hey, I'm developing a plugin for which I'm dynamically rendering a table in HTML, but I wanted to add a feature to make the table permanent, so I need to convert it into Markdown. I tried doing it myself, but it seemed like doing double the work when I already have the table in HTML and I ran into some problems escaping special characters.
In my search for a better solution, I found Joplin's turndown package and the one specifically for tables. It's working great, however importing these packages takes the size of the .jpl file of the plugin from ~200KB to more than 3MB. This seems like a lot for a plugin and I'm wondering, whether I can import these packages a different way to reduce the size (especially since the packages are from Joplin already)?

2

do you know where this bloat comes from? If you untar the .jpl file you can view the exact files. I guess something that's not supposed to be there gets imported somehow.

3

I checked and there is an extra file called index.js.LICENSE.txt, which contains a lot of imports
index.js.LICENSE.txt (20.2 KB).
And the index.js file is about 3MB larger in the version with the imports.

The only thing I'm changing between the two versions is (un)commenting these lines:

import TurndownService = require("@joplin/turndown");
const tables = require('@joplin/turndown-plugin-gfm').tables;
.
.
.
const turndownService = new TurndownService();
turndownService.use(tables);

// turn HTML tables to MD
for (const overview of overviews) {
	const tableHTML = await renderOverview(overview[1]);
	const markdownTable = turndownService.turndown(tableHTML);
	body = body.replace(overview[0], markdownTable);
	await joplin.data.put(["notes", selectedNote.id], null, { body: body });
}
4

That file seems to be 20.2 KB though?

5

Yeah but the generated index.js file is about 3MB larger than the one in the version without the imports. And that "LICENSE"-file isn't generated at all, so I thought that might be an indication on what is being imported unneccessarily

6

I tried using the normal turndown and turndown-plugin-gfm instead of Joplin's versions and my file is only 467KB instead of 3MB now.
There is still an added index.js.LICENSE.txt file compared to building the plugin without any of the turndown packages, but it only contains the imports for the buffer and ieee754, instead of 20KB worth of imports.

7

Any chance you could share the bloated plugin so that we can check exactly what file is added and how? I know we have something that generates a license file but I'm surprised it ends up there

8

Sure, this is it:
frontmatter-overview-plugin.jpl (3.2 MB)
The source code is here.

9

I think the reason why it's large is due to jsdom which is 2.4 MB, but I guess it's required.

The normal Turndown is only 467KB because it looks like in recent versions they dropped JSDOM in favour of a smaller parser. Maybe we should update this too but this is likely to break certain things.

10

Okay, thanks for taking a look. Not sure yet how I will proceed with my plugin, but at least I know where the large file size is coming from.

11

I agree that 3MB is excessive but maybe not such a huge issue. It's gzipped when downloaded anyway and then 3MB on a hard drive is not too bad.

But we'll look and see if that can be reduced anyway, as that would benefit the main app too:

1 Like