Using native node modules in plugins

shikuz · 2023-07-26T19:00:46.385Z

I recently mentioned this problem I'm having in another thread. Here are more details. Will appreciate anyone's advice. I'm not a JS dev - I'm familiar with Joplin's plugin API, but not with Joplin's core or more advanced stuff.

transformers.js is a package for running AI language models locally that is endorsed by the Hugging Face model repository. It uses ONNX Runtime to run models in the browser, which is a compiled package. The distribution of transformers includes 6 .node binaries for [3 OSs] X [2 architectures].

In order to use it, I configured Webpack to bundle the native modules using node-loader. So I have the following code in my webpack.config.js:

module: {
	rules: [...
		{
			test: /\.node$/,
			loader: 'node-loader',
			options: {
				name: '[path][name].[ext]' // without the options this also fails
			}
		}
	],
},

I can confirm that the .node files are bundled in the /dist folder, and preserve their relative location in the /dist/node_modules sub-folder.

The problem is that during runtime I get the following error in the debug console:

Uncaught Error: node-loader:
Error: ENOENT, services/plugins/node_modules/@xenova/transformers/node_modules/onnxruntime-node/bin/napi-v3/darwin/arm64/onnxruntime_binding.node not found in /Applications/Joplin.app/Contents/Resources/app.asar

laurent · 2023-07-26T20:07:51.256Z

Thanks for the info, do you have a repo we could use to replicate this issue?

shikuz · 2023-07-26T20:41:37.989Z

Thanks @laurent. I have a branch here that I used to reproduce the error. If you prefer a minimal environment, I can create a new plugin repo just for this.

laurent · 2023-08-07T17:18:47.239Z

I spent some time trying to get this working but unfortunately didn't get anywhere. I don't think it's an issue with not being able to use native modules, because this package works even in Chrome Extensions (they have some examples of it). From what I can read in their repo and issues, it's about bundling the package properly with WebPack, maybe by ignoring or disabling certain files.

I tried the solutions they propose using IgnorePlugin and resolve.alias but nothing seems to work. Maybe you could ask them and see if they have any ideas why it doesn't work.

shikuz · 2023-08-07T17:59:55.393Z

Thank you very much Laurent for looking into it. I'll try to ask in their community.

I think that for browsers the package switches (automatically) to ONNX WebAssembly runtime. Maybe there's a way to make it unaware that it's running in node. In any case, I'll try to experiment with it occasionally, and perhaps similar use cases will pop up on GitHub at some point.

Since I ran into similar problems with other native modules (lower priority packages), I thought there might be a Joplin trick to it.

laurent · 2023-08-07T18:06:26.406Z

If you have another example of native module, please let me know.

For this particular module I only tried to get it working as non-native because it seems possible so I didn't really test any native module loading capabilities

shikuz · 2023-08-07T18:13:47.569Z

Sure, I'll try to reproduce this with other packages and update.

shikuz · 2023-08-11T12:39:42.991Z

I added a new branch dev-llama with a runtime import error of a native module based on the package llama-node. Maybe the underlying problem is the same (the .node files are included in the dist folder, but cannot be found at runtime).

I saw that other people ran into similar issues with webpack, but I did not see a solution.

rxliuli · 2023-10-13T13:59:38.636Z

Dealing with .node modules using build tools can be complicated. I've encountered similar problems before using esbuild. The author's reply was that you can copy the .node file and change the import path after the bundle. Again, this is complicated, especially Usually cjs will use dynamic paths.
refer to: Incorrect .node module require · Issue #3294 · evanw/esbuild · GitHub

If you want to use a build tool, I recommend using wasm anyway, it works in the browser and nodejs, and the path to introduce wasm is usually explicitly declared manually.

HahaBill · 2024-05-26T19:49:58.315Z

Hi everyone,

I am currently trying to set up Transformers.js in a plugin too. However, I've encountered several problems such as:

I have same problem with node-loader not working and seeing .node files are in dist folder.
Using resolve.alias and externals solve the loader issue but the problem now seems to be that onnxruntime-web is not part of the build since it's unable to find the InferenceSession object to call create.

Does anyone have ideas how to tackled this issue?

I am currently reading about webpack since I have only basic knowledge about that so I might be missing something. I also saw that you can import the library via CDN:

<script type="module">
    import { pipeline } from 'https://cdn.jsdelivr.net/npm/@xenova/transformers@2.17.1';
</script>

Would we be able to import modules from CDNs into a plugin if we could not make Transformers.js work? I think it still keeps privacy for users but unfortunately they have to be connected to the internet.