Bonding Period Update - Week 1-2

HahaBill · 2024-05-19T21:44:56.441Z

Progress

Attending the "Contributor Summit" (Tuesday, May 7 1500-1730 UTC)
Introductory meeting with all the mentors
Meeting with Daeraxa
Finished 2 plugin tutorials on Joplin's documentation website
Setting up the dev environment for summarise-note feature
Accessed note content from the app and managed to summarise it with extractive summarisation

Plans

Engaging more with the community on Discourse, Discord, GitHub, etc.
Looking out for updates from Daeraxa about newly created repository for summarise-note feature in Joplin's GitHub repositories
Continuing working on setting up Transformers.js in a plugin

Problems

Encountered a problem with running Transformers.js (abstractive summarisation) in a plugin,
- The problem was first with webpack, solve it with this: [Bug] Problem working with Next.js · Issue #210 · xenova/transformers.js · GitHub
- Another issue is that in runtime I go this error displayed below. The problem seems to be disabling onnxruntime-node which is necessary to solve the error in webpack. I managed to run Transformers.js in a project without webpack and during the model construction I saw the library uses onnx. This link confirms my thoughts: TypeError: Cannot read properties of undefined (reading 'wasm') · Issue #589 · xenova/transformers.js · GitHub
- Solution: Creating Node.js Child Process is one way to solve the issue, where we could use Python instead since it has a rich ML ecosystem. However, for this project using Transformers.js is better because is in Javascript which keeps things consistent and it contains already converted models in onnx to use. I am happy to hear more suggestions! Node.js Child Processes: Everything you need to know

Something went wrong during model construction (most likely a missing operation). Using `wasm` as a fallback. 
Kr @ plugin_com.example.JoplinSummarizeAILocal.js:2
2plugin_com.example.JoplinSummarizeAILocal.js:2 Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'create')
   at Kr (plugin_com.example.JoplinSummarizeAILocal.js:2:681968)
   at async Promise.all (/Users/billtonhoang/Documents/GitHub/joplin-hahabill/joplin-hahabill-resolved/packages/app-desktop/services/plugins/index 1)
   at async zo.from_pretrained (plugin_com.example.JoplinSummarizeAILocal.js:2:688061)
   at async Mc.from_pretrained (plugin_com.example.JoplinSummarizeAILocal.js:2:721345)
   at async Promise.all (/Users/billtonhoang/Documents/GitHub/joplin-hahabill/joplin-hahabill-resolved/packages/app-desktop/services/plugins/index 1)
   at async plugin_com.example.JoplinSummarizeAILocal.js:2:788312
   at async Zh (plugin_com.example.JoplinSummarizeAILocal.js:2:787993)
   at async a.predict (plugin_com.example.JoplinSummarizeAILocal.js:2:561693)

allanmax · 2024-05-20T07:13:26.029Z

Solutions:

Check onnx dependency:

Make sure onnx is installed in your project using npm install onnx or yarn add onnx.
Verify the version of onnx is compatible with Transformers.js (check documentation).
Investigate model loading:

Double-check the path you're providing to the model file.
Ensure the model format is compatible with Transformers.js (usually ONNX).

HahaBill · 2024-05-20T08:39:54.707Z

Hi allanmax! Thank you for your suggestions, I will give it a try!

allanmax · 2024-05-20T08:45:49.468Z

Let me know is it works or not @HahaBill

HahaBill · 2024-05-21T19:57:45.791Z

Hi, thank you again for providing your solution. I tried it and it doesn't work, mostly the same as you can see in the post. Transformers.js has already some dependencies and they are also installed when you install Transformers.js. Therefore, the onnx libraries are already installed.

It might be worth it to post to the HuggingFace about this.

HahaBill · 2024-05-21T20:44:41.122Z

HuggingFace discourse and GitHub issue in Transformers.js:

Transformers.js went wrong during the model construction - 🤗Transformers - Hugging Face Forums
Something went wrong during model construction (most likely a missing operation). Using `wasm` as a fallback. · Issue #769 · xenova/transformers.js · GitHub