Bonding Period Update - Week 1-2

[HahaBill]

Progress

• Attending the "Contributor Summit" (Tuesday, May 7 1500-1730 UTC)
• Introductory meeting with all the mentors
• Meeting with Daeraxa
• Finished 2 plugin tutorials on Joplin's documentation website
• Setting up the dev environment for summarise-note feature
• Accessed note content from the app and managed to summarise it with extractive summarisation

Plans

• Engaging more with the community on Discourse, Discord, GitHub, etc.
• Looking out for updates from Daeraxa about newly created repository for summarise-note feature in Joplin's GitHub repositories
• Continuing working on setting up Transformers.js in a plugin

Problems

• Encountered a problem with running Transformers.js (abstractive summarisation) in a plugin,
  • The problem was first with webpack, solve it with this: [Bug] Problem working with Next.js · Issue #210 · xenova/transformers.js · GitHub
  • Another issue is that in runtime I go this error displayed below. The problem seems to be disabling onnxruntime-node which is necessary to solve the error in webpack. I managed to run Transformers.js in a project without webpack and during the model construction I saw the library uses onnx. This link confirms my thoughts: TypeError: Cannot read properties of undefined (reading 'wasm') · Issue #589 · xenova/transformers.js · GitHub
  • Solution: Creating Node.js Child Process is one way to solve the issue, where we could use Python instead since it has a rich ML ecosystem. However, for this project using Transformers.js is better because is in Javascript which keeps things consistent and it contains already converted models in onnx to use. I am happy to hear more suggestions! Node.js Child Processes: Everything you need to know

Something went wrong during model construction (most likely a missing operation). Using `wasm` as a fallback. 
Kr @ plugin_com.example.JoplinSummarizeAILocal.js:2
2plugin_com.example.JoplinSummarizeAILocal.js:2 Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'create')
   at Kr (plugin_com.example.JoplinSummarizeAILocal.js:2:681968)
   at async Promise.all (/Users/billtonhoang/Documents/GitHub/joplin-hahabill/joplin-hahabill-resolved/packages/app-desktop/services/plugins/index 1)
   at async zo.from_pretrained (plugin_com.example.JoplinSummarizeAILocal.js:2:688061)
   at async Mc.from_pretrained (plugin_com.example.JoplinSummarizeAILocal.js:2:721345)
   at async Promise.all (/Users/billtonhoang/Documents/GitHub/joplin-hahabill/joplin-hahabill-resolved/packages/app-desktop/services/plugins/index 1)
   at async plugin_com.example.JoplinSummarizeAILocal.js:2:788312
   at async Zh (plugin_com.example.JoplinSummarizeAILocal.js:2:787993)
   at async a.predict (plugin_com.example.JoplinSummarizeAILocal.js:2:561693)

[allanmax]

Solutions:

Check onnx dependency:

Make sure onnx is installed in your project using npm install onnx or yarn add onnx.
Verify the version of onnx is compatible with Transformers.js (check documentation).
Investigate model loading:

Double-check the path you're providing to the model file.
Ensure the model format is compatible with Transformers.js (usually ONNX).

[HahaBill]

Hi allanmax! Thank you for your suggestions, I will give it a try!

[allanmax]

Let me know is it works or not @HahaBill

[HahaBill]

Hi, thank you again for providing your solution. I tried it and it doesn't work, mostly the same as you can see in the post. Transformers.js has already some dependencies and they are also installed when you install Transformers.js. Therefore, the onnx libraries are already installed.

It might be worth it to post to the HuggingFace about this.

[HahaBill]

HuggingFace discourse and GitHub issue in Transformers.js:

• Transformers.js went wrong during the model construction - 🤗Transformers - Hugging Face Forums

• Something went wrong during model construction (most likely a missing operation). Using `wasm` as a fallback. · Issue #769 · xenova/transformers.js · GitHub