Mobile plugin error: sqlite3

shikuz · 2024-03-17T11:30:30.530Z

I'm developing a plugin, and would be happy to make it available on mobile once plugin support is released. I'm facing 2 issues:

The first is that I don't know CodeMirror6 well enough to build a scrollToLine plugin for it (TBH, I never knew CM5 well enough to do so either), but I assume that at some point someone will solve it and I'll adopt her code.

The second, is that I'm using sqlite3. It works perfectly well on the desktop client, but seems to be raising errors on mobile (using Prerelease 3). That is, if I understood the logs correctly. I'm attaching them here.

mobile-log.log (5.5 KB)

personalizedrefriger · 2024-03-17T14:13:42.515Z

shikuz:

The first is that I don't know CodeMirror6 well enough to build a scrollToLine plugin for it (TBH, I never knew CM5 well enough to do so either), but I assume that at some point someone will solve it and I'll adopt her code.

The following content script is mostly untested, but should work.

export default (_context: { contentScriptId: string, postMessage: any }) => {
    return {
        plugin: (codeMirrorWrapper: any) => {
            if (!codeMirrorWrapper.cm6) return;
            codeMirrorWrapper.registerCommand('scroll-to-line', (lineNumber: number) => {
                const editor = codeMirrorWrapper.editor; // as EditorView
                const lineInfo = editor.state.doc.line(lineNumber);
                editor.dispatch(editor.state.update({
                    selection: { anchor: lineInfo.from },
                    scrollIntoView: true,
                }));
            });
        },
    };
};

Notes:

I found a bug that will cause custom editor commands to fail on mobile. It should be fixed soon . Edit: Pull request.
See the beta markdown editor plugin tutorial for information about how to import CodeMirror 6 objects/types without bundling a copy of CodeMirror 6 with your plugin.
See also the CM6 plugin example and CodeMirrorContol.ts.

shikuz:

The second, is that I'm using sqlite3. It works perfectly well on the desktop client, but seems to be raising errors on mobile (using Prerelease 3). That is, if I understood the logs correctly. I'm attaching them here.

Allowing plugins to import sqlite3 on mobile would be difficult for a few reasons:

The mobile app uses React Native. The SQLite library we use has a different API from the sqlite3 library available on desktop.
Currently, plugins are run in mobile within iframes nested within a WebView. While this gives plugins access to DOM APIs and ES6, this means that any communication with the main React Native process needs to be async.

diagram: An IPC bridge connects the WebView to the main React Native process, IPC connects the WebView to an iframe for each plugin.525×600 23.3 KB

As such, it could make sense to expose SQLite functionality to plugins, but it would likely need to be a part of Joplin's plugin API, rather than a mock of Node's sqlite3.

Notes:

It may be possible to use Web SQLite here -- it's roughly 3 MB, so I don't think it would make sense to include in the mobile app by default.
- Ideally, it wouldn't be necessary to always include libraries like this when targeting both desktop and mobile. It might make sense to allow plugin authors to publish different versions of a plugin for different platforms.
- To store larger amounts of information (over a few megabytes), Web SQLite seems to need the Origin Private Filesystem, which, according to their documentation, is only available in web workers. (Presumably, Web SQLite needs the synchronous OPFS methods).
There's also absurd-sql, which uses indexedDB for persistent storage.
- It seems to be about 45 MB (based on its NPM page). Libraries this large are not well supported by the plugin loader on mobile.

Thank you for helping test mobile plugin support!

laurent · 2024-03-17T14:52:08.978Z

Or can we expose our SQLite driver? Via database-driver-node and database-driver-react-native? That way the interface would be the same on desktop and mobile

JackGruber · 2024-03-17T19:56:48.378Z

I'm not sure, but when you try to save for each note the position or something like that, you can use the user data API (userDataSet / userDataGet) and this data is synced with the note!

joplinapp.org

joplin.data | Joplin Plugin API Documentation

Documentation for Joplin Plugin API Documentation

shikuz · 2024-03-19T00:36:29.461Z

personalizedrefriger:

The following content script is mostly untested, but should work.

Thanks so much @personalizedrefriger, appreciate the help. I had all kinds of similar-ish attempts, but never got CM6 to accept my code. I tested your suggestion here and it worked great. In Joplin (desktop client), though, CM6 registers the function, calls it fine (prints to log with lineInfo), but does not actually scroll (no errors reported). What I did was to simply register your script with joplin.contentScripts.register instead of the old CM5 plugin.

shikuz · 2024-03-19T00:45:12.067Z

Thank you all for your help, this gives a better context of why the plugin failed.

If any of the mentioned database solutions is to be implemented at some point, are they expected to support a memory-based database? I'm asking because this is my specific use case (I assumed that the file system will just add an unnecessary overhead, but perhaps I should benchmark it properly). In case this is not an option, I could think of alternative ways to store and query the data.

personalizedrefriger · 2024-03-19T02:41:44.811Z

shikuz:

I tested your suggestion here and it worked great. In Joplin (desktop client), though, CM6 registers the function, calls it fine (prints to log with lineInfo), but does not actually scroll (no errors reported). What I did was to simply register your script with joplin.contentScripts.register instead of the old CM5 plugin.

Here's an example plugin that works for me in Joplin 2.14.19: GitHub - personalizedrefrigerator/bug-report at example/plugin-scroll-to-line

shikuz · 2024-03-19T14:40:33.212Z

Thanks again, grateful for your help. I'm sure that other plugins will find these examples helpful.

After playing with the various versions of the scripts I realized that what I thought was broken code in my last reply was actually very different behaviour exhibited by CM6 compared to CM5. CM5 scrolled the line to the top of the view (or at least that's what the snippet I had did), while CM6 (by default) only keeps the line in view (and sometimes not even that, if it's at the bottom). The result was that on most of my tests there was zero change in the scroll position.

I modified the command to scroll the line to the top of the editor, if anyone finds this useful:

const lineInfo = editor.state.doc.line(lineNumber);
editor.dispatch(editor.state.update({
    effects: EditorView.scrollIntoView(lineInfo.from, {y: 'start'})
}));

personalizedrefriger · 2024-03-20T14:53:57.517Z

shikuz:

are they expected to support a memory-based database?

We pass databaseName directly to the underlying library on both desktop and mobile. As such, on desktop, this should support creating a database with the name :memory:. On mobile, I'm not sure whether the library we use passes the database name directly to SQLite.