Mobile: Plugin support

Medullitus · 2024-02-28T21:08:48.749Z

A fantastic project!

Thank you very much @personalizedrefriger

cwhittl · 2024-03-06T16:50:24.932Z

If you need an IOS beta, let me know. I have a Dev account so I can deploy just might need some links on how to for this project.

personalizedrefriger · 2024-03-09T01:18:06.830Z

The iOS AppStore guidelines has restrictions on 3rd-party code that we'll need to make sure we comply with. At the time of this writing, the guidelines were:

Guidelines

See App Review Guidelines - Apple Developer

4.7 Mini apps, mini games, streaming games, chatbots, and plug-ins

Apps may offer certain software that is not embedded in the binary, specifically mini apps, mini games, streaming games, chatbots, and plug-ins. You are responsible for all such software offered in your app, including ensuring that such software complies with these Guidelines and all applicable laws. Software that does not comply with one or more guidelines will lead to the rejection of your app. You must also ensure that the software adheres to the additional rules that follow in 4.7.1 and 4.7.5. These additional rules are important to preserve the experience that App Store customers expect, and to help ensure user safety.

4.7.1 Software offered in apps under this rule must:

follow all privacy guidelines, including but not limited to the rules set forth in Guideline 5.1 concerning collection, use, and sharing of data, and sensitive data (such as health and personal data from kids);

include a method for filtering objectionable material, a mechanism to report content and timely responses to concerns, and the ability to block abusive users; and

use in-app purchase in order to offer digital goods or services to end users.

4.7.2 Your app may not extend or expose native platform APIs to the software without prior permission from Apple.
4.7.3 Your app may not share data or privacy permissions to any individual software offered in your app without explicit user consent in each instance.
4.7.4 You must provide an index of software and metadata available in your app. It must include universal links that lead to all of the software offered in your app.
4.7.5 Your app must share the age rating of the highest age-rated content available in your app.

As such, before creating a version that can be distributed for iOS, I suspect that we'll need to allow only a specific set of plugins and versions (though perhaps the restrictions are different for TestFlight?). We may also need to get permission from Apple to allow plugins to access the clipboard (which currently uses a native API) and Joplin's database (this includes access to notes).

personalizedrefriger · 2024-03-21T23:56:47.137Z

Mobile plugin support has been merged and should be present in the Joplin v3.0.1 pre-release for Android:

GitHub

Release android-v3.0.1 · laurent22/joplin-android

Android releases for Joplin. Contribute to laurent22/joplin-android development by creating an account on GitHub.

See the following GitHub issue for more information about plans for plugin support on iOS:

github.com/laurent22/joplin

iOS: Enable plugin support

opened 04:29PM - 18 Mar 24 UTC

personalizedrefrigerator

enhancement medium

### Operating system iOS ### Joplin version 3 ### Desktop version in…fo _No response_ ### Current behaviour Currently, plugin support is disabled in the iOS app because of [this section of the iOS AppStore guidelines](https://developer.apple.com/app-store/review/guidelines/#third-party-software) (accessed March 18 2024): > **4.7 Mini apps, mini games, streaming games, chatbots, and plug-ins** > > Apps may offer certain software that is not embedded in the binary, specifically mini apps, mini games, streaming games, chatbots, and plug-ins. You are responsible for all such software offered in your app, including ensuring that such software complies with these Guidelines and all applicable laws. Software that does not comply with one or more guidelines will lead to the rejection of your app. You must also ensure that the software adheres to the additional rules that follow in 4.7.1 and 4.7.5. These additional rules are important to preserve the experience that App Store customers expect, and to help ensure user safety. > > - **4.7.1** Software offered in apps under this rule must: > - follow all privacy guidelines, including but not limited to the rules set forth in Guideline 5.1 concerning collection, use, and sharing of data, and sensitive data (such as health and personal data from kids); > - include a method for filtering objectionable material, a mechanism to report content and timely responses to concerns, and the ability to block abusive users; and > - use in-app purchase in order to offer digital goods or services to end users. > - **4.7.2** Your app may not extend or expose native platform APIs to the software without prior permission from Apple. > - **4.7.3** Your app may not share data or privacy permissions to any individual software offered in your app without explicit user consent in each instance. > - **4.7.4** You must provide an index of software and metadata available in your app. It must include universal links that lead to all of the software offered in your app. > - **4.7.5** Your app must share the age rating of the highest age-rated content available in your app. To support this, we will likely need to: 1. Review and approve a subset of plugins. 2. Disable access to certain parts of the plugin API (e.g. clipboard access) on iOS. ## Supporting plugin review Here are a few ideas for how we might support only showing reviewed plugins on iOS: > [!NOTE] > > Both of the ideas mentioned below continue to use GitHub as the source for downloading plugins. Depending on how trusted [the GitHub mirrors](https://github.com/laurent22/joplin/blob/56b010ba0ea22b03619f387ecfd428837095f761/packages/lib/services/plugins/RepositoryApi.ts#L19) are, we may need to disable them on iOS. Alternatively, reviewed plugins could be signed. > ### 1. Create a separate repository for reviewed plugins This repository could be located at `joplin/plugins-reviewed` or perhaps be a separate branch of the existing plugin repository. Joplin maintainers manually update plugins in this repository after reviewing them. With this method, the plugin review process would be similar to the following: 1. A maintainer either 1. Reviews the source code of the plugin locally, then builds the plugin and uploads it to the "reviewed" repository, or 2. Extracts the plugin, reviews the (probably minified) JavaScript source, then uploads the `.jpl` file to the private repository. **Pros**: - This method should require very little code change on iOS. - We could support this by using a different plugin repository URL (this could be changed [here](https://github.com/laurent22/joplin/blob/56b010ba0ea22b03619f387ecfd428837095f761/packages/app-mobile/components/screens/ConfigScreen/plugins/utils/useRepoApi.ts#L19)). - We could more easily have the @joplinbot build plugins from a specific commit and auto-upload to the repository. **Cons**: - The plugin repository would be split across two GitHub repositories or branches. This may make it more difficult to work with (e.g. when reviewing plugins). - If we want to show certain plugins/versions as `reviewed` on other clients, the client will need to check both repositories. - All plugins in the `reviewed` repository will be older versions of plugins in the main plugin repository. As such, reviewed plugins will need to update from the reviewed repository and not the main repository. To support this on non-iOS clients, we'll need to store additional data (e.g. add a `_reviewed` manifest key). ### 2. Create a new `reviewed` subdirectory in the [`@joplin/plugins`](https://github.com/joplin/plugins/tree/master) repository The review process for this option would be very similar to the process outlined above for option 1. **Pros**: - We still only need one plugin repository/branch. - Creating a separate folder for reviewed plugins makes them easier to find and update. **Cons**: - May be more complicated to implement than option `1`. ### 3. Add a new private manifest property that indicates the last reviewed commit A new private manifest key ([similar to `_recommended`](https://github.com/laurent22/joplin/blob/0715340dc0437efe785b227c2ea61820728b0485/packages/plugin-repo-cli/lib/validateUntrustedManifest.ts#L17)), `_lastReviewedCommit` could be assigned to plugins using [`manifestOverrides.json`](https://github.com/joplin/plugins/blob/master/manifestOverrides.json). On iOS, plugins would then be downloaded using the JPL from that commit. The download URL might then, for example, be `https://github.com/joplin/plugins/raw/e45d1b6361e986a88ff2bb63a51134588b5991f6/plugins/io.github.personalizedrefrigerator.joplin-vimrc/plugin.jpl`. The plugin review process would then be similar to the following: 1. A maintainer downloads a plugin's `.jpl` file from a specific commit, then extracts it and reviews the (probably minified) JavaScript). - Note: `.jpl` files can also be inspected using the [Plugin Website](https://joplinapp.org/plugins/view-source.html?plugin=io.github.personalizedrefrigerator.joplin-vimrc#index.js) by drag-and-dropping a `.jpl` file into the source viewer, which can format minified files with `prettier`. - We could also encourage plugin authors to [disable minification](https://webpack.js.org/configuration/optimization/#optimizationminimize) to make the review process easier. 2. After reviewing, a `_lastReviewedCommit` field can be added to [`manifestOverrides.json`](https://github.com/joplin/plugins/blob/master/manifestOverrides.json) to indicate which commit was last reviewed. **Pros**: - No need to maintain multiple plugin repositories. - No need to change the overall structure of the main plugin repository. **Cons**: - We can't build plugins' `.jpl` files ourselves when reviewing. - This could be fine, particularly if we encourage plugin developers to create [NPM packages that are verifiably linked to a CI build](https://github.blog/2023-04-19-introducing-npm-package-provenance/) for their plugins. - Likely requires more code changes than option 1. - May be unable to use mirrors to download plugins from a specific commit.

crtkugrrvjy · 2024-03-25T19:54:27.727Z

Amazing work thank you!

Embedded Search works!

personalizedrefriger · 2024-04-04T15:59:01.031Z

I've tested several plugins with a recent version of Joplin (around this commit). I've listed the plugins that seem to work and the plugins that are broken below.

The most common issues seemed to be caused by lack of file system access for plugins on mobile.

Broken plugins

Backup
BibTeX
- Requires filesystem access
- Fails on startup when trying to require child_process (required by a dependency? Doesn't seem to be in the main codebase...)
Freehand drawing
- Uses fs
Favorites
- Seem to require toplevel menubar items to add tabs (there's no menubar on mobile)
Combine notes
- Uses fs
- UI is in the list of actions when selecting multiple notes, which hasn't been exposed to plugins yet.
Conflict resolution
- UI for showing the conflict resolution window (right-click on a note) isn't exposed to plugins.
- (Based on the codebase) Uses fs-extra
Hotfolder
- Uses the file system
Templates
- Uses the file system
Persistent editor layout
- Mobile only has "editor" and "viewer" and doesn't expose commands for switching layouts to plugins.
Resource search
- Requires filesystem access.
Table formatter plugin
- Extends the right-click menu, which isn't available on mobile.
Note overview
- Expects the global process variable to exist.
Better code blocks
- No CodeMirror 6 support
Note list (Preview)
- No note list API

Partially working plugins

Note tabs
- There doesn't seem to be a way to pin tabs (except by enabling the "pin on editing" setting)
- There doesn't seem to be a way to close tabs
RevealJS slides
- Printing doesn't work
- Users need to click the "start slideshow" button, wait for the "switch to the note viewer" alert dialog, then switch to the note viewer to start a presentation.
Outline
- Its scrollToLine command only works in CodeMirror 5 (so jump-to-heading doesn't work).

Mostly working plugins

History panel
- Difficulty during setup — need to set a history note in advanced settings.
Kanban
ABC Sheet Music
- Has trouble with dark mode
Menu items, shortcuts, toolbar icons
- Extra CodeMirror toolbar buttons work
Rich markdown
- No alt key on many mobile devices to trigger the "show image on hover"
  - Works if "render images on their own line" is enabled.
- No ctrl key to trigger ctrl-click behavior.
CodeMirror 6 settings
- Logs an error when trying to update editor settings if no note is open (but otherwise seems to work).
Link Graph UI
- Seems to have trouble when switching between notes (or no note selected). Otherwise, seems to work.
Spoilers
- Shows an empty box, rather than an arrow, to the left of dropdowns rendered by the plugin. (This may be related to the fonts installed on my test device).

Working plugins

Quick Links
Inline Tags
Markdown table: Colorize
Function plot
Alerts Perfect
Admonition Markdown Extension

kentonself · 2024-04-08T02:07:53.620Z

I have been able to get the spoilers plugin and the admonitions plugin to work on my android phone and tablet.

yourmother9 · 2024-04-23T19:00:39.763Z

what about the calendar and agenda plugin on android.
this is very hard needed.

Benjamin-Watson0 · 2024-07-10T14:03:24.336Z

GitHub

GitHub - SeptemberHX/joplin-plugin-bundle: Gather several plugins together...

Gather several plugins together into one panel with more elegant UI design! - SeptemberHX/joplin-plugin-bundle

To-dos are really useful, this plugin does a great job of seeing all the to-dos, would you like to implement it on mobile or even desktop? Thank you.
@personalizedrefriger

shikuz · 2024-07-11T01:12:09.303Z

I love the bundle plugin (even used to contribute to it). It's strange that it's not displayed in the Joplin plugin store, because it used to be. Could it be that its manifest needs an update @personalizedrefriger?

I just re-built the plugin in a separate fork, so anyone can download the jpl and install it on desktop or mobile. It doesn't work on mobile because of dependencies that are not available (it will probably also need some modifications).

However, until it's migrated to the mobile app, if you're interested in displaying TODOs in a panel I recommend checking (disclaimer: my) Inline Tag Navigator plugin, which is supported on mobile. There are simple instructions how to set it up to display inline TODOs. You may use it to filter them and toggle checkboxes.

personalizedrefriger · 2024-07-11T01:31:08.969Z

shikuz:

I love the bundle plugin (even used to contribute to it). It's strange that it's not displayed in the Joplin plugin store, because it used to be. Could it be that its manifest needs an update @personalizedrefriger?

It seems to have been removed. The commit message states:

Remove plugin "Plugin Bundle" · joplin/plugins@c63147f · GitHub
Causes CPU to be at 100% all the time and bundles too many plugins.
Cannot be uninstalled in some cases.

Other plugins were removed on the same day.

Edit: Reworded, reformatted.
Edit 2: I don't see a corresponding GitHub issue. Edit 3: Corresponding GitHub issue.

shikuz · 2024-07-11T01:49:00.539Z

Thanks @personalizedrefriger, I wasn't aware of this process. And thanks @laurent for maintaining a clean and safe plugin repository! (I hope that @SeptemberHX knows.)

hieuthi · 2024-07-11T11:41:04.824Z

Look like Mobile support for plugin is up and running.

What is the requirement to make my plugins available on mobile? My understanding is some of them is compatible with mobile version already but I don’t see them in the install list.

personalizedrefriger · 2024-07-11T13:59:46.823Z

hieuthi:

What is the requirement to make my plugins available on mobile? My understanding is some of them is compatible with mobile version already but I don’t see them in the install list.

Setup

Add "platforms": ["desktop", "mobile"] to the plugin's manifest to declare that the plugin supports mobile.
Build the plugin with npm run dist.

Testing

Next, debug and test the plugin either in the mobile app itself, or with the work-in-progress web build of the mobile app.

Testing on a web build of Joplin mobile

The work-in-progress web build of the mobile app has two ways to test custom-built plugins:

Configuration > Plugins > Advanced > Install from file: Allows installing custom plugins. For local changes to be applied, plugins installed this way will need to be manually reinstalled.
Configuration > Plugins > Advanced > Development plugins:
- This only works in Chromium-based browsers.
- The development plugin should be reloaded 0-6 seconds after being changed on disk. However, for editor plugins, it may be necessary to exit, then re-enter edit mode for the changes to be visible.
  - It may be necessary to re-add the plugin in settings after refreshing the page.

Plugin logs (and a large number of unrelated warnings) can be seen using the browser development tools.

Testing on a physical device

On Android and development versions of the iOS app, plugins can be installed from the file system from Configuration > Plugins > Advanced > Install From File. See debugging mobile plugins for how to inspect plugins using the Chrome dev tools.

Running on iOS: Information about how to build and run Joplin in development mode can be found here. See iOS: Enable plugin support · Issue #10154 · laurent22/joplin · GitHub for why this requires a development build of Joplin for iOS.

Print statement debugging

console.info, console.warn, and console.error should all be sent to Joplin's logs.

Benjamin-Watson0 · 2024-07-22T14:01:21.344Z

I guess it doesn't support real-time updates of the to-do list on mobile devices.

Benjamin-Watson0 · 2024-07-22T14:07:49.664Z

Would you like to add a plugin that can display all the to-do items on the mobile side?
Based on my guess, the probability of other contributors doing this specifically for the mobile side is not high. But I am very much looking forward to being able to retrieve all the to-do items on the mobile side.

shikuz · 2024-07-22T14:40:05.906Z

I may be misinterpreting what you wrote, but if you edit a note or sync it, the panel should update the corresponding inline TODOs from that note. If there are issues I'd be happy to look into them, but perhaps let's continue in this thread.

djsudduth · 2024-07-25T22:07:14.680Z

@personalizedrefriger - I've added the platforms list to my Paragraph Extractor plugin and tested it on the work-in-progress web build. It has the fs error. What do I need to do here? Do I need to wait some more? It is similar to the Combine Notes plugin which also has the selection of multiple notes - not ready yet?

personalizedrefriger · 2024-07-25T22:33:29.219Z

Based on index.js in the built version of Paragraph Extractor, node:fs, node:os, node:util, and node:tty are both used by the plugin^[1]. These libraries are unavailable on mobile.

I suspect that fs, os, util, and tty are being required by i18n, though they could also be required by moment. Both libraries are imported in extractParagraphs.ts.

On mobile, plugins run in a WebView, without access to NodeJS APIs. i18n has an open GitHub issue requesting support for web browsers, so likely won't work in a mobile-compatible Joplin plugin.

Possible fixes:

Migrate to an i18n library that supports running within a browser JavaScript environment.
Use custom localization logic (e.g. different JavaScript objects for different locales). (Example).
Require i18n dynamically and only support localization on desktop.

Additional notes:

The path and url imports should work — polyfills for these libraries should be available to all plugins.
Requiring fs may seem to work in some mobile plugins: Joplin currently provides a non-functional fs mock to certain mobile plugins. However, adding Paragraph Extractor to this list would require releasing a new version of Joplin. This also wouldn't provide mocks for tty, os, or util.

djsudduth:

also has the selection of multiple notes

Be aware that the joplin.views.menus API is currently desktop-only. While it's possible to register menus with joplin.views.menus on mobile, they won't be visible in the UI.

Two possible alternatives are joplin.views.toolbarButtons and joplin.views.panels. The joplin.versionInfo API can be used to determine whether the current device is mobile or desktop.

Edited: Reformatting, linked to additional information about the i18n incompatibility, and added a section about other possible incompatibilities.

This is based on a ctrl+f search for require('fs'). ↩︎

djsudduth · 2024-07-25T23:33:34.262Z

@personalizedrefriger thank you for the comprehensive reply! I'll take a look at possibly disabling i18n for now and digging into the alternative APIs for views.