I've opened an RFC here to see whether we can integrate a review process to plugin publishing. If you have any comments or questions about it feel free to post here or on GitHub!
The full post:
Currently, the Joplin application gets the list of plugins from our GitHub repository, and that repository is automatically built based on the third-party plugins that have been published to npm. We generally don't do any checks on these plugins except for those that are marked as "recommended", which are those developed by the team or long term contributors.
In order to make it safer for users to install plugins, it would be good to have a review process. Many plugins are relatively simple and we have perhaps 3 or 4 updates per week so it should be doable. The advantage is that we can guarantee that all reviewed plugins are safe to install, which would be interesting to users who no longer need to limit themselves to recommended plugins only.
To minimise changes to the existing process, we could integrate the review process in that way:
Publishing to npm will no longer be relevant
Instead, we would check the code directly on the repository
The code itself
But also the dependencies - we should only accept dependencies to well-known, safe packages
If all is good, we record the repository url and commit that we've reviewed
When building the plugin repository, we would automatically:
Fetch the third-party repositories at the previously recorded commit
Build the plugin
Add the .jpl to our plugin repository
Once we have the .jpl the rest of our process will be identical.
Questions
What to do about plugins that we have not yet reviewed - should we include them and mark them as "not reviewed". Or exclude them completely?
How about plugins that are not published to any public repository? (not sure if they exist)
For review, we should have some tool to generate a diff between the last reviewed commit and the latest one
We should probably have a process to allow the plugin developer to tell us when their plugin is ready for review, otherwise we may fetch a commit that's still under development
I would display them anyway, as there may be many useful plugins that are no longer listed and all reviewed are marked or the other ones marked as not reviewed.
I would still have them installed manually, but as these are not reviews, it is everyone's responsibility
How would it be that there simply has to be a git tag for the published versions?
I think the ideal situation would be that only "recommended" plugins were visible by default, with a checkbox that allows viewing the rest but labelled advising the users that they haven't been checked for quality and may not get future maintenance.
I think the maintenance part of this is as important to consider as e.g security. Whilst we want the plugins to be secure, you'd also not want the plugin ecosystem to rot overtime, where people might refuse to upgrade Joplin because an essential plugin for their workflow doesn't work with newer releases. Or where a plugin might begin to actively interfere with the core app and might not be ever updated; harming the opinion of the core app when people attribute blame to the core app and not the plugin.
But this is coming from the perspective of someone who absolutely tries minimise how many programs/tools/plugins I install regardless of what they relate to, not opinions about Joplin itself specifically since I don't use Joplin plugins anyway.
Asking plugin authors to promise a best effort going forward as a requirement to becoming recommended is probably as important as the security review side of things.
Show all public plugins, regardless of review status, to make them discoverable without extra clicks. Categorize plugins by review status like how the plugin discovery site does for recommended plugins. This visually separates plugins by status and designates a spot to explain each status.
Is there any protocol for handling abandoned plugins, broken plugins, or those incompatible with newer versions of Joplin?
Firefox has a model y'all may consider adapting, in the Recommended Extensions program. My suggestion would be to get started by adopting just the basic tenets. Consider the rest later as the Joplin plugin program develops.
tl:dr
What is this?
"Recommended extensions differ from other extensions that are regularly reviewed by Firefox staff in that they are curated extensions that meet the highest standards of security, functionality, and user experience. Firefox staff thoroughly evaluate each extension before it receives Recommended status."
Discovery
Those Recommended get a badge, and can be found through filters.
For unmaintained plugins, there is also a mechanism for transferring the source NPM package used for updates. This could be useful if, for example, the Joplin team decides to start maintaining an abandoned plugin. So far, this mechanism hasn't been used.