I was wondering what the vision is for Joplin's plugin system.
Something like https://packagecontrol.io/ would be awesome, but I think hosting such a system can get out of hand.
On the other side, we could just use a github repository and Joplin can retrieve the plugins from there.
So for people to publish a plugin, they only have to create a PR in e.g. https://github.com/joplinapp/plugins
Of course we have to define a few rules, like
creating something similar to a package.json file, although we should name it differently. e.g. jplugin.json which includes the version it runs on (semver), name, description, platform, ....
certain file/directory structure
I believe we can use GitHub's GraphQL to retrieve available plugins from the repo and present them in Joplin.
That's a good question and I'd be glad to hear what everyone think about this. We'd need a plugin repository eventually and need to figure out what shape it should take.
A GitHub repo where users can submit a plugin by creating a pull request indeed makes sense. What I'm wondering though is how to get that data into Joplin - for example let's say we want to display the list of available plugins in Joplin. Is there a good API for this in GitHub? Surely we don't want to want to have a git client running in Joplin.
I love this idea. Just found Joplin a month ago and collapsed my note-taking from about six different platforms to joplin with several coworkers and friends following suite after I introduced them to the platform.
One of the major selling points is the open architecture and forward-thinking regarding integrations and plugins. Having an effective a 'library' for plugins would be awesome. It makes the publishing process much more effective ... and provides the capability for collecting metrics for a recommendation/popularity engine around plugins which ... is would be super awesome. Peer review and reputation ftw!
I guess that would be straightforward, although the drawback is that we don't know what the jpl file contains.
use the repository to host the source code of the plugins
In that case we can know which is good, but it's more complex to do this I guess. We would need some CI to build the .jpl files based on the source code.
I think maintaining a list of repos would be enough, plugin developers would be expected to host the plugin on github (or potentially a different compatible server). Then the Joplin project would maintain a list of approved plugins that the Joplin clients read from. This is kind of how vim plugins are handled, but they don't have a central repository for approved plugins (I think Joplin should).
In that case, we can just read a JSON file from a separate repo e.g. joplin/plugins or just add this one file to the joplin repo itself: plugin_list.json
This file just includes the name, version, link to README (doc), and link to the .jpl file (in an array of course).
That seems reasonable, we just ask developers to tell us where they host their plugin, we add the link and that's it. It's simpler for us.
What I'm wondering also is what should be our stance in terms of security? We can't audit all the plugins and updates like Google or Mozilla are doing with the browser extensions, but maybe that's fine? Plugins are like any other software and users shouldn't blindly install everything. How do other package repositories handle this?
Yes, and perhaps we should also require plugin developers to host something like a "latest.json" file from which we can read the plugin version. That way we can notify when an update is available. Or if they use standard GitHub release, we could read the tag name too I guess.
Looks like vscode doesn't do much basically just verifies the package file and makes it available.
I think they can get away with this because of the large number of users though, I wonder if it wouldn't be prudent for us to manually screen a few plugins and mark them "reviewed". Although as I type that out I really don't like it, we can probably get by using peer review i.e. users will use the forum to informally rate plugins, we can remove any from the list that are found to be malicious.
Or if they use standard GitHub release, we could read the tag name too I guess.
We could probably just read the manifest file from the repository, it's already generated and should contain all the information we need.
There might be a way with being officially review without putting to much burden:
Review plugins when there is spare time and mark them reviewed.
Let Joplin check if plugin is officially reviewed, when adding it.
If plugin is not reviewed yet, put warning to user telling him
that's its dangerous and he does it on his own risk.
Or even help new users that will install unverified plugin by doing full data backup. It won't mitigate all risks but at least users won't lose their notes.
Not sure I follow. Do you mean the info from the gihub api or do you mean to download the .jpl file, extract the manifest.json file and use that info.
The former requires that people actually use gh releases and the latter requires to transfer the entire .jpl file just to display its information.
My worry here is that we will need to review not just plugin submissions but also plugin updates. This would mean that large (useful) plugins could fall out of verified status just because no-one has the time to do a full review and keep up with the development.
Plugin developers should be required to host the entire source of the plugin, just as shantanugoel did we can then just grab the manifest.json file from the src folder, at least I think that's possible?
Yes, this makes sense, but we also need the .jpl file to install it. Unless we fetch the source tree to a dev server and create the .jpl on the fly. Or maybe we should require a travis pipeline that builds the .jpl file in those repos. We might be able to create a pipeline template everyone can use.
The plugin development process already creates a .jpl file when the plugin developer runs npm dist. We can just require that developers push the .jpl file (and the dist directory).
Simply linking to a repo and requiring developers to publish their JPL file indeed makes things easier as we don't need to do anything on our side, beside maintaining a list of repo URLs.
On the other hand, it means we don't have much control over what's published. Even if we review, the developer could change files after that and we won't know about it. Or perhaps we need a repo url + tag, as I think tags are immutable (or maybe not?).
Then whenever a developer wants to update their plugin, they create a pull request on our repo and change the tag name. At that point we can review if we want to.
Right, additionally we could create a sha256 sum of the .jpl file. It's not signing, but the gpg npm packages are rather big 8-10MB, and it's better than nothing. Otherwise, if we really want to go with a verification process, we could also sign a package (.jpl file) with a Joplin key, but use something other than gpg, e.g. miniLock. I'm sure there are smaller libs than gpg out there that allow to digisign files. git tags are not immutable, but asset ids or node ids are.
+1 for the GitHub repository for plugins. The store/repo will be very important in order to find good plugins and maintain them easily - install, update, rate.
Thanks everyone for the feedback so far. From our discussion I'm putting below the requirements for a plugin repository. I think the most important question now is what information exactly a developer should provide.
Developers submit or update a plugin by creating a pull request
That's the easiest way both for us and developers.
Plugin location provided by developer should be immutable
In other words the content pointed by that reference should not change once it's in our repo. We still need to define what exactly the developer should provide - release URLs, tags and commits aren't good because the content pointed by them can be changed.
Optionally, developer should provide a tag or commit associated with the JPL file
So that it can be built and checked independently. Not necessarily by us, but reproducible builds are important to prove that nothing suspicious has been added to the minified JPL code.
It should be possible to access the manifest file
The client will need to access the manifest file, which is inside the JPL code. Downloading all the JPL files to get that info is not ideal, so that manifest file should be obtainable from somewhere else. If the developer provides a GitHub URL, we can find it in src/manifest.json for example.
Security policy
We cannot possibly review all plugins, just like Homebrew for example is not going to check every single package. However we can enforce important checks via the immutable URLs mentioned above, the ability to build the plugin independently, and generally a transparent process with the way plugins are added or updated in the repo.