Plugin repository?

Features
21

I've thought further about this, and taking into account the above requirements, I think we could have something like this:

To publish a plugin on the Joplin plugin repository, a developer would need to create a pull request with these two files:

  • plugin.jpl: This is the compiled plugin. Since it's likely to be relatively small I think it makes sense to host it directly on the repository. Long term it should make it easier to manage the repo.
  • plugin.json: This is just a copy of manifest.json. It is also packaged inside plugin.jpl but duplicated here so that the Joplin app can display information about the plugin, such as its name, description, etc. without having to download the complete plugin.
    • This file could have optional keys specific for publishing:
      • _commit: The commit associated with plugin.jpl, if it could be determined at build time
      • _hash: A hash of plugin.jpl
    • Both of those keys would be optional but recommended. The first one allows reproducible builds, and the second is to verify the file integrity (although if we're hosting it it's less important).

Both those files should be added to directory named after the plugin ID. So the repo structure would be like so:

README.md
/plugins
	/com.example.MyPlugin
		plugin.jpl
		plugin.json
	/com.example.OtherPlugin
		plugin.jpl
		plugin.json
...

If you can see any issue or something missed with this setup please let me know.

The plugin generator will also be updated so that the plugin.json is created, and the files are named as they are when they are published.

22

I'm assuming this is all with the intention that eventually you will be able to search and install plugins directly via Joplin without requiring the user to manually download and install the .jpl from the repo.
If that is the case (and apologies if I'm stating the obvious here) then, like the min version tag in the manifest/plugin.json (which I assume would prevent clients below that version from installing it) would it be useful to have a pre-release/beta/dev tag to prevent 'normal' users accidentally installing it whilst allowing others to opt in and help test/provide feedback without having to resort to the repo to download and install it manually?

23

This process does not allow for an automated update process.

e.g. A plugin developer updates their plugin in their repo and creates a new release (and even has the hash and commit somewhere, since you can't have the commit hash in that commit.)

But the suggested workflow allows for "vettting" new plugins and updates. So that makes sense too.

24

That's correct. Step 1 is to have a central place where developers can publish their plugin. Step 2, soon after, is to add a way to search this repository from the Joplin app.

That's a good point, but to keep it simple we'll only have one release channel for this release. Later on, I guess we could indeed have a pre-release channel which users can subscribe to if they want.

25

Yes and this is by design. The idea is that once we make a plugin available, we don't want it to change without us knowing it. So by hosting the plugin file directly, we can know what's been changed and when.

We want to avoid the scenario where a developer give us a URL, then the content at that URL is swapped to malware either because they got their account hacked, or transferred the repo to a malicious user.

On the other hand, it's true it makes it a bit more tedious for a developer to publish a plugin. They need to create a pull request on our repo, add the files, etc. Perhaps we can make that easier later on, for example by adding an npm run publish command to the plugin generator.

1 Like
26

In terms of security, I think too much. As far as vscode is concerned, I can directly publish every time I publish, and there is no review process.

27

Their process is very similar to ours as far as I can see. What's missing at this point is what I mentioned above, a command to directly publish to the repo.

But that makes me think, perhaps we could leverage npmjs - we'd require developers to publish to npm, which is very easy, and then we can fetch the plugin from there. We know packages over there are immutable so that would solve the security issue I mentioned above.

28

After a few tests, I think we are going to use instead the same approach as yeoman generators (Under "How to add your generator").

  • We'll use npmjs to store the plugin files, which will have many benefits:
    • We'll have built-in support for immutable versions (npm lets you remove package versions, but not change them)
    • Developers just need to run the regular npm publish to release their plugin, and it means we don't need to build tooling for publishing, auto-creating pull requests, etc.

So on developer side, it's very simple - just npm publish and it's done.

On our side, we'll still build a repo (similar to the structure mentioned above) based on these published plugins, so that the desktop clients can load them.

We can use npm search to find all the relevant plugins, then npm install them on some virtual machine. From there, we'll have the plugin files which we can push to some repository. All this is relatively easy to create, it will probably be just one cron script that runs a few times a day.

4 Likes