Changelog :
-
Edited: Section 3 - Committed to LLM as the selected scanner. Added Socket.dev testing conclusion and open gap for malicious dependency detection. Clarified
@joplinchain-based filter as the solution for transitive dependency noise. -
Included few questions at the end of section 3
1. Problem Statement :
The current Joplin plugin publishing pipeline relies on a "trust-by-default" architecture. When a developer publishes an update to the public NPM registry, an automated bot blindly pulls that package into the central Joplin ecosystem.
While this creates a frictionless Developer Experience (DX), it completely lacks automated security scanning, human reviews, and traceability back to the original source code. If a malicious or compromised package hits NPM, it is immediately distributed to Joplin users without any checking.
The Objective: Transition the Joplin plugin registry to a GitHub Actions-based submission queue. To minimize risk, the immediate deliverable for this project is a fully functional Proof of Concept (PoC) that operates in parallel to the existing system using a test repository, ensuring the architecture can be evaluated safely before affecting the live ecosystem.
2. Proposed Architecture & Developer Experience (DX) :
2.1 Current State Analysis :
The current Joplin plugin registry operates on a strictly automated architecture tied to the NPM registry.
The Automated Pipeline (update-repo.yml)
The central registry relies entirely on a 30-minute cron job that executes the @joplin/plugin-repo-cli. This backend tool continuously scans the public NPM registry for packages tagged with joplin-plugin. It fetches the .tgz payload, extracts the manifest.json, generates the .jpl artifact, and updates the master manifests.json registry with tracking fields. Because it directly mutates the registry and README.md, this workflow operates with high-privilege contents: write permissions.
Manifest Data Structures
The ecosystem relies on two core JSON configurations to manage registry state:
-
manifests.json: The generated master registry containing all metadata. -
manifestOverrides.json: A manual control file allowing maintainers to override the automated pipeline (e.g., flagging plugins as_recommended: trueor forcefully deprecating them via_obsolete: true).
2.2 The Proposed DX: A Dedicated CLI Gateway :
Previously, I was suggesting to build a seperate npm package to handle the new publish flow, depriciation error, authentication, etc. This was suggested thinking that the logic will be then isolated from the reach of the developer (As it will be in a dedicated package so dev cannot edit the file to edit the logic.. etc)
I was treating the CLI as the absolute safegaurd before the issue is created.
By your feedback :-
I came to realize treating CLI as the safety boundary is not important. It can purely be used for developer convinience... So, even if some dev try to do something malicious... The github action will work as the ultimate security boundary.
So, Instead of making a seperate package we can just inject the logic in generator-joplin package. We will still not use @joplin/plugin-repo-cli to divide the server side work and client side work (publsih flow).
As part of this transition, commandBuild in @joplin/plugin-repo-cli which currently excecute the NPM package getting cron job will be removed. The GitHub Releases upload logic from commandUpdateRelease will be migrated directly into the GitHub Actions publish job, eliminating the need for the CLI to hold GitHub credentials.
2.3 Authentication :
To maximize Developer Experience (DX), the CLI will use GitHub Device Flow to authenticate the user creating the submission issue.
This eliminates the need for developers to manually generate Personal Access Tokens (PATs). The CLI simply provides a code, opens the user's browser, and securely requests the minimum permissions required to open an issue in the Joplin registry repository on their behalf.
Here's a POC :
3. Threat Model & Security Scanning Pipeline :
3.1 The Threat Model :
For a deeper research please see :
Plugin Security Tool Comparison -- CodeQL, Semgrep & Gemini CLI
3.2 Tooling Selection: (LLM-assisted review)
Since, the threat model is more of a data flow kind now, AST scanner tools like semgrep , sonarQube (primary code quality scanner), etc does not have ability to scan them. Even if extra complex peice of custom rules are written, the main trade off will be that they will be doing something they are not designed for, hence a huge blind spot will always be left for the senarious which are not kept in mind while writting the rules + they will produce a lot of noise on normal plugins too.
The tool which stand chance was CodeQl, after several tests I came to realize the same things as the above tools, even though it has ability of taint tracking, there will always be blind spots for the senarious not taken in mind while writting the rules.
The LLM performed well in all of these cases, not only it was accurate at scanning.. It seldomly warned me about new things which are potential warning too in the plugins.
LLM would be a great call for use in this pipeline.
Pricing : (Taking plugin-yesyoukan as a base)
Gemini Pro API pricing for contexts larger than 128k tokens.
Input Cost: $2.50 per 1 Million tokens : 210,000 tokens = ~$0.53 USD
Output Cost: $7.50 per 1 Million tokens : 500 tokens = ~$0.004 USD
Total : ~$0.534 USD
The most token utilzed was in package-lock.json (~130,000 tokens)
By replacing the massive package-lock.json with a custom, SCA summary by filtering out useless noise from the package-lock.json file, we reduce the AI's context payload to roughly 82,000 tokens. The SCA summary will include :
-
Packages with Install Scripts: Any dependency marked with
hasInstallScript: true -
Suspicious Origins: Any dependency where the
resolvedURL points to a random GitHub repository. -
Direct Root Dependencies: The packages listed in the
package.jsonto check for typosquatting.
Using this the cost drops to :
Input Cost: $1.25 per 1 Million tokens : 80,000 tokens = ~$0.10 USD
Output Cost: $3.75 per 1 Million tokens : Assuming ~500 tokens for the report = ~$0.002 USD
Total : ~$0.102 USD
On dependency scanning
This seems to be due to "@joplin/utils": "^3.0.1" dep which explicitly lists @joplin/fork-htmlparser2 as a direct dependency. The security tool looks at that entire chain.
Suppressing them :
The 1st method would be ignoring the package-lock.json file and only scanning the package.json but that would leave a blindspot, as we should scan every code the plugin is using... that include what package the package in package.json is using`.
We cannot rely on the ommiting dev dependencies (npx snyk test --production ) either as the package was present in the dependecy.
On Snyk : The only remaining way left is to use .snyk yaml file which has an ignore: flag to ignore certain packages, I tried supressing these joplin/* :
The result was manually supressing each package in the
.snyk file. This will break if in future more packages are added, as we need to add the package here too.
For semgrep there was no such way to supress certain package. The only way was to use the .semgrepignore file to ignore the package-lock.json.
Solution :
Since, joplin packages like "@joplin/utils": "^3.0.1" are trusted we can run a filter for the SCA scan result and filter out the @joplin starting packages.
Lastly :
Both semgrep sca and snyk works same way as npm audit - flagging the known CVE's with a little larger CVE database.
The scanning dependencies for actual malicious code is actually done by the tool Socket.dev.
Socket.dev was tested against real published plugins and a purpose-built malicious postinstall script exfiltrating process.env.AWS_SECRET_ACCESS_KEY. It failed to detect the script because it only analyzes packages published to the npm registry, not author written scripts.
Its only alerts were criticalCVE warnings for Joplin dependencies and obfuscatedFile false positives on minified bundles.
The LLM scanner partially addresses this it can detect if a plugin's source code directly invokes suspicious functions from a third-party dependency. However it cannot detect malicious logic that runs entirely within a dependency's own lifecycle scripts (e.g. postinstall).
LLM will be used for both source code scanning and dependency analysis in the PoC, also keeping it convinient to replace it with any tool if needed further.
4. Approval, Registry Mutation, and UI Integration :
Automated scanning alone cannot prevent targeted attacks if a bad actor updates their logic to bypass static analysis. Therefore, human review remains the final gatekeeper.
Once a Joplin maintainer reviews the automated Markdown report, they approve the plugin by applying a specific GitHub Label (e.g., status: approved) to the Issue. This label triggers the secure mutation pipeline.
The build workflow will be kept in an group to prevent concurrency :
concurrency:
group: global-joplin-registry-mutation
cancel-in-progress: false
4.1 Separating Build and Publish Jobs :
Previously, due to too much complex wording I think I was not able to make you understand what does this part meant.
I've simplified the language.
Building a plugin requires the CI to run the developer's local npm run dist script. Because this is untrusted, third-party code, a malicious build script could potentially read the CI environment variables and steal the GITHUB_TOKEN to push unauthorized changes to the Joplin repository.
To prevent token theft, the GitHub Actions workflow is split into two separate jobs:
-
The Build Step (No Permissions): When a maintainer applies the
status: approvedlabel, the CI clones the repository and runsnpm ciandnpm run distto compile the.jplfile. This job is explicitly stripped of all repository permissions (permissions: read-all). The runner then saves the compiled.jplfile as a temporary GitHub Artifact. -
The Publish Step (Has Permissions): A second job starts. This job is granted the
GITHUB_TOKENpermissions needed to update the registry, but it does not execute any third-party code. It simply downloads the.jpl, pushes it to GitHub Releases, compiles the updated download counts intostats.json, and updates the central registry via the REST API.
Note: Unlike the current pipeline which installs packages with --ignore-scripts, the new build job must run npm ci and npm run dist to compile the .jpl from source. This necessarily executes developer-defined scripts, which is why the build job is stripped of all write permissions cause it can contain any malicious script that attempts to steal the GITHUB_TOKEN.
4.2 Error Handling & CI Failure Recovery :
Because this architecture relies heavily on GitHub Actions event triggers, it must gracefully account for runner failures or API timeouts.
To ensure maintainers can recover from failures without requiring developers to resubmit their plugins, the architecture provides two recovery mechanisms:
-
Native Job Re-runs: For standard failures, maintainers will utilize GitHub's native "Re-run failed jobs" button. Because the workflow context is preserved, the CI will re-evaluate the original Issue payload or Label event without requiring any manual data entry.
-
Manual Override (
workflow_dispatch): To protect against outages where an event trigger is completely dropped by GitHub, both the Review CI and Approval CI workflows will include aworkflow_dispatchhook. This provides maintainers with a manual "Run Workflow" button in the GitHub UI, allowing them to explicitly trigger a scan or approval by manually inputting the target GitHub Issue number.
4.3 Parsing & Update Lifecycle :
Structured Issue Payloads: To avoid the fragility of parsing free-form Markdown, the CLI programmatically generates the submission Issue with a JSON payload embedded in the body.
Comprehensive Update Scanning: When a developer submits a version update, the pipeline scans the entire codebase at the new commit, not just the diff. This guarantees that cross-file vulnerabilities (where malicious logic is split between old approved code and new updates) are fully detected.
The publish job will also replicate the existing updateReadme.ts logic , regenerating the plugin table in README.md after every approved submission, maintaining consistency with the current behavior.
5. Project Scope: Proof of Concept (PoC) :
Given the timeline, the immediate focus is to build a Proof of Concept that can be evaluated without disrupting the existing ecosystem.
- Parallel Submission Flow: The
generator-joplinCLI will be updated to include thenpm run publishcommand. However, the existingnpm publishworkflow will not be disabled. - Test Target: The new CLI will authenticate and open submission Issues strictly against a dedicated
joplin-plugin-test-registryrepository, leaving the livejoplin/pluginsregistry untouched. - Workflow Evaluation: All labeling, scanning, and Split-Job CI mechanics will be built and tested within this sandbox environment to allow the maintainer team to evaluate the generated security reports before full integration.
6. Plugin ID Uniqueness :
The Current Mechanism:
The existing pipeline already enforces plugin ID uniqueness by using _npm_package_name in manifests.json via checkIfPluginCanBeAdded.ts. By moving away from NPM, we lose this anchor. The new system replaces _npm_package_name with repository_url as the binding key, enforcing the same "first-come, first-served" guarantee through the CI check instead.
The Proposed Mechanism:
By shifting the publishing pipeline away from NPM and directly to GitHub Actions, we lose NPM's global id protection. Therefore, the central manifests.json file inside the joplin/plugins repository must become the Ultimate Source of Truth for identity and uniqueness.
To enforce uniqueness, the CI will utilize a "First-Come, First-Served" locking mechanism :
-
New Plugins (Registration): When a developer submits a brand-new plugin, the CI checks the
manifests.jsonregistry. If theplugin_iddoes not exist, the submission proceeds. Upon approval, the ID is permanently bound to the developer'srepository_urlin the registry. Once published, a plugin's registered repository cannot be changed or moved to a different URL. In case user want to change therepository_url, he would have to contact someone from the Joplin's team to get the url manually changed and verified. -
URL Normalization: Before any comparison, the CI will normalize all incoming URLs (stripping
https://, trailing.git, etc.) to prevent duplicate registrations likegithub.com/Foo/Barvshttps://github.com/Foo/Bar.git. -
Plugin Updates: When an update is submitted for an existing plugin, the CI performs a lookup. It compares the GitHub issue Repository URL of the incoming submission against the
repository_urlregistered to thatplugin_idinmanifests.json. -
If a user attempts to submit a plugin using an ID they do not own, the normalized URLs will not match. The CI will instantly reject the payload and close the submission issue with an automated comment:
"Error: This Plugin ID is already registered to another repository."
7. Conclusion: The Complete End-to-End Lifecycle :
To summarize how this architecture secures the ecosystem without sacrificing Developer Experience (DX), here is the complete journey of a plugin, from the developer's terminal to the end-user's application:
-
A developer finishes coding and runs
npm run publish. The CLI takes over, verifying the build, authenticating the user via GitHub Device Flow, and securely submitting the payload to the central repository as a structured Issue. -
The event-driven CI instantly wakes up and executes the scanner security scan, and generate the report.
-
A Joplin maintainer reviews the report and applies a
status: approvedlabel. This triggers the build job, which compiles the.jplartifact in a standard GitHub Actions runner without write permissions. -
The publish job (which has registry write permissions) downloads the
.jplartifact, uploads it to GitHub Releases, and updates themanifest.jsonregistry via the REST API. -
An end-user opens their Joplin Desktop app and browses the plugins. They see the newly published plugin with a crown badge, and install it normally.
By moving from NPM polling to a GitHub Actions submission queue, this project introduces necessary review steps to protect Joplin's users without adding excessive friction for plugin developers.


