Weekly Update 2: UMAP Integration & Multi-Algorithm Clustering Benchmark

GSoC Projects Note Categorisation
1

Hello everyone! Hope you all had a great week. Here is a quick update on our note categorization plugin:

Progress from last week:

  • Switched to umap-js: I originally tried druidjs for UMAP dimensionality reduction but ran into security issues and TypeScript bugs. I switched to umap-js which works perfectly. In my proposal, I had discarded umap-js because the repository is not actively maintained, but after reading more about it, I found out it was developed by Google under PAIR-code, so I thought it was worth giving it a try, and it turned out to be a highly reliable and stable choice.
  • Implemented 3 Algorithms from Scratch: Wrote K-Means, X-Means, and K-Medoids directly to keep dependencies minimal.
  • Added HDBSCAN: Integrated hdbscan-ts for density-based clustering. I added a new parameter (minSamples) to control how many notes get flagged as noise, because a lot of notes were being left as noise without it.
  • Created a Benchmark Pipeline: Set up a pipeline to test all these algorithms side-by-side. I wanted to test on a real set of 48 notes rather than fake data, which made this PR (feat/umap-clustering-benchmark) quite large.

Benchmark Results:

Here is how the 7 algorithms compared when clustering 48 real notes:

Strategy Algorithm K (Clusters) Silhouette Score (Higher is Better) Outliers Time
hdbscan-5-ms2 HDBSCAN 4 0.7504 12 4ms
hdbscan-3 HDBSCAN 6 0.6794 14 8ms
kmedoids-5 K-Medoids 5 0.6722 0 6ms
hdbscan-3-ms2 HDBSCAN 6 0.6700 11 4ms
kmeans-5 K-Means 5 0.6686 0 1ms
kmeans-8 K-Means 8 0.5164 0 0ms
xmeans-auto X-Means 15 0.2825 0 1ms

I have added the clustering results in the issue comments: [Feature] UMAP with various clustering algorithms · Issue #15 · joplin/plugin-note-categorization · GitHub

HDBSCAN with tuned parameters (hdbscan-5-ms2) performed best. It successfully grouped topic-specific notes (AI, Finance, Health) and kept unrelated notes (like food recipes) separate as noise.

Plan for this week:

  • Work more on the PR: Since this PR is one of the most important ones, I will finish the remaining tests and get the benchmark PR merged.
  • Joplin Settings UI: Move the configuration settings out of the code and into the Joplin settings panel.

Problems faced this week:

  • Resolving library conflicts with druidjs and configuring skipLibCheck: true to bypass TypeScript errors.
  • Tuning HDBSCAN parameters so it doesn't classify too many notes as noise.

References:

For the mathematical details behind these algorithms, you can refer to these papers. Since the papers were hard to understand at first glance, I also referred to these documentation guides:

3 Likes