Hey everyone!
This week was huge because we finally launched v1! The core features are fully ready and the plugin is officially launched.
This week, I worked on three main things. First, I added a full unit testing suite to cover our core algorithm logic - making sure K-Means, K-Medoids, HDBSCAN, distance metrics, and TF-IDF tag extraction are working reliably. Next, I did a big code cleanup for v1 by refactoring and splitting up some of the monolithic files into smaller, cleaner modules. Finally, I am working on implementing an automatic K selection mechanism so the plugin can dynamically find the best number of clusters instead of using a hardcoded K=6.
Problem Faced While testing the Auto K selection, I ran into an issue when switching to Joplin's native AI search API. The auto K selection works perfectly for our local ONNX model, but when using the native API, it keeps returning K=2 as the best cluster count. Because the native embeddings don't blend note titles the same way, the vector space ends up more clumped, so K=2 mathematically gets the highest silhouette score even when there are clearly 6-7 categories. Also, I'm still noticing some noise notes sitting right on the border between clusters.
Another thing I noticed is that the cluster names generated right now aren't that good. I tried my best to keep everything running 100% locally, but extracting keywords locally has its limits for naming. To get really meaningful cluster names, I think we will eventually have to use AI to generate the cluster titles.
Plan for Next Week is to experiment with multiple things to improve the overall clustering quality and fix the K selection issue for the native API. I'll also look into better ways for cluster naming, while keeping an eye out for any feedback from the v1 launch!