Hey everyone!
This week, I worked on three main things. First, I fixed a critical bug in the native AI embedding path (PR #36) that I mentioned in the last weekly report. It turns out native embeddings (768D or 1536D) were getting rejected because of a hardcoded 384D check, which forced the plugin back to the local model every time. Once I fixed that validation and improved how chunk vectors get weighted and ordered, native AI clustering started working properly. Next, I added AI cluster naming using Joplin's built-in AI chat feature (PR #35). Instead of just picking raw keywords, it now uses note titles and top keywords to give clusters clean, natural names. I added a fallback so if the AI feature isn't set up, it smoothly falls back to local keyword naming. Finally, I spent time cleaning up residual any types.
Problem Faced: While digging into the Auto K issue from last week where native AI kept picking K=2, I realized that silhouette scoring naturally prefers smaller cluster counts on continuous note vectors. Running a full sweep for every option was also adding unnecessary overhead. To fix this, I am planning to use a simple formula based on the total number of notes; for example, picking K=5 for 50 notes or K=7 for 100 notes (I haven't finalized it yet, I will test it first and then decide).
Plan for Next Week is to open a small follow-up PR with this adaptive K scaling and tune UMAP parameters so clusters group together even tighter (it will help in reducing the number of noise notes), and release an important update to the plugin with all these fixes.