Links
1. Introduction
Hello! I’m Madhan S, a second-year B.Tech student at Sri Eshwar College of Engineering(India), specializing in Artificial Intelligence and Data Science. My academic focus on machine learning, neural networks, and software engineering provides me with a strong technical foundation that aligns perfectly with the requirements of this project.
Technical Background
I’ve spent significant time working with TypeScript and JavaScript, particularly building Node.js applications and managing asynchronous patterns. My experience with React has also prepared me to handle the UI components of the chat panel effectively. Additionally, I’m proficient in Python, which I’ve used extensively for data wrangling and ML-related scripting.
On the AI front, I have hands-on experience working with LLM APIs, vector embeddings, and RAG pipelines through both my coursework and personal projects.
Open Source Experience
I’m an active contributor to the open-source community, with merged PRs in:
I’ve already set up the full Joplin development environment locally and spent time diving into the plugin API and data access patterns. To ensure I hit the ground running, I’ve even built a small proof-of-concept before finalizing this proposal.
2. Project Summary
The Challenge
For many Joplin users, their note collection is a vast repository of research, personal insights, and project logs. However, as these collections grow into the thousands, traditional keyword search becomes a bottleneck. If a user cannot recall the exact phrasing used in a note from a year ago, that information is effectively buried. Currently, there is no way to truly "interrogate" your knowledge base; you can only search for specific terms.
Solution & Its Impact
This project introduces a native Joplin plugin that enables natural, conversational interaction with your personal knowledge base. By implementing a Retrieval-Augmented Generation (RAG) pipeline, users can query their notes as they would a knowledgeable colleague using plain language and following up with clarifying questions.
Crucially, this maintains Joplin's privacy-first, local-first philosophy. By prioritizing Ollama as the default local LLM runner, users gain the power of conversational AI with the assurance that zero data leaves their machine, a significant privacy advantage over cloud-based alternatives like Notion AI.
Key Deliverables
I aim to deliver a robust, end-to-end RAG implementation with the following core features:
- Intuitive Interface: A ChatGPT-style React UI integrated directly into a Joplin panel, featuring real-time streaming responses.
- Context-Aware Chat: Support for multi-turn conversations, where follow-up questions retain the context of the previous three interactions.
- Citations & Navigation: Every AI-generated answer will include clickable citations, allowing users to jump directly to the source note within Joplin.
- Flexible Backends: Default support for Ollama (local) with an opt-in for OpenAI (API-based) for maximum user choice.
Performance & Reliability
I have set clear benchmarks for success, including a Recall@3 score of ≥ 0.85 on a standardized test set and high-speed indexing (5,000 notes in under 3 minutes). The final delivery will include a comprehensive test suite and full documentation for both users and future developers.
Project Scope
To ensure a high-quality initial release, the project will focus exclusively on the core RAG experience within the chat interface. Features such as cloud-sync collaboration, additional LLM providers, or direct note editing via the chat are considered out of scope for this initial phase.
3. Technical Approach
Architecture & Components
The plugin is designed with a modular, three-tier architecture to ensure reliability and ease of testing:
-
Layer A - Ingestion Pipeline: This layer handles note discovery via the joplin.data.get() API. To maintain performance, notes are processed into overlapping 512-token chunks using a Markdown-aware splitter. These are then embedded and stored in LanceDB, a high-performance, file-based vector database that requires no external server. To keep the index current, the pipeline listens for onNoteChange events to re-index only modified content.
-
Layer B - Retrieval Engine: When a user asks a question, this engine performs an Approximate Nearest Neighbor (ANN) search. I am implementing a Hybrid Ranking system (combining semantic search with BM25 keyword scores via RRF) to ensure the most relevant context is found. To prevent context window overflow, tiktoken manages token counts, while the last three dialogue turns are preserved for conversational continuity.
-
Layer C - Chat UI: Built as a native React panel, the interface communicates with the backend via Joplin's postMessage bridge. It supports real-time token streaming for a responsive feel and includes clickable citations that link directly to the referenced notes.
Technology Stack
| Component |
Technology |
| Language & UI |
TypeScript, React, Joplin Panel API |
| Vector Database |
LanceDB (Local, file-based) |
| AI Models (Local) |
Ollama: llama3 (LLM) & nomic-embed-text (Embeddings) |
| AI Models (Cloud) |
OpenAI: GPT-4o & text-embedding-3-small |
| Processing |
LangChain.js (Chunking), tiktoken (Token management) |
| Search/Ranking |
flexsearch (BM25) + RRF Re-ranking |
Addressing Potential Challenges
- Scalability: For users with 10,000+ notes, indexing will run in batched background workers with a progress indicator to ensure the UI remains fluid.
- Markdown Formatting: I’ll extend the
MarkdownTextSplitter to handle Joplin-specific syntax, ensuring code blocks and tables remain intact and readable for the LLM.
- Environment Readiness: If Ollama isn't detected, the plugin will provide a graceful setup guide rather than failing, offering a seamless fallback to OpenAI if a key is provided.
Testing & Documentation
I am committing to a rigorous testing strategy, moving from unit tests for chunking and database integrity to full integration tests that validate multi-turn coherence. The project will be benchmarked on a 10,000-note collection to ensure we maintain a Recall@3 ≥ 0.85.
Final deliverables will include a comprehensive User README (with a dedicated Ollama walkthrough), a Developer Architecture Doc, and full JSDoc coverage for the codebase to ensure long-term maintainability by the Joplin community.
4. Implementation Plan
The following roadmap outlines the development phases, with primary deliverables marked as [R] and optional stretch goals as [O]. I have allocated a total of 350 hours for this project, including a dedicated buffer for mentor feedback and rigorous testing.
Phase 1: Foundation & Ingestion (Weeks 1 - 6)
- Weeks 1 - 2 (May 27 - Jun 9): Community Bonding & Prototyping [R]
Deep-dive into Joplin’s API and the LanceDB SDK. Prototype the MarkdownTextSplitter using real-world note exports to ensure code block integrity and finalize the design doc with my mentors.
- Week 3 (Jun 10 - 16): Note Fetcher Implementation [R]
Develop the paginated joplin.data.get() loop. Validation will include unit tests for pagination accuracy and ensuring a 250-note fixture returns full data.
- Week 4 (Jun 17 - 23): Advanced Chunking [R]
Integrate the text splitter with specific focus on chunk size bounds (512 tokens), 64-token overlaps, and maintaining heading-boundary integrity.
- Week 5 (Jun 24 - 30): Vector Database Integration [R]
Set up LanceDB and define the schema (id, noteId, title, chunk, vector). Conduct performance benchmarks for indexing speeds at scales of 1k, 5k, and 10k notes.
- Week 6 (Jul 1 - 7): Embedding Providers [R]
Build the EmbeddingProvider interface to support both Ollama and OpenAI backends. Test for vector dimensionality and seamless provider switching.
Phase 2: Retrieval & Intelligence (Weeks 7 - 10)
- Week 7 (Jul 8 - 14): Midterm Checkpoint [R]
Demonstrate an end-to-end ingestion pipeline. Goal: 500 notes indexed with a terminal query successfully retrieving the top-3 relevant chunks with source IDs. Initial Recall@3 measurement.
- Week 8 (Jul 15 - 21): Hybrid Retrieval Engine [R]
Implement hybridRetrieve() combining ANN search with BM25 keyword scoring via RRF re-ranking. Integrate tiktoken for precise context window management.
- Week 9 (Jul 22 - 28): LLM & Streaming Integration [R]
Connect the retrieval engine to Ollama and OpenAI. Enable token streaming to the Joplin panel via the postMessage bridge and verify source citation accuracy.
- Week 10 (Jul 29 - Aug 4): React Chat Interface [R]
Build the frontend panel featuring the message thread, real-time streaming display, and interactive citations that navigate directly to notes in Joplin.
Phase 3: Refinement & Scale (Weeks 11 - 14)
- Week 11 (Aug 5 - 11): Multi-turn Conversation & QA [R]
Implement conversational memory (last 3 turns). Execute comprehensive integration tests for dialogue coherence and bridge latency.
- Week 12 (Aug 12 - 18): UX & Incremental Indexing [R]
Develop the settings panel for provider/API configuration. Implement onNoteChange() background indexing and build robust error handling for edge cases (e.g., Ollama missing, disk full).
- Week 13 (Aug 19 - 25): Large-Scale Validation [R]
Perform end-to-end stress tests on a 10,000-note collection to validate Recall@3 ≥ 0.85. [O] Implement a visual progress bar for the initial indexing build.
- Week 14 (Aug 26 - Sep 1): Final Documentation & Launch [R]
Complete the User README, Developer Architecture Doc, and full JSDoc pass. Submit the plugin to the Joplin marketplace and finalize the project evaluation.
Resource Allocation
- Development: 340 Hours
- Buffer: 10 Hours (Reserved for PR reviews and mentor feedback iterations)
5. Deliverables
The primary outcome of this project will be a production-ready, installable Joplin plugin that provides a seamless "Chat with your Notes" experience. The specific deliverables include:
- Production Plugin: A fully functional Joplin plugin implementing an end-to-end RAG (Retrieval-Augmented Generation) pipeline with a native React chat interface.
- Core Ingestion Pipeline:
- A robust, paginated note fetcher.
- A customized
MarkdownTextSplitter for intelligent note chunking.
- A local, high-performance LanceDB vector store.
- Support for both Ollama (local) and OpenAI (cloud) embedding backends.
- Advanced Retrieval Engine:
- Hybrid search capabilities (ANN + BM25) with RRF re-ranking for maximum accuracy.
- Precise context management using
tiktoken.
- Multi-turn conversational memory for follow-up questions.
- Conversational UI:
- A responsive chat panel featuring real-time token streaming.
- Interactive, clickable citations that link directly to source notes.
- A comprehensive settings panel for model and API configuration.
- Smart Indexing:
- Incremental indexing triggered via
onNoteChange() to keep the database current.
- A manual re-indexing option for full library refreshes.
- Quality Assurance Suite:
- A full suite of unit, integration, and scale tests.
- Performance validation on a 10,000-note collection (targeting Recall@3 ≥ 0.85).
- Documentation:
- User README: Clear setup and usage instructions, including an Ollama walkthrough.
- Developer Architecture Doc: A deep dive into the data flow and extension points.
- Code Documentation: Full JSDoc coverage for all public functions to ensure maintainability.
6. Availability
Weekly Commitment
I am fully committed to dedicating approximately 30 hours per week throughout the GSoC coding period (May 27 - September 1, 2024). My goal is to maintain a consistent development pace to meet all milestones on schedule.
Communication & Time Zone
- Time Zone: IST (UTC +5:30).
- Sync-ups: I am available for weekly video calls with mentors and will be highly responsive to asynchronous communication via the Joplin forum and GitHub.
Academic & Other Commitments
My end-of-semester exams typically conclude in early May, ensuring I am clear of academic hurdles before the coding period begins. I have no conflicting internships, part-time jobs, or other programs during this window. Should any minor academic assessments arise during the summer, I will provide mentors with at least one week's notice to ensure project continuity.
Post-Program Maintenance
I view this project as a long-term contribution to the Joplin ecosystem. Beyond the GSoC period, I intend to maintain the plugin by incorporating community feedback, expanding support for additional LLM providers, and ensuring compatibility with future Joplin API updates.
