AI powered searching of notes?

Features
14

I recommend getting familiar with the readme and guide in order to understand the features that you're trying to use better.

It's worth noting that:

  1. The Chat command is unrelated to your notes. It's just like ChatGPT, but your conversation is stored in a note, and whatever you enter in that note can be used for context.

  2. Chat with notes adds to the context of the conversation (hopefully-)relevant sections from your notes, so that you can ask the AI about them. For example, if you have many travel logs you could ask Jarvis: "What was the thing I liked the most in our trip to Georgia?" and it will use the appropriate note to answer the question. There are additional usage examples from users in this thread.

  3. Semantic search is a different thing. It uses AI, but doesn't follow instructions. See this video for an introduction on the subject.

  4. Something we didn't mention yet, but could be relevant when searching for new information, is Research with Jarvis, which searches for summaries of academic papers online (see also this post). This is not the state of the art for this kind of research, but still worth mentioning.

All of these are described briefly in the documentation I linked to, BTW.

It's also important to note that (1), (2) and (4) depend on a powerful LMM to get intelligent responses, GPT-4 or Claude recommended for complex prompts, while (3) can work with the default offline model reasonably well (the online models are still a bit better).

In general, Jarvis is not an agent (yet!), meaning that it doesn't perform operations in the app and on your notes based on free-text, such as: "list all tags in notebook X" (a prompt that should run an API call to Joplin to get all tags and analyze them), "move all notes from X to Y" (another API call to Joplin), or "create a new note about heat insulation" (another API call, you get the idea...).[*] What it has access to is an indexed database of note sections based on embeddings. Once information is retrieved, a LMM can analyze it along with your prompt. What it can output is text that is inserted into the currently open note. It has its limitations and strengths.

[*] One exception is auto-tagging, a specific operation that is implemented as a feature in Jarvis.

2 Likes