Just that you know Ollama is now also available for windows. 
I will try running Jarvis with ollama and litellm and see how it goes. Mistral 7b seems to have good performance in terms of capabilities and despite its size, Ollama somehow manages to run it on my PC! Gemma 2b also seems to work well and is super fast, but I think this model will need more tweaking in the system prompt to make it work with Notes.
@shikuz Any luck testing other local embedding models? I tried Ollama with AnythingLLM and AnythingLLM has a list of supported LLMs including tools that allow to run local embedding models. One of them is localai.io. From the GitHub repository it does not seem to have a windows installer, but maybe it could work on docker in WSL... Maybe you could check it to get some inspiration for Jarvis.
I also saw this embedding model on huggingface:
distiluse-base-multilingual-cased-v1
But I'm struggling to find a way to run it locally as it doesn't have a gguf file. And I want to avoid having to make it work with python. 