Hello everyone,
I am new to this forum, although I often read posts in the past. Now I am (first time) writing my own post, because I want to tell you about a journey of mine and maybe, this journey is also interesting for you.
TL;DR: What started as a small side project to improve my approach in making handwritten notes in Joplin eventually turned into a setup where I use my Android tablet almost like an additional input device for my notebook. But getting there was quite a journey.
So for context, I am a long-time Joplin user. I love markdown and the simple and efficient layout of Joplin that allows me to efficiently lay down my thoughts. Also it is open source and provides an ever-improving plugin API that allows me to adapt Joplin to my individual needs and workflows. I use Joplin more or less as my "second brain" - and so far this works extremely well for me.
However, of course I faced a (small) number of limitations, that send me off on my journey. What started as a side-quest to make my handwriting process in Joplin more natural eventually turned into a rather elaborate composition involving my notebook, my Android tablet and a native android app. Getting there took quite a few detours, and more than two years ( on a rollercoaster - although not uninterrupted ).
I work a lot with math and therefore I love the latex integration, but the active thinking process often requires a more dynamic way of laying down math expressions - they are often intertwined with drawings and arrows and exclamation marks 
. Besides, although I like markdown, sometimes I have this connection to handwritten writing. It feels as some thoughts might flow better when writing them by hand. Anyway, the point I want to make is - I was extremely excited when the joplin-plugin-freehand-drawing plugin was announced. In fact I am a heavy user since day one and I used it together with my Samsung Galaxy Note tablet.
It worked okay. Don't get me wrong. It's a wonderful tooling and I am very thankful to the developers. I think they did a tremendous job here and for most use cases it works not okay - but absolutely fantastic, but I am something of an oddball here - it worked for my (personal) way of working okay.
Let me give you a brief overview of my issues - they are not all related directly to Joplin or the freehand drawing plugin, but more to my workflow. (Please note, that those are only my personal impressions, in my individual setup and do not reflect the workings of the software. If in doubt I configured something poorly.)
- When using the Joplin app on my Samsung tablet with the freehand-drawing the input was often lagging. In particular if drawings became large. Tbh the writing experience was lacking in particular when I compared it to other - let's say more native - drawing apps.
- Also I actually do not like keyboard-writing on my tablet - I am so slow with these screen keyboards, so I was always switching between my notebook Joplin where I entered markdown notes and the freehand notes on my tablet and waited for the synchronization ( which, btw, worked really well). However, this interrupted the flow state and was too cumbersome most of the time.
- I am a notebook-first person - for me the tablet is basically an additional input device, like a mouse or a keyboard, I want to use it to quickly sketch, add notes or annotate a drawing with some arrows and a smiley or so - and no my current notebook has no pen-sensitive screen.
So I was wondering can I make this setup better support my workflow - or can I embed my tablet into the workflow, such that using it feels equally natural to when I am using for instance my mouse?
For my first tests I used apps like SuperDisplay or VirtualTablet - what they do is, they mirror the notebook screen on the tablet and allow one to draw on it with the pen. I like the idea, but it was very laggy, even in my local wifi and I had also trouble connecting my linux notebook, which is - to make it even more interesting - also connected to multiple monitors, with the tablet and have a resolution configured that still let's me write by hand. In conclusion, as a first proof-of-concept this was already promising, but far from what I was actually looking for - and I learned I needed a more custom application.
However, I did not want to write my own android app, because creating a hand-writing widget is a Herculean task and I have a lot of respect for the persons that actually do this, as for instance the developers of js-draw, on which the freehand drawing plugin is based on. I had no delusions at all, that I would manage to build such an usable widget with high UX or at least I realized, that this would become a task in itself.
So I decided to look deeper into the handrawing plugin - maybe I am able to tweak it in a way that it is better working with my Samsung tablet - or so I thought at the time. I added a remote session tool, which I can use with js-drawings. See the image below. Next to the regular 'edit' button, which is displayed on the top-right corner of drawings, I added a 'remote edit' button, that opens a dialog.
(redacted due to new user)
The dialog shows an URL and a QR code that I can use with my tablet to connect to the remote session via browser. The plugin starts a webserver and serves a website with js-draw embedded that allows me to edit drawing in the browser.
(redacted due to new user)
Note: Up to this point I did it without AI, but from here on on-wards I heavily adopted it and used agentic coding - just for transparency. Many research "branches" would not have been possible without the extensive time-saving and support of AI.
This was already more usable than the screen mirroring apps and it supported my 'tablet-as-device' workflow really well. Also by using the browser on my tablet I am not forced to create my own app, but can focus on tweaking the served website. However, there is a but, and the but refers to the browser pen input support. I tested different browsers, in particular Chrome and Firefox, but both provided more or less the same writing-experience as the Joplin android app. The handwriting was just not "there" particularly when compared to other native apps on my Samsung tablet.
I used this setup for some time and did a lot of tweaking along the way. Sometimes I felt that it was getting better but then at some point suddenly I created a drawing and the experience was extremely poor again - honestly, it felt not like a controlled approach, but rather like a game of chance and although, I used it actually quite a lot, it quickly became clear what I was not at the end of my journey yet.
However, I learned a lot about the workflow that I was trying to bring to life. So for instance it quickly became annoying to always scan the QR code for editing a drawing, so that I added a companion mode, which basically provided a website (see image below), that listed drawings, after I clicked the 'remote editing' button.
(redacted due to new user)
I spent a lot of time trying to improve and tweak the handwriting UX. After trying to implement an elaborate test harness and reading a lot about the pen-support / implementation in modern webs browsers I finally had another idea. An idea that is my current state and which feels so natural and has such a high UX, that I want to share it with you. Maybe I am not the only oddball out there and someone is looking for exactly this or a very similar workflow.
Let's summarize my dilemma, browser handwriting was a pain, good handwriting-support was not on the roadmap for any browser and I did not feel confident in any way to build such an app by myself.
Back at the time I often used and tested different handwriting apps with my android tablet for comparing them to the UX of my tweaked implementation. That brought me to the idea to look for an open source native handwriting app and test its UX. In my search I stumbled over xnotes. Which is awesome. Writing in it feels really really good. It is responsive, has sensible menus and options and also my beloved infinite canvas. So I switched my entire approach - I dropped the browser support (or actually I kept it - but you know it's just there), and connected xnotes to the remote drawing sessions of my adapted Joplin freehand drawing plugin.
The workflow is depicted in the image below. A remote drawing session is started either by appending a new drawing or clicking the edit button on the top-right corner of an existing drawing (this also works with regular images in which case one can draw on top of them). Then on the tablet the session becomes available on the companion website and I can open it in xnotes. Here I have the full xnotes experience and can write, sketch and draw whatever I fancy. On the top right corner I added a 'save' button, which will synchronize the image back to Joplin, where it is then displayed in the notes.
Summarizing my journey now:
There are a lot of details I like about the current solution. Also I notice the path that led me here, because many 'annoying' details are already tweaked away and the process is streamlined according to my individual needs. It comes really natural to me when I work on notes to just start the remote drawing session, take my tablet and start scribbling. Also annotating images, which was initially only a side quest, comes in handy more often than I would have expected. I also added the other way around - what I mean is, that sometimes I start a note in xnotes on my tablet and then want to append it to a Joplin note.
Let's talk about all the open issues. The solution is by no means production ready or - "properly functioning". I use it on a daily basis and I often meet nuisances. For instance for some reason xnotes does not open the note and then I need to open it again or things like that. It is continuously improved and bug-fixed. Also the 'remote-drawing' connection is really only a solution for trusted local networks. There is no encryption yet. On top of that, the entire setup of the project is patch work. It is not like I can simply point someone to a repository and the whole setup is self-explainatory.
Then there is the whole can of worms regarding SVG compatibility - xnotes uses it's own format .xcanvas, while the handdrawing plugin is strongly based on SVG files. Right now it supports only a subset of drawing elements / tools - although, to me it feels like a sufficient subset (for my needs).
In conclusion, an idea, that started with a one-day-build / let's-see-how-far-we-get send me on a whole rollercoaster journey spanning more than two years (not uninterrupted). I am at a point now, where I am very happy with the existing solution. I feel like I still need to tweak a few things here and there to streamline my workflow even better, but the heavy lifting is mostly done. Of course, one get's ideas...
So, why am I writing all this, and what am I really interested in?
Well, first of all, how "oddbally" am I?
Does anyone else use a similar workflow? In particular, I would be interested to hear if you use your tablet more as an additional input device for your notebook or desktop rather than as independent note-taking device.
And for those of you using the Joplin Android app with a pen – what is your writing experience? Maybe other tablets harmonize better with Joplin than my Samsung? Or do you prefer to stick to Markdown notes altogether?
I would be really interested in hearing about your experiences and workflows.
Disclaimer: The described setup is by no means finished, production-ready or even simply reusable. I am sorry to say it so bluntly, but providing a usable setup for others would require a lot of additional work and time that I currently do not have. So please don't expect a GitHub repository or a downloadable solution at this point. I am primarily sharing my journey and would be interested in hearing about your experiences with similar workflows.