Home / GitHub Page

Web Clipper architecture

#1

I’d like to start a discussion about the current Web Clipper architecture.

I stumbled across a few questions when running my standard Joplin app and 2 other Joplin instances for development. There is no way to set the port number in Joplin, nor in the extension. So how could I ever use WebClipper with a running development version?
Currently it seems I have to stop all but one Joplin instance and then I should be able to use the WebClipper.

Wouldn’t it be better to be able to specify the port, even if it were just a hack or config variable (not changeable via the UI)? I am also ok with a UI setting, but I think the reason for not making it available in the UI was the ease of use.

#2

The app and clipper connect via a different port on dev and prod: https://github.com/laurent22/joplin/blob/808413d0bf1e90923fba0bb57c76d4a5e481e88e/Clipper/joplin-webclipper/popup/src/randomClipperPort.js#L2

If that port is not available, the next one is check and so on.

#3

Although this works in most cases, it makes it a little undeterministic which actual Joplin instance you’ll be talking to.

I’ve already found out (the hard way) that when you pass “–env foo” to joplin it will consider this a development version.

#4

If the env is correctly set as either prod or dev, it should be predictable what port is going to use (basically the first available port starting from the start port).

There’s indeed no validation of what the env is set to since for a long time it was a hidden setting just for myself, but maybe we could add a basic validation check now to avoid any surprise.

#5

I run ‘stock’ Joplin and Joplin from git. Both are production versions.
To distinguish them I run the git version with --env git-$gitrev (where gitrev contains the short revision), so I get this in the about box:

scrot20190510192658

However, since Joplin treats everything that is not "prod" as "dev", the git version is seen as development. I had to change the code to fix that (if not dev then prod, instead of if not prod then dev).

Have you considered adding Avahi (Bonjour, ZeroConf) support in Joplin? Then each Joplin can announce its port and applications can find it. Future proof.

#6

The current solution works, so I haven’t considered it. If it ain’t broke ¯\_(ツ)_/¯

#7

I can easily imagine a situation with several joplins for distinct purposes. If one joplin would announce itself as joplin-prod._tcp._local and another instance as joplin-aux._tcp._local, then specific applications at least have a chance to look up the instance they expect before falling back to the current method of checking ports.

Sure, nothing is broken. But improvements are nice.

#8

This is fine and useful, but my concern is as follows:

I’m running my “stock” Joplin same as @scurius. But then I also run a Joplin version via the ./run.sh script.
So how do I tell the WebClipper browser extension to use the API from the .run.sh Joplin instance?

#9

You need to run the clipper in dev mode too. This is the mode when you start the clipper extension with npm run watch (from the “popup” directory). If you try any of the packaged version it will be in prod mode.

You can also force a particular mode - for example by changing what’s returned from joplinEnv()

#10

Yea, this looks very cumbersome to me. An option to specify the port in the settings for the extension would be so much easier. By default it would be set to Auto and does what it does now. When choosing Manual, a new field to enter a port number could show up and voilà - problem solved.

#11

That option would be cumbersome to me anyway since I’ll have to hard-code ports in the clipper and the desktop app and make sure they match, basically having to do manual work that can be easily automated.

Now it’s simple: dev clipper connects to dev desktop app; prod clipper connects to prod desktop app. I’m not even sure why anyone would want something different at this point. Maybe when the app supports multiple instances we can think about it, but for now there’s no need.

#12

Now I am genuinely confused. Maybe we are talking passed each other.

Nothing changes in Joplin. Nothing changes in the web clipper web extension. The only thing that would be added is a way to enter a port manually. Should someone enter a port number that number would be used instead of the current algorithm - that’s all.

#13

I see what you mean, but I’m not sure how that would be implemented without modifying the clipper or desktop app.

#14

Ok, let me rephrase: Nothing changes when it comes to the bahavior.

But, I was currently only talking about the possibility to choose the port number (to connect to) in the clipper browser extension. This would make it a lot easier to tell the extension to use the port that is used by Jooplin instance X (e.g. the dev version with profile X).
I didn’t understand your remarks about hardcoded ports, because by default it would still use the current search algorithm (as it used now). Also, if you entered a port number, it would still not be a hardcoded port, but a dynamic one - the one you entered.

Setting the port number in the Jioplin app (the listening port) is a nice thing to have, but not a must have. It’s way more important to be able to set where to connect to.

#15

Dare I mention mDNS (avahi, bonjour, zeroconf) again? This is exactly intended to solve problems like this.

#16

Sorry I feel I’m being dense here, but I still don’t know what problem we’re trying to solve. There’s two env - dev and prod, both use different ports, so you can have prod and dev apps and clippers running at the same time if needed.

Sure we can do something more sophisticated, and maybe we’ll need it at some point, but for now the current solution is enough for development.

#17

It’s just a usability thing. Also, I hope there will be a version of Joplin that supports multiple instances, in which case the clipper needs a way to allow which port (instance) to use.

But you are right, it is not necessary, but would make things easier - that’s all.