Major breaking change in coming Joplin Server 2.0

Actually, I should've guessed given the topic name. For some reason I thought I was replying on a separate topic.

I'm not that familiar with the server so will let someone more knowledgable to comment.

2 Likes

What version of joplin desktop?

Joplin 2.0.1 (prod, win32)

Sync Version: 2
Profile Version: 37
Keychain Supported: Yes

Revision: 74d8fec98

And server is 2.0?

I believe so. I'm using this Docker Build

https://hub.docker.com/r/florider89/joplin-server

Ok then it's the wrong forum to ask because this is not the official release.

By the way @florider, could you remind me what is the purpose of this release? Is it to support ARM64 or is there more to it? I don't mind that you maintain a separate release, but it would be useful to mention that it's not the official one and what are the differences with it (or overall goal). This is mainly so that people know what they are installing and perhaps go to the right place for support.

I think florider's uses persistent storage, while yours does not.

This means should someone ever drop the container (or upgrade, which always drops the container) all data is lost.

1 Like

I believe the official was having problems too. I'll load it and try again.

  • know @florider's version has PostGres and not SQLite.

Ok, I used the standard instruction and got a new server going and I get the same exact error.

But why does Postgres recommend storing the data in the container? docs/README.md at master · docker-library/docs · GitHub You'd think they know what they are doing? In any case the pattern to upgrade a Docker container that uses the internal volume management is to pgdump_all, upgrade the image, then re-import the SQL dump.

Having the files in an external volume perhaps make the upgrade easier, but it has other drawbacks in terms of security and ease of use.

1 Like

It does not seem to me like they recommend it, at least not on the page you linked. The way I read it they just list various options without specifying what's preferable.

It's true they don't explicitly recommend it but they make it the default. If people lose their data every time they upgrade, then surely they'll set a different default?

@laurent do you have any idea why the server is not able to sync and getting the Not Allowed Put (Code 400) error?

1 Like

It would help if you were more precise when reporting bugs. What's the exact Joplin Server version you're using?

1 Like

Unfortunately they don't. People are supposed to know how to use Docker according to their philosophy.

This is why this makes it so dangerous. People who don't use Docker all the time might just use your docker file and think everything will be ok, when they upgrade, but then all their data is gone. I seriously doubt that they go into the postgres container and run daily backups of the database.

1 Like

Yup - only change is that it supports arm64 and the base image is ubuntu, which is slightly lighter. The rest is pretty much the same.

I've updated the tagline of the repo to make it clearer that it's not official.

Sure sorry, how can I help?

I'm not exactly sure where to find the version in Server, but all this is in the last day so I'm assuming all is latest. I saw you guys have an update a few days ago and wanted to stay current.

This is all fresh install stuff, not existing systems.

I can provide the login if it helps.

Sorry, my Docker Knowledge is lacking. I can't keep up with you new programmers, Haha.

Yeah, I'm really struggling with the Docker stuff at the moment. New Tech for me. DevOps!

Docker is an interesting topic.

It makes perfect sense to use Docker for testing and stateless services. But this is only my personal opinion. And I agree there are exceptions to that rule in certain circumstances.
Personally I dislike using backend sytems with persistent storgae requirements in Docker.

It doesn't mean it can't be done or that it is even bad practice. It makes orchestration with k8s and Nomad a bit more complex, since you need (1) a storage solution like Rook or Portworx. Also, I'm old school and come from a performance background. Doing perf tuning on container based systems is often a problem and sometimes very complex (and in rare cases impossible) to accomplish drilling into all the different layers.

A while back I came across "The Docker Book", which is a great resource for starting with Docker. It also includes more advanced topics. I think you can get a PDF copy for 10 bucks or so.

(1): In the meantime most orchestration solutions provide stable (although mostly basic) persistent storage.

2 Likes

Don't wanna nitpick or pretend I understand Docker all that well, but maybe I'll try to point out a difference: the link you posted mentions two options, "writing the database files to disk on the host system using its own internal volume management" and "Create a data directory on the host system". Neither of those is "storing the data in the container."

The "data directory on the host system" is what you'd expect: you say map "/my/local/path" to a location in the container, and that is then used to store files. It looks like "/my/local/path:/path/in/container". (This lets you specify where the files will be, but can sometimes be a bit annoying, since you might encounter issues with permissions etc.)

The "internal volume management" however doesn't refer to storing the data in the container, but in a named volume. Specifying a named volume looks like this: "volume_name:/path/in/container". This basically is a data container, created and managed by Docker, that lives outside the container. The advantages of this approach are that Docker handles everything for you: there usually are no permission errors, and you won't wipe/modify the data by accident. When you upgrade a container, you don't lose data, because the connected volume is not touched when the container is deleted. (The docs say "Volumes are easier to back up or migrate than bind mounts", but when I checked last year, backing them up did not look easy at all. Might be a PEBKAC, though.)

Any of the two approaches is fine; perhaps the volumes are better as a default for Joplin server, because they will most likely just work, and will not require the user to create (and then remember to not delete :smiley: ) any folders. On the other hand, just leaving the data in the container is definitely a recipe for disaster, because they will be wiped on each upgrade.

7 Likes