Add ability to use docker secrets in Joplin server

Feature Request

As a user bringing up a Joplin server Docker container, I'd like the ability to set paths to secrets containing sensitive data such as user credentials, in environment variables in docker-compose.yml, so that I don't need to store plain text passwords and usernames inside my docker-compose.yml.

Disclaimer

I am not a security expert by any stretch of the imagination. My knowledge of attach vectors and best practices regarding security, storing user credentials etc. is fairly limited. However, I'm fairly confident that my hesitance to store plain text credentials in files I'd like to commit into version control is a bad idea.

I have made a suggestion for a change to the codebase that I believe would implement the base of my feature request. I have never touched Typescript in my life (except in the snippets below) and so I could be missing something obvious that makes this problem less trivial to solve than I think. This is also the primary reason why I didn't just submit a pull request via github.

Context

I'm setting up a new self hosted Joplin server using Docker. All of my server setup scripts and configs are stored in a (private) github repo that allows me to keep all my server config easily backed up and version controlled.

When setting up the Joplin server via Docker, I don't really like how database and mailer credentials are required to be stored in docker-compose.yml in plain text.

What I don't want to do:

  • Store passwords, tokens or secrets in plain text in any files I need to commit
  • Commit them as blanked out, and then have to go populate all of them after I clone the repo (this would basically guarantee the scenario of me forgetting to clear passwords before committing...)

The postgres credentials could be argued are less of a big deal since the connection is isolated to between the two containers, and I'd always randomly generate a new unique set of credentials purely for this connection anyway. It still feels not good though.

Storing mailer auth credentials in the clear feels a bit more dangerous. Although I can get a unique set of gmail smpt credentials for this particular application, if someone somehow got hold of these creds, I'd like them not to be able to do email stuff on my behalf.

What I do want to do:

  • Use docker secrets
  • Add a .gitignore catch for anything under .secrets, and never commit them
  • Store the secrets with appropriate ownership and access permissions
  • Allow Joplin server to take in a bunch of credential file paths in place of the raw credentials themselves

I have an example docker-compose.yml below. I've trimmed out some of the less relevant config, but the example should allow me to illustrate my envisaged usage.

# docker-compose.yml

version: '3.9'

services:
  joplin-db:
    image: postgres:15
    environment:
      - POSTGRES_DB_FILE=/run/secrets/joplin_db_name
      - POSTGRES_USER_FILE=/run/secrets/joplin_db_username
      - POSTGRES_PASSWORD_FILE=/run/secrets/joplin_db_password
    secrets:
      - postgres_db_name
      - postgres_username
      - postgres_password
    volumes:
      - joplin-db:/var/lib/postgresql/data

  joplin:
    container_name: joplin
    image: joplin/server:latest
    depends_on:
      - joplin-db
    environment:
      - APP_PORT=22300
      - APP_BASE_URL=https://example.com
      - STORAGE_DRIVER=Type=Filesystem; Path=/home/joplin/storage
      - DB_CLIENT=pg
      - POSTGRES_DATABASE_FILE=/run/secrets/postgres_db_name
      - POSTGRES_USER_FILE=/run/secrets/postgres_username
      - POSTGRES_PASSWORD_FILE=/run/secrets/postgres_password
      - POSTGRES_PORT=5432
      - POSTGRES_HOST=joplin-db
      - MAILER_ENABLED=1
      - MAILER_HOST=smtp.gmail.com
      - MAILER_PORT=465
      - MAILER_SECURE=1
      - MAILER_AUTH_USER_FILE=/run/secrets/gmail_smtp_username
      - MAILER_AUTH_PASSWORD_FILE=/run/secrets/gmail_smtp_token
      - MAILER_NOREPLY_NAME=Joplin
      - MAILER_NOREPLY_EMAIL=admin@example.com
    secrets:
      - postgres_db_name
      - postgres_username
      - postgres_password
      - gmail_smtp_username
      - gmail_smtp_token
    volumes:
      - joplin-storage:/home/joplin/storage

volumes:
  joplin-db:
    external: true
  joplin-storage:
    external: true

secrets:
  postgres_db_name:
    file: ~/.secrets/joplin_db_name
  postgres_username:
    file: ~/.secrets/joplin_db_username
  postgres_password:
    file: ~/.secrets/joplin_db_password
  gmail_smtp_username:
    file: ~/.secrets/mail_smtp_username
  gmail_smtp_token:
    file: ~/.secrets/mail_smtp_token

Note that it's completely safe for me to paste this configuration here, since there's no chance I've forgotten to strip out any prod credentials before doing so :slight_smile:

It's also worth noting that support for this usage pattern seems pretty common and in fact the postgres image already supports the use of the *_FILE environment variable already (I'd have to check if postgres v15 supports this though).

One further note is that it seems not very widely known that docker secrets can be used without using a docker swarm, however, docker compose will indeed handle the above example just fine on its own.

Solution Space

Looking through the Joplin server code, it seems like it would be pretty straight forward to implement this functionality:

In env.ts::defaultEnvValues() add the following:

POSTGRES_DATABASE_FILE: '',
POSTGRES_USER_FILE: '',
POSTGRES_PASSWORD_FILE: '',

and

MAILER_AUTH_USER_FILE: '',
MAILER_AUTH_PASSWORD_FILE: ''

Then in config.ts add a check in databaseConfigFromEnv() and mailerConfigFromEnv() for the appropriate config fields to see if the *_FILE version of that environment variable is set, and if so try to read the contents of the file path stored in the environement variable, otherwise, use the regular environment variable as before.

Something like this:

function getEnvVarFromFile(path, fallback) {
	if (path.length == 0) {
		return fallback;
	}
	let envValue = fallback;
	try {
		const fileContents = readFileSync(path, 'utf-8');
		if (fileContents.length > 0) {
			envValue = fileContents;
		}
	} catch (error) {
		// Warn here maybe?
	}
	return envValue;
}

function databaseConfigFromEnv(runningInDocker: boolean, env: EnvVariables): DatabaseConfig {
	// ... Code ommited for clarity
	if (env.DB_CLIENT === 'pg') {
		// ... Code ommited for clarity

			return {
				...databaseConfig,
				name: getEnvVarFromFile(env.POSTGRES_DATABASE_FILE, env.POSTGRES_DATABASE),
				user: getEnvVarFromFile(env.POSTGRES_USER_FILE, env.POSTGRES_USER),
				password: getEnvVarFromFile(env.POSTGRES_PASSWORD_FILE, env.POSTGRES_PASSWORD),
				port: env.POSTGRES_PORT,
				host: databaseHostFromEnv(runningInDocker, env) || 'localhost',
			};
		}
	}

	// ... Code ommited for clarity
}

function mailerConfigFromEnv(env: EnvVariables): MailerConfig {
	return {
		enabled: env.MAILER_ENABLED,
		host: env.MAILER_HOST,
		port: env.MAILER_PORT,
		security: env.MAILER_SECURITY,
		authUser: getEnvVarFromFile(env.MAILER_AUTH_USER_FILE, env.MAILER_AUTH_USER),
		authPassword: getEnvVarFromFile(env.MAILER_AUTH_PASSWORD_FILE, env.MAILER_AUTH_PASSWORD),
		noReplyName: env.MAILER_NOREPLY_NAME,
		noReplyEmail: env.MAILER_NOREPLY_EMAIL,
	};
}

Remarks

I feel like this feature is definitely worth adding. To my knowledge it should be a fairly safe and isolated change to make and will result in a more secure experience for anyone standing up a Joplin server instance using Docker's compose methods.

What do you think?

1 Like

@jmdejoanelli welcome to the forum.

Currently you can remove the environment variables and put them in a separate environment file, say, .env.

Then, assuming that the .env file is in the same directory as the docker-compose.yml file, in the app section of the docker-compose.yml file add:

env_file: .env

You can then gitignore .env.

Example

...
    app:
        restart: unless-stopped
        image: joplin/server:latest
        depends_on:
            - db
        ports:
            - "22300:22300"
        env_file: .env
...

Your .env file would then look something like:

APP_NAME=Joplin Server
APP_BASE_URL=https://joplin.example.com
APP_PORT=22300
POSTGRES_PASSWORD=secretpostgrespass
POSTGRES_DATABASE=joplin
POSTGRES_USER=joplin
POSTGRES_PORT=5432
POSTGRES_HOST=db
DB_CLIENT=pg
MAILER_ENABLED=1
MAILER_HOST=smtp.example.com
MAILER_PORT=465
MAILER_SECURITY=tls
MAILER_AUTH_USER=mailacc@example.com
MAILER_AUTH_PASSWORD=secretSMTPpassword
MAILER_NOREPLY_NAME=JoplinServer
MAILER_NOREPLY_EMAIL=noreply@example.com

Would that help?

That would certainly work, however the .env solution is only half way to what I'd like to see.

Is it possible to have a hybrid setup where I set only the credentials in the .env file and then set all the other non sensitive settings in docker-compose.yml as environment variables? That way I could treat the entire .env file as a secret. If it contains all the non-sensitive config however, I'd have to rewrite that anytime I need to rebuild/reinstance my server, which doesn't bode well for my gold fish memory and my goal to automate server bring up as much as possible.

I still think supporting secrets would be a useful feature to have. As it unifies credential handling across most containers and semi-automates secret management (via Docker secrets).

I forgot to mention as well, that I'd still need to store the postgres credentials in two different places using the .env method. Once in the docker secret files, and once in the .env file for Joplin.

I.e. splitting the credentials out into their own secrets also makes sharing secrets among containers slightly cleaner, and the credentials always only exist in one place which makes them easier to keep track of and secure.