Mark-magic: Post joplin notes to blog

rxliuli · 11 October 2022 16:01

If you don't know what joplin-blog is, let me briefly introduce it: it is a cli tool for converting joplin notes into other forms of content, supporting blog/wiki frameworks such as hexo, vuepress, docsify, jeykll. ref: https://discourse.joplinapp.org/t/joplin-note-sharing-tool/13480

Since joplin-blog => mami => mark-magic, this has been rewritten for the third time, but this time it has become better and can support exporting to EPUB and online novel sites (if you are using joplin to write novels) . The following is an introduction.

Introduction

A markdown-based data connection and conversion tool that solves data conversion between different tools and coordination between some commonly used tools.

Some use cases:

Publishing notes online, such as generating content required for a hexo blog from the Joplin note tool and publishing it online at https://blog.rxliuli.com/.
Publishing a novel online, such as generating a novel website from a local markdown file and publishing it at https://tts.liuli.moe/.
Packaging a novel into an epub file, such as packaging "To the Stars" into an epub file for offline reading after downloading at https://github.com/liuli-moe/to-the-stars/releases.

Some previous community use cases:

Joplin => Hugo for generating a blog.
Joplin => Jekyll for generating a blog.
Possibly others...

rxliuli · 12 October 2022 16:55

@mami/plugin-hugo-output It has been simply implemented, it can generate the files required by hugo from joplin notes, basically just make some simple modifications to hexo, refer to the example project: mami/mami.config.ts at master · rxliuli/mami · GitHub

rxliuli · 4 November 2022 14:11

joplin <=> obsidian completed

ref: GitHub - rxliuli/mami: cross-tool document converter

Usage

Requirements

you need to install nodejs 18

Step 1: Create a new project

The following uses pnpm as the package manager, but you can replace it with npm

Create a new directory and enter

mkdir mami-starter && cd mami-starter

Then use your favorite package manager to initialize

pnpm init

Step 2: Install mami

Add @mami/cli and typescript as development dependencies of the project

pnpm i -D @mami/cli typescript

Add some scripts to package.json

{
  ...
  "scripts": {
    "gen": "mami"
  },
  ...
}

Create your config file mami.config.ts

import { defineConfig } from '@mami/cli'

export default defineConfig({
  input: [],
  output: [],
})

then run

$ pnpm run gen

> joplin2obsidian-demo@1.0.0 gen
> mami

start
end

Well, nothing happens because you have no input or output plugin defined. Continue to the next step.

Step 3: Install the required plugins

Install the required plug-ins to connect the required tools. Here we use joplin => obsidian as an example

pnpm i -D @mami/plugin-joplin @mami/plugin-obsidian

Modify your config file mami.config.ts

The token required by the joplin plugin here comes from the web clipper service

import { defineConfig } from '@mami/cli'
import * as joplin from '@mami/plugin-joplin'
import * as obsidian from '@mami/plugin-obsidian'
import path from 'path'

export default defineConfig({
  input: [
    joplin.input({
      baseUrl: 'http://localhost:41184',
      token:
        '5bcfa49330788dd68efea27a0a133d2df24df68c3fd78731eaa9914ef34811a34a782233025ed8a651677ec303de6a04e54b57a27d48898ff043fd812d8e0b31',
      tag: '',
    }),
  ],
  output: [
    obsidian.output({
      root: path.resolve(__dirname, 'dist'),
    }),
  ],
})

Step 4: Perform the conversion

Then you can rerun the following command

pnpm run gen

Now you will see the converted obsidian file in dist

example

plugins

API Documentation

Roughly speaking, plugins are divided into input and output plugins. The input plugin will return an AsyncGenerator, while the output plugin will return an AsyncGenerator will consume it in the handle hook function.

design

Intermediate format

{
  "id": "0e2510c9272449dbafe3e0f3fba12d74",
  "title": "Welcome to Joplin!",
  "content": "content body",
  "createAt": 1666288266591,
  "updateAt": 1666288266591,
  "path": ["Welcome! (Desktop)"],
  "tags": [
    {
      "id": "04dfa5cf19e4435f9f3f09a73a7edfb2",
      "title": "blog"
    }
  ],
  "resources": [
    {
      "id": "63b83e548b7b4adfae18544b7038b0bc",
      "title": "AllClients.png",
      "raw": "<nodejs buffer>"
    }
  ]
}

Writing plugins involves some markdown ast operations. For example, you may need to convert links in markdown. It is recommended to use mdast to handle it.

rxliuli · 18 November 2022 06:02

mami-cli.exe demo

Publishing a mami-cli binary, I tried making an exe that doesn't require installing nodejs or npm dependencies, see if anyone is interested.

Instructions

Download mami-cli.exe
Put it in the directory that needs to be manipulated

Add a config file, basically of the form [name, config], an example of converting from joplin to hexo

{
  "input": [
    [
      "joplin",
      {
        "baseUrl": "http://127.0.0.1:27583",
        "token": "5bcfa49330788dd68efea27a0a133d2df24df68c3fd78731eaa9914ef34811a34a782233025ed8a651677ec303de6a04e54b57a27d48898ff043fd812d8e0b31",
        "tag": "blog"
      }
    ]
  ],
  "output": [["hugo", { "baseUrl": "/demo/joplin2hugo-for-release/" }]]
}

Run the command ./mami-cli

An example project is https://github.com/rxliuli/mami/tree/master/demos/joplin2hugo-for-release-demo

release: Release mami-cli.exe · rxliuli/mami · GitHub

rxliuli · 26 November 2022 20:35

Implement the docsify plugin，ref: mami/demos/joplin2docsify-demo at master · rxliuli/mami · GitHub

demo: https://mami.rxliuli.com/demo/joplin2docsify/

maphew · 11 December 2022 10:49

Conversion from joplin to other formats is much faster than I expected, so nice!

The Obsidian output is nicer to work with than Hexo or Hugo because the filenames are the page titles instead of Hex ID string.

Is it possible to select a particular Joplin notebook to export instead of everything? I have a lot of notes that aren't suitable for web publishing. (Though it is comforting to have an quick and easy external backup of all of it.)

Pictures seem to be broken in the output files. The links I'm getting look like
src=":/2373913a38e3428eba18192417db21dd"
instead of
src="../resources/2373913a38e3428eba18192417db21dd.jpg"

Update: it's only pictures using html tags that break, regular markdown pics are okay.

rxliuli · 12 December 2022 02:20

The Obsidian output is nicer to work with than Hexo or Hugo because the filenames are the page titles instead of Hex ID string.

Not exactly, because hexo/hugo essentially expects no manual editing after the output, so directly using id as the file name is a convenient method, and it does not affect the final rendering of hexo/hugo. You can refer to the example: https://github.com/rxliuli/mami/tree/master/demos/joplin2hexo-demo

Is it possible to select a particular Joplin notebook to export instead of everything? I have a lot of notes that aren't suitable for web publishing. (Though it is comforting to have an quick and easy external backup of all of it.)

Yes, you can filter using the tag field, for example the following configurations are supported

{
  baseUrl: 'http://127.0.0.1:27583',
  token:
    '',
  tag: 'blog',
}

it's only pictures using html tags that break, regular markdown pics are okay

Yes, html is not supported yet, and I'm still not sure if I should support it, because html parser is another thing, and I should not add support for it for the time being, but anyway, the plugin can handle converting html in markdown by itself.

ps: The main reason why joplin to other formats is fast is that the intermediate format uses joplin. Although some problems have been found, the main problem is that the attachments and links cannot be distinguished, especially when the notes of attachments or links do not exist, it is impossible to judge

maphew · 12 December 2022 02:31

thanks for the extra info!

rxliuli · 6 January 2023 17:25

Published @mami/plugin-vuepress, support to generate vuepress files, refer to the example

sample project：mami/demos/joplin2vuepress-demo at master · rxliuli/mami · GitHub
sample site: joplin2vuepress | joplin2vuepress

rxliuli · 6 January 2023 18:36

Published @mami/plugin-vitepress, support to generate vitepress files, refer to the example

sample project：mami/demos/joplin2vitepress-demo at master · rxliuli/mami · GitHub
sample site: joplin2vitepress | joplin2vitepress

rxliuli · 18 December 2023 14:23

Long time no see. I have rewritten the project again to support the requirement of outputting it as an e-book epub, and have also added better documentation. Is anyone interested?

rxliuli · 6 February 2024 10:21

Release @mark-magic/plugin-joplin@0.11.7

fix: Fix the error that the generated path lacks the last file name
fix: Fix the error that the generated content lacks a first-level title
fix: Fix the generated content reference link error

ref: plugin-joplin | Mark Magic

nkapila6 · 11 August 2024 21:41

This looks great, can I do a jekyll blog instead of a hexo one?

Topic		Replies	Views
Publish notes to the internet Support	4	1051	15 February 2023
Is anyone using Joplin for blogging? Lounge	3	1595	26 February 2021
Introduce the plugin Joplin Publisher Plugins	28	1266	19 November 2024
Hability to create a blog from Joplin Notes Features	7	1834	29 May 2022
General Usage Pattern Questions Lounge	6	620	24 November 2021