Template Plugin

[nishantwrp]

Hi everyone!

I'm Nishant Mittal (https://www.github.com/nishantwrp) from India and this summer I'll be working with Joplin on the template plugin!

Abstract

Joplin’s desktop app has a template feature i.e. users can make their templates and reuse the templates. For example - SWOT Analysis, Time Table, Meeting Notes, Journal, etc. and what not. But Joplin also has an awesome plugin architecture.

So it would be better if we repackage the template feature as a plugin due to many reasons like

• Better maintainability
• Features packaged as plugins would be more robust
• The feature would’ve better scope of improvement.

I basically propose to remove the existing feature and repackage it as a plugin while improving the template functionality.

Project goals

There are two main objectives of this project.

• Create a new plugin with improved template functionality
• Remove the template functionality from the main application

Plugin Design

The main improvements I plan to bring in the template functionality are as follows.

• Removal of Refresh Templates - Currently there is a refresh templates option in the templates menu that reloads the templates from the template directory. Removing this option and loading the templates everytime user tried to select a template would improve the usability of the feature.

• Default Templates - It is highly likely that most users have one or two templates that they use the most. And they want to create a new note using that template they’ll currently have to create a new note and then select a template that they want to insert. So, to make it more useful I’d like to introduce default templates feature. This feature was also once requested in the Joplin Forum, however my proposal of this feature is not exactly similar to the request of this user. The details of this feature can be found below in this section itself.

• Pre Installed templates - When I tried to see the templates functionality in Evernote, I found that they provide some templates by default such as Meeting Note, Essay Outline, Meal Planner, Daily Reflection etc. I personally think that it would be great if we do it too in this plugin.

• Save as template - Currently, if a user wants to create a new template they have to open the templates directory and create a new markdown file and maybe then use a third party editor to write the template. The point is that there is no convenient way to do so. So, I’d like to add a save as template option both in the menu and as a toolbar icon.

• Toolbar icons - We can include two icons Insert Template and Save as template in the editor toolbar which I believe would be the best fit.

Basically, the plugin in the menu would look something like this.

Here, new note from template, new to-do from template, insert template would work in the same way as they currently do, the only difference being that there will be some pre-installed templates in the dropdown too.

The open templates directory will work in a similar way as it does now. It open a file manager with the templates directory.

The save as template option would save the current markdown content of the note after taking the name of the template as input from the user.

In the default template functionality the user will be able to set two default templates one for notes and one for to-dos. And as it can be seen in the picture there are two options: new note from default template and new to-do from default template. As the name suggests they would create a new note and to-do from their respective defaults. These options also have keyboard shortcuts which would make this feature a lot more usable.

Also, we can add two toolbar buttons that would look something like

The last two icons denote “Insert Template” and “Save as template”. In my opinion these two options would make the most sense in the editor toolbar.

Documentation

I’ll document the usage of the plugin in the README of the plugin repository. Also, I’ll try to use comments while coding wherever necessary to make the code easy to understand.

NOTE - This post will serve as a live specification for this project, I will be updating it regularly as I work on this project.

[nishantwrp]

Working Plan

Weeks 1-3

• Initialize the plugin repository.
• Make changes in the Joplin main app to add insert template functionality.
• Add insert template functionality in the plugin.
• Setup jest and unit testing in the plugin.
• Setup CI/CD using Github actions for unit tests and lint checks to ensure quality of the code is good.
• Make changes in the Joplin main add to allow opening template directory.
• Add Open Template directory functionality.
• Write unit tests for the plugin code written.
• Make weekly blog posts on the progress.

Weeks 4-5

• Add new note / to-do from template functionality.
• Add save as template functionality.
• Add toolbar buttons.
• Write unit tests for the plugin code written.
• Make weekly blog posts on the progress.

Weeks 6-7

• Add the default template functionality.
• Add some pre-installed templates.
• Write unit tests for the plugin code written.
• Document the usage of the plugin completely in the readme file.
• Make weekly blog posts on the progress.

Weeks 8-10

• Wrap up the plugin so that it can be used completely.
• Discuss any improvements / features that can be done further in the plugin.
• Remove the template functionality from the main Joplin desktop app safely so that it doesn’t break anything.
• Write relevant unit tests.
• Update documentation on the Joplin home page accordingly so that the community can know that the Template functionality is now replaced by a plugin.
• Make weekly blog posts on the progress.
• Make a final GSoC project report.

[nishantwrp]

reserved for future content 2

[nishantwrp]

reserved for future content 3

[nishantwrp]

@nishantwrp

The last two icons denote “Insert Template” and “Save as template”. In my opinion these two options would make the most sense in the editor toolbar.

I would like to ask the community about what they feel about what all toolbar icons should be present?

[Yann1ck]

I don’t think “save as template” is needed in the toolbar. A typical user will not create many templates or change them in a daily basis. So it’s not a tool used regular und just takes away space. (Besides that a Save-Icon could be misleading). From my point of view it can just be in the drop down menu.

[JackGruber]

I am already very excited about this plugin

If I times my small wish list

• It should be possible to use tags in the templates
• I would love it if the templates can also be saved as a note in a notebook and not only in the template directory, so that they are syncronized across multiple devices and I don't have to maintain the templates over and over again on each device.
• Variables and a Dialog for the variables when creating a note from the template
  • {{VARIABLENAME:TYPE:OPTIONS}}
  • Dropdown: {{room:dropdown:[Selection 1, Selection 2, Selection 3]}}
  • Select: {{dark:select:[Yes, No]}}
    ...
• Multiple use of variables in the document, but fill only once

[uxamanda]

You mentioned default templates, which is cool. I like the idea of being able to assign a keyboard shortcut to different templates so that I can use them as snippets. I think if they appear in the menu they could be assigned a shortcut? Not sure if that is out of scope of this.

@JackGruber

Variables and a Dialog for the variables when creating a note from the template

This is cool. I have a CLI tool I built to make daily notes that does this. Basically it's a form I fill out that creates a new note with the variables from the form that I add to throughout the day. Would be nice to have a built in solution!

[nishantwrp]

@uxamanda

You mentioned default templates, which is cool. I like the idea of being able to assign a keyboard shortcut to different templates so that I can use them as snippets. I think if they appear in the menu they could be assigned a shortcut? Not sure if that is out of scope of this.

Yup, the main idea was to have two default templates one for note and one for todo and assign a keyboard shortcut to each of them.

[nishantwrp]

@JackGruber

I would love it if the templates can also be saved as a note in a notebook and not only in the template directory, so that they are syncronized across multiple devices and I don't have to maintain the templates over and over again on each device.

I was thinking about this, and yes maybe this is possible. Instead of storing the templates in a directory, they can be stored in a separate notebook.

The way this can be implemented is

1. Register a new setting that stores the id of the notebook the templates are in.
2. If the notebook doesn't exist we create one and update the setting.
3. The notes in these notebooks would represent templates.
4. This way tags can also be added to templates.

This all can be done using Joplin Data API and settings Plugin API. What do you guys think about it?

/cc @JackGruber @CalebJohn

[rxliuli]

In my opinion, what you may need is Snippet instead of File Template. The former is to enter some preset keywords in the notes, and then generate some template text. The latter is to generate fixed content when creating a file (notes in Joplin). The main difference between the two is that the former is much more than the latter.

I would recommend to implement Snippet support, and then it would be better for users to add them easily, refer to:

• Snippets in Visual Studio Code
• javascript - How to customize Emmet (snippets.json) for codemirror? - Stack Overflow

[nishantwrp]

@JackGruber

Variables and a Dialog for the variables when creating a note from the template

• {{VARIABLENAME:TYPE:OPTIONS}}
• Dropdown: {{room:dropdown:[Selection 1, Selection 2, Selection 3]}}
• Select: {{dark:select:[Yes, No]}}

For this, here's an idea.

Initially, we allow two types of variables text and dropdown. The variables need to be first declared like for text {{ Variable Name: Text }} and for dropdown {{ Variable Name: dropdown: [ Selection1, Section 2, Selection 3 ] }}. And then the values can be simply used as {{ Variable Name }}.

Implementation

1. We first scan the text using regex and find all variables, their types. And remove the declarations from the text.
2. Display a popup for taking input from the user for the variables.
3. Use the mustache library, to replace all the variables names from the resultant text from step 1 with their values.
4. Render the final text.

The future scope of this project can also be then to use an advanced templating engine to allow boolean variables etc. and if, else logic within the template.

Thoughts?

[laurent]

Are there any existing templating engine that support this kind of syntax? I think it would worth investigating a bit what's being done so that we don't reinvent the wheel.

I feel mixing type definition with variable names makes for a heavy syntax and it means you need to redefine the type every time you use the variable in the document. For example, The {{room:dropdown:[Selection 1, Selection 2, Selection 3]}} is {{dark:select:[Yes, No]}} and the sky is {{dark:select:[Yes, No]}}.

I wonder if it should be separated - on one side the placeholders, on the other side the type definition. For example like this (using R Markdown header):

---
room: enum(Selection 1, Selection 2, Selection 3)
dark: boolean
---

The {{room}} is {{dark}} and the sky is {{dark}}

[nishantwrp]

@laurent

Are there any existing templating engine that support this kind of syntax? I think it would worth investigating a bit what's being done so that we don't reinvent the wheel.

Yes, makes sense. I'll investigate a bit on this and get back.

@laurent

I feel mixing type definition with variable names makes for a heavy syntax and it means you need to redefine the type every time you use the variable in the document. For example, The {{room:dropdown:[Selection 1, Selection 2, Selection 3]}} is {{dark:select:[Yes, No]}} and the sky is {{dark:select:[Yes, No]}} .

Actually, my idea was to separately declare variables something like

{{room:dropdown:[Selection 1, Selection 2, Selection 3]}}
{{dark:select:[Yes, No]}}

The {{ room }} is {{ dark ]}} and the sky is {{dark }}.

This is kind of similar to your R Markdown header idea but the syntax you suggested looks better.

---
room: enum(Selection 1, Selection 2, Selection 3)
dark: boolean
---

The {{room}} is {{dark}} and the sky is {{dark}}

[nishantwrp]

So, I did some research on templating systems and I couldn't find a templating system that lets to declare to declare the variables and their types in the template itself. So, I guess we'll have to do that ourselves. Here's an updated idea of the implementation.

We follow the template syntax @laurent suggested,

---
room: enum(Selection 1, Selection 2, Selection 3)
dark: boolean
---

The {{room}} is {{dark}} and the sky is {{dark}}

1. We use a library like markdown-it-front-matter and extract the YAML headers and use a regex like /---(.|\n)*---/ to strip off the yaml headers. Now, we have the variables and we can parse their values.

2. We display a dialog to collect the user input for the variables. And then render the template.

3. Also, I propose that we switch from mustache to handlebars because it's more powerful and it supports logic like if-else.

Thoughts?

[laurent]

@nishantwrp

• We use a library like markdown-it-front-matter and extract the YAML headers and use a regex like /---(.|\n)*---/ to strip off the yaml headers. Now, we have the variables and we can parse their values.

That makes sense. Is there any standard syntax you could use for the type definition? I kind of made that one up, and if you can it's best to use an existing one so that you don't need to needlessly develop a custom parser.

@nishantwrp

1. Also, I propose that we switch from mustache to handlebars because it's more powerful and it supports logic like if-else .

It's up to you. Handlebars is indeed a good library.

[Sento]

Hey,
This sounds great!
If I may suggest a feature I was hoping to see in the near future:

• Allow scripts inside the templates

Let me make a small example:
When researching papers I often find my sources on arxiv. To keep an overview about all the papers I've read and how they are linked to each other, I create a note based on a 'paper' template which has the title, fields for the author, abstract, link and the pdf file itself. At the moment, I always have to copy and paste the needed information although it should be relatively easy to extract them from arxiv directly, given the specific url.
Maybe allowing scripts in or around templates could allow for an autofill feature of parts of the templates by using Regular Expressions (or any other functionality to keep it general) on websites/files/any other source by only passing the url/path/.. to the template.

I'm not sure if this is too much to ask for in this plugin, maybe it could be separated into another plugin or maybe there is already a solution to this that I haven't found so far.

Anyway, good luck with the plugin and thank you for all your efforts!

[nishantwrp]

@Sento

Allow scripts inside the templates

Hmmm, I can't think of a good UX to implement this feature. Can you think of how this feature would look?

[nishantwrp]

@laurent

. Is there any standard syntax you could use for the type definition? I kind of made that one up, and if you can it's best to use an existing one so that you don't need to needlessly develop a custom parser.

Good idea. But I did some research and couldn't find any such parser that we can use in this case. We need a parser that could output the variables and type declarations in a string. I couldn't find something of that sort. So, I guess we can enforce our custom syntax for now. For the start I guess three types text, boolean and enum would be good enough. And the syntax that you made up looks good enough for that.

[Sento]

Mh, good question. Maybe it is possible to keep the script language inside the template .md file, like the declared variables.

Something like this with pseudo code:

---
url: text
parse_url: function(input1, ...)->text{
    return regex_magic(input1)
}
---

The abstract of the paper is {{parse_url(url)}}

I guess that could be one way to include functions in the template, how exactly it is implemented in the end is up to you. Of course, only if you think it is actually possible (and in your time frame).