Kanban Plugin Weekly Report

It's because, the way I have it now, there's no need for a store, which redux would provide. The frontend dispatches an action, like "move note", to the backend, which then chooses some data api query(ies) (via a reducer-like function, but maybe it's more like a map, because it maps actions to queries) and executes them through the plugin api. Finally it runs the search, created from the column sorting rules, and returns the updated sorted notes to the frontend.

Like this, the only state the plugin keeps is the parsed configuration, but that's assumed to be static (if it changes the board would be removed and recreated), so I think there's no need for a state management library.

Thanks for clarifying and yes it looks like Redux wouldn't be needed in that case. It also makes sense to keep the app data as the single source of truth, as you've done, as that should make your code simpler and more reliable.

1 Like

Week 3

June 21. - 27.

Progress

This week was mostly about university exams for me, so I apologize for not being able to show much, but I've still managed to make some progress. Most notably, here's the first screenshot of an acutal UI (no mockup)!

I was planning on releasing a working test version with this progress update, that's why it got delayed to Monday, but unfortunately a roadblock came up which I haven't managed to resolve.

The way the plugin is constructed, the state of the board (which column each note belongs to) is not kept by the plugin. It fetches the data directly from Joplin, using its native search feature. When a note is dragged between columns, the plugin first updates Joplin's database, then fetches the board state again.

This is really neat, because it keeps the plugin code simple, but there's one major issue: getting the updated board state after a drag is asynchronous, because it involves sending a query to the main Joplin process and waiting for a reply. Of course, this happens very fast, for a user the delay is not noticeable. The problem is that React, the framework I use for the UI, doesn't really know what to render in this short time. If I tell it to render nothing, the UI would flicker. If I don't tell it what to render, it would render the previous last known state, so the dragged note would just snap back for a split second.

Ideally, I'd like to tell it to hold on, and keep the dragged card "floating" in place just for a few more milliseconds, while the state is updated in the background. Sadly, even though I've been trying for the past 1,5 days, I just couldn't make it work with the drag and drop library (react-beautiful-dnd) I was planning to use (which is a really great lib, would be awesome if I could get it to work).

I will keep trying, I'm thinking about switching to a different, lower-level drag&drop library, which hopefully would be more flexible with async updates. If all else fails, I can replicate the board state in the plugin, though I'd prefer to avoid this.

Achieved

  • Got a basic UI (drag&drop not working yet)
  • Wrote full test suite for backend
  • Smaller fixes to backend

Planned for next week

  • Fix drag&drop
  • Release first working test version
  • Fix recursive notebook search
  • (if there's time) Get started on the config UI

Thanks for following along, as always feedback is appreciated!

7 Likes

:tada: :raised_hand: :rocket: So excited! It is so cool to see the first version. Congrats on juggling all of this with your exams!

As for the drag and drop itself, had a random idea – can you duplicate the card in the meantime, so the user is dragging a static version temporarily and the ones in the columns are the current state? Seems tricky to get the logic right, not sure it helps.

1 Like

I got the drag&drop working! What I had to do in the end was to duplicate the two columns involved and move the note on the cloned columns. Not as nice and simple as I had imagined but it works!

dragopt

7 Likes

Week 4

June 28. - July 4.

Progress

This week I've been making incremental improvements to the plugin in order to release a test version along with the progress report. So without further ado, here it is, the first test (super-early, super-buggy) release of the kanban plugin :partying_face: :
com.mablin7.test.kanban.jpl (1.6 MB)

(Please read further for notes for testers)

Most significant of the changes I've made this week, is that I finally gave up trying to use the Joplin search to sort notes into columns. I thought it's clever, but it created more problems than it solved, so I ended up writing my own sorting algorithm. It fetches all viable notes, then matches them against the rules defined in the config. This gives way more control over how the rules can work and also solves the 2 roadblocks mentioned in the Week 2 report:

Both of these are now solvable, though the completedTag and completedNotebook rules are not yet implemented.

I've also improved the UI, bringing it closer to my mockup, but not quite there yet. The board opening should also work more reliably now, and some basic error messages are also shown, if the config is invalid.

Achieved

  • Refactored the sorting logic (again), squished multiple bugs
  • Improved visuals
  • Basic error messages
  • Improved board opening

Planned for next week

From now on, I'm hoping to release a new version every week!

  • Better error messages
  • Automatic reloading
  • completedTag and completedNotebook rules
  • More forgiving YAML parsing

Notes for testers

First of all, thank you if you're willing to give this plugin a try, user testing truly helps a lot in development.

  1. Though the plugin doesn't touch the contents of your notes, it does reorganize them, and if there are unexpected bugs it can certainly mess up your collection. Make sure to have a backup of your notes before trying out the plugin!
  2. You're going to need a board config. To start, try out this one, I've been using for testing, but please do not be afraid to experiment with it, since that's the point of user testing:
columns: 
  - 
    backlog: true
    name: Backlog
  - 
    name: "Ready for review"
    tag: ready
  - 
    name: Working
    notebookPath: working
  - 
    completed: true
    tag: done
    name: Done
filters: 
  rootNotebookPath: "test/nested test"
  tag: task
  1. The config parsing is currently quite fragile, for example tabs are not allowed for indentation. If you're getting a YAML parse error, try this online validator, it should give you more details than the plugin. I am working on more informative error messages and making the parser more forgiving.
  2. The board currently only gets updated if you reopen it. This means that if you changed the config, or added/edited the notes on the board, you need to open an unrelated note to close the board, the open it again to reload. The next release will surely include a better solution.
  3. If you find any bugs, or have any suggestions, post on the main thread please.
  4. Enjoy!
3 Likes

There was a bug in the test release, which prevented the board from opening. Please use this fixed version:
com.mablin7.test.kanban.jpl (1.6 MB)

1 Like

Hi @mablin7

Thanks for that. I try to do a quick test but the tasks do not appear (I took exactly your test configuration). In the "nested test" folder you insert tasks with a "ready" tag for example? And that's all. Is this right?

Hey all! Here's another hotfix which allows not to specify rootNotebookPath
com.mablin7.test.kanban.jpl (1.6 MB)

@bepolymathe Sorry, I wrote the progress report in a hurry and forgot to actually include usage instructions. So here it is:

Installation

  1. Download the .jpl file
  2. Open Joplin > Tools > Options > Plugins > Gear icon > Install from file
  3. Select the file you downloaded
  4. Restart Joplin

Usage

  1. Create a new note anywhere. This will be the "board note"
  2. Paste this basic config into your board note
```kanban
columns: 
  - 
    name: Backlog    
    backlog: true
  - 
    name: Working
    tag: wip
  - 
    name: Done
    completed: true
    tag: done
```​
  1. Switch to another note and switch back, or restart Joplin. (this will be improved soon!)
  2. Now you should see the board open on the right side with all your notes, which were in the same notebook as the board.
  3. Drag the notes around and see their tags update.

You can also try notebook-based columns instead of tag based ones. Replace the config with this:

```kanban
columns: 
  - 
    name: Backlog    
    backlog: true
  - 
    name: Working
    notebookPath: wip
  - 
    name: Done
    completed: true
    notebookPath: done
```​

Make sure that the notebooks you specify here exists in the same notebook where the board is!

Example notebook structure:

📁 board
├─ 🗒️ board note
├─ 🗒️ backlog notes/tasks...
├─ 📁 wip
│ ├─ 🗒️ wip notes/tasks...
├─ 📁 done
│ ├─ 🗒️ done notes/tasks...
2 Likes

Thank you for this clarification. After a first try following your instructions, it seems that the tags are not updated if I move a note or a task.

Hm, that's bad. If it's not private, could you send me a screenshot of your setup? I just want to see your board note, and the notebook where it's in.

Also, could you try enabling debug mode, as described here:
https://joplinapp.org/debugging/#desktop-application

and send me the logs you see? You should see two consoled pop up, one in the main joplin window and one in a separate window. I need logs from both

Yes of course.

That's strange... Could you also share the logs?

Fun to play with! The drag and drop feels nice and responsive. I know you are not done, so my apologies if some of my "bugs" are just TBD!

I tested this on Mac, Joplin 2.2.1 (prod, darwin).

Some things I noticed with the Tag-based Example:

  • the board wouldn't load notes if the name of the notebook has a space in it (e.g. notebook Testing Kanban)
  • the first time i moved a note to the "working" column it went back to the backlog without "sticking". This seems to be because I didn't have the WIP tag created yet. once i moved it again, it worked properly adding and removing the tag
  • have to switch to a new notebook (not just a new note) to get the board to re-render (because clicking notes in the same notebook keeps the board on the screen I think)
  • sometimes moving a note caused another note to move. for example, in this case I move the note called "test" to the working column and the note "test note" moved too. This doesnt seem to happen every time, and as you can see "test note" is still marked as done. They are only semi-linked and don't always do the same thing when I move them, maybe a bug because they are named similarly?
  • moving a note (not a todo) to done moves it visually, but no tag is added, so it doesnt "stick" when you reload
  • errors i see in the console (could be from other issues)
Warning: Can't perform a React state update on an unmounted component. This is a no-op, but it indicates a memory leak in your application. To fix, cancel all subscriptions and asynchronous tasks in a useEffect cleanup function.
    in NoteEditor (created by Connect(NoteEditor))
    in Connect(NoteEditor) (created by ResizableLayout)
    in div (created by styled.div)
    in styled.div (created by ResizableLayout)
    in div (created by Resizable)
    in Resizable (created by ResizableLayout)
UserWebviewIndex.html:1 Uncaught (in promise) Error: Not Found
    at Object.<anonymous> (/Applications/Joplin.app/Contents/Resources/app.asar/node_modules/@joplin/lib/services/rest/routes/tags.js:25)
    at Generator.next (<anonymous>)
    at fulfilled (/Applications/Joplin.app/Contents/Resources/app.asar/node_modules/@joplin/lib/services/rest/routes/tags.js:5)
  • Testing the notebook path example didn't seem to work for me. Only the backlog notes loaded, and moving notes into a different column just disappeared them from the board (although it did move them to the correct folder.)
    Screen Shot 2021-07-10 at 15.07.59
  • For the tag based example, the notes with the tag "WIP" needed to be in the same folder as the board note to appear, notes with that tag in different folders were ignored.

FYI, I only see the dev console when I run in debug, not sure where to see the separate window.

1 Like

You need to use dev version of the plugin, i.e. check out the sources, run npm i from there, and point Joplin to that dir. And if you have previously installed the plugin from .jpl, you'll need to remove it first.

Thank you for the detailed feedback!

Yeah these seem like serious bugs, I'll look into them ASAP.

Yes, notes in the same notebook should keep the board open, but I'm working on the auto reload.

This will be solved when I add support for the completedTag rule, so very soon.

I guess it's related to the WIP tag not existing. For some reason I thought tags are created automatically by Joplin if they don't exist yet, but it seems that's not the case. I'll add the check for that.

That's the intended behavior, unless they're in sub-notebooks of the board notebook, in which case it's a bug.

I figured most people would have one notebook for the board, and would want to leave all other notes out. But I could surely add an option to include all notebooks. What do you think?

I fixed all the bugs you discovered except this one, because I couldn't reproduce it. Did you notice maybe, when this happens, do both notes move smoothly, or does one jump there a split second later?

Yea, I like the idea of having a board that is generated purely from tags and doesn't care what notebook it is in. That allows me to keep project notes in their notebooks, but see all my "work" across a bunch of projects. In that case I probably would want my backlog to be all notes in a general Inbox notebook, but I am less concerned with that.

Hmm, seems to depend? As you can see it isn't consistent.

Tried it yesterday, very nice. Great work!

I could not reproduce the issue that @uxamanda mentioned, but at some point I grabbed a task and started dragging it back and forth and after a few seconds my whole system froze.
However, I think this is not something specific to your plugin and more of a general stability issue on Linux.