New: Joplin Plugin Graph (knowledge graph visualization)

Yes, agree with zblesk. Current performance seems to be a major issue.

Joplin can barely start on my system, UI and Graph becoming unresponsive and not rendering.

As starting point, I would suggest following changes:

  • Make "Show selected nodes only" the default setting
  • Persist configured settings otherwise
  • Provide performance optimization options similar to the other graph plugin
    • Max nodes in graph: should be self-explanatory
    • Max degree of separation: Maximum number of link/edge jumps, starting at selected node

Strangely even with "Show selected nodes only", the graph most of the times does not render changes, when switching the active node (constant > 70% CPU). Tried raising COOLDOWN_TIME to 10sec - without effect.

1 Like

Sorry for the inconvenience! Please update to v1.0.5. I have added a fix that prevents the graph from refreshing when the graph is hidden, hence there is very little impact of the plugin on CPU usage when the graph panel is hidden.

For the CPU usage while using the graph, it is mostly bound to the underlying graph library. Animating graphs over canvas theoretically scales better than over svg, but it also means refreshing canvas pixels every frame if you want it to be dynamic. I will eventually do some experiments and go for optimizing low-hanging fruits but looking at simple examples from the library as a baseline, it won't be that significative of an improvement at the moment.
I do run Joplin on an old mac laptop (2014) and if I notice the fan at times, I have not experienced lag (except in tests with several thousands nodes and links) or unresponsiveness. Please open an issue with details about your hardware config / graph size if this is the case.

For your last question: SHIFT+CLICK all the nodes and check "Show selected nodes only" in the control panel should redraw only the selected nodes. You may need to zoom out to see them.
I do notice a problem at times when tinkering with multiple node selection in the note list. The ui seems one "selection event" behind. I will create an issue to track this down. Thanks for reporting!

1 Like
  • Provide performance optimization options similar to the other graph plugin
    • Max nodes in graph: should be self-explanatory
    • Max degree of separation: Maximum number of link/edge jumps, starting at selected node

I do have implemented those methods already. But both have their set of problems in terms of communicating info back to the user so I stay away from using them until I can figure out a good way to implement dynamically expanding graph regions.
Also the sheer model size has not really proven to be a bottleneck. It will be even less so after I switch to the /events api (soon). The ui will eventually lag on large graphs (probably larger than most users knowledge base though), so this may become more critical for some users but not most. For now filtering data should help.

Strangely even with "Show selected nodes only", the graph most of the times does not render changes, when switching the active node

Try zooming out. The graph underlying quadtree is not readily accessible so I use simple heuristics for zoom/pan but I will refine those later.

Tried raising COOLDOWN_TIME to 10sec - without effect.

Simulation COOLDOWN_TIME and COOLDOWN_TICK can be safely kept quite low (defaults should be plenty enough). For most use cases, the layout converges rapidly.

1 Like

If you run 1.0.6, please upgrade to 1.0.7.
1.0.6 was relying on the remark markdown parser.
After some (overdue) benchmarks, I realized it was 200x slower than evaluating regexps and way too slow for the current batch processing.

3 Likes

Thanks for the update, appreciate your time.

Yes, sample size for me were a couple thousand notes.

Yes, tried that. Later on I realized, that this is a pure performance issue, where the graph took too much time to render after selecting a different note (with "show selected nodes only").
It also seem to show the previous selection, not current one. Though it might be rather related to performance issue.

This looks very promising, looking forward to try that change out!

Mind updating your github code base to v1.0.7 as well? I have forked the repo to learn a bit more about joplin plugins in general and used graph libraries.

Mind updating your github code base to v1.0.7 as well? I have forked the repo to learn a bit more about joplin plugins in general and used graph libraries.

Done.

I had a few surprises myself when trying to figure this out. Actually the next release of the plugin will focus on rewriting the model management. The current approach must deal with costly workarounds and I have a couple ideas that should improve performance quite a bit.
Next I plan to look at what can be done to optimize the rendering. And also maybe finally take the time to get rid of some of the dependencies we talked about.

If you are so inclined, I am happy to welcome new contributors!

2 Likes

Thanks for your great work!

Is it possible to show tag nodes when 'only selected nodes' is checked? They only seem to show up when displaying all notes.

You should be able to. If have both only selected nodes and show tag nodes checked, make sure you are not filtering on other tags.

sharing experience with 1.0.7:

this definitely goes in the right direction performance-wise.

CPU idle before: 70-85%
CPU idle after: 30-40%

Graph takes 15-20sec to re-render after switching the active note (and sometimes not at all?).
So currently still not quite usable with bigger note count - but much better than the previous version, well done!

The graph in facts seems to show the previous note selection, not the current one.

The plugin definitely deserves contributors. But I probably need a bit more of learning time :slight_smile: . Let me try to help you as tester first.

Hello!
I have just published a complete overhaul of the plugin (release 1.1.0).
It is focused on improving performance and scalability.
You should see dramatic improvements there.
The UI style has also changed quite a bit.
It is still a work in progress and it may requires some tweaking to get what you want (check the options in settings and in the control menu) but I do think it is a significant step forward.
Enjoy and let me know what you think!

Screenshot from 2022-01-14 12-09-51

5 Likes

Looking forward to using this plugin!

I just tried 1.1.0 and in my case the plugin never manages to render the graph. I am using the default settings and I get an empty dark view and high CPU load.

A few other suggestions:

  • F8 to toggle the graph view on and off (similar to Link Graph UI).
  • What about other color schemes in addition to the current dark mode? Maybe this can be made configurable by users.

Looking forward to using this plugin!

I just tried 1.1.0 and in my case the plugin never manages to render the graph. I am using the default settings and I get an empty dark view and high CPU load.

I would need more information to provide any guidance on this. Maybe related to Unchecking SHOW_ON_START results in blank graph · Issue #14 · agerardin/joplin-plugin-knowledge-graph · GitHub? Please open an issue if you suspect a bug.

A few other suggestions:

  • F8 to toggle the graph view on and off (similar to Link Graph UI).

That should be doable.

  • What about other color schemes in addition to the current dark mode? Maybe this can be made configurable by users.

Joplin themes should affect the control menu. I don't anticipate any changes on the graph view until its implementation is more stable. Progress on this topic can be tracked here :

Had a chance to try out new version 1.1.0 today.
There still seems to be a problem of rendering the graph, when "Show selected nodes only" is enabled:

This is vanilla Joplin + plugin + demo notes + additional "note 1" note linked from the welcome note.

If setting is enabled, an empty graph is shown - just black background.

Desired result would be, that selected note ("Welcome ....") is shown together with the link to "note 1".
Zooming out does not help here. Or I may have misunderstood the setting.

Do you experience similar issues?

Hello @pacific_bagel,
With v1.1.0, selected nodes should appear in blue.
Nodes can be selected by opening a note in the editor or clicking on the graph.
The selection boundaries are sometimes off and require zooming out, which I shall try to correct for in the next release (time permitting, at the end of week).

2 Likes

Thanks. No hurry, take your time :-).

From my experience, the missing rendering of the graph with "Show selected nodes only" has nothing to do with need to zoom out/in. I almost broke my mouse wheel in hope to discover some nodes.

The latest version doesn't render a graph at all. :frowning: I run Joplin 2.6.10. When I click the Graph button in the toolbar, an empty pane opens (with no controls or anything). I've opened the dev console, there are no errors there. Anything else I could try?

Sorry for the delayed response @zblesk. Release v1.1.0 seems to create problems for many. I also upgraded to 2.6.10 and did not experience similar issues, so it is not clear what is going on. I plan is to release a new version in the next days to see it this helps. Time has been hard to come by lately.

5 Likes

We all have plenty things to do. I do not expect immediate responses. Take your time, and thanks. :slight_smile:

2 Likes

@agerardin in case you don't have time for the plugin, could you maybe give me a few pointers as to how could I go about debugging the issue myself?
I haven't worked on a Joplin plugin yet, so I don't know.

So far I have created a separate Joplin instance with only this plugin installed. It starts out okay, but when I import my notes, it no longer renders anything.
When I do things like change notes, or click the refresh arrow, it doesn't log anything into the dev console. (When the app starts, it reports that the plugin was loaded, but I've found nothing else regardless of my actions.)

2 Likes

I'm just writing to know if there is a plan for update for this plugin. I still like it, but am in the same way still a bit annoyed by the size of the text. I have tons of notes (1000+) and when I zoom, the text get so small that I can't read it.

1 Like