Performance issue

[Steph]

TEST 1
Computer 1 with Windows 11, Joplin 3.0.2

1. I export all into a jex file
2. I create a new profile
3. I import the previous jex file into this new profile => I have to try many times before it works… ( I have many error message from Joplin, like "out of delay").
   When the importation is done, in fact everything is so slow that it's absolutely impossible to use ! 10 to 12 seconds after a click to open a notebook or a note...

TEST 2
Same computer (Windows 11, Joplin 3.0.2)
I want to install the previous version of Joplin (2.14.20): IMPOSSIBLE (only error messages)
So : no turning back possible after an upgrade.

TEST 3
I have two identical laptops.
On the second one I have a 2.12. and I have never install the 3.0.2.

1. I install Joplin 2.14.20 (latest version of the 2 branch)
2. I import the jex file that I did on the other Joplin on the other computer => it works "immediatly" (but one hour and a half needed = normal = 2 Go of datas !). I mean no stop and no error messages from JOPLIN,
3. It works quickly ! I can go back to work.

So there was 2 issues with the 3.0.2 under windows 11 for me => delay and exportation-importation. The trash works fine, Like I created many profiles just for the tests on the first computer, I noticed that's it's also too difficult (delay) to delete the content because of delay and error messages.

To be followed…

I will try

1. an 2.14.20 update to 3.0.2 on a third computer after the importation of the same all.jex -
2. an upgrade of Joplin on Kubuntu...

[personalizedrefriger]

I'm able to reproduce a performance issue that, based on the descriptions above, seems similar (in Joplin 3.0.2 with about 9000 tags, 2000 notebooks (many nested), and 13000 notes).

Note/resource/tag/notebook counts:

Note: 0/13785
Folder: 0/1889
Resource: 0/15
Tag: 0/9524
NoteTag: 0/8997
Revision: 0/68
Total: 0/34278

Joplin version information:

Joplin 3.0.2 (dev, linux)

Client ID: 3a953c01b7df45eab0fcf3b3410b7451
Sync Version: 3
Profile Version: 46
Keychain Supported: No

Revision: b3f441402

Backup: 1.4.1

Time to create and tag a note from the console

When I run the following in Joplin's development tools console with the notes list, notebook list, and tag list open,

(async () => {
    console.time('createNote');
    const note = await Note.save({ title: 'Test...', parent_id: '08e57e090e11466d833925c8e8c4f062' });
    console.time('tagNote');
    console.timeEnd('createNote');
    const tag = await Tag.save({ title: 'tag-new-note' });
    await Tag.addNote(tag.id, note.id);
    console.timeEnd('tagNote');
})();

I get the following output:

// Console output:
-> createNote: 55.760009765625 ms
-> 10:04:57: Scheduling sync operation... 15000
-> 10:05:03: App: Refreshing notes: 2 88c0d524c4a8467eac36e0de2493352a
-> tagNote: 8669.56884765625 ms

Running it again with the notebook list and note list closed (closed by pressing F10, F11):

-> createNote: 45.0830078125 ms
-> 10:07:07: Scheduling sync operation... 15000
-> 10:07:13: App: Refreshing notes: 2 88c0d524c4a8467eac36e0de2493352a
-> tagNote: 9182.297119140625 ms

With settings open:

-> createNote: 39.320068359375 ms
-> 10:08:45: Scheduling sync operation... 15000
-> 10:08:46: App: Refreshing notes: 2 88c0d524c4a8467eac36e0de2493352a
-> tagNote: 143.031982421875 ms

Notebook switching time

When I run the following from Joplin's development tools,

(async () => {
let avgTime = 0;
const trialCount = 5;
for (let i = 0; i < trialCount; i++) {
    await joplin.commandService.execute('openItem', ':/ec609ce22d0645afad68b3e261f516e5');
    const startTime = performance.now();
    await joplin.commandService.execute('openItem', ':/227759c8f2134dadbf62365db5e7571e');
    const deltaTime = performance.now() - startTime;
    avgTime += deltaTime;
}
avgTime /= trialCount;
console.log('Average switch notebook time', avgTime);
})()

In addition to a number of other logs, I get,

Average switch notebook time 2727.820000004768

Full log

CommandService::execute: openItem :/227759c8f2134dadbf62365db5e7571e
main-html.js:54 10:15:10: App: Refreshing notes: 2 08e57e090e11466d833925c8e8c4f062
main-html.js:54 10:15:13: Loading existing note 227759c8f2134dadbf62365db5e7571e
main-html.js:54 10:15:13: CommandService::execute: openItem :/ec609ce22d0645afad68b3e261f516e5
main-html.js:54 10:15:13: Loaded note: {id: '227759c8f2134dadbf62365db5e7571e', parent_id: 'b9671d8f2e6b42e983e331f9ccf27b5c', title: 'Test note 9', body: 'Test note 9', created_time: 1712589885051, …}
main-html.js:54 10:15:15: CommandService::execute: openItem :/227759c8f2134dadbf62365db5e7571e
main-html.js:54 10:15:15: App: Refreshing notes: 2 b9671d8f2e6b42e983e331f9ccf27b5c
main-html.js:54 10:15:15: Loading existing note ec609ce22d0645afad68b3e261f516e5
main-html.js:54 10:15:15: App: Refreshing notes: 2 08e57e090e11466d833925c8e8c4f062
main-html.js:54 10:15:18: CommandService::execute: openItem :/ec609ce22d0645afad68b3e261f516e5
main-html.js:54 10:15:18: Loading existing note ec609ce22d0645afad68b3e261f516e5
main-html.js:54 10:15:21: CommandService::execute: openItem :/227759c8f2134dadbf62365db5e7571e
main-html.js:54 10:15:21: App: Refreshing notes: 2 b9671d8f2e6b42e983e331f9ccf27b5c
main-html.js:54 10:15:21: PluginRunner: Got message (3): joplin.settings.value backupInterval
main-html.js:54 10:15:23: CommandService::execute: openItem :/ec609ce22d0645afad68b3e261f516e5
main-html.js:54 10:15:23: App: Refreshing notes: 2 08e57e090e11466d833925c8e8c4f062
main-html.js:54 10:15:23: PluginRunner: Got message (3): joplin.settings.value lastBackup
main-html.js:54 Got ipc-message: noteRenderComplete undefined
main-html.js:54 10:15:23: PluginRunner: Got message (3): joplin.settings.value onlyOnChange
main-html.js:54 10:15:23: App: Refreshing notes: 2 b9671d8f2e6b42e983e331f9ccf27b5c
main-html.js:54 10:15:26: Loading existing note ec609ce22d0645afad68b3e261f516e5
main-html.js:54 10:15:26: CommandService::execute: openItem :/227759c8f2134dadbf62365db5e7571e
main-html.js:54 10:15:26: PluginRunner: Got message (3): joplin.data.get ['folders'] {fields: 'title, id, updated_time', order_by: 'updated_time', order_dir: 'DESC', limit: 10, page: 1}
main-html.js:54 Got ipc-message: noteRenderComplete undefined
main-html.js:54 10:15:26: App: Refreshing notes: 2 08e57e090e11466d833925c8e8c4f062
main-html.js:54 10:15:29: CommandService::execute: openItem :/ec609ce22d0645afad68b3e261f516e5
main-html.js:54 10:15:29: PluginRunner: Got message (3): joplin.data.get ['notes'] {fields: 'title, id, updated_time', order_by: 'updated_time', order_dir: 'DESC', limit: 10, page: 1}
main-html.js:54 10:15:29: App: Refreshing notes: 2 b9671d8f2e6b42e983e331f9ccf27b5c
main-html.js:54 10:15:29: Loading existing note ec609ce22d0645afad68b3e261f516e5
main-html.js:54 10:15:32: CommandService::execute: openItem :/227759c8f2134dadbf62365db5e7571e
main-html.js:54 10:15:32: PluginRunner: Got message (3): joplin.data.get ['resources'] {fields: 'title, id, updated_time', order_by: 'updated_time', order_dir: 'DESC', limit: 10, page: 1}
10:15:32: App: Refreshing notes: 2 08e57e090e11466d833925c8e8c4f062

JEX Export of the data I'm testing with

big.jex.gz (5.4 MB)

(Partially created with Stress testing/Python script: Generating a large number of notes and tags)

Because adding/updating notes is much faster when settings is open, I suspect that part of the performance issue is some component in the main Joplin screen that updates for every note/notebook.

I'm also experiencing a .5-2 second freeze when creating a new note (after starting to type the title).

Note that an import of the relevant .JEX file in another 3.0.2 copy of Joplin and a 2.14.20 version of Joplin are in progress, so I'm unsure whether the performance issues listed above are regressions or not.

• Edit: With version 2.14.20 (release), importing finished faster than with version 3.0.2 (dev) (edit 2 so far, more than 7 minutes faster).
• Edit: I'm also not noticing the freeze when updating the notebook title in v2.14.20 (release).

Edit 3: Similar GitHub issue.

@HarSel

have a small notebook of perhaps 6 notes at most (out of an entire collection of 2400 notes) that I use to track incoming purchases. When I receive notice that an item has shipped I merely stick a dash in front of the title so it moves to the top of the list (sorted by title.) Under 2.13 and previous versions the note list updated too fast to measure but as soon as I got onto 2.14 the time for the note list to update jumped to around 2 seconds.

I'm able to reproduce this in a new profile with two notes and one notebook (edit: in Joplin 3.0.2 (dev) on Ubuntu 23.10). Edit 2: GitHub issue.

[personalizedrefriger]

Related pull request:

Desktop: Resolves #4251: Refactor sidebar to better handle thousands tags and notebooks