Pre-release 1.3 with plugin support is now available! (Updated 31/10/20)

wandern · 2020-10-29T17:43:55.996Z

I've run into various types of errors with 1.3.10 while importing various .enex files. Here are three examples. Most of them have been encountered more than once.

TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string or an instance of Buffer or URL. Received undefined
TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string or an instance of Buffer or URL. Received undefined
    at Object.stat (node:fs:1061:10)
    at Object.stat (/Users/someuser/.joplin-bin/lib/node_modules/joplin/node_modules/graceful-fs/polyfills.js:298:16)
    at isSrcSubdir (/Users/someuser/.joplin-bin/lib/node_modules/joplin/node_modules/fs-extra/lib/move/index.js:153:6)
    at Object.move (/Users/someuser/.joplin-bin/lib/node_modules/joplin/node_modules/fs-extra/lib/move/index.js:24:3)
    at /Users/someuser/.joplin-bin/lib/node_modules/joplin/node_modules/universalify/index.js:13:12
    at new Promise (<anonymous>)
    at Object.move (/Users/someuser/.joplin-bin/lib/node_modules/joplin/node_modules/universalify/index.js:7:14)
    at saveNoteResources (/Users/someuser/.joplin-bin/lib/node_modules/joplin/lib/import-enex.js:183:12)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:93:5) {
  code: 'ERR_INVALID_ARG_TYPE'
}

...

[Error: ENOENT: no such file or directory, stat '/Users/someuser/.config/joplin/tmp/717f4e9cfd5d88b3fd8c040ccde840de.base64.decoded'] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'stat',
  path: '/Users/someuser/.config/joplin/tmp/717f4e9cfd5d88b3fd8c040ccde840de.base64.decoded'
}

...

Error: EISDIR: illegal operation on a directory, read

wandern · 2020-10-29T17:47:28.931Z

Regarding atomic database operations, etc. This is what happens to an import process when a 'joplin sync' is triggered in the background.

It's unclear whether the "ERROR" is actually an error, or if the import process is retrying, or if a DELETE is actually failing (and potentially leaving something inconsistent):

Error: Error: SQLITE_BUSY: database is locked: INSERT INTO `note_tags` (`note_id`, `tag_id`, `updated_time`, `user_updated_time`, `id`, `created_time`, `user_created_time`) VALUES (?, ?, ?, ?, ?, ?, ?): 65982470864b43d1ab2e0d034f44e649,864e84baa5ef4dfca7d47a86035eae3d,1603993373023,1603993373023,5e5e6d57862c4cd193fa965ac820129a,1603993373023,1603993373023
    at DatabaseDriverNode.sqliteErrorToJsError (/Users/someuser/.joplin-bin/lib/node_modules/joplin/lib/database-driver-node.js:21:18)
    at JoplinDatabase.sqliteErrorToJsError (/Users/someuser/.joplin-bin/lib/node_modules/joplin/lib/database.js:25:24)
    at JoplinDatabase.tryCall (/Users/someuser/.joplin-bin/lib/node_modules/joplin/lib/database.js:104:45)
    at processTicksAndRejections (node:internal/process/task_queues:93:5)
    at async JoplinDatabase.transactionExecBatch (/Users/someuser/.joplin-bin/lib/node_modules/joplin/lib/database.js:161:4)
    at async Function.save (/Users/someuser/.joplin-bin/lib/node_modules/joplin/lib/BaseModel.js:503:4)
    at async Function.addNote (/Users/someuser/.joplin-bin/lib/node_modules/joplin/lib/models/Tag.js:64:18)
    at async saveNoteTags (/Users/someuser/.joplin-bin/lib/node_modules/joplin/lib/import-enex.js:198:3)
    at async saveNoteToStorage (/Users/someuser/.joplin-bin/lib/node_modules/joplin/lib/import-enex.js:225:22)
    at async processNotes (/Users/someuser/.joplin-bin/lib/node_modules/joplin/lib/import-enex.js:345:21) {
  code: 'SQLITE_BUSY'
}
/Users/someuser/.joplin-bin/lib/node_modules/joplin/lib/database-driver-node.js:21
        const output = new Error(msg.join(': '));
                       ^

Error: Error: SQLITE_BUSY: database is locked: DELETE FROM item_changes WHERE item_id = ?: 9896f891caef4ecc9a932985d0b191ee
    at DatabaseDriverNode.sqliteErrorToJsError (/Users/someuser/.joplin-bin/lib/node_modules/joplin/lib/database-driver-node.js:21:18)
    at JoplinDatabase.sqliteErrorToJsError (/Users/someuser/.joplin-bin/lib/node_modules/joplin/lib/database.js:25:24)
    at JoplinDatabase.tryCall (/Users/someuser/.joplin-bin/lib/node_modules/joplin/lib/database.js:104:45)
    at processTicksAndRejections (node:internal/process/task_queues:93:5)
    at async JoplinDatabase.transactionExecBatch (/Users/someuser/.joplin-bin/lib/node_modules/joplin/lib/database.js:173:5)
    at async Function.add (/Users/someuser/.joplin-bin/lib/node_modules/joplin/lib/models/ItemChange.js:25:4) {
  code: 'SQLITE_BUSY'
}

laurent · 2020-10-29T18:51:08.176Z

wandern:

I've run into various types of errors with 1.3.10 while importing various .enex files. Here are three examples. Most of them have been encountered more than once.

Please provide an ENEX file to replicate the issue.

wandern · 2020-10-29T19:20:15.298Z

laurent:

Please provide an ENEX file to replicate the issue.

I plan on doing that, but since I have personal information in the large .enex files, I'm trying to determine the specific notes that are triggering the issue, allowing me to create an .enex containing only those notes.

However, the information on debugging the command line app here: joplin/readme/debugging.md at master · laurent22/joplin · GitHub is not working as I would expect.

joplin --debug --log-level debug import filename.enex

is not adding any additional debug information in $HOME/.config/joplin/log.txt

I can't find anything in log-database.txt that lets me correlate the error to a specific note. Do you have any suggestions?

Thanks!

laurent · 2020-10-29T22:18:18.894Z

wandern:

is not adding any additional debug information in $HOME/.config/joplin/log.txt

The error you've posted above is already good enough, but it's even better if I can replicate with an ENEX, then it's much easier to find a fix. So indeed if you can narrow it down to a few notes that you can share that would be perfect. You can also send them by PM if you prefer.

tessus · 2020-10-30T02:08:36.431Z

Some combinations of modifier keys can't be assigned via the UI keyboard shortcut editor:

e.g.:

Ctrl+Option+Shift+Cmd+[A-Z]
Ctrl+Option+Cmd+[A-Z]
Option+Shift+[A-Z]
Option+Cmd+[A-Z]
Option+Shift+Cmd+[A-Z]
Ctrl+Option+[A-Z]

You can use them in the json file, but not via the UI.

tessus · 2020-10-30T18:55:57.552Z

@laurent shall I open a gh issue for this ^^^ or is above post enough?

laurent · 2020-10-30T19:00:04.103Z

I don't know how to fix this to be honest as I don't understand why the Option key wouldn't work. Perhaps @anjulalk would know something?

anjulalk · 2020-10-30T19:25:25.435Z

tessus:

Some combinations of modifier keys can't be assigned via the UI keyboard shortcut editor:

e.g.:
Ctrl+Option+Shift+Cmd+[A-Z]
Ctrl+Option+Cmd+[A-Z]
Option+Shift+[A-Z]
Option+Cmd+[A-Z]
Option+Shift+Cmd+[A-Z]
Ctrl+Option+[A-Z]
You can use them in the json file, but not via the UI.

This is very strange. I have thoroughly tested the editor in macOS, and it worked fine. However it was a virtualized instance, and I used the accessibility keyboard for sending key presses as I don't have access to a Mac.

Let me look into this issue. I don't know what's causing the issue at the moment, but I'll try to give a fix as soon as possible.

laurent · 2020-10-30T22:48:21.740Z

No I think that's reasonable, I thought about something like this too, and the underlying component is made with that in mind. The layout in particular can be saved/loaded from JSON if needed.

The problem indeed is how to create a UI that can be developed relatively easily. Drag and drop is often messy to implement so I'd favour a solution without it but not sure what.

laurent · 2020-10-31T13:22:58.675Z

Version 1.3.11 is now available:

Improved: Make sure all commands appear in keymap editor
Fixed: Add history backward and forward commands to keymap and menus (#4010)
Fixed: Fixed handling of Option key for shortcuts in macOS
Security: Remove "link" and "meta" tags from notes to prevent XSS

GitHub

Release v1.3.11 · laurent22/joplin

Improved: Make sure all commands appear in keymap editor Fixed: Add history backward and forward commands to keymap and menus (#4010) Fixed: Fixed handling of Option key for shortcuts in macOS Secu...

benji300 · 2020-10-31T13:23:24.484Z

That would be great. I didn't think you thought so, too.
I understand the drag and drop problem.

Initially, you could let users edit the layout directly in the JSON file. So Joplin would read this "layout-desktop.json" only once at startup. This is not for "standard" users, but if they can currently install a plugin, they should also be able to edit the JSON file.

Later it might be possible to implement an UI (whatever that might look like) in the options view to define the layout graphically. From which the JSON file is written then.

laurent · 2020-10-31T13:23:30.601Z

dpoulton · 2020-10-31T15:48:11.479Z

laurent:

Version 1.3.11 is now available:

A little bit of beta-testing

Windows 10 Pro x64
Joplin 1.3.11 Portable

Placing a plugin in a plugins folder in the profile directory (\plugins\SideBarToggle.js) works.
Moving the plugin to a sub-directory of the plugins folder in the profile directory (\plugins\SideBarToggle\SideBarToggle.js) does not.
Previous post indicates that the plugin in the sub-directory should be loaded.
Joplin was restarted after every change.
Keyboard shortcuts settings page appears to have two entries for rename (Rename Folder & Rename Tag?)

laurent · 2020-10-31T16:38:29.869Z

dpoulton:

Placing a plugin in a plugins folder in the profile directory ( \plugins\SideBarToggle.js ) works.
Moving the plugin to a sub-directory of the plugins folder in the profile directory ( \plugins\SideBarToggle\SideBarToggle.js ) does not.
Previous post indicates that the plugin in the sub-directory should be loaded.
Joplin was restarted after every change.

Right, the doc wasn't clear. If it's in a sub-directory, Joplin expects a certain structure, as built by the yo plugin command, but if it's a single JS file it can be name anything. So the rule is actually:

The plugin service will look at the following locations:

plugins/(PLUGIN_ID).js

plugins/(PLUGIN_ID)/index.js

plugins/(PLUGIN_ID)/dist/index.js

Any directory or file that starts with "_" will be excluded (convenient to quickly disable a plugin)

PLUGIN_ID can be any string but it must be unique.

laurent · 2020-10-31T16:45:09.416Z

dpoulton:

Keyboard shortcuts settings page appears to have two entries for rename (Rename Folder & Rename Tag?)

Thanks, for now these two commands will have to be removed anyway because they don't appear in the main menu (only commands in that menu can have a shortcut associated with them).

tessus · 2020-10-31T17:13:54.834Z

Unfortunately the fix (using the KeyCode) had a side effect, which makes it even less usable now.

If you press

option+cmd the following appears in the field: Option+Cmd+[
Cmd+Left (for the history back operation), the following appears in the field: Cmd+%
The are not only shown like this, they are registered a such. So Cmd+Left wouldn't do anything. To use the shortvut to go back I have to press Cmd+Shift+5 (shift 5 is % on the US keyboard)

Some other keys can't be assigned anymore. e.g. the backtick.

I'm trying to debug this, but there's no debug output in that code, so it's hard....

tessus · 2020-10-31T18:57:13.017Z

tessus:

I'm trying to debug this,

I think I found a way to make this work. The problem is that fromCharCode returns incorrect characters.
I know how to fix this, but it will take a while to get it right. I have a few other things to do this weekend, but I should be able to get it done.

Here's the fix:

github.com/laurent22/joplin

Desktop: fix issues with shortcut editor

laurent22:dev ← tessus:fix/shortcut-editor

opened 03:18AM - 02 Nov 20 UTC

tessus

+270 -34

String.fromCharCode expects unicode charcodes as an argument; e.keyCode returns …javascript keycodes. Javascript keycodes and unicode charcodes are not the same thing! Therefore I used an array to map keycodes. This seems to be the only way to make this work properly. I also changed the width of the shortcut recorder input field to 200px, since certain key combinations are cut off, because they don't fit in the field.

laurent · 2020-11-04T12:24:01.944Z

Version 1.3.15 is now available. I expect this will be the final release for 1.3 although there's still one issue with the keymap editor on macOS but it's difficult to fix so will be in 1.4

Improved: Remove from keymap editor commands that cannot be associated with a shortcut
Regression: Reverted keymap Option fix as it was making problem worse

GitHub

Release v1.3.15 · laurent22/joplin

Improved: Remove from keymap editor commands that cannot be associated with a shortcut Regression: Reverted keymap Option fix as it was making problem worse New: Add support for application plug...

laurent · 2020-11-06T12:53:34.869Z