Importing markdown .md files with front matter throws errors

Operating system

Windows

Joplin version

3.6.15

Desktop version info

Joplin 3.6.15 (prod, win32)

Device: win32, Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
Client ID: f56ced2ebb8e4e58a59824aab101be67
Sync Version: 3
Profile Version: 49
Keychain Supported: Yes
Alternative instance ID: -
Sync target: Dropbox
Editor: Rich Text

Revision: c615726

Backup: 1.5.1
Favorites: 1.3.2
Freehand Drawing: 4.3.0
Jarvis: 0.13.4
Note Statistics: 1.0.4
Note Tabs: 1.4.0
Paste HTML as Markdown: 1.1.12

What issue do you have?

Am trying to import around 2k markdown files in a directory with front matter headers.

I think the files are good, but the import is throwing a lot of errors on a large number of files:

" Bad indentation of mapping entry 1:10"

Using the import->file instead of import->directory throws the same errors

Seems to be a similar issue to the one reported here: While importing md files I get "bad indentation of a mapping entry (3:12)"

...although that seems to be a slightly different error code if that is what the post fix is!!??

Could it be a joplin markdown implementation issue?

Fwiw one common denominator in the files throwing the error is that titles with a stray double quote " seem to be the culprit, in other words an odd number of "s even if the title itself is properly contained in an opening and closing " pair.

Could this be the issue, and is it possible to relax parsing?

Can you share an example front matter? I think this would make it easier for the developers to reproduce the issue.

Thanks very much for your reply, Marph!

This may sound a little ridiculous but the these are old confidential client notes, so I don't really want to put this up.

But as i said the consistent theme with the notes that failed to import is the title front matter has an unmatched ending double quote, which presumably leaves a text string hanging without a tag in the front matter conversion.

Seems to me very likely that is the issue, easily replicated with any random string, but the question is could the import logic be tweaked to address that gracefully.

Again thanks for your interest

update: Ok, it is definitely the unpaired double quote throwing the error.

However, the file actually seems to be imported, but the error crashes the wider import routine which then exits (causing extensive processing on sync and subsequent deletion/trash emptying to retry the process.

I don't expect we could (or should) import an invalid FrontMatter header. Importing invalid data can have side effects, including subtle data loss that may not be noticed until it's too late.

What app exported these files? Because that's what should be fixed. Otherwise the only solution would probably be a bespoke script that fixes the files for you before your import them. ChatGPT would do that easily for you in Python or your language of choice

Right, but the note itself is in fact imported but without any front matter. And only after import does it throw the error and then exit the import. I suggest the behaviour should be either

a) throws the error and discards the note or
b) discards the front matter and continues the import

My preference is b :wink: