Feature request/discussion: Custom API metadata

Hello,

since Joplin now has a shiny new API that seems pretty cool, I'm tempted to integrate it into my workflows. (That's something I've always wanted to do, but didn't really have a good editor to do it in. :slight_smile: )

I sync data from various systems to my blog (some were automated, most manual). That kinda sorta works when I just create a new note, but to automate updates, I'd need to be able to find and identify a note to update - ideally by storing an ID used by the external system. (E. g. when I sync something from Airtable or Goodreads, I might want to store an Airtable or Goodreads ID to find the note by.)

So getting to the point, my question: would it perhaps be viable to add a new field, something along the lines of 'additional data', to the notes' metadata? From Joplin's point of view it could be a plain string that it doesn't process in any way, just lets me get, set or search it via the API.

I've already noted the benefits this would have for me; I could, for instance, store a json that says 'this note is from system A where it has the ID XYZ'. I could then find and update this note whenever necessary.

When it comes to drawbacks, there are the usual obvious ones: it would need to be implemented, and I'm not sure how many users would actually benefit from it/want to use this feature.
Other than that, noting comes to mind; seems to me this should be pretty orthogonal to all the other existing features and should not cause conflicts or require much upkeep. (But I might be wrong here, I don't know the internals that well.)

Thoughts?

3 Likes

While reading the API docs, I've noticed this:

Resources larger than 10 MB are not currently supported on mobile. They will crash the application so it is recommended not to attach such resources at the moment. The issue is being looked at.

If I understand it correctly, this is no longer the case? I'm not sure and have no way to test RN.

Could you perhaps use tags for this, or would the number of them become unwieldy?

You mean one tag per note? That would be unwieldy indeed, and also a completely wrong use for tags.

Bumping up this thread - any plugin or future interest in a custom metadata feature?

Any plugin can already associate metadata with a note if needed. The metadata can be stored by the plugin in their own JSON file, SQLite database, etc.

I haven't been able to quickly find an answer in the docs: if I use a JSON file, SQLite database, etc, I assume they will stay local and will not be fixed?

So presumably the best solution still is having a custom JSON file, but keeping it in a note to ensure syncing? (I've been doing it like this in my Ghoplin tool ever since this thread started, which I now see has been 2 years ago already =-o )

Thank you - can you point me to an example of such a plugin and/or the documentation of how to do this?

How would I integrate the custom metadata with the main Joplin interface so I can enter this field while editing/viewing a note rather than only via the API?

Right, if you need the metadata to be synced you'd need to save it to the note itself. If it was really needed, I guess we could add a "userdata" field to notes which could be set by plugins and external apps, and it would be synced. I don't know if many plugins would need this though.

4 Likes

You could add an extra panel or have a dialog using joplin.views.panels

1 Like

This will open up many possibilities.

For example,

I'm currently maintaining my book reviews using a custom template. It has 2 custom fields for rating and relevant notes.

Now consider that, If I want to use the Note Overview plugin to create a dashboard for all reviews. I can't get the review data in the table because it's just part of a body.

With an additional userdata/metadata field, plugins can add metadata (ideally namespaced, in JSON.) and other plugins can run queries on that.

1 Like

Hi laurent,

I would like to raise again the topic of notes and user data / metadata / properties that can be set for plugins per note. I have listed all the toppics in the issue where where this topic was discussed.

it just happens again and again that somewon want to store some information per note for plugins, which is of course possible in a separate database local. But this is only a locale information and not shared between the different installations :frowning:

A few examples from existing use cases which I know:

  • IDs of the saved note at HackMD, gist, ...
  • Display settings for the note (currently saved as tags)

I think it would be an extreme enrichment for the plugins eco system if they could write metadata for the notes that are synced, as this could improve existing use cases or offer new possibilities.

1 Like

I think we discussed this in depth in an issue. Would it make sense to bring that one up, as it would have more information to help us decide how to proceed?

Not sure if I've seen the issue, but I'd be glad if this topic was reopened.
I also think having plugins use local, unsynced storage is an unfortunate default.

Yes and add this feature to the enhancement issues.

Plugin note metadata

  • Data must be synchronised across all devices
  • Plugins only have access to key/values of their namspaces
  • Include the data in the RAW, JEX, and MD exports
  • [Optional] Define a key/values pair which has public as option and other plugins can access this value
1 Like

What is that metadata attached to? To notes, notebooks, tags, etc? Or are we talking about synchronising the plugin settings?

I think it's most important to be able to attach metadata to notes; next come plugins, next - notebooks and tags are 'nice to have' and I wouldn't miss them if they weren't available.

My reasoning is: The notes are the most important, because that's the entire point. I can, for instance, say 'this note is synchronized to system X, where it has ID Y'. I need that per-note.

Per-plugin data would be great for configuration sharing and the like - connection strings, settings, etc. It can be worked around using notes - currently, I'm still using this:

Summary, so you don't need to click that link: my CLI app, plugin, or whatever, have a hard-coded GUID that refers to a note with the plugin's configuration; it stores any config there. (Or, if need be, any per-note metadata.) I just put anything in a data structure, then serialize it to JSON and store that in a specific note.

That works, because since the ID is hard-coded, there's no need to sync it between instances; and the contents themselves are synced as a standard note.

It's kinda okay when it comes to syncing plugin config; not so great for per-note metadata, because it requires having a huge note with a serialized dictionary with KVPs for each note.

Last but not least: we would need a UI to show the metadata. I think it'd be quite sufficient to have a link for 'show note metadata' in the Info section where History is, which would just show the data for the note. (A read-only table would be fine.)

Is it also worth bringing this discussion up too? - Ability to add arbitrary properties to Note Properties, and hence Markdown frontmatter - #5 by gotofritz

It is about also being able to use the YAML frontmatter to store additional key-value pairs within the note metadata and it seems to me (probably naively) that it could use the same functionality as could be provided via the plugin api.

Yes primarily for notes

A option to sync the plugins settings, or some configured plugin settings/Joplin app settings would be nice but this topic is for Custom API metadata on note level.

Hi guys, the latest pre-release 2.11.11 adds support for user data. More info here:

If you can, please give it a try and let me know if you notice any issue. We can still change the API while it's in prerelease if there's something that doesn't quite work with your use case.

The prerelease is there:

1 Like