Plugin: Frontmatter Overview

Hello again. The plugin seems to only return 99 rows. Is that intentional?

And do you have any plans to maybe add simple filtering? I have numeric attributes and it would be handy to be able to filter non-empty rows, and have less-than/greater-than/equal operators, even if just with constants.

Definitely not intentional, but fixed now! Thanks for the bug report, I don’t know how I hadn’t noticed it myself.

I’ve thought about adding support for filtering and now that I know it would actually be useful for a user, I will probably implement it, though it may take some time. Just in case, this helps you in the mean time: Filtering out completely empty rows is already possible with the excludeEmpty: true parameter.

When I implement this feature I would probably have an SQL-like syntax like this: where: attribute='filter' or where: attribute > 0 for numeric values. I would also add the ability to filter the attribute for being empty or not empty. Is there another use case you would find useful?

1 Like

Thanks, that was fast! It works well now.

I have read the docs and know about excludeEmpty, but that doesn't help.

For now, basic filters would be enough for me: i.e. checks for things being empty, non-empty, and trichotomic comparisons to constants.
It's easy to imagine other operations, such as comparisons of columns (i.e. instead of price > 4 or rating = 2, having price < rating) and substring/regex search ('i.e. tags include 'fps', but I don't really need those at the moment.

Here's one of my use-cases, in case you're interested!
I keep various kinds of records - about books, games, etc - in NocoDB, because those are well suited for tabular data like that.
But I also need them in my notes.
So they're integrated, with all my games having auto-generated yaml frontmatter. (Btw I started doing that purely because of your plugin. :slight_smile: )

I'm going through my records and notes, cleaning up, adding data, etc. That's where this plugin comes in.

So I generate tables that have a link to every note, a line number (so I know how many records there are to process), and columns for things like price, rating, play status, total playtime, etc.
I slice the data in various ways - I want to see the best rated games, or the ones I dropped, or the one I played at least 1 hour, etc.
As it is, without filters, I always get all the games in the list, even if most of it has no data in the relevant column. (E.g. I haven't played it yet, so Playtime is empty.)
I work around this by using Sort, but having filters would be better.

1 Like

I was thinking today to create something similar to this lol, and it's done, yay. Thanks for the work. I have several suggestions to add new features.
Here you have one, will copy this post to an issue in github.

Didn't tested it, I dont have idea yet how to set a development enviroment for joplin.

But doing this small changes to this functions, a new parsing is available to be able to do aggregates. Using for example $$ as aggregate definition.
Example:

from: notebook:"Books 2025"
properties:
  - LINE_NUM AS Number
  - NOTE_LINK AS Title
  - title AS Book $$ count
  - author
  - price as Price $$ sum
  - rating AS ⭐
sort: rating DESC

This should add in the footer, at the same column as title, the count of elements returned in the table, with this result:

Number Title Book author Price :star:
1 Note 1 Book 1 5,99 5
2 Note 2 Book 2 Author 2 7,99 2
Count:
2
Sum:
13,98

Available options:
count Total elements in the table.
sum Sum of all the values of the column of this table.
avg Average value of the values of the column

function getOverviewSettings(overview: string): overviewSettings | string {
	let overviewSettings: overviewSettings | null = null;
	try {
		overviewSettings = yaml.load(overview) as overviewSettings;
	}
	catch (error: any) {
		console.log("YAML parsing error:", error);
		return `YAML parsing error: ${error.message}`;
	}
	// check basic structure
	if (!settingsValid(overviewSettings)) {
		return "Invalid overview settings";
	}
	// get aliases and aggregates
	overviewSettings.properties = overviewSettings.properties.map(prop => {
		// Busca agregado separado por $$
		const [mainPart, aggregatePart] = prop.split('$$').map(s => s.trim());

		const match = mainPart.match(/^(.+?)\s+AS\s+(.+)$/);
		const baseProp = match ? match[1].trim() : mainPart;
		const alias = match ? match[2].trim() : mainPart;

		return {
			original: baseProp,
			alias: alias,
			aggregate: aggregatePart || null
		};
	});

	if (overviewSettings.sort && overviewSettings.sort.endsWith(" DESC")) {
		overviewSettings.reverseSort = true;
		overviewSettings.sort = overviewSettings.sort.slice(0, -5);
	}
	// check sort
	if (!sortValid(overviewSettings)) {
		return `Invalid sort parameter: Please check that it matches one of the original property names, except ${LINE_NUM}.`
	}

	return overviewSettings;
}


function makeTableOverview(properties: {original: string, alias: string, aggregate: string | null}[], notes: any[]): string {
	let tableOverview = "<table><thead><tr>";
	for (const prop of properties) {
		tableOverview += `<td> ${prop.alias} </td>`;
	}
	tableOverview += "</tr></thead>";

	// Cuerpo de la tabla
	notes.forEach((note, index)  => {
		tableOverview += "<tr>";
		for (const prop of properties) {
			let propValue = note.frontmatter[prop.original] || "";
			if (prop.original === "NOTE_LINK") {
				const titleDiv = (document.createElement("div"));
				titleDiv.textContent = note.title;
				propValue = `<a href=":/${note.id}">${titleDiv.innerHTML}</a>`;
			}
			else if (prop.original === "LINE_NUM") {
				propValue = (index + 1).toString();
			}
			tableOverview += `<td> ${propValue} </td>`;
		}
		tableOverview += "</tr>";
	});

	// Footer con agregados
	if (properties.some(p => p.aggregate)) {
		tableOverview += "<tfoot><tr>";
		for (const prop of properties) {
			if (prop.aggregate) {
				let result = "";
				let label = "";
				if (prop.aggregate === "count") {
					result = notes.length.toString();
					label = "Count:";
				} else if (prop.aggregate === "sum") {
					result = notes.reduce((acc, note) => {
						const value = Number(note.frontmatter[prop.original]);
						return acc + (isNaN(value) ? 0 : value);
					}, 0).toString();
					label = "Sum:";
				} else if (prop.aggregate === "avg") {
					const values = notes
						.map(note => Number(note.frontmatter[prop.original]))
						.filter(v => !isNaN(v));
					result = values.length ? (values.reduce((a, b) => a + b, 0) / values.length).toFixed(2) : "";
					label = "Avg:";
				}
				tableOverview += `<td><strong>${label}</strong><br>${result}</td>`;
			} else {
				tableOverview += `<td></td>`;
			}
		}
		tableOverview += "</tr></tfoot>";
	}

	tableOverview += "</table>";
	return tableOverview;
}

Thank you for the feature request and for even writing the possible logic to implement it!

It’s a good idea and I’ll take a closer look when I have the time to implement new features. Just one question: what do you think the result should be if you added a count to the author column in your example? Should it always just be the number of rows or would that result in 1 since only one of the rows has an author value?

Well, I told you I was thinking to develop something similar :stuck_out_tongue:

You only have to copy-paste :stuck_out_tongue:

But I agree with @laurent and could be better to rename the plugin and take in consideration more things than frontmatter like native properties like filter by tags for example. This makes the plugin more powerfull, and usable by people who don't use or don't want to use frontmatter. I will try to get the time to clone the repo and play with this things in local

Good question. The best IMHO is with 2 counts. Something similar like excel.
count counts all non-empty cells with values
counta (count all) counts the total number of cells.