Talking about Markdown's CodeMirror theme update

Hi @pokemonleo, welcome to the forum!

First off, I'd like to say thank you for the detailed post. It can be hard to express frustration and I think you've done an excellent job.

I think you've done a great job making your point. For many users, markdown should be treated as code, and for many of those users code needs to have syntax highlighting. While this may seem like a matter if preference, it can have a very real impact on an individuals ability to use the application. This is not something that I had considered when reviewing the recent change. At the time, my thoughts were mostly on making Joplin more approachable for users that are uncomfortable with code and my assumption was that users who care about markdown-as-code are the same users that don't shy away from custom css. This was wrong.

Overall, I see 2 primary issues:

  1. Many users deeply care about the Joplin visuals, but don't necessarily care about maintaining custom CSS.
    • As a sub-issue, even with a wiki page css is not easily discoverable.
  2. Many users are passionate about the current state of Joplin, but don't care to or don't have the time to take an active role in providing feedback.
    • This change was shared and available for 5 days before being accepted. It received no engagement from the broader community.
    • It has also been in pre-release for the past 25 days, which is meant to give users time for this kind of feedback. But again, no complaints.

By the way, I'm not criticizing anyone for not taking a proactive role in the development process. I only mention it to point out that this change didn't appear out of nowhere, and there are users who have been happily using it for almost a month. So this is not a black and white change, I'm certain that many users (myself included) see this as a change for the better.

What does this mean?

What this means to me is that there is a desperate need for community-supported user-friendly customization. What I envision is a plugin similar to the awesome macOS theme plugin, but would allow users to select from a dropdown list of curated themes. Ideally there would also be a tooling in place to allow users to customize their Joplin experience without having to understand CSS. Finally, a method of sharing created themes and making created themes available to all users would complete the tool.
(I believe this has already been suggested by @uxamanda but I can't find the post.)

Why community supported?

Presently, there are not many consistent contributors to the main Joplin application. I believe one of the reasons for this is that Joplin is a large application, and the barrier to contributing can seem very high. I observed that there have been more people willing to contribute to Joplin by creating plugins, likely do to a lowering of this barrier. By lowering this barrier to entry as much as possible, we should see more, and better, themes and styles that cater to all Joplin users.

Okay, but why not officially support it?

The Joplin application has many competing considerations when it comes to developing new features. For example, ideally, anything added to the core Joplin app must be supported on all desktop platforms as well as mobile platforms (if you've been around the forum you'll know that this isn't always achieved). Joplin also has to consider the needs of new users, many of who are using markdown for the first time and can be quite put off by it. Finally, there are the needs of the core developers. As mentioned above, the barrier to contributing to the core app can be very high, adding additional requirements (maintaining legacy styles would be a requirement) just makes that barrier even higher. I encourage anyone curious to take a look at the Joplin contribution plots to get an understanding of how many/how frequent contributions are. It also demonstrates just how much work Laurent is putting in relative to the rest of us. So please take some time to thank him (or donate if you're able).

What now?

Like all initiatives, someone is going to have to put in the work to make it happen (planning and execution). While I think this is a good solution for providing long term styling support, I don't pretend to think it's the best. I've mostly just posted this here as an example of the kind of solution that I think might be agreeable. What I'd really like is to see some suggestions about what we as a open source community can do to support the unique needs of all Joplin's users. So please reply with your thoughts and suggestions!

6 Likes

Yes, so many community tools I created rely on joplin data api, because its api is very small and easy to try and use.

2 Likes

So, I in no way want to criticize anyone, and I know how much work just a few individuals are putting in.

I just now followed the link above and tried to figure out what it was about. As a somewhat aware user who is not really a coder it took me a bit to understand what was going on. So, for things that the developers want feedback on, these are some thoughts to help get more feedback. Now, I know people could respond with "but that should be obvious" or "If people just made some extra effort..." Sure. But it's not, and people probably won't. Any way,

  • Does RFC mean "Request for Comment"? I'm assuming this is a common developer acronym. If you want engagement from non-developers, I'd recommend instead saying "Give your feedback" or "Requesting opinions."
  • You might also want to write headings like that in a way that makes it explicit how it will affect regular users. Perhaps something like, "Changes to the default theme are coming. Give your opinions." Fortunately changing headlines is something anyone with a certain engagement level can do on Discourse, so it wouldn't have to be on the OP to come up with it.
  • For non-tech users, sharing a link to a Github pull request just isn't going to get them involved. I know you shouldn't have to but pasting the images and text from the PR into the OP for sure would get more people engaged.
  • Unless people are posting to a thread, it's going to disappear to the bottom of the page very quickly. If the devs want feedback, these things should be pinned. Otherwise if people only visit the forum once or twice a week, they are not likely to see it.
  • I know it's maybe tacky, but choosing an emoji to use for posts seeking feedback could increase engagement. Something like :question: or :loudspeaker:.
  • The post should include a very plain language explanation of what kind of feedback is wanted. Something like "We are changing how the markdown editor text look. From now on the theme you have chosen will determine what color some text is, for example headings and bold text. Tell us what you think."

This particular change, in retrospect, could have been expected to make people unhappy. Basically you are changing the colours of their interface when they are not expecting it, and there's no one-click way to go back to the way it was.

As far as mitigating any unhappiness, having a thread to announce the change and then giving people specific instructions on what css to add to their custom file to make things look like they did before. New users won't know the difference, so having a thread here for current users to be pointed to should help. (If indeed I'm even understanding the problem)

Just some thoughts.

6 Likes

I agree with most of what you say, anything we can clarify beforehand and any way we can get people involved in such decisions is great. In practice we just do what we can though with limited resources. Having the screenshots in the post and the pull request for example would be nice, but it's extra troubles for us to follow conversations on both the PR and forum, and having to copy and paste relevant info from one to another.

Otherwise I still think this change was mostly a good one, although it's possible the removal of formatting went a bit too far and certain parts should still be highlighted. There's an open issue about the code syntax highlighting for instance.

For the lists, if you look at most Markdown editors (I've checked Discourse, GitHub, StackOverflow, VSCode), none of them highlight bullet point text in purple like we had. VSCode highlight the dash, but I don't think CodeMirror allows this. Perhaps we can find some middle ground, a colour less flashy than purple but that can still make it clear that it's a syntactically valid list.

4 Likes

I totally hear that! And I appreciate what you do.

Very good point. Perhaps a big notice that you should post on the PR if you have a Github account.

Sure! But I would expect a whole lot of "What the #$%^#$!" messages anyway.

Honestly I always thought it was odd that lists were in a different colour, but I grew to like it. Before all this I actually experimented with keeping them the regular text colour but went back to using an accent colour.

I think many people just don't know how to provide feedback. For sure most people don't know how to provide useful feedback.

2 Likes

I could not agree more with @pokemonleo. Markdown is code. Making markdown look more like a WYSIWYG is antithetical to what markdown is for. For me, this update makes Joplin largely unusable. I am a software developer, and use Joplin because it binds all my notes together in a functional, syncable, UI. The Markdown syntax highlighting was the actual selling point for me for Joplin. Now that its no longer there, Joplin is less useful. I personally don't want to maintain a custom CSS file just for Joplin though I am sure other do,

From a community perspective, these kinds of changes break down trust between the contributors and the users. Joplin has far more casual users than it has users on the forums or contributing. If changes are put forth that are not backwards compatible, and not major versioned, you telegraph to your community that development is not stable. Why would I spend time developing custom CSS when you might decide to change the api while I'm not paying attention. Put another way, if you move my cheese, I am going to look elsewhere to get what I want.

In short, this should have been a feature users could opt into, or opt out of, not one that suddenly requires custom work to maintain parity with what they are used to.

Did you read the rest of the thread? There's work in progress to improve the styling.

1 Like

Whilst I'm personally not overly keen on the changes, this really isn't the discussion at this point. What is done is done and we shouldn’t be flip flopping back and forth based on negative feedback after the case. Instead it has highlighted the need for systems both in terms of change communication and application features that can appease and benefit a whole bunch of different audiences.

I think this was a somewhat perfect example of the disconnect that can happen, as CalebJohn mentioned the amount of work that Laurent puts into this project is somewhat disproportionate and like many other changes proposed and discussed without comment, this one was also brought up to the community and went by without disagreement or any comment at all so therefore was justified. Obviously in this particular case it has become clear that the people affected were not of a demographic that had really been considered – as CalebJohn also mentions it was assumed that the main users would be either casual (who would likely welcome the change) or technical/enthusiast who wouldn’t think twice about tweaking or loading a css and therefore the change would barely affect them.

Posts like uxamanda’s theme discussion and andrejilderda’s macOS theme together with this situation shows that there is definitely a desire or even a requirement to make accessible user styling or theming tweaks that is available for all kinds of audiences. This would include a good, accessible starting theme to not dissuade newcomers, a good modular theme system that can allow less coding savvy users to still contribute to the project just as the plugin system has provided as well a system that allows the tinkering/enthusiast types to modify the application to suit their needs exactly.

I think overall that this is a good change even if for no other reason other than that it has shown just how much people do actually care about this aspect of the application.

4 Likes

When I made this change, the idea was 1. to make the colour scheme less distracting and 2. to use the same consistent style across the app. For example if a user chooses a black colour for the font, and gray for codeblocks, it's strange that the item lists would look purple and the code block brown.

So now we have consistency. But from the feedback I realise that some degree of syntax highlighting is important in Markdown. Caleb has already implemented a great fix for code blocks and inline code. I think we should try to improve list syntax highlighting too - perhaps by highlighting the bullet point?

Are there other parts of the syntax that any of you think should be improved?

1 Like

IMO, the background is a bit distracting because empty lines do not have a background.
The entire code block should have a background.

Unfortunately I think there's not much one can do except remove the background with css again, because there's no div for the entire code block.

1 Like

@tessus Looks like that's a bug (maybe something in your userchrome?). Here's how it looks for me
image
image

Works for me too, the empty lines have the expected colour too.

That's why I deactivated any css before I posted this. :wink:

But I was able to narrow this down:
It happens on macOS 10.14.x. On 10.15.x it looks the same as yours. This is extremly puzzling. I'm using the same binary on both macOS versions.

1 Like

This is the key bit for many I think, I had done a bit of .css fiddling but I found myself reverting to default more than a few times because I either didn't like it or I couldn't work out how to change some of the other elements to match. It wasn't overly pleasing aesthetically but the obvious default syntax highlighting just made it very clear whilst typing if I was making any inadvertant syntax mistakes or leaving sections/tags open rather than having to repeatedly check the output.

I think most of the old default elements that were differentiated in highlighting worked well, for me but I can't say I made use of all of them (like nested ordered lists) I can't think of any severe omission off the top of my head - however I'm yet to update to the latest version as my sync target is somewhat of a mess which I've been having issues with, I've only had a quick play in the portable version without my notes.

Thanks a lot for the attention. And appreciate to definitely the thought from developers. And sorry for the late reply.

As a user of Joplin, I am very grateful for the people who create Joplin and make Joplin come into the world. This app does really improving my productivity, by keeping notes, tasks, posts, and a lot of other things.

And an apologize, I didn't really check often the prereleases and how the dev team approaching the app, which I am focusing on my job and have not really concerning about the community of what I am using, which make me unnoticed every detail in the skipped version, and even unnoticed to the big efforts what dev team is done on it.

Actually, as a user, we—or I—had never needed a list to be colorized before we using Joplin. At the very first moment of thought to using an app to take note, we never thought we have such demand for us.

But Joplin did colorize their list in their Markdown editor in the previous versions. This teaching us users to learn: When we need to find any small piece in a long nested list, the green and purple are helping us to distinguish the levels and find what we want. And we will be imprinted that when we want to find what level of the list on the next time, then we will go to find what color of the text.

And this is also what the color scheme in the programmatic codes inside all the editors on the market is doing, just except the Notepad. The color scheme is not necessary to change and should not change so frequently, and we will deeply remember what color expressing for what type to the text by the time we get used to using the app. The imprinted color scheme may not be carried across multiple apps, but it will keep unchanged in the brain while using the same app no matter the versions.

So as a user, the visual change in the update that making a flowery garden becoming plain is really drastically and it may even change the way users use their tools. Fortunately, there is userchrome.css to turn it back with some of the user efforts or make it even better to suit the user's own eyes, which I had done already. But unfortunately, it is really not easy to start until someone tells you that you can do this and how to do this, or you know how to find the answers. OFF TOPIC: This is also what I want to talk about some may want to have an option to run Joplin while the starting of OS, which people may don't know this could be done on an OS basis, manually. Some app is done this on the options page and some app is done on the installer.

Based on the way of thinking to use the app—maybe it won't apply to all the users, but I know the dev team is also coding in their own way. I hope the thought is got well-expressed—it would be a good practice to keep the default theme unchanged or change it minorly. For example, like only change the color contrast; or add/remove things with a long-term basis: split into pieces and check two or three of them every version, or to notify the users in a more clear way, just like @whitewall had mentioned.

The new CodeMirror theme seems to be the new default. It is even okay that keeps the color of the list unchanged from the new default. If you had well discussed and decided, then you should keep it, long-termed, in my opinion. I am not going to order you to go back or help me to colorize again the list. I just talked about my thought on using Joplin. On using any editor. Although I may not join the conversation about the upcoming changes, by knowing the dev team is ongoing to pay a lot of energy to improve the new CodeMirror theme, I am eagerly looking forward to the upcoming updates.

Cheers
And I keep using Joplin.

4 Likes

I totally agree with this. :slight_smile:

I want to thanks @laurent for all the nice job, and also @CalebJohn for his contribution, always constructive,
We really love Joplin, otherwise we wouldn't be here discussing it. :slight_smile:
But I too I was disappointed with this change, and as the OP noted finding the way to go back (I mean the comments in PR and in the Forum) wasn't so easy.

An "excuse" :sweat_smile: for the absence of comments could be that everything started shortly before the summer holidays, when we find ourselves having to finish some work, and in August personally I was away from the computer, but I think it is not just for me.

I also agree with this idea.
I am very, very, very habitual, :grimacing: and I apply the same theme on Vim, or VScode.
Learning to distinguish certain parts at a glance is of great help for me.

One last consideration.
For my use of Joplin, keeping the editor and rendering styles separate is not a problem, on the contrary, it is sometimes more important to have a clear highlight of where I am typing, and go and see the result later. Sometimes I use HTML tags, or long tables in markdown, and it's more useful to know where I'm putting my hands clearly.

However, I understand the need to have consistency, or rather, to have a style that is not unsettling between the markdown editor and the panel with the rendering.
But here a doubt arises: Is this need perhaps felt more by those who use markdown only to write predominantly textual notes? And that maybe it uses the rich text editor instead of the plain?
I don't know, I don't have the answer. But I would be curious to know how annoying this change was based on the type of user.

I know that this is extremely late, but I wanted to add my two cents and this was the most recent thread I could find on this topic. I didn't think it was appropriate to make another thread, so if I am breaking any particular rule or guideline, someone please let me know. Also, apologies in advance if I ramble, I'm in the midst of a covid brain fog and I'm running mostly on caffeine and spite. (But not against Joplin or laurent. I love this app. I just wanted to write about my experience and leave a helpful snippet for anyone having the same issue.)

I would say that this change was very annoying for my type of user. For context: I make my living by editing large amounts of text, both fiction and nonfiction. Other than occasionally modding video games or working with virtual machines to play the ridiculously old games of my childhood, I have very little experience with "tech stuff" - I want to learn how to do programming, including CSS, but between a schedule that includes multiple projects and a growing family I have precious little time to learn brand-new skills. So customizing Joplin is largely beyond me.

After struggling for years with Evernote and OneNote, Joplin has been a godsend for me: I can make each project into a separate notebook, or even sort notebooks into groups according to contract. I use a lot of outlines to keep my thoughts and proposed edits organized - many editors make an outline and then rewrite the text proper. Joplin's previous behavior of alternating colors when handling lists was great for this! I realize that this was a side-effect of text coloration for the convenience of coders, but there is usually quite a bit going on in a manual, and even more going on in fiction, so having outlines that alternated colors according to their tab depth to organize my notes was great on a level I couldn't describe. Seriously. It was transcendent. I might write a song about how awesome Joplin is someday.

When the change happened, I actually didn't think much about it at first because, again, busy, but over time it got to the point where I was having trouble keeping track of my notes. I was looking at an endless field of white text on a dark background and there wasn't anything to differentiate one piece from another except for the tab depth of the bullet, which wasn't helpful when I was neck-deep in someone's deathless prose.

I still soldiered on, but when I caught covid and my contracts were kind enough to allow me time to recover, in a stunning display of wisdom, I decided to finally restore the alternate colors instead of hibernating. Thus began a six hour long saga of experimenting with the CSS posted here on the forums and elsewhere. Very little of it worked. It was only after cobbling together material from at least three different sources that I found that I had to:

  1. Do a find-replace to get the dark theme to change colors, as the only available snippet that functioned at all is for the light theme.
  2. Figure out which portions of the CSS apply to the lists. If anyone finds this post having suffered the same issue these are, in order of their depth in a list: cm-variable-2, cm-variable-3, and cm-keyword. That is, cm-variable-2 will always be the first depth in the list (starting a list without hitting tab), cm-variable-3 will always be depth 2 in the list, and cm-keyword will always be depth 3 in the list, and afterward they repeat every time you add a depth level. I do not know why cm-keyword comes before the other two in the code but after them in a list, or why cm-variable appears to do nothing.
  3. Paste the CSS into userchrome.css (which for future reference can be accessed through Tools -> Appearance -> Advanced Options -> Custom stylesheet for Joplin-wide app styles). I will include the CSS snippet that I'm using below.
  4. Save userchrome.css, then exit Joplin completely - do not minimize it to the tray.
  5. Sacrifice a goat to the Elder Gods to make my stylesheet work. Unfortunately I had to dispose of my ritual robes afterward, as blood is impossible to get out of fabric.
  6. Restart Joplin.

Mine is probably a very niche use case, I know, but I feel that allowing the option to swap between the colored text and the more standard text might be a good future feature. I do imagine that most coders would be able to whip up something for themselves, though, so maybe I'm the only one.

Thank you all for reading, and thank you Laurent for what is otherwise a flawless app. I think I'll actually go to sleep now.

Note that the below is incomplete; I only changed the colors for the lists and a lot of this looks pretty terrible in dark mode as-is. Still, all you have to do is plug in the hex code for the color you want.

/* For styling the entire Joplin app (except the rendered Markdown, which is defined in `userstyle.css`) */

div.CodeMirror.cm-s-material-darker span.cm-header {color: blue;}
div.CodeMirror.cm-s-material-darker span.cm-quote {color: #090;}
div.CodeMirror.cm-s-material-darker span.cm-negative {color: #d44;}
div.CodeMirror.cm-s-material-darker span.cm-positive {color: #292;}
div.CodeMirror.cm-s-material-darker span.cm-header, span.cm-strong {font-weight: bold;}
div.CodeMirror.cm-s-material-darker span.cm-em {font-style: italic;}
div.CodeMirror.cm-s-material-darker span.cm-link {text-decoration: underline;}
div.CodeMirror.cm-s-material-darker span.cm-strikethrough {text-decoration: line-through;}

div.CodeMirror.cm-s-material-darker span.cm-keyword {color: #70bf5d;}
div.CodeMirror.cm-s-material-darker span.cm-atom {color: #219;}
div.CodeMirror.cm-s-material-darker span.cm-number {color: #164;}
div.CodeMirror.cm-s-material-darker span.cm-def {color: #00f;}
div.CodeMirror.cm-s-material-darker span.cm-variable,
div.CodeMirror.cm-s-material-darker span.cm-punctuation,
div.CodeMirror.cm-s-material-darker span.cm-property,
div.CodeMirror.cm-s-material-darker span.cm-operator {}
div.CodeMirror.cm-s-material-darker span.cm-variable-2 {color: #71bebd;}
div.CodeMirror.cm-s-material-darker span.cm-variable-3, div.CodeMirror.cm-s-material-darker span.cm-type {color: #c2bd60;}
div.CodeMirror.cm-s-material-darker span.cm-comment {color: #a50;}
div.CodeMirror.cm-s-material-darker span.cm-string {color: #a11;}
div.CodeMirror.cm-s-material-darker span.cm-string-2 {color: #f50;}
div.CodeMirror.cm-s-material-darker span.cm-meta {color: #555;}
div.CodeMirror.cm-s-material-darker span.cm-qualifier {color: #555;}
div.CodeMirror.cm-s-material-darker span.cm-builtin {color: #30a;}
div.CodeMirror.cm-s-material-darker span.cm-bracket {color: #997;}
div.CodeMirror.cm-s-material-darker span.cm-tag {color: #170;}
div.CodeMirror.cm-s-material-darker span.cm-attribute {color: #00c;}
div.CodeMirror.cm-s-material-darker span.cm-hr {color: #999;}
div.CodeMirror.cm-s-material-darker span.cm-link {color: #00c;}

div.CodeMirror.cm-s-material-darker span.cm-error {color: #f00;}
span.cm-invalidchar {color: #f00;}

In case you do want the rest I did have a go at creating the css to restore the rest of the dark theme a while ago - Dark theme syntax highlighting - #4 by Daeraxa

You're a hero, thank you very much.