GSoC 2026: Proposal Draft - Idea 5:Automatically Label Images Using AI

Also, I'm attaching a quick reminder of Joplin's policy for AI use in proposals. If AI was used (e.g. for rewording or spelling/grammar correction), be sure to include an "AI Assistance Disclosure" section. Please read Joplin's GSoC AI Policy for details. (If AI wasn't used for the proposal, no need to change anything :slight_smile: ).

thanks, good to know about the separate BrowserWindow, that simplifies things and the contentScripts.register approach for injecting alt attributes makes sense, i'll look into that

this is interesting, @laurent had mentioned concerns about modifying note bodies because of sync conflicts and resources being in multiple notes but the benefits you listed-search working out of the box, screen readers in the editor, alt text syncing to mobile without the plugin ,those are pretty strong reasons to go this route. what if the plugin only modifies the alt text part like ![ generated labels] (:/resourceId) and skips read-only or mid-sync notes? would that be enough to handle the conflict risk or is it still too fragile?

yeah i need to look into this more,mobileclip is trained on english so labels would be english by default .For non-english users we could either keep english labels which would still be useful for search, or add a lightweight translation step like the firefox approach you mentioned. i'll check what's realistic within 175h and add it to the proposal.

Hey @personalizedrefriger @laurent , I've uploaded my final proposal to the GSoC portal. Here's the PDF as well in case you'd like to take a look. Happy to make any last adjustments if you have suggestions before the deadline closes :slight_smile:

GSoC_2026_Kanishka_Idea5_Automatically_Label_Images_using_AI.pdf (1.1 MB)