Hello World! From Kishlay-notabot

GSoC
1

Hi there! I am Kishlay, a creative and passionate student developer. I feel excited to finally have an official start in the open source domain. I'd like to contribute in one of the proposed ideas which is the OCR plugin.

Coincidentally I just finished a research project (A corresponding research paper is also planned with it) a few weeks ago which uses the same technology, Tesseract.js [I will link the project below]. So I have learnt and understood this tool very well and I also raised some PRs and issues in the Tesseract.js repository. I am up to date and have a good grasp with the latest v5 of the Tesseract OCR engine.

This is my first time participating in GSoC, I did read all the info pages but it would be a great help if anyone gives me just a TL;DR for the steps to successfully contribute here, and what to do before the registration window starts.
Thankyou!

2 Likes
2

I had some questions, now that I've decided to work on a project idea, Do I have to contact the mentor for that program right away, or wait for the registration process? I really am new to this landscape, so I apologize in advance.

3

Hi @kish!

Tesseract support was recently added to Joplin Desktop by @laurent, though there's still room for improvement. Here are a few things that are missing:

  • Functionality:
    • No OCR for handwriting
      • Joplin's mobile handwriting feature stores drawings as SVGs. These are currently skipped by OCR.
      • A possible fix would be to render SVGs as bitmaps before OCR, though this discards information from strokes not visible in the drawing.
    • No OCR on mobile
      • At present, OCR only happens on desktop. OCR search results must then be synced with mobile.
  • UI
    • No visible progress indicator.
    • Doesn't scroll to search results and OCR matches aren't highlighted in the UI.

Note: At the time of this writing, the GSoC ideas list on Joplin's website is outdated (this is being fixed). You can find a current list of GSoC ideas in Joplin's GSoC repository.

4

Oh hi, okay so if I work on making the OCR process more transparent and better, can it be counted under GSoC? And please educate me about who to contact if I wanna explore this domain, and pursue it as an idea.
And, I don't have the perms to post links, but you can see my latest project using tesseract here:
github[dot]com[/]Kishlay-notabot[/]dcda

2 Likes
5

I've included the link!

Okay so if I work on making the OCR process more transparent and better, can it be counted under GSoC? And please educate me about who to contact if I wanna explore this domain, and pursue it as an idea.

I would suggest talking to @ Laurent about this for both of those questions :slight_smile: .

Edit: 1) Removed the @ mention as per Joplin's GSoC guidelines. 2) Wording adjustments.

6

I agree with your OCR ideas - would you mind adding them to the ideal list? (as just one idea, under the "OCR" theme)

Basically also to support scenario like these:

  • Someone take a photo of their handwritten note, and we'd like to extract the text from it
  • Display the OCRed text as selected text above an image (as is done I think in Evernote and probably in other note-taking apps)
1 Like
7

@laurent Both of the 2 points you have mentioned can be achieved through Tesseract. And sorry for my misunderstanding but are you the mentor of the same proposed plugin project ?
Edit: typo

8

Thankyou for posting the link!