Email Plugin Implementation & UI

The Implementation & UI has been updated after discussion with the mentors and community.

Hi Everyone :wave: :grin:

let's continue the previous post :sweat_smile:

I will try to explain the implementation as simply as possible.

Implementation

IMAP

One of the main goals of the plugin is the ability to log in to different email providers like Gmail, Outlook, AOL, Yahoo, ProtonMail, etc. And I see this as the main challenge of the project, and for this to happen the IMAP protocol gives the plugin the ability to connect to different email services (receiving and managing) emails.

There are three main ways different email services give you to connect to IMAP:

  • IMAP can be used by just passing an email and a password after enabling IMAP in the settings of the email provider, like Outlook and Gmail.
  • Using "generate a password for 3rd-party apps," the email service will give the user a
    new password that can connect to IMAP like AOL, Yahoo.
  • using OAuth like Gmail.

The three methods require IMAP configuration, such as a hostname and port number = 993 (default), TLS, And the first two ways are to just pass in email and password or (Generated password) and the third way takes the access token and converts it to XOAuth Base64(email + access token) and just passes it to IMAP.

Examples showing the different connections of IMAP:

// conventionla connection
let imap = new Imap({
    user: 'example@example.com',
    password: '**********************',
    host: 'imap.example.com',
    port: 993,
    tls: true,
});

// connection using OAuth2.0
let imap = new Imap({
    xoauth2: 'string - Base64-encoded',
    host: 'imap.example.com',
    port: 993,
    tls: true,
});

successful connection

It consists of two main parts:

  1. fetch all emails and monitor any new emails based on the 'from' attribute

By linking the plugin to an external account and forwarding through your main account all the emails you want to convert to an external account, or you can also forward all the emails you want to yourself (same account). In this case, write your email in the From field.
UI1


  1. fetch all emails and monitor any new emails based on specific mailboxes or folders.

Or you specify the mailbox or folder that you want to convert all emails and also monitor any new email that reaches this mailbox or folder and you can also remove monitoring.
UI2

Exporting

IMAP will return the full data of the messages as a sequence of base64 chunks.
After the message is collected, I will pass this message to postal-mime to parse the
message and I will get an object that contains all the data inside the message like
this:

const parser = new PostalMime()

// email of type RFC822
const data = await parser.parse(email);
/*
data.html // HTML content of message as a string
data.text // Plaintext content of message
data.attachments // Array that includes message attachments
data.subject, data.to, email.cc, ...etc
*/

Following the parsing of the email, we have two more challenges:

  1. Convert the message's HTML content to markdown language.
  2. Add email attachments to Joplin.
  • The first challenge can be completed by using Turndown.
  • The second challenge: Each attachment from the parser is as an ArrayBuffer, so it will be converted to an actual file and then by using joplin.data.post to create resources for Joplin. Thus, we will get an id for each resource (attachment)

Before sending the note to Joplin, at the end of the note content, each attachment
will be added with an id by (:/id)


UI of a plugin

A manual screen is just an HTML page that takes email, password, hostname, port, tls.

How to locate converted emails?
by using @ for nookbook and # for tag (work only on forwarded emails to the email that was specified).

@

Plugin Setting


Please don't hesitate to make any recommendations or suggestions you have, whether it is in the UI or implementation.

2 Likes

as you show
image
how is your OAuth implementation?

1 Like

By using Window.open() and it will direct the user to the scope endpoint of the google account like this:

And if the user accepts the permissions (email account and access email), it will direct the user to a specific URL(localhost) with code, and I will create an HTTP server to take this code

Once I get the code, I will send a post request with the code to get the refresh token
and store it in the login setting to enter directly when the user logs in again. And also once I have a refresh token, I will send another post request with a refresh token to get an
access token

For the UI I'd like to have an option to not have any UI at all. Just run on timer, fetch new emails based on the config and convert them to notes without any manual action from the user.
This is roughly how Evernote works and I have a feeling this is what most people will be interested in. May be worth creating a poll

6 Likes

which browser backend your are going to use to open the login page?

I did a little research on Evernote and what I found so far is that they use Evernote's unique mail for each user to send them emails to Evernote notes.

But I have a suggestion:
The user can move the emails that need to be converted into notes in a folder (Joplin) and the plugin checks from time to time if there are new emails that need to be converted and after each email is moved, it removes a Joplin folder flag from it..

Is this what you mean?

I just used Window.open() because it is a pop window and I think the kind of browser depends on what is user-installed, and this method is available in all browsers: Chrome, Firefox, Safari,.. etc.

1 Like

Pretty much. I think it should be more flexible than just a folder. For instance, once the plugin is ready, I'll set up another inbox with my email providers, something like joplin@mydomain.net and configure Joplin to fetch all emails from that address. Then from my main email account I will forward to my joplin email and expect new notes to be created -- just like Evernote only with better email address.

4 Likes

In Electron it will create a new BrowserWindow instance. It's irrelevant to the native browser since Electron is built on Chromium. :grinning:

1 Like

Some ideas/questions:

1.) Say I cannot or do not want Joplin itself fetch emails online via IMAP (e.g. due to offline system).
Is it possible to specify an alternative mail source like

  • a local filesystem folder of .eml files
  • Thunderbird (or different mail client) profile directory

to grap mails from?

2.) What about GPG encrypted mails? Is there some possibility to include a GPG agent in your workflow, so content gets decrypted before importing it to Joplin note?

1 Like

I totally agree. It would allow to benefit from the plugin even when IMAP is not allowed (OWA pro for example)...

1 Like

I think for client like Thunderbird this solution is better --> Joplin Export: Export emails from Thunderbird to Joplin

2 Likes

I think the picture now is more clear and I think this way will be more efficient than searching inside the plugin and more reliable.

But I have some questions, please:

  • Will the user enter this email and password of their email provider or will an email be generated somehow like Evernote?
  • Is the plugin that creates another inbox? and why do we want to create it?

Can we only convert all the emails to notes and then add a flag to these emails (To keep it from converting again to a note).

Thank you for this clarification :grinning:

1 Like

I think this idea is applicable, But now we are looking for a standard way to avoid different errors and the different ways to connect to IMAP because each mail provider has its own connection methods.

I'm not familiar with GPG, but if you mean to store data securely I think the secure will suffice or If your meaning is to securely transfer data, you just set TLS to configure IMAP with true and the data is transmitted encrypted(almost all email providers use TLS)

The former. I don't see how Joplin can provide an autogenerated email like Evernote. Unlike Evernote, there's no central Joplin server so each user will have to set up their own email (or use their existing account).

Not quite sure what you mean? All the plugin needs to do is connect via IMAP, fetch emails, and create notes. Setting up the account is best left to the user.

I can see a few options:

  • add a flag
  • delete processed emails
  • mark unread
  • move to another folder
3 Likes

To make sure I understood correctly:

  1. The user enters the email address, password, and configuration to connect with IMAP.
  2. The plugin checks from time to time, like every half a minute, if there are new emails in the inbox or any folder the user previously selected.
  3. Once the plugin finds a new email, it converts it to a note and then adds a flag to it or deleted it, etc.

Is that what you mean? :grinning:

I also think it would be a good idea for the user to type something like @NoteBookName #TageName before forwarding the message to specifically locate the note and add tags to note.

I think the main screen and display email screen don't have meaning when you use this mechanism, right?

Do you have any comments about the login screen?

Shall we leave these options to the user?

I was just asking about "I'm going to set up another inbox with my email providers," Is this feature available for every email provider(set up another 'inbox' with the same name)?

It’s a good idea yes. You think for example in the title of the email ? Between two signs (eg "<>") ? Todoist offers something like that I think...

2 Likes

Yes

Correct. Someone might prefer fetching emails manually though. But I think adding the GUI can be done at a later stage, if needed.

I can't say for all providers. Even if you use something like gmail, nothing's stopping you from simply registering another account and using it only for Joplin.

1 Like

On the other hand, I think that some of the users would like to use this by archiving emails in Joplin and other users would rather like to collect emails as tasks. It might therefore be interesting to provide an option "convert as notes" or "convert as tasks" (as the plugin does for Thunderbird)...