The Implementation & UI has been updated after discussion with the mentors and community.
Hi Everyone
let's continue the previous post
I will try to explain the implementation as simply as possible.
Implementation
IMAP
One of the main goals of the plugin is the ability to log in to different email providers like Gmail, Outlook, AOL, Yahoo, ProtonMail, etc. And I see this as the main challenge of the project, and for this to happen the IMAP protocol gives the plugin the ability to connect to different email services (receiving and managing) emails.
There are three main ways different email services give you to connect to IMAP:
- IMAP can be used by just passing an email and a password after enabling IMAP in the settings of the email provider, like Outlook and Gmail.
- Using "generate a password for 3rd-party apps," the email service will give the user a
new password that can connect to IMAP like AOL, Yahoo. - using OAuth like Gmail.
The three methods require IMAP configuration, such as a hostname and port number = 993 (default), TLS, And the first two ways are to just pass in email and password or (Generated password) and the third way takes the access token and converts it to XOAuth Base64(email + access token) and just passes it to IMAP.
Examples showing the different connections of IMAP:
// conventionla connection
let imap = new Imap({
user: 'example@example.com',
password: '**********************',
host: 'imap.example.com',
port: 993,
tls: true,
});
// connection using OAuth2.0
let imap = new Imap({
xoauth2: 'string - Base64-encoded',
host: 'imap.example.com',
port: 993,
tls: true,
});
successful connection
It consists of two main parts:
- fetch all emails and monitor any new emails based on the 'from' attribute
By linking the plugin to an external account and forwarding through your main account all the emails you want to convert to an external account, or you can also forward all the emails you want to yourself (same account). In this case, write your email in the From field.
- fetch all emails and monitor any new emails based on specific mailboxes or folders.
Or you specify the mailbox or folder that you want to convert all emails and also monitor any new email that reaches this mailbox or folder and you can also remove monitoring.
Exporting
IMAP will return the full data of the messages as a sequence of base64 chunks.
After the message is collected, I will pass this message to postal-mime to parse the
message and I will get an object that contains all the data inside the message like
this:
const parser = new PostalMime()
// email of type RFC822
const data = await parser.parse(email);
/*
data.html // HTML content of message as a string
data.text // Plaintext content of message
data.attachments // Array that includes message attachments
data.subject, data.to, email.cc, ...etc
*/
Following the parsing of the email, we have two more challenges:
- Convert the message's HTML content to markdown language.
- Add email attachments to Joplin.
- The first challenge can be completed by using Turndown.
- The second challenge: Each attachment from the parser is as an ArrayBuffer, so it will be converted to an actual file and then by using joplin.data.post to create resources for Joplin. Thus, we will get an id for each resource (attachment)
Before sending the note to Joplin, at the end of the note content, each attachment
will be added with an id by (:/id)
UI of a plugin
A manual screen is just an HTML page that takes email, password, hostname, port, tls.
How to locate converted emails?
by using @ for nookbook and # for tag (work only on forwarded emails to the email that was specified).
Plugin Setting
Please don't hesitate to make any recommendations or suggestions you have, whether it is in the UI or implementation.