[Devlog] - Mail Organizer update: Week 1

Although I'm not sure I'll be able to stick to this regularly at the start, I'd still like to establish the routine of creating a weekly update (at minimum) for the projects I'm working on. This is the first update for the mail organizer project I'm working on.

Current status

The current state of the project is that my little sandbox is now working as a CLI to organize my mail. That's right. I've essentially built a fully functional mail organizer system that runs from my terminal. It's very verbose and it's got the jank you'd expect but I'm super happy with the progress that's been made on it so far. It currently works in 2 steps.

1. First, you setup your rules by running node index.js. At this stage, your email is downloaded from the "Inbox" folder and the script iterates through it. When it comes across a mail address it hasn't seen before, it asks you what to do with it. You enter your option using a single character and the options are:
   1. Screen out - o
   2. Screen into conversations - c
   3. Screen into feed - f
   4. Screen into paper trail - p
   5. Ignore for now - i
2. You then sort your mail by running node sort.js. The script once again goes through all the mail and sorts it based on the rules you specified in the first step. For example, screened out mails are marked as read and put into the "Screened out ". Only mails that are marked for "Conversation" will be left as "Unread" when going into the conversations folder.

Here's a quick taste of what it looks like:

That's the mail organizing process

That's the mail sorting process

Next steps

Currently the mail configuration gets stored in the same directory as the source code inside a file called sorted-mail.json. While the json format works fine, I need to ensure that the path to the file can be configured. In my case, I'd like to configure it to a directory that's synced to Dropbox.

Additionally, even though the repo is a playground and not meant for public consumption I need to organize it a bit better. A small README on how to get setup + a license would make it ready for public consumption. I do need to be mindful of the time limit here. It's 1 week down already and I have only until June 6th.

That timeline matters because the real major next step will be to start making this more accessible. This was always meant to be accessed via a GUI. I think having this work inside of an Electron app might be the way to go. I don't know for sure though and I need to make this decision soon. Exploring this will be my weekend jam I guess. I'll also be creating a fresh repo to make the more accessible version. It'll use some of this code as its underlying logic.

Developer notes

After going through the options for working with imap, I settled on the excellent imapflow library. Not going to lie, although it is excellent, the examples on working with the emails themselves could be a little more extended. It's an open source project though, so I'm considering contributing some time towards this once I finish this project.

The primary challenge I ran into here was how to fetch mails reliably. This is not technically difficult. But I was a little confused with the difference between grabbing a mail by sequence number vs UID. Looking back at it, I should have realised it sooner, but at the start I didn't understand why I couldn't use a single UID as an integer to specify the mail I wanted. By default, to retrieve mails I needed to use a "Sequence string" which looks something like this: 1:3 which retrieves mails with sequence numbers 1 to 3. Because of the idea of a sequence number and the wording "sequence string" I assumed that if I was using UID I'd use it as a single integer. But UID does need to be converted to a string to be used.

Another challenge I ran into but have not solved was operating on the emails when iterating through them using fetch() . An example:

client = new ImapFlow({config_obj})

//connect and open Inbox.

for await (let message of client.fetch('1:*', {envelope: true})){
  //store relevant details in vars
  await client.messageMove(uid.toString(), 'Folder to move to', {uid: true})
}

This code hangs at the client.messageMove step. My guess is that it's because inside that for await loop. But I don't understand really what's happening. So to not waste time, I store all the details inside a new object with the structure:

      
let organizedMessages = []

for await (let message of client.fetch('1:\*', {envelope: true})){
  // store relevant details in vars
  
  organizedMessages.push({
    uid: message.uid.toString(),
    sender: fromAddress,
    formattedDetails: formattedMessageDetails
  })
}

I then process it in a standard for (let i=0; i<organizedMessage.length; i++) loop. For whatever reason, that works fine. I'll try and look into this later.

Also, although I could have programmed the whole process to be a single step, I opted out of that in order to not trigger something that broke my entire email box. Because we be testing in "production"!

Finally, there was the excellent prompt library that I used to get the configuration value for each email during the screening process. I haven't added any validation though so if a person enters a non valid answer, the system is going to take it and say "thanks!" and I have no idea what class of errors will be introduced from there 😅.

And that's it for the weekly update!

The update is part of a series of updates I'm doing as part of my mail organizing project where I'm building a tool to organize and control mail using a workflow inspired by Hey's mailing system. To find out more and see other posts related to the project, you can find the central hub here

This blog doesn't have a comment box. But I'd love to hear any thoughts y'all might have. Send them to [email protected]

Posted on May 13 2021 by Adnan Issadeen