Data formats

Email reader

Read raw emails: sender, recipients, subject, date, the text and HTML bodies and the attachments, for support inboxes and replies by email.

Version 1.0.0 · 3 functions · 1 crate

#email#import#support#attachments

Functions

What use email_parse.qwer gives a program, with the types Quillo sees. To take only some, or under other names: use email_parse, email_parse_file from email_parse (Quillo 0.6).

email_parse(raw: str) → anyor an error

An email read from its raw text: { subject, from, to, cc, reply_to, date, message_id, in_reply_to, text, html, attachments, headers }. from is { name, email } (null when missing), to, cc and reply_to lists of them; date is RFC 3339; text is the plain body and html the HTML one, each made from the other when the email has only one; attachments are { name, type, size }; headers the raw headers by lower-case name (the last of a repeated one). Encoded words (=?UTF-8?Q?...?=), quoted-printable and base64 are decoded.

email_parse_file(path: str) → anyor an error

An .eml file read as email_parse reads text.

email_save_attachments(raw: str, folder: str) → listor an error

Saves an email's attachments into folder (made when missing), each under its own name with anything but letters, digits, ., -, _ and spaces replaced, so no name can leave the folder; gives { name, path, size } for each. A name used twice gets -2, -3 before its extension.

Get it

$ quillo install email_parse               # email_parse.qwer into this folder$ quillo install email_parse --example     # and its example, email_parse.qwe

Or with curl:

$ curl -O https://quillolang.org/plugins/files/email_parse.qwer$ curl -O https://quillolang.org/plugins/files/email_parse.qwe$ curl -O https://quillolang.org/plugins/files/email_parse_sample.eml

Then, in your program, all of it or only what you call:

main.qwe
use email_parse.qweruse email_parse, email_parse_file from email_parse      // Quillo 0.6: only these

It uses this crate, which cargo fetches the first time the plugin is built:

[dependencies]
mail-parser = "0.9"

Example

email_parse.qwe
// A support inbox: an incoming email turned into a ticket, with its attachments saved.////   quillo run email_parse.qwe      then open http://localhost:8080/ticket//// Inbound-mail services (Postmark, SendGrid, Mailgun, SES) can post the raw email to POST /inbound.use "email_parse.qwer"fn ticket(raw: str) {    let m = email_parse(raw)?    let files = email_save_attachments(raw, `data/tickets/${crypto.sha256(m.message_id ?? raw)}`)?    { from: m.from.email, name: m.from.name, subject: m.subject, at: m.date, replying_to: m.in_reply_to, body: m.text, files: files | map((f) => f.name) }}route GET /ticket = ticket(read_file("email_parse_sample.eml")?)route POST /inbound = ticket(request.body)route GET /parts {    let m = email_parse_file("email_parse_sample.eml")?    { to: m.to, cc: m.cc, attachments: m.attachments, html: m.html, content_type: m.headers["content-type"] }}
$ quillo run email_parse.qwe$ curl 'localhost:8080/ticket'$ curl 'localhost:8080/parts'

Source

email_parse.qwer
//! Read raw emails (.eml, or what an inbound-mail webhook posts): sender, recipients, subject, date, the text and HTML bodies and the attachments, for support inboxes and replies by email.//! version: 1.0.0//!//! [dependencies]//! mail-parser = "0.9"use mail_parser::{Address, MessageParser, MimeHeaders};use serde_json::{json, Map, Value};/// Emails up to 50 MB are read.const MAX: usize = 50 * 1024 * 1024;fn people(a: Option<&Address>) -> Vec<Value> {    a.map(|a| a.iter().map(|p| json!({ "name": p.name(), "email": p.address() })).collect()).unwrap_or_default()}fn safe_name(name: &str, i: usize) -> String {    let n: String = name.chars().map(|c| if c.is_alphanumeric() || ".-_ ".contains(c) { c } else { '_' }).collect();    let n = n.trim().trim_start_matches('.').to_string();    if n.is_empty() { format!("attachment-{}", i + 1) } else { n.chars().take(120).collect() }}fn parse(raw: &[u8]) -> Result<mail_parser::Message<'_>, String> {    if raw.len() > MAX {        return Err("emails up to 50 MB".into());    }    MessageParser::default().parse(raw).ok_or_else(|| "not an email".to_string())}/// An email read from its raw text: `{ subject, from, to, cc, reply_to, date, message_id, in_reply_to, text, html,/// attachments, headers }`. `from` is `{ name, email }` (null when missing), `to`, `cc` and `reply_to` lists of them; `date`/// is RFC 3339; `text` is the plain body and `html` the HTML one, each made from the other when the email has only one;/// `attachments` are `{ name, type, size }`; `headers` the raw headers by lower-case name (the last of a repeated one)./// Encoded words (`=?UTF-8?Q?...?=`), quoted-printable and base64 are decoded.pub fn email_parse(raw: &str) -> Result<Value, String> {    let m = parse(raw.as_bytes())?;    let mut headers = Map::new();    for (k, v) in m.headers_raw() {        headers.insert(k.to_lowercase(), Value::String(v.trim().to_string()));    }    let attachments: Vec<Value> = m.attachments().map(|a| json!({        "name": a.attachment_name(),        "type": a.content_type().map(|c| format!("{}/{}", c.c_type, c.c_subtype.as_deref().unwrap_or("octet-stream"))),        "size": a.contents().len(),    })).collect();    Ok(json!({        "subject": m.subject(),        "from": people(m.from()).into_iter().next(),        "to": people(m.to()),        "cc": people(m.cc()),        "reply_to": people(m.reply_to()),        "date": m.date().map(|d| d.to_rfc3339()),        "message_id": m.message_id(),        "in_reply_to": m.in_reply_to().as_text(),        "text": m.body_text(0).map(|t| t.into_owned()),        "html": m.body_html(0).map(|t| t.into_owned()),        "attachments": attachments,        "headers": headers,    }))}/// An .eml file read as email_parse reads text.pub fn email_parse_file(path: &str) -> Result<Value, String> {    let size = std::fs::metadata(path).map_err(|e| format!("{}: {}", path, e))?.len();    if size as usize > MAX {        return Err(format!("{}: emails up to 50 MB", path));    }    email_parse(&String::from_utf8_lossy(&std::fs::read(path).map_err(|e| format!("{}: {}", path, e))?))}/// Saves an email's attachments into `folder` (made when missing), each under its own name with anything but letters,/// digits, `.`, `-`, `_` and spaces replaced, so no name can leave the folder; gives `{ name, path, size }` for each./// A name used twice gets `-2`, `-3` before its extension.pub fn email_save_attachments(raw: &str, folder: &str) -> Result<Vec<Value>, String> {    let m = parse(raw.as_bytes())?;    std::fs::create_dir_all(folder).map_err(|e| format!("{}: {}", folder, e))?;    let mut out = vec![];    let mut used = std::collections::HashSet::new();    for (i, a) in m.attachments().enumerate().take(1000) {        let base = safe_name(a.attachment_name().unwrap_or(""), i);        let mut name = base.clone();        let mut n = 2;        while !used.insert(name.clone()) {            name = match base.rsplit_once('.') {                Some((stem, ext)) => format!("{}-{}.{}", stem, n, ext),                None => format!("{}-{}", base, n),            };            n += 1;        }        let path = std::path::Path::new(folder).join(&name);        std::fs::write(&path, a.contents()).map_err(|e| format!("{}: {}", path.display(), e))?;        out.push(json!({ "name": name, "path": path.to_string_lossy(), "size": a.contents().len() }));    }    Ok(out)}
Thread
86Useful? Like it, or add to its thread below.

Thread 0 posts

On Email reader: questions, tips, corrections, and code that uses it.

No posts yet. The first one opens the thread.