Data formats

BibTeX

BibTeX and BibLaTeX bibliographies read, with LaTeX accents and macros resolved; reference lists in APA or IEEE style, CSL-JSON for Zotero and pandoc, and entries written back.

Version 1.0.0 · 5 functions · 2 crates

#books#import#export#writing#docs

Functions

What use bibtex.qwer gives a program, with the types Quillo sees. To take only some, or under other names: use bibtex_parse, bibtex_read_file from bibtex (Quillo 0.6).

bibtex_parse(text: str) → listor an error

The entries of a BibTeX or BibLaTeX text, in order: { key, type, title, authors, editors, year, month, journal, booktitle, publisher, volume, number, pages, doi, url, fields }. People are { family, given, prefix, suffix } ("von Neumann, John" has prefix "von"); LaTeX accents (G{\"o}del) and @string macros are resolved; month is 1 to 12; fields has every field as plain text.

bibtex_read_file(path: str) → listor an error

The entries of the .bib file at path, as bibtex_parse gives them.

bibtex_format(text: str, style: str) → listor an error

A reference list from a .bib text, in style "apa" (sorted by author and year, titles in sentence case) or "ieee" (numbered, in the file's order): [{ key, text, html }], html with titles of books and journals in <i>.

bibtex_to_csl(text: str) → listor an error

A .bib file's entries as CSL-JSON items, what Zotero, pandoc and citeproc read: [{ id, type, title, author, editor, issued, container-title, volume, issue, page, publisher, DOI, URL }].

bibtex_write(entries: list) → stror an error

Entries as BibTeX text: each { key, type, fields } (as bibtex_parse gives, or made by hand: { key: "lovelace1843", type: "article", fields: { author: "Lovelace, Ada", title: "...", year: "1843" } }), fields in the order given, with & % $ # _ escaped and braces dropped from values.

Get it

$ quillo install bibtex               # bibtex.qwer into this folder$ quillo install bibtex --example     # and its example, bibtex.qwe

Or with curl:

$ curl -O https://quillolang.org/plugins/files/bibtex.qwer$ curl -O https://quillolang.org/plugins/files/bibtex.qwe$ curl -O https://quillolang.org/plugins/files/bibtex_sample.bib

Then, in your program, all of it or only what you call:

main.qwe
use bibtex.qweruse bibtex_parse, bibtex_read_file from bibtex      // Quillo 0.6: only these

It uses these crates, which cargo fetches the first time the plugin is built:

[dependencies]
biblatex = "0.10"serde_json = { version = "1", features = ["preserve_order"] }

Example

bibtex.qwe
// A thesis's bibliography: the .bib file read, its reference list in APA and IEEE, CSL-JSON for pandoc, and a new entry written.////   quillo run bibtex.qwe      then open http://localhost:8080/apause "bibtex.qwer"route GET /entries = bibtex_read_file("bibtex_sample.bib")? | map((e) => { key: e.key, type: e.type, title: e.title, authors: e.authors | map((p) => p.family), year: e.year, month: e.month, journal: e.journal })route GET /apa = bibtex_format(read_file("bibtex_sample.bib")?, "apa")? | map((r) => r.text)route GET /ieee = bibtex_format(read_file("bibtex_sample.bib")?, "ieee")?route GET /csl = bibtex_to_csl(read_file("bibtex_sample.bib")?)?[2]route GET /write = respond(200, bibtex_write([{ key: "lovelace1843", type: "article", fields: { author: "Lovelace, Ada", title: "Notes on the Analytical Engine", journal: "Taylor's Scientific Memoirs", year: "1843", note: "Notes A & G" } }])?, { "content-type": "text/plain; charset=utf-8" })
$ quillo run bibtex.qwe$ curl 'localhost:8080/entries'$ curl 'localhost:8080/apa'$ curl 'localhost:8080/ieee'$ curl 'localhost:8080/write'

Source

bibtex.qwer
//! BibTeX and BibLaTeX bibliographies: entries read (LaTeX accents and @string macros resolved), references formatted in APA or IEEE style, written back, and turned into CSL-JSON for Zotero and pandoc.//! version: 1.0.0//!//! [dependencies]//! biblatex = "0.10"//! serde_json = { version = "1", features = ["preserve_order"] }use biblatex::{Bibliography, ChunksExt, Entry, Person};use serde_json::{json, Map, Value};const MONTHS: [&str; 12] = ["Jan.", "Feb.", "Mar.", "Apr.", "May", "Jun.", "Jul.", "Aug.", "Sep.", "Oct.", "Nov.", "Dec."];fn parse(text: &str) -> Result<Bibliography, String> {    Bibliography::parse(text).map_err(|e| {        let line = text[..e.span.start.min(text.len())].matches('\n').count() + 1;        format!("line {}: {}", line, e.kind)    })}fn field(e: &Entry, names: &[&str]) -> Option<String> {    names.iter().find_map(|n| e.fields.get(*n).map(|c| c.format_verbatim())).filter(|s| !s.trim().is_empty())}fn people(p: Result<Vec<Person>, biblatex::RetrievalError>) -> Vec<Person> {    p.unwrap_or_default()}fn editors(e: &Entry) -> Vec<Person> {    e.editors().map(|v| v.into_iter().flat_map(|(p, _)| p).collect()).unwrap_or_default()}fn year_month(e: &Entry) -> (Option<i64>, Option<i64>) {    use biblatex::{DateValue, PermissiveType};    match e.date() {        Ok(PermissiveType::Typed(d)) => {            let dt = match d.value {                DateValue::At(x) | DateValue::After(x) | DateValue::Before(x) | DateValue::Between(x, _) => x,            };            (Some(dt.year as i64), dt.month.map(|m| m as i64 + 1))        }        _ => (field(e, &["year"]).and_then(|y| y.trim().get(..4).and_then(|y| y.parse().ok())), None),    }}fn family(p: &Person) -> String {    if p.prefix.is_empty() { p.name.clone() } else { format!("{} {}", p.prefix, p.name) }}fn initials(given: &str) -> String {    given        .split_whitespace()        .map(|w| w.split('-').filter_map(|part| part.chars().next().map(|c| format!("{}.", c))).collect::<Vec<_>>().join("-"))        .collect::<Vec<_>>()        .join(" ")}fn person_json(p: &Person) -> Value {    json!({ "family": p.name, "given": p.given_name, "prefix": p.prefix, "suffix": p.suffix })}fn entry_json(e: &Entry) -> Value {    let (year, month) = year_month(e);    let mut fields = Map::new();    for (k, v) in &e.fields {        fields.insert(k.clone(), json!(v.format_verbatim()));    }    json!({        "key": e.key,        "type": e.entry_type.to_string(),        "title": field(e, &["title"]),        "authors": people(e.author()).iter().map(person_json).collect::<Vec<_>>(),        "editors": editors(e).iter().map(person_json).collect::<Vec<_>>(),        "year": year,        "month": month,        "journal": field(e, &["journal", "journaltitle"]),        "booktitle": field(e, &["booktitle"]),        "publisher": field(e, &["publisher"]),        "volume": field(e, &["volume"]),        "number": field(e, &["number"]),        "pages": field(e, &["pages"]),        "doi": field(e, &["doi"]),        "url": field(e, &["url"]),        "fields": fields,    })}fn and_list(names: &[String], amp: bool) -> String {    match names.len() {        0 => String::new(),        1 => names[0].clone(),        2 if !amp => format!("{} and {}", names[0], names[1]),        n => format!("{}, {} {}", names[..n - 1].join(", "), if amp { "&" } else { "and" }, names[n - 1]),    }}/// One reference: (plain text, HTML with the title or journal in italics).fn format_entry(e: &Entry, style: &str, n: usize) -> (String, String) {    let esc = |s: &str| s.replace('&', "&amp;").replace('<', "&lt;").replace('>', "&gt;");    let authors = people(e.author());    let (year, month) = year_month(e);    let title = field(e, &["title"]).unwrap_or_default();    let ty = e.entry_type.to_string();    let journal = field(e, &["journal", "journaltitle"]);    let book = field(e, &["booktitle"]);    let publisher = field(e, &["publisher", "institution", "school", "organization", "howpublished"]);    let (vol, num, pages) = (field(e, &["volume"]), field(e, &["number", "issue"]), field(e, &["pages"]));    let doi = field(e, &["doi"]);    let url = field(e, &["url"]);    // Pieces as (text, italic).    let mut parts: Vec<(String, bool)> = vec![];    let push = |parts: &mut Vec<(String, bool)>, s: String, it: bool| parts.push((s, it));    if style == "ieee" {        let names: Vec<String> = authors.iter().map(|p| format!("{} {}", initials(&p.given_name), family(p)).trim().to_string()).collect();        push(&mut parts, format!("[{}] ", n), false);        if !names.is_empty() {            push(&mut parts, format!("{}, ", and_list(&names, false)), false);        }        let when = match (month, year) {            (Some(m), Some(y)) => format!("{} {}", MONTHS[(m as usize - 1).min(11)], y),            (None, Some(y)) => y.to_string(),            _ => String::new(),        };        if ty == "book" {            push(&mut parts, title.clone(), true);            push(&mut parts, format!(". {}{}.", publisher.map(|p| format!("{}, ", p)).unwrap_or_default(), when), false);        } else {            push(&mut parts, format!("\u{201c}{},\u{201d} ", title), false);            if let Some(j) = journal.clone() {                push(&mut parts, j, true);                let mut t = String::new();                if let Some(v) = &vol { t.push_str(&format!(", vol. {}", v)); }                if let Some(x) = &num { t.push_str(&format!(", no. {}", x)); }                if let Some(p) = &pages { t.push_str(&format!(", pp. {}", p)); }                t.push_str(&format!(", {}", when));                push(&mut parts, t, false);            } else if let Some(b) = book.clone() {                push(&mut parts, "in ".into(), false);                push(&mut parts, b, true);                push(&mut parts, format!(", {}{}", when, pages.as_ref().map(|p| format!(", pp. {}", p)).unwrap_or_default()), false);            } else {                push(&mut parts, format!("{}{}", publisher.map(|p| format!("{}, ", p)).unwrap_or_default(), when), false);            }            match (&doi, &url) {                (Some(d), _) => push(&mut parts, format!(", doi: {}.", d), false),                (None, Some(u)) => push(&mut parts, format!(". [Online]. Available: {}", u), false),                _ => push(&mut parts, ".".into(), false),            }        }    } else {        let names: Vec<String> = authors.iter().map(|p| format!("{}, {}", family(p), initials(&p.given_name)).trim_end_matches(", ").to_string()).collect();        let who = if names.is_empty() { String::new() } else { format!("{} ", and_list(&names, true)) };        push(&mut parts, format!("{}({}). ", who, year.map(|y| y.to_string()).unwrap_or_else(|| "n.d.".into())), false);        let sentence = e.fields.get("title").map(|c| c.format_sentence()).unwrap_or_default();        let end = |s: &str| if s.ends_with(['.', '?', '!']) { "" } else { "." };        if ty == "book" {            push(&mut parts, sentence.clone(), true);            push(&mut parts, format!("{} {}", end(&sentence), publisher.map(|p| format!("{}.", p)).unwrap_or_default()).trim_end().to_string(), false);        } else if let Some(j) = journal {            push(&mut parts, format!("{}{} ", sentence, end(&sentence)), false);            push(&mut parts, j, true);            if let Some(v) = &vol {                push(&mut parts, ", ".into(), false);                push(&mut parts, v.clone(), true);            }            let mut t = String::new();            if let Some(x) = &num { t.push_str(&format!("({})", x)); }            if let Some(p) = &pages { t.push_str(&format!(", {}", p)); }            t.push('.');            push(&mut parts, t, false);        } else if let Some(b) = book {            push(&mut parts, format!("{}{} In ", sentence, end(&sentence)), false);            push(&mut parts, b, true);            push(&mut parts, format!("{}.{}", pages.as_ref().map(|p| format!(" (pp. {})", p)).unwrap_or_default(), publisher.map(|p| format!(" {}.", p)).unwrap_or_default()), false);        } else {            push(&mut parts, sentence.clone(), true);            push(&mut parts, format!("{}{}", end(&sentence), publisher.map(|p| format!(" {}.", p)).unwrap_or_default()), false);        }        match (&doi, &url) {            (Some(d), _) => push(&mut parts, format!(" https://doi.org/{}", d.trim_start_matches("https://doi.org/")), false),            (None, Some(u)) => push(&mut parts, format!(" {}", u), false),            _ => {}        }    }    let text: String = parts.iter().map(|(s, _)| s.as_str()).collect();    let html: String = parts.iter().map(|(s, it)| if *it && !s.is_empty() { format!("<i>{}</i>", esc(s)) } else { esc(s) }).collect();    (text, html)}fn csl_type(t: &str) -> &'static str {    match t {        "article" => "article-journal",        "book" | "mvbook" => "book",        "inbook" | "incollection" => "chapter",        "inproceedings" | "conference" => "paper-conference",        "proceedings" => "book",        "phdthesis" | "mastersthesis" | "thesis" => "thesis",        "techreport" | "report" => "report",        "online" | "electronic" | "www" => "webpage",        "manual" => "book",        "patent" => "patent",        "dataset" => "dataset",        "software" => "software",        _ => "document",    }}/// The entries of a BibTeX or BibLaTeX text, in order: `{ key, type, title, authors, editors, year, month, journal,/// booktitle, publisher, volume, number, pages, doi, url, fields }`. People are `{ family, given, prefix, suffix }`/// ("von Neumann, John" has prefix "von"); LaTeX accents (`G{\"o}del`) and `@string` macros are resolved; `month` is/// 1 to 12; `fields` has every field as plain text.pub fn bibtex_parse(text: &str) -> Result<Vec<Value>, String> {    Ok(parse(text)?.iter().map(entry_json).collect())}/// The entries of the .bib file at `path`, as `bibtex_parse` gives them.pub fn bibtex_read_file(path: &str) -> Result<Vec<Value>, String> {    let text = std::fs::read_to_string(path).map_err(|e| format!("{}: {}", path, e))?;    bibtex_parse(&text).map_err(|e| format!("{}: {}", path, e))}/// A reference list from a .bib text, in `style` "apa" (sorted by author and year, titles in sentence case) or "ieee"/// (numbered, in the file's order): `[{ key, text, html }]`, `html` with titles of books and journals in `<i>`.pub fn bibtex_format(text: &str, style: &str) -> Result<Vec<Value>, String> {    let style = style.to_lowercase();    if style != "apa" && style != "ieee" {        return Err(format!("unknown style `{}`: apa or ieee", style));    }    let bib = parse(text)?;    let mut entries: Vec<&Entry> = bib.iter().collect();    if style == "apa" {        entries.sort_by_key(|e| {            let first = people(e.author()).first().map(|p| p.name.to_lowercase()).unwrap_or_else(|| field(e, &["title"]).unwrap_or_default().to_lowercase());            (first, year_month(e).0.unwrap_or(i64::MAX))        });    }    Ok(entries        .iter()        .enumerate()        .map(|(i, e)| {            let (t, h) = format_entry(e, &style, i + 1);            json!({ "key": e.key, "text": t, "html": h })        })        .collect())}/// A .bib file's entries as CSL-JSON items, what Zotero, pandoc and citeproc read: `[{ id, type, title, author,/// editor, issued, container-title, volume, issue, page, publisher, DOI, URL }]`.pub fn bibtex_to_csl(text: &str) -> Result<Vec<Value>, String> {    let bib = parse(text)?;    Ok(bib        .iter()        .map(|e| {            let mut o = Map::new();            o.insert("id".into(), json!(e.key));            o.insert("type".into(), json!(csl_type(&e.entry_type.to_string())));            let names = |ps: Vec<Person>| -> Value {                ps.iter()                    .map(|p| {                        let mut n = Map::new();                        n.insert("family".into(), json!(p.name));                        n.insert("given".into(), json!(p.given_name));                        if !p.prefix.is_empty() { n.insert("non-dropping-particle".into(), json!(p.prefix)); }                        if !p.suffix.is_empty() { n.insert("suffix".into(), json!(p.suffix)); }                        Value::Object(n)                    })                    .collect()            };            let a = people(e.author());            if !a.is_empty() { o.insert("author".into(), names(a)); }            let ed = editors(e);            if !ed.is_empty() { o.insert("editor".into(), names(ed)); }            if let (Some(y), m) = year_month(e) {                let parts: Vec<i64> = std::iter::once(y).chain(m).collect();                o.insert("issued".into(), json!({ "date-parts": [parts] }));            }            for (csl, bib) in [("title", &["title"][..]), ("container-title", &["journal", "journaltitle", "booktitle"]), ("volume", &["volume"]), ("issue", &["number", "issue"]), ("page", &["pages"]), ("publisher", &["publisher", "institution", "school"]), ("DOI", &["doi"]), ("URL", &["url"]), ("ISBN", &["isbn"]), ("abstract", &["abstract"])] {                if let Some(v) = field(e, bib) {                    o.insert(csl.into(), json!(if csl == "page" { v.replace('\u{2013}', "-") } else { v }));                }            }            Value::Object(o)        })        .collect())}fn tex_escape(s: &str) -> String {    let mut o = String::new();    for c in s.chars() {        match c {            '&' | '%' | '$' | '#' | '_' => { o.push('\\'); o.push(c); }            '{' | '}' => {}            c => o.push(c),        }    }    o}/// Entries as BibTeX text: each `{ key, type, fields }` (as `bibtex_parse` gives, or made by hand: `{ key: "lovelace1843",/// type: "article", fields: { author: "Lovelace, Ada", title: "...", year: "1843" } }`), fields in the order given,/// with `& % $ # _` escaped and braces dropped from values.pub fn bibtex_write(entries: Vec<serde_json::Value>) -> Result<String, String> {    let mut out = String::new();    for (i, e) in entries.iter().enumerate() {        let key = e["key"].as_str().filter(|k| !k.is_empty() && !k.contains([',', ' ', '{', '}'])).ok_or_else(|| format!("entry {}: a `key` without spaces or commas", i))?;        let ty = e["type"].as_str().unwrap_or("misc");        let fields = e["fields"].as_object().ok_or_else(|| format!("entry `{}`: `fields` is a map", key))?;        out.push_str(&format!("@{}{{{},\n", ty, key));        for (k, v) in fields {            let v = match v {                Value::String(s) => s.clone(),                Value::Null => continue,                other => other.to_string(),            };            out.push_str(&format!("  {} = {{{}}},\n", k, tex_escape(&v)));        }        out.push_str("}\n\n");    }    parse(&out).map_err(|e| format!("the text written does not read back: {}", e))?;    Ok(out)}
Thread
76Useful? Like it, or add to its thread below.

Thread 0 posts

On BibTeX: questions, tips, corrections, and code that uses it.

No posts yet. The first one opens the thread.