Data formats

PEM

PEM bundles read block by block (certificates, keys and requests, with their labels, headers, key algorithm and SHA-256), blocks picked out, written, and turned into DER files and back.

Version 1.0.0 · 7 functions · 3 crates

#certificates#keys#security#encoding

Functions

What use pem.qwer gives a program, with the types Quillo sees. To take only some, or under other names: use pem_blocks, pem_read_file from pem (Quillo 0.6).

pem_blocks(text: str) → listor an error

Every PEM block in text, in order: [{ index, label, kind, algorithm, encrypted, headers, bytes, sha256, base64 }]. label is what follows BEGIN ("CERTIFICATE", "PRIVATE KEY"...); kind says it in words; algorithm is the key's ("RSA", "EC P-256", "Ed25519"...) when it can be seen; headers are the old RFC 1421 ones (Proc-Type, DEK-Info); bytes and sha256 are the DER's length and hash (a certificate's fingerprint). Text between blocks is skipped.

pem_read_file(path: str) → listor an error

pem_blocks of a file (a .pem, .crt, .key or CA bundle), up to 32 MB.

pem_only(text: str, labels: list) → stror an error

The blocks of text whose label is one of labels ("CERTIFICATE", "PRIVATE KEY"...; case does not matter), as PEM text again, in their order: the chain from a file that also holds the key, or the key alone. Empty when none matches.

pem_encode(label: str, der_base64: str, headers: map) → stror an error

One PEM block: label after BEGIN, der_base64 the content as base64, headers ({} mostly; Proc-Type is written first, as RFC 1421 wants). Lines of 64 characters, \n line ends.

pem_decode(text: str, label: str) → stror an error

The DER of the first block labelled label in text ("" for the first block of any label), as base64.

pem_to_der_file(text: str, label: str, out: str) → intor an error

Writes the DER of the first block labelled label ("" for any) to the file out (a .der or .cer, folders made): the bytes written.

pem_from_der_file(path: str, label: str) → stror an error

A DER file (.der, .cer, a key from Java or Windows) as one PEM block labelled label ("CERTIFICATE", "PRIVATE KEY"...). A file that already is PEM is an error, to catch the wrong file.

Get it

$ quillo install pem               # pem.qwer into this folder$ quillo install pem --example     # and its example, pem.qwe

Or with curl:

$ curl -O https://quillolang.org/plugins/files/pem.qwer$ curl -O https://quillolang.org/plugins/files/pem.qwe$ curl -O https://quillolang.org/plugins/files/pem_sample.pem

Then, in your program, all of it or only what you call:

main.qwe
use pem.qweruse pem_blocks, pem_read_file from pem      // Quillo 0.6: only these

It uses these crates, which cargo fetches the first time the plugin is built:

[dependencies]
pem = "3.0.6"base64 = "0.22"sha2 = "0.10"

Example

pem.qwe
// A shop's TLS bundle looked into: what each block is, the certificate picked out, written as DER and read back.////   quillo run pem.qwe      then open http://localhost:8080/bundleuse "pem.qwer"route GET /bundle = pem_read_file("pem_sample.pem")? | map((b) => { label: b.label, kind: b.kind, algorithm: b.algorithm, encrypted: b.encrypted, headers: b.headers, bytes: b.bytes, sha256: b.sha256 })// The certificate alone, to a .der file for a Java keystore, and back to PEM: the same fingerprint.route GET /convert {    let text = read_file("pem_sample.pem")    let cert = pem_only(text, ["certificate"])?    let written = pem_to_der_file(text, "CERTIFICATE", "data/shop.der")?    let again = pem_from_der_file("data/shop.der", "CERTIFICATE")?    {        der_bytes: written,        same: again == cert,        fingerprint: pem_blocks(again)?[0].sha256,        keys_only: len(pem_blocks(pem_only(text, ["PUBLIC KEY", "RSA PRIVATE KEY"])?)?),    }}// A block of your own, with the old-style headers.route GET /write = respond(200, pem_encode("QUILLO LICENSE", "SGVsbG8sIFF1aWxsbyE=", { "Proc-Type": "4,CRL", "Comment": "licence for shop.example" })?, { "content-type": "text/plain" })
$ quillo run pem.qwe$ curl 'localhost:8080/bundle'$ curl 'localhost:8080/convert'$ curl 'localhost:8080/write'

Source

pem.qwer
//! PEM blocks of any kind read and written: what a bundle holds (certificates, keys, requests, each with its label, headers, size and SHA-256), blocks picked out of it, and DER files to PEM and back.//! version: 1.0.0//!//! [dependencies]//! pem = "3.0.6"//! base64 = "0.22"//! sha2 = "0.10"use base64::engine::general_purpose::STANDARD;use base64::Engine;use serde_json::{json, Map, Value};use sha2::{Digest, Sha256};const MAX: usize = 32 * 1024 * 1024;fn has(der: &[u8], oid: &[u8]) -> bool {    der.windows(oid.len()).any(|w| w == oid)}/// The key algorithm named inside a block's DER, by its object identifier: "RSA", "EC P-256", "Ed25519"...fn algorithm(der: &[u8]) -> Option<String> {    let curves: [(&[u8], &str); 3] = [        (&[0x06, 0x08, 0x2a, 0x86, 0x48, 0xce, 0x3d, 0x03, 0x01, 0x07], "P-256"),        (&[0x06, 0x05, 0x2b, 0x81, 0x04, 0x00, 0x22], "P-384"),        (&[0x06, 0x05, 0x2b, 0x81, 0x04, 0x00, 0x23], "P-521"),    ];    if has(der, &[0x06, 0x09, 0x2a, 0x86, 0x48, 0x86, 0xf7, 0x0d, 0x01, 0x01, 0x01]) {        return Some("RSA".into());    }    if has(der, &[0x06, 0x07, 0x2a, 0x86, 0x48, 0xce, 0x3d, 0x02, 0x01]) || curves.iter().any(|(o, _)| has(der, o)) {        let curve = curves.iter().find(|(o, _)| has(der, o)).map(|(_, n)| *n).unwrap_or("other curve");        return Some(format!("EC {}", curve));    }    for (oid, name) in [([0x06u8, 0x03, 0x2b, 0x65, 0x70], "Ed25519"), ([0x06, 0x03, 0x2b, 0x65, 0x71], "Ed448"), ([0x06, 0x03, 0x2b, 0x65, 0x6e], "X25519"), ([0x06, 0x03, 0x2b, 0x65, 0x6f], "X448")] {        if has(der, &oid) {            return Some(name.into());        }    }    None}fn describe(p: &pem::Pem, index: usize) -> Value {    let der = p.contents();    let label = p.tag().to_string();    let encrypted = label.contains("ENCRYPTED") || p.headers().get("Proc-Type").is_some_and(|v| v.contains("ENCRYPTED"));    let alg = match label.as_str() {        "RSA PRIVATE KEY" | "RSA PUBLIC KEY" => Some("RSA".to_string()),        "EC PRIVATE KEY" => algorithm(der).or(Some("EC".into())),        "ENCRYPTED PRIVATE KEY" | "OPENSSH PRIVATE KEY" => None,        _ if encrypted => None,        _ => algorithm(der),    };    let kind = match label.as_str() {        "CERTIFICATE" | "X509 CERTIFICATE" => "certificate",        "TRUSTED CERTIFICATE" => "trusted certificate (OpenSSL)",        "CERTIFICATE REQUEST" | "NEW CERTIFICATE REQUEST" => "certificate request",        "X509 CRL" => "certificate revocation list",        "PRIVATE KEY" => "private key (PKCS#8)",        "ENCRYPTED PRIVATE KEY" => "encrypted private key (PKCS#8)",        "RSA PRIVATE KEY" => "private key (PKCS#1)",        "EC PRIVATE KEY" => "private key (SEC1)",        "PUBLIC KEY" => "public key (SPKI)",        "RSA PUBLIC KEY" => "public key (PKCS#1)",        "OPENSSH PRIVATE KEY" => "OpenSSH private key",        "DH PARAMETERS" => "Diffie-Hellman parameters",        "EC PARAMETERS" => "EC parameters",        "PKCS7" | "CMS" => "signed or enveloped data (PKCS#7 / CMS)",        "ATTRIBUTE CERTIFICATE" => "attribute certificate",        _ => "other",    };    let mut headers = Map::new();    for (k, v) in p.headers().iter() {        headers.insert(k.to_string(), Value::String(v.to_string()));    }    json!({        "index": index,        "label": label,        "kind": kind,        "algorithm": alg,        "encrypted": encrypted,        "headers": headers,        "bytes": der.len(),        "sha256": Sha256::digest(der).iter().map(|b| format!("{:02x}", b)).collect::<String>(),        "base64": STANDARD.encode(der),    })}fn parse(text: &str) -> Result<Vec<pem::Pem>, String> {    if text.len() > MAX {        return Err("the text is over 32 MB".into());    }    let blocks = pem::parse_many(text).map_err(|e| format!("not PEM: {}", e))?;    if blocks.is_empty() {        return Err("no PEM block (-----BEGIN ...-----) in the text".into());    }    Ok(blocks)}fn lf(p: &pem::Pem) -> String {    pem::encode_config(p, pem::EncodeConfig::new().set_line_ending(pem::LineEnding::LF))}/// Every PEM block in `text`, in order: `[{ index, label, kind, algorithm, encrypted, headers, bytes, sha256, base64 }]`./// `label` is what follows BEGIN ("CERTIFICATE", "PRIVATE KEY"...); `kind` says it in words; `algorithm` is the key's/// ("RSA", "EC P-256", "Ed25519"...) when it can be seen; `headers` are the old RFC 1421 ones (`Proc-Type`, `DEK-Info`);/// `bytes` and `sha256` are the DER's length and hash (a certificate's fingerprint). Text between blocks is skipped.pub fn pem_blocks(text: &str) -> Result<Vec<Value>, String> {    Ok(parse(text)?.iter().enumerate().map(|(i, p)| describe(p, i)).collect())}/// `pem_blocks` of a file (a .pem, .crt, .key or CA bundle), up to 32 MB.pub fn pem_read_file(path: &str) -> Result<Vec<Value>, String> {    let text = std::fs::read_to_string(path).map_err(|e| format!("{}: {}", path, e))?;    pem_blocks(&text)}/// The blocks of `text` whose label is one of `labels` ("CERTIFICATE", "PRIVATE KEY"...; case does not matter), as PEM/// text again, in their order: the chain from a file that also holds the key, or the key alone. Empty when none matches.pub fn pem_only(text: &str, labels: Vec<String>) -> Result<String, String> {    let want: Vec<String> = labels.iter().map(|l| l.trim().to_uppercase()).collect();    Ok(parse(text)?.iter().filter(|p| want.contains(&p.tag().to_uppercase())).map(lf).collect())}/// One PEM block: `label` after BEGIN, `der_base64` the content as base64, `headers` (`{}` mostly; `Proc-Type` is/// written first, as RFC 1421 wants). Lines of 64 characters, `\n` line ends.pub fn pem_encode(label: &str, der_base64: &str, headers: std::collections::HashMap<String, String>) -> Result<String, String> {    let label = label.trim();    if label.is_empty() || label.contains('-') || label.contains('\n') {        return Err(format!("`{}` is not a PEM label", label));    }    let clean: String = der_base64.chars().filter(|c| !c.is_whitespace()).collect();    let der = STANDARD.decode(clean).map_err(|e| format!("der_base64 is not base64: {}", e))?;    let mut p = pem::Pem::new(label, der);    let mut keys: Vec<&String> = headers.keys().collect();    keys.sort_by_key(|k| (k.as_str() != "Proc-Type", k.to_string()));    for k in keys {        p.headers_mut().add(k, &headers[k]).map_err(|e| format!("header `{}`: {}", k, e))?;    }    Ok(lf(&p))}/// The DER of the first block labelled `label` in `text` ("" for the first block of any label), as base64.pub fn pem_decode(text: &str, label: &str) -> Result<String, String> {    let blocks = parse(text)?;    let p = blocks.iter().find(|p| label.is_empty() || p.tag().eq_ignore_ascii_case(label.trim())).ok_or_else(|| format!("no block labelled `{}`", label))?;    Ok(STANDARD.encode(p.contents()))}/// Writes the DER of the first block labelled `label` ("" for any) to the file `out` (a .der or .cer, folders made):/// the bytes written.pub fn pem_to_der_file(text: &str, label: &str, out: &str) -> Result<i64, String> {    let der = STANDARD.decode(pem_decode(text, label)?).map_err(|e| e.to_string())?;    if let Some(dir) = std::path::Path::new(out).parent().filter(|d| !d.as_os_str().is_empty()) {        std::fs::create_dir_all(dir).map_err(|e| format!("{}: {}", dir.display(), e))?;    }    std::fs::write(out, &der).map_err(|e| format!("{}: {}", out, e))?;    Ok(der.len() as i64)}/// A DER file (.der, .cer, a key from Java or Windows) as one PEM block labelled `label` ("CERTIFICATE", "PRIVATE KEY"...)./// A file that already is PEM is an error, to catch the wrong file.pub fn pem_from_der_file(path: &str, label: &str) -> Result<String, String> {    let der = std::fs::read(path).map_err(|e| format!("{}: {}", path, e))?;    if der.len() > MAX {        return Err(format!("{}: over 32 MB", path));    }    if der.starts_with(b"-----BEGIN") {        return Err(format!("{} is PEM already", path));    }    if der.first() != Some(&0x30) {        return Err(format!("{} is not DER (it does not start with a SEQUENCE)", path));    }    pem_encode(label, &STANDARD.encode(&der), Default::default())}
Thread
66Useful? Like it, or add to its thread below.

Thread 0 posts

On PEM: questions, tips, corrections, and code that uses it.

No posts yet. The first one opens the thread.