Data formats

CBOR

CBOR (RFC 8949), the binary format of passkeys, COSE and IoT devices: values to CBOR and back, as base64 or files.

Version 1.0.0 · 4 functions · 2 crates

#binary#passkeys#iot#import

Functions

What use cbor.qwer gives a program, with the types Quillo sees. To take only some, or under other names: use cbor_encode, cbor_decode from cbor (Quillo 0.6).

cbor_encode(value: any) → stror an error

value as CBOR, in base64.

cbor_decode(base64_text: str) → anyor an error

The value in CBOR given as base64 or base64url (as WebAuthn's attestationObject comes). Byte strings come back as base64 text, tagged values as their value, map keys that are numbers (COSE keys: 1, -2) as text.

cbor_write_file(path: str, value: any) → intor an error

Writes value as CBOR to path (folders made); gives the bytes written.

cbor_read_file(path: str) → anyor an error

The value in a CBOR file.

Get it

$ quillo install cbor               # cbor.qwer into this folder$ quillo install cbor --example     # and its example, cbor.qwe

Or with curl:

$ curl -O https://quillolang.org/plugins/files/cbor.qwer$ curl -O https://quillolang.org/plugins/files/cbor.qwe

Then, in your program, all of it or only what you call:

main.qwe
use cbor.qweruse cbor_encode, cbor_decode from cbor      // Quillo 0.6: only these

It uses these crates, which cargo fetches the first time the plugin is built:

[dependencies]
ciborium = "0.2"base64 = "0.22"

Example

cbor.qwe
// Reading the COSE public key a passkey sends at registration, and CBOR both ways.////   quillo run cbor.qwe      then open http://localhost:8080/coseuse "cbor.qwer"// A COSE EC2 key as WebAuthn sends it, base64url: { 1: 2 (EC2), 3: -7 (ES256), -1: 1 (P-256), -2: x, -3: y }.let COSE_KEY = "pQECAyYgASFYIAABAgMEBQYHCAkKCwwNDg8QERITFBUWFxgZGhscHR4fIlggICEiIyQlJicoKSorLC0uLzAxMjM0NTY3ODk6Ozw9Pj8"route GET /cose {    let k = cbor_decode(COSE_KEY)?    { key_type: k["1"] == 2 ? "EC2" : "other", algorithm: k["3"] == -7 ? "ES256" : "other", curve: k["-1"] == 1 ? "P-256" : "other", x: k["-2"] }}route GET /roundtrip {    let reading = { sensor: "greenhouse-3", celsius: 21.5, humidity: 64, at: 1793530800, ok: true }    let bytes = cbor_write_file("data/reading.cbor", reading)?    { bytes, json_bytes: len(json.stringify(reading)), back: cbor_read_file("data/reading.cbor")?, base64: cbor_encode(reading)? }}
$ quillo run cbor.qwe$ curl 'localhost:8080/cose'$ curl 'localhost:8080/roundtrip'

Source

cbor.qwer
//! CBOR (RFC 8949), the binary format of WebAuthn passkeys, COSE and IoT devices: values to CBOR as base64 or a file, and CBOR read back as values.//! version: 1.0.0//!//! [dependencies]//! ciborium = "0.2"//! base64 = "0.22"use base64::engine::general_purpose::{STANDARD, URL_SAFE_NO_PAD};use base64::Engine;use ciborium::Value as C;use serde_json::{json, Map, Value};/// At most 64 MiB of CBOR is read or written.const MAX: usize = 64 * 1024 * 1024;fn key_text(k: C) -> String {    match k {        C::Text(s) => s,        C::Integer(i) => i128::from(i).to_string(),        C::Bytes(b) => STANDARD.encode(b),        C::Bool(b) => b.to_string(),        C::Float(f) => f.to_string(),        other => format!("{:?}", other),    }}fn to_json(v: C, depth: usize) -> Value {    if depth > 256 {        return Value::Null;    }    match v {        C::Null => Value::Null,        C::Bool(b) => Value::Bool(b),        C::Integer(i) => {            let n = i128::from(i);            i64::try_from(n).map(Value::from).or_else(|_| u64::try_from(n).map(Value::from)).unwrap_or_else(|_| json!(n as f64))        }        C::Float(f) => json!(f),        C::Text(s) => Value::String(s),        // Byte strings have no JSON form: they come back as base64 text.        C::Bytes(b) => Value::String(STANDARD.encode(b)),        C::Array(a) => Value::Array(a.into_iter().map(|x| to_json(x, depth + 1)).collect()),        C::Map(m) => {            let mut out = Map::new();            for (k, v) in m {                out.insert(key_text(k), to_json(v, depth + 1));            }            Value::Object(out)        }        // A tag (0: a date as text, 1: a date as seconds, 32: a URL...) gives its value.        C::Tag(_, inner) => to_json(*inner, depth + 1),        _ => Value::Null,    }}fn decode(bytes: &[u8]) -> Result<Value, String> {    if bytes.len() > MAX {        return Err("CBOR over 64 MiB".into());    }    let v: C = ciborium::from_reader(bytes).map_err(|e| format!("not CBOR: {}", e))?;    Ok(to_json(v, 0))}fn encode(value: &Value) -> Result<Vec<u8>, String> {    let mut out = Vec::new();    ciborium::into_writer(value, &mut out).map_err(|e| e.to_string())?;    if out.len() > MAX { Err("CBOR over 64 MiB".into()) } else { Ok(out) }}fn unbase64(t: &str) -> Result<Vec<u8>, String> {    let t = t.trim();    STANDARD.decode(t).or_else(|_| URL_SAFE_NO_PAD.decode(t.trim_end_matches('='))).map_err(|_| "not base64".to_string())}/// `value` as CBOR, in base64.pub fn cbor_encode(value: serde_json::Value) -> Result<String, String> {    Ok(STANDARD.encode(encode(&value)?))}/// The value in CBOR given as base64 or base64url (as WebAuthn's `attestationObject` comes). Byte strings come back as base64/// text, tagged values as their value, map keys that are numbers (COSE keys: `1`, `-2`) as text.pub fn cbor_decode(base64_text: &str) -> Result<Value, String> {    decode(&unbase64(base64_text)?)}/// Writes `value` as CBOR to `path` (folders made); gives the bytes written.pub fn cbor_write_file(path: &str, value: serde_json::Value) -> Result<i64, String> {    let b = encode(&value)?;    if let Some(dir) = std::path::Path::new(path).parent() {        if !dir.as_os_str().is_empty() {            std::fs::create_dir_all(dir).map_err(|e| format!("{}: {}", dir.display(), e))?;        }    }    std::fs::write(path, &b).map_err(|e| format!("{}: {}", path, e))?;    Ok(b.len() as i64)}/// The value in a CBOR file.pub fn cbor_read_file(path: &str) -> Result<Value, String> {    let size = std::fs::metadata(path).map_err(|e| format!("{}: {}", path, e))?.len();    if size as usize > MAX {        return Err(format!("{}: over 64 MiB", path));    }    decode(&std::fs::read(path).map_err(|e| format!("{}: {}", path, e))?)}
BSON bson.qwer · 1.0.0BSON, MongoDB's binary format: documents to BSON and back as Extended JSON, and ObjectIds made and read.mongodbbinaryidsimport6 views · 6 downloadsProtobuf wire format protobuf_wire.qwer · 1.0.0Protocol Buffers without generated code: any message decoded field by field with no schema, decoded and encoded by name with the field numbers given, and length-delimited files split.binaryapiimportintegration7 views · 8 downloadsAmazon Ion amazon_ion.qwer · 1.0.0Amazon Ion 1.0 text and binary read and written, with a value form that keeps every Ion type: annotations, symbols and local symbol tables, typed nulls, decimals with their exact digits, timestamps at each precision and offset, integers of any size, blobs, clobs, sexps, long strings. Plain JSON both ways, and lengths past the end or nesting too deep refused.ionbinaryserializationaws6 views · 8 downloadsApache Arrow IPC arrow_ipc.qwer · 1.0.0Apache Arrow IPC stream and file formats read and written, through the official Rust crates: schemas, record batches split by size, rows to columns and back with nulls, lists, structs, dictionaries, decimals, dates and timestamps, LZ4 compression, and a schema inferred from rows. Values that do not fit their type name the row, and damaged data is an error.arrowcolumnaranalyticsbinary4 views · 7 downloadsApache Thrift thrift.qwer · 1.0.0Apache Thrift with no generated code: the binary and compact protocols encoded and decoded from a schema-less value form, field deltas, zigzag ints, bools in field headers and little-endian doubles included; TMessage framing, the old unversioned form too; and calls to a framed server with results, declared exceptions and TApplicationException read.rpcbinaryserializationservices5 views · 7 downloadsASN.1 DER and BER asn1_der.qwer · 1.0.0ASN.1 read to a tree and written back, with DER's rules enforced and BER's looseness when asked: tags of any size, OIDs with a table of names, UTCTime and GeneralizedTime, integers of any size, every string type, SETs sorted, PEM, a dump in the manner of `openssl asn1parse`. Indefinite lengths are refused in DER, and bad lengths, tags and values are errors naming their offset, never panics.asn1certificatesbinaryx5094 views · 8 downloads
Thread
77Useful? Like it, or add to its thread below.

Thread 0 posts

On CBOR: questions, tips, corrections, and code that uses it.

No posts yet. The first one opens the thread.