MessagePack
MessagePack, the compact binary JSON: values to MessagePack as base64 or a file, and back.
Version 1.0.0 · 4 functions · 3 crates
Functions
What use msgpack.qwer gives a program, with the types Quillo sees. To take only some, or under other names: use msgpack_encode, msgpack_decode from msgpack (Quillo 0.6).
msgpack_encode(value: any) → stror an errorvalue as MessagePack, in base64: maps keep their keys' names, ints stay ints and floats floats.
msgpack_decode(base64_text: str) → anyor an errorThe value in MessagePack given as base64 (standard or URL-safe). Binary fields come back as base64 text, extension types as { ext_type, data }, and map keys that are not text as their text.
msgpack_write_file(path: str, value: any) → intor an errorWrites value as MessagePack to the file at path (folders made); gives the bytes written.
msgpack_read_file(path: str) → anyor an errorThe value in a MessagePack file.
Get it
$ quillo install msgpack # msgpack.qwer into this folder$ quillo install msgpack --example # and its example, msgpack.qweOr with curl:
$ curl -O https://quillolang.org/plugins/files/msgpack.qwer$ curl -O https://quillolang.org/plugins/files/msgpack.qweThen, in your program, all of it or only what you call:
use msgpack.qweruse msgpack_encode, msgpack_decode from msgpack // Quillo 0.6: only theseIt uses these crates, which cargo fetches the first time the plugin is built:
rmp-serde = "1"rmpv = "1"base64 = "0.22"Example
// A cache of orders kept as MessagePack, smaller than JSON, and read back.//// quillo run msgpack.qwe then open http://localhost:8080/sizesuse "msgpack.qwer"let orders = range(50) | map((i) => { id: i, customer: `customer-${i}`, total: 10.5 + i, paid: i % 2 == 0, items: [i, i + 1] })route GET /sizes { let bytes = msgpack_write_file("data/orders.msgpack", orders)? let back = msgpack_read_file("data/orders.msgpack")? { msgpack_bytes: bytes, json_bytes: len(json.stringify(orders)), same: back == orders, third: back[2] }}route GET /roundtrip { let packed = msgpack_encode({ event: "signup", user: 7, at: "2026-11-02T09:00:00Z", tags: ["beta"] })? { packed, back: msgpack_decode(packed)? }}$ quillo run msgpack.qwe$ curl 'localhost:8080/sizes'$ curl 'localhost:8080/roundtrip'Source
//! MessagePack, the compact binary JSON: values to MessagePack (as base64 or a file) and back, for caches, queues and APIs that speak it.//! version: 1.0.0//!//! [dependencies]//! rmp-serde = "1"//! rmpv = "1"//! base64 = "0.22"use base64::engine::general_purpose::{STANDARD, URL_SAFE_NO_PAD};use base64::Engine;use serde_json::{json, Map, Value};/// At most 64 MiB of MessagePack is read or written.const MAX: usize = 64 * 1024 * 1024;fn to_json(v: rmpv::Value) -> Value { use rmpv::Value as M; match v { M::Nil => Value::Null, M::Boolean(b) => Value::Bool(b), M::Integer(i) => i.as_i64().map(Value::from).or_else(|| i.as_u64().map(Value::from)).or_else(|| i.as_f64().map(Value::from)).unwrap_or(Value::Null), M::F32(f) => json!(f as f64), M::F64(f) => json!(f), M::String(s) => Value::String(s.into_str().unwrap_or_default()), // Bytes have no JSON form: they come back as base64 text. M::Binary(b) => Value::String(STANDARD.encode(b)), M::Array(a) => Value::Array(a.into_iter().map(to_json).collect()), M::Map(m) => { let mut out = Map::new(); for (k, v) in m { let key = match k { M::String(s) => s.into_str().unwrap_or_default(), other => other.to_string(), }; out.insert(key, to_json(v)); } Value::Object(out) } M::Ext(t, data) => json!({ "ext_type": t, "data": STANDARD.encode(data) }), }}fn encode(value: &Value) -> Result<Vec<u8>, String> { let b = rmp_serde::to_vec_named(value).map_err(|e| e.to_string())?; if b.len() > MAX { Err("MessagePack over 64 MiB".into()) } else { Ok(b) }}fn decode(bytes: &[u8]) -> Result<Value, String> { if bytes.len() > MAX { return Err("MessagePack over 64 MiB".into()); } let mut r = bytes; let v = rmpv::decode::read_value(&mut r).map_err(|e| format!("not MessagePack: {}", e))?; Ok(to_json(v))}/// `value` as MessagePack, in base64: maps keep their keys' names, ints stay ints and floats floats.pub fn msgpack_encode(value: serde_json::Value) -> Result<String, String> { Ok(STANDARD.encode(encode(&value)?))}/// The value in MessagePack given as base64 (standard or URL-safe). Binary fields come back as base64 text, extension/// types as `{ ext_type, data }`, and map keys that are not text as their text.pub fn msgpack_decode(base64_text: &str) -> Result<Value, String> { let t = base64_text.trim(); let bytes = STANDARD.decode(t).or_else(|_| URL_SAFE_NO_PAD.decode(t.trim_end_matches('='))).map_err(|_| "not base64".to_string())?; decode(&bytes)}/// Writes `value` as MessagePack to the file at `path` (folders made); gives the bytes written.pub fn msgpack_write_file(path: &str, value: serde_json::Value) -> Result<i64, String> { let b = encode(&value)?; if let Some(dir) = std::path::Path::new(path).parent() { if !dir.as_os_str().is_empty() { std::fs::create_dir_all(dir).map_err(|e| format!("{}: {}", dir.display(), e))?; } } std::fs::write(path, &b).map_err(|e| format!("{}: {}", path, e))?; Ok(b.len() as i64)}/// The value in a MessagePack file.pub fn msgpack_read_file(path: &str) -> Result<Value, String> { let size = std::fs::metadata(path).map_err(|e| format!("{}: {}", path, e))?.len(); if size as usize > MAX { return Err(format!("{}: over 64 MiB", path)); } decode(&std::fs::read(path).map_err(|e| format!("{}: {}", path, e))?)}Related
protobuf_wire.qwer · 1.0.0Protocol Buffers without generated code: any message decoded field by field with no schema, decoded and encoded by name with the field numbers given, and length-delimited files split.7 views · 8 downloadsAmazon Ion amazon_ion.qwer · 1.0.0Amazon Ion 1.0 text and binary read and written, with a value form that keeps every Ion type: annotations, symbols and local symbol tables, typed nulls, decimals with their exact digits, timestamps at each precision and offset, integers of any size, blobs, clobs, sexps, long strings. Plain JSON both ways, and lengths past the end or nesting too deep refused.6 views · 8 downloadsApache Arrow IPC arrow_ipc.qwer · 1.0.0Apache Arrow IPC stream and file formats read and written, through the official Rust crates: schemas, record batches split by size, rows to columns and back with nulls, lists, structs, dictionaries, decimals, dates and timestamps, LZ4 compression, and a schema inferred from rows. Values that do not fit their type name the row, and damaged data is an error.4 views · 7 downloadsApache Thrift thrift.qwer · 1.0.0Apache Thrift with no generated code: the binary and compact protocols encoded and decoded from a schema-less value form, field deltas, zigzag ints, bools in field headers and little-endian doubles included; TMessage framing, the old unversioned form too; and calls to a framed server with results, declared exceptions and TApplicationException read.5 views · 7 downloadsASN.1 DER and BER asn1_der.qwer · 1.0.0ASN.1 read to a tree and written back, with DER's rules enforced and BER's looseness when asked: tags of any size, OIDs with a table of names, UTCTime and GeneralizedTime, integers of any size, every string type, SETs sorted, PEM, a dump in the manner of `openssl asn1parse`. Indefinite lengths are refused in DER, and bad lengths, tags and values are errors naming their offset, never panics.4 views · 8 downloadsAvro avro.qwer · 1.0.0Apache Avro data files: rows written with a schema, as Kafka, Hadoop and BigQuery use, and read back with the schema they carry.9 views · 6 downloadsThread 0 posts
On MessagePack: questions, tips, corrections, and code that uses it.
No posts yet. The first one opens the thread.