MBTiles
MBTiles files read by a small SQLite reader of its own, in pure Rust and with no locks, so no bundled SQLite sits beside Quillo's: metadata, tiles by web-map z/x/y with the TMS row flip, the unique index walked where there is one, overflow pages followed, mbutil's `map` and `images` layout joined, and tiles written out for `file()`.
Version 1.0.0 · 5 functions · 3 crates
Functions
What use mbtiles.qwer gives a program, with the types Quillo sees. To take only some, or under other names: use mbtiles_info, mbtiles_tile from mbtiles (Quillo 0.6).
mbtiles_info(path: str) → anyor an errorWhat a file holds: { layout, indexed, name, format, type, bounds, center, minzoom, maxzoom, attribution, vector_layers, metadata, page_size }. The metadata map keeps every row. An MBTiles 1.3 json row's vector layers are read out.
mbtiles_tile(path: str, z: int, x: int, y: int) → anyor an errorA tile by web-map z/x/y, the y flipped to MBTiles's row for you: { found, z, x, y, row, bytes, format, compression, base64 }. found false is a tile the file lacks, which a map shows as empty: not an error. Quillo has no bytes value, so the tile comes as base64; mbtiles_tile_to_file writes it for file().
mbtiles_tile_to_file(path: str, z: int, x: int, y: int, to: str) → anyor an errorA tile written to a file, to answer a map with file(to): { found, bytes, format, content_type }. A gzipped vector tile is unpacked first, as file() cannot say Content-Encoding. Nothing is written for a tile the file lacks.
mbtiles_zooms(path: str) → anyor an errorHow many tiles there are at each zoom and the x and y they span: [{ zoom, tiles, min_x, max_x, min_y, max_y }], y as web maps count it. Reads the index where it can, the whole table where not.
mbtiles_flip_y(z: int, y: int) → intor an errorThe TMS row of a web-map y, or back, as it is its own inverse: at zoom 3, 2 and 5 swap.
Get it
$ quillo install mbtiles # mbtiles.qwer into this folder$ quillo install mbtiles --example # and its example, mbtiles.qweOr with curl:
$ curl -O https://quillolang.org/plugins/files/mbtiles.qwer$ curl -O https://quillolang.org/plugins/files/mbtiles.qwe$ curl -O https://quillolang.org/plugins/files/mbtiles_sample.mbtiles$ curl -O https://quillolang.org/plugins/files/mbtiles_dedupe.mbtiles$ curl -O https://quillolang.org/plugins/files/mbtiles_plain.mbtilesThen, in your program, all of it or only what you call:
use mbtiles.qweruse mbtiles_info, mbtiles_tile from mbtiles // Quillo 0.6: only theseIt uses these crates, which cargo fetches the first time the plugin is built:
serde_json = { version = "1", features = ["preserve_order"] }base64 = "0.22"flate2 = "1"Example
// MBTiles: a tileset's metadata, a tile by z/x/y with the row flipped, a little tile server, and// each of the layouts in use. The plugin reads the SQLite file itself, so nothing is locked and no// second SQLite is linked beside Quillo's.//// node standins/mbtiles_sample.mjs makes the sample files// quillo run mbtiles.qwe// then open http://localhost:8080/info, /tile/1/0/0, /tiles/2/1/3.png, /layouts and /refuseduse "mbtiles.qwer"let sample = "mbtiles_sample.mbtiles"route GET /info = { info: mbtiles_info(sample)?, zooms: mbtiles_zooms(sample)?,}route GET /tile/:z(int)/:x(int)/:y(int) { let t = mbtiles_tile(sample, z, x, y) if is_err(t) { reply 400 t.error } t}// What a map library asks for. A real server would keep these files only for a moment.route GET /tiles/:z(int)/:x(int)/:name { let y = int(split(name, ".")[0]) let to = "data/mbtiles-" + str(z) + "-" + str(x) + "-" + str(y) + ".png" let t = mbtiles_tile_to_file(sample, z, x, y, to)? if !t.found { reply 404 "no such tile" } file(to)}// A tile runs onto overflow pages at zoom 3; each y is its own row, the flip showing.route GET /check { let t = mbtiles_tile(sample, 3, 2, 5)? let low = mbtiles_tile(sample, 1, 1, 0)? { row: t.row, bytes: t.bytes, format: t.format, small: low.bytes, low_row: low.row, missing: mbtiles_tile(sample, 9, 0, 0)?.found, flip: [mbtiles_flip_y(3, 2)?, mbtiles_flip_y(3, 5)?], written: mbtiles_tile_to_file(sample, 1, 1, 0, "data/mbtiles-one.png")?, }}route GET /layouts { let d = "mbtiles_dedupe.mbtiles" let p = "mbtiles_plain.mbtiles" { dedupe: mbtiles_info(d)?.layout, shared: mbtiles_tile(d, 3, 0, 0)?.bytes == mbtiles_tile(d, 3, 7, 7)?.bytes, one: mbtiles_tile(d, 2, 1, 1)?.bytes, dedupe_zooms: mbtiles_zooms(d)?, plain: mbtiles_info(p)?, no_index: mbtiles_tile(p, 2, 1, 2)?, no_index_zooms: mbtiles_zooms(p)?, unpacked: mbtiles_tile_to_file(p, 2, 1, 2, "data/mbtiles-vector.pbf")?, }}route GET /refused = { off_the_map: mbtiles_tile(sample, 1, 5, 0).error, no_file: mbtiles_info("nothing.mbtiles").error, not_sqlite: mbtiles_info("mbtiles.qwe").error, zoom: mbtiles_flip_y(40, 0).error,}$ quillo run mbtiles.qwe$ curl 'localhost:8080/info'$ curl 'localhost:8080/tile/1/0/0'$ curl 'localhost:8080/tile/9/0/0'$ curl 'localhost:8080/tiles/2/1/3.png'$ curl 'localhost:8080/check'$ curl 'localhost:8080/layouts'$ curl 'localhost:8080/refused'Source
//! MBTiles files read by a small SQLite reader of its own, so no bundled SQLite sits beside Quillo's: metadata, tiles by z/x/y with the TMS row flip, the index used where there is one, and mbutil's deduplicated layout.//! version: 1.0.0//!//! [dependencies]//! serde_json = { version = "1", features = ["preserve_order"] }//! base64 = "0.22"//! flate2 = "1"use base64::Engine;use serde_json::{json, Map, Value};use std::collections::HashMap;use std::fs::File;use std::io::{Read, Seek, SeekFrom};// The file is read as pages, never written, and takes no lock: a file being written may be read// half way, so serve MBTiles that are finished. A `-wal` beside it with content is refused.struct Db { f: File, page: usize, usable: usize, pages: u64, cache: HashMap<u32, Vec<u8>>,}#[derive(Clone, Debug, PartialEq)]enum Col { Null, Int(i64), Real(f64), Text(String), Blob(Vec<u8>),}fn varint(b: &[u8]) -> Result<(u64, usize), String> { let mut v = 0u64; for i in 0..9 { let c = *b.get(i).ok_or("the file is cut short inside a record")?; if i == 8 { return Ok((v << 8 | c as u64, 9)); } v = v << 7 | (c & 0x7f) as u64; if c & 0x80 == 0 { return Ok((v, i + 1)); } } Ok((v, 9))}fn be(b: &[u8]) -> u64 { b.iter().fold(0u64, |a, c| a << 8 | *c as u64)}impl Db { fn open(path: &str) -> Result<Db, String> { let mut f = File::open(path).map_err(|e| format!("cannot open {}: {}", path, e))?; let wal = format!("{}-wal", path); if std::fs::metadata(&wal).map(|m| m.len() > 0).unwrap_or(false) { return Err(format!("{} has a write-ahead log beside it; run `PRAGMA wal_checkpoint(TRUNCATE)` on it first, as this reader sees only the main file", path)); } let mut h = [0u8; 100]; f.read_exact(&mut h).map_err(|_| format!("{} is too short to be SQLite", path))?; if &h[..16] != b"SQLite format 3\0" { return Err(format!("{} is not a SQLite file, so not MBTiles", path)); } let ps = be(&h[16..18]) as usize; let page = if ps == 1 { 65536 } else { ps }; if page < 512 || !page.is_power_of_two() { return Err(format!("{} has a bad page size", path)); } if be(&h[56..60]) > 1 { return Err(format!("{} is not UTF-8, which MBTiles requires", path)); } let usable = page - h[20] as usize; let len = f.metadata().map_err(|e| e.to_string())?.len(); Ok(Db { f, page, usable, pages: len / page as u64, cache: HashMap::new() }) } fn page(&mut self, n: u32) -> Result<&Vec<u8>, String> { if n == 0 || n as u64 > self.pages { return Err(format!("page {} is past the end of the file", n)); } if !self.cache.contains_key(&n) { if self.cache.len() > 256 { self.cache.clear(); } let mut b = vec![0u8; self.page]; self.f.seek(SeekFrom::Start((n as u64 - 1) * self.page as u64)).map_err(|e| e.to_string())?; self.f.read_exact(&mut b).map_err(|e| e.to_string())?; self.cache.insert(n, b); } Ok(&self.cache[&n]) } // The bytes of a cell's payload, following its overflow chain. fn payload(&mut self, page_no: u32, at: usize, size: usize, leaf_table: bool) -> Result<Vec<u8>, String> { let u = self.usable; let x = if leaf_table { u - 35 } else { (u - 12) * 64 / 255 - 23 }; let m = (u - 12) * 32 / 255 - 23; let local = if size <= x { size } else { let k = m + (size - m) % (u - 4); if k <= x { k } else { m } }; let p = self.page(page_no)?; if at + local > p.len() { return Err("a cell runs off its page".into()); } let mut out = p[at..at + local].to_vec(); if local < size { let mut next = be(&p[at + local..at + local + 4]) as u32; let mut guard = 0u64; while next != 0 && out.len() < size { guard += 1; if guard > self.pages { return Err("an overflow chain loops".into()); } let pg = self.page(next)?; let want = (size - out.len()).min(u - 4); next = be(&pg[..4]) as u32; let chunk = pg[4..4 + want].to_vec(); out.extend_from_slice(&chunk); } } Ok(out) } fn cells(&mut self, n: u32) -> Result<(u8, Vec<usize>, u32), String> { let off = if n == 1 { 100 } else { 0 }; let p = self.page(n)?; let kind = p[off]; let count = be(&p[off + 3..off + 5]) as usize; let (head, right) = match kind { 2 | 5 => (12, be(&p[off + 8..off + 12]) as u32), 10 | 13 => (8, 0), k => return Err(format!("page {} has an unknown kind {}", n, k)), }; let mut c = Vec::with_capacity(count); for i in 0..count { c.push(be(&p[off + head + i * 2..off + head + i * 2 + 2]) as usize); } Ok((kind, c, right)) } // Every row of a table, in rowid order; `f` returns false to stop. fn walk(&mut self, root: u32, f: &mut dyn FnMut(&mut Db, i64, Vec<Col>) -> Result<bool, String>) -> Result<bool, String> { let (kind, cells, right) = self.cells(root)?; for c in cells { if kind == 5 { let child = be(&self.page(root)?[c..c + 4]) as u32; if !self.walk(child, f)? { return Ok(false); } } else { let (size, a) = varint(&self.page(root)?[c..])?; let (rowid, b) = varint(&self.page(root)?[c + a..])?; let data = self.payload(root, c + a + b, size as usize, true)?; if !f(self, rowid as i64, record(&data)?)? { return Ok(false); } } } if kind == 5 { return self.walk(right, f); } Ok(true) } // One row by rowid, down the table's tree. fn row(&mut self, root: u32, rowid: i64) -> Result<Option<Vec<Col>>, String> { let mut pg = root; loop { let (kind, cells, right) = self.cells(pg)?; if kind == 5 { let mut next = right; for c in cells { let p = self.page(pg)?; let (key, _) = varint(&p[c + 4..])?; if rowid <= key as i64 { next = be(&p[c..c + 4]) as u32; break; } } pg = next; } else { for c in cells { let (size, a) = varint(&self.page(pg)?[c..])?; let (id, b) = varint(&self.page(pg)?[c + a..])?; if id as i64 == rowid { let d = self.payload(pg, c + a + b, size as usize, true)?; return Ok(Some(record(&d)?)); } } return Ok(None); } } } // Down an index for a key of leading columns: gives the record found. fn seek(&mut self, root: u32, key: &[Col]) -> Result<Option<Vec<Col>>, String> { let mut pg = root; loop { let (kind, cells, right) = self.cells(pg)?; let inner = kind == 2; let (mut lo, mut hi) = (0usize, cells.len()); while lo < hi { let mid = (lo + hi) / 2; let c = cells[mid]; let at = if inner { c + 4 } else { c }; let (size, a) = varint(&self.page(pg)?[at..])?; let d = self.payload(pg, at + a, size as usize, false)?; let r = record(&d)?; match compare(&r, key) { std::cmp::Ordering::Equal => return Ok(Some(r)), std::cmp::Ordering::Less => lo = mid + 1, std::cmp::Ordering::Greater => hi = mid, } } if !inner { return Ok(None); } pg = if lo < cells.len() { be(&self.page(pg)?[cells[lo]..cells[lo] + 4]) as u32 } else { right }; } } // Every record of an index in key order. fn scan_index(&mut self, root: u32, f: &mut dyn FnMut(Vec<Col>) -> Result<(), String>) -> Result<(), String> { let (kind, cells, right) = self.cells(root)?; for c in cells { if kind == 2 { let child = be(&self.page(root)?[c..c + 4]) as u32; self.scan_index(child, f)?; } let at = if kind == 2 { c + 4 } else { c }; let (size, a) = varint(&self.page(root)?[at..])?; let d = self.payload(root, at + a, size as usize, false)?; f(record(&d)?)?; } if kind == 2 { self.scan_index(right, f)?; } Ok(()) }}fn compare(r: &[Col], key: &[Col]) -> std::cmp::Ordering { use std::cmp::Ordering::*; for (a, b) in r.iter().zip(key) { let o = match (a, b) { (Col::Int(x), Col::Int(y)) => x.cmp(y), (Col::Text(x), Col::Text(y)) => x.as_bytes().cmp(y.as_bytes()), (Col::Blob(x), Col::Blob(y)) => x.cmp(y), (Col::Null, Col::Null) => Equal, (Col::Null, _) => Less, (_, Col::Null) => Greater, (Col::Int(_), _) | (Col::Real(_), _) => Less, (Col::Text(_), Col::Blob(_)) => Less, _ => Greater, }; if o != Equal { return o; } } Equal}fn record(d: &[u8]) -> Result<Vec<Col>, String> { let (head, mut h) = varint(d)?; let head = head as usize; if head > d.len() { return Err("a record's header runs past it".into()); } let mut body = head; let mut out = vec![]; while h < head { let (t, n) = varint(&d[h..])?; h += n; let take = |len: usize, b: usize| -> Result<&[u8], String> { d.get(b..b + len).ok_or("a record is cut short".to_string()) }; let (v, len) = match t { 0 => (Col::Null, 0), 1..=4 => { let l = t as usize; (Col::Int(signed(take(l, body)?)), l) } 5 => (Col::Int(signed(take(6, body)?)), 6), 6 => (Col::Int(signed(take(8, body)?)), 8), 7 => (Col::Real(f64::from_bits(be(take(8, body)?))), 8), 8 => (Col::Int(0), 0), 9 => (Col::Int(1), 0), n if n >= 12 && n % 2 == 0 => { let l = (n as usize - 12) / 2; (Col::Blob(take(l, body)?.to_vec()), l) } n if n >= 13 => { let l = (n as usize - 13) / 2; (Col::Text(String::from_utf8_lossy(take(l, body)?).into_owned()), l) } n => return Err(format!("a record has a reserved type {}", n)), }; body += len; out.push(v); } Ok(out)}fn signed(b: &[u8]) -> i64 { let mut v = if b[0] & 0x80 != 0 { -1i64 } else { 0 }; for c in b { v = v << 8 | *c as i64; } v}struct Item { kind: String, name: String, table: String, root: u32, sql: String,}fn schema(db: &mut Db) -> Result<Vec<Item>, String> { let mut out = vec![]; db.walk(1, &mut |_, _, r| { let text = |i: usize| match r.get(i) { Some(Col::Text(s)) => s.clone(), _ => String::new(), }; let root = match r.get(3) { Some(Col::Int(n)) => *n as u32, _ => 0, }; out.push(Item { kind: text(0), name: text(1), table: text(2), root, sql: text(4) }); Ok(true) })?; Ok(out)}// An index whose first three columns are the tile's, in order.fn tile_index(s: &[Item], table: &str) -> Option<u32> { s.iter().find(|i| { let low = i.sql.to_lowercase().replace(['"', '`', '[', ']'], ""); let cols = low.split_once('(').map(|x| x.1).unwrap_or("").replace(' ', ""); i.kind == "index" && i.table == table && i.root != 0 && cols.starts_with("zoom_level,tile_column,tile_row") }).map(|i| i.root)}fn column_pos(sql: &str, want: &str) -> Option<usize> { let body = sql.split_once('(')?.1; let body = body.rsplit_once(')')?.0; let mut depth = 0; let mut i = 0; let mut piece = String::new(); let mut pieces = vec![]; for c in body.chars() { match c { '(' => depth += 1, ')' => depth -= 1, ',' if depth == 0 => { pieces.push(std::mem::take(&mut piece)); continue; } _ => {} } piece.push(c); } pieces.push(piece); for p in pieces { let n = p.trim().trim_matches(|c| matches!(c, '"' | '`' | '[' | ']')).split(|c: char| c.is_whitespace() || c == '"' || c == '`' || c == ']').next().unwrap_or("").to_lowercase(); if n == want { return Some(i); } i += 1; } None}struct Layout { db: Db, table: Option<(u32, [usize; 4])>, // A plain `tiles` table: root, and z, x, y, data columns. index: Option<u32>, map: Option<(u32, [usize; 4], Option<u32>)>, images: Option<(u32, usize, usize, Option<u32>)>, kind: &'static str,}fn layout(path: &str) -> Result<Layout, String> { let mut db = Db::open(path)?; let s = schema(&mut db)?; let find = |n: &str, k: &str| s.iter().find(|i| i.name == n && i.kind == k); if find("metadata", "table").is_none() { return Err(format!("{} has no `metadata` table, so it is not MBTiles", path)); } let cols = |sql: &str, names: [&str; 4]| -> Result<[usize; 4], String> { let mut o = [0; 4]; for (i, n) in names.iter().enumerate() { o[i] = column_pos(sql, n).ok_or(format!("no column `{}`", n))?; } Ok(o) }; let mut l = Layout { db, table: None, index: None, map: None, images: None, kind: "tiles" }; if let Some(t) = find("tiles", "table") { if t.sql.to_uppercase().contains("WITHOUT ROWID") { return Err("a WITHOUT ROWID `tiles` table is not read here".into()); } l.table = Some((t.root, cols(&t.sql, ["zoom_level", "tile_column", "tile_row", "tile_data"])?)); l.index = tile_index(&s, "tiles"); } else if find("tiles", "view").is_some() { // mbutil's compact layout, joined by hand on tile_id. let (m, im) = (find("map", "table"), find("images", "table")); let (Some(m), Some(im)) = (m, im) else { return Err("`tiles` is a view over something other than `map` and `images`".into()); }; l.map = Some((m.root, cols(&m.sql, ["zoom_level", "tile_column", "tile_row", "tile_id"])?, tile_index(&s, "map"))); let idx = s.iter().find(|i| i.kind == "index" && i.table == "images" && i.sql.to_lowercase().contains("tile_id")).map(|i| i.root); l.images = Some((im.root, column_pos(&im.sql, "tile_data").ok_or("no `tile_data`")?, column_pos(&im.sql, "tile_id").ok_or("no `tile_id`")?, idx)); l.kind = "map+images"; } else { return Err(format!("{} has no `tiles` table or view, so it is not MBTiles", path)); } Ok(l)}fn meta(path: &str, l: &mut Layout) -> Result<Map<String, Value>, String> { let s = schema(&mut l.db)?; let t = s.iter().find(|i| i.name == "metadata" && i.kind == "table").ok_or(format!("{} has no metadata", path))?; let (name, value) = (column_pos(&t.sql, "name").unwrap_or(0), column_pos(&t.sql, "value").unwrap_or(1)); let mut m = Map::new(); let root = t.root; l.db.walk(root, &mut |_, _, r| { if let (Some(Col::Text(k)), Some(Col::Text(v))) = (r.get(name), r.get(value)) { m.insert(k.clone(), json!(v)); } Ok(true) })?; Ok(m)}fn int(c: Option<&Col>) -> i64 { match c { Some(Col::Int(n)) => *n, _ => -1, }}fn flip(z: i64, y: i64) -> i64 { (1i64 << z) - 1 - y}fn check_tile(z: i64, x: i64, y: i64) -> Result<(), String> { if !(0..=30).contains(&z) { return Err(format!("zoom is 0 to 30, not {}", z)); } if x < 0 || y < 0 || x >= 1 << z || y >= 1 << z { return Err(format!("{}/{}/{} is off the map: at zoom {} x and y are 0 to {}", z, x, y, z, (1i64 << z) - 1)); } Ok(())}fn fetch(l: &mut Layout, z: i64, x: i64, row: i64) -> Result<Option<Vec<u8>>, String> { let key = [Col::Int(z), Col::Int(x), Col::Int(row)]; let blob = |c: Option<&Col>| match c { Some(Col::Blob(b)) => Some(b.clone()), Some(Col::Text(t)) => Some(t.clone().into_bytes()), _ => None, }; if let Some((root, cols)) = l.table { if let Some(ix) = l.index { // The index's last column is the rowid. let Some(r) = l.db.seek(ix, &key)? else { return Ok(None) }; let rowid = int(r.last()); return Ok(l.db.row(root, rowid)?.and_then(|t| blob(t.get(cols[3])))); } let mut found = None; l.db.walk(root, &mut |_, _, r| { if int(r.get(cols[0])) == z && int(r.get(cols[1])) == x && int(r.get(cols[2])) == row { found = blob(r.get(cols[3])); return Ok(false); } Ok(true) })?; return Ok(found); } let (root, cols, ix) = l.map.ok_or("no tiles")?; let (iroot, data, id, iix) = l.images.ok_or("no images")?; let mut tile_id = None; if let Some(ix) = ix { if let Some(r) = l.db.seek(ix, &key)? { let rowid = int(r.last()); tile_id = l.db.row(root, rowid)?.and_then(|t| t.get(cols[3]).cloned()); } } else { l.db.walk(root, &mut |_, _, r| { if int(r.get(cols[0])) == z && int(r.get(cols[1])) == x && int(r.get(cols[2])) == row { tile_id = r.get(cols[3]).cloned(); return Ok(false); } Ok(true) })?; } let Some(tile_id) = tile_id else { return Ok(None) }; if let Some(iix) = iix { if let Some(r) = l.db.seek(iix, &[tile_id.clone()])? { let rowid = int(r.last()); return Ok(l.db.row(iroot, rowid)?.and_then(|t| blob(t.get(data)))); } return Ok(None); } let mut found = None; l.db.walk(iroot, &mut |_, _, r| { if r.get(id) == Some(&tile_id) { found = blob(r.get(data)); return Ok(false); } Ok(true) })?; Ok(found)}fn sniff(b: &[u8]) -> (&'static str, Option<&'static str>) { if b.starts_with(&[0x89, b'P', b'N', b'G']) { ("png", None) } else if b.starts_with(&[0xff, 0xd8]) { ("jpg", None) } else if b.starts_with(b"RIFF") && b.get(8..12) == Some(b"WEBP") { ("webp", None) } else if b.starts_with(&[0x1f, 0x8b]) { ("pbf", Some("gzip")) } else if b.starts_with(&[0x78, 0x9c]) { ("pbf", Some("deflate")) } else { ("pbf", None) }}fn numbers(s: Option<&Value>) -> Option<Vec<f64>> { let v: Vec<f64> = s?.as_str()?.split(',').filter_map(|x| x.trim().parse().ok()).collect(); if v.is_empty() { None } else { Some(v) }}/// What a file holds: { layout, indexed, name, format, type, bounds, center, minzoom, maxzoom, attribution, vector_layers, metadata, page_size }. The `metadata` map keeps every row. An MBTiles 1.3 `json` row's vector layers are read out.pub fn mbtiles_info(path: &str) -> Result<Value, String> { let mut l = layout(path)?; let m = meta(path, &mut l)?; let get = |k: &str| m.get(k).cloned().unwrap_or(Value::Null); let layers = m.get("json").and_then(|j| j.as_str()).and_then(|j| serde_json::from_str::<Value>(j).ok()).and_then(|v| v.get("vector_layers").cloned()); let int_of = |k: &str| m.get(k).and_then(|v| v.as_str()).and_then(|v| v.parse::<i64>().ok()); Ok(json!({ "layout": l.kind, "indexed": l.index.is_some() || l.map.map(|x| x.2.is_some()).unwrap_or(false), "name": get("name"), "format": get("format"), "type": get("type"), "bounds": numbers(m.get("bounds")), "center": numbers(m.get("center")), "minzoom": int_of("minzoom"), "maxzoom": int_of("maxzoom"), "attribution": get("attribution"), "vector_layers": layers, "page_size": l.db.page, "metadata": m, }))}/// A tile by web-map z/x/y, the y flipped to MBTiles's row for you: { found, z, x, y, row, bytes, format, compression, base64 }. `found` false is a tile the file lacks, which a map shows as empty: not an error. Quillo has no bytes value, so the tile comes as base64; mbtiles_tile_to_file writes it for `file()`.pub fn mbtiles_tile(path: &str, z: i64, x: i64, y: i64) -> Result<Value, String> { check_tile(z, x, y)?; let mut l = layout(path)?; let row = flip(z, y); Ok(match fetch(&mut l, z, x, row)? { Some(b) => { let (fmt, comp) = sniff(&b); json!({ "found": true, "z": z, "x": x, "y": y, "row": row, "bytes": b.len(), "format": fmt, "compression": comp, "base64": base64::engine::general_purpose::STANDARD.encode(&b) }) } None => json!({ "found": false, "z": z, "x": x, "y": y, "row": row, "bytes": 0, "format": null, "compression": null, "base64": null }), })}/// A tile written to a file, to answer a map with `file(to)`: { found, bytes, format, content_type }. A gzipped vector tile is unpacked first, as `file()` cannot say `Content-Encoding`. Nothing is written for a tile the file lacks.pub fn mbtiles_tile_to_file(path: &str, z: i64, x: i64, y: i64, to: &str) -> Result<Value, String> { check_tile(z, x, y)?; let mut l = layout(path)?; match fetch(&mut l, z, x, flip(z, y))? { None => Ok(json!({ "found": false, "bytes": 0, "format": null, "content_type": null })), Some(b) => { if let Some(dir) = std::path::Path::new(to).parent() { std::fs::create_dir_all(dir).map_err(|e| e.to_string())?; } let (fmt, comp) = sniff(&b); let mut plain = vec![]; match comp { Some("gzip") => { flate2::read::MultiGzDecoder::new(&b[..]).take(1 << 28).read_to_end(&mut plain).map_err(|e| format!("the tile's gzip is bad: {}", e))?; } Some(_) => { flate2::read::ZlibDecoder::new(&b[..]).take(1 << 28).read_to_end(&mut plain).map_err(|e| format!("the tile's deflate is bad: {}", e))?; } None => plain = b, } std::fs::write(to, &plain).map_err(|e| format!("cannot write {}: {}", to, e))?; let ct = match fmt { "png" => "image/png", "jpg" => "image/jpeg", "webp" => "image/webp", _ => "application/x-protobuf", }; Ok(json!({ "found": true, "bytes": plain.len(), "format": fmt, "content_type": ct })) } }}/// How many tiles there are at each zoom and the x and y they span: [{ zoom, tiles, min_x, max_x, min_y, max_y }], y as web maps count it. Reads the index where it can, the whole table where not.pub fn mbtiles_zooms(path: &str) -> Result<Value, String> { let mut l = layout(path)?; let mut z: std::collections::BTreeMap<i64, (i64, i64, i64, i64, i64)> = std::collections::BTreeMap::new(); let mut add = |zoom: i64, x: i64, row: i64| { if !(0..=30).contains(&zoom) { return; } let y = flip(zoom, row); let e = z.entry(zoom).or_insert((0, i64::MAX, i64::MIN, i64::MAX, i64::MIN)); e.0 += 1; e.1 = e.1.min(x); e.2 = e.2.max(x); e.3 = e.3.min(y); e.4 = e.4.max(y); }; let (index, table) = (l.index.or(l.map.and_then(|m| m.2)), l.table.map(|t| (t.0, t.1)).or(l.map.map(|m| (m.0, m.1)))); if let Some(ix) = index { l.db.scan_index(ix, &mut |r| { add(int(r.first()), int(r.get(1)), int(r.get(2))); Ok(()) })?; } else if let Some((root, c)) = table { l.db.walk(root, &mut |_, _, r| { add(int(r.get(c[0])), int(r.get(c[1])), int(r.get(c[2]))); Ok(true) })?; } Ok(json!(z.iter().map(|(k, v)| json!({ "zoom": k, "tiles": v.0, "min_x": v.1, "max_x": v.2, "min_y": v.3, "max_y": v.4 })).collect::<Vec<_>>()))}/// The TMS row of a web-map y, or back, as it is its own inverse: at zoom 3, 2 and 5 swap.pub fn mbtiles_flip_y(z: i64, y: i64) -> Result<i64, String> { check_tile(z, 0, y)?; Ok(flip(z, y))}Related
mapbox.qwer · 1.0.0Mapbox from the server: Geocoding v6 forward and reverse with address parts read out, directions with polyline6 decoded, travel time matrices, Static Images URLs with markers, paths and auto fit, slippy-map tile numbers and tile URLs. A 429 is waited out by its reset time, and the token stays out of error messages.3 views · 9 downloadsOpenStreetMap PBF osm_pbf.qwer · 1.0.0OpenStreetMap PBF extracts read in pure Rust with no database: the header and its required features, nodes, dense nodes, ways and relations filtered by tags and a bounding box, ways given their coordinates and length by a second pass, tags counted, and matches written as GeoJSON.3 views · 5 downloadsPMTiles pmtiles.qwer · 1.0.0PMTiles v3 archives read from a file or from any server answering Range requests, with no tile server: the header, Hilbert tile ids both ways, directories with run lengths and leaves, metadata, gzip, brotli and zstd, and tiles by z/x/y. Directories are cached, and an archive replaced under the reader is noticed by its ETag.3 views · 7 downloadsTopoJSON topojson.qwer · 1.0.0TopoJSON read: arcs decoded with quantization and delta encoding, reversed arcs followed, and any object turned into GeoJSON with its ids and properties. GeoJSON made into a topology, where a border two features share is one arc; neighbours found and borders drawn as a mesh.5 views · 8 downloadsCoordinates coordinates.qwer · 1.0.0Coordinates as people write them, such as 52°22'N 4°53'E or N 52° 22.2', read as one, written as degrees-minutes-seconds, and converted to and from UTM.7 views · 8 downloadsGeoTIFF geotiff.qwer · 1.0.0GeoTIFF files read in pure Rust, for elevation data above all: tags and GeoKeys, tie points or a transformation matrix, bounds, EPSG code, no-data, and the value at a coordinate, nearest or bilinear, reading only the strip or tile it needs. Profiles along a line, statistics, strips or tiles, classic or BigTIFF, either byte order, plain, deflate, LZW or PackBits, with predictors.3 views · 7 downloadsThread 0 posts
On MBTiles: questions, tips, corrections, and code that uses it.
No posts yet. The first one opens the thread.