Places

MBTiles

MBTiles files read by a small SQLite reader of its own, in pure Rust and with no locks, so no bundled SQLite sits beside Quillo's: metadata, tiles by web-map z/x/y with the TMS row flip, the unique index walked where there is one, overflow pages followed, mbutil's `map` and `images` layout joined, and tiles written out for `file()`.

Version 1.0.0 · 5 functions · 3 crates

#mbtiles#maps#tiles#sqlite#gis

Functions

What use mbtiles.qwer gives a program, with the types Quillo sees. To take only some, or under other names: use mbtiles_info, mbtiles_tile from mbtiles (Quillo 0.6).

mbtiles_info(path: str) → anyor an error

What a file holds: { layout, indexed, name, format, type, bounds, center, minzoom, maxzoom, attribution, vector_layers, metadata, page_size }. The metadata map keeps every row. An MBTiles 1.3 json row's vector layers are read out.

mbtiles_tile(path: str, z: int, x: int, y: int) → anyor an error

A tile by web-map z/x/y, the y flipped to MBTiles's row for you: { found, z, x, y, row, bytes, format, compression, base64 }. found false is a tile the file lacks, which a map shows as empty: not an error. Quillo has no bytes value, so the tile comes as base64; mbtiles_tile_to_file writes it for file().

mbtiles_tile_to_file(path: str, z: int, x: int, y: int, to: str) → anyor an error

A tile written to a file, to answer a map with file(to): { found, bytes, format, content_type }. A gzipped vector tile is unpacked first, as file() cannot say Content-Encoding. Nothing is written for a tile the file lacks.

mbtiles_zooms(path: str) → anyor an error

How many tiles there are at each zoom and the x and y they span: [{ zoom, tiles, min_x, max_x, min_y, max_y }], y as web maps count it. Reads the index where it can, the whole table where not.

mbtiles_flip_y(z: int, y: int) → intor an error

The TMS row of a web-map y, or back, as it is its own inverse: at zoom 3, 2 and 5 swap.

Get it

$ quillo install mbtiles               # mbtiles.qwer into this folder$ quillo install mbtiles --example     # and its example, mbtiles.qwe

Or with curl:

$ curl -O https://quillolang.org/plugins/files/mbtiles.qwer$ curl -O https://quillolang.org/plugins/files/mbtiles.qwe$ curl -O https://quillolang.org/plugins/files/mbtiles_sample.mbtiles$ curl -O https://quillolang.org/plugins/files/mbtiles_dedupe.mbtiles$ curl -O https://quillolang.org/plugins/files/mbtiles_plain.mbtiles

Then, in your program, all of it or only what you call:

main.qwe
use mbtiles.qweruse mbtiles_info, mbtiles_tile from mbtiles      // Quillo 0.6: only these

It uses these crates, which cargo fetches the first time the plugin is built:

[dependencies]
serde_json = { version = "1", features = ["preserve_order"] }base64 = "0.22"flate2 = "1"

Example

mbtiles.qwe
// MBTiles: a tileset's metadata, a tile by z/x/y with the row flipped, a little tile server, and// each of the layouts in use. The plugin reads the SQLite file itself, so nothing is locked and no// second SQLite is linked beside Quillo's.////   node standins/mbtiles_sample.mjs      makes the sample files//   quillo run mbtiles.qwe//   then open http://localhost:8080/info, /tile/1/0/0, /tiles/2/1/3.png, /layouts and /refuseduse "mbtiles.qwer"let sample = "mbtiles_sample.mbtiles"route GET /info = {    info: mbtiles_info(sample)?,    zooms: mbtiles_zooms(sample)?,}route GET /tile/:z(int)/:x(int)/:y(int) {    let t = mbtiles_tile(sample, z, x, y)    if is_err(t) { reply 400 t.error }    t}// What a map library asks for. A real server would keep these files only for a moment.route GET /tiles/:z(int)/:x(int)/:name {    let y = int(split(name, ".")[0])    let to = "data/mbtiles-" + str(z) + "-" + str(x) + "-" + str(y) + ".png"    let t = mbtiles_tile_to_file(sample, z, x, y, to)?    if !t.found { reply 404 "no such tile" }    file(to)}// A tile runs onto overflow pages at zoom 3; each y is its own row, the flip showing.route GET /check {    let t = mbtiles_tile(sample, 3, 2, 5)?    let low = mbtiles_tile(sample, 1, 1, 0)?    {        row: t.row,        bytes: t.bytes,        format: t.format,        small: low.bytes,        low_row: low.row,        missing: mbtiles_tile(sample, 9, 0, 0)?.found,        flip: [mbtiles_flip_y(3, 2)?, mbtiles_flip_y(3, 5)?],        written: mbtiles_tile_to_file(sample, 1, 1, 0, "data/mbtiles-one.png")?,    }}route GET /layouts {    let d = "mbtiles_dedupe.mbtiles"    let p = "mbtiles_plain.mbtiles"    {        dedupe: mbtiles_info(d)?.layout,        shared: mbtiles_tile(d, 3, 0, 0)?.bytes == mbtiles_tile(d, 3, 7, 7)?.bytes,        one: mbtiles_tile(d, 2, 1, 1)?.bytes,        dedupe_zooms: mbtiles_zooms(d)?,        plain: mbtiles_info(p)?,        no_index: mbtiles_tile(p, 2, 1, 2)?,        no_index_zooms: mbtiles_zooms(p)?,        unpacked: mbtiles_tile_to_file(p, 2, 1, 2, "data/mbtiles-vector.pbf")?,    }}route GET /refused = {    off_the_map: mbtiles_tile(sample, 1, 5, 0).error,    no_file: mbtiles_info("nothing.mbtiles").error,    not_sqlite: mbtiles_info("mbtiles.qwe").error,    zoom: mbtiles_flip_y(40, 0).error,}
$ quillo run mbtiles.qwe$ curl 'localhost:8080/info'$ curl 'localhost:8080/tile/1/0/0'$ curl 'localhost:8080/tile/9/0/0'$ curl 'localhost:8080/tiles/2/1/3.png'$ curl 'localhost:8080/check'$ curl 'localhost:8080/layouts'$ curl 'localhost:8080/refused'

Source

mbtiles.qwer
//! MBTiles files read by a small SQLite reader of its own, so no bundled SQLite sits beside Quillo's: metadata, tiles by z/x/y with the TMS row flip, the index used where there is one, and mbutil's deduplicated layout.//! version: 1.0.0//!//! [dependencies]//! serde_json = { version = "1", features = ["preserve_order"] }//! base64 = "0.22"//! flate2 = "1"use base64::Engine;use serde_json::{json, Map, Value};use std::collections::HashMap;use std::fs::File;use std::io::{Read, Seek, SeekFrom};// The file is read as pages, never written, and takes no lock: a file being written may be read// half way, so serve MBTiles that are finished. A `-wal` beside it with content is refused.struct Db {    f: File,    page: usize,    usable: usize,    pages: u64,    cache: HashMap<u32, Vec<u8>>,}#[derive(Clone, Debug, PartialEq)]enum Col {    Null,    Int(i64),    Real(f64),    Text(String),    Blob(Vec<u8>),}fn varint(b: &[u8]) -> Result<(u64, usize), String> {    let mut v = 0u64;    for i in 0..9 {        let c = *b.get(i).ok_or("the file is cut short inside a record")?;        if i == 8 {            return Ok((v << 8 | c as u64, 9));        }        v = v << 7 | (c & 0x7f) as u64;        if c & 0x80 == 0 {            return Ok((v, i + 1));        }    }    Ok((v, 9))}fn be(b: &[u8]) -> u64 {    b.iter().fold(0u64, |a, c| a << 8 | *c as u64)}impl Db {    fn open(path: &str) -> Result<Db, String> {        let mut f = File::open(path).map_err(|e| format!("cannot open {}: {}", path, e))?;        let wal = format!("{}-wal", path);        if std::fs::metadata(&wal).map(|m| m.len() > 0).unwrap_or(false) {            return Err(format!("{} has a write-ahead log beside it; run `PRAGMA wal_checkpoint(TRUNCATE)` on it first, as this reader sees only the main file", path));        }        let mut h = [0u8; 100];        f.read_exact(&mut h).map_err(|_| format!("{} is too short to be SQLite", path))?;        if &h[..16] != b"SQLite format 3\0" {            return Err(format!("{} is not a SQLite file, so not MBTiles", path));        }        let ps = be(&h[16..18]) as usize;        let page = if ps == 1 { 65536 } else { ps };        if page < 512 || !page.is_power_of_two() {            return Err(format!("{} has a bad page size", path));        }        if be(&h[56..60]) > 1 {            return Err(format!("{} is not UTF-8, which MBTiles requires", path));        }        let usable = page - h[20] as usize;        let len = f.metadata().map_err(|e| e.to_string())?.len();        Ok(Db { f, page, usable, pages: len / page as u64, cache: HashMap::new() })    }    fn page(&mut self, n: u32) -> Result<&Vec<u8>, String> {        if n == 0 || n as u64 > self.pages {            return Err(format!("page {} is past the end of the file", n));        }        if !self.cache.contains_key(&n) {            if self.cache.len() > 256 {                self.cache.clear();            }            let mut b = vec![0u8; self.page];            self.f.seek(SeekFrom::Start((n as u64 - 1) * self.page as u64)).map_err(|e| e.to_string())?;            self.f.read_exact(&mut b).map_err(|e| e.to_string())?;            self.cache.insert(n, b);        }        Ok(&self.cache[&n])    }    // The bytes of a cell's payload, following its overflow chain.    fn payload(&mut self, page_no: u32, at: usize, size: usize, leaf_table: bool) -> Result<Vec<u8>, String> {        let u = self.usable;        let x = if leaf_table { u - 35 } else { (u - 12) * 64 / 255 - 23 };        let m = (u - 12) * 32 / 255 - 23;        let local = if size <= x {            size        } else {            let k = m + (size - m) % (u - 4);            if k <= x { k } else { m }        };        let p = self.page(page_no)?;        if at + local > p.len() {            return Err("a cell runs off its page".into());        }        let mut out = p[at..at + local].to_vec();        if local < size {            let mut next = be(&p[at + local..at + local + 4]) as u32;            let mut guard = 0u64;            while next != 0 && out.len() < size {                guard += 1;                if guard > self.pages {                    return Err("an overflow chain loops".into());                }                let pg = self.page(next)?;                let want = (size - out.len()).min(u - 4);                next = be(&pg[..4]) as u32;                let chunk = pg[4..4 + want].to_vec();                out.extend_from_slice(&chunk);            }        }        Ok(out)    }    fn cells(&mut self, n: u32) -> Result<(u8, Vec<usize>, u32), String> {        let off = if n == 1 { 100 } else { 0 };        let p = self.page(n)?;        let kind = p[off];        let count = be(&p[off + 3..off + 5]) as usize;        let (head, right) = match kind {            2 | 5 => (12, be(&p[off + 8..off + 12]) as u32),            10 | 13 => (8, 0),            k => return Err(format!("page {} has an unknown kind {}", n, k)),        };        let mut c = Vec::with_capacity(count);        for i in 0..count {            c.push(be(&p[off + head + i * 2..off + head + i * 2 + 2]) as usize);        }        Ok((kind, c, right))    }    // Every row of a table, in rowid order; `f` returns false to stop.    fn walk(&mut self, root: u32, f: &mut dyn FnMut(&mut Db, i64, Vec<Col>) -> Result<bool, String>) -> Result<bool, String> {        let (kind, cells, right) = self.cells(root)?;        for c in cells {            if kind == 5 {                let child = be(&self.page(root)?[c..c + 4]) as u32;                if !self.walk(child, f)? {                    return Ok(false);                }            } else {                let (size, a) = varint(&self.page(root)?[c..])?;                let (rowid, b) = varint(&self.page(root)?[c + a..])?;                let data = self.payload(root, c + a + b, size as usize, true)?;                if !f(self, rowid as i64, record(&data)?)? {                    return Ok(false);                }            }        }        if kind == 5 {            return self.walk(right, f);        }        Ok(true)    }    // One row by rowid, down the table's tree.    fn row(&mut self, root: u32, rowid: i64) -> Result<Option<Vec<Col>>, String> {        let mut pg = root;        loop {            let (kind, cells, right) = self.cells(pg)?;            if kind == 5 {                let mut next = right;                for c in cells {                    let p = self.page(pg)?;                    let (key, _) = varint(&p[c + 4..])?;                    if rowid <= key as i64 {                        next = be(&p[c..c + 4]) as u32;                        break;                    }                }                pg = next;            } else {                for c in cells {                    let (size, a) = varint(&self.page(pg)?[c..])?;                    let (id, b) = varint(&self.page(pg)?[c + a..])?;                    if id as i64 == rowid {                        let d = self.payload(pg, c + a + b, size as usize, true)?;                        return Ok(Some(record(&d)?));                    }                }                return Ok(None);            }        }    }    // Down an index for a key of leading columns: gives the record found.    fn seek(&mut self, root: u32, key: &[Col]) -> Result<Option<Vec<Col>>, String> {        let mut pg = root;        loop {            let (kind, cells, right) = self.cells(pg)?;            let inner = kind == 2;            let (mut lo, mut hi) = (0usize, cells.len());            while lo < hi {                let mid = (lo + hi) / 2;                let c = cells[mid];                let at = if inner { c + 4 } else { c };                let (size, a) = varint(&self.page(pg)?[at..])?;                let d = self.payload(pg, at + a, size as usize, false)?;                let r = record(&d)?;                match compare(&r, key) {                    std::cmp::Ordering::Equal => return Ok(Some(r)),                    std::cmp::Ordering::Less => lo = mid + 1,                    std::cmp::Ordering::Greater => hi = mid,                }            }            if !inner {                return Ok(None);            }            pg = if lo < cells.len() { be(&self.page(pg)?[cells[lo]..cells[lo] + 4]) as u32 } else { right };        }    }    // Every record of an index in key order.    fn scan_index(&mut self, root: u32, f: &mut dyn FnMut(Vec<Col>) -> Result<(), String>) -> Result<(), String> {        let (kind, cells, right) = self.cells(root)?;        for c in cells {            if kind == 2 {                let child = be(&self.page(root)?[c..c + 4]) as u32;                self.scan_index(child, f)?;            }            let at = if kind == 2 { c + 4 } else { c };            let (size, a) = varint(&self.page(root)?[at..])?;            let d = self.payload(root, at + a, size as usize, false)?;            f(record(&d)?)?;        }        if kind == 2 {            self.scan_index(right, f)?;        }        Ok(())    }}fn compare(r: &[Col], key: &[Col]) -> std::cmp::Ordering {    use std::cmp::Ordering::*;    for (a, b) in r.iter().zip(key) {        let o = match (a, b) {            (Col::Int(x), Col::Int(y)) => x.cmp(y),            (Col::Text(x), Col::Text(y)) => x.as_bytes().cmp(y.as_bytes()),            (Col::Blob(x), Col::Blob(y)) => x.cmp(y),            (Col::Null, Col::Null) => Equal,            (Col::Null, _) => Less,            (_, Col::Null) => Greater,            (Col::Int(_), _) | (Col::Real(_), _) => Less,            (Col::Text(_), Col::Blob(_)) => Less,            _ => Greater,        };        if o != Equal {            return o;        }    }    Equal}fn record(d: &[u8]) -> Result<Vec<Col>, String> {    let (head, mut h) = varint(d)?;    let head = head as usize;    if head > d.len() {        return Err("a record's header runs past it".into());    }    let mut body = head;    let mut out = vec![];    while h < head {        let (t, n) = varint(&d[h..])?;        h += n;        let take = |len: usize, b: usize| -> Result<&[u8], String> { d.get(b..b + len).ok_or("a record is cut short".to_string()) };        let (v, len) = match t {            0 => (Col::Null, 0),            1..=4 => {                let l = t as usize;                (Col::Int(signed(take(l, body)?)), l)            }            5 => (Col::Int(signed(take(6, body)?)), 6),            6 => (Col::Int(signed(take(8, body)?)), 8),            7 => (Col::Real(f64::from_bits(be(take(8, body)?))), 8),            8 => (Col::Int(0), 0),            9 => (Col::Int(1), 0),            n if n >= 12 && n % 2 == 0 => {                let l = (n as usize - 12) / 2;                (Col::Blob(take(l, body)?.to_vec()), l)            }            n if n >= 13 => {                let l = (n as usize - 13) / 2;                (Col::Text(String::from_utf8_lossy(take(l, body)?).into_owned()), l)            }            n => return Err(format!("a record has a reserved type {}", n)),        };        body += len;        out.push(v);    }    Ok(out)}fn signed(b: &[u8]) -> i64 {    let mut v = if b[0] & 0x80 != 0 { -1i64 } else { 0 };    for c in b {        v = v << 8 | *c as i64;    }    v}struct Item {    kind: String,    name: String,    table: String,    root: u32,    sql: String,}fn schema(db: &mut Db) -> Result<Vec<Item>, String> {    let mut out = vec![];    db.walk(1, &mut |_, _, r| {        let text = |i: usize| match r.get(i) {            Some(Col::Text(s)) => s.clone(),            _ => String::new(),        };        let root = match r.get(3) {            Some(Col::Int(n)) => *n as u32,            _ => 0,        };        out.push(Item { kind: text(0), name: text(1), table: text(2), root, sql: text(4) });        Ok(true)    })?;    Ok(out)}// An index whose first three columns are the tile's, in order.fn tile_index(s: &[Item], table: &str) -> Option<u32> {    s.iter().find(|i| {        let low = i.sql.to_lowercase().replace(['"', '`', '[', ']'], "");        let cols = low.split_once('(').map(|x| x.1).unwrap_or("").replace(' ', "");        i.kind == "index" && i.table == table && i.root != 0 && cols.starts_with("zoom_level,tile_column,tile_row")    }).map(|i| i.root)}fn column_pos(sql: &str, want: &str) -> Option<usize> {    let body = sql.split_once('(')?.1;    let body = body.rsplit_once(')')?.0;    let mut depth = 0;    let mut i = 0;    let mut piece = String::new();    let mut pieces = vec![];    for c in body.chars() {        match c {            '(' => depth += 1,            ')' => depth -= 1,            ',' if depth == 0 => {                pieces.push(std::mem::take(&mut piece));                continue;            }            _ => {}        }        piece.push(c);    }    pieces.push(piece);    for p in pieces {        let n = p.trim().trim_matches(|c| matches!(c, '"' | '`' | '[' | ']')).split(|c: char| c.is_whitespace() || c == '"' || c == '`' || c == ']').next().unwrap_or("").to_lowercase();        if n == want {            return Some(i);        }        i += 1;    }    None}struct Layout {    db: Db,    table: Option<(u32, [usize; 4])>, // A plain `tiles` table: root, and z, x, y, data columns.    index: Option<u32>,    map: Option<(u32, [usize; 4], Option<u32>)>,    images: Option<(u32, usize, usize, Option<u32>)>,    kind: &'static str,}fn layout(path: &str) -> Result<Layout, String> {    let mut db = Db::open(path)?;    let s = schema(&mut db)?;    let find = |n: &str, k: &str| s.iter().find(|i| i.name == n && i.kind == k);    if find("metadata", "table").is_none() {        return Err(format!("{} has no `metadata` table, so it is not MBTiles", path));    }    let cols = |sql: &str, names: [&str; 4]| -> Result<[usize; 4], String> {        let mut o = [0; 4];        for (i, n) in names.iter().enumerate() {            o[i] = column_pos(sql, n).ok_or(format!("no column `{}`", n))?;        }        Ok(o)    };    let mut l = Layout { db, table: None, index: None, map: None, images: None, kind: "tiles" };    if let Some(t) = find("tiles", "table") {        if t.sql.to_uppercase().contains("WITHOUT ROWID") {            return Err("a WITHOUT ROWID `tiles` table is not read here".into());        }        l.table = Some((t.root, cols(&t.sql, ["zoom_level", "tile_column", "tile_row", "tile_data"])?));        l.index = tile_index(&s, "tiles");    } else if find("tiles", "view").is_some() {        // mbutil's compact layout, joined by hand on tile_id.        let (m, im) = (find("map", "table"), find("images", "table"));        let (Some(m), Some(im)) = (m, im) else {            return Err("`tiles` is a view over something other than `map` and `images`".into());        };        l.map = Some((m.root, cols(&m.sql, ["zoom_level", "tile_column", "tile_row", "tile_id"])?, tile_index(&s, "map")));        let idx = s.iter().find(|i| i.kind == "index" && i.table == "images" && i.sql.to_lowercase().contains("tile_id")).map(|i| i.root);        l.images = Some((im.root, column_pos(&im.sql, "tile_data").ok_or("no `tile_data`")?, column_pos(&im.sql, "tile_id").ok_or("no `tile_id`")?, idx));        l.kind = "map+images";    } else {        return Err(format!("{} has no `tiles` table or view, so it is not MBTiles", path));    }    Ok(l)}fn meta(path: &str, l: &mut Layout) -> Result<Map<String, Value>, String> {    let s = schema(&mut l.db)?;    let t = s.iter().find(|i| i.name == "metadata" && i.kind == "table").ok_or(format!("{} has no metadata", path))?;    let (name, value) = (column_pos(&t.sql, "name").unwrap_or(0), column_pos(&t.sql, "value").unwrap_or(1));    let mut m = Map::new();    let root = t.root;    l.db.walk(root, &mut |_, _, r| {        if let (Some(Col::Text(k)), Some(Col::Text(v))) = (r.get(name), r.get(value)) {            m.insert(k.clone(), json!(v));        }        Ok(true)    })?;    Ok(m)}fn int(c: Option<&Col>) -> i64 {    match c {        Some(Col::Int(n)) => *n,        _ => -1,    }}fn flip(z: i64, y: i64) -> i64 {    (1i64 << z) - 1 - y}fn check_tile(z: i64, x: i64, y: i64) -> Result<(), String> {    if !(0..=30).contains(&z) {        return Err(format!("zoom is 0 to 30, not {}", z));    }    if x < 0 || y < 0 || x >= 1 << z || y >= 1 << z {        return Err(format!("{}/{}/{} is off the map: at zoom {} x and y are 0 to {}", z, x, y, z, (1i64 << z) - 1));    }    Ok(())}fn fetch(l: &mut Layout, z: i64, x: i64, row: i64) -> Result<Option<Vec<u8>>, String> {    let key = [Col::Int(z), Col::Int(x), Col::Int(row)];    let blob = |c: Option<&Col>| match c {        Some(Col::Blob(b)) => Some(b.clone()),        Some(Col::Text(t)) => Some(t.clone().into_bytes()),        _ => None,    };    if let Some((root, cols)) = l.table {        if let Some(ix) = l.index {            // The index's last column is the rowid.            let Some(r) = l.db.seek(ix, &key)? else { return Ok(None) };            let rowid = int(r.last());            return Ok(l.db.row(root, rowid)?.and_then(|t| blob(t.get(cols[3]))));        }        let mut found = None;        l.db.walk(root, &mut |_, _, r| {            if int(r.get(cols[0])) == z && int(r.get(cols[1])) == x && int(r.get(cols[2])) == row {                found = blob(r.get(cols[3]));                return Ok(false);            }            Ok(true)        })?;        return Ok(found);    }    let (root, cols, ix) = l.map.ok_or("no tiles")?;    let (iroot, data, id, iix) = l.images.ok_or("no images")?;    let mut tile_id = None;    if let Some(ix) = ix {        if let Some(r) = l.db.seek(ix, &key)? {            let rowid = int(r.last());            tile_id = l.db.row(root, rowid)?.and_then(|t| t.get(cols[3]).cloned());        }    } else {        l.db.walk(root, &mut |_, _, r| {            if int(r.get(cols[0])) == z && int(r.get(cols[1])) == x && int(r.get(cols[2])) == row {                tile_id = r.get(cols[3]).cloned();                return Ok(false);            }            Ok(true)        })?;    }    let Some(tile_id) = tile_id else { return Ok(None) };    if let Some(iix) = iix {        if let Some(r) = l.db.seek(iix, &[tile_id.clone()])? {            let rowid = int(r.last());            return Ok(l.db.row(iroot, rowid)?.and_then(|t| blob(t.get(data))));        }        return Ok(None);    }    let mut found = None;    l.db.walk(iroot, &mut |_, _, r| {        if r.get(id) == Some(&tile_id) {            found = blob(r.get(data));            return Ok(false);        }        Ok(true)    })?;    Ok(found)}fn sniff(b: &[u8]) -> (&'static str, Option<&'static str>) {    if b.starts_with(&[0x89, b'P', b'N', b'G']) {        ("png", None)    } else if b.starts_with(&[0xff, 0xd8]) {        ("jpg", None)    } else if b.starts_with(b"RIFF") && b.get(8..12) == Some(b"WEBP") {        ("webp", None)    } else if b.starts_with(&[0x1f, 0x8b]) {        ("pbf", Some("gzip"))    } else if b.starts_with(&[0x78, 0x9c]) {        ("pbf", Some("deflate"))    } else {        ("pbf", None)    }}fn numbers(s: Option<&Value>) -> Option<Vec<f64>> {    let v: Vec<f64> = s?.as_str()?.split(',').filter_map(|x| x.trim().parse().ok()).collect();    if v.is_empty() { None } else { Some(v) }}/// What a file holds: { layout, indexed, name, format, type, bounds, center, minzoom, maxzoom, attribution, vector_layers, metadata, page_size }. The `metadata` map keeps every row. An MBTiles 1.3 `json` row's vector layers are read out.pub fn mbtiles_info(path: &str) -> Result<Value, String> {    let mut l = layout(path)?;    let m = meta(path, &mut l)?;    let get = |k: &str| m.get(k).cloned().unwrap_or(Value::Null);    let layers = m.get("json").and_then(|j| j.as_str()).and_then(|j| serde_json::from_str::<Value>(j).ok()).and_then(|v| v.get("vector_layers").cloned());    let int_of = |k: &str| m.get(k).and_then(|v| v.as_str()).and_then(|v| v.parse::<i64>().ok());    Ok(json!({        "layout": l.kind, "indexed": l.index.is_some() || l.map.map(|x| x.2.is_some()).unwrap_or(false),        "name": get("name"), "format": get("format"), "type": get("type"), "bounds": numbers(m.get("bounds")), "center": numbers(m.get("center")),        "minzoom": int_of("minzoom"), "maxzoom": int_of("maxzoom"), "attribution": get("attribution"), "vector_layers": layers,        "page_size": l.db.page, "metadata": m,    }))}/// A tile by web-map z/x/y, the y flipped to MBTiles's row for you: { found, z, x, y, row, bytes, format, compression, base64 }. `found` false is a tile the file lacks, which a map shows as empty: not an error. Quillo has no bytes value, so the tile comes as base64; mbtiles_tile_to_file writes it for `file()`.pub fn mbtiles_tile(path: &str, z: i64, x: i64, y: i64) -> Result<Value, String> {    check_tile(z, x, y)?;    let mut l = layout(path)?;    let row = flip(z, y);    Ok(match fetch(&mut l, z, x, row)? {        Some(b) => {            let (fmt, comp) = sniff(&b);            json!({ "found": true, "z": z, "x": x, "y": y, "row": row, "bytes": b.len(), "format": fmt, "compression": comp, "base64": base64::engine::general_purpose::STANDARD.encode(&b) })        }        None => json!({ "found": false, "z": z, "x": x, "y": y, "row": row, "bytes": 0, "format": null, "compression": null, "base64": null }),    })}/// A tile written to a file, to answer a map with `file(to)`: { found, bytes, format, content_type }. A gzipped vector tile is unpacked first, as `file()` cannot say `Content-Encoding`. Nothing is written for a tile the file lacks.pub fn mbtiles_tile_to_file(path: &str, z: i64, x: i64, y: i64, to: &str) -> Result<Value, String> {    check_tile(z, x, y)?;    let mut l = layout(path)?;    match fetch(&mut l, z, x, flip(z, y))? {        None => Ok(json!({ "found": false, "bytes": 0, "format": null, "content_type": null })),        Some(b) => {            if let Some(dir) = std::path::Path::new(to).parent() {                std::fs::create_dir_all(dir).map_err(|e| e.to_string())?;            }            let (fmt, comp) = sniff(&b);            let mut plain = vec![];            match comp {                Some("gzip") => {                    flate2::read::MultiGzDecoder::new(&b[..]).take(1 << 28).read_to_end(&mut plain).map_err(|e| format!("the tile's gzip is bad: {}", e))?;                }                Some(_) => {                    flate2::read::ZlibDecoder::new(&b[..]).take(1 << 28).read_to_end(&mut plain).map_err(|e| format!("the tile's deflate is bad: {}", e))?;                }                None => plain = b,            }            std::fs::write(to, &plain).map_err(|e| format!("cannot write {}: {}", to, e))?;            let ct = match fmt {                "png" => "image/png",                "jpg" => "image/jpeg",                "webp" => "image/webp",                _ => "application/x-protobuf",            };            Ok(json!({ "found": true, "bytes": plain.len(), "format": fmt, "content_type": ct }))        }    }}/// How many tiles there are at each zoom and the x and y they span: [{ zoom, tiles, min_x, max_x, min_y, max_y }], y as web maps count it. Reads the index where it can, the whole table where not.pub fn mbtiles_zooms(path: &str) -> Result<Value, String> {    let mut l = layout(path)?;    let mut z: std::collections::BTreeMap<i64, (i64, i64, i64, i64, i64)> = std::collections::BTreeMap::new();    let mut add = |zoom: i64, x: i64, row: i64| {        if !(0..=30).contains(&zoom) {            return;        }        let y = flip(zoom, row);        let e = z.entry(zoom).or_insert((0, i64::MAX, i64::MIN, i64::MAX, i64::MIN));        e.0 += 1;        e.1 = e.1.min(x);        e.2 = e.2.max(x);        e.3 = e.3.min(y);        e.4 = e.4.max(y);    };    let (index, table) = (l.index.or(l.map.and_then(|m| m.2)), l.table.map(|t| (t.0, t.1)).or(l.map.map(|m| (m.0, m.1))));    if let Some(ix) = index {        l.db.scan_index(ix, &mut |r| {            add(int(r.first()), int(r.get(1)), int(r.get(2)));            Ok(())        })?;    } else if let Some((root, c)) = table {        l.db.walk(root, &mut |_, _, r| {            add(int(r.get(c[0])), int(r.get(c[1])), int(r.get(c[2])));            Ok(true)        })?;    }    Ok(json!(z.iter().map(|(k, v)| json!({ "zoom": k, "tiles": v.0, "min_x": v.1, "max_x": v.2, "min_y": v.3, "max_y": v.4 })).collect::<Vec<_>>()))}/// The TMS row of a web-map y, or back, as it is its own inverse: at zoom 3, 2 and 5 swap.pub fn mbtiles_flip_y(z: i64, y: i64) -> Result<i64, String> {    check_tile(z, 0, y)?;    Ok(flip(z, y))}
Mapbox mapbox.qwer · 1.0.0Mapbox from the server: Geocoding v6 forward and reverse with address parts read out, directions with polyline6 decoded, travel time matrices, Static Images URLs with markers, paths and auto fit, slippy-map tile numbers and tile URLs. A 429 is waited out by its reset time, and the token stays out of error messages.mapsgeocodingdirectionsmapboxtiles3 views · 9 downloadsOpenStreetMap PBF osm_pbf.qwer · 1.0.0OpenStreetMap PBF extracts read in pure Rust with no database: the header and its required features, nodes, dense nodes, ways and relations filtered by tags and a bounding box, ways given their coordinates and length by a second pass, tags counted, and matches written as GeoJSON.openstreetmaposmpbfmapsgis3 views · 5 downloadsPMTiles pmtiles.qwer · 1.0.0PMTiles v3 archives read from a file or from any server answering Range requests, with no tile server: the header, Hilbert tile ids both ways, directories with run lengths and leaves, metadata, gzip, brotli and zstd, and tiles by z/x/y. Directories are cached, and an archive replaced under the reader is noticed by its ETag.pmtilesmapstileshilbertrange-requests3 views · 7 downloadsTopoJSON topojson.qwer · 1.0.0TopoJSON read: arcs decoded with quantization and delta encoding, reversed arcs followed, and any object turned into GeoJSON with its ids and properties. GeoJSON made into a topology, where a border two features share is one arc; neighbours found and borders drawn as a mesh.topojsongeojsonmapsgistopology5 views · 8 downloadsCoordinates coordinates.qwer · 1.0.0Coordinates as people write them, such as 52°22'N 4°53'E or N 52° 22.2', read as one, written as degrees-minutes-seconds, and converted to and from UTM.geomapsformsgps7 views · 8 downloadsGeoTIFF geotiff.qwer · 1.0.0GeoTIFF files read in pure Rust, for elevation data above all: tags and GeoKeys, tie points or a transformation matrix, bounds, EPSG code, no-data, and the value at a coordinate, nearest or bilinear, reading only the strip or tile it needs. Profiles along a line, statistics, strips or tiles, classic or BigTIFF, either byte order, plain, deflate, LZW or PackBits, with predictors.geotiffelevationdemrastergis3 views · 7 downloads
Thread
28Useful? Like it, or add to its thread below.

Thread 0 posts

On MBTiles: questions, tips, corrections, and code that uses it.

No posts yet. The first one opens the thread.