Infrastructure

DigitalOcean

Make, list and destroy droplets, wait on their actions, keep domains and DNS records and Spaces keys, with every list paged by its links and rate limits waited out by their headers.

Version 1.0.0 · 19 functions · 2 crates

#cloud#servers#dns#api

Functions

What use digitalocean.qwer gives a program, with the types Quillo sees. To take only some, or under other names: use digitalocean_rate, digitalocean_request from digitalocean (Quillo 0.6).

digitalocean_rate() → any

What the last answer said of the rate limit: { limit, remaining, reset }.

digitalocean_request(method: str, path: str, body: any, options: any) → anyor an error

Any call: method, path like /v2/account, body or null. Gives the JSON answer, null for a 204.

digitalocean_all(path: str, key: str, options: any) → listor an error

Every item of a list under key (droplets, domains...), following links.pages.next.

digitalocean_action_wait(id: int, options: any) → anyor an error

Waits for an action to end: { id, status, type, ... }. errored is an error value. Options: wait_ms (5 min), every_ms (2 s).

digitalocean_droplets(options: any) → listor an error

All droplets, each with public_ip and private_ip picked out. Option tag limits them to a tag.

digitalocean_droplet(id: int, options: any) → anyor an error

One droplet.

digitalocean_droplet_create(spec: any, options: any) → anyor an error

Makes a droplet: spec is { name, region, size, image, ssh_keys, tags, user_data, ... }. With the option wait, waits for its create action and gives it again, by then with addresses.

digitalocean_droplet_delete(id: int, options: any) → boolor an error

Destroys a droplet.

digitalocean_droplet_action(id: int, kind: str, args: any, options: any) → anyor an error

Starts an action on a droplet: kind is reboot, power_off, power_on, shutdown, snapshot, resize...; args the action's own fields or null. With the option wait, waits for it. Gives the action.

digitalocean_domains(options: any) → listor an error

All domains.

digitalocean_domain_create(name: str, ip: str, options: any) → anyor an error

Adds a domain, with an A record at the apex when ip is given.

digitalocean_domain_delete(name: str, options: any) → boolor an error

Removes a domain and its records.

digitalocean_records(domain: str, options: any) → listor an error

A domain's records, all pages. Options: type and name (the whole name, like www.example.com) filter them.

digitalocean_record_create(domain: str, record: any, options: any) → anyor an error

Adds a record: { type, name, data, ttl, priority, port, weight, flags, tag }. name is relative, @ the apex.

digitalocean_record_update(domain: str, id: int, changes: any, options: any) → anyor an error

Changes fields of a record.

digitalocean_record_delete(domain: str, id: int, options: any) → boolor an error

Removes a record.

digitalocean_spaces_keys(options: any) → listor an error

The account's Spaces access keys, without their secrets.

digitalocean_spaces_key_create(name: str, grants: any, options: any) → anyor an error

Makes a Spaces key: grants is [{ bucket, permission }], permission being read, readwrite or fullaccess (bucket "" for fullaccess). Gives { name, access_key, secret_key, grants }: the secret is shown this once.

digitalocean_spaces_key_delete(access_key: str, options: any) → boolor an error

Revokes a Spaces key by its access key.

Get it

$ quillo install digitalocean               # digitalocean.qwer into this folder$ quillo install digitalocean --example     # and its example, digitalocean.qwe

Or with curl:

$ curl -O https://quillolang.org/plugins/files/digitalocean.qwer$ curl -O https://quillolang.org/plugins/files/digitalocean.qwe

Then, in your program, all of it or only what you call:

main.qwe
use digitalocean.qweruse digitalocean_rate, digitalocean_request from digitalocean      // Quillo 0.6: only these

It uses these crates, which cargo fetches the first time the plugin is built:

[dependencies]
ureq = "2"serde_json = { version = "1", features = ["preserve_order"] }

Example

digitalocean.qwe
// A small fleet on DigitalOcean: a droplet made and waited for until it has an address, rebooted,// listed past one page, a domain with its records, and a Spaces key made and revoked.////   DIGITALOCEAN_TOKEN=... quillo run digitalocean.qwe      then open http://localhost:8080/droplets, /action, /dns, /spaces and /failures//// The token needs write scope. These routes spend money on a real account. DIGITALOCEAN_API_URL// reaches a test server.use "digitalocean.qwer"route GET /droplets {    let made = digitalocean_droplet_create({ name: "web-9", region: "ams3", size: "s-1vcpu-1gb", image: "ubuntu-24-04-x64", tags: ["web"] }, { wait: true, every_ms: 100 })?    let all = digitalocean_droplets({})?    let web = digitalocean_droplets({ tag: "web" })?    { status: made.status, ip: made.public_ip, total: len(all), tagged: len(web), one: digitalocean_droplet(made.id, {})?.name, rate: digitalocean_rate().remaining != null }}route GET /action {    let done = digitalocean_droplet_action(1001, "reboot", null, { wait: true, every_ms: 100 })?    let queued = digitalocean_droplet_action(1001, "snapshot", { name: "before-upgrade" }, {})?    let spare = digitalocean_droplet_create({ name: "spare", region: "ams3", size: "s-1vcpu-1gb", image: "ubuntu-24-04-x64" }, {})?    { reboot: done.status, snapshot: queued.status, failed: digitalocean_droplet_action(1002, "power_on", null, { wait: true, every_ms: 50 }).error, deleted: digitalocean_droplet_delete(spare.id, {})? }}route GET /dns {    digitalocean_domain_create("shop.example", "192.0.2.10", {})?    let rec = digitalocean_record_create("shop.example", { type: "A", name: "www", data: "192.0.2.11", ttl: 300 }, {})?    let txt = digitalocean_record_create("shop.example", { type: "TXT", name: "@", data: "v=spf1 -all", ttl: 3600 }, {})?    let changed = digitalocean_record_update("shop.example", rec.id, { data: "192.0.2.12" }, {})?    digitalocean_record_delete("shop.example", txt.id, {})?    {        domains: map(digitalocean_domains({})?, (d) => d.name),        www: changed.data,        records: len(digitalocean_records("shop.example", {})?),        only_a: map(digitalocean_records("shop.example", { type: "A", name: "www.shop.example" })?, (r) => r.name),        taken: digitalocean_domain_create("shop.example", "", {}).error,        removed: digitalocean_domain_delete("shop.example", {})?,    }}route GET /spaces {    let key = digitalocean_spaces_key_create("backup", [{ bucket: "photos", permission: "read" }], {})?    let listed = digitalocean_spaces_keys({})?    { access: key.access_key, secret_shown: len(key.secret_key) > 0, listed: len(listed), hides_secret: listed[0].secret_key == "", bad: digitalocean_spaces_key_create("x", [{ bucket: "a", permission: "write" }], {}).error, revoked: digitalocean_spaces_key_delete(key.access_key, {})? }}route GET /failures = {    missing: digitalocean_droplet(404404, {}).error,    incomplete: digitalocean_droplet_create({ name: "x" }, {}).error,    limited: digitalocean_request("GET", "/v2/account", null, {})?.account.status,}
$ quillo run digitalocean.qwe$ curl 'localhost:8080/droplets'$ curl 'localhost:8080/action'$ curl 'localhost:8080/dns'$ curl 'localhost:8080/spaces'$ curl 'localhost:8080/failures'

It talks to the Redis that REDIS_URL names, else Redis on this machine. The site's test runs it against a stand-in.

Source

digitalocean.qwer
//! DigitalOcean's API: droplets listed, made and destroyed, power and snapshot actions waited on, domains and DNS records, Spaces access keys, every list paged by following its links, and 429s waited out by the rate-limit headers.//! version: 1.0.0//!//! [dependencies]//! ureq = "2"//! serde_json = { version = "1", features = ["preserve_order"] }use serde_json::{json, Value};use std::sync::Mutex;use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH};static RATE: Mutex<Value> = Mutex::new(Value::Null);fn enc(s: &str) -> String {    s.bytes().map(|b| if b.is_ascii_alphanumeric() || b"-_.~".contains(&b) { (b as char).to_string() } else { format!("%{:02X}", b) }).collect()}fn base() -> String {    std::env::var("DIGITALOCEAN_API_URL").ok().filter(|v| !v.is_empty()).unwrap_or("https://api.digitalocean.com".into()).trim_end_matches('/').to_string()}fn now() -> u64 {    SystemTime::now().duration_since(UNIX_EPOCH).map_or(0, |d| d.as_secs())}/// One call. A 429 waits for RateLimit-Reset, at most the option max_wait_s (10), and goes again; 5xx on a GET goes again too. An API error becomes `DigitalOcean 404 not_found: ...`.fn call(method: &str, path: &str, body: Option<&Value>, o: &Value) -> Result<Value, String> {    let token = std::env::var("DIGITALOCEAN_TOKEN").ok().filter(|v| !v.is_empty()).ok_or("no token: set DIGITALOCEAN_TOKEN")?;    let url = if path.starts_with("http") { path.to_string() } else { format!("{}{}", base(), path) };    let agent = ureq::AgentBuilder::new().timeout_connect(Duration::from_secs(10)).timeout_read(Duration::from_secs(60)).build();    let most = o.get("max_wait_s").and_then(|v| v.as_u64()).unwrap_or(10);    let mut tries = 0;    loop {        tries += 1;        let r = agent.request(method, &url).set("Authorization", &format!("Bearer {}", token)).set("Accept", "application/json");        let res = match body {            Some(b) => r.set("Content-Type", "application/json").send_string(&b.to_string()),            None => r.call(),        };        let resp = match res {            Ok(x) => x,            Err(ureq::Error::Status(_, x)) => x,            Err(e) => return Err(format!("cannot reach DigitalOcean: {}", e)),        };        let status = resp.status();        let limit = |n: &str| resp.header(n).and_then(|h| h.trim().parse::<i64>().ok());        let (lim, left, reset) = (limit("ratelimit-limit"), limit("ratelimit-remaining"), limit("ratelimit-reset"));        if lim.is_some() {            if let Ok(mut g) = RATE.lock() {                *g = json!({ "limit": lim, "remaining": left, "reset": reset });            }        }        let text = resp.into_string().map_err(|e| e.to_string())?;        if (status == 429 || (status >= 500 && method == "GET")) && tries < 5 {            let wait = if status == 429 { reset.map(|r| (r - now() as i64).max(1) as u64).unwrap_or(1) } else { tries };            if wait <= most {                std::thread::sleep(Duration::from_secs(wait));                continue;            }        }        if status >= 300 {            let v: Value = serde_json::from_str(&text).unwrap_or(Value::Null);            return Err(format!("DigitalOcean {} {}: {}", status, v["id"].as_str().unwrap_or("error"), v["message"].as_str().map(str::to_string).unwrap_or_else(|| text.chars().take(160).collect())));        }        if text.trim().is_empty() {            return Ok(Value::Null);        }        return serde_json::from_str(&text).map_err(|e| format!("DigitalOcean: not JSON: {}", e));    }}/// What the last answer said of the rate limit: { limit, remaining, reset }.pub fn digitalocean_rate() -> serde_json::Value {    RATE.lock().map(|g| g.clone()).unwrap_or(Value::Null)}/// Any call: `method`, `path` like /v2/account, `body` or null. Gives the JSON answer, null for a 204.pub fn digitalocean_request(method: &str, path: &str, body: serde_json::Value, options: serde_json::Value) -> Result<serde_json::Value, String> {    call(&method.to_uppercase(), path, if body.is_null() { None } else { Some(&body) }, &options)}/// Every item of a list under `key` (droplets, domains...), following `links.pages.next`.pub fn digitalocean_all(path: &str, key: &str, options: serde_json::Value) -> Result<Vec<serde_json::Value>, String> {    let size = options.get("per_page").and_then(|v| v.as_u64()).unwrap_or(200);    let mut next = format!("{}{}per_page={}", path, if path.contains('?') { "&" } else { "?" }, size);    let mut out = vec![];    loop {        let v = call("GET", &next, None, &options)?;        out.extend(v[key].as_array().cloned().unwrap_or_default());        match v.pointer("/links/pages/next").and_then(|n| n.as_str()) {            // The link names the real host: stay on the one configured.            Some(n) => next = n.find("/v2/").map_or(n.to_string(), |i| n[i..].to_string()),            None => return Ok(out),        }    }}/// Waits for an action to end: { id, status, type, ... }. `errored` is an error value. Options: wait_ms (5 min), every_ms (2 s).pub fn digitalocean_action_wait(id: i64, options: serde_json::Value) -> Result<serde_json::Value, String> {    let limit = Duration::from_millis(options.get("wait_ms").and_then(|v| v.as_u64()).unwrap_or(300_000));    let every = Duration::from_millis(options.get("every_ms").and_then(|v| v.as_u64()).unwrap_or(2_000));    let start = Instant::now();    loop {        let a = call("GET", &format!("/v2/actions/{}", id), None, &options)?["action"].clone();        match a["status"].as_str() {            Some("completed") => return Ok(a),            Some("errored") => return Err(format!("action {} ({}) ended in an error", id, a["type"].as_str().unwrap_or("?"))),            _ => {}        }        if start.elapsed() > limit {            return Err(format!("action {} is still {} after {} ms", id, a["status"].as_str().unwrap_or("?"), limit.as_millis()));        }        std::thread::sleep(every);    }}fn plain(d: &Value) -> Value {    let mut out = d.clone();    let ip = |kind: &str| d.pointer("/networks/v4").and_then(|n| n.as_array()).and_then(|l| l.iter().find(|a| a["type"] == kind)).map(|a| a["ip_address"].clone());    out["public_ip"] = ip("public").unwrap_or(Value::Null);    out["private_ip"] = ip("private").unwrap_or(Value::Null);    out}/// All droplets, each with `public_ip` and `private_ip` picked out. Option tag limits them to a tag.pub fn digitalocean_droplets(options: serde_json::Value) -> Result<Vec<serde_json::Value>, String> {    let path = options.get("tag").and_then(|t| t.as_str()).map_or("/v2/droplets".to_string(), |t| format!("/v2/droplets?tag_name={}", enc(t)));    Ok(digitalocean_all(&path, "droplets", options)?.iter().map(plain).collect())}/// One droplet.pub fn digitalocean_droplet(id: i64, options: serde_json::Value) -> Result<serde_json::Value, String> {    Ok(plain(&call("GET", &format!("/v2/droplets/{}", id), None, &options)?["droplet"]))}/// Makes a droplet: `spec` is { name, region, size, image, ssh_keys, tags, user_data, ... }. With the option wait, waits for its create action and gives it again, by then with addresses.pub fn digitalocean_droplet_create(spec: serde_json::Value, options: serde_json::Value) -> Result<serde_json::Value, String> {    for k in ["name", "region", "size", "image"] {        if spec.get(k).is_none() {            return Err(format!("the droplet needs `{}`", k));        }    }    let r = call("POST", "/v2/droplets", Some(&spec), &options)?;    let d = &r["droplet"];    if options.get("wait") == Some(&Value::Bool(true)) {        let act = r.pointer("/links/actions/0/id").and_then(|i| i.as_i64()).ok_or("the answer names no create action")?;        digitalocean_action_wait(act, options.clone())?;        return digitalocean_droplet(d["id"].as_i64().unwrap_or(0), options);    }    Ok(plain(d))}/// Destroys a droplet.pub fn digitalocean_droplet_delete(id: i64, options: serde_json::Value) -> Result<bool, String> {    call("DELETE", &format!("/v2/droplets/{}", id), None, &options)?;    Ok(true)}/// Starts an action on a droplet: `kind` is reboot, power_off, power_on, shutdown, snapshot, resize...; `args` the action's own fields or null. With the option wait, waits for it. Gives the action.pub fn digitalocean_droplet_action(id: i64, kind: &str, args: serde_json::Value, options: serde_json::Value) -> Result<serde_json::Value, String> {    let mut body = if args.is_object() { args } else { json!({}) };    body["type"] = json!(kind);    let a = call("POST", &format!("/v2/droplets/{}/actions", id), Some(&body), &options)?["action"].clone();    if options.get("wait") == Some(&Value::Bool(true)) {        return digitalocean_action_wait(a["id"].as_i64().unwrap_or(0), options);    }    Ok(a)}/// All domains.pub fn digitalocean_domains(options: serde_json::Value) -> Result<Vec<serde_json::Value>, String> {    digitalocean_all("/v2/domains", "domains", options)}/// Adds a domain, with an A record at the apex when `ip` is given.pub fn digitalocean_domain_create(name: &str, ip: &str, options: serde_json::Value) -> Result<serde_json::Value, String> {    let body = if ip.is_empty() { json!({ "name": name }) } else { json!({ "name": name, "ip_address": ip }) };    Ok(call("POST", "/v2/domains", Some(&body), &options)?["domain"].clone())}/// Removes a domain and its records.pub fn digitalocean_domain_delete(name: &str, options: serde_json::Value) -> Result<bool, String> {    call("DELETE", &format!("/v2/domains/{}", enc(name)), None, &options)?;    Ok(true)}/// A domain's records, all pages. Options: type and name (the whole name, like www.example.com) filter them.pub fn digitalocean_records(domain: &str, options: serde_json::Value) -> Result<Vec<serde_json::Value>, String> {    let mut path = format!("/v2/domains/{}/records", enc(domain));    let mut q = vec![];    for k in ["type", "name"] {        if let Some(v) = options.get(k).and_then(|v| v.as_str()) {            q.push(format!("{}={}", k, enc(v)));        }    }    if !q.is_empty() {        path = format!("{}?{}", path, q.join("&"));    }    digitalocean_all(&path, "domain_records", options)}/// Adds a record: { type, name, data, ttl, priority, port, weight, flags, tag }. `name` is relative, `@` the apex.pub fn digitalocean_record_create(domain: &str, record: serde_json::Value, options: serde_json::Value) -> Result<serde_json::Value, String> {    Ok(call("POST", &format!("/v2/domains/{}/records", enc(domain)), Some(&record), &options)?["domain_record"].clone())}/// Changes fields of a record.pub fn digitalocean_record_update(domain: &str, id: i64, changes: serde_json::Value, options: serde_json::Value) -> Result<serde_json::Value, String> {    Ok(call("PATCH", &format!("/v2/domains/{}/records/{}", enc(domain), id), Some(&changes), &options)?["domain_record"].clone())}/// Removes a record.pub fn digitalocean_record_delete(domain: &str, id: i64, options: serde_json::Value) -> Result<bool, String> {    call("DELETE", &format!("/v2/domains/{}/records/{}", enc(domain), id), None, &options)?;    Ok(true)}/// The account's Spaces access keys, without their secrets.pub fn digitalocean_spaces_keys(options: serde_json::Value) -> Result<Vec<serde_json::Value>, String> {    digitalocean_all("/v2/spaces/keys", "keys", options)}/// Makes a Spaces key: `grants` is [{ bucket, permission }], permission being read, readwrite or fullaccess (bucket "" for fullaccess). Gives { name, access_key, secret_key, grants }: the secret is shown this once.pub fn digitalocean_spaces_key_create(name: &str, grants: serde_json::Value, options: serde_json::Value) -> Result<serde_json::Value, String> {    for g in grants.as_array().ok_or("grants is a list of { bucket, permission }")? {        let p = g["permission"].as_str().unwrap_or("");        if !["read", "readwrite", "fullaccess"].contains(&p) {            return Err(format!("`{}` is not a permission: read, readwrite or fullaccess", p));        }    }    Ok(call("POST", "/v2/spaces/keys", Some(&json!({ "name": name, "grants": grants })), &options)?["key"].clone())}/// Revokes a Spaces key by its access key.pub fn digitalocean_spaces_key_delete(access_key: &str, options: serde_json::Value) -> Result<bool, String> {    call("DELETE", &format!("/v2/spaces/keys/{}", enc(access_key)), None, &options)?;    Ok(true)}
Hetzner Cloud hetzner.qwer · 1.0.0Make, power and remove Hetzner Cloud servers, volumes and firewalls, wait on each action and the ones it starts next, read pages and rate-limit headers, and try again when a resource is locked or the limit is reached.cloudserversfirewallapi3 views · 8 downloadsAmazon DynamoDB aws_dynamodb.qwer · 1.0.0Get, put, update, delete and query DynamoDB with plain maps turned into attribute values and back, pages followed, batch writes with UnprocessedItems sent again, throttling retried and each answer's CRC32 checked, signed with Signature V4.awsdatabaseapisigning2 views · 7 downloadsAWS Lambda aws_lambda.qwer · 1.0.0Invoke Lambda functions and wait, or queue them to run later, with function errors read from their header and the log tail's REPORT line turned into numbers, throttling waited out as Retry-After says, and functions listed and waited on, signed with Signature V4.awsserverlessapisigning3 views · 8 downloadsGrafana grafana.qwer · 1.0.0Save, find and remove Grafana dashboards and folders, mark deploys with annotations, run data-source queries with their frames read as rows, keep alert rules, and make service accounts and tokens, with version conflicts and a query's own failure turned into clear errors.monitoringdashboardsalertsapi3 views · 7 downloadsGrafana Loki loki.qwer · 1.0.0Push log lines to Loki with labels and nanosecond times kept as text, run LogQL range and instant queries with streams flattened into one sorted list and metric results as series, and read long ranges past the limit without losing or doubling a line.loggingmonitoringapitime series4 views · 7 downloads
Thread
39Useful? Like it, or add to its thread below.

Thread 0 posts

On DigitalOcean: questions, tips, corrections, and code that uses it.

No posts yet. The first one opens the thread.