Quillo.qweDocs

Guide

Routes

route METHOD /path/:param/:typed(int)/*rest [? { query pattern }] [{ request pattern }] [with effects] = exprroute METHOD /path [with effects] { ... block ... }

If no route matches, the response is 404 for an unknown path, 405 for a known path with the wrong method (with Allow, the methods it takes), and 400 when a route matched the path and method but none of its patterns matched. The 400 is logged with the route that came closest and its first miss (POST /form (main.qwe:6): `form.agree` is missing, and must be a bool), and under --watch the answer says it too (since 0.7). Before any route is tried, a POST, PUT, PATCH or DELETE with a form, multipart or plain-text body (or none) that the browser marks as coming from another site is refused with 403 (see allow_cross_site under Standard library). A last route ANY /*rest is a 404 page of your own; since ANY matches every method, no path then answers 405 or 400.

The statuses the server answers by itself:

StatusWhen
204OPTIONS on a path no OPTIONS route takes, and OPTIONS *, with Allow
304a file's, or a route's own etag, that the browser already holds
400a malformed request line, header or chunked body; a body that is not the gzip it says; a route matched the path and method but no pattern; a server fn call whose body is not a JSON array; a WebSocket handshake that is not a GET with Upgrade and Connection
403a cross-site form post, server fn call or live connection; a phone app built the old way, on 127.0.0.1, whose launch token is missing
404no route; an unknown /_quillo/ path; a file() path refused
405the path is known, the method is not (with Allow)
408a request head not whole after 30 s, or a body of which nothing came for 30 s (logged: POST /up HTTP/1.1 -> 408 (nothing of the body came for 30 s))
409, 422an idempotent request running, or sent again with another body
412a file whose If-Match or If-Unmodified-Since it does not meet
413a body larger than --max-body, sent or once decoded
414a request line over 64 KiB
415a body in a coding other than gzip; a server fn call without content-type: application/json
416a Range past the end of a file
417an Expect other than 100-continue
426a WebSocket handshake of a version other than 13
431a request head over 64 KiB, or more than 100 headers
500a runtime error in the route (the body is internal error; the message is on stderr, and on the page under --watch)
501a transfer-encoding other than chunked; a method HTTP does not define
503the server stopping or its program failed to load; a prefetch refused; too many files being sent or pages waiting (with retry-after: 5)

Route groups

Routes that share a start of their path, middleware or a request pattern are written once for all of them, in a routes group, as Laravel's Route::prefix()->middleware()->group():

fn signed_in(r) = session().user != null ? r : reply 303 redirect(`/login?next=${url_encode(r.path)}`)fn admins(r) = is_admin(session().user) ? r : respond(403, "admins only")routes /admin | signed_in | admins {    route GET / = <AdminHome/>                      // GET /admin    route GET /users = list_users()                 // GET /admin/users    route POST /users/:id(int)/ban { ban(id) }    routes /teams/:team(int) {                      // groups nest: /admin/teams/7/members        route GET /members = members(team)    }}routes /api { headers: { host: "api.example.com" } } {    // Laravel's domain()    route GET /me = me()}routes / | signed_in {                              // no prefix: middleware only    route GET /settings = <Settings/>}routes /reports | signed_in with db {               // what its routes may do, at most    route GET /daily = daily()}

Uploads

A form with enctype="multipart/form-data" sends files. Its text fields are in request.form, as for any form, and each file is in request.files under its field's name: { field, filename, type, size }. The shape follows the count: a field that sent one file holds its record, and a field with several, from multiple, a list of them. A pattern makes it one shape: { files: { docs: [map] } } takes every file of that field as a list, one file a list of one and none an empty list, as [str] does for form fields:

route GET /docs =    <form method="post" action="/docs" enctype="multipart/form-data">        <input type="text" name="title"/>        <input type="file" name="docs" accept=".pdf" multiple/>        <button>Send</button>    </form>route POST /docs { files: { docs: [map] } } {    for d in docs { save_upload(d, `uploads/${uuid()}.pdf`)? }    redirect("/docs")}

The whole round trip for one file: the page with its form, then the route the form posts to. The bytes stay with the request until saved:

route GET /avatar =    <form method="post" action="/avatar" enctype="multipart/form-data">        <label>Your photo <input type="file" name="photo" accept="image/*" required/></label>        <button>Upload</button>    </form>route POST /avatar {    let f = request.files.photo    if f == null { reply 400 "Choose a photo" }    if f.size > 2_000_000 { reply 413 "Photos up to 2 MB" }    let path = save_upload(f, `uploads/${uuid()}.png`)?   // into the program's folder    redirect(`/${path}`)}route GET /uploads/:name = file(`uploads/${name}`)

The form needs method="post" and enctype="multipart/form-data": without the enctype, the browser sends only the file's name. Each file input's name is its key in request.files; accept and required are the browser's own checks, which a visitor can skip, so the route checks again (f == null, f.size). A file chosen in a component goes to a server fn instead, without a form (see save_upload in the reference).

From a component, a file goes to a server fn as an argument: pass it the input's file, and the function gets the same { field, filename, type, size } record a form's upload is, for save_upload:

server fn add_photo(file: map) -> str = save_upload(file, `photos/${uuid()}.jpg`)component Photo() {    state saved = null    <input type="file" accept="image/*" onchange={(e) => { saved = add_photo(e.target.files[0]) }}/>}

A file anywhere in the arguments goes as a file, inside a list or a map too, and a FileList (e.target.files) is a list of files, so add_photos(e.target.files) gets a list of records. Such a call is sent as multipart with an X-Quillo-Rpc header, which pages of other sites cannot add without asking first; a form another site posts is refused.

Checking code from a program

check_program("main.qwe", { "plugins/x.qwe": text }) gives what quillo check says of a program, as { file, line, severity, message } maps, with some files' text given in place of what is on disk: a tool that writes Quillo code can check it before writing it. The code checked may come from users, so it reads only files inside the program's folder: a use, mount or service path with .., starting with /, or leading out of the folder through a link is reported as an error (path must stay inside the program's folder: data/x.qwe), not read. It reads only Quillo's files, .qwe (and the signatures of .qwer and .rs files for a use), no path with a dot folder or file in it, and a link judged by where it leads, so use .env or use .git/config reads nothing; its messages never show a file's text. delete_file(path) removes a file or folder inside the program's folder, but not a folder that holds a database the program has open.

Scripts from users

run_script(source, vars) runs code a program's users write (a rule on a form's field, who may see a record) on the server, and gives its last expression's value, or an error value saying what went wrong:

run_script("value > 3 ? null : `at least 4, got ${value}`", { value: 5 })   // null

Images

image.info("photos/a.jpg")              // { width, height, format, alpha }, or an errorimage.convert("photos/a.jpg", "cache/a-800.jpg", {    crop: { x: 100, y: 0, w: 1200, h: 675 },   // in the image's pixels    width: 800,                                 // shrunk to it, never enlarged    format: "jpeg", quality: 82,                // jpeg, png or webp (lossless)})

format is what the target's name says unless given: .jpg or .jpeg, .png, .webp, in any case (since 0.7: it was JPEG whatever the name, and out.png held a JPEG without its transparency). A name that says none of them (.gif, no extension), with no format, is an error value, and nothing is written. A format given wins over the name.

A photo is read as it is shown: its EXIF orientation is applied, so image.info's width and height are those of the turned image, a crop is in its pixels, and the copy is upright. JPEG has no transparency, so transparent pixels are laid on white in a JPEG (they went black); there is no option for another colour. Each conversion writes to a temporary file of its own and moves it in place, so two at once to one target both finish, the last one's file kept.

JPEG, PNG, WebP and GIF are read. image.info reads only the header, so it is a cheap check that an upload is an image at all. Decoding is limited to 12,000 pixels a side and 512 MiB, so a small file claiming to be enormous is refused before anything is allocated; both run off the event loop. At most four conversions run at once (fewer on a machine with fewer cores), so many uploads together cannot each take 512 MiB; the rest wait their turn, and past 64 waiting image.convert gives an error value at once. Paths stay inside the program's folder.

An upload made into a picture for the page: the form, the route that keeps the file and makes an 800-pixel copy, and the page that shows it:

route GET / =    <form method="post" action="/photos" enctype="multipart/form-data">        <input type="file" name="photo" accept="image/*" required/>        <button>Upload</button>    </form>route POST /photos {    let f = request.files.photo ?? reply 400 "Choose a photo"    let id = uuid()    let original = save_upload(f, `uploads/${id}`)?    let info = image.info(original)    if is_err(info) { delete_file(original); reply 415 "That file is not a JPEG, PNG, WebP or GIF image" }    image.convert(original, `thumbs/${id}.jpg`, { width: 800, format: "jpeg", quality: 82 })?    redirect(`/photos/${id}`)}route GET /photos/:id = <figure>    <img src={`/thumbs/${id}.jpg`} width="800"/>    <figcaption>{image.info(`uploads/${id}`).width} pixels wide, shown at 800</figcaption></figure>route GET /thumbs/:name = file(`thumbs/${name}`, { cache: "1y" })

A photo 1,600 pixels wide goes to its page, 1600 pixels wide, shown at 800, with the copy served as image/jpeg. A text file named .jpg is not kept, and the answer is That file is not a JPEG, PNG, WebP or GIF image (415).

Cookies and sessions

route POST /login { form: { email: str, password: str } } with db {    // stored the same way at signup: one Unicode form, trimmed, in lower case    let user = db.one("select * from users where email = ?", lower(normalize(trim(email))))    if !crypto.verify_password(password, user.password_hash) { reply 401 "Wrong email or password" }    set_session({ user: user.id })    redirect("/")}route GET /me with db {    let id = session().user ?? reply 401 "Log in first"    db.one("select name from users where id = ?", id)}route POST /logout { end_session(); redirect("/") }

The pages that call them: a form that posts the email and password, and a logout button, which is a form too, since /logout takes a POST (a link would be a GET, which a prefetch could follow):

route GET /login =    <html><body>        <form method="post" action="/login">            <label>Email <input type="email" name="email" required/></label>            <label>Password <input type="password" name="password" required/></label>            <button>Log in</button>        </form>    </body></html>route GET / = <html><body>    {session().user == null        ? <a href="/login">Log in</a>        : <form method="post" action="/logout"><button>Log out</button></form>}</body></html>

A wrong password, or an email no one has (user is null, so its password_hash is null, which verify_password says is false), shows the plain text Wrong email or password with status 401. The right one sets the cookie and goes to /, which now shows the button; /me then gives {"name":"Ana"}, and after logging out Log in first (401).

fn touch() {    let s = session()    let day = 24 * 60 * 60 * 1000                      // now() is milliseconds    if s.user != null && now() - (s.at ?? 0) > day {        set_session({ ...s, at: now() }, { max_age: "30d" })    }}

What a route returns

ValueResponse
html200 text/html; XML when its root names a namespace (<urlset xmlns="...">, a sitemap) or it starts <?xml: application/xml, application/rss+xml for <rss>, application/atom+xml for <feed xmlns="...">, and image/svg+xml for <svg xmlns="...">
str200 text/plain
map, list, number, bool200 JSON; one holding a function is a 500, its log line naming where (the answer's value cannot be written as JSON: f is a function, ...), and quillo check warns of a function written in the answer
null204
error500 {"error": ...}
respond(status, body, headers)exactly that
redirect(url, status = 303)a redirect
file(path, { cache })a file under the program's folder (see Static files)
files(folder, path, options) (since 0.7)a folder of files, as a static host serves it (see Static files)
markdown_page(text, { title, lang, html }) (since 0.7)Markdown as a whole page: a doctype, a <head> with the title (the option's, else the first level-one heading's text, # or underlined with =), and the HTML
stream(f, { type, status, headers })sent as f makes it (see Streaming)
events(f, { status, headers })Server-Sent Events, as f sends them
a runtime error500 text/plain, internal error

reply status [body] ends the request right away, from anywhere: inside a helper, a pipeline stage or a ?? fallback. This is how middleware works:

let auth = {    required: (req) =>        req.headers.authorization == `Bearer ${env("TOKEN")}`            ? req            : reply 401 { error: "unauthorized" },}route GET /me = request | auth.required | (r) => { ip: r.ip }route GET /users/:id(int) {    let user = users | find((u) => u.id == id) ?? reply 404 { error: `no user ${id}` }    user}

What a route cannot do

The questions that come up once an app is past its first pages, answered from the server's code:

Idempotent requests

A client that sent a payment and got no answer (a timeout, a dropped connection) cannot tell whether it was made. idempotent(f) lets it send the request again: a request with an Idempotency-Key header runs once, and the same request again with the same key gets the first answer, without running again. It works as Stripe's API does (idempotent requests; the header is the IETF's draft):

route POST /payments { json: { to: str, amount: dec } } = idempotent(() => {    let id = pay(to, amount)?    respond(201, { id })})server fn transfer(from: str, to: str, amount: str) =    idempotent({ isolation: "serializable", retries: 5 }, () => move(from, to, amount))

A client of the route sends the key in a header, and the same key again when it retries:

curl localhost:8080/payments -H "Idempotency-Key: $(uuidgen)" --json '{"to": "bob", "amount": "12.50"}'

A page calls the server fn as any other; it makes the key and sends it by itself, so the component has nothing to add for it:

component Pay() {    state sent = null    <div>        <button onclick={() => { sent = transfer("ana", "bob", "12.50") }}>Send 12.50 to Bob</button>        {is_err(sent) ? <p>Not sent: {error(sent)}</p> : sent && <p>Sent, transfer {sent.id}</p>}    </div>}

Sent twice with one key, the second answer is the first, with Idempotent-Replayed: true, and move has run once:

curl localhost:8080/_quillo/rpc/transfer -H "Idempotency-Key: k-77" --json '["ana", "bob", "12.50"]'# {"ok":{"id":1}}curl -i localhost:8080/_quillo/rpc/transfer -H "Idempotency-Key: k-77" --json '["ana", "bob", "12.50"]'# idempotent-replayed: true# {"ok":{"id":1}}

What is guaranteed. The answers are kept in _quillo_idempotency, in the program's database (the tenant's, with a database per tenant), and f runs in a transaction on that database, which idempotent begins (so it takes isolation and retries, and is called outside db.transaction). The answer is written in that transaction, before it commits: the work and its answer are kept together, or neither is.

Prefetching

Every page a route makes asks for the pages it links to before they are clicked: a link of the same site pointed at (for 65 ms), pressed, touched or reached with Tab. By the time the click lands, the page is there. Browsers with speculation rules (Chrome, Edge) keep the page for the navigation themselves; others get it in the background and keep it in their cache for 30 seconds, which the server allows for this, unless someone has logged in or out.

Streaming

A route can send its answer as it makes it: a large export row by row, a job's progress, a model's reply word by word. The client has the head at once, and each send as it is made.

route GET /orders.csv with db = stream((send) => {    send("id,total\n")    for o in db.query("select id, total from orders") { send(`${o.id},${o.total}\n`) }}, { type: "text/csv", headers: { "content-disposition": "attachment" } })route GET /jobs/:id/progress = events((send) => {    for step in range(10) {        work(id, step)        send({ done: step + 1, of: 10 }, "progress")   // a value, and an event name    }})// A model's answer, passed on as it arrives.route POST /ask with http, env = events((send) => http.stream("https://api.example.com/v1/chat", send, {    body: { prompt: request.json.prompt, stream: true },    headers: { authorization: `Bearer ${env("API_KEY")}` },}))
component Chat() {    state answer = ""    <div>        <button onclick={() => receive("/ask", (part) => { answer = answer + part.text }, { body: { prompt: "hi" } })}>Ask</button>        <p>{answer}</p>    </div>}component Progress(job: int) {    state done = 0    listen(`/jobs/${job}/progress`, (p) => { done = p.done })    <progress value={done} max="10" />}

Each open listen holds one of the six connections a browser keeps to a site over HTTP/1.1, across its tabs. That suits a job's progress or a reply being written; for data every page follows, live (see Live data) shares one WebSocket. Outside the server (a program's own requests in tests), the function runs first and the whole answer is sent at once.

UDP

A program can take UDP packets on a port, answer them, and send its own: readings from sensors, syslog, a game's moves, metrics.

let readings = []udp.listen(9000, (p) => {    readings = readings + [json.parse(p.text)]   // pages with live(...) on it follow    udp.reply(p, "ok")                           // sent back from port 9000})udp.send("10.0.0.5", 514, "<14>app started")

A page that shows the last readings as they come, and a sensor's packet sent from a terminal (nc stands in for the device):

server fn latest() -> list = readings | drop(len(readings) - 10)     // the last tencomponent Readings() {    state rs = live(latest)    <ul>{for r in rs => <li>{r.sensor}: {r.c} °C</li>}</ul>}route GET / = <html><body><Readings/></body></html>
echo '{"sensor": "hall", "c": 21.5}' | nc -u -w1 127.0.0.1 9000     # prints ok, the reply

Each packet adds a line, hall: 21.5 °C, to every page open on /, without a reload.

Channels

A channel carries messages between the server and its pages as they happen: moves in a game, cursors, readings from a device, a chat. Pages send to the program, which answers one page, all of them, or none.

channel.on("game", (msg, peer) => {    // Everyone else in the channel sees the move.    channel.broadcast("game", { id: peer.id, pos: msg.pos }, { except: peer })})component Board() {    state others = {}    let game = channel.open("game", (m) => { others = set(others, m.id, m.pos) })    <canvas onpointermove={(e) => game.send({ pos: [e.clientX, e.clientY] })}/>}

How messages travel:

UDP and channels together pass a device's packets on to pages as they come; see telemetry.qwe.

Media

A program can take a camera's or an encoder's stream as RTP, or a page's camera, and play it in its pages, a fraction of a second behind:

media.rtp("cam", { video: 5004, audio: 5006 })media.on("cam", { watch: (peer) => peer.session.user != null })route GET / = <html><body><quillo-media stream="cam" controls muted/></body></html>
ffmpeg -f avfoundation -framerate 30 -video_size 1280x720 -i "0:0" \  -map 0:v -c:v libx264 -tune zerolatency -profile:v baseline -pix_fmt yuv420p -g 60 \  -f rtp rtp://127.0.0.1:5004 -map 0:a -c:a libopus -f rtp rtp://127.0.0.1:5006

Static files

route GET /assets/*path = files("public", path)route GET /build/*path = files("dist", path, { cache: "1y" })   // names that change with contentroute GET /report = file("exports/report.pdf")

A folder of files, as a static host serves it, is files(folder, path, options):

route GET /*path = files("site", path)route GET /docs/*p = files("manual", p, { index: "home.htm", extensions: ["htm"], not_found: null, cache: "1h" })

quillo serve site serves a folder that way with no program at all: index.html, clean URLs and 404.html, on --port (8080 unless given: quillo serve site --port 3000) and with the other options of run, read as run reads them. Its URLs are its own, so a file asked for with a / after it (/a.txt/, /about/) is moved to its path without one (301, cache-control: max-age=3600), only when it would be served there. A Markdown file goes out as text/markdown; to show one as a page, markdown_page(read_file("notes.md")) gives a whole page (a doctype, a <head> with <title>, the text of the first level-one heading unless title is given, and lang and html as options; html: true keeps HTML written in the text, as markdown does).

A file the program makes as it starts, a stylesheet compiled by an engine in Rust, is declared with build at the top level:

use "tailwind.rs"build "/app.css" = tailwind(css`@import "tailwindcss";`, classes())

Translations

translations("locales")          // locales/en.json, locales/fr.json, locales/pt-BR.jsonview Header(user) = <header>    <a href="/">{t("nav.home")}</a>    <span>{t("greeting", { name: user.name })}</span>    <span>{t("cart.items", { count: len(user.cart) })}</span></header>
{ "nav": { "home": "Accueil" },  "greeting": "Bonjour, {name} !",  "cart": { "items": { "one": "{count} article", "other": "{count} articles" } } }

Mounting programs

mount serves another .qwe program under a path prefix:

mount /notes = "notes.qwe"mount /shop = "shop/app.qwe"route GET / = <a href="/notes">Notes</a>

The mounted program runs unchanged, with its own globals and its own database, and sees paths without the prefix (url().path is /products, not /shop/products). Its root-relative links and form actions (href, src, action), its redirects, its browser bundle, its server fn calls, live data and page changes are all moved under the prefix. Its cookies are set for the prefix (Path=/shop), its session cookie is quillo_session and is signed for that mount, so the two programs' sessions never pass for each other. Its file paths count from its own folder; the services it declares are shared with the program that mounts it. /_quillo/health and /_quillo/metrics are the process's, at the root only. mount, like service, is written at the top level. Changes to mounted files are picked up by --watch (quillo run main.qwe --watch). See index.qwe, which mounts every example.

Services

A part of an app can run in the same process or as a server of its own, without its code changing. service names a program whose server fns the others call:

// billing.qwe: the servicedb.open("billing.db")db.exec("create table if not exists invoices (id integer primary key, name text not null, cents int not null)")server fn invoice(name: str, cents: int) -> map =    db.one("insert into invoices (name, cents) values (?, ?) returning id", name, cents)// shop.qweservice billing = "billing.qwe"route GET / =    <form method="post" action="/buy">            // /shop/buy once served under /shop        <input name="name" placeholder="Your name" required/>        <button>Buy, 12.00</button>    </form>route POST /buy with http {    let inv = billing.invoice(request.form.name, 1200)?    // billing's `server fn invoice`    <p>Invoice #{inv.id}</p>}// app.qwe: the app, made of bothservice /shop = "shop.qwe"          // named after the file; its routes under /shopservice /billing = "billing.qwe"

quillo run app.qwe serves the form at /shop/; sent, it shows Invoice #1, the row billing wrote in its own database. The shop's own paths are written as the shop sees them: action="/buy" reaches the page as /shop/buy, as the shop's links and redirects do (action="/shop/buy" would become /shop/shop/buy). quillo run app.qwe --split answers the same, with billing in a process of its own.

When a service runs elsewhere, its calls are made to hold up:

See services: a shop that bills through a billing service.

Authentication between services

A call signed with QUILLO_SERVICE_SECRET proves one thing: the caller holds the secret. Every service holding it can sign as any other, call any function of any service, and name any tenant, so the most exposed service in a system holds the key to all the others. A caller can prove more than that, and a service can say whom it takes calls from (since 0.7).

The caller says how it proves who it is, where it declares the service:

service billing = "billing.qwe" { auth: { key: env("SHOP_KEY"), as: "shop" } }
authThe call
"quillo" (the default)signed with the shared secret
{ key, as }signed with a key of the caller's own, which the service knows it by: as is the name (the program's service name unless given). A key of notify's cannot sign as shop
{ bearer: f }Authorization: Bearer <token>, the token from f (a function, Quillo's or a Rust file's) at each call, or text
{ oauth: { token_url, client_id, client_secret, scope, audience, method } }a token fetched by OAuth 2.0's client credentials grant (method: "post", the client's secret in the form, or "basic", in Authorization), kept until 30 s before it expires, and fetched again once when the service answers 401
{ mtls: { cert, key, ca } }a client certificate on the connection (PEM files; ca a file or "system"), alone or beside one of the others

A call that proves itself with a token (bearer, oauth) is not also signed with the shared secret, so the service checks the token, not the secret.

service billing = "billing.qwe" { auth: { oauth: {    token_url: env("IDP_TOKEN_URL"), client_id: "shop", client_secret: env("SHOP_CLIENT_SECRET"),    scope: "billing", audience: "billing",} }, forward: ["authorization"] }

forward: ["authorization"] passes the visitor's own headers of the request the call is made in along with it; the service reads them in request.forwarded. A call signed with a key or the secret signs them too, so they cannot be changed on the way; they are still what the visitor sent. quillo check warns of a key or client_secret written in the source, where every copy of the program has it (service billing: its `key` is written in the source, ... read it with env("...")), and of a misspelt option (auht: did you mean auth?). A wrong value stops the program as it loads, saying what is wrong (service billing: `oauth` needs `client_id`).

The service says what it takes, once, at its top level:

// billing.qweaccept_calls {    keys: { shop: [env("SHOP_KEY"), env("SHOP_KEY_OLD")], orders: env("ORDERS_KEY") },    jwt: { issuer: env("IDP"), audience: "billing", jwks: env("IDP") + "/.well-known/jwks.json" },    client_cert: { header: "x-forwarded-client-cert" },    check: (caller) => caller.name != "mallory",}server fn refund(account: int, cents: int) for ["shop"] {    // request.caller: { name: "shop", by: "key" }    { refunded: cents }}

server fn refund(...) for ["shop"] takes calls from those callers only, checked before the arguments are read: another gets 403 (`refund` takes calls from shop only; this is [`notify`](/docs/reference/notify/)). Pages cannot call it at /_quillo/rpc/refund, which answers 404, and their script has no server.refund (since 0.7: anyone could). quillo check refuses for [], for [shop] (a caller is named in plain text, "shop") and for "shop", and quillo docs puts each function's callers in x-quillo-callers, for a gateway to enforce too.

What the function sees, where a service runs elsewhere (in one process a call is a function's call: request.caller is null, and for is not checked):

A call proven several ways is taken by the first that holds, in this order: a key of the caller's own, the shared secret, a token, a certificate. A caller with QUILLO_SERVICE_SECRET set signs every call with it, a bearer one too, so a service that is to check tokens says shared: false.

Answers are signed back with the key the call was signed with, and a caller that signs with a key of its own takes no answer without it (service billing: the answer from http://... is not signed by the service), so a copy that only says it is the service cannot answer for it. Signatures are now version 3, which also covers the caller's name, the deadline and the contract; services still take version 2, so upgrade the services before their callers.

The shared secret is changed without stopping anything: QUILLO_SERVICE_SECRET=new,old (or QUILLO_SERVICE_SECRET=new and QUILLO_SERVICE_SECRET_PREVIOUS=old) signs with the first and takes any. Set both on every machine, then drop the old one. keys take a list for the same.

For a caller that is not a Quillo service (a gateway, a program in another language with an OAuth token), accept_calls { jwt } takes its token at /_quillo/service/<fn> with x-quillo-to and x-quillo-call. Or the service offers routes, a JSON API, checked as middleware with the catalog's jwt.qwer (2.0.0: jwt_check(token, secret, { audience, issuer, require }), exp required unless told otherwise, and jwt_verify_jwks for an identity provider's keys) or oidc.qwer (1.1.0: oidc_client_credentials fetches a token, oidc_verify_access checks one), and called with http.*. That gives up what service brings: quillo check of each call, contracts, retries run once, deadlines, .later and published topics.

use "jwt.qwer"route POST /api/refund {    let token = replace(request.headers.authorization ?? "", "Bearer ", "")    let claims = jwt_check(token, env("JWT_KEY"), { audience: "orders", issuer: "idp" })    if is_err(claims) { reply 401 { error: error(claims) } }    { refunded: request.json.cents, for: claims.sub }}

Copies that join

quillo run billing.qwe --as-service --port 8081 --join http://app:8080 starts one more copy of billing that joins the app while it runs: it says so, signed, every 5 s, and that it is leaving when stopped (the app drops one it has not heard from in 15 s). Its address is the one it connects from, with its --port, called over http://; --join-as https://billing-3.internal:8443 gives the address it is called at instead, which must lead to where it joined from.

Services are found by registration: a copy joins the app (--join), and any server joins limer's pool (--register). A caller does not look a service up by name, so a service the app does not run itself needs --service-url or QUILLO_SERVICE_<NAME>, and QUILLO_SERVICE_SECRET is set on every machine by hand.

The program's API (OpenAPI)

quillo docs app.qwe writes the program's API as OpenAPI 3, from its source: each route's path, query and payload patterns, and each server fn's contract (x-quillo-server-fns, with x-quillo-callers for one declared for [...]), with the routes of the services it serves under their prefixes. Started with the option --describe, quillo run app.qwe --describe, the server serves the same at GET /_quillo/describe, for a gateway to import. A prefixed service's server fns are not in the app's document: they are called by other services, signed, not at /billing/_quillo/rpc. Under --split the app's document still holds the services' routes, read from their source, and --describe is passed down, so /billing/_quillo/describe answers the service's own document, its paths without the prefix.

What the document holds:

/_quillo/describe, when on, is public: it lists every route, those no page links to as well, each server fn's signature, the file and line of every route, and the version of Quillo. Keep it private as /_quillo/metrics is, with a rule at the proxy, or do not turn it on in production.

Calling another Quillo site

(since 0.7) site shop = "https://shop.example.com" names a deployed Quillo site, and shop.notes(a, b) calls its server fn notes as the site's own pages do: POST <url>/_quillo/rpc/notes with the arguments as a JSON array. It gives the answer's value, the site's error as an error value, or an error value when the site cannot be reached. Decimals and big ints cross as they do for pages. The address is any expression:

// shop.qwe, deployed at https://shop.example.comserver fn notes(n: int) -> list = range(1, n + 1) | map((i) => { id: i, text: `note ${i}` })// main.qwe, elsewheresite shop = env("SHOP_URL", "https://shop.example.com")server fn my_notes() = shop.notes(10)route GET / with http = <ul>{for n in my_notes() ?? [] => <li>{n.text}</li>}</ul>

SHOP_URL=http://127.0.0.1:8081 quillo run main.qwe with the shop on that port lists note 1 to note 10; with the shop down, my_notes() is an error value and the list is empty.

Sagas

A saga is a business operation spread over services that share no transaction. Each service commits its own part, and if a later part cannot happen, the earlier parts are undone by compensating steps you write: an order reserves stock, charges a card and books a parcel, and if the parcel cannot be booked, the charge is refunded and the stock released.

Quillo has no saga word. One is a table, a job that moves each row a step at a time, and ordinary calls, and it needs nothing else. The state is written before anything else happens, so a server that stops goes on from there:

db.open("orders.db")db.exec("create table if not exists sagas (id text primary key, cents int not null, state text not null, step int not null, tries int not null, next_at bigint not null, why text)")service inventory = "inventory.qwe" { timeout: "1s", retries: 1 }service payments = "payments.qwe" { timeout: "1s", retries: 1 }let steps = 2fn forward(s) = match s.step {    0 => inventory.reserve(s.id)    _ => payments.charge(s.id, s.cents)}fn undo(s) = match s.step {    0 => inventory.release(s.id)    _ => payments.refund(s.id)}fn move(s, state, step, why) -> map =    db.exec("update sagas set state = ?, step = ?, tries = 0, next_at = 0, why = ? where id = ? and state = ? and step = ?", state, step, why, s.id, s.state, s.step)fn try_again(s, why) -> map =    db.exec("update sagas set tries = ?, next_at = ?, why = ? where id = ?", s.tries + 1, now() + min([1000 * pow(2, s.tries + 1), 60000]), why, s.id)fn advance(s) {    if s.state == "running" {        let r = forward(s)        if is_err(r) {            // No answer: it may have run. Try again, then undo this step too.            if s.tries + 1 >= 3 {                move(s, "undoing", s.step, str(r))            } else {                try_again(s, str(r))            }        } else if !r.ok {            move(s, "undoing", s.step - 1, r.why)        } else if s.step + 1 == steps {            move(s, "done", s.step, null)        } else {            move(s, "running", s.step + 1, null)        }    } else if s.step < 0 {        move(s, "undone", s.step, s.why)    } else {        let r = undo(s)        if is_err(r) {            // An undo is never given up on: it waits for the service.            try_again(s, str(r))        } else {            move(s, "undoing", s.step - 1, s.why)        }    }}fn advance_all() {    for s in db.query("select * from sagas where state in ('running', 'undoing') and next_at <= ? order by id limit 100", now()) {        advance(s)    }}schedule("1s", advance_all)

And a step, the same in every service:

server fn charge(ref: str, cents: int) -> map {    db.transaction(() => {        let was = db.one("select state from charges where ref = ?", ref)        if was != null {            return { ok: was.state == "charged", why: was.state }        }        if cents > 100000 {            return { ok: false, why: "declined" }        }        db.exec("insert into charges (ref, cents, state) values (?, ?, 'charged')", ref, cents)        { ok: true }    })}server fn refund(ref: str) -> map {    db.exec("insert into charges (ref, cents, state) values (?, 0, 'refunded') on conflict (ref) do update set state = 'refunded'", ref)    { ok: true }}

What starts a saga is a row: the order's route writes it and answers at once, and the job does the rest. The visitor follows it on a page that reads the row:

route GET /order =    <form method="post" action="/order">        <input type="number" name="cents" value="2500"/>        <button>Order</button>    </form>route POST /order { form: { cents: int } } with db, random {    let id = uuid()    db.exec("insert into sagas (id, cents, state, step, tries, next_at) values (?, ?, 'running', 0, 0, 0)", id, cents)    redirect(`/order/${id}`)}route GET /order/:id with db {    let s = db.one("select state, why from sagas where id = ?", id) ?? reply 404 "No such order"    match s.state {        "done" => <p>Your order is placed.</p>        "undone" => <p>Your order could not be placed ({s.why}); nothing was charged.</p>        _ => <p>Placing your order… <a href={`/order/${id}`}>See how it is going</a></p>    }}

An order of 2,500 shows "Placing your order…", then, a couple of seconds later, "Your order is placed." One of 200,000 is declined at the charge: the stock is released, and the page says Your order could not be placed (declined); nothing was charged.

Five rules carry it, and leaving out any one breaks it:

  1. The reference is the key. The caller makes it up, and a step keyed on it changes nothing the second time. A job's tries are new calls with new ids, so request.call and idempotent do not do this job here, though they do for one call sent again.
  2. A "no" is an answer, not an error. A step gives { ok: false, why }. An error value means no answer came and the step may have run, which is a different decision.
  3. An undo works on what never happened, and leaves a mark. A refund with no charge writes refunded, so a charge that arrives late is refused. A call that times out keeps running at the service, so this order really happens.
  4. A step with no answer is undone too. It may have run, and rule 3 makes that safe.
  5. An undo never gives up. It waits for the service. If it can never work, a person has to look, so a saga long in undoing is what to alert on.

Put the step that cannot be undone last. And sagas do not isolate each other: between steps, other requests see the stock reserved and the charge taken, so a state such as held must mean something to them.

.later can carry a step too, committing the move with the call: later.status(id) tells how it ended, and a server fn with idempotent is safe across a crash. A job as above stays the simpler choice when the next move depends on the answer.

Thread
35Useful? Like it, or add to its thread below.

Thread 1 post

On Routes: questions, tips, corrections, and code that uses it. Read the thread on its own page →

  1. restimguayopened this thread10 d ago

    Testing Thread

    Edit this page on GitHub · made from the user guide · questions on Discord or r/quillo