UrlInspector
Splits a URL into parts with a query table to edit, encodes on rebuild, flags tracking and double encoding, and shows punycode.
Demo
URL inspector
An absolute URL
- It holds a password, which anyone who sees the URL can read.
- Port 443 is what https uses anyway, so it can go.
- The path looks encoded twice: it has %25 before an escape.
- Scheme
- https
- User
- ada
- Password
- s3cret
- Host
- xn--mnchen-3ya.example
- Reads as
- münchen.example
- Port
- 443
- Path
- /caf%C3%A9/a%2520b
- Path, decoded
- /café/a%20b
- Fragment
- top
| Name | Value | Flags | Remove |
|---|---|---|---|
| tracking | |||
| tracking | |||
| encoded twice | |||
| tracking |
Rebuilt URL
https://ada:s3cret@xn--mnchen-3ya.example:443/caf%C3%A9/a%2520b?utm_source=news&utm_medium=email&q=caf%C3%A9+au+lait&next=%252Fhome&flag&fbclid=abc123#top
Relative, with on_change
A relative URL: it needs a base to lead anywhere
- The path holds a space, which the rebuilt URL encodes.
- Path
- ../img/a b.png
| Name | Value | Flags | Remove |
|---|---|---|---|
Rebuilt URL
../img/a%20b.png?v=2
Resolves to https://example.com/blog/img/a%20b.png?v=2
Change the URL to see on_change.
Get it
It needs theme.qwe, as every component does; or take the whole kit, quillo-ui.qwe. Each component brings its own styles to the pages that show it.
$ curl -O https://quillolang.org/components/ui/theme.qwe$ curl -O https://quillolang.org/components/ui/url_inspector.qwemain.qwe
import ui/theme.qwe use theme_cssuse ui/url_inspector.qweroute GET / = <html><head><style>{theme_css}</style></head><body class="ui-root">...</body></html>Example
The code of the preview above.
demos/url_inspector.qwe
component UrlInspectorDemoLive() { state seen = null <div.demo-stack> <UrlInspector id="url-inspector-demo-live" title="Relative, with on_change" name="link" url="../img/a b.png?v=2" base="https://example.com/blog/post/" on_change={(r) => { seen = r }}/> <p.demo-note>{seen == null ? "Change the URL to see on_change." : `on_change: ${seen.kind}, ${seen.tracking} tracking, ${seen.rebuilt}.`}</p> </div>}view UrlInspectorDemo() = <div.demo-stack> <UrlInspector id="url-inspector-demo" url="https://ada:s3cret@xn--mnchen-3ya.example:443/caf%C3%A9/a%2520b?utm_source=news&utm_medium=email&q=caf%C3%A9+au+lait&next=%252Fhome&flag&fbclid=abc123#top"/> <UrlInspectorDemoLive/> </div>Source
ui/url_inspector.qwe
// UrlInspector: takes a URL apart and puts it back. It splits it into scheme, user, host,// port, path, query and fragment, shows the query as a table to edit, and rebuilds the// URL with what needs encoding encoded and what was fine left as written. It flags// tracking parameters (and takes them out at a press), text encoded twice, bad percent// escapes, a password in the URL and hosts that mix alphabets, shows punycode both ways// (münchen.de is xn--mnchen-3ya.de), and copes with what is not an absolute URL: it says// so of a relative one and resolves it against a base, and says what is wrong with an// invalid one, by part.//// <UrlInspector url="https://example.com/a?utm_source=x&id=7#top"/>// <UrlInspector url="../img/a.png" base="https://example.com/blog/post/" on_change={(r) => { clean = r.rebuilt }}/>//// url: the URL it starts with. base: what a relative URL is resolved against; it gets a// field when the URL is relative. title: its heading. id: a prefix for its ids, unique on// the page. name: the field's name ("url"; the base's is <name>_base). tracking: more// parameter names to flag, as a list. on_change(result): inside one of your components,// given { url, rebuilt, kind, valid, tracking } after a change, tracking being how many// are flagged. class: more classes for its outer element.//// url_inspector_parse(text) gives the parts: { kind ("absolute", "relative", "scheme-// relative", "opaque", "invalid" or "empty"), scheme, user, password, host, host_ascii,// host_unicode, port, path, query, fragment, problems, notes }; url_inspector_rows(query)// the rows; url_inspector_rebuild(parts, rows) the URL; url_inspector_resolve(base, ref)// RFC 3986's resolution; url_inspector_to_ascii(host) and url_inspector_to_unicode(host)// punycode. Host names are lowered and encoded by RFC 3492, without the rest of UTS 46.import ui/theme.qwe use ui_clslet url_inspector_css = css`.ui-url-insp { display: grid; gap: 12px; min-width: 0; padding: 20px 22px; border: 1px solid var(--ui-border); border-radius: var(--ui-radius); background: var(--ui-surface); color: var(--ui-fg); font: 14px/1.45 var(--ui-font); }.ui-url-insp .ui-url-insp-title { margin: 0; font-size: 16px; font-weight: 650; }.ui-url-insp-field { display: grid; gap: 5px; min-width: 0; }.ui-url-insp-field label { font-size: 13px; font-weight: 500; }.ui-url-insp-input { box-sizing: border-box; width: 100%; height: 34px; padding: 0 10px; border: 1px solid var(--ui-border); border-radius: var(--ui-radius-sm); background: var(--ui-surface); color: var(--ui-fg); font: 13px var(--ui-mono); }.ui-url-insp-input:focus-visible { outline: none; border-color: var(--ui-accent); box-shadow: var(--ui-focus); }.ui-url-insp-input[aria-invalid] { border-color: var(--ui-danger); }.ui-url-insp-kind { margin: 0; font-weight: 600; }.ui-url-insp-list { display: grid; gap: 4px; margin: 0; padding: 0; list-style: none; font-size: 13px; }.ui-url-insp-list li { padding: 6px 10px; border-left: 4px solid var(--ui-warning); border-radius: var(--ui-radius-sm); background: var(--ui-warning-soft); overflow-wrap: anywhere; }.ui-url-insp-list.ui-url-insp-bad li { border-left-color: var(--ui-danger); background: var(--ui-danger-soft); }.ui-url-insp-parts { display: grid; gap: 2px; margin: 0; font-size: 13px; }.ui-url-insp-part { display: grid; grid-template-columns: 100px minmax(0, 1fr); gap: 12px; }.ui-url-insp-parts dt { color: var(--ui-muted); }.ui-url-insp-parts dd { margin: 0; font-family: var(--ui-mono); overflow-wrap: anywhere; }.ui-url-insp-scroll { position: relative; overflow-x: auto; }.ui-url-insp-scroll:focus-visible { outline: none; box-shadow: var(--ui-focus); }.ui-url-insp table { width: 100%; border-collapse: collapse; font-size: 13px; }.ui-url-insp caption { padding: 0 0 6px; text-align: left; font-weight: 600; }.ui-url-insp th, .ui-url-insp td { padding: 4px 6px 4px 0; border-bottom: 1px solid var(--ui-border); text-align: left; vertical-align: middle; }.ui-url-insp thead th { color: var(--ui-muted); font-weight: 500; }.ui-url-insp td .ui-url-insp-input { min-width: 110px; }.ui-url-insp-flag { display: inline-block; margin: 1px 4px 1px 0; padding: 0 6px; border-radius: 999px; background: var(--ui-warning-soft); color: var(--ui-warning); font-size: 11px; font-weight: 600; white-space: nowrap; }.ui-url-insp-actions { display: flex; flex-wrap: wrap; align-items: center; gap: 8px; }.ui-url-insp-btn { height: 30px; padding: 0 10px; border: 1px solid var(--ui-border); border-radius: var(--ui-radius-sm); background: var(--ui-surface); color: var(--ui-fg); font: 500 13px/1 var(--ui-font); white-space: nowrap; cursor: pointer; }.ui-url-insp-btn:hover { background: var(--ui-subtle); }.ui-url-insp-btn:focus-visible { outline: none; box-shadow: var(--ui-focus); }.ui-url-insp-out { margin: 0; padding: 8px 10px; border-radius: var(--ui-radius-sm); background: var(--ui-subtle); font: 13px/1.5 var(--ui-mono); overflow-wrap: anywhere; }.ui-url-insp-note { margin: 0; color: var(--ui-muted); font-size: 13px; }`let url_inspector_most = 20000let url_inspector_most_rows = 500let url_inspector_re = "^(?:([^:/?#]+):)?(?://([^/?#]*))?([^?#]*)(?:\\?([^#]*))?(?:#(.*))?$"let url_inspector_hier = ["http", "https", "ws", "wss", "ftp", "ftps"]let url_inspector_ports = { http: "80", https: "443", ws: "80", wss: "443", ftp: "21" }let url_inspector_track_names = ["gclid", "gclsrc", "dclid", "gbraid", "wbraid", "fbclid", "msclkid", "mc_cid", "mc_eid", "igshid", "yclid", "twclid", "ttclid", "li_fat_id", "_hsenc", "_hsmi", "vero_id", "mkt_tok", "ref_src", "s_cid", "spm", "_ga", "_gl"]let url_inspector_digits = "abcdefghijklmnopqrstuvwxyz0123456789"let url_inspector_hex = "0123456789abcdef"fn url_inspector_hex_value(c: str) = index_of(url_inspector_hex, lower(c))// A code point as text: JSON's escapes are the only way Quillo has to make a character// from its number.fn url_inspector_escape(u: int) -> str { let h = (n: int) => slice(url_inspector_hex, n, n + 1) `\\u${h(floor(u / 4096))}${h(floor(u / 256) % 16)}${h(floor(u / 16) % 16)}${h(u % 16)}`}fn url_inspector_char(cp: int) -> str { if cp < 65536 { return json.parse(`"${url_inspector_escape(cp)}"`) } let v = cp - 65536 json.parse(`"${url_inspector_escape(55296 + floor(v / 1024))}${url_inspector_escape(56320 + v % 1024)}"`)}// The bytes of `%41%42`.fn url_inspector_bytes(run: str) -> list = for i in 0..floor(len(run) / 3) => url_inspector_hex_value(run[i * 3 + 1]) * 16 + url_inspector_hex_value(run[i * 3 + 2])let url_inspector_ascii = join(range(32, 127) | map(url_inspector_char), "")// A character's number, by the only road there is: a table for ASCII, which url_encode// may leave alone, and its percent escapes for the rest.fn url_inspector_code(c: str) -> int { let at = index_of(url_inspector_ascii, c) if at != null { return at + 32 } let b = url_inspector_bytes(url_encode(c)) if len(b) == 1 { return b[0] } if len(b) == 2 { return (b[0] - 192) * 64 + b[1] - 128 } if len(b) == 3 { return (b[0] - 224) * 4096 + (b[1] - 128) * 64 + b[2] - 128 } (b[0] - 240) * 262144 + (b[1] - 128) * 4096 + (b[2] - 128) * 64 + b[3] - 128}// Bytes as UTF-8: { text, bad (some were not) }. Bad ones become U+FFFD.fn url_inspector_utf8(bytes: list) -> map { let parts = [] let bad = false let n = len(bytes) let i = 0 while i < n { let b = bytes[i] let need = b < 128 ? 0 : (b >= 194 && b <= 223 ? 1 : (b >= 224 && b <= 239 ? 2 : (b >= 240 && b <= 244 ? 3 : -1))) let cp = need == 0 ? b : (need == 1 ? b - 192 : (need == 2 ? b - 224 : b - 240)) let ok = need >= 0 let j = 1 while ok && j <= need { let c = i + j < n ? bytes[i + j] : 0 if c < 128 || c > 191 { ok = false } else { cp = cp * 64 + c - 128; j += 1 } } if ok && ((need == 2 && (cp < 2048 || (cp >= 55296 && cp <= 57343))) || (need == 3 && (cp < 65536 || cp > 1114111))) { ok = false } if ok { parts = push(parts, url_inspector_char(cp)) i += need + 1 } else { parts = push(parts, "\u{FFFD}") bad = true i += 1 } } { text: join(parts, ""), bad }}// Percent escapes read, `+` as a space in a query: { text, bad_escape, bad_utf8, twice }.// twice: what came out still looks escaped, so the text was encoded two times.fn url_inspector_decode(raw: str, plus: bool) -> map { let spaced = plus ? replace(raw, "+", " ") : raw let bad_utf8 = false let text = re.replace(spaced, "(?:%[0-9A-Fa-f]{2})+", (m) => { let r = url_inspector_utf8(url_inspector_bytes(m.text)) if r.bad { bad_utf8 = true } r.text }) let bad_escape = any(re.find_all(spaced, "%[0-9A-Fa-f]{2}|%.{0,2}", "s"), (m) => !re.test(m.text, "^%[0-9A-Fa-f]{2}$")) { text, bad_escape, bad_utf8, twice: re.test(raw, "%25[0-9A-Fa-f]{2}") }}// Text with what a URL cannot hold encoded, and valid escapes and legal characters as// they are. `allowed` is a class of what else may stay.fn url_inspector_keep(text: str, allowed: str) -> str = re.replace(text, `%[0-9A-Fa-f]{2}|[^A-Za-z0-9._~!$&'()*+,;=:@${allowed}-]`, (m) => len(m.text) == 3 && starts_with(m.text, "%") ? m.text : url_encode(m.text))// ---- Punycode, RFC 3492 ----------------------------------------------------fn url_inspector_adapt(delta0: int, points: int, first: bool) -> int { let delta = floor((first ? floor(delta0 / 700) : floor(delta0 / 2))) delta = delta + floor(delta / points) let k = 0 while delta > 455 { delta = floor(delta / 35) k += 36 } k + floor(36 * delta / (delta + 38))}fn url_inspector_threshold(k: int, bias: int) -> int = k <= bias ? 1 : (k >= bias + 26 ? 26 : k - bias)// A label's code points from what follows `xn--`, or null if it cannot be.fn url_inspector_puny_decode(input: str) { let chars = split(input, "") let dash = -1 for p in enumerate(chars) { if p[1] == "-" { dash = p[0] } } let basic = dash < 0 ? [] : take(chars, dash) if any(basic, (c) => !re.test(c, "^[\\x00-\\x7f]$")) { return null } let output = basic | map((c) => url_inspector_code(c)) let rest = drop(chars, dash + 1) let n = 128 let i = 0 let bias = 72 let pos = 0 let total = len(rest) while pos < total { let old = i let w = 1 let k = 36 let going = true while going { if pos >= total { return null } let digit = index_of(url_inspector_digits, lower(rest[pos])) pos += 1 if digit == null { return null } i = i + digit * w let t = url_inspector_threshold(k, bias) if digit < t { going = false } else { w = w * (36 - t) k += 36 } if i > 2000000000 { return null } } let size = len(output) + 1 bias = url_inspector_adapt(i - old, size, old == 0) n = n + floor(i / size) i = i % size if n > 1114111 || (n >= 55296 && n <= 57343) { return null } output = take(output, i) + [n] + drop(output, i) i += 1 } // Control characters, and a label with nothing to encode, decode by the letter of // the RFC, but no name is made of them. if any(output, (c) => c >= 128 && c < 160) || all(output, (c) => c < 128) { return null } output}// What goes after `xn--` for a label of code points.fn url_inspector_puny_encode(cps: list) -> str { let basic = cps | filter((c) => c < 128) let out = basic | map((c) => url_inspector_char(c)) let b = len(basic) let h = b if b > 0 { out = push(out, "-") } let n = 128 let delta = 0 let bias = 72 let total = len(cps) while h < total { let m = min(cps | filter((c) => c >= n)) delta = delta + (m - n) * (h + 1) n = m for c in cps { if c < n { delta += 1 } if c == n { let q = delta let k = 36 let going = true while going { let t = url_inspector_threshold(k, bias) if q < t { going = false } else { let d = t + (q - t) % (36 - t) out = push(out, slice(url_inspector_digits, d, d + 1)) q = floor((q - t) / (36 - t)) k += 36 } } out = push(out, slice(url_inspector_digits, q, q + 1)) bias = url_inspector_adapt(delta, h + 1, h == b) delta = 0 h += 1 } } delta += 1 n += 1 } join(out, "")}// A host in ASCII, each label that needs it as xn--.fn url_inspector_to_ascii(host: str) -> str = join(split(lower(host), ".") | map((label) => re.test(label, "^[\\x00-\\x7f]*$") ? label : `xn--${url_inspector_puny_encode(split(label, "") | map((c) => url_inspector_code(c)))}`), ".")// A host for reading, each xn-- label decoded; a label that will not decode stays.fn url_inspector_to_unicode(host: str) -> str = join(split(host, ".") | map((label) => { if !starts_with(lower(label), "xn--") { return label } let d = url_inspector_puny_decode(slice(label, 4, len(label))) d == null ? label : join(d | map((c) => url_inspector_char(c)), "") }), ".")// ---- The URL ----------------------------------------------------------------// Trouble with a host: text for each.fn url_inspector_host_problems(host: str) -> list { let out = [] if re.test(host, "[\\s<>\"\\\\^`{|}%]") { out = push(out, `The host "${host}" holds a character a host cannot: a space, or one of < > " \\ ^ \` { | } %.`) } if len(host) > 253 { out = push(out, "The host is longer than the 253 characters a name can have.") } for label in split(host, ".") { if len(label) > 63 { out = push(out, `The host part "${slice(label, 0, 20)}…" is longer than the 63 characters a part can have.`) } } if contains(host, "..") || starts_with(host, ".") { out = push(out, `The host "${host}" has an empty part between dots.`) } for label in split(host, ".") { if starts_with(lower(label), "xn--") && url_inspector_puny_decode(slice(label, 4, len(label))) == null { out = push(out, `The host part "${label}" is not valid punycode.`) } } out}// A host that mixes alphabets can pass for another: аpple.com with a Cyrillic а.fn url_inspector_mixed(unicode_host: str) -> str? { let found = find(split(unicode_host, "."), (l) => re.test(l, "[a-zA-Z]") && re.test(l, "[Ѐ-ӿͰ-Ͽ]")) found == null ? null : `The host part "${found}" mixes Latin with Cyrillic or Greek letters, which can pass for another name.`}// Authority text into its parts, and what is wrong with it.fn url_inspector_authority(a: str) -> map { let at = re.find_all(a, "@") let info = len(at) > 0 ? slice(a, 0, at[-1].start) : null let rest = len(at) > 0 ? slice(a, at[-1].end, len(a)) : a let colon = info == null ? null : index_of(info, ":") let user = info == null ? null : (colon == null ? info : slice(info, 0, colon)) let password = colon == null ? null : slice(info, colon + 1, len(info)) let v6 = re.find(rest, "^(\\[[^\\]]*\\])(?::(.*))?$") let plain = v6 == null ? re.find(rest, "^([^:]*)(?::([^:]*))?$") : null let open_bracket = v6 == null && starts_with(rest, "[") let host = open_bracket ? rest : (v6 != null ? v6.groups[0] : (plain != null ? plain.groups[0] : slice(rest, 0, index_of(rest, ":")))) let port = open_bracket ? null : (v6 != null ? v6.groups[1] : (plain != null ? plain.groups[1] : null)) let problems = [] if open_bracket { problems = push(problems, `The host "${rest}" opens a [ and never closes it.`) } if v6 != null && !re.test(v6.groups[0], "^\\[[0-9A-Fa-f:.]+\\]$") { problems = push(problems, `${v6.groups[0]} is not an IPv6 address.`) } if v6 == null && !open_bracket && plain == null { problems = push(problems, `The host "${rest}" has more than one colon. An IPv6 address needs [ ].`) } if port != null && port != "" { let bad = !re.test(port, "^\\d{1,5}$") || int(port) > 65535 if bad { problems = push(problems, `The port "${port}" is not a number from 0 to 65535.`) } } { user, password, host, port: port == "" ? null : port, problems: v6 != null || open_bracket || plain == null ? problems : problems + url_inspector_host_problems(host) }}// Reads a URL. See the top of the file.fn url_inspector_parse(text: str) -> map { let parts = { raw: text, kind: "empty", scheme: null, user: null, password: null, host: null, host_ascii: null, host_unicode: null, port: null, path: "", query: null, fragment: null, problems: [], notes: [] } let t = trim(text) if t == "" { return parts } if len(t) > url_inspector_most { return { ...parts, kind: "invalid", problems: [`This is ${format_number(len(t), "en-US", 0)} characters, past the ${format_number(url_inspector_most, "en-US", 0)} it will read.`] } } let notes = t != text ? ["Space around the URL is left out."] : [] let problems = [] let inner = re.test(t, "[\\t\\n\\r]") if inner { notes = push(notes, "Tabs and line breaks inside it are left out, as browsers do.") } let s = re.replace(t, "[\\t\\n\\r]", "") let m = re.find(s, url_inspector_re, "s") let g = m.groups let scheme_raw = g[0] let good_scheme = scheme_raw != null && re.test(scheme_raw, "^[A-Za-z][A-Za-z0-9+.-]*$") if scheme_raw != null && !good_scheme && g[1] != null { return { ...parts, kind: "invalid", path: g[2] ?? "", problems: [`"${scheme_raw}" cannot be a scheme: it starts with a letter, then letters, digits, + . or -.`], notes } } // host:8080/path reads as the scheme "host", which is never what was meant. if good_scheme && g[1] == null && re.test(g[2] ?? "", "^\\d{1,5}(/|$)") && !contains(url_inspector_hier, lower(scheme_raw)) { return { ...parts, kind: "invalid", problems: [`This reads as the scheme "${scheme_raw}", but looks like a host and port. Add https:// in front.`], notes } } let scheme = good_scheme ? lower(scheme_raw) : null let path = scheme_raw != null && !good_scheme ? `${scheme_raw}:${g[2] ?? ""}` : (g[2] ?? "") let authority = g[1] if contains(s, "\\") { problems = push(problems, "It holds a backslash, which a URL does not: browsers read it as /, others do not.") } if scheme != null && authority == null && contains(url_inspector_hier, scheme) { return { ...parts, kind: "invalid", scheme, path, problems: problems + [`${scheme} needs // after the colon: ${scheme}://${re.replace(path, "^/+", "")}`], notes } } if scheme != null && authority == null { if scheme == "javascript" { notes = push(notes, "This runs script when followed. Never put one from a visitor in a link.") } return { ...parts, kind: "opaque", scheme, path, query: g[3], fragment: g[4], problems, notes } } let kind = scheme != null ? "absolute" : (authority != null ? "scheme-relative" : "relative") let a = authority != null ? url_inspector_authority(authority) : { user: null, password: null, host: null, port: null, problems: [] } if authority != null && (a.host ?? "") == "" && contains(url_inspector_hier, scheme ?? "http") { problems = push(problems, "There is no host after the //.") } problems = problems + a.problems if kind == "relative" && re.test(path, "^[a-z0-9-]+(\\.[a-z0-9-]+)*\\.[a-z]{2,}(/|$)", "i") { notes = push(notes, `This is a relative path. If ${split(path, "/")[0]} is a host, add https:// in front.`) } let host_ascii = a.host == null ? null : (starts_with(a.host, "[") ? lower(a.host) : url_inspector_to_ascii(a.host)) let host_unicode = host_ascii == null ? null : url_inspector_to_unicode(host_ascii) if a.password != null { notes = push(notes, "It holds a password, which anyone who sees the URL can read.") } let mixed = host_unicode == null ? null : url_inspector_mixed(host_unicode) if mixed != null { notes = push(notes, mixed) } if a.port != null && get(url_inspector_ports, scheme ?? "") == a.port { notes = push(notes, `Port ${a.port} is what ${scheme} uses anyway, so it can go.`) } if url_inspector_decode(path, false).twice { notes = push(notes, "The path looks encoded twice: it has %25 before an escape.") } if url_inspector_decode(path, false).bad_escape { problems = push(problems, "The path has a % that does not start an escape.") } if re.test(path, "\\s") { notes = push(notes, "The path holds a space, which the rebuilt URL encodes.") } { ...parts, kind: len(problems) > 0 ? "invalid" : kind, scheme, user: a.user, password: a.password, host: a.host, host_ascii, host_unicode, port: a.port, path, query: g[3], fragment: g[4], problems, notes }}// Whether a parameter is a tracking one.fn url_inspector_is_tracking(key: str, extra: list) -> bool { let k = lower(key) starts_with(k, "utm_") || contains(url_inspector_track_names, k) || contains(extra | map((x) => lower(x)), k)}// The query's rows: { id, key, value, eq (had an =), raw (the text as it came, so an// unedited row is not re-encoded), flags }. A list made by pieces, past the limit left.fn url_inspector_rows(query, first_id: int = 0) -> list { if query == null || query == "" { return [] } let pieces = take(split(query, "&") | filter((p) => p != ""), url_inspector_most_rows) enumerate(pieces) | map((p) => { let at = index_of(p[1], "=") let raw_key = at == null ? p[1] : slice(p[1], 0, at) let raw_value = at == null ? "" : slice(p[1], at + 1, len(p[1])) let k = url_inspector_decode(raw_key, true) let v = url_inspector_decode(raw_value, true) { id: first_id + p[0], key: k.text, value: v.text, eq: at != null, raw: p[1], twice: k.twice || v.twice, bad_escape: k.bad_escape || v.bad_escape, bad_utf8: k.bad_utf8 || v.bad_utf8 } })}fn url_inspector_flags(r: map, extra: list) -> list = [url_inspector_is_tracking(r.key, extra) ? "tracking" : null, r.twice ? "encoded twice" : null, r.bad_escape ? "bad escape" : null, r.bad_utf8 ? "not UTF-8" : null, r.key == "" ? "no name" : null] | filter((f) => f != null)fn url_inspector_row_text(r: map) -> str { if r.raw != null { return url_inspector_keep(r.raw, "/?") } r.eq ? `${url_encode(r.key)}=${url_encode(r.value)}` : url_encode(r.key)}// The URL again, from the parts and the rows, encoded.fn url_inspector_rebuild(parts: map, rows: list) -> str { let scheme = parts.scheme != null ? `${parts.scheme}:` : "" let info = parts.user == null ? "" : `${url_inspector_keep(parts.user, "")}${parts.password != null ? `:${url_inspector_keep(parts.password, "")}` : ""}@` let port = parts.port != null ? `:${parts.port}` : "" let authority = parts.host != null ? `//${info}${parts.host_ascii}${port}` : "" let path = url_inspector_keep(parts.path, "/") let q = join(rows | map(url_inspector_row_text), "&") let query = len(rows) > 0 ? `?${q}` : (parts.query != null && parts.kind == "opaque" ? `?${parts.query}` : "") let fragment = parts.fragment != null ? `#${url_inspector_keep(parts.fragment, "/?")}` : "" `${scheme}${authority}${path}${query}${fragment}`}// RFC 3986 section 5.2.4.fn url_inspector_dots(path: str) -> str { let segs = split(path, "/") let out = [] let n = len(segs) for p in enumerate(segs) { let s = p[1] let last = p[0] == n - 1 if s == ".." { if len(out) > 1 { out = slice(out, 0, len(out) - 1) } if last { out = push(out, "") } } else if s == "." { if last { out = push(out, "") } } else { out = push(out, s) } } join(out, "/")}// RFC 3986 section 5.2.2: ref against an absolute base, or null if base is not one.fn url_inspector_resolve(base: str, ref: str) -> str? { let b = url_inspector_parse(base) if b.kind != "absolute" { return null } let g = re.find(re.replace(trim(ref), "[\\t\\n\\r]", ""), url_inspector_re, "s").groups let auth = (a: str) => `${b.scheme}://${a}` let tail = (q, f) => `${q != null ? `?${q}` : ""}${f != null ? `#${f}` : ""}` if g[0] != null { return `${lower(g[0])}:${g[1] != null ? `//${g[1]}` : ""}${url_inspector_dots(g[2] ?? "")}${tail(g[3], g[4])}` } if g[1] != null { return `${auth(g[1])}${url_inspector_dots(g[2] ?? "")}${tail(g[3], g[4])}` } let here = re.find(trim(base), url_inspector_re, "s").groups let base_path = here[2] ?? "" let path = g[2] ?? "" if path == "" { return `${auth(here[1])}${base_path}${tail(g[3] ?? here[3], g[4])}` } if starts_with(path, "/") { return `${auth(here[1])}${url_inspector_dots(path)}${tail(g[3], g[4])}` } let cut = re.find_all(base_path, "/") let dir = len(cut) == 0 ? "/" : slice(base_path, 0, cut[-1].end) `${auth(here[1])}${url_inspector_dots(`${dir}${path}`)}${tail(g[3], g[4])}`}fn url_inspector_kind_text(kind: str) -> str = match kind { "absolute" => "An absolute URL" "relative" => "A relative URL: it needs a base to lead anywhere" "scheme-relative" => "A scheme-relative URL: it takes http or https from the page" "opaque" => "A URL with no host" "invalid" => "Not a valid URL" _ => ""}view UrlInspectorPart(label: str, part: str, value: str) = <div.ui-url-insp-part><dt>{label}</dt><dd data-part={part}>{value}</dd></div>component UrlInspector(url: str = "", base: str = "", title: str = "URL inspector", id: str = "url-inspector", name: str = "url", tracking = [], on_change = null, class: str? = null) { style global url_inspector_css state text = url state base_text = base let first = url_inspector_parse(url) state rows = url_inspector_rows(first.query) state next = 1000 state copied = "" let p = url_inspector_parse(text) let usable = p.kind != "empty" && p.kind != "invalid" let rebuilt = usable ? url_inspector_rebuild(p, rows) : "" let resolved = usable && (p.kind == "relative" || p.kind == "scheme-relative") && trim(base_text) != "" ? url_inspector_resolve(base_text, rebuilt) : null let base_bad = usable && (p.kind == "relative" || p.kind == "scheme-relative") && trim(base_text) != "" && resolved == null let flagged = rows | filter((r) => url_inspector_is_tracking(r.key, tracking)) let cut_rows = len(split(p.query ?? "", "&")) > url_inspector_most_rows let tell = () => { if on_change != null { let q = url_inspector_parse(text) let ok = q.kind != "empty" && q.kind != "invalid" on_change({ url: text, rebuilt: ok ? url_inspector_rebuild(q, rows) : null, kind: q.kind, valid: ok, tracking: count(rows, (r) => url_inspector_is_tracking(r.key, tracking)) }) } } let typed = (e) => { text = e.value rows = url_inspector_rows(url_inspector_parse(e.value).query) copied = "" tell() } let edit = (r, field: str, v: str) => { rows = rows | map((x) => x.id == r.id ? (field == "key" ? { ...x, key: v, raw: null, eq: true, twice: false, bad_escape: false, bad_utf8: false } : { ...x, value: v, raw: null, eq: true, twice: false, bad_escape: false, bad_utf8: false }) : x) copied = "" tell() } let copy = () => { js("navigator.clipboard.writeText", rebuilt).then((_ = null) => { copied = "Copied" }, (_ = null) => { copied = "Could not copy: select the text instead" }) } <section class={ui_cls(["ui-url-insp", class])} aria-labelledby={`${id}-title`}> <h3.ui-url-insp-title id={`${id}-title`}>{title}</h3> <div.ui-url-insp-field> <label for={`${id}-url`}>URL</label> <input.ui-url-insp-input type="text" inputmode="url" id={`${id}-url`} name={name} value={text} spellcheck="false" autocomplete="off" placeholder="https://example.com/path?a=1#top" aria-invalid={p.kind == "invalid" ? "true" : null} @input={typed}/> </div> {p.kind == "empty" && <p.ui-url-insp-note>Paste a URL to take it apart.</p>} {p.kind != "empty" && <p.ui-url-insp-kind data-kind={p.kind} aria-live="polite">{url_inspector_kind_text(p.kind)}</p>} {len(p.problems) > 0 && <ul class="ui-url-insp-list ui-url-insp-bad" role="alert">{for x in p.problems => <li key={x}>{x}</li>}</ul>} {len(p.notes) > 0 && <ul.ui-url-insp-list aria-label="Notes">{for x in p.notes => <li key={x}>{x}</li>}</ul>} {usable && (p.kind == "relative" || p.kind == "scheme-relative") && <div.ui-url-insp-field> <label for={`${id}-base`}>Base URL</label> <input.ui-url-insp-input type="text" inputmode="url" id={`${id}-base`} name={`${name}_base`} value={base_text} spellcheck="false" autocomplete="off" placeholder="https://example.com/page/" aria-invalid={base_bad ? "true" : null} @input={(e) => { base_text = e.value }}/> </div>} {base_bad && <p.ui-url-insp-note role="alert">The base must be an absolute URL.</p>} {usable && <dl.ui-url-insp-parts> {p.scheme != null && <UrlInspectorPart label="Scheme" part="scheme" value={p.scheme}/>} {p.user != null && <UrlInspectorPart label="User" part="user" value={p.user}/>} {p.password != null && <UrlInspectorPart label="Password" part="password" value={p.password}/>} {p.host != null && <UrlInspectorPart label="Host" part="host" value={p.host_ascii}/>} {p.host != null && p.host_unicode != p.host_ascii && <UrlInspectorPart label="Reads as" part="unicode" value={p.host_unicode}/>} {p.port != null && <UrlInspectorPart label="Port" part="port" value={p.port}/>} {p.path != "" && <UrlInspectorPart label="Path" part="path" value={p.path}/>} {p.path != "" && url_inspector_decode(p.path, false).text != p.path && <UrlInspectorPart label="Path, decoded" part="decoded" value={url_inspector_decode(p.path, false).text}/>} {p.fragment != null && <UrlInspectorPart label="Fragment" part="fragment" value={p.fragment}/>} </dl>} {usable && p.kind != "opaque" && <div.ui-url-insp-field> {len(rows) > 0 && <div.ui-url-insp-scroll tabindex="0" role="region" aria-label="Query parameters"> <table> <caption>{`Query: ${len(rows)} ${len(rows) == 1 ? "parameter" : "parameters"}`}</caption> <thead><tr><th scope="col">Name</th><th scope="col">Value</th><th scope="col">Flags</th><th scope="col"><span.ui-sr>Remove</span></th></tr></thead> <tbody> {for r in rows => <tr key={str(r.id)}> <td><input.ui-url-insp-input type="text" name={`${name}_key`} aria-label={`Name of parameter ${r.key}`} value={r.key} spellcheck="false" autocomplete="off" @input={(e) => { edit(r, "key", e.value) }}/></td> <td><input.ui-url-insp-input type="text" name={`${name}_value`} aria-label={`Value of ${r.key}`} value={r.value} spellcheck="false" autocomplete="off" @input={(e) => { edit(r, "value", e.value) }}/></td> <td data-flags>{for f in url_inspector_flags(r, tracking) => <span.ui-url-insp-flag key={f}>{f}</span>}</td> <td><button.ui-url-insp-btn type="button" aria-label={`Remove ${r.key}`} @click={() => { rows = remove(rows, r); tell() }}>Remove</button></td> </tr>} </tbody> </table> </div>} {cut_rows && <p.ui-url-insp-note>{`Only the first ${url_inspector_most_rows} parameters are read.`}</p>} <div.ui-url-insp-actions> <button.ui-url-insp-btn type="button" data-add @click={() => { rows = push(rows, { id: next, key: "", value: "", eq: true, raw: null, twice: false, bad_escape: false, bad_utf8: false }); next += 1; tell() }}>Add parameter</button> {len(flagged) > 0 && <button.ui-url-insp-btn type="button" data-strip @click={() => { rows = rows | filter((r) => !url_inspector_is_tracking(r.key, tracking)); tell() }}>{`Remove ${len(flagged)} tracking ${len(flagged) == 1 ? "parameter" : "parameters"}`}</button>} </div> </div>} {usable && <div.ui-url-insp-field> <span.ui-sr id={`${id}-rebuilt-label`}>Rebuilt URL</span> <p.ui-url-insp-out data-rebuilt aria-labelledby={`${id}-rebuilt-label`}>{rebuilt}</p> {resolved != null && <p.ui-url-insp-note>Resolves to <span data-resolved>{resolved}</span></p>} <div.ui-url-insp-actions> <button.ui-url-insp-btn type="button" data-copy @click={copy}>Copy</button> <span.ui-url-insp-note aria-live="polite">{copied}</span> </div> </div>} </section>}Related
HttpRequestBuilderBuilds an HTTP request as curl, fetch and HTTPie with correct shell escaping, warns on a body on GET and a missing content type, and hides secrets until asked.3 views · 5 downloadsJsonFormatterFormats, minifies and sorts JSON, keeps numbers exactly, names the line and column of an error, and repairs trailing commas and single quotes.3 views · 5 downloadsJwtInspectorDecodes a JSON Web Token locally, reads its dates relative to now, warns on alg none and a missing exp, and says the signature was not verified.3 views · 7 downloadsCronBuilderBuilds a five-field cron line from dropdowns or reads one typed, says it in English, lists the next runs in a time zone, and rejects a day that never comes.2 views · 6 downloadsRegexTesterTests a pattern live: matches marked, groups listed, a replace preview, the pattern explained, errors at their place, and a guard against patterns that hang.3 views · 10 downloadsSlugInputMakes a URL slug from a title as it is typed, writing accents plainly, stops following once edited by hand, and checks taken and reserved links, offering the next free one.3 views · 6 downloads
Thread
Thread 0 posts
On UrlInspector: questions, tips, corrections, and code that uses it.
No posts yet. The first one opens the thread.