IP anonymizing
IP addresses made GDPR-friendly: the last part cut off (/24, /48) for logs, or a salted hash that counts visitors without keeping them.
Version 1.0.0 · 4 functions · 3 crates
Functions
What use ip_anonymize.qwer gives a program, with the types Quillo sees. To take only some, or under other names: use ip_truncate, ip_anonymize from ip_anonymize (Quillo 0.6).
ip_truncate(ip: str, v4_bits: int, v6_bits: int) → stror an errorip with all but its first v4_bits (IPv4) or v6_bits (IPv6) bits set to zero: 203.0.113.77 with 24 is 203.0.113.0, 2001:db8:85a3:8d3:1319:8a2e:370:7348 with 48 is 2001:db8:85a3::. Bits are kept within 0 to 32 and 0 to 128.
ip_anonymize(ip: str) → stror an errorip cut as most analytics tools cut it for the GDPR: the last byte of an IPv4 (/24), the last 80 bits of an IPv6 (/48).
ip_network(ip: str, v4_bits: int, v6_bits: int) → stror an errorThe network ip falls in, cut as ip_truncate cuts it, written with its prefix: 203.0.113.0/24. For grouping visitors.
ip_pseudonym(ip: str, salt: str) → stror an errorA pseudonym for ip: HMAC-SHA256 keyed with salt, the first 16 bytes in hex. The same IP and salt give the same pseudonym, so visits can be counted; change the salt each day (and forget the old one) and yesterday's pseudonyms can no longer be tied to anyone. With an empty salt it is an error: an unsalted hash of an IP is easy to reverse.
Get it
$ quillo install ip_anonymize # ip_anonymize.qwer into this folder$ quillo install ip_anonymize --example # and its example, ip_anonymize.qweOr with curl:
$ curl -O https://quillolang.org/plugins/files/ip_anonymize.qwer$ curl -O https://quillolang.org/plugins/files/ip_anonymize.qweThen, in your program, all of it or only what you call:
use ip_anonymize.qweruse ip_truncate, ip_anonymize from ip_anonymize // Quillo 0.6: only theseIt uses these crates, which cargo fetches the first time the plugin is built:
ipnet = "2"hmac = "0.12"sha2 = "0.10"Example
// Privacy-friendly access logs and a daily visitor count that keep no full IP address.//// quillo run ip_anonymize.qwe then open http://localhost:8080/samplesuse "ip_anonymize.qwer"let SECRET = env("IP_SALT_SECRET") ?? crypto.token(32)// A new salt each day: today's pseudonyms cannot be tied to tomorrow's.fn todays_salt() = crypto.sha256(`${SECRET}:${date.today()}`)route GET /visit { let line = { at: now(), ip: ip_anonymize(request.ip)?, path: request.path, visitor: ip_pseudonym(request.ip, todays_salt())? } write_file("data/access.log", json.stringify(line)) line}route GET /samples = { v4: ip_anonymize("203.0.113.77")?, v6: ip_anonymize("2001:db8:85a3:8d3:1319:8a2e:370:7348")?, mapped: ip_anonymize("::ffff:198.51.100.23")?, coarse: ip_network("203.0.113.77", 16, 32)?, same_visitor: ip_pseudonym("203.0.113.77", "a salt of at least sixteen") == ip_pseudonym("203.0.113.77", "a salt of at least sixteen"), no_salt: ip_pseudonym("203.0.113.77", "").error,}$ quillo run ip_anonymize.qwe$ curl 'localhost:8080/samples'$ curl 'localhost:8080/visit'Source
//! IP addresses made GDPR-friendly for logs and analytics: the last part cut off (/24 for IPv4, /48 for IPv6, or your own), or a salted hash that counts visitors without keeping who they were.//! version: 1.0.0//!//! [dependencies]//! ipnet = "2"//! hmac = "0.12"//! sha2 = "0.10"use hmac::{Hmac, Mac};use ipnet::{IpNet, Ipv4Net, Ipv6Net};use sha2::Sha256;use std::net::IpAddr;fn parse(ip: &str) -> Result<IpAddr, String> { let ip: IpAddr = ip.trim().trim_start_matches('[').trim_end_matches(']').parse().map_err(|_| format!("`{}` is not an IP address", ip))?; // An IPv4 address seen through IPv6 (::ffff:203.0.113.9) is cut as IPv4. Ok(match ip { IpAddr::V6(v6) => v6.to_ipv4_mapped().map(IpAddr::V4).unwrap_or(ip), v4 => v4, })}fn net(ip: &str, v4_bits: i64, v6_bits: i64) -> Result<IpNet, String> { Ok(match parse(ip)? { IpAddr::V4(a) => IpNet::V4(Ipv4Net::new(a, v4_bits.clamp(0, 32) as u8).map_err(|e| e.to_string())?.trunc()), IpAddr::V6(a) => IpNet::V6(Ipv6Net::new(a, v6_bits.clamp(0, 128) as u8).map_err(|e| e.to_string())?.trunc()), })}/// `ip` with all but its first `v4_bits` (IPv4) or `v6_bits` (IPv6) bits set to zero: `203.0.113.77` with 24 is `203.0.113.0`,/// `2001:db8:85a3:8d3:1319:8a2e:370:7348` with 48 is `2001:db8:85a3::`. Bits are kept within 0 to 32 and 0 to 128.pub fn ip_truncate(ip: &str, v4_bits: i64, v6_bits: i64) -> Result<String, String> { Ok(net(ip, v4_bits, v6_bits)?.network().to_string())}/// `ip` cut as most analytics tools cut it for the GDPR: the last byte of an IPv4 (/24), the last 80 bits of an IPv6 (/48).pub fn ip_anonymize(ip: &str) -> Result<String, String> { ip_truncate(ip, 24, 48)}/// The network `ip` falls in, cut as ip_truncate cuts it, written with its prefix: `203.0.113.0/24`. For grouping visitors.pub fn ip_network(ip: &str, v4_bits: i64, v6_bits: i64) -> Result<String, String> { Ok(net(ip, v4_bits, v6_bits)?.to_string())}/// A pseudonym for `ip`: HMAC-SHA256 keyed with `salt`, the first 16 bytes in hex. The same IP and salt give the same/// pseudonym, so visits can be counted; change the salt each day (and forget the old one) and yesterday's pseudonyms/// can no longer be tied to anyone. With an empty salt it is an error: an unsalted hash of an IP is easy to reverse.pub fn ip_pseudonym(ip: &str, salt: &str) -> Result<String, String> { if salt.len() < 16 { return Err("give a salt of at least 16 random characters: an IP hashed without a secret can be found again".into()); } let mut m = Hmac::<Sha256>::new_from_slice(salt.as_bytes()).map_err(|e| e.to_string())?; m.update(parse(ip)?.to_string().as_bytes()); Ok(m.finalize().into_bytes()[..16].iter().map(|b| format!("{:02x}", b)).collect())}Related
pii_redact.qwer · 1.0.0Find and mask personal data in text before logs or outside services see it: emails, phones, card numbers, IBANs and IP addresses.8 views · 8 downloadsSecrets scanner secrets_scan.qwer · 1.0.0API keys, tokens, private keys, JWTs and passwords in URLs found in text by gitleaks-style rules, with documentation placeholders told apart, and the text given back with them masked.7 views · 8 downloadsCrypto box crypto_box.qwer · 1.0.0Public-key encryption between users (libsodium's crypto_box): only the recipient reads it and knows the sender; and anonymous sealed boxes.7 views · 8 downloadsEncrypt encrypt.qwer · 1.0.0Encrypt and decrypt text and files with a secret key (AES-256-GCM): API tokens, notes and uploads kept unreadable at rest.7 views · 7 downloadsHash chain hash_chain.qwer · 1.0.0A tamper-evident audit log: each entry's SHA-256 covers the one before, so a changed, removed or reordered entry is found.7 views · 11 downloadsLog parser log_parse.qwer · 1.0.0Log lines into rows: nginx and Apache access logs, syslog, logfmt and JSON lines, the format found by itself, with counts by any field.7 views · 7 downloadsThread 0 posts
On IP anonymizing: questions, tips, corrections, and code that uses it.
No posts yet. The first one opens the thread.