Text

Phonetic matching

Names matched by how they sound for search and duplicate checks: Soundex, Metaphone, Double Metaphone, NYSIIS, Caverphone and Cologne codes, accents ignored.

Version 1.0.0 · 4 functions · 2 crates

#search#names#text#dedupe

Functions

What use phonetics.qwer gives a program, with the types Quillo sees. To take only some, or under other names: use phonetic_code, phonetic_double_metaphone from phonetics (Quillo 0.6).

phonetic_code(text: str, algorithm: str) → stror an error

The code of text by algorithm, accents ignored: "soundex" ("Robert" and "Rupert" are both R163), "refined_soundex", "metaphone" (TH, PH and silent letters known), "double_metaphone" (its first code), "nysiis", "caverphone" (made for New Zealand names) or "cologne" (for German names: "Meier" and "Mayr" match).

phonetic_double_metaphone(text: str) → anyor an error

Both Double Metaphone codes of a name, { primary, alternate }: the alternate is another way it is said ("Schmidt" as German and as English). Two names match when any code of one equals any code of the other.

phonetic_match(a: str, b: str, algorithm: str) → boolor an error

Whether a and b sound alike by algorithm; for "double_metaphone", by any of their codes.

phonetic_find(name: str, names: list) → listor an error

The names that sound like name by Double Metaphone, in their order: for "did you mean Katherine?" and finding duplicates in a list of up to 100,000.

Get it

$ quillo install phonetics               # phonetics.qwer into this folder$ quillo install phonetics --example     # and its example, phonetics.qwe

Or with curl:

$ curl -O https://quillolang.org/plugins/files/phonetics.qwer$ curl -O https://quillolang.org/plugins/files/phonetics.qwe

Then, in your program, all of it or only what you call:

main.qwe
use phonetics.qweruse phonetic_code, phonetic_double_metaphone from phonetics      // Quillo 0.6: only these

It uses these crates, which cargo fetches the first time the plugin is built:

[dependencies]
rphonetic = "4"unicode-normalization = "0.1"

Example

phonetics.qwe
// A customer search that finds "Catherine" when someone types "Kathryn", and a duplicate check.////   quillo run phonetics.qwe      then open http://localhost:8080/customers?name=Kathrynuse "phonetics.qwer"let customers = ["Catherine Smith", "Katherine", "Kathryn", "Karen", "Smyth", "Schmidt", "Mayr", "Meier", "Robert", "Rupert"]route GET /customers ? { name: str } = { name, sounds_like: phonetic_find(name, customers)? }route GET /codes ? { name: str = "Robert" } = {    soundex: phonetic_code(name, "soundex")?,    metaphone: phonetic_code(name, "metaphone")?,    double: phonetic_double_metaphone(name)?,    nysiis: phonetic_code(name, "nysiis")?,    cologne: phonetic_code(name, "cologne")?,}route GET /same = {    robert_rupert: phonetic_match("Robert", "Rupert", "soundex")?,    smith_smyth: phonetic_match("Smith", "Smyth", "double_metaphone")?,    meier_mayr: phonetic_match("Meier", "Mayr", "cologne")?,    smith_jones: phonetic_match("Smith", "Jones", "double_metaphone")?,}
$ quillo run phonetics.qwe$ curl 'localhost:8080/customers?name=Kathryn'$ curl 'localhost:8080/same'

Source

phonetics.qwer
//! Names matched by how they sound, for search, duplicate checks and genealogy: Soundex, Metaphone, Double Metaphone, NYSIIS, Caverphone and Cologne (German) codes.//! version: 1.0.0//!//! [dependencies]//! rphonetic = "4"//! unicode-normalization = "0.1"use rphonetic::{Caverphone2, Cologne, DoubleMetaphone, Encoder, Metaphone, Nysiis, RefinedSoundex, Soundex};use serde_json::{json, Value};fn guard<T>(f: impl FnOnce() -> T + std::panic::UnwindSafe) -> Result<T, String> {    std::panic::catch_unwind(f).map_err(|_| "could not encode this text".to_string())}/// The name in ASCII letters: accents taken off ("Élodie" is "Elodie"), other characters dropped, as the English codes expect.fn ascii(text: &str) -> String {    use unicode_normalization::UnicodeNormalization;    text.replace('ß', "ss").nfd().filter(|c| c.is_ascii_alphabetic() || *c == ' ' || *c == '-' || *c == '\'').collect()}fn encode(text: &str, algorithm: &str) -> Result<String, String> {    if text.len() > 1000 {        return Err("names up to 1,000 characters".into());    }    let a = algorithm.to_lowercase();    // Cologne knows ä, ö, ü and ß itself.    let t = if a == "cologne" { text.to_string() } else { ascii(text) };    if !["soundex", "refined_soundex", "metaphone", "double_metaphone", "nysiis", "caverphone", "cologne"].contains(&a.as_str()) {        return Err(format!("no algorithm {:?}: soundex, refined_soundex, metaphone, double_metaphone, nysiis, caverphone or cologne", algorithm));    }    guard(move || match a.as_str() {        "soundex" => Soundex::default().encode(&t),        "refined_soundex" => RefinedSoundex::default().encode(&t),        "metaphone" => Metaphone::default().encode(&t),        "double_metaphone" => DoubleMetaphone::default().encode(&t),        "nysiis" => Nysiis::default().encode(&t),        "caverphone" => Caverphone2.encode(&t),        _ => Cologne.encode(&t),    })}/// The code of `text` by `algorithm`, accents ignored: "soundex" ("Robert" and "Rupert" are both R163), "refined_soundex", "metaphone" (TH, PH and silent letters known), "double_metaphone" (its first code), "nysiis", "caverphone" (made for New Zealand names) or "cologne" (for German names: "Meier" and "Mayr" match).pub fn phonetic_code(text: &str, algorithm: &str) -> Result<String, String> {    encode(text, algorithm)}/// Both Double Metaphone codes of a name, `{ primary, alternate }`: the alternate is another way it is said ("Schmidt" as German and as English). Two names match when any code of one equals any code of the other.pub fn phonetic_double_metaphone(text: &str) -> Result<Value, String> {    if text.len() > 1000 {        return Err("names up to 1,000 characters".into());    }    let t = ascii(text);    let r = guard(move || DoubleMetaphone::default().double_metaphone(&t))?;    Ok(json!({ "primary": r.primary(), "alternate": r.alternate() }))}/// Whether `a` and `b` sound alike by `algorithm`; for "double_metaphone", by any of their codes.pub fn phonetic_match(a: &str, b: &str, algorithm: &str) -> Result<bool, String> {    if algorithm.eq_ignore_ascii_case("double_metaphone") {        let (x, y) = (phonetic_double_metaphone(a)?, phonetic_double_metaphone(b)?);        let codes = |v: &Value| [v["primary"].as_str().unwrap_or("").to_string(), v["alternate"].as_str().unwrap_or("").to_string()];        let (cx, cy) = (codes(&x), codes(&y));        return Ok(cx.iter().any(|c| !c.is_empty() && cy.contains(c)));    }    let (x, y) = (encode(a, algorithm)?, encode(b, algorithm)?);    Ok(!x.is_empty() && x == y)}/// The `names` that sound like `name` by Double Metaphone, in their order: for "did you mean Katherine?" and finding duplicates in a list of up to 100,000.pub fn phonetic_find(name: &str, names: Vec<String>) -> Result<Vec<String>, String> {    if names.len() > 100_000 {        return Err("up to 100,000 names".into());    }    let mut out = Vec::new();    for n in names {        if phonetic_match(name, &n, "double_metaphone")? {            out.push(n);        }    }    Ok(out)}
Thread
89Useful? Like it, or add to its thread below.

Thread 0 posts

On Phonetic matching: questions, tips, corrections, and code that uses it.

No posts yet. The first one opens the thread.