Phonetic matching
Names matched by how they sound for search and duplicate checks: Soundex, Metaphone, Double Metaphone, NYSIIS, Caverphone and Cologne codes, accents ignored.
Version 1.0.0 · 4 functions · 2 crates
Functions
What use phonetics.qwer gives a program, with the types Quillo sees. To take only some, or under other names: use phonetic_code, phonetic_double_metaphone from phonetics (Quillo 0.6).
phonetic_code(text: str, algorithm: str) → stror an errorThe code of text by algorithm, accents ignored: "soundex" ("Robert" and "Rupert" are both R163), "refined_soundex", "metaphone" (TH, PH and silent letters known), "double_metaphone" (its first code), "nysiis", "caverphone" (made for New Zealand names) or "cologne" (for German names: "Meier" and "Mayr" match).
phonetic_double_metaphone(text: str) → anyor an errorBoth Double Metaphone codes of a name, { primary, alternate }: the alternate is another way it is said ("Schmidt" as German and as English). Two names match when any code of one equals any code of the other.
phonetic_match(a: str, b: str, algorithm: str) → boolor an errorWhether a and b sound alike by algorithm; for "double_metaphone", by any of their codes.
phonetic_find(name: str, names: list) → listor an errorThe names that sound like name by Double Metaphone, in their order: for "did you mean Katherine?" and finding duplicates in a list of up to 100,000.
Get it
$ quillo install phonetics # phonetics.qwer into this folder$ quillo install phonetics --example # and its example, phonetics.qweOr with curl:
$ curl -O https://quillolang.org/plugins/files/phonetics.qwer$ curl -O https://quillolang.org/plugins/files/phonetics.qweThen, in your program, all of it or only what you call:
use phonetics.qweruse phonetic_code, phonetic_double_metaphone from phonetics // Quillo 0.6: only theseIt uses these crates, which cargo fetches the first time the plugin is built:
rphonetic = "4"unicode-normalization = "0.1"Example
// A customer search that finds "Catherine" when someone types "Kathryn", and a duplicate check.//// quillo run phonetics.qwe then open http://localhost:8080/customers?name=Kathrynuse "phonetics.qwer"let customers = ["Catherine Smith", "Katherine", "Kathryn", "Karen", "Smyth", "Schmidt", "Mayr", "Meier", "Robert", "Rupert"]route GET /customers ? { name: str } = { name, sounds_like: phonetic_find(name, customers)? }route GET /codes ? { name: str = "Robert" } = { soundex: phonetic_code(name, "soundex")?, metaphone: phonetic_code(name, "metaphone")?, double: phonetic_double_metaphone(name)?, nysiis: phonetic_code(name, "nysiis")?, cologne: phonetic_code(name, "cologne")?,}route GET /same = { robert_rupert: phonetic_match("Robert", "Rupert", "soundex")?, smith_smyth: phonetic_match("Smith", "Smyth", "double_metaphone")?, meier_mayr: phonetic_match("Meier", "Mayr", "cologne")?, smith_jones: phonetic_match("Smith", "Jones", "double_metaphone")?,}$ quillo run phonetics.qwe$ curl 'localhost:8080/customers?name=Kathryn'$ curl 'localhost:8080/same'Source
//! Names matched by how they sound, for search, duplicate checks and genealogy: Soundex, Metaphone, Double Metaphone, NYSIIS, Caverphone and Cologne (German) codes.//! version: 1.0.0//!//! [dependencies]//! rphonetic = "4"//! unicode-normalization = "0.1"use rphonetic::{Caverphone2, Cologne, DoubleMetaphone, Encoder, Metaphone, Nysiis, RefinedSoundex, Soundex};use serde_json::{json, Value};fn guard<T>(f: impl FnOnce() -> T + std::panic::UnwindSafe) -> Result<T, String> { std::panic::catch_unwind(f).map_err(|_| "could not encode this text".to_string())}/// The name in ASCII letters: accents taken off ("Élodie" is "Elodie"), other characters dropped, as the English codes expect.fn ascii(text: &str) -> String { use unicode_normalization::UnicodeNormalization; text.replace('ß', "ss").nfd().filter(|c| c.is_ascii_alphabetic() || *c == ' ' || *c == '-' || *c == '\'').collect()}fn encode(text: &str, algorithm: &str) -> Result<String, String> { if text.len() > 1000 { return Err("names up to 1,000 characters".into()); } let a = algorithm.to_lowercase(); // Cologne knows ä, ö, ü and ß itself. let t = if a == "cologne" { text.to_string() } else { ascii(text) }; if !["soundex", "refined_soundex", "metaphone", "double_metaphone", "nysiis", "caverphone", "cologne"].contains(&a.as_str()) { return Err(format!("no algorithm {:?}: soundex, refined_soundex, metaphone, double_metaphone, nysiis, caverphone or cologne", algorithm)); } guard(move || match a.as_str() { "soundex" => Soundex::default().encode(&t), "refined_soundex" => RefinedSoundex::default().encode(&t), "metaphone" => Metaphone::default().encode(&t), "double_metaphone" => DoubleMetaphone::default().encode(&t), "nysiis" => Nysiis::default().encode(&t), "caverphone" => Caverphone2.encode(&t), _ => Cologne.encode(&t), })}/// The code of `text` by `algorithm`, accents ignored: "soundex" ("Robert" and "Rupert" are both R163), "refined_soundex", "metaphone" (TH, PH and silent letters known), "double_metaphone" (its first code), "nysiis", "caverphone" (made for New Zealand names) or "cologne" (for German names: "Meier" and "Mayr" match).pub fn phonetic_code(text: &str, algorithm: &str) -> Result<String, String> { encode(text, algorithm)}/// Both Double Metaphone codes of a name, `{ primary, alternate }`: the alternate is another way it is said ("Schmidt" as German and as English). Two names match when any code of one equals any code of the other.pub fn phonetic_double_metaphone(text: &str) -> Result<Value, String> { if text.len() > 1000 { return Err("names up to 1,000 characters".into()); } let t = ascii(text); let r = guard(move || DoubleMetaphone::default().double_metaphone(&t))?; Ok(json!({ "primary": r.primary(), "alternate": r.alternate() }))}/// Whether `a` and `b` sound alike by `algorithm`; for "double_metaphone", by any of their codes.pub fn phonetic_match(a: &str, b: &str, algorithm: &str) -> Result<bool, String> { if algorithm.eq_ignore_ascii_case("double_metaphone") { let (x, y) = (phonetic_double_metaphone(a)?, phonetic_double_metaphone(b)?); let codes = |v: &Value| [v["primary"].as_str().unwrap_or("").to_string(), v["alternate"].as_str().unwrap_or("").to_string()]; let (cx, cy) = (codes(&x), codes(&y)); return Ok(cx.iter().any(|c| !c.is_empty() && cy.contains(c))); } let (x, y) = (encode(a, algorithm)?, encode(b, algorithm)?); Ok(!x.is_empty() && x == y)}/// The `names` that sound like `name` by Double Metaphone, in their order: for "did you mean Katherine?" and finding duplicates in a list of up to 100,000.pub fn phonetic_find(name: &str, names: Vec<String>) -> Result<Vec<String>, String> { if names.len() > 100_000 { return Err("up to 100,000 names".into()); } let mut out = Vec::new(); for n in names { if phonetic_match(name, &n, "double_metaphone")? { out.push(n); } } Ok(out)}Related
chinese_segment.qwer · 1.0.0Chinese text split into words for search indexes and counts, with parts of speech, and written in pinyin with tones, tone numbers or initials for sorting and finding names.7 views · 7 downloadsFuzzy search fuzzy.qwer · 1.0.0Rank names against what someone typed, typos and all, edit distances, and did-you-mean.7 views · 8 downloadsKeywords keywords.qwer · 1.0.0The keywords and key phrases of a text by RAKE, TextRank or TF-IDF, with stop words of 58 languages left out: tags, SEO and related posts.7 views · 6 downloadsN-grams ngrams.qwer · 1.0.0N-grams and shingles: word and character n-grams, the phrases a text uses most, how alike two texts are by shared shingles despite typos, and near-duplicates found among thousands of texts with MinHash.6 views · 9 downloadsSpellcheck spellcheck.qwer · 1.0.0Spelling checked against a word list you load, in any language: unknown words found with their places, suggestions ranked by closeness and frequency (SymSpell's method), text corrected for "did you mean", and your own words added.7 views · 8 downloadsStemming stemming.qwer · 1.0.0Words cut to their stems in 18 languages, so running, runs and run count as one word in a search or a tag cloud.7 views · 8 downloadsThread 0 posts
On Phonetic matching: questions, tips, corrections, and code that uses it.
No posts yet. The first one opens the thread.