Right-to-left text
Right-to-left text done right: a text's direction, mixed Arabic or Hebrew text put in display order for printers, terminals and images, user text isolated so it cannot flip a page, and Trojan Source direction controls found and removed.
Version 1.0.0 · 5 functions · 2 crates
Functions
What use bidi.qwer gives a program, with the types Quillo sees. To take only some, or under other names: use bidi_direction, bidi_visual from bidi (Quillo 0.6).
bidi_direction(text: str) → anyThe direction of text: "rtl" or "ltr" by its first strong letter (as dir="auto" decides), "neutral" when it has none (digits and punctuation only); { direction, has_rtl, has_ltr, mixed }.
bidi_visual(text: str, base: str) → stror an errortext in display order, line by line (the Unicode Bidirectional Algorithm), for output that shows characters left to right as stored: thermal printers, terminals, text drawn on images. base is "auto" (each paragraph by its first strong letter), "ltr" or "rtl".
bidi_isolate(text: str) → strtext wrapped in a first-strong isolate (U+2068 … U+2069) with any controls inside removed: a user's name or a product title in Arabic or Hebrew put into a sentence without turning the words around it. (In HTML, <bdi> does the same.)
bidi_strip(text: str) → strtext without the invisible direction controls (embeddings, overrides, isolates and marks): for usernames, file names and code, where they only disguise.
bidi_check(text: str) → anyThe direction controls in text, "Trojan Source" style: { safe, controls: [{ position, code, name }], unbalanced }; unbalanced when an embedding, override or isolate is never closed, which lets it flip what follows. safe when there are no overrides, embeddings or isolates (lone marks are allowed).
Get it
$ quillo install bidi # bidi.qwer into this folder$ quillo install bidi --example # and its example, bidi.qweOr with curl:
$ curl -O https://quillolang.org/plugins/files/bidi.qwer$ curl -O https://quillolang.org/plugins/files/bidi.qweThen, in your program, all of it or only what you call:
use bidi.qweruse bidi_direction, bidi_visual from bidi // Quillo 0.6: only theseIt uses these crates, which cargo fetches the first time the plugin is built:
unicode-bidi = "=0.3.18"serde_json = { version = "1", features = ["preserve_order"] }Example
// A marketplace with sellers writing in Arabic and Hebrew: the page direction of each listing, names isolated in English sentences, receipts for a printer with no bidi, and usernames checked for hidden direction tricks.//// quillo run bidi.qwe then open http://localhost:8080/bidiuse "bidi.qwer"route GET /bidi { { arabic: bidi_direction("مرحبا بالعالم"), english: bidi_direction("Hello, world").direction, mixed: bidi_direction("שלום world"), digits: bidi_direction("123-456").direction, // "Order 15 for עלמה": in memory the Hebrew letters come first to last; a printer must get them reversed. receipt: bidi_visual("Order 15 for עלמה", "ltr")?, hebrew_line: bidi_visual("abc שלום", "rtl")?, isolated: replace(replace(bidi_isolate("עלמה"), "\u{2068}", "[FSI]"), "\u{2069}", "[PDI]"), // A file name that shows as "invoicefdp.exe" reads "invoiceexe.pdf" on screen. trick: bidi_check("invoice\u{202E}fdp.exe"), cleaned: bidi_strip("invoice\u{202E}fdp.exe"), plain: bidi_check("Plain name").safe, }}route GET /listing ? { title: str = "שולחן עץ" } { let dir = bidi_direction(title).direction <html><body> <h1 dir={dir == "rtl" ? "rtl" : "ltr"}>{title}</h1> <p>Sold by <bdi>{title}</bdi> in 3 days.</p> </body></html>}$ quillo run bidi.qwe$ curl 'localhost:8080/bidi'Source
//! Right-to-left text done right: a text's direction (Arabic, Hebrew, Persian, Urdu...), mixed text put in display order for places with no bidi support (receipt printers, terminals, images, PDFs), user text isolated so it cannot flip a page's layout, and the invisible direction controls of "Trojan Source" attacks found and removed.//! version: 1.0.0//!//! [dependencies]//! unicode-bidi = "=0.3.18"//! serde_json = { version = "1", features = ["preserve_order"] }use serde_json::{json, Value};use unicode_bidi::{bidi_class, BidiClass, BidiInfo, Level};const CONTROLS: [(char, &str); 12] = [ ('\u{202A}', "LEFT-TO-RIGHT EMBEDDING"), ('\u{202B}', "RIGHT-TO-LEFT EMBEDDING"), ('\u{202C}', "POP DIRECTIONAL FORMATTING"), ('\u{202D}', "LEFT-TO-RIGHT OVERRIDE"), ('\u{202E}', "RIGHT-TO-LEFT OVERRIDE"), ('\u{2066}', "LEFT-TO-RIGHT ISOLATE"), ('\u{2067}', "RIGHT-TO-LEFT ISOLATE"), ('\u{2068}', "FIRST STRONG ISOLATE"), ('\u{2069}', "POP DIRECTIONAL ISOLATE"), ('\u{200E}', "LEFT-TO-RIGHT MARK"), ('\u{200F}', "RIGHT-TO-LEFT MARK"), ('\u{061C}', "ARABIC LETTER MARK"),];fn is_rtl_class(c: BidiClass) -> bool { matches!(c, BidiClass::R | BidiClass::AL) }/// The direction of `text`: `"rtl"` or `"ltr"` by its first strong letter (as `dir="auto"` decides), `"neutral"` when it has none (digits and punctuation only); `{ direction, has_rtl, has_ltr, mixed }`.pub fn bidi_direction(text: &str) -> Value { let mut first = None; let (mut r, mut l) = (false, false); for c in text.chars() { let cls = bidi_class(c); if is_rtl_class(cls) { r = true; if first.is_none() { first = Some("rtl") } } else if cls == BidiClass::L { l = true; if first.is_none() { first = Some("ltr") } } } json!({ "direction": first.unwrap_or("neutral"), "has_rtl": r, "has_ltr": l, "mixed": r && l })}/// `text` in display order, line by line (the Unicode Bidirectional Algorithm), for output that shows characters left to right as stored: thermal printers, terminals, text drawn on images./// `base` is `"auto"` (each paragraph by its first strong letter), `"ltr"` or `"rtl"`.pub fn bidi_visual(text: &str, base: &str) -> Result<String, String> { let level = match base.trim().to_lowercase().as_str() { "" | "auto" => None, "ltr" => Some(Level::ltr()), "rtl" => Some(Level::rtl()), b => return Err(format!("base is auto, ltr or rtl, not `{}`", b)) }; let mut out = Vec::new(); for line in text.split('\n') { let info = BidiInfo::new(line, level); let mut s = String::new(); for para in &info.paragraphs { s.push_str(&info.reorder_line(para, para.range.clone())) } out.push(s); } Ok(out.join("\n"))}/// `text` wrapped in a first-strong isolate (U+2068 … U+2069) with any controls inside removed: a user's name or a product title in Arabic or Hebrew put into a sentence without turning the words around it. (In HTML, `<bdi>` does the same.)pub fn bidi_isolate(text: &str) -> String { format!("\u{2068}{}\u{2069}", bidi_strip(text))}/// `text` without the invisible direction controls (embeddings, overrides, isolates and marks): for usernames, file names and code, where they only disguise.pub fn bidi_strip(text: &str) -> String { text.chars().filter(|c| !CONTROLS.iter().any(|(x, _)| x == c)).collect()}/// The direction controls in `text`, "Trojan Source" style: `{ safe, controls: [{ position, code, name }], unbalanced }`; `unbalanced` when an embedding, override or isolate is never closed, which lets it flip what follows. `safe` when there are no overrides, embeddings or isolates (lone marks are allowed).pub fn bidi_check(text: &str) -> Value { let mut found = Vec::new(); let (mut emb, mut iso) = (0i64, 0i64); let mut dangerous = false; for (i, c) in text.chars().enumerate() { if let Some((_, name)) = CONTROLS.iter().find(|(x, _)| *x == c) { found.push(json!({ "position": i, "code": format!("U+{:04X}", c as u32), "name": name })); match c { '\u{202A}'..='\u{202B}' | '\u{202D}'..='\u{202E}' => { emb += 1; dangerous = true } '\u{202C}' => emb -= 1, '\u{2066}'..='\u{2068}' => { iso += 1; dangerous = true } '\u{2069}' => iso -= 1, _ => {} } } if c == '\n' { if emb != 0 || iso != 0 { dangerous = true } } } json!({ "safe": !dangerous, "controls": found, "unbalanced": emb != 0 || iso != 0 })}Related
kana.qwer · 1.0.0Japanese kana: romaji typed as hiragana or katakana, kana written in Hepburn romaji, hiragana and katakana each to the other, half-width katakana from bank and till systems made full-width, and which scripts a text uses.6 views · 10 downloadsBraille braille.qwer · 1.0.0Unicode braille from English text: grade 1 letter by letter, and Unified English Braille grade 2 with its wordsigns, groupsigns, positional rules and shortforms; read back cell by cell where the cells allow it; dot numbers and the Braille ASCII of `.brf` files both ways.3 views · 6 downloadsChinese words chinese_segment.qwer · 1.0.0Chinese text split into words for search indexes and counts, with parts of speech, and written in pinyin with tones, tone numbers or initials for sorting and finding names.7 views · 7 downloadsEmoji emoji.qwer · 1.0.0Emoji for chat and comments: :shortcodes: turned into emoji and back, emoji found with their names, stripped from subjects, and searched by name for a picker.7 views · 8 downloadsHyphenation hyphenation.qwer · 1.0.0Words hyphenated by the rules of 80 languages and spellings, patterns built in: a word's syllables, soft hyphens put in text and HTML for narrow columns and e-mails, and text wrapped at a width with words broken where they may be.6 views · 7 downloadsNumbers in words number_words.qwer · 1.0.0Numbers and amounts of money in words for cheques, invoices and contracts, in English, French and Ukrainian, with ordinals, years and Roman numerals both ways.8 views · 7 downloadsThread 0 posts
On Right-to-left text: questions, tips, corrections, and code that uses it.
No posts yet. The first one opens the thread.