Unidecode → disarm recipes¶

A survey of 58 real projects (beets, saleor, investpy, python-slugify, django-autoslug, …) found that unidecode is almost never the last step. It sits inside a hand-rolled normalisation pipeline — lowercase, strip, collapse, re-encode — and every one of those pipelines already has a single-call equivalent in disarm. The gap is discoverability, not capability.

This page maps each observed pattern to its one-liner. The "after" snippets are executed and asserted in CI, so the equivalences below cannot silently rot. The "before" snippets show the legacy hand-rolled code and are not executed.

At a glance¶

Hand-rolled with unidecode	disarm one-liner
re.sub(r"\W+", "-", unidecode(t).lower().strip())	slugify(t)
unidecode(name).replace(os.sep, "_")	sanitize_filename(name, platform=...)
unidecode(t).encode("ascii")	transliterate(t).encode("ascii")
unidecode(t.lower().strip()) for dict / search keys	search_key(t) / catalog_key(t) (both fold accents like unidecode; sort_key keeps them)
quote(unidecode(t))	quote(transliterate(t)) (or slugify for URLs)

URL slugs¶

# Before — hand-rolled slug
import re
from unidecode import unidecode

def slug(t):
    return re.sub(r"\W+", "-", unidecode(t).lower().strip()).strip("-")

# After
from disarm import slugify

assert slugify("Café del Mar!!") == "cafe-del-mar"
assert slugify("Хлеб с маслом") == "khleb-s-maslom"

slugify() lowercases, transliterates, replaces runs of non-word characters with a single -, and trims — in the right order — so you do not re-implement (and mis-order) the pipeline. See the Slugification guide.

Safe filenames¶

# Before — transliterate then patch path separators by hand
import os
from unidecode import unidecode

safe = unidecode("Naïve/Résumé.txt").replace(os.sep, "_")

# After
from disarm import sanitize_filename

assert sanitize_filename("Naïve/Résumé.txt") == "Naive_Resume.txt"

sanitize_filename() does more than swap separators: it removes OS-illegal characters, handles Windows reserved names, and truncates to a byte budget. Pick the rule set with platform="universal" | "posix" | "windows". See Filename Sanitization.

ASCII bytes¶

# Before
from unidecode import unidecode

raw = unidecode("Café").encode("ascii")

# After
from disarm import transliterate

assert transliterate("Café").encode("ascii") == b"Cafe"

transliterate() already guarantees ASCII-only output, so the .encode("ascii") never raises UnicodeEncodeError. Use errors="ignore" | "preserve" | "replace" to choose what happens to characters with no mapping.

Dictionary / search keys¶

# Before — a casefolded, accent-folded lookup key
from unidecode import unidecode

key = unidecode("Café del Mar".lower().strip())

# After — pick the helper that documents intent
from disarm import search_key, catalog_key, sort_key

# search_key / catalog_key fold accents to ASCII (the direct unidecode swap):
assert search_key("  Café  RÉSUMÉ  ") == "cafe resume"
assert catalog_key("Café del Mar") == "cafe del mar"
# sort_key keeps accented Latin so the accent can order the collation key:
assert sort_key("Café del Mar") == "café del mar"

search_key and catalog_key fold accents to ASCII — the direct replacements for unidecode-based keys. sort_key deliberately does not. They differ by the extra pass each adds:

• search_key — lightest; folds accents, nothing else. For search / lookup indexes.
• catalog_key — adds a TR39 confusable fold, which canonicalises look-alikes that survive transliteration (curly quotes, primes, and similar punctuation). Built for bibliographic de-duplication where typographic variants of the same title should collapse to one key.
• sort_key — preserves base accented characters ("Café" → "café") so the accent can order the key, folding only non-Latin scripts to Latin and stripping bidi controls for stable ordering. Its accent-folding counterpart is search_key.

The confusable fold is what sets catalog_key apart — typographic variants of a title collapse to the same key:

assert catalog_key('naïve “quote”') == "naive ''quote''"   # quotes folded
assert search_key('naïve “quote”') == 'naive "quote"'      # quotes preserved

That fold runs after transliteration, so it does not reverse cross-script homoglyph spoofs (Cyrillic р is already romanised to r by then). For that, see confusable defense.

URL-encoded query parameters¶

# Before
from urllib.parse import quote
from unidecode import unidecode

q = quote(unidecode("Café del Mar"))

# After
from urllib.parse import quote
from disarm import transliterate, slugify

assert quote(transliterate("Café del Mar")) == "Cafe%20del%20Mar"
# For a clean URL path segment, prefer a slug:
assert slugify("Café del Mar") == "cafe-del-mar"

The ordering footgun¶

#88 found pipelines that lowercase before transliterating. That order is subtly wrong whenever transliteration introduces an uppercase letter. The numero sign № expands to No — a capital N that a prior .lower() cannot see:

from disarm import transliterate, search_key

# Lowercasing BEFORE transliteration misses the introduced capital:
assert transliterate("№5".lower()) == "No5"     # wrong — stray capital N
# Lowercasing AFTER transliteration is correct:
assert transliterate("№5").lower() == "no5"
# The disarm helpers fold case after transliterating, so they get it right:
assert search_key("№5") == "no5"

This is exactly the class of bug the one-call helpers remove: the correct order is baked in. When in doubt, reach for slugify / search_key / sanitize_filename rather than re-assembling the steps.

See also¶

• Migrating from Unidecode — the drop-in alias and the security caveat (unidecode is not a confusable-defense tool).
• Slugification, Filename Sanitization, Normalization.