sanitize - convert Unicode text into safe ASCII strings - Emacs Lisp package
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
data
tools
COPYING
README.org
unidecode-test.el
unidecode.el

README.org

unidecode.el

Functions for transliterating Unicode text to ASCII:

  • (unidecode STRING)
  • (unidecode-region BEG END)
  • (unidecode-sanitize STRING)
  • (unidecode-sanitize-region BEG END)

This package was derived from the Unidecode module for Python, which was in turn derived from the Text::Unidecode package for Perl.

Example use

(require 'unidecode)

(unidecode "Déjà vu")
;=> "Deja vu"
(unidecode "北亰")
;=> "Bei Jing "
(unidecode-sanitize "Déjà vu, all over again")
;=> "deja-vu-all-over-again"

How this package was made

This package uses a number of data files (see the data directory), which were converted from the corresponding files in the Python Unidecode module.

The scripts used to perform this conversion are tools/unidecode-convert.el and tools/unidecode_convert.py.

To perform the conversion yourself, load unidecode-convert.el and run the unidecode-convert command (e.g. M-x unidecode-convert RET). When prompted, enter the path to Unidecode’s source directory as the first argument, and where you want the files to be saved as the second argument. It uses the Python script to convert the Unidecode files to JSON, then uses Emacs’s json package to load them, and finally prints them as read-able Lisp data as stored in the data directory.

License

This project is free software, available under the terms of the GPL, version 2 or later. See COPYING for details.