Getting Started

Getting Started

Runtime: Python 3.9–3.13. CI on GitHub Actions covers that range. The libmagic shared library is required (python-magic is only the ctypes binding).

libmagic

man libmagic    # should exist after install

macOS (Homebrew):

brew install libmagic

Debian / Ubuntu:

sudo apt-get install libmagic1

Install ETLLib

git clone https://github.com/chrismattmann/etllib.git
cd etllib
python3 -m pip install -e .

That puts tsvtojson, repackage, poster, repackageandpost, translatejson, and similarity on your PATH. Check:

tsvtojson -h

hirlite is declared as a dependency for translatejson's translation cache. It often fails to build on current Python. If pip install -e . dies on hirlite, install the rest and skip it:

python3 -m pip install setuptools iso8601 python-magic 'tika>=1.13'
python3 -m pip install --no-deps -e .

translatejson needs hirlite. For many-to-English at scale, use BigTranslate / Pantogloss instead.

Tika is a normal dependency (tika>=1.13). There is no separate buildout install with-tika path anymore.

Next

• Tutorial — CSV/TSV → aggregate JSON → one file per row
• Commands

Python 2.7, bootstrap.py, and buildout notes are under Old.