Getting Started

Getting Started

Runtime: Python 3.10–3.13. CI on GitHub Actions covers that range. Packaging is pyproject.toml (setuptools). The libmagic shared library is required (python-magic is only the ctypes binding).

libmagic

man libmagic    # should exist after install

macOS (Homebrew):

brew install libmagic

Debian / Ubuntu:

sudo apt-get install libmagic1

Install ETLLib

git clone https://github.com/chrismattmann/etllib.git
cd etllib
python3 -m pip install -e .

That puts tsvtojson, repackage, poster, repackageandpost, translatejson, and similarity on your PATH. Check:

tsvtojson -h

translatejson needs hirlite, which often fails to build on current Python. It is an extra, not a required dependency:

python3 -m pip install -e ".[translate]"

For many-to-English at scale, use BigTranslate / Pantogloss instead. Tika (tika>=1.13) is a normal dependency.

Tests:

python3 -m pip install -e . --group=tests
python3 -m pytest

Next

• Tutorial — CSV/TSV → aggregate JSON → one file per row
• Commands

Python 2.7, bootstrap.py, and buildout notes are under Old.