This project has only been tested on Python 3. We highly recommend creating a virtual environment and installing the python requirements there.
# Setting up virtual environment
python -m venv venv_athena
source venv_athena/bin/activate
For more information, you can checkout the tensorflow website.
# we highly recommend firstly update pip
pip install --upgrade pip
pip install tensorflow==2.0.0
For multiple GPU/CPU training You have to install the horovod, you can find out more information from the horovod website.
For WFST decoding You have to install pydecoder, installation guide for pydecoder can be found athena-decoder website
git clone https://github.com/athena-team/athena.git
cd athena
pip install -r requirements.txt
python setup.py bdist_wheel sdist
python -m pip install --ignore-installed dist/athena-0.1.0*.whl
- Once athena is successfully installed , you should do
source tools/env.sh
firstly before doing other things. - For installing some other supporting tools, you can check the
tools/install*.sh
to install kenlm, sph2pipe, spm and ... [Optional]
- On a single cpu/gpu
source tools/env.sh
python examples/translate/spa-eng-example/prepare_data.py examples/translate/spa-eng-example/data/train.csv
python athena/main.py examples/translate/spa-eng-example/transformer.json
- On multiple cpu/gpu in one machine (you should make sure your hovorod is successfully installed)
source tools/env.sh
python examples/translate/spa-eng-example/prepare_data.py examples/translate/spa-eng-example/data/train.csv
horovodrun -np 4 -H localhost:4 athena/horovod_main.py examples/translate/spa-eng-example/transformer.json
- If you see errors such as
ERROR: Cannot uninstall 'wrapt'
while installing TensorFlow, try updating it using commandconda update wrapt
. Same for similar dependencies such asentrypoints
,llvmlite
and so on. - You may want to make sure you have
g++
version 7 or above to make sure you can successfully install TensorFlow.