Adjust setup.py. Map scripts to binary names. Adjust Readme docs.

[mavlyutovr]

Adjust setup.py:

• import requirements from files
• setup metadata
• add scripts as separate projects
• add script->binary name mapping
• add symlinks from project modules when installing in editable mode

Adjust Readme docs:

• use binary names instead of python .. path to script.py
• add a mention of an extra requirement for libsndfile (for fairseq2)

Adjustments in the script files:

• set logging
• move argument parsing under main()

E2E evaluation:

conda create  -y -n sc309 python=3.9
conda activate sc309
conda install -y -c conda-forge libsndfile
pip install .
mkdir -p ~/dataset
m4t_prepare_dataset  --source_lang eng --target_lang kor --split validation --save_dir ~/dataset
m4t_predict ~/dataset/downloads/extracted/633d087081013940f8c282cc7b5155e838ece8e0293d1651b7c684c47c563276/dev/10010138729160973689.wav s2st eng --output_path ~/out.wav
m4t_prepare_dataset  --source_lang eng --target_lang kor --split train --save_dir ~/dataset
m4t_finetune --train_dataset ~/dataset/train_manifest.json --eval_dataset ~/dataset/validation_manifest.json  --model_name seamlessM4T_medium --save_model_to ~/checkpoint.pt --log_steps 1 --batch_size 2

Same if installing in editable mode (also testing python 3.10):

conda create  -y -n sc310 python=3.10
conda activate sc310
conda install -y -c conda-forge libsndfile
pip install -e .
mkdir -p ~/dataset
python scripts/m4t/finetune/dataset.py --source_lang eng --target_lang kor --split validation --save_dir ~/dataset
python scripts/m4t/predict/predict.py ~/dataset/downloads/extracted/633d087081013940f8c282cc7b5155e838ece8e0293d1651b7c684c47c563276/dev/10010138729160973689.wav s2st eng --output_path ~/out.wav
python scripts/m4t/finetune/dataset.py --source_lang eng --target_lang kor --split train --save_dir ~/dataset
python scripts/m4t/finetune/finetune.py --train_dataset ~/dataset/train_manifest.json --eval_dataset ~/dataset/validation_manifest.json  --model_name seamlessM4T_medium --save_model_to ~/checkpoint.pt --log_steps 1 --batch_size 2

[cndn]

As it's early stage of fairseq2 shall we add version to avoid compatibility issues?

[mavlyutovr]

Sg, fixed

[cndn]

Shall we make it explicit that macOS support ✅ but pre-built package ❌, as Can mentioned in #19

right now we don't have pre-built packages available for macOS. For the time-being I suggest using a Linux box. We plan to have macOS wheels available pretty soon.

[mavlyutovr]

Added

[cndn]

Do we expect heavy code (that's not main entry) in finetune / predict or something importable? I was thinking we should move things that need to be imported to src/ all. But open to this change here.

[mavlyutovr]

This is to expose CLI scripts as executables.

[kauterry]

Could you rename python scripts/m4t/predict/predict.py to m4t_predict here too? https://github.com/facebookresearch/seamless_communication/blob/main/scripts/m4t/predict/README.md

[kauterry]

We create new directories seamless_communication and m4t_scripts, do we want that? cc @cndn

[cbalioglu]

Maybe I am missing something, but are these symlinks necessary? Normally pip install -e . adds the relative "src" directory to Python's sys.path, so Python should be able to resolve.

[mavlyutovr]

Editable mode of Python creates a symlink in site-packages to the module's parent folder. This logic does not work if you have more than one module to expose and they don't share their parent folder.

[kauterry]

What Ruslan means by "more than one module to expose" @cbalioglu