CLI helper for CodeSearchNet to OpenNMT

[bzz]

Converts a split of CodeSearchNet to the text format that OpenNMT uses

Example:

wget 'https://s3.amazonaws.com/code-search-net/CodeSearchNet/v2/java.zip'
unzip java.zip

python notebooks/codesearchnet-opennmt.py --data_dir='java/final/jsonl/valid' --newline='\\n'

wc -l src-trian.txt tgt-trian.txt
  15328 src-trian.txt
  15328 tgt-trian.txt
  30656 total

[m09]

👍