What is MD example save path?

[ButteredGroove]

The README.md states:

Training

❱❱❱ python3 myTrain.py -dec=TRADE -bsz=32 -dr=0.2 -lr=0.001 -le=1

Testing

❱❱❱ python3 myTest.py -path=${save_path}

Based on the training command, what should save_path be set to?

I ran the myTrain.py command successfully. But the testing command fails if I don't supply a path:

$ python myTest.py
{'dataset': 'multiwoz', 'task': 'dst', 'path': None, 'sample': None, 'patience': 6, 'earlyStop': 'BLEU', 'all_vocab': 1, 'imbalance_sampler': 0, 'data_ratio': 100, 'unk_mask': 1, 'batch': None, 'run_dev_testing': 0, 'vizualization': 0, 'genSample': 0, 'evalp': 1, 'addName': '', 'eval_batch': 0, 'use_gate': 1, 'load_embedding': 0, 'fix_embedding': 0, 'parallel_decode': 0, 'decoder': None, 'hidden': 400, 'learn': None, 'drop': None, 'limit': -10000, 'clip': 10, 'teacher_forcing_ratio': 0.5, 'lambda_ewc': 0.01, 'fisher_sample': 0, 'all_model': False, 'domain_as_task': False, 'run_except_4d': 1, 'strict_domain': False, 'except_domain': '', 'only_domain': ''}
Traceback (most recent call last):
  File "myTest.py", line 8, in <module>
    directory = args['path'].split("/")
AttributeError: 'NoneType' object has no attribute 'split'

[ButteredGroove]

Some attempts:

$ python myTest.py -path="save"
{'dataset': 'multiwoz', 'task': 'dst', 'path': 'save', 'sample': None, 'patience': 6, 'earlyStop': 'BLEU', 'all_vocab': 1, 'imbalance_sampler': 0, 'data_ratio': 100, 'unk_mask': 1, 'batch': None, 'run_dev_testing': 0, 'vizualization': 0, 'genSample': 0, 'evalp': 1, 'addName': '', 'eval_batch': 0, 'use_gate': 1, 'load_embedding': 0, 'fix_embedding': 0, 'parallel_decode': 0, 'decoder': None, 'hidden': 400, 'learn': None, 'drop': None, 'limit': -10000, 'clip': 10, 'teacher_forcing_ratio': 0.5, 'lambda_ewc': 0.01, 'fisher_sample': 0, 'all_model': False, 'domain_as_task': False, 'run_except_4d': 1, 'strict_domain': False, 'except_domain': '', 'only_domain': ''}
Traceback (most recent call last):
  File "myTest.py", line 9, in <module>
    HDD = directory[2].split('HDD')[1].split('BSZ')[0]
IndexError: list index out of range

$ python myTest.py -path="save/TRADE-multiwozdst"
{'dataset': 'multiwoz', 'task': 'dst', 'path': 'save/TRADE-multiwozdst', 'sample': None, 'patience': 6, 'earlyStop': 'BLEU', 'all_vocab': 1, 'imbalance_sampler': 0, 'data_ratio': 100, 'unk_mask': 1, 'batch': None, 'run_dev_testing': 0, 'vizualization': 0, 'genSample': 0, 'evalp': 1, 'addName': '', 'eval_batch': 0, 'use_gate': 1, 'load_embedding': 0, 'fix_embedding': 0, 'parallel_decode': 0, 'decoder': None, 'hidden': 400, 'learn': None, 'drop': None, 'limit': -10000, 'clip': 10, 'teacher_forcing_ratio': 0.5, 'lambda_ewc': 0.01, 'fisher_sample': 0, 'all_model': False, 'domain_as_task': False, 'run_except_4d': 1, 'strict_domain': False, 'except_domain': '', 'only_domain': ''}
Traceback (most recent call last):
  File "myTest.py", line 9, in <module>
    HDD = directory[2].split('HDD')[1].split('BSZ')[0]
IndexError: list index out of range

$ python myTest.py -path="save/TRADE-multiwozdst/HDD400BSZ32DR0.2ACC-0.4950"
{'dataset': 'multiwoz', 'task': 'dst', 'path': 'save/TRADE-multiwozdst/HDD400BSZ32DR0.2ACC-0.4950', 'sample': None, 'patience': 6, 'earlyStop': 'BLEU', 'all_vocab': 1, 'imbalance_sampler': 0, 'data_ratio': 100, 'unk_mask': 1, 'batch': None, 'run_dev_testing': 0, 'vizualization': 0, 'genSample': 0, 'evalp': 1, 'addName': '', 'eval_batch': 0, 'use_gate': 1, 'load_embedding': 0, 'fix_embedding': 0, 'parallel_decode': 0, 'decoder': None, 'hidden': 400, 'learn': None, 'drop': None, 'limit': -10000, 'clip': 10, 'teacher_forcing_ratio': 0.5, 'lambda_ewc': 0.01, 'fisher_sample': 0, 'all_model': False, 'domain_as_task': False, 'run_except_4d': 1, 'strict_domain': False, 'except_domain': '', 'only_domain': ''}
HDD 400 decoder TRADE BSZ 32
folder_name save/
Traceback (most recent call last):
  File "myTest.py", line 21, in <module>
    train, dev, test, test_special, lang, SLOTS_LIST, gating_dict, max_word = prepare_data_seq(False, args['task'], False, batch_size=BSZ)
  File "/home/user/trade-dst/utils/utils_multiWOZ_DST.py", line 428, in prepare_data_seq
    with open(folder_name+lang_name, 'rb') as handle:
FileNotFoundError: [Errno 2] No such file or directory: 'save/lang-all.pkl'

There is a lang-all.pkl file, but it's in save/TRADE-multiwozdst/

[ButteredGroove]

I was able to get myTrain to run by creating symbolic links in the save directory to the lang files in save/TRADE-multiwozdst and then running:
$ python myTest.py -path="save/TRADE-multiwozdst/HDD400BSZ32DR0.2ACC-0.4950"

[jasonwu0731]

Yes, the original testing script depends on the way (the exact path) we saved the file. Please modify the test script if you would like to change the saving path.