More help info for archive_surgery.py
#2327
Conversation
* script for doing archive surgery * simplify script
* Update the elmo command help text. * Update the find-lr subcommand help text.
As it currently stands, the following is logged during training:
```
2019-01-06 10:46:21,832 - INFO - allennlp.common.from_params - instantiating class <class 'allennlp.model
s.language_model.LanguageModel'> from params {'bidirectional': False, 'contextualizer': {'bidirectional':
False, 'dropout': 0.5, 'hidden_size': 200, 'input_size': 200, 'num_layers': 2, 'type': 'lstm'}, 'dropout
': 0.5, 'text_field_embedder': {'token_embedders': {'tokens': {'embedding_dim': 200, 'type': 'embedding'}
}}} and extras {'vocab': <allennlp.data.vocabulary.Vocabulary object at 0x7ff7811665f8>}
```
Note that the `Vocabulary` does not provide any useful information, since it doesn't have `__repr__` defined. This provides a fix.
…#2264) Fixes allenai#2255 This PR replaces the `BidirectionalLM` class with a more-general `LanguageModel` that can be used in either the unidirectional/forward setting or the bidirectional setting. It also accordingly replaces the `BidirectionalLanguageModelTokenEmbedder` with a `LanguageModelTokenEmbedder`. Also fixes bug in the experiment_unsampled.jsonnet config that was preventing a test from actually being unsampled. TODO: - [x] test the unidirectional case - [x] properly deprecate `BidirectionalLM` and `BidirectionalLanguageModelTokenEmbedder` - [x] check docs for accuracy - [x] fix user-facing training configs
Merge from allenai/allennlp master
There was a problem hiding this comment.
I find the new interplay between force and output-file and inplace very confusing. If I do inplace do I need to force or not? I can read the code, but it's not obvious from the names or the documentation.
What can we do to make this cleaner / easier to understand?
scripts/archive_surgery.py
Outdated
| raise RuntimeError("please specify an editor or set the $EDITOR environment variable") | ||
|
|
||
| if os.path.exists(args.output_file): | ||
| if not args.inplace and os.path.exists(args.output_file) and not args.force: | ||
| raise ValueError("output file already exists") |
There was a problem hiding this comment.
"output file already exists, use --force to override"
|
@joelgrus the args seem reasonable to me. You can assume you don't need force until an error message tells you to use it (once my comment is addressed). You don't need |
|
Or change the help string of |
|
@WrRan that strikes me as an improvement! |
|
Thanks for your feedback. 😄 |
|
@joelgrus does this look good to you? |
|
Another question, should we put this very useful scripts under python -m allennlp.scripts.archive_surgery --helpPersonally, I think it will be more convenient. What do you think? And thanks for your attention, again. 😃 |
Before, typing
python scripts/archive_surgery.py --help, got:usage: archive_surgery.py [-h] --input-file INPUT_FILE --output-file OUTPUT_FILE [--editor EDITOR] Perform surgery on a model.tar.gz archive optional arguments: -h, --help show this help message and exit --input-file INPUT_FILE --output-file OUTPUT_FILE --editor EDITORNow, you will get:
usage: archive_surgery.py [-h] --input-file INPUT_FILE --output-file OUTPUT_FILE [--editor EDITOR] Perform surgery on a model.tar.gz archive optional arguments: -h, --help show this help message and exit --input-file INPUT_FILE path to input file (default: None) --output-file OUTPUT_FILE path to output file (default: None) --editor EDITOR editor to launch, whose default value is the environment vaiable `$EDITOR` (default: vim)May help users to understand the meaning and state of the parameters.