Merge from awslabs/sockeye #2
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Updating default parameters with values from our arXiv paper.
Adding an end of sentence symbol to the source side.
#398) * Removed separate beam prune system test and added pruning to another system test. Prevents timeouts on CRON job for Travis. * fix
…ture. (#399) * Argument specifications for each CLI are now stored in a static structure.
Some notes on tests and PyPI.
* Partially reverted 5a3bf5f * Added support for config files Specify a config file with --config. Command line parameters have precedence over the values read from the config file, as expected. The config file is a json serialization of the namespace returned by argparse, casted to a dictionary. The file args.json produced by the training can be used as config file. The config file does not need to be complete. Missing parameters will be read from the command line or will take default values. Additionally the functionality introduced in 5a3bf5f has been reimplemented, accessing the argument_definitions member of ConfigArgumentParser. * Added unit tests for config files * Addressed (most of) github comments * switch from json to yaml * changelog, minor version * Changed config file format to YAML * Typo * mypi * Added test overwriting config file argument with command line
* Fix logic with training resumption * fix
* Merge Sockeye Autopilot. * Typing cleanup. * Update version, changelog. * Update description of Autopilot in changelog.
* Added hard lexical constraints (Post & Vilar, NAACL 2018)
This commit adds hard lexical constraints as described in Post & Vilar,
Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation (NAACL 2018).
Hard lexical constraints are words or phrases that must appear in the translation output.
To use this feature, for a particular input sentences, create a (1-line) JSON object of the following form:
```
{ "text": "Einer soll ein hoch@@ rangi@@ ges Mitglied aus Berlin gewesen sein .",
"constraints": ["is said to", "powerful"] }
```
You then need to pass the JSON input flag (`--json-input`) to `sockeye.translate`.
…gged the memory on the primary GPU device over time when many checkpoints were done. Gradient histograms are now logged to Tensorboard separated by device. (#407)
* added wmt18 data sets
Included tutorial about adapting NMT models, including LHUC.
This PR updates Sockeye to MXNet 1.2 which was released May 21st 2018. Core change to Sockeye is the use of the new LayerNormalization operator which reduces GPU memory usage. It uses the same set of parameters existing models are compatible, but running sockeye now requires mxnet 1.2.
Introducing the image captioning module. Type of models supported: ConvNet encoder and Sockeye NMT decoders Features: - Image encoder that extracts features using preetrained nets: `image_captioning.encoder` - Feature extraction script to dump features to disk `image_captioning.extract_features` - Pass-through embedding, since we do not need it for images - Image-text iterator that loads features on the fly during training with the option of loading all to memory: `image_captioning.data_io` - Training and inference pipelines for image captioning: `image_captioning.train`, `image_captioning.inference` and `image_captioning.captioner` - README with instructions on how to use the image captioning module: `image_captioning/README.md` - Visualization script that loads images and captions (prediction+ground truth) and display them: `image_captioning.visualize`
Also some cleanup of overly long lines, imports etc.
* Added LHUC for transformer model Note: Changed apply() function of LHUC to __call__ * Sized down transformer-lhuc integration test
* Downsized integration tests for faster test runs * downsized integration test data set sizes
…nd some reordering (#417)
This cleans up the pruning logic a little bit and continues work started in #422. Changes include: 1) the modifications on various data structures as a result of pruning are now more local to where it matters. 2) vectorized the pruning function and moved it to `utils.py` (similar to `topk()`). Also using a partial now. Vectorization may help us in moving operations to HybridBlocks in the future.
* bugfix: added cast; plus fixed stupid mistakes in test cases that let it sneak in
This refactors beam search to group fixed-size operations in beam search into cached ops through Gluon HybridBlocks. My testing showed ~3% speed improvement. Not much, but consistent. Another change that is included here is to not use columns from `sequences` to pass into the decoder step module, but use `best_word_indices` from the previous iteration. NDArray indexing seems expensive and ideally we should aim for avoiding all indexing ops in an iteration.
* Beam search concat and mypy fix. * fix
* Fix: Word based batching memory with longer source sentences. * comment clarification. * typos
* added wmt18 en<>de to autopilot
* Fix for transformer-with-conv-embed encoder. * changelog * version fix
Surprisingly, allocating memory for sequences/attentions once in beam search and writing to index t seems to be slower than concatenating with every step. Likewise, getting rid of the pad_dist write to C.PAD_ID index is faster. This change gives about +1.5 sent/sec on a laptop/CPU and +0.1–0.5 on P2/P3s with latest MXNet.
* updated tutorial parameters to use RNN * typo * updated layers
* Update requirements to mxnet-*mkl. * Update documentation and setup for requirements dir. * Use isclose to slightly relax equality check for coverage test. (Test was failing with MKL version of MXNet) * Add requirements to MANIFEST.in. * Update changelog.
Occasionally this symlink exists and then the whole training procedures dies. My guess is that stray NFS files prevent the deletion of the temporary directory from the previous round. The other files are safely overwritten because they use direct writes instead of a symlink. This should solve that problem.
ROUGE is now available as the stopping criterion for tasks such as summarization.
* Fix source factor splitting for single factor. - Use version of ndarray split that always returns a list for uniform handling. * Unit test for factor splitting. * Update version, changelog. * Keep original split call for sym_gen.
The new pyyaml version is introducing some issues, which we should fix. For now let's depend on the old version to unblock Travis.
…#361) * add chapter multi-instance translate for README.md and script mlt-trans to run multi-instance * add chapter multi-instance translate for README.md * change method to get physical cores number and add hyperlink for C4 and C5 in README.md * add option for running as benchmark or not * rewrite mlt-trans script using python and move it to tutorials * change code according to the commnets * add chapter multi-instance translate for README.md and script mlt-trans to run multi-instance * add chapter multi-instance translate for README.md * change method to get physical cores number and add hyperlink for C4 and C5 in README.md * move mlt_cpu_trans_benchmark to tutorials and rename it process_per_core_translation * fix typo * add option for running as benchmark or not * rewrite mlt-trans script using python and move it to tutorials * change code according to the commnets * move mlt_cpu_trans_benchmark to tutorials and rename it process_per_core_translation * add python script in tutortial * add license (Apache) and authors
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.