Skip to content
This repository has been archived by the owner on Dec 16, 2022. It is now read-only.

Commit

Permalink
Browse files Browse the repository at this point in the history
Merging vision into main (#4800)
* An initial VilBERT model for NLVR2 (#4423)

* Some initial work; lots left to do

* Initial test mostly passing, though things are still a bit of a mess

* tests are passing with small fixtures

* remove prints

* Test more stuff

* PathLike

* Make vilbert pass tests

* PR comments

* call float before log

* add CI

Co-authored-by: Dirk Groeneveld <dirkg@allenai.org>

* Initializing a VilBERT model from a pre-trained transformer (#4495)

* saving state

* Code is running, though it is returning zero gradients (but not None)

* initial test passing, still working on albert

* albert works, but bert-base-uncased still gives zero gradients

* Loading of weights should now work

* black, flake, mypy

* remove drop and mask functionality from reader

* make comment better

* fix tests

* flake

Co-authored-by: Dirk Groeneveld <dirkg@allenai.org>

* new data loading (#4497)

* first implementation

* update docstrings

* fixes

* fix sharding logic

* clean up DatasetReader

* fix samplers

* fixes

* fixes

* patch models for now

* more fixes

* fix linting error

* fix model test case

* some fixes

* fix linting err

* updates

* rename dataloader -> data_loader

* fixes

* more JoinableQueue

* set daemon=True

* fixes

* fix

* fixes

* fix

* update shuffle logic

* load instances right away when not lazy

* add tqdm when num_workers <= 0

* apply_token_indexers

* fix bug causing high mem usage

* address some of @dirkgr's comments

* fix lazy

* use sensible default for max_batches_in_mem

* ensure workers terminated on err

* fix

* start adding some tests

* more tests

* add some more tests

* address most of Matt's comments

* update PyTorchDataLoader test

* get rid of lazy option

* fix linting

* update docs, change max_batches_per_epoch to max_instances_per_epcoh

* update CHANGELOG

* fix drop_last validation

* fix py2md test fixture

* handle drop_last

* update docs

* implement sharding for most readers

* fix worker init fn

* limit tqdm output

* fixes

* ensure vision CI runs on each commit (#4582)

* ensure vision CI runs on each commit

* fix

* try fix CHANGELOG check

* ensure models check runs on right branch

* Formatting updates for new version of black (#4607)

* reformat for new version of black (#4605)

* reformat for new version of black

* pin black

* reformat for black

* fix

* rename 'node_rank' to 'global_rank' in dataset reader 'DistributedInfo' (#4608)

* rename 'node_rank' to 'global_rank'

* Clarify doc comments

* fix line length

* remove duplicate padding calculations in collate fn (#4617)

* fix len calculation for new data loader (#4618)

* fix len calculation for new data loader

* add test

Co-authored-by: Dirk Groeneveld <dirkg@allenai.org>

* make existing readers work with multi-process loading (#4597)

* make existing readers work with multi-process loading

* add 'overrides' decorator

* call apply_token_indexers in predictor

* clean up

* fix tests

* Add MultiTaskModel (#4601)

* Initial design of the multi-task model

* PR comments, more implementation

* changelog and docs fix

* More tests, and fixes for those tests

* mypy and make test less flaky

* Update allennlp/models/multitask.py

* Update allennlp/models/multitask.py

Co-authored-by: Dirk Groeneveld <groeneveld@gmail.com>

* Update allennlp/models/multitask.py

Co-authored-by: James Barry <james.barry26@mail.dcu.ie>

* respect active heads in get_metrics

* Clean up changelog

* black (apparently github UI doesn't add newlines?)

Co-authored-by: Dirk Groeneveld <dirkg@allenai.org>
Co-authored-by: Dirk Groeneveld <groeneveld@gmail.com>
Co-authored-by: James Barry <james.barry26@mail.dcu.ie>

* Detectron NLVR2 (#4481)

* Passes a batch of detectron images to the model in the correct format

* Loads a model and runs inference on it

* Some initial work; lots left to do

* Initial test mostly passing, though things are still a bit of a mess

* tests are passing with small fixtures

* remove prints

* More configurable reader

* add image_root and feature extraction to detectron model

* Use general detectron cfg functions

* Adds TensorField

* Fix detectron dependency

* Adds a detectron processor that we can use in dataset readers

* Test more stuff

* PathLike

* Make vilbert pass tests

* PR comments

* call float before log

* add CI

* PathLike

* Adds another NLVR2 reader

* add region feature and grid feature configuration json and attrtibute to cfg file

* change detectron_utils based on https://github.com/vedanuj/grid-feats-vqa/blob/master/extract_feature.py

* add bottom up and top down roi head into detectron2 based on allennlp/models/detectron.py

* Fix padding in TensorField

* Fix field construction

* Adds ability to read an arbitrary file

* More type annotations

* Remove old reader, add test for new one

* Use the right kind of field

* Run Jiasen's configs as tests

* We don't need this field

* Removes detectron reader

* Remove detectron reader and field

* Unify ArrayField and TensorField

* Making sure that no merge will go cleanly from now on

* Clean up the new output from the detectron processor a bit

* Fix Detectron2 version as v0.2

* saving state

* Code is running, though it is returning zero gradients (but not None)

* initial test passing, still working on albert

* albert works, but bert-base-uncased still gives zero gradients

* Note

* Formatting

* Adds Registrable base classes for image operations

* Adds a real example of a image2image module

* Run the new code (without implementation) in the nlvr2 reader

* Solve some issue involving circular imports

* add new modules for vilbert

* add parameters for detectron image loader.

* push current code on implementing proposal generator.

* push current progress on proposal generator

* Update FasterRCNNProposalGenerator & Merge Detectron2 config

* Loading of weights should now work

* black, flake, mypy

* Run detectron pipeline pieces one at a time

This is unfinished and will not run this way.

* Fix the data format for the backbone

* Handle image sizes separately

* remove drop and mask functionality from reader

* make comment better

* remove proposal_embedder, and finish proposal generator

* working on grid embedder

* added simple test for resnet backbone, which passes

* Got proposal generator test passing

* Change default number of detections per image: 100 => 36

* Fix detectron config hierarchy: test_detectron_per_image

* Make number of detections configurable & Add test

* rename ProposalGenerator to RegionDetector

* try to fix makefile

* another attempt at makefile

* quotes in the pip command...

* added a simple test for the dataset reader, made it pass

* add feature caching to the dataset reader

* another try with the makefile

* a better temporary fix for installing detectron

* writing files before committing is good...

* fix tests

* fix (at least part of) the vilbert tests

* ok, this makefile change should actually work

* add torchvision, try to remove eager import of detectron code

* flake

* cleanup

* more cleanup

* mypy, flake

* add back code I shouldn't have removed

* black

* test and flake fixes

* fix region_detector for multiple images and add feature and coords padding

* fix imports

* restore null grid embedder

* add back (todo) null region detector

* Bring back import changes, to fix circular imports caused by NLVR2
reader

* region detector test passing

* model test finally passing

* update torchvision version

* add vqav2 dataset

* add gpu support for detectron feature extraction

* add lmdbCache to cache feature into lmdb database

* fix typo

* update vqa jsonnet

* fix url adding by cat

* Fixes type annotation

* Fixes borked error message

* New feature cache

* Formatting

* Fix the tensor cache

* Be explicit about our dependencies

* Use the new tensor cache

* Adds a test using the tensor cache

* Run NLVR dataprep on GPU

* Tqdm when finding images

* Fixes padding in array field

* Adjust max_length when truncating in PretrainedTransformerTokenizer

* Fewer print statements

* remove VQA from this branch and copy default vilbert parameters.

* Sanjay's vision features cache script (#4633)

* Use LMDB cache in NLVR2 dataset reader; fix a few typos

* Standalone script for caching image features

* Removing reference to LMDB cache in NLVR2 dataset reader

* Adding back asterisk in nlvr2 dataset reader

* Fixing one variable name mistake

* Decreasing batch size and making a few cuda-related changes

* Loading images in batches to avoid GPU OOM error

* Pedantic changes for consistency

* Run the pre-processing with the models and not the data loading

* Filter out paths of images already cached

* Add image extensions other than png

* Fixes import error

* Makes the vision features script work alongside other scripts or training runs

Co-authored-by: sanjays <sanjays@ip-10-0-0-157.us-west-2.compute.internal>
Co-authored-by: sanjays <sanjays@ip-10-1-10-157.us-west-2.compute.internal>
Co-authored-by: Sanjay Subramanian <sanjays@allennlp-server1.corp.ai2>
Co-authored-by: Sanjay Subramanian <sanjays_ssubramanian@hotmail.com>

* Adds missing imports

* Makes TensorCache into a real MutableMapping

* Formatting

* Changelog

* Fix typecheck

* Makes the NLVR2 reader work with Pete's new code

* Fix type annotation

* Formatting

* Backwards compatibility

* Fix tests

* Fix broken config

* Update grid embedder test

* Fix vilbert_from_huggingface configuration

* Don't run the vilbert_from_huggingface test anymore

* Remove unused test fixtures

* Fix the region detector test

* Fix vilbert-from-huggingface and bring it back

* Fuck the linter

* Run the region detector test on GPU

* Run more stuff on GPU

The CPU test runner doesn't have enough memory.

* Depend on newer version of Detectron

* Reinstall Detectron before running tests

* Just force CUDA to be on, instead of reinstalling Detecton2

* Detectron needs CUDA_HOME to be set during install

At least this thing fails quickly.

* Try a different way of wrangling the detectron installer

* Bring back amp

* Trying to make tests faster, and passing

* use two regions, to make tests pass

* black

* Documentation for TensorCache

* Documentation for the NLVR2 dataset reader

* Rename ArrayField to TensorField

Co-authored-by: Matt Gardner <mattg@allenai.org>
Co-authored-by: jiasenlu <jiasenlu@gatech.edu>
Co-authored-by: Jaemin Cho <heythisischo@gmail.com>
Co-authored-by: jiasenlu <echosenm@gmail.com>
Co-authored-by: sanjays <sanjays@ip-10-0-0-157.us-west-2.compute.internal>
Co-authored-by: sanjays <sanjays@ip-10-1-10-157.us-west-2.compute.internal>
Co-authored-by: Sanjay Subramanian <sanjays@allennlp-server1.corp.ai2>
Co-authored-by: Sanjay Subramanian <sanjays_ssubramanian@hotmail.com>

* This should have been part of the previously merged PR

* Transformer toolkit (#4577)

* transformer toolkit: BertEmbeddings

* transformer toolkit: BertSelfAttention

* transformer toolkit: BertSelfOutput

* transformer toolkit: BertAttention

* transformer toolkit: BertIntermediate

* transformer toolkit: BertOutput

* transformer toolkit: BertLayer

* transformer toolkit: BertBiAttention

* transformer toolkit: BertEmbeddings

* transformer toolkit: BertSelfAttention

* transformer toolkit: BertSelfOutput

* transformer toolkit: BertAttention

* transformer toolkit: BertIntermediate

* transformer toolkit: BertOutput

* transformer toolkit: BertLayer

* transformer toolkit: BertBiAttention

* Attention scoring functions

* merging output and self output

* utility to replicate layers, further cleanup

* adding sinusoidal positional encoding

* adding activation layer

* adding base class for generic loading of pretrained weights

* further generalizing, adding tests

* updates

* adding bimodal encoder, kwargs in from_pretrained_module

* vilbert using transformer toolkit

* fixing test function

* changing to torch.allclose

* fixing attention score api

* bug fix in bimodal output

* changing to older attention modules

* _construct_default_mapping returns mapping

* adding kwargs to _get_input_arguments, adding examples

* using cached_transformers

* making transformer_encoder more general

* added get_relevant_module, loading by name

* fixing constructor name

* undoing failure after merge

* misc minor changes

Co-authored-by: Dirk Groeneveld <dirkg@allenai.org>

* Transformer toolkit: BiModalEncoder now has separate `num_attention_heads` for both modalities (#4728)

* separate num_attention_heads for both modalities, default arguments

* adding tests for toolkit examples

* debug statements for failing test

* removing debug statements, reordering

* Let's be more tolerant

* removing commented code

Co-authored-by: Dirk Groeneveld <dirkg@allenai.org>

* separating TransformerPooler as a new module (#4730)

* separating TransformerPooler as a new module

* adding size check

* fix failing tests

* Generalizing self attention (#4756)

* generalizing SelfAttention

* typecheck changes

* adding shape information to docstring

Co-authored-by: Dirk Groeneveld <dirkg@allenai.org>

* Multitask data loading and scheduling (#4625)

* Some initial work, still a bunch left to do

* Adds a utility function that can shuffle iterables

* remove shuffle

* Getting close; saving state before fixing lint and adding tests

* mypy and flake

* put in some initial schedulers and samplers; just need to write tests

* added some tests

* changelog

* add more-itertools to setup.py

* finish docstring

* some PR comments addressed

* mypy

* use homogeneous scheduler by default, not the non-homogeneous one

* add option to not shuffle

* normalize dataset proportions

* Update allennlp/data/data_loaders/multitask_data_loader.py

Co-authored-by: Dirk Groeneveld <dirkg@allenai.org>

* improve independence of vision components (#4793)

* improve independence of vision components

* fix install

* fix failing test

* haha, actually fix

* include torchvision exception too

* fix torchvision install

* remove vision push trigger

* VQAv2 (#4639)

* albert works, but bert-base-uncased still gives zero gradients

* Note

* Formatting

* Adds Registrable base classes for image operations

* Adds a real example of a image2image module

* Run the new code (without implementation) in the nlvr2 reader

* Solve some issue involving circular imports

* add new modules for vilbert

* add parameters for detectron image loader.

* push current code on implementing proposal generator.

* push current progress on proposal generator

* Update FasterRCNNProposalGenerator & Merge Detectron2 config

* Loading of weights should now work

* black, flake, mypy

* Run detectron pipeline pieces one at a time

This is unfinished and will not run this way.

* Fix the data format for the backbone

* Handle image sizes separately

* remove drop and mask functionality from reader

* make comment better

* remove proposal_embedder, and finish proposal generator

* working on grid embedder

* added simple test for resnet backbone, which passes

* Got proposal generator test passing

* Change default number of detections per image: 100 => 36

* Fix detectron config hierarchy: test_detectron_per_image

* Make number of detections configurable & Add test

* rename ProposalGenerator to RegionDetector

* try to fix makefile

* another attempt at makefile

* quotes in the pip command...

* added a simple test for the dataset reader, made it pass

* add feature caching to the dataset reader

* another try with the makefile

* a better temporary fix for installing detectron

* writing files before committing is good...

* fix tests

* fix (at least part of) the vilbert tests

* ok, this makefile change should actually work

* add torchvision, try to remove eager import of detectron code

* flake

* cleanup

* more cleanup

* mypy, flake

* add back code I shouldn't have removed

* black

* test and flake fixes

* fix region_detector for multiple images and add feature and coords padding

* fix imports

* restore null grid embedder

* add back (todo) null region detector

* Bring back import changes, to fix circular imports caused by NLVR2
reader

* region detector test passing

* model test finally passing

* update torchvision version

* add vqav2 dataset

* add gpu support for detectron feature extraction

* add lmdbCache to cache feature into lmdb database

* fix typo

* update vqa jsonnet

* fix url adding by cat

* Fixes type annotation

* Fixes borked error message

* New feature cache

* Formatting

* Fix the tensor cache

* Be explicit about our dependencies

* Use the new tensor cache

* Adds a test using the tensor cache

* Run NLVR dataprep on GPU

* Tqdm when finding images

* Fixes padding in array field

* Adjust max_length when truncating in PretrainedTransformerTokenizer

* Fewer print statements

* remove VQA from this branch and copy default vilbert parameters.

* add VQAv2 dataset

* Added dataset reader and model tests, which are now passing

* Sanjay's vision features cache script (#4633)

* Use LMDB cache in NLVR2 dataset reader; fix a few typos

* Standalone script for caching image features

* Removing reference to LMDB cache in NLVR2 dataset reader

* Adding back asterisk in nlvr2 dataset reader

* Fixing one variable name mistake

* Decreasing batch size and making a few cuda-related changes

* Loading images in batches to avoid GPU OOM error

* Pedantic changes for consistency

* Run the pre-processing with the models and not the data loading

* Filter out paths of images already cached

* Add image extensions other than png

* Fixes import error

* Makes the vision features script work alongside other scripts or training runs

Co-authored-by: sanjays <sanjays@ip-10-0-0-157.us-west-2.compute.internal>
Co-authored-by: sanjays <sanjays@ip-10-1-10-157.us-west-2.compute.internal>
Co-authored-by: Sanjay Subramanian <sanjays@allennlp-server1.corp.ai2>
Co-authored-by: Sanjay Subramanian <sanjays_ssubramanian@hotmail.com>

* Adds missing imports

* Makes TensorCache into a real MutableMapping

* Formatting

* Changelog

* Fix typecheck

* Makes the NLVR2 reader work with Pete's new code

* Fix type annotation

* Formatting

* Backwards compatibility

* Restore NLVR to former glory

* Types and multi-process reading for VQAv2

* Formatting

* Fix tests

* Fix broken config

* Update grid embedder test

* Fix vilbert_from_huggingface configuration

* Don't run the vilbert_from_huggingface test anymore

* Remove unused test fixtures

* Fix the region detector test

* Fix vilbert-from-huggingface and bring it back

* Fuck the linter

* Fix for VQA test

* Why was this metric disabled?

* Black and flake

* Re-add VQA reader

* Image featurizers now need to be called with sizes

* Run the region detector test on GPU

* Run more stuff on GPU

The CPU test runner doesn't have enough memory.

* Depend on newer version of Detectron

* Reinstall Detectron before running tests

* Just force CUDA to be on, instead of reinstalling Detecton2

* Fixes VQA2 DatasetReader

* Fix documentation

* Detectron needs CUDA_HOME to be set during install

At least this thing fails quickly.

* Try a different way of wrangling the detectron installer

* Try a different way of wrangling the detectron installer

* Bring back amp

* Refactored VQA reader

* More training paths

* Remove debug code

* Don't check in debug code

* Auto-detect GPU to use

* Apply indexers later

* Fix typo

* Register the model

* Fields live on CPU. Only batches get GPUs.

* black

* black, flake

* mypy

* more flake

* More realistic training config

* Adds a basic Predictor for VQAv2

* Make vilbert output human-readable

* Forgot to enumerate

* Use the right namspace

* Trying to make tests faster, and passing

* add image prefix when loading coco image

* fix vqav2 dataset reader and config file

* use two regions, to make tests pass

* black

* Output probabilities in addition to logits

* Make it possible to turn off the cache

* Turn off the cache in the predictor

* Fix the VQA predictor

* change the experiment to the defualt vilbert hyperparams.

* add default experiment_from_huggingface.json

* fix typos in vqa reader

* Proper probabilities

* Formatting

* Remove unused variable

* Make mypy happy

* Fixed loss function, metric, and got tests to pass

* Updates the big training config

* Put real settings into the vilbert_vqa config

* Strings are lists in Python

* Make mypy happy

* Formatting

* Unsatisfying mypy

* Config changes to make this run

* Fix dimensionality of embeddings

* clean the code and add the image_num_heads and combine_num_heads

* fix answer vocab and add save and load from pre-extracted vocab

* fix loss and update save_answer_vocab script

* Typo

* Fixed fusion method

* Tweaking the VQA config some more

* Moved the from_huggingface config

* 20 epochs

* Set up the learning rate properly

* Simplify

* Hardcoded answer vocab

* Don't be lazy

* Steps per epoch cannot be None

* Let's chase the right score

* Fixing some parameter names

* Fields are stored on CPUs

* Bigger batch size, easier distributed training

* Don't run the debug code by default

* VQA with the Transformer Toolkit (#4729)

* transformer toolkit: BertEmbeddings

* transformer toolkit: BertSelfAttention

* transformer toolkit: BertSelfOutput

* transformer toolkit: BertAttention

* transformer toolkit: BertIntermediate

* transformer toolkit: BertOutput

* transformer toolkit: BertLayer

* transformer toolkit: BertBiAttention

* transformer toolkit: BertEmbeddings

* transformer toolkit: BertSelfAttention

* transformer toolkit: BertSelfOutput

* transformer toolkit: BertAttention

* transformer toolkit: BertIntermediate

* transformer toolkit: BertOutput

* transformer toolkit: BertLayer

* transformer toolkit: BertBiAttention

* Attention scoring functions

* merging output and self output

* utility to replicate layers, further cleanup

* adding sinusoidal positional encoding

* adding activation layer

* adding base class for generic loading of pretrained weights

* further generalizing, adding tests

* updates

* adding bimodal encoder, kwargs in from_pretrained_module

* vilbert using transformer toolkit

* fixing test function

* changing to torch.allclose

* fixing attention score api

* bug fix in bimodal output

* changing to older attention modules

* _construct_default_mapping returns mapping

* adding kwargs to _get_input_arguments, adding examples

* using cached_transformers

* making transformer_encoder more general

* added get_relevant_module, loading by name

* fixing constructor name

* undoing failure after merge

* misc minor changes

* Transformer toolkit (#4577)

* transformer toolkit: BertEmbeddings

* transformer toolkit: BertSelfAttention

* transformer toolkit: BertSelfOutput

* transformer toolkit: BertAttention

* transformer toolkit: BertIntermediate

* transformer toolkit: BertOutput

* transformer toolkit: BertLayer

* transformer toolkit: BertBiAttention

* transformer toolkit: BertEmbeddings

* transformer toolkit: BertSelfAttention

* transformer toolkit: BertSelfOutput

* transformer toolkit: BertAttention

* transformer toolkit: BertIntermediate

* transformer toolkit: BertOutput

* transformer toolkit: BertLayer

* transformer toolkit: BertBiAttention

* Attention scoring functions

* merging output and self output

* utility to replicate layers, further cleanup

* adding sinusoidal positional encoding

* adding activation layer

* adding base class for generic loading of pretrained weights

* further generalizing, adding tests

* updates

* adding bimodal encoder, kwargs in from_pretrained_module

* vilbert using transformer toolkit

* fixing test function

* changing to torch.allclose

* fixing attention score api

* bug fix in bimodal output

* changing to older attention modules

* _construct_default_mapping returns mapping

* adding kwargs to _get_input_arguments, adding examples

* using cached_transformers

* making transformer_encoder more general

* added get_relevant_module, loading by name

* fixing constructor name

* undoing failure after merge

* misc minor changes

Co-authored-by: Dirk Groeneveld <dirkg@allenai.org>

* separate num_attention_heads for both modalities, default arguments

* adding tests for toolkit examples

* debug statements for failing test

* removing debug statements, reordering

* Typo

* Some compatibility with the transformer toolkit

* Reorganize the image inputs

* More transformer toolkit compatibility

* Debug settings

* Let's be more tolerant

* Fix how VilBERT runs

Co-authored-by: Akshita Bhagia <akshita23bhagia@gmail.com>

* Make the region detector and region embedder lazy

* Fix references to the model

* Make various automated tests pass

* Formatting

* More logging

* One more logging statement

* Read answer vocab from vocab file instead of determining it automatically

* Don't keep the files open so long

* Use most of the validation set for training as well

* Get ready to be lazy

* Upgrade paths

* Be lazy

* Keep unanswerable questions only during test time

* Fix the from_huggingface config

* Fixes the VQA score

* VQA specific metric

* Fixes some tests

* Tests pass!

* Formatting

* Use the correct directory

* Use the region detector that's meant for testing

* Read the test split properly

* Be a little more verbose while discovering images

* Modernize Vilbert VQA

* Update NLVR, but it still doesn't run

* Formatting

* Remove NLVR

* Fix the last test

* Formatting

* Conditionally export the VilbertVqaPredictor

* ModuleNotFoundError is a type of ImportError

* Fix test-install

* Try the broken test with a fixed seed

* Try a bunch of seeds

* Smaller model to get bigger magnitudes

* Now that the test works, we don't need to specify the seeds anymore

Co-authored-by: Matt Gardner <mattg@allenai.org>
Co-authored-by: jiasenlu <jiasenlu@gatech.edu>
Co-authored-by: Jaemin Cho <heythisischo@gmail.com>
Co-authored-by: jiasenlu <echosenm@gmail.com>
Co-authored-by: sanjays <sanjays@ip-10-0-0-157.us-west-2.compute.internal>
Co-authored-by: sanjays <sanjays@ip-10-1-10-157.us-west-2.compute.internal>
Co-authored-by: Sanjay Subramanian <sanjays@allennlp-server1.corp.ai2>
Co-authored-by: Sanjay Subramanian <sanjays_ssubramanian@hotmail.com>
Co-authored-by: Akshita Bhagia <akshita23bhagia@gmail.com>
Co-authored-by: Evan Pete Walsh <epwalsh10@gmail.com>

* SNLI_VE dataset reader (#4799)

* adding VE reader

* removing jsonlines

* blackify

* intial VE model

* adding VisionReader for common vision components

* fix test file

* fix doc

* temporarily removing VE model

* bug fix

* cleanup

* removing unnecessary check

* simplify

* Visual entailment model code (#4822)

* VE model code

* adding VE model

* misc minor updates

* update changelog

* Added GQA reader (#4832)

* Adds reader for GQA dataset. Will download questions from https://cs.stanford.edu/people/dorarad/gqa/download.html.

* Cleaned up GQA reader tests

* Other VQA datasets (#4834)

* Make the VQA reader work for the other datasets

* Also find pngs

* Really support pngs

* Remove debug code

* More logging

* Unexpected formatting

* Respect the device

* This is how your replace things in named tuples.

* Remove unused import

* This is how you override a method properly.

* This is how you set parameters in detectron.

* Also set the device for the region detector

* Training configs for all three datasets contained in VQA

* Bigger batches

* Bigger batches for image processing

* Fix vilbert-from-huggingface config

* Make the config switch modes for constructing vocab

* More vocab, more docs, better way of deriving vocab

* Modernize the from_huggingface config

* More updates to the from_huggingface config

* Better hyperparameters stolen from another project

* Fix for inverted parameter

* Formatting

* Throw a meaningful error message when we don't have images

* Add a warning that includes instructions for how to fix things

* Remove unused script

* Merge issue

* adding multilabel option (#4843)

* Generalizing transformer layers (#4776)

* adding HF tests, docstrings for AttentionLayer, TransformerLayer, TransformerBlock

* temp change to check if tests pass

* undoing temp change

* ci update

* more ci updates

* changing test run

* update makefile

* temp change

* isolating failing case

* further debugging

* fail check

* reverting to older CI

* test with reduced batch size

* cleanup

* more cleanup

* oops, fix

* gqa reader fixes during vilbert training (#4851)

* Refactored shared code

* typecheck fix

* rebase

* Refactored shared code

* typecheck fix

* rebase

* Cleaned up GQA reader tests

* Modify instance format for vilbert-vqa model

* update for vision branch bump

Co-authored-by: Jackson Stokes <jacksons@Jacksons-MacBook-Pro.local>
Co-authored-by: Dirk Groeneveld <dirkg@allenai.org>

* Toolkit: Adding documentation and small changes for `BiModalAttention` (#4859)

* adding documentation for bimodal attn, small fixes

* changing the way mask is applied

* using large value rather than inf

* Update comment

Co-authored-by: Dirk Groeneveld <groeneveld@gmail.com>

* moving apply_mask to util

Co-authored-by: Dirk Groeneveld <groeneveld@gmail.com>

* Make tests work again (#4865)

* New import paths

* Duplicate entries

* Dataset readers can't be lazy anymore

* Switch to torchvision for vision components 👀, simplify and improve MultiProcessDataLoader (#4821)

* implement TorchImageLoader

* implement ResnetBackbone

* add resize + normalize to image loader

* finalize FasterRcnnRegionDetector

* pin torchvision

* fix VQAv2Reader

* add box mask field

* dataset reader fixes

* fix model tests

* doc fixes

* add threshold parameters to FasterRcnnRegionDetector

* address @dirkgr comments

* mask fixes

* shape comments

* add some more comments

* cache answers_by_question_id

* implement LocalCacheResource

* fix

* add read-only option to cache

* fix

* simplify data loader

* make featurizer and detector optional in readers

* Cache in memory

* back pressure is important I guess

* merge

* Updated configs

* Fixes the way we apply masks

* Use more of Jiasen's real settings

* Upgrade the from_huggingface config

* Switch back to the images on corpnet

* Fix random seeds

* Bigger model needs smaller batch size

* Adds ability to selectively ignore one input

* address some comments

* format + lint

* fixes

* Bring back bert-base configs

* fix error handling

* fix test

* fix typo

* use lock when possible

Co-authored-by: Dirk Groeneveld <dirkg@allenai.org>

* doc fixes

* Only cache, no featurizing (#4870)

* implement TorchImageLoader

* implement ResnetBackbone

* add resize + normalize to image loader

* finalize FasterRcnnRegionDetector

* pin torchvision

* fix VQAv2Reader

* add box mask field

* dataset reader fixes

* fix model tests

* doc fixes

* add threshold parameters to FasterRcnnRegionDetector

* address @dirkgr comments

* mask fixes

* shape comments

* add some more comments

* cache answers_by_question_id

* implement LocalCacheResource

* fix

* add read-only option to cache

* fix

* simplify data loader

* make featurizer and detector optional in readers

* Cache in memory

* back pressure is important I guess

* merge

* Updated configs

* Fixes the way we apply masks

* Use more of Jiasen's real settings

* Upgrade the from_huggingface config

* Switch back to the images on corpnet

* Fix random seeds

* Bigger model needs smaller batch size

* Adds ability to selectively ignore one input

* address some comments

* format + lint

* fixes

* Bring back bert-base configs

* fix error handling

* fix test

* Adds the ability to read from a feature cache, but not run any featurization

* Update tests

* Let's stick with "feature_cache"

As long as we're consistent ...

* More epochs, more random

* Use the new parameters

* Fix initialization

* Make tests work, add some documentation

* Remove the read_from_cache parameter

* Cleanup of training configs

* Typecheck

* Building docs right

* Better settings for VQA

* Leave the image_feature_dim at 1024

Co-authored-by: epwalsh <epwalsh10@gmail.com>

* Make images easier to find for Visual Entailment (#4878)

* implement TorchImageLoader

* implement ResnetBackbone

* add resize + normalize to image loader

* finalize FasterRcnnRegionDetector

* pin torchvision

* fix VQAv2Reader

* add box mask field

* dataset reader fixes

* fix model tests

* doc fixes

* add threshold parameters to FasterRcnnRegionDetector

* address @dirkgr comments

* mask fixes

* shape comments

* add some more comments

* cache answers_by_question_id

* implement LocalCacheResource

* fix

* add read-only option to cache

* fix

* simplify data loader

* make featurizer and detector optional in readers

* Cache in memory

* back pressure is important I guess

* merge

* Updated configs

* Fixes the way we apply masks

* Use more of Jiasen's real settings

* Upgrade the from_huggingface config

* Switch back to the images on corpnet

* Fix random seeds

* Bigger model needs smaller batch size

* Adds ability to selectively ignore one input

* address some comments

* format + lint

* fixes

* Bring back bert-base configs

* fix error handling

* fix test

* Adds the ability to read from a feature cache, but not run any featurization

* Update tests

* Let's stick with "feature_cache"

As long as we're consistent ...

* More epochs, more random

* Use the new parameters

* Fix initialization

* Make tests work, add some documentation

* Remove the read_from_cache parameter

* Cleanup of training configs

* Typecheck

* Building docs right

* Better settings for VQA

* Open cached paths when reading json lines

* By default, autodetect GPUs when training

* Switch to torchvision

* Download training data from the web

* This needs to stay at 1024 until we get the new featurization model

* Have a more descriptive error message when images are missing

* Update vilbert_ve_from_huggingface.jsonnet

Co-authored-by: epwalsh <epwalsh10@gmail.com>
Co-authored-by: Akshita Bhagia <akshita23bhagia@gmail.com>

* Adding f1 score (#4890)

* adding f1 score

* updated config

* import MultiTaskDataLoader to data_loaders/__init__.py (#4885)

* Make GQA work (#4884)

* Refactored shared code

* typecheck fix

* rebase

* Refactored shared code

* typecheck fix

* rebase

* Cleaned up GQA reader tests

* Modify instance format for vilbert-vqa model

* update for vision branch bump

* Adding training config for GQA

* Unnamed variable

* Various GQA fixes

* Temporary extra configs needed to make vocab

* Remove unused file

* Optimize VQA score instead of F-Score

* Use our newly created vocab

* Remove temporary configs

* Don't fail when we don't need to create a directory

* Make a config that works on the servers as well

* Update comment

* A new command to count instances

* Temporary config to count instances

* Undo temporary changes

* Put in the correct number of steps per epoch

* Remove this number from the config because it's almost certainly wrong

* Don't put Fields in Tuples

* Formatting

* More informative error message when batches are heterogeneous

* Formatting

* Not my type

* Generate the fields properly when answers are missing

* Properly discard instances with missing answers

* Changelog

* Update number of steps per epoch

* Adds a config for balanced GQA

* fix file_utils extract with directory

* fix Batch._check_types

* Fill in URL

Co-authored-by: Jackson Stokes <jacksons@Jacksons-MacBook-Pro.local>
Co-authored-by: Akshita Bhagia <akshita23bhagia@gmail.com>
Co-authored-by: Evan Pete Walsh <epwalsh10@gmail.com>

* Toolkit: Cleaning up TransformerEmbeddings (#4900)

* fixing issue of non-deterministic dropout

* updating TransformerEmbeddings

* ImageFeatureEmbeddings is now a subclass of Embeddings

* allowing for no token type embeddings

* fixing kwargs for loading pretrained module

* Data loading cuda device (#4879)

* add test with tensor fields

* improve nn.util.move_to_device

* ensure start_method is 'spawn' when using lazy and mem pin

* add 'non_blocking' arg to 'move_to_device'

* fix fake test tensor

* fix sampler test

* lint

* fix 'move_to_device'

* fix condition check

* add device to data loader

* clean up doc string

* rename 'device' arg to 'cuda_device'

* pinning is very slow, revert

* DataLoaders load to CUDA device

* fix evaluate test

* rename 'multi_process_*' -> 'multiprocess' for consistency (#4906)

* MultiProcessDataLoader takes PathLike data_path (#4908)

* remove PyTorchDataLoader, add SimpleDataLoader for testing (#4907)

* remove PyTorchDataLoader, add SimpleDataLoader for testing

* fix test

* comments

* improve data loading docs (#4909)

* improve data loading docs

* document best practices, add 'get_batch_size' method to samplers

* try fix annoying unrelated test

* revert that

* clarify handling of 'max_instances_in_memory'

* fix imports in file_utils

* rename 'master' -> 'primary' for distributed training (#4910)

* improve worker error handling in MultiProcessDataLoader (#4912)

* improve worker error handling

* rename test file

* Toolkit decoder (#4914)

* adding cross_attention, renaming block -> stack

* stack can be initialized with layer too

Co-authored-by: Dirk Groeneveld <dirkg@allenai.org>

* resolve _read type (#4916)

* resolve _read type

* fix sharded reader

* fix data loader arg

* Multitask example (#4898)

* Make the VQA reader work for the other datasets

* Also find pngs

* Really support pngs

* Remove debug code

* More logging

* Unexpected formatting

* Respect the device

* This is how your replace things in named tuples.

* Remove unused import

* This is how you override a method properly.

* This is how you set parameters in detectron.

* Also set the device for the region detector

* Training configs for all three datasets contained in VQA

* Bigger batches

* Bigger batches for image processing

* Fix vilbert-from-huggingface config

* Make the config switch modes for constructing vocab

* More vocab, more docs, better way of deriving vocab

* Modernize the from_huggingface config

* More updates to the from_huggingface config

* Better hyperparameters stolen from another project

* Fix for inverted parameter

* Formatting

* Throw a meaningful error message when we don't have images

* Add a warning that includes instructions for how to fix things

* Remove unused script

* Merge issue

* Adds named splits to the SNLI-VE reader

* Make the multitask data loader discoverable

* Formatting

* More flexible inputs to the dataset readers

* Prototype config for the multitask training job

* json_lines_from_file() already calls cached_path()

* Visual entailment should track accuracy

* Switching to torch

* Fixing VE image paths

* Formatting

* Experimentally use threaded_generator to read instances from readers simultaneously

* Vilbert backbone

* Fixed paths

* Formatting

* Adds heads

* Revert "Experimentally use threaded_generator to read instances from readers simultaneously"

This reverts commit a633e67.

* Multitask trains now!

* Remove useless parameter from GQA reader

* Updated multitask config

* Schedulers produce batches, not instances

* Track multiple metrics

* Make mypy happy

* Formatting

* Keep better track of which heads have been called

* Fix the merge

* We have more than strings for input

* Remove unused imports

* -1 is CPU

* Go back to tracking instances per epoch so that the samplers can work

* Better error message

* A useful sampler to have

* We haven't indexed until we've indexed

* Makes tests pass

* Formatting

* Fine-tuning the metric tracker

* Update model configs for my changes

* Fixing model configs for Akshita's changes

* Implement VisionTextModel in terms of VilbertBackbone

* Formatting

* Fix stale comment

* Use the server paths by default, not Dirk's desktop

* Fix tests

* Formatting again

* Removed data loader parameters that don't exist anymore

* Clarified comment

Co-authored-by: Evan Pete Walsh <epwalsh10@gmail.com>

* Moves vision models to allennlp-models (#4918)

* Moves vision models to allennlp-models

* Also move test fixtures

* Don't return so many instances if we're cutting them out later anyways

* We actually need this image

* Formatting

* Fixing more paths

* Prepare for release v2.0.0rc1

* Make releasing work with the renamed master branch, and with the vision branch

* Debugging the release process in the slowest way possible

* Another attempt at fixing the release process

* Generic Callbacks (#4917)

* Better Callbacks

* Reformatting

* Fixes

* Tests for updated TrainerCallback

* Formatting and Type-Checking fixes

* Consistent metric tracker (#4928)

* Makes the metric tracker more consistent

* Turns out we need best_epoch_metrics after all.

* Backwards compatibility

* Formatting

* Remove old script

* Changes CI since we won't have a `vision` branch anymore

* fix up CHANGELOG

Co-authored-by: Matt Gardner <mattg@allenai.org>
Co-authored-by: epwalsh <epwalsh10@gmail.com>
Co-authored-by: James Barry <james.barry26@mail.dcu.ie>
Co-authored-by: jiasenlu <jiasenlu@gatech.edu>
Co-authored-by: Jaemin Cho <heythisischo@gmail.com>
Co-authored-by: jiasenlu <echosenm@gmail.com>
Co-authored-by: sanjays <sanjays@ip-10-0-0-157.us-west-2.compute.internal>
Co-authored-by: sanjays <sanjays@ip-10-1-10-157.us-west-2.compute.internal>
Co-authored-by: Sanjay Subramanian <sanjays@allennlp-server1.corp.ai2>
Co-authored-by: Sanjay Subramanian <sanjays_ssubramanian@hotmail.com>
Co-authored-by: Akshita Bhagia <akshita23bhagia@gmail.com>
Co-authored-by: jvstokes <40584422+jvstokes@users.noreply.github.com>
Co-authored-by: Jackson Stokes <jacksons@Jacksons-MacBook-Pro.local>
Co-authored-by: Karen Hambardzumyan <mahnerak@gmail.com>
  • Loading branch information
15 people committed Jan 26, 2021
1 parent 65e50b3 commit 67fa291
Show file tree
Hide file tree
Showing 164 changed files with 9,012 additions and 2,499 deletions.
17 changes: 11 additions & 6 deletions .github/workflows/ci.yml
Expand Up @@ -151,7 +151,8 @@ jobs:
ALLENNLP_VERSION_OVERRIDE: "" # Don't replace the core library.
run: |
git clone https://github.com/allenai/allennlp-models.git
cd allennlp-models && pip install --upgrade --upgrade-strategy eager -e . -r dev-requirements.txt
cd allennlp-models
pip install --upgrade --upgrade-strategy eager -e . -r dev-requirements.txt
- name: Run models tests
run: |
Expand Down Expand Up @@ -288,11 +289,11 @@ jobs:
run: |
# Check the install instructions on https://pytorch.org/ to keep these up-to-date.
if [[ $CUDA == '10.1' ]]; then
echo "DOCKER_TORCH_VERSION='torch==1.7.1+cu101 -f https://download.pytorch.org/whl/torch_stable.html'" >> $GITHUB_ENV;
echo "DOCKER_TORCH_VERSION='torch==1.7.1+cu101 torchvision==0.8.2+cu101 -f https://download.pytorch.org/whl/torch_stable.html'" >> $GITHUB_ENV;
elif [[ $CUDA == '10.2' ]]; then
echo "DOCKER_TORCH_VERSION='torch==1.7.1'" >> $GITHUB_ENV;
elif [[ $CUDA == '11.0' ]]; then
echo "DOCKER_TORCH_VERSION='torch==1.7.1+cu110 -f https://download.pytorch.org/whl/torch_stable.html'" >> $GITHUB_ENV;
echo "DOCKER_TORCH_VERSION='torch==1.7.1+cu110 torchvision==0.8.2+cu110 -f https://download.pytorch.org/whl/torch_stable.html'" >> $GITHUB_ENV;
else
echo "Unhandled CUDA version $CUDA";
exit 1;
Expand Down Expand Up @@ -389,15 +390,19 @@ jobs:
run: |
./scripts/build_docs.sh
- name: Print the ref
run: |
echo ${{ github.ref }}
- name: Configure Git
if: github.event_name == 'release' || github.event_name == 'push'
if: github.event_name == 'release' || (github.event_name == 'push' && github.ref == 'refs/heads/main')
run: |
git config --global user.email "ai2service@allenai.org"
git config --global user.name "ai2service"
git config --global push.default simple
- name: Stage docs
if: github.event_name == 'release' || github.event_name == 'push'
if: github.event_name == 'release' || (github.event_name == 'push' && github.ref == 'refs/heads/main')
run: |
echo "Staging docs to $DOCS_FOLDER"
Expand Down Expand Up @@ -449,7 +454,7 @@ jobs:
EOL
- name: Deploy docs
if: github.event_name == 'release' || github.event_name == 'push'
if: github.event_name == 'release' || (github.event_name == 'push' && github.ref == 'refs/heads/main')
run: |
# And push them up to GitHub
cd ~/allennlp-docs/
Expand Down
80 changes: 76 additions & 4 deletions CHANGELOG.md
Expand Up @@ -10,6 +10,49 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

- The `TrainerCallback` constructor accepts `serialization_dir` provided by `Trainer`. This can be useful for `Logger` callbacks those need to store files in the run directory.
- The `TrainerCallback.on_start()` is fired at the start of the training.
- The `TrainerCallback` event methods now accept `**kwargs`. This may be useful to maintain backwards-compability of callbacks easier in the future. E.g. we may decide to pass the exception/traceback object in case of failure to `on_end()` and this older callbacks may simply ignore the argument instead of raising a `TypeError`.

### Changed

- The `TrainerCallack.on_epoch()` does not fire with `epoch=-1` at the start of the training.
Instead, `TrainerCallback.on_start()` should be used for these cases.
- `TensorBoardBatchMemoryUsage` is converted from `BatchCallback` into `TrainerCallback`.
- `TrackEpochCallback` is converted from `EpochCallback` into `TrainerCallback`.
- `Trainer` can accept callbacks simply with name `callbacks` instead of `trainer_callbacks`.

### Removed

- Removed `EpochCallback`, `BatchCallback` in favour of `TrainerCallback`.
The metaclass-wrapping implementation is removed as well.

### Fixed

- Now Trainer always fires `TrainerCallback.on_end()` so all the resources can be cleaned up properly.
- Fixed the misspelling, changed `TensoboardBatchMemoryUsage` to `TensorBoardBatchMemoryUsage`.
- We set a value to `epoch` so in case of firing `TrainerCallback.on_end()` the variable is bound.
This could have lead to an error in case of trying to recover a run after it was finished training.


## [v2.0.0rc1](https://github.com/allenai/allennlp/releases/tag/v2.0.0rc1) - 2021-01-21

### Added

- Added `TensorCache` class for caching tensors on disk
- Added abstraction and concrete implementation for image loading
- Added abstraction and concrete implementation for `GridEmbedder`
- Added abstraction and demo implementation for an image augmentation module.
- Added abstraction and concrete implementation for region detectors.
- A new high-performance default `DataLoader`: `MultiProcessDataLoading`.
- A `MultiTaskModel` and abstractions to use with it, including `Backbone` and `Head`. The
`MultiTaskModel` first runs its inputs through the `Backbone`, then passes the result (and
whatever other relevant inputs it got) to each `Head` that's in use.
- A `MultiTaskDataLoader`, with a corresponding `MultiTaskDatasetReader`, and a couple of new
configuration objects: `MultiTaskEpochSampler` (for deciding what proportion to sample from each
dataset at every epoch) and a `MultiTaskScheduler` (for ordering the instances within an epoch).
- Transformer toolkit to plug and play with modular components of transformer architectures.
- Added a command to count the number of instances we're going to be training with
- Added a `FileLock` class to `common.file_utils`. This is just like the `FileLock` from the `filelock` library, except that
it adds an optional flag `read_only_ok: bool`, which when set to `True` changes the behavior so that a warning will be emitted
instead of an exception when lacking write permissions on an existing file lock.
Expand All @@ -19,14 +62,42 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Moving `ModelCard` and `TaskCard` abstractions into the main repository.
- Added a util function `allennlp.nn.util.dist_reduce(...)` for handling distributed reductions.
This is especially useful when implementing a distributed `Metric`.
- Added a `FileLock` class to `common.file_utils`. This is just like the `FileLock` from the `filelock` library, except that
it adds an optional flag `read_only_ok: bool`, which when set to `True` changes the behavior so that a warning will be emitted
instead of an exception when lacking write permissions on an existing file lock.
This makes it possible to use the `FileLock` class on a read-only file system.
- Added a new learning rate scheduler: `CombinedLearningRateScheduler`. This can be used to combine different LR schedulers, using one after the other.
- Moving `ModelCard` and `TaskCard` abstractions into the main repository.

### Changed

- `DatasetReader`s are now always lazy. This means there is no `lazy` parameter in the base
class, and the `_read()` method should always be a generator.
- The `DataLoader` now decides whether to load instances lazily or not.
With the `PyTorchDataLoader` this is controlled with the `lazy` parameter, but with
the `MultiProcessDataLoading` this is controlled by the `max_instances_in_memory` setting.
- `ArrayField` is now called `TensorField`, and implemented in terms of torch tensors, not numpy.
- Improved `nn.util.move_to_device` function by avoiding an unnecessary recursive check for tensors and
adding a `non_blocking` optional argument, which is the same argument as in `torch.Tensor.to()`.
- If you are trying to create a heterogeneous batch, you now get a better error message.
- Readers using the new vision features now explicitly log how they are featurizing images.
- `master_addr` and `master_port` renamed to `primary_addr` and `primary_port`, respectively.
- `is_master` parameter for training callbacks renamed to `is_primary`.
- `master` branch renamed to `main`
- Torch version bumped to 1.7.1 in Docker images.
- 'master' branch renamed to 'main'
- Torch version bumped to 1.7.1 in Docker images.

### Removed

- Removed `nn.util.has_tensor`.

### Fixed

- The `build-vocab` command no longer crashes when the resulting vocab file is
in the current working directory.
- VQA models now use the `vqa_score` metric for early stopping. This results in
much better scores.
- Fixed typo with `LabelField` string representation: removed trailing apostrophe.
- `Vocabulary.from_files` and `cached_path` will issue a warning, instead of failing, when a lock on an existing resource
can't be acquired because the file system is read-only.
Expand Down Expand Up @@ -58,11 +129,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
were not passed to the constructor if the value of the parameter was equal to the default value.
This caused bugs in some edge cases where a subclass that takes `**kwargs` needs to inspect
`kwargs` before passing them to its superclass.
- Improved the band-aid solution for segmentation faults and the "ImportError: dlopen: cannot load any more object with static TLS"
- Improved the band-aid solution for segmentation faults and the "ImportError: dlopen: cannot load any more object with static TLS"
by adding a `transformers` import.
- Added safety checks for extracting tar files
- Turned superfluous warning to info when extending the vocab in the embedding matrix, if no pretrained file was provided


## [v1.2.2](https://github.com/allenai/allennlp/releases/tag/v1.2.2) - 2020-11-17

### Added
Expand Down Expand Up @@ -213,6 +285,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Fixed a bug in the cnn_encoder where activations involving masked tokens could be picked up by the max
- Fix intra word tokenization for `PretrainedTransformerTokenizer` when disabling fast tokenizer.


## [v1.1.0](https://github.com/allenai/allennlp/releases/tag/v1.1.0) - 2020-09-08

### Fixed
Expand All @@ -227,8 +300,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

- `Predictor.capture_model_internals()` now accepts a regex specifying
which modules to capture
- `Predictor.capture_model_internals()` now accepts a regex specifying which modules to capture.


## [v1.1.0rc4](https://github.com/allenai/allennlp/releases/tag/v1.1.0rc4) - 2020-08-20
Expand Down Expand Up @@ -295,7 +367,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
in the log output even when `train_parameters` was set to `False`.
- Fixed a bug with the sharded dataset reader where it would only read a fraction of the instances
in distributed training.
- Fixed checking equality of `ArrayField`s.
- Fixed checking equality of `TensorField`s.
- Fixed a bug where `NamespaceSwappingField` did not work correctly with `.empty_field()`.
- Put more sensible defaults on the `huggingface_adamw` optimizer.
- Simplified logging so that all logging output always goes to one file.
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Expand Up @@ -18,7 +18,7 @@ LABEL com.nvidia.volumes.needed="nvidia_driver"

WORKDIR /stage/allennlp

# Install torch first. This build arg should be in the form of a version requirement,
# Install torch ecosystem first. This build arg should be in the form of a version requirement,
# like 'torch==1.7' or 'torch==1.7+cu102 -f https://download.pytorch.org/whl/torch_stable.html'.
ARG TORCH
RUN pip install --no-cache-dir ${TORCH}
Expand Down
4 changes: 2 additions & 2 deletions Dockerfile.test
Expand Up @@ -17,7 +17,7 @@ LABEL com.nvidia.volumes.needed="nvidia_driver"

WORKDIR /stage/allennlp

# Install torch first. This build arg should be in the form of a version requirement,
# Install torch ecosystem first. This build arg should be in the form of a version requirement,
# like 'torch==1.7' or 'torch==1.7+cu102 -f https://download.pytorch.org/whl/torch_stable.html'.
ARG TORCH
RUN pip install --no-cache-dir ${TORCH}
Expand All @@ -30,7 +30,7 @@ COPY setup.py .
COPY dev-requirements.txt .
RUN touch allennlp/__init__.py \
&& touch README.md \
&& pip install --no-cache-dir -r dev-requirements.txt -e .
&& pip install --no-cache-dir -e . -r dev-requirements.txt

# Now add the full package source and re-install just the package.
COPY . .
Expand Down
5 changes: 2 additions & 3 deletions Makefile
Expand Up @@ -13,9 +13,9 @@ MD_DOCS_EXTRAS = $(addprefix $(MD_DOCS_ROOT),README.md CHANGELOG.md CONTRIBUTING
DOCKER_TAG = latest
DOCKER_IMAGE_NAME = allennlp/allennlp:$(DOCKER_TAG)
DOCKER_TEST_IMAGE_NAME = allennlp/test:$(DOCKER_TAG)
DOCKER_TORCH_VERSION = 'torch==1.7.1'
DOCKER_TORCH_VERSION = 'torch==1.7.1 torchvision==0.8.2'
# Our self-hosted runner currently has CUDA 11.0.
DOCKER_TEST_TORCH_VERSION = 'torch==1.7.1+cu110 -f https://download.pytorch.org/whl/torch_stable.html'
DOCKER_TEST_TORCH_VERSION = 'torch==1.7.1+cu110 torchvision==0.8.2+cu110 -f https://download.pytorch.org/whl/torch_stable.html'
DOCKER_RUN_CMD = docker run --rm \
-v $$HOME/.allennlp:/root/.allennlp \
-v $$HOME/.cache/huggingface:/root/.cache/huggingface \
Expand Down Expand Up @@ -85,7 +85,6 @@ install :
# Due to a weird thing with pip, we may need egg-info before running `pip install -e`.
# See https://github.com/pypa/pip/issues/4537.
python setup.py install_egg_info
# Install allennlp as editable and all dependencies.
pip install --upgrade --upgrade-strategy eager -e . -r dev-requirements.txt

#
Expand Down
5 changes: 4 additions & 1 deletion README.md
Expand Up @@ -148,7 +148,10 @@ to distribute as a plugin, see the [subcommand API docs](https://docs.allennlp.o

## Installation

AllenNLP requires Python 3.6.1 or later. The preferred way to install AllenNLP is via `pip`. Just run `pip install allennlp` in your Python environment and you're good to go!
AllenNLP requires Python 3.6.1 or later and [PyTorch](https://pytorch.org/).
It's recommended that you install the PyTorch ecosystem **before** installing AllenNLP by following the instructions on [pytorch.org](https://pytorch.org/).

The preferred way to install AllenNLP is via `pip`. Just run `pip install allennlp`.

> ⚠️ If you're using Python 3.7 or greater, you should ensure that you don't have the PyPI version of `dataclasses` installed after running the above command, as this could cause issues on certain platforms. You can quickly check this by running `pip freeze | grep dataclasses`. If you see something like `dataclasses=0.6` in the output, then just run `pip uninstall -y dataclasses`.
Expand Down
1 change: 1 addition & 0 deletions allennlp/commands/__init__.py
Expand Up @@ -15,6 +15,7 @@
from allennlp.commands.subcommand import Subcommand
from allennlp.commands.test_install import TestInstall
from allennlp.commands.train import Train
from allennlp.commands.count_instances import CountInstances
from allennlp.common.plugins import import_plugins
from allennlp.common.util import import_module_and_submodules

Expand Down
3 changes: 2 additions & 1 deletion allennlp/commands/build_vocab.py
Expand Up @@ -65,7 +65,8 @@ def build_vocab_from_args(args: argparse.Namespace):
raise RuntimeError(f"{args.output_path} already exists. Use --force to overwrite.")

output_directory = os.path.dirname(args.output_path)
os.makedirs(output_directory, exist_ok=True)
if len(output_directory) > 0:
os.makedirs(output_directory, exist_ok=True)

params = Params.from_file(args.param_path)

Expand Down
52 changes: 52 additions & 0 deletions allennlp/commands/count_instances.py
@@ -0,0 +1,52 @@
"""
Subcommand for counting the number of instances from a training config.
"""

import argparse
import logging

from overrides import overrides

from allennlp.commands.subcommand import Subcommand
from allennlp.common.params import Params


logger = logging.getLogger(__name__)


@Subcommand.register("count-instances")
class CountInstances(Subcommand):
@overrides
def add_subparser(self, parser: argparse._SubParsersAction) -> argparse.ArgumentParser:
description = """Count the number of training instances in an experiment config file."""
subparser = parser.add_parser(self.name, description=description, help=description)
subparser.add_argument("param_path", type=str, help="path to an experiment config file")

subparser.add_argument(
"-o",
"--overrides",
type=str,
default="",
help=(
"a json(net) structure used to override the experiment configuration, e.g., "
"'{\"vocabulary.min_count.labels\": 10}'. Nested parameters can be specified either"
" with nested dictionaries or with dot syntax."
),
)

subparser.set_defaults(func=count_instances_from_args)

return subparser


def count_instances_from_args(args: argparse.Namespace):
from allennlp.training.util import data_loaders_from_params

params = Params.from_file(args.param_path)

data_loaders = data_loaders_from_params(params, train=True, validation=False, test=False)
instances = sum(
1 for data_loader in data_loaders.values() for _ in data_loader.iter_instances()
)

print(f"Success! One epoch of training contains {instances} instances.")
20 changes: 11 additions & 9 deletions allennlp/commands/evaluate.py
Expand Up @@ -134,24 +134,26 @@ def evaluate_from_args(args: argparse.Namespace) -> Dict[str, Any]:

evaluation_data_path = args.input_file
logger.info("Reading evaluation data from %s", evaluation_data_path)
instances = dataset_reader.read(evaluation_data_path)

data_loader_params = config.pop("validation_data_loader", None)
if data_loader_params is None:
data_loader_params = config.pop("data_loader")
if args.batch_size:
data_loader_params["batch_size"] = args.batch_size
data_loader = DataLoader.from_params(
params=data_loader_params, reader=dataset_reader, data_path=evaluation_data_path
)

embedding_sources = (
json.loads(args.embedding_sources_mapping) if args.embedding_sources_mapping else {}
)

if args.extend_vocab:
logger.info("Vocabulary is being extended with test instances.")
model.vocab.extend_from_instances(instances=instances)
model.vocab.extend_from_instances(instances=data_loader.iter_instances())
model.extend_embedder_vocab(embedding_sources)

instances.index_with(model.vocab)
data_loader_params = config.pop("validation_data_loader", None)
if data_loader_params is None:
data_loader_params = config.pop("data_loader")
if args.batch_size:
data_loader_params["batch_size"] = args.batch_size
data_loader = DataLoader.from_params(dataset=instances, params=data_loader_params)
data_loader.index_with(model.vocab)

metrics = evaluate(
model,
Expand Down

0 comments on commit 67fa291

Please sign in to comment.