Skip to content
This repository has been archived by the owner on Dec 16, 2022. It is now read-only.

VQA with the Transformer Toolkit #4729

Merged
merged 58 commits into from
Oct 15, 2020
Merged

VQA with the Transformer Toolkit #4729

merged 58 commits into from
Oct 15, 2020

Commits on Aug 14, 2020

  1. transformer toolkit: BertEmbeddings

    @AkshitaB
    AkshitaB committed Aug 14, 2020
    Configuration menu
    Copy the full SHA
    db26b83 View commit details
    Browse the repository at this point in the history

Commits on Aug 18, 2020

  1. transformer toolkit: BertSelfAttention

    @AkshitaB
    AkshitaB committed Aug 18, 2020
    Configuration menu
    Copy the full SHA
    7cdc7b9 View commit details
    Browse the repository at this point in the history

Commits on Aug 19, 2020

  1. transformer toolkit: BertSelfOutput

    @AkshitaB
    AkshitaB committed Aug 19, 2020
    Configuration menu
    Copy the full SHA
    4647527 View commit details
    Browse the repository at this point in the history

  2. transformer toolkit: BertAttention

    @AkshitaB
    AkshitaB committed Aug 19, 2020
    Configuration menu
    Copy the full SHA
    941de0a View commit details
    Browse the repository at this point in the history

  3. transformer toolkit: BertIntermediate

    @AkshitaB
    AkshitaB committed Aug 19, 2020
    Configuration menu
    Copy the full SHA
    1e279fa View commit details
    Browse the repository at this point in the history

  4. transformer toolkit: BertOutput

    @AkshitaB
    AkshitaB committed Aug 19, 2020
    Configuration menu
    Copy the full SHA
    0dbd17e View commit details
    Browse the repository at this point in the history

  5. transformer toolkit: BertLayer

    @AkshitaB
    AkshitaB committed Aug 19, 2020
    Configuration menu
    Copy the full SHA
    84ebf9b View commit details
    Browse the repository at this point in the history

  6. transformer toolkit: BertBiAttention

    @AkshitaB
    AkshitaB committed Aug 19, 2020
    Configuration menu
    Copy the full SHA
    b3c6262 View commit details
    Browse the repository at this point in the history

Commits on Sep 2, 2020

  1. transformer toolkit: BertEmbeddings

    @AkshitaB
    AkshitaB committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    dec734f View commit details
    Browse the repository at this point in the history

  2. transformer toolkit: BertSelfAttention

    @AkshitaB
    AkshitaB committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    aa907a9 View commit details
    Browse the repository at this point in the history

  3. transformer toolkit: BertSelfOutput

    @AkshitaB
    AkshitaB committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    6a6c05b View commit details
    Browse the repository at this point in the history

  4. transformer toolkit: BertAttention

    @AkshitaB
    AkshitaB committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    be896da View commit details
    Browse the repository at this point in the history

  5. transformer toolkit: BertIntermediate

    @AkshitaB
    AkshitaB committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    2da9c7b View commit details
    Browse the repository at this point in the history

  6. transformer toolkit: BertOutput

    @AkshitaB
    AkshitaB committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    1698fed View commit details
    Browse the repository at this point in the history

  7. transformer toolkit: BertLayer

    @AkshitaB
    AkshitaB committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    c58d703 View commit details
    Browse the repository at this point in the history

  8. transformer toolkit: BertBiAttention

    @AkshitaB
    AkshitaB committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    812b276 View commit details
    Browse the repository at this point in the history

  9. Attention scoring functions

    @AkshitaB
    AkshitaB committed Sep 2, 2020
    Configuration menu
    Copy the full SHA
    4796129 View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2020

  1. merging output and self output

    @AkshitaB
    AkshitaB committed Sep 3, 2020
    Configuration menu
    Copy the full SHA
    ec37dc2 View commit details
    Browse the repository at this point in the history

  2. cleanup

    @AkshitaB
    AkshitaB committed Sep 3, 2020
    Configuration menu
    Copy the full SHA
    f6266b3 View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2020

  1. utility to replicate layers, further cleanup

    @AkshitaB
    AkshitaB committed Sep 9, 2020
    Configuration menu
    Copy the full SHA
    6b72fd4 View commit details
    Browse the repository at this point in the history

Commits on Sep 14, 2020

  1. adding sinusoidal positional encoding

    @AkshitaB
    AkshitaB committed Sep 14, 2020
    Configuration menu
    Copy the full SHA
    0f1e86e View commit details
    Browse the repository at this point in the history

  2. adding activation layer

    @AkshitaB
    AkshitaB committed Sep 14, 2020
    Configuration menu
    Copy the full SHA
    745a567 View commit details
    Browse the repository at this point in the history

  3. adding base class for generic loading of pretrained weights

    @AkshitaB
    AkshitaB committed Sep 14, 2020
    Configuration menu
    Copy the full SHA
    7b1c586 View commit details
    Browse the repository at this point in the history

Commits on Sep 15, 2020

  1. further generalizing, adding tests

    @AkshitaB
    AkshitaB committed Sep 15, 2020
    Configuration menu
    Copy the full SHA
    21d8690 View commit details
    Browse the repository at this point in the history

  2. updates

    @AkshitaB
    AkshitaB committed Sep 15, 2020
    Configuration menu
    Copy the full SHA
    58eb166 View commit details
    Browse the repository at this point in the history

  3. adding bimodal encoder, kwargs in from_pretrained_module

    @AkshitaB
    AkshitaB committed Sep 15, 2020
    Configuration menu
    Copy the full SHA
    f979664 View commit details
    Browse the repository at this point in the history

Commits on Sep 16, 2020

  1. vilbert using transformer toolkit

    @AkshitaB
    AkshitaB committed Sep 16, 2020
    Configuration menu
    Copy the full SHA
    a354c55 View commit details
    Browse the repository at this point in the history

  2. fixing test function

    @AkshitaB
    AkshitaB committed Sep 16, 2020
    Configuration menu
    Copy the full SHA
    294ef45 View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2020

  1. Merge branch 'vision' into transformer-toolkit

    @AkshitaB
    AkshitaB committed Sep 18, 2020
    Configuration menu
    Copy the full SHA
    275d799 View commit details
    Browse the repository at this point in the history

Commits on Sep 22, 2020

  1. changing to torch.allclose

    @AkshitaB
    AkshitaB committed Sep 22, 2020
    Configuration menu
    Copy the full SHA
    2cc0eea View commit details
    Browse the repository at this point in the history

  2. fixing attention score api

    @AkshitaB
    AkshitaB committed Sep 22, 2020
    Configuration menu
    Copy the full SHA
    7459efb View commit details
    Browse the repository at this point in the history

Commits on Sep 23, 2020

  1. bug fix in bimodal output

    @AkshitaB
    AkshitaB committed Sep 23, 2020
    Configuration menu
    Copy the full SHA
    f8183f3 View commit details
    Browse the repository at this point in the history

  2. changing to older attention modules

    @AkshitaB
    AkshitaB committed Sep 23, 2020
    Configuration menu
    Copy the full SHA
    850fde4 View commit details
    Browse the repository at this point in the history

  3. _construct_default_mapping returns mapping

    @AkshitaB
    AkshitaB committed Sep 23, 2020
    Configuration menu
    Copy the full SHA
    2420314 View commit details
    Browse the repository at this point in the history

Commits on Sep 29, 2020

  1. adding kwargs to _get_input_arguments, adding examples

    @AkshitaB
    AkshitaB committed Sep 29, 2020
    Configuration menu
    Copy the full SHA
    ae09756 View commit details
    Browse the repository at this point in the history

Commits on Sep 30, 2020

  1. Merge branch 'vision' into transformer-toolkit

    @AkshitaB
    AkshitaB committed Sep 30, 2020
    Configuration menu
    Copy the full SHA
    143725f View commit details
    Browse the repository at this point in the history

  2. using cached_transformers

    @AkshitaB
    AkshitaB committed Sep 30, 2020
    Configuration menu
    Copy the full SHA
    05c331a View commit details
    Browse the repository at this point in the history

Commits on Oct 2, 2020

  1. making transformer_encoder more general

    @AkshitaB
    AkshitaB committed Oct 2, 2020
    Configuration menu
    Copy the full SHA
    616ce2c View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2020

  1. added get_relevant_module, loading by name

    @AkshitaB
    AkshitaB committed Oct 3, 2020
    Configuration menu
    Copy the full SHA
    85ad6fd View commit details
    Browse the repository at this point in the history

  2. fixing constructor name

    @AkshitaB
    AkshitaB committed Oct 3, 2020
    Configuration menu
    Copy the full SHA
    42bedf1 View commit details
    Browse the repository at this point in the history

Commits on Oct 9, 2020

  1. Merge remote-tracking branch 'origin/vision' into transformer-toolkit 

    # Conflicts:
#	CHANGELOG.md
#	allennlp/models/vilbert.py
#	test_fixtures/vilbert/experiment.json
#	test_fixtures/vilbert/experiment_from_huggingface.jsonnet
    @dirkgr
    dirkgr committed Oct 9, 2020
    Configuration menu
    Copy the full SHA
    f5d04d5 View commit details
    Browse the repository at this point in the history

  2. undoing failure after merge

    @AkshitaB
    AkshitaB committed Oct 9, 2020
    Configuration menu
    Copy the full SHA
    566e523 View commit details
    Browse the repository at this point in the history

  3. misc minor changes

    @AkshitaB
    AkshitaB committed Oct 9, 2020
    Configuration menu
    Copy the full SHA
    5472ac5 View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2020

  1. Transformer toolkit (#4577) 

    * transformer toolkit: BertEmbeddings

* transformer toolkit: BertSelfAttention

* transformer toolkit: BertSelfOutput

* transformer toolkit: BertAttention

* transformer toolkit: BertIntermediate

* transformer toolkit: BertOutput

* transformer toolkit: BertLayer

* transformer toolkit: BertBiAttention

* transformer toolkit: BertEmbeddings

* transformer toolkit: BertSelfAttention

* transformer toolkit: BertSelfOutput

* transformer toolkit: BertAttention

* transformer toolkit: BertIntermediate

* transformer toolkit: BertOutput

* transformer toolkit: BertLayer

* transformer toolkit: BertBiAttention

* Attention scoring functions

* merging output and self output

* utility to replicate layers, further cleanup

* adding sinusoidal positional encoding

* adding activation layer

* adding base class for generic loading of pretrained weights

* further generalizing, adding tests

* updates

* adding bimodal encoder, kwargs in from_pretrained_module

* vilbert using transformer toolkit

* fixing test function

* changing to torch.allclose

* fixing attention score api

* bug fix in bimodal output

* changing to older attention modules

* _construct_default_mapping returns mapping

* adding kwargs to _get_input_arguments, adding examples

* using cached_transformers

* making transformer_encoder more general

* added get_relevant_module, loading by name

* fixing constructor name

* undoing failure after merge

* misc minor changes

Co-authored-by: Dirk Groeneveld <dirkg@allenai.org>
    @AkshitaB @dirkgr
    AkshitaB and dirkgr committed Oct 10, 2020
    Configuration menu
    Copy the full SHA
    91631ef View commit details
    Browse the repository at this point in the history

Commits on Oct 12, 2020

  1. Merge remote-tracking branch 'origin/vision' into VQA 

    # Conflicts:
#	allennlp/models/vilbert.py
    @dirkgr
    dirkgr committed Oct 12, 2020
    Configuration menu
    Copy the full SHA
    a56944b View commit details
    Browse the repository at this point in the history

Commits on Oct 13, 2020

  1. separate num_attention_heads for both modalities, default arguments

    @AkshitaB
    AkshitaB committed Oct 13, 2020
    Configuration menu
    Copy the full SHA
    9af3cee View commit details
    Browse the repository at this point in the history

  2. Merge branch 'vision' into transformer-toolkit

    @AkshitaB
    AkshitaB committed Oct 13, 2020
    Configuration menu
    Copy the full SHA
    6d19592 View commit details
    Browse the repository at this point in the history

Commits on Oct 14, 2020

  1. adding tests for toolkit examples

    @AkshitaB
    AkshitaB committed Oct 14, 2020
    Configuration menu
    Copy the full SHA
    fa2ab22 View commit details
    Browse the repository at this point in the history

  2. debug statements for failing test

    @AkshitaB
    AkshitaB committed Oct 14, 2020
    Configuration menu
    Copy the full SHA
    9013766 View commit details
    Browse the repository at this point in the history

  3. removing debug statements, reordering

    @AkshitaB
    AkshitaB committed Oct 14, 2020
    Configuration menu
    Copy the full SHA
    290ec6f View commit details
    Browse the repository at this point in the history

  4. Typo

    @dirkgr
    dirkgr committed Oct 14, 2020
    Configuration menu
    Copy the full SHA
    5387f53 View commit details
    Browse the repository at this point in the history

  5. Some compatibility with the transformer toolkit

    @dirkgr
    dirkgr committed Oct 14, 2020
    Configuration menu
    Copy the full SHA
    0e1a2e3 View commit details
    Browse the repository at this point in the history

  6. Reorganize the image inputs

    @dirkgr
    dirkgr committed Oct 14, 2020
    Configuration menu
    Copy the full SHA
    d571d93 View commit details
    Browse the repository at this point in the history

  7. More transformer toolkit compatibility

    @dirkgr
    dirkgr committed Oct 14, 2020
    Configuration menu
    Copy the full SHA
    69cd8e8 View commit details
    Browse the repository at this point in the history

  8. Debug settings

    @dirkgr
    dirkgr committed Oct 14, 2020
    Configuration menu
    Copy the full SHA
    32e9b67 View commit details
    Browse the repository at this point in the history

Commits on Oct 15, 2020

  1. Let's be more tolerant

    @dirkgr
    dirkgr committed Oct 15, 2020
    Configuration menu
    Copy the full SHA
    8f4660a View commit details
    Browse the repository at this point in the history

  2. Merge branch 'transformer-toolkit' into VQAWithTToolkit

    @dirkgr
    dirkgr committed Oct 15, 2020
    Configuration menu
    Copy the full SHA
    7e2b85f View commit details
    Browse the repository at this point in the history

  3. Fix how VilBERT runs

    @dirkgr
    dirkgr committed Oct 15, 2020
    Configuration menu
    Copy the full SHA
    5881d0f View commit details
    Browse the repository at this point in the history