Add PoolFormer #15250
Closed
Add PoolFormer #15250
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
fix few spelling mistakes
* fix import * fix style Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
… labels (#15234) * Fixes tf_default_data_collator sometimes guessing the wrong dtype for labels * Add test for numpy scalar inputs
NielsRogge
reviewed
Jan 21, 2022
docs/source/model_doc/poolformer.mdx
Outdated
| @@ -0,0 +1,90 @@ | |||
| <!--Copyright 2021 The HuggingFace Team. All rights reserved. | |||
There was a problem hiding this comment.
Suggested change
| <!--Copyright 2021 The HuggingFace Team. All rights reserved. | |
| <!--Copyright 2022 The HuggingFace Team. All rights reserved. |
NielsRogge
reviewed
Jan 21, 2022
docs/source/model_doc/poolformer.mdx
Outdated
Comment on lines
29
to
31
| ## XLNetConfig | ||
|
|
||
| [[autodoc]] XLNetConfig |
There was a problem hiding this comment.
Suggested change
| ## XLNetConfig | |
| [[autodoc]] XLNetConfig |
This was a mistake in the CookieCutter template.
NielsRogge
reviewed
Jan 21, 2022
docs/source/model_doc/poolformer.mdx
Outdated
Comment on lines
38
to
49
| ## PoolFormerTokenizer | ||
|
|
||
| [[autodoc]] PoolFormerTokenizer | ||
| - build_inputs_with_special_tokens | ||
| - get_special_tokens_mask | ||
| - create_token_type_ids_from_sequences | ||
| - save_vocabulary | ||
|
|
||
|
|
||
| ## PoolFormerTokenizerFast | ||
|
|
||
| [[autodoc]] PoolFormerTokenizerFast |
There was a problem hiding this comment.
Can be replaced by PoolFormerFeatureExtractor (in case there's one needed).
NielsRogge
reviewed
Jan 21, 2022
docs/source/model_doc/poolformer.mdx
Outdated
Comment on lines
58
to
90
| ## PoolFormerForCausalLM | ||
|
|
||
| [[autodoc]] PoolFormerForCausalLM | ||
| - forward | ||
|
|
||
|
|
||
| ## PoolFormerForMaskedLM | ||
|
|
||
| [[autodoc]] PoolFormerForMaskedLM | ||
| - forward | ||
|
|
||
|
|
||
| ## PoolFormerForSequenceClassification | ||
|
|
||
| [[autodoc]] transformers.PoolFormerForSequenceClassification | ||
| - forward | ||
|
|
||
| ## PoolFormerForMultipleChoice | ||
|
|
||
| [[autodoc]] transformers.PoolFormerForMultipleChoice | ||
| - forward | ||
|
|
||
|
|
||
| ## PoolFormerForTokenClassification | ||
|
|
||
| [[autodoc]] transformers.PoolFormerForTokenClassification | ||
| - forward | ||
|
|
||
|
|
||
| ## PoolFormerForQuestionAnswering | ||
|
|
||
| [[autodoc]] PoolFormerForQuestionAnswering | ||
| - forward |
There was a problem hiding this comment.
Suggested change
| ## PoolFormerForCausalLM | |
| [[autodoc]] PoolFormerForCausalLM | |
| - forward | |
| ## PoolFormerForMaskedLM | |
| [[autodoc]] PoolFormerForMaskedLM | |
| - forward | |
| ## PoolFormerForSequenceClassification | |
| [[autodoc]] transformers.PoolFormerForSequenceClassification | |
| - forward | |
| ## PoolFormerForMultipleChoice | |
| [[autodoc]] transformers.PoolFormerForMultipleChoice | |
| - forward | |
| ## PoolFormerForTokenClassification | |
| [[autodoc]] transformers.PoolFormerForTokenClassification | |
| - forward | |
| ## PoolFormerForQuestionAnswering | |
| [[autodoc]] PoolFormerForQuestionAnswering | |
| - forward | |
| [[autodoc]] PoolFormerForImageClassification | |
| - forward |
NielsRogge
reviewed
Jan 21, 2022
src/transformers/__init__.py
Outdated
| @@ -140,6 +140,7 @@ | |||
| "load_tf2_weights_in_pytorch_model", | |||
| ], | |||
| # Models | |||
| "models.poolformer": ["POOLFORMER_PRETRAINED_CONFIG_ARCHIVE_MAP", "PoolFormerConfig", "PoolFormerTokenizer"], | |||
There was a problem hiding this comment.
You can remove the tokenizer everywhere.
NielsRogge
reviewed
Jan 21, 2022
src/transformers/__init__.py
Outdated
| @@ -398,6 +399,7 @@ | |||
| # tokenizers-backed objects | |||
| if is_tokenizers_available(): | |||
| # Fast tokenizers | |||
| _import_structure["models.poolformer"].append("PoolFormerTokenizerFast") | |||
NielsRogge
reviewed
Jan 21, 2022
src/transformers/__init__.py
Outdated
Comment on lines
611
to
625
| _import_structure["models.poolformer"].extend( | ||
| [ | ||
| "POOLFORMER_PRETRAINED_MODEL_ARCHIVE_LIST", | ||
| "PoolFormerForMaskedLM", | ||
| "PoolFormerForCausalLM", | ||
| "PoolFormerForMultipleChoice", | ||
| "PoolFormerForQuestionAnswering", | ||
| "PoolFormerForSequenceClassification", | ||
| "PoolFormerForTokenClassification", | ||
| "PoolFormerLayer", | ||
| "PoolFormerModel", | ||
| "PoolFormerPreTrainedModel", | ||
| "load_tf_weights_in_poolformer", | ||
| ] | ||
| ) |
There was a problem hiding this comment.
Suggested change
| _import_structure["models.poolformer"].extend( | |
| [ | |
| "POOLFORMER_PRETRAINED_MODEL_ARCHIVE_LIST", | |
| "PoolFormerForMaskedLM", | |
| "PoolFormerForCausalLM", | |
| "PoolFormerForMultipleChoice", | |
| "PoolFormerForQuestionAnswering", | |
| "PoolFormerForSequenceClassification", | |
| "PoolFormerForTokenClassification", | |
| "PoolFormerLayer", | |
| "PoolFormerModel", | |
| "PoolFormerPreTrainedModel", | |
| "load_tf_weights_in_poolformer", | |
| ] | |
| ) | |
| _import_structure["models.poolformer"].extend( | |
| [ | |
| "POOLFORMER_PRETRAINED_MODEL_ARCHIVE_LIST", | |
| "PoolFormerLayer", | |
| "PoolFormerModel", | |
| "PoolFormerPreTrainedModel", | |
| "PoolFormerForImageClassification", | |
| ] | |
| ) |
Not sure if PoolFormerLayer is present.
NielsRogge
reviewed
Jan 21, 2022
src/transformers/__init__.py
Outdated
| @@ -2213,6 +2231,7 @@ | |||
| load_tf2_weights_in_pytorch_model, | |||
| ) | |||
| from .models.albert import ALBERT_PRETRAINED_CONFIG_ARCHIVE_MAP, AlbertConfig | |||
| from .models.poolformer import POOLFORMER_PRETRAINED_CONFIG_ARCHIVE_MAP, PoolFormerConfig, PoolFormerTokenizer | |||
There was a problem hiding this comment.
Suggested change
| from .models.poolformer import POOLFORMER_PRETRAINED_CONFIG_ARCHIVE_MAP, PoolFormerConfig, PoolFormerTokenizer | |
| from .models.poolformer import POOLFORMER_PRETRAINED_CONFIG_ARCHIVE_MAP, PoolFormerConfig |
NielsRogge
reviewed
Jan 21, 2022
src/transformers/__init__.py
Outdated
| @@ -2443,6 +2462,7 @@ | |||
| from .utils.dummy_sentencepiece_objects import * | |||
|
|
|||
| if is_tokenizers_available(): | |||
| from .models.poolformer import PoolFormerTokenizerFast | |||
There was a problem hiding this comment.
Suggested change
| from .models.poolformer import PoolFormerTokenizerFast |
NielsRogge
reviewed
Jan 21, 2022
src/transformers/__init__.py
Outdated
Comment on lines
2587
to
2599
| from .models.poolformer import ( | ||
| POOLFORMER_PRETRAINED_MODEL_ARCHIVE_LIST, | ||
| PoolFormerForMaskedLM, | ||
| PoolFormerForCausalLM, | ||
| PoolFormerForMultipleChoice, | ||
| PoolFormerForQuestionAnswering, | ||
| PoolFormerForSequenceClassification, | ||
| PoolFormerForTokenClassification, | ||
| PoolFormerLayer, | ||
| PoolFormerModel, | ||
| PoolFormerPreTrainedModel, | ||
| load_tf_weights_in_poolformer, | ||
| ) |
There was a problem hiding this comment.
Suggested change
| from .models.poolformer import ( | |
| POOLFORMER_PRETRAINED_MODEL_ARCHIVE_LIST, | |
| PoolFormerForMaskedLM, | |
| PoolFormerForCausalLM, | |
| PoolFormerForMultipleChoice, | |
| PoolFormerForQuestionAnswering, | |
| PoolFormerForSequenceClassification, | |
| PoolFormerForTokenClassification, | |
| PoolFormerLayer, | |
| PoolFormerModel, | |
| PoolFormerPreTrainedModel, | |
| load_tf_weights_in_poolformer, | |
| ) | |
| from .models.poolformer import ( | |
| POOLFORMER_PRETRAINED_MODEL_ARCHIVE_LIST, | |
| PoolFormerLayer, | |
| PoolFormerModel, | |
| PoolFormerPreTrainedModel, | |
| PoolFormerForImageClassification, | |
| ) |
NielsRogge
reviewed
Jan 21, 2022
| @@ -150,6 +151,7 @@ | |||
| MODEL_WITH_LM_HEAD_MAPPING_NAMES = OrderedDict( | |||
| [ | |||
| # Model with LM heads mapping | |||
| ("poolformer", "PoolFormerForMaskedLM"), | |||
There was a problem hiding this comment.
Suggested change
| ("poolformer", "PoolFormerForMaskedLM"), |
NielsRogge
reviewed
Jan 21, 2022
| @@ -201,6 +203,7 @@ | |||
| MODEL_FOR_CAUSAL_LM_MAPPING_NAMES = OrderedDict( | |||
| [ | |||
| # Model for Causal LM mapping | |||
| ("poolformer", "PoolFormerForCausalLM"), | |||
There was a problem hiding this comment.
Suggested change
| ("poolformer", "PoolFormerForCausalLM"), |
NielsRogge
reviewed
Jan 21, 2022
| @@ -277,6 +280,7 @@ | |||
| MODEL_FOR_MASKED_LM_MAPPING_NAMES = OrderedDict( | |||
| [ | |||
| # Model for Masked LM mapping | |||
| ("poolformer", "PoolFormerForMaskedLM"), | |||
There was a problem hiding this comment.
Suggested change
| ("poolformer", "PoolFormerForMaskedLM"), |
NielsRogge
reviewed
Jan 21, 2022
| @@ -349,6 +353,7 @@ | |||
| MODEL_FOR_SEQUENCE_CLASSIFICATION_MAPPING_NAMES = OrderedDict( | |||
| [ | |||
| # Model for Sequence Classification mapping | |||
| ("poolformer", "PoolFormerForSequenceClassification"), | |||
There was a problem hiding this comment.
Suggested change
| ("poolformer", "PoolFormerForSequenceClassification"), |
NielsRogge
reviewed
Jan 21, 2022
| @@ -396,6 +401,7 @@ | |||
| MODEL_FOR_QUESTION_ANSWERING_MAPPING_NAMES = OrderedDict( | |||
| [ | |||
| # Model for Question Answering mapping | |||
| ("poolformer", "PoolFormerForQuestionAnswering"), | |||
There was a problem hiding this comment.
Suggested change
| ("poolformer", "PoolFormerForQuestionAnswering"), |
NielsRogge
reviewed
Jan 21, 2022
| @@ -444,6 +450,7 @@ | |||
| MODEL_FOR_TOKEN_CLASSIFICATION_MAPPING_NAMES = OrderedDict( | |||
| [ | |||
| # Model for Token Classification mapping | |||
| ("poolformer", "PoolFormerForTokenClassification"), | |||
There was a problem hiding this comment.
Suggested change
| ("poolformer", "PoolFormerForTokenClassification"), |
NielsRogge
reviewed
Jan 21, 2022
| @@ -479,6 +486,7 @@ | |||
| MODEL_FOR_MULTIPLE_CHOICE_MAPPING_NAMES = OrderedDict( | |||
| [ | |||
| # Model for Multiple Choice mapping | |||
| ("poolformer", "PoolFormerForMultipleChoice"), | |||
There was a problem hiding this comment.
Suggested change
| ("poolformer", "PoolFormerForMultipleChoice"), |
You can add PoolFormerForImageClassification to the MODEL_FOR_IMAGE_CLASSIFICATION_MAPPING_NAMES.
NielsRogge
reviewed
Jan 21, 2022
| # There's no way to ignore "F401 '...' imported but unused" warnings in this | ||
| # module, but to preserve other warnings. So, don't check this module at all. | ||
|
|
||
| # Copyright 2020 The HuggingFace Team. All rights reserved. |
There was a problem hiding this comment.
Suggested change
| # Copyright 2020 The HuggingFace Team. All rights reserved. | |
| # Copyright 2022 The HuggingFace Team. All rights reserved. |
* Adding support for `microphone` streaming within pipeline.
- Uses `ffmpeg` to get microphone data.
- Makes sure alignment is made to `size_of_sample`.
- Works by sending `{"raw": ..data.., "stride": (n, left, right),
"partial": bool}`
directly to the pipeline enabling to stream partial results and still
get inference.
- Let's `partial` information flow through the pipeline to enable caller
to get it back and choose to display text or not.
- The striding reconstitution is bound to have errors since CTC does not
keep previous state. Currently most of the errors are we don't know if
there's a space or not between two chunks.
Since we have some left striding info, we could use that during decoding
to choose what to do with those spaces and even extra letters maybe (if
the stride is long enough, it's bound to cover at least a few symbols)
Fixing tests.
Protecting with `require_torch`.
`raw_ctc` support for nicer demo.
Post rebase fixes.
Revamp to split raw_mic_data from it's live chunking.
- Requires a refactor to make everything a bit cleaner.
Automatic resampling.
Small fix.
Small fix.
* Post rebase fix (need to let super handle more logic, reorder args.)
* Update docstrings
* Docstring format.
* Remove print.
* Prevent flow of `input_values`.
* Fixing `stride` too.
* Fixing the PR by removing `raw_ctc`.
* Better docstrings.
* Fixing init.
* Update src/transformers/pipelines/audio_utils.py
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
* Update tests/test_pipelines_automatic_speech_recognition.py
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
* Quality.
Co-authored-by: Anton Lozhkov <aglozhkov@gmail.com>
* Allow dynamic modules to use relative imports * Work for configs * Fix last merge conflict * Save code of registered custom objects * Map strings to strings * Fix test * Add tokenizer * Rework tests * Tests * Ignore fixtures py files for tests * Tokenizer test + fix collection * With full path * Rework integration * Fix typo * Remove changes in conftest * Test for tokenizers * Add documentation * Update docs/source/custom_models.mdx Co-authored-by: Lysandre Debut <lysandre@huggingface.co> * Add file structure and file content * Add more doc * Style * Update docs/source/custom_models.mdx Co-authored-by: Suraj Patil <surajp815@gmail.com> * Address review comments Co-authored-by: Lysandre Debut <lysandre@huggingface.co> Co-authored-by: Suraj Patil <surajp815@gmail.com>
# Add support for W&B hyperparameter sweep
This PR:
* allows using wandb for running hyperparameter search.
* The runs are visualized on W&B sweeps dashboard
* This supports runnning sweeps on parallel devices, all reporting to the same central dashboard.
### Usage
**To run new a hyperparameter search:**
```
trainer.hyperparameter_search(
backend="wandb",
project="transformers_sweep", # name of the project
n_trials=5,
metric="eval/loss", # metric to be optimized, default 'eval/loss'. A warning is raised if the passed metric is not found
)
```
This outputs a sweep id. Eg. `my_project/sweep_id`
**To run sweeps on parallel devices:**
Just pass sweep id which you want to run parallel
```
trainer.hyperparameter_search(
backend="wandb",
sweep_id = "my_project/sweep_id"
)
```
…15482) * Playing * Properly set labels in model config for token classification example * Port to run_ner_no_trainer * Quality
…e` (#15456) * change truncation_side in init of `PreTrainedTokenizerBase` Co-authored-by: LSinev <LSinev@users.noreply.github.com> * add test * Revert "replace assert with exception for `padding_side` arg in `PreTrainedTokenizerBase` `__init__`" This reverts commit 7a98b87. * fix kwargs * Revert "fix kwargs" This reverts commit 67b0a52. * Update tests/test_tokenization_common.py Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com> * delete truncation_side variable * reorganize test * format * complete doc * Revert "Revert "replace assert with exception for `padding_side` arg in `PreTrainedTokenizerBase` `__init__`"" This reverts commit d5a10a7. * fix typo * fix typos to render documentation * Revert "Revert "Revert "replace assert with exception for `padding_side` arg in `PreTrainedTokenizerBase` `__init__`""" This reverts commit 16cf588. * format Co-authored-by: LSinev <LSinev@users.noreply.github.com> Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>
* Correct eos_token_id set in generate * Set eos_token_id in test * Correct eos_token_id set in generate * Set eos_token_id in test
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Add general docstrings * Remove legacy docstrings * Add BEiT * Add DEiT * Add SegFormer * Fix beit output class * Fix missing return_dict
* [deepspeed] fix a bug in a test * consistency
* Add preprocess_logits_for_metrics Trainer param * Compute accuracy in LM examples * Improve comments
* Remove return_loss from Flax models * fix more * fix Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR adds the PoolFormer model to the 🤗 repository.
I also opened an Issue for adding the model #14584
Who can review?
@NielsRogge