Skip to content

Commit

Permalink
Docs - Guide to add a new TensorFlow model (#19256)
Browse files Browse the repository at this point in the history
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>
  • Loading branch information
3 people committed Sep 30, 2022
1 parent 6a08162 commit cfb777f
Show file tree
Hide file tree
Showing 4 changed files with 359 additions and 9 deletions.
6 changes: 4 additions & 2 deletions docs/source/en/_toctree.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
- local: sagemaker
title: Run training on Amazon SageMaker
- local: converting_tensorflow_models
title: Converting TensorFlow Checkpoints
title: Converting from TensorFlow checkpoints
- local: serialization
title: Export 🤗 Transformers models
- local: troubleshooting
Expand Down Expand Up @@ -109,6 +109,8 @@
title: How to contribute to transformers?
- local: add_new_model
title: How to add a model to 🤗 Transformers?
- local: add_tensorflow_model
title: How to convert a 🤗 Transformers model to TensorFlow?
- local: add_new_pipeline
title: How to add a pipeline to 🤗 Transformers?
- local: testing
Expand Down Expand Up @@ -511,4 +513,4 @@
- local: internal/file_utils
title: General Utilities
title: Internal Helpers
title: API
title: API
14 changes: 8 additions & 6 deletions docs/source/en/add_new_model.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ own regarding how code should be written :-)
for a good example).
2. The code should be fully understandable, even by a non-native English speaker. This means you should pick
descriptive variable names and avoid abbreviations. As an example, `activation` is preferred to `act`.
One-letter variable names are strongly discouraged unless it's an index in a for loop.
One-letter variable names are strongly discouraged unless it's an index in a for loop.
3. More generally we prefer longer explicit code to short magical one.
4. Avoid subclassing `nn.Sequential` in PyTorch but subclass `nn.Module` and write the forward pass, so that anyone
using your code can quickly debug it by adding print statements or breaking points.
Expand Down Expand Up @@ -222,7 +222,7 @@ cd ..
5. To port *brand_new_bert*, you will also need access to its original repository:

```bash
git clone https://github.com/org_that_created_brand_new_bert_org/brand_new_bert.git
git clone https://github.com/org_that_created_brand_new_bert_org/brand_new_bert.git
cd brand_new_bert
pip install -e .
```
Expand Down Expand Up @@ -683,10 +683,11 @@ work left to be done should be a cakewalk 😊.
At this point, you have successfully added a new model. However, it is very much possible that the model does not yet
fully comply with the required design. To make sure, the implementation is fully compatible with 🤗 Transformers, all
common tests should pass. The Cookiecutter should have automatically added a test file for your model, probably under
the same `tests/test_modeling_brand_new_bert.py`. Run this test file to verify that all common tests pass:
the same `tests/models/brand_new_bert/test_modeling_brand_new_bert.py`. Run this test file to verify that all common
tests pass:

```bash
pytest tests/test_modeling_brand_new_bert.py
pytest tests/models/brand_new_bert/test_modeling_brand_new_bert.py
```

Having fixed all common tests, it is now crucial to ensure that all the nice work you have done is well tested, so that
Expand All @@ -700,7 +701,7 @@ Cookiecutter, called `BrandNewBertModelIntegrationTests` and only has to be fill
tests are passing, run

```bash
RUN_SLOW=1 pytest -sv tests/test_modeling_brand_new_bert.py::BrandNewBertModelIntegrationTests
RUN_SLOW=1 pytest -sv tests/models/brand_new_bert/test_modeling_brand_new_bert.py::BrandNewBertModelIntegrationTests
```

<Tip>
Expand Down Expand Up @@ -758,7 +759,8 @@ contain a couple of hard-coded integration tests.
**10. Run End-to-end integration tests**

Having added the tokenizer, you should also add a couple of end-to-end integration tests using both the model and the
tokenizer to `tests/test_modeling_brand_new_bert.py` in 🤗 Transformers. Such a test should show on a meaningful
tokenizer to `tests/models/brand_new_bert/test_modeling_brand_new_bert.py` in 🤗 Transformers.
Such a test should show on a meaningful
text-to-text sample that the 🤗 Transformers implementation works as expected. A meaningful text-to-text sample can
include *e.g.* a source-to-target-translation pair, an article-to-summary pair, a question-to-answer pair, etc… If none
of the ported checkpoints has been fine-tuned on a downstream task it is enough to simply rely on the model tests. In a
Expand Down

0 comments on commit cfb777f

Please sign in to comment.