Add Slack notification support for doc tests #16253
Conversation
…e/transformers into add_doc_tests_reports
patrickvonplaten
changed the title
|
The documentation is not available anymore as the PR was closed or merged. |
patrickvonplaten
requested review from
sgugger,
LysandreJik,
ydshieh and
patil-suraj
docs/source/quicktour.mdx
Outdated
| <tf.Tensor: shape=(2, 5), dtype=float32, numpy= | ||
| array([[0.00205667, 0.00176607, 0.01154934, 0.21209256, 0.7725354 ], | ||
| [0.20841935, 0.1826222 , 0.19692972, 0.17550416, 0.2365246 ]], | ||
| dtype=float32)> |
There was a problem hiding this comment.
This is super flaky and won't pass on any other machine or at the new TF release. We need to test at a lesser precision or ignore the output.
There was a problem hiding this comment.
Yeah true! Still don't understand why my local pytest or python -m pytest doesn't pick up the precision flag here...Will adapt!
There was a problem hiding this comment.
I tried to fight with it too on a branch with little success. I'd advocate to not lose sleep on it and just ignore the output since we can't get a result that passes both locally for me and on the runner :-)
| n_blanks_left = 25 | ||
| n_blanks_right = 15 | ||
| category_failures = {k: len(v["failed"]) for k, v in doc_test_results.items() if isinstance(v, dict)} | ||
| category_failures_report = [f"| {category}".ljust(n_blanks_left, " ") + f"| {num}".ljust(n_blanks_right, " ") + "|" for category, num in category_failures.items()] |
|
I will look this later today. Still learn this kind of things. If it is urgent, don't hesitate to go ahead though. |
utils/documentation_tests.txt
Outdated
| @@ -31,6 +31,6 @@ src/transformers/models/blenderbot/modeling_blenderbot.py | |||
| src/transformers/models/blenderbot_small/modeling_blenderbot_small.py | |||
| src/transformers/models/plbart/modeling_plbart.py | |||
| src/transformers/generation_utils.py | |||
| src/transformers/models/resnet/modeling_resnet.py | |||
There was a problem hiding this comment.
I kind get the idea. If I understand it correctly, here are my 2 cents:
- maybe move this line before
src/transformers/generation_utils.py? - maybe in a future PR, we can have a script to sort the lines by some order (say, alphabetic)
There was a problem hiding this comment.
Alphabetic sort is a very good idea!
.github/workflows/doctests.yml
Outdated
| - "docs/**" | ||
| types: [assigned, opened, synchronize, reopened] | ||
| # schedule: | ||
| # - cron: "0 0 * * *" |
There was a problem hiding this comment.
Guess this is temporary (to check this PR works), and will be changed back to schedule at the end?
|
LGTM (without looking the notification script in detail). Just 2 questions (due to my lack of expertise):
with:
repository: 'huggingface/transformers'
path: transformers(just wondering why we don't need this) |
|
Very good question! And I don't really know the answer :D I just noticed that it didn't work with with:
repository: 'huggingface/transformers'
path: transformersvery well and I also think it's not necessary here so I removed it. Maybe @LysandreJik can give you a better answer. |
* up * up * up * fix * yeh * ups * Empty test commit * correct quicktour * correct * correct * up * up * uP * uP * up * up * uP * up * up * up * up * up * up * up * up * up * up * Update src/transformers/models/van/modeling_van.py * finish * apply suggestions * remove folder * revert to daily testing
What does this PR do?
This PR enables Slack reports for the daily doc tests.