Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tweak tests #410

Merged
merged 1 commit into from
May 11, 2022
Merged

Tweak tests #410

merged 1 commit into from
May 11, 2022

Conversation

benfred
Copy link
Member

@benfred benfred commented May 10, 2022

Report skipped tests during unitests, and change one skipped test to a
xfail

Report skipped tests during unitests, and change one skipped test to a
xfail
@github-actions
Copy link

@nvidia-merlin-bot
Copy link

Click to view CI Results
GitHub pull request #410 of commit a61299113af56caeab9ec9b9ff2fa5cd674cf23a, no merge conflicts.
Running as SYSTEM
Setting status of a61299113af56caeab9ec9b9ff2fa5cd674cf23a to PENDING with url http://10.20.13.93:8080/job/transformers4rec_tests/112/ and message: 'Pending'
Using context: Jenkins Unit Test Run
Building on master in workspace /var/jenkins_home/workspace/transformers4rec_tests
using credential nvidia-merlin-bot
Cloning the remote Git repository
Cloning repository https://github.com/NVIDIA-Merlin/Transformers4Rec.git
 > git init /var/jenkins_home/workspace/transformers4rec_tests/transformers4rec # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/Transformers4Rec.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Transformers4Rec.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/Transformers4Rec.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/Transformers4Rec.git # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/Transformers4Rec.git
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/Transformers4Rec.git +refs/pull/410/*:refs/remotes/origin/pr/410/* # timeout=10
 > git rev-parse a61299113af56caeab9ec9b9ff2fa5cd674cf23a^{commit} # timeout=10
Checking out Revision a61299113af56caeab9ec9b9ff2fa5cd674cf23a (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f a61299113af56caeab9ec9b9ff2fa5cd674cf23a # timeout=10
Commit message: "Tweak tests"
 > git rev-list --no-walk 11540230a6fa73ba04ea4238a5faacbeeb5f6fa3 # timeout=10
First time build. Skipping changelog.
[transformers4rec_tests] $ /bin/bash /tmp/jenkins3755971308863208368.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: pip in /var/jenkins_home/.local/lib/python3.8/site-packages (22.0.4)
rm: cannot remove '/usr/lib/python3.8/site-packages/nvtabular.egg-link': No such file or directory
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: pyarrow in /usr/local/lib/python3.8/dist-packages (5.0.0)
Requirement already satisfied: numpy>=1.16.6 in /var/jenkins_home/.local/lib/python3.8/site-packages (from pyarrow) (1.20.3)
Installing models
Found existing installation: nvtabular 1.0.0+10.g4df99eb4
Can't uninstall 'nvtabular'. No files were found to uninstall.
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/NVIDIA-Merlin/NVTabular.git
  Cloning https://github.com/NVIDIA-Merlin/NVTabular.git to /tmp/pip-req-build-pxi7q6lz
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/NVTabular.git /tmp/pip-req-build-pxi7q6lz
  Resolved https://github.com/NVIDIA-Merlin/NVTabular.git to commit abadde07f0198d844bfe73a52176865e8bb7709f
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Building wheels for collected packages: nvtabular
  Building wheel for nvtabular (pyproject.toml): started
  Building wheel for nvtabular (pyproject.toml): finished with status 'done'
  Created wheel for nvtabular: filename=nvtabular-1.1.1-cp38-cp38-linux_x86_64.whl size=400464 sha256=8566919b6f2ba47914a1b6e8f45b764cef30c38288c9a875acc74afa75d37ad9
  Stored in directory: /tmp/pip-ephem-wheel-cache-j1jho9og/wheels/c2/16/76/39994bff39d812513de5b5572bff0903b9eb8f6c645b44cedc
Successfully built nvtabular
Installing collected packages: nvtabular
Successfully installed nvtabular-1.1.1
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Processing /var/jenkins_home/workspace/transformers4rec_tests/transformers4rec
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Building wheels for collected packages: transformers4rec
  Building wheel for transformers4rec (pyproject.toml): started
  Building wheel for transformers4rec (pyproject.toml): finished with status 'done'
  Created wheel for transformers4rec: filename=transformers4rec-0.1.8+1.ga6129911-py3-none-any.whl size=437233 sha256=cf993f094f59f9721204dd28a55e91b1f8e444aea694f6ac37ecbe3d80af1637
  Stored in directory: /tmp/pip-ephem-wheel-cache-6qkxddwm/wheels/cb/5d/b4/e081835ae498194a418e957657f998bdff0fa2bd103855a861
Successfully built transformers4rec
Installing collected packages: transformers4rec
Successfully installed transformers4rec-0.1.8+1.ga6129911
Running black --check
All done! ✨ 🍰 ✨
167 files would be left unchanged.
Running flake8
Running isort
Skipped 1 files
Running bandit
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/transformers4rec_tests/transformers4rec
plugins: anyio-3.5.0, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 478 items / 1 skipped

tests/tf/test_masking.py ................... [ 3%]
tests/tf/test_public_api.py ....... [ 5%]
tests/tf/block/test_base.py .. [ 5%]
tests/tf/block/test_dlrm.py . [ 6%]
tests/tf/block/test_mlp.py ........................ [ 11%]
tests/tf/block/test_transformer.py .................. [ 14%]
tests/tf/features/test_continuous.py ..... [ 15%]
tests/tf/features/test_embedding.py ........ [ 17%]
tests/tf/features/test_sequence.py ............ [ 20%]
tests/tf/features/test_tabular.py ....... [ 21%]
tests/tf/model/test_head.py ................................ [ 28%]
tests/tf/model/test_model.py ............................... [ 34%]
tests/tf/tabular/test_aggregation.py ......... [ 36%]
tests/tf/tabular/test_tabular.py ......... [ 38%]
tests/tf/tabular/test_transformations.py XXXXXXXXXXXXX [ 41%]
tests/tf/utils/test_schema_utils.py .. [ 41%]
tests/torch/test_dataloader_utils.py ... [ 42%]
tests/torch/test_losses.py ... [ 42%]
tests/torch/test_masking.py ............................................ [ 52%]
........... [ 54%]
tests/torch/test_public_api.py ....... [ 55%]
tests/torch/test_ranking_metrics.py .................................... [ 63%]
.... [ 64%]
tests/torch/test_readme.py . [ 64%]
tests/torch/test_trainer.py .............. [ 67%]
tests/torch/block/test_base.py .... [ 68%]
tests/torch/block/test_mlp.py . [ 68%]
tests/torch/block/test_transformer.py .................. [ 72%]
tests/torch/features/test_continuous.py .. [ 72%]
tests/torch/features/test_embedding.py .............. [ 75%]
tests/torch/features/test_sequential.py ......... [ 77%]
tests/torch/features/test_tabular.py .... [ 78%]
tests/torch/model/test_head.py ....................................... [ 86%]
tests/torch/model/test_model.py .......................... [ 91%]
tests/torch/tabular/test_aggregation.py ......... [ 93%]
tests/torch/tabular/test_tabular.py ..... [ 94%]
tests/torch/tabular/test_transformations.py ....................... [ 99%]
tests/torch/utils/test_schema_utils.py .. [100%]

=============================== warnings summary ===============================
tests/tf/model/test_model.py::test_save_and_load_transformer_model[LongformerConfig-True]
tests/tf/model/test_model.py::test_save_and_load_transformer_model[LongformerConfig-False]
tests/tf/model/test_model.py::test_save_and_load_transformer_model[BertConfig-True]
tests/tf/model/test_model.py::test_save_and_load_transformer_model[BertConfig-False]
tests/tf/model/test_model.py::test_save_and_load_transformer_model[RobertaConfig-True]
tests/tf/model/test_model.py::test_save_and_load_transformer_model[RobertaConfig-False]
tests/tf/model/test_model.py::test_save_and_load_transformer_model[AlbertConfig-True]
tests/tf/model/test_model.py::test_save_and_load_transformer_model[AlbertConfig-False]
/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/generic_utils.py:494: CustomMaskWarning: Custom mask layers require a config and must override get_config. When loading, the custom mask layer must be passed to the custom_objects argument.
warnings.warn('Custom mask layers require a config and must override '

tests/tf/tabular/test_transformations.py::test_stochastic_swap_noise[0.1]
tests/tf/tabular/test_transformations.py::test_stochastic_swap_noise[0.3]
tests/tf/tabular/test_transformations.py::test_stochastic_swap_noise[0.5]
tests/tf/tabular/test_transformations.py::test_stochastic_swap_noise[0.7]
tests/tf/tabular/test_transformations.py::test_stochastic_swap_noise_with_tabular_features[0.1]
tests/tf/tabular/test_transformations.py::test_stochastic_swap_noise_with_tabular_features[0.3]
tests/tf/tabular/test_transformations.py::test_stochastic_swap_noise_with_tabular_features[0.5]
tests/tf/tabular/test_transformations.py::test_stochastic_swap_noise_with_tabular_features[0.7]
/var/jenkins_home/.local/lib/python3.8/site-packages/keras/backend.py:6089: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.
warnings.warn('tf.keras.backend.random_binomial is deprecated, '

tests/torch/test_trainer.py::test_set_train_eval_loaders_attributes[16]
tests/torch/test_trainer.py::test_set_train_eval_loaders_attributes[16]
/usr/local/lib/python3.8/dist-packages/torch/utils/tensorboard/init.py:6: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
if not hasattr(tensorboard, 'version') or LooseVersion(tensorboard.version) < LooseVersion('1.15'):

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

---------- coverage: platform linux, python 3.8.10-final-0 -----------
Name Stmts Miss Branch BrPart Cover Missing

merlin_standard_lib/init.py 9 0 0 0 100%
merlin_standard_lib/proto/init.py 0 0 0 0 100%
merlin_standard_lib/proto/schema_bp.py 271 4 98 0 98% 236-240
merlin_standard_lib/registry.py 102 28 40 7 68% 33, 51, 55, 135, 139, 144, 149, 165, 171, 173, 223, 232, 246, 249, 252, 255, 258, 261, 287-296
merlin_standard_lib/schema/init.py 0 0 0 0 100%
merlin_standard_lib/schema/schema.py 276 114 150 9 55% 26-30, 57-60, 105-115, 127, 135, 142-158, 161-170, 173-175, 183-188, 191-194, 197, 200-203, 224-239, 242-250, 253-256, 259-264, 275-285, 288-298, 314-320, 349-353, 358-363, 397-399, 402-404, 411, 413, 417-421, 430-431, 449, 485, 495, 498-510
merlin_standard_lib/schema/tag.py 25 0 2 0 100%
merlin_standard_lib/utils/init.py 0 0 0 0 100%
merlin_standard_lib/utils/doc_utils.py 10 0 6 0 100%
merlin_standard_lib/utils/embedding_utils.py 8 0 2 0 100%
merlin_standard_lib/utils/misc_utils.py 118 91 76 1 18% 31, 45-53, 57-58, 65-74, 81-89, 93-116, 123-132, 136-143, 146-147, 150, 159-175, 198-239
merlin_standard_lib/utils/proto_utils.py 31 15 8 1 44% 34, 42-44, 50-66
transformers4rec/init.py 2 0 0 0 100%
transformers4rec/_version.py 337 196 190 27 34% 89-100, 103-106, 117-137, 147-165, 172, 174->186, 189, 191-230, 249, 253-255, 266, 270, 281, 284->308, 291, 295->299, 301, 306, 318, 327-328, 333-337, 348-350, 365, 380->390, 384-389, 402-419, 428-435, 448-462, 473-491, 502-513, 524-533, 545-553, 559, 568, 572-587, 620-621, 632-641
transformers4rec/config/init.py 0 0 0 0 100%
transformers4rec/config/schema.py 36 0 12 0 100%
transformers4rec/config/trainer.py 22 0 6 0 100%
transformers4rec/config/transformer.py 81 7 32 5 89% 23-24, 47, 49, 66->103, 80, 82, 249
transformers4rec/data/init.py 3 0 0 0 100%
transformers4rec/data/dataset.py 22 1 6 1 93% 28
transformers4rec/data/preprocessing.py 134 134 40 0 0% 18-379
transformers4rec/data/synthetic.py 37 37 10 0 0% 1-168
transformers4rec/data/testing/init.py 0 0 0 0 100%
transformers4rec/data/testing/dataset.py 3 0 0 0 100%
transformers4rec/data/testing/tabular_data/init.py 0 0 0 0 100%
transformers4rec/data/testing/tabular_data/dataset.py 3 0 0 0 100%
transformers4rec/data/yoochoose.py 18 18 2 0 0% 18-88
transformers4rec/tf/init.py 35 0 0 0 100%
transformers4rec/tf/block/init.py 0 0 0 0 100%
transformers4rec/tf/block/base.py 135 19 56 3 84% 30-42, 45-50, 86, 94-96, 107-110, 187
transformers4rec/tf/block/dlrm.py 39 5 14 4 83% 49, 56, 67-68, 92
transformers4rec/tf/block/mlp.py 18 1 12 1 93% 47
transformers4rec/tf/block/transformer.py 69 13 32 7 80% 38-40, 78-79, 81, 92, 101, 103, 148-155, 171, 175
transformers4rec/tf/features/init.py 0 0 0 0 100%
transformers4rec/tf/features/base.py 3 0 2 0 100%
transformers4rec/tf/features/continuous.py 33 3 6 0 92% 84, 87, 90
transformers4rec/tf/features/embedding.py 146 13 64 13 87% 83, 153, 165->163, 212, 217, 232, 248->250, 260, 263->265, 277->279, 279->282, 286->288, 289, 296-303
transformers4rec/tf/features/sequence.py 115 3 38 4 95% 103, 244->247, 302, 309
transformers4rec/tf/features/tabular.py 73 5 24 9 86% 84->86, 87, 118->121, 147->152, 152->158, 159, 183, 190, 195
transformers4rec/tf/features/text.py 54 42 28 0 20% 25-26, 29-47, 50-64, 77-80, 83-93, 96-110
transformers4rec/tf/layers/init.py 3 0 0 0 100%
transformers4rec/tf/layers/interaction.py 47 16 22 6 57% 49, 59-71, 79-84, 91, 96->98, 107, 113
transformers4rec/tf/layers/outer_product.py 30 24 12 0 19% 37-38, 41-60, 71-84, 87
transformers4rec/tf/masking.py 126 0 14 0 100%
transformers4rec/tf/model/init.py 0 0 0 0 100%
transformers4rec/tf/model/base.py 295 47 142 24 80% 105->108, 122, 141->143, 145, 156, 159, 163, 166, 176, 179, 182-184, 192-193, 229, 255->261, 263-264, 279-296, 318-324, 333, 343, 350-355, 358-362, 378->381, 419-420, 450, 504, 506, 524, 554->557, 576
transformers4rec/tf/model/prediction_task.py 160 21 70 15 83% 14, 156, 161->163, 163->176, 190, 231, 234, 257, 263, 266-273, 283, 286, 289-291, 354, 380, 382->387, 391
transformers4rec/tf/ranking_metric.py 151 80 36 0 48% 75-77, 82-90, 94, 119-121, 137-153, 171-205, 225-250, 270-296, 320-336, 340-351
transformers4rec/tf/tabular/init.py 0 0 0 0 100%
transformers4rec/tf/tabular/aggregation.py 101 9 30 1 91% 86-90, 145, 156-160
transformers4rec/tf/tabular/base.py 279 19 122 13 91% 86, 98, 134, 136, 140, 252-254, 309->311, 422->430, 424, 433-437, 476, 478, 558-559, 600->exit, 607, 638, 658
transformers4rec/tf/tabular/transformations.py 70 18 26 3 74% 35-37, 51-60, 63, 95, 102->106, 136, 139-144
transformers4rec/tf/typing.py 7 0 0 0 100%
transformers4rec/tf/utils/init.py 0 0 0 0 100%
transformers4rec/tf/utils/repr_utils.py 67 40 30 5 37% 21-29, 33-48, 52-65, 73, 76->91, 83, 88-90, 97-100, 112, 116
transformers4rec/tf/utils/schema_utils.py 75 2 34 1 97% 113-114
transformers4rec/tf/utils/testing_utils.py 28 1 2 1 93% 30
transformers4rec/tf/utils/tf_utils.py 62 22 42 1 66% 97-112, 138, 169-172, 176, 180, 185-195
transformers4rec/torch/init.py 21 0 0 0 100%
transformers4rec/torch/block/init.py 0 0 0 0 100%
transformers4rec/torch/block/base.py 173 24 91 10 83% 35-47, 50-55, 73, 97-100, 103, 113->exit, 136, 168-173, 226->231, 228->231, 248->256, 250, 253->255, 272
transformers4rec/torch/block/mlp.py 38 5 16 4 80% 36, 79->81, 82-83, 85, 92
transformers4rec/torch/block/transformer.py 70 5 34 1 92% 85-89, 185
transformers4rec/torch/features/init.py 0 0 0 0 100%
transformers4rec/torch/features/base.py 4 0 2 0 100%
transformers4rec/torch/features/continuous.py 22 0 6 0 100%
transformers4rec/torch/features/embedding.py 162 10 74 13 89% 79->76, 96->98, 198, 202->212, 222-227, 289->294, 348->351, 359->357, 380, 384->393, 409, 412, 415
transformers4rec/torch/features/sequence.py 96 4 38 3 93% 79->81, 241, 245-248, 267->270
transformers4rec/torch/features/tabular.py 66 6 28 8 83% 78->80, 81, 103->106, 159->170, 170->175, 182->192, 194, 210-213, 220
transformers4rec/torch/features/text.py 0 0 0 0 100%
transformers4rec/torch/losses.py 28 5 10 3 74% 45, 50, 53-56
transformers4rec/torch/masking.py 202 4 38 2 98% 201, 206, 451, 502
transformers4rec/torch/model/init.py 0 0 0 0 100%
transformers4rec/torch/model/base.py 292 48 144 20 81% 131->137, 133, 138-140, 152->155, 191, 267->273, 275-276, 290->exit, 299, 334-350, 371-375, 504, 505->510, 567-590, 603, 607->610, 616->618, 620-621, 627->618, 631->635, 634, 641, 646->648, 655-658
transformers4rec/torch/model/prediction_task.py 134 7 64 8 92% 170, 178, 183->185, 199->202, 224, 262, 268-269, 377->382, 387
transformers4rec/torch/ranking_metric.py 106 0 28 1 99% 132->142
transformers4rec/torch/tabular/init.py 0 0 0 0 100%
transformers4rec/torch/tabular/aggregation.py 88 3 32 3 95% 84, 95, 112
transformers4rec/torch/tabular/base.py 244 23 114 10 89% 73, 85, 124, 126, 130, 211-213, 330->332, 450-451, 490-496, 500, 529, 539, 567->exit, 572, 598-603
transformers4rec/torch/tabular/transformations.py 85 1 40 3 97% 50, 88->91, 138->142
transformers4rec/torch/trainer.py 266 70 100 30 68% 111-113, 164, 203->exit, 252, 259, 294-295, 302, 360->363, 364, 375, 401->405, 410, 419->426, 420->426, 426->431, 431->439, 439->466, 476-502, 506, 509->514, 514->519, 519->531, 536->538, 538->540, 540->544, 546->554, 577-578, 590-594, 610-636, 640, 644, 681, 693, 696-700, 713-723, 730-731, 748, 751, 755, 760, 773
transformers4rec/torch/typing.py 6 0 0 0 100%
transformers4rec/torch/utils/init.py 0 0 0 0 100%
transformers4rec/torch/utils/data_utils.py 114 31 38 6 73% 56->173, 108, 115, 173->331, 188-190, 194->exit, 219-255, 258-265, 268->exit, 300-328, 360-362, 366-368, 372
transformers4rec/torch/utils/examples_utils.py 44 44 16 0 0% 1-103
transformers4rec/torch/utils/schema_utils.py 74 2 34 1 97% 108-109
transformers4rec/torch/utils/torch_utils.py 109 11 44 10 85% 131-133, 156, 172, 175, 178, 181, 238, 240, 242, 246, 249->251
transformers4rec/types.py 0 0 0 0 100%
transformers4rec/utils/init.py 0 0 0 0 100%
transformers4rec/utils/dependencies.py 20 10 0 0 50% 19-23, 30-32, 39-40
versioneer.py 643 643 300 0 0% 3-2088

TOTAL 6876 2004 2829 298 68%
Coverage XML written to file coverage.xml

===== 465 passed, 1 skipped, 13 xpassed, 18 warnings in 1079.86s (0:17:59) =====
Performing Post build task...
Match found for : : True
Logical operation result is TRUE
Running script : #!/bin/bash
cd /var/jenkins_home/
CUDA_VISIBLE_DEVICES=2 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/Transformers4Rec/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"
[transformers4rec_tests] $ /bin/bash /tmp/jenkins6540019070906430261.sh

@benfred benfred merged commit 180da13 into main May 11, 2022
@benfred benfred added the skip-changelog Exclude from change log label Jun 9, 2022
@benfred benfred deleted the tweak_tests branch June 9, 2022 17:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
skip-changelog Exclude from change log
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants