Use shuffle_df from merlin-core by benfred · Pull Request #413 · NVIDIA-Merlin/models

benfred · 2022-05-04T22:14:02Z

The dataloader was including the shuffle code that is now hosted in merlin-core. This
change updates to use the merlin-core version, rather than redefine here.

The shuffle_df code had an issue with cudf 22.04 that is fixed in merlin-core by
NVIDIA-Merlin/core#82

The dataloader was including the shuffle code that is now hosted in merlin-core. This change updates to use the merlin-core version, rather than redefine here. The shuffle_df code had an issue with cudf 22.04 that is fixed in merlin-core by NVIDIA-Merlin/core#82

nvidia-merlin-bot · 2022-05-04T22:25:46Z

Click to view CI Results

GitHub pull request #413 of commit d6b4dc1d3355773848c9ac71db1827ca8f55239e, no merge conflicts.
Running as SYSTEM
Setting status of d6b4dc1d3355773848c9ac71db1827ca8f55239e to PENDING with url https://10.20.13.93:8080/job/merlin_models/239/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/413/*:refs/remotes/origin/pr/413/* # timeout=10
 > git rev-parse d6b4dc1d3355773848c9ac71db1827ca8f55239e^{commit} # timeout=10
Checking out Revision d6b4dc1d3355773848c9ac71db1827ca8f55239e (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f d6b4dc1d3355773848c9ac71db1827ca8f55239e # timeout=10
Commit message: "Use shuffle_df from merlin-core"
 > git rev-list --no-walk a77b6f21032e8bad5a8faf5f12eb235b9533f9ba # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins8584993502440324171.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.4.0)
Requirement already satisfied: traitlets>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (5.1.1)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.0)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.4.0)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.15.3)
Requirement already satisfied: jupyter-core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.10.0)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.7.1)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (21.4.0)
Requirement already satisfied: pyzmq>=22.3 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (22.3.0)
Requirement already satisfied: tornado>=6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.1)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.0)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 429 items / 2 skipped
tests/datasets/test_advertising.py .s tests/datasets/test_ecommerce.py ..sss tests/datasets/test_entertainment.py ....sss tests/datasets/test_social.py . tests/datasets/test_synthetic.py ..... tests/tf/test_core.py ................ tests/tf/test_dataset.py .............. tests/tf/test_public_api.py . tests/tf/blocks/test_cross.py ............ tests/tf/blocks/test_dlrm.py ........ tests/tf/blocks/test_interactions.py . tests/tf/blocks/test_mlp.py ............................. tests/tf/blocks/core/test_aggregation.py ......... tests/tf/blocks/core/test_base.py . tests/tf/blocks/core/test_index.py ... tests/tf/blocks/core/test_masking.py ....... tests/tf/blocks/core/test_transformations.py ........... tests/tf/blocks/retrieval/test_matrix_factorization.py tests/tf/blocks/retrieval/test_two_tower.py ........... tests/tf/examples/test_01_getting_started.py . tests/tf/examples/test_02_dataschema.py . tests/tf/examples/test_03_exploring_different_models.py tests/tf/examples/test_04_export_ranking_models.py . tests/tf/examples/test_05_export_retrieval_model.py . tests/tf/examples/test_06_advanced_own_architecture.py . tests/tf/features/test_continuous.py ..... tests/tf/features/test_embedding.py .......... tests/tf/features/test_tabular.py ....... tests/tf/layers/test_queue.py .............. tests/tf/losses/test_losses.py ....................... tests/tf/metrics/test_metrics_popularity.py ..... tests/tf/metrics/test_metrics_ranking.py tests/tf/models/test_benchmark.py . tests/tf/models/test_ranking.py ...................... tests/tf/models/test_retrieval.py .................. tests/tf/prediction_tasks/test_classification.py .. tests/tf/prediction_tasks/test_multi_task.py ....... tests/tf/prediction_tasks/test_next_item.py tests/tf/prediction_tasks/test_regression.py .. tests/tf/prediction_tasks/test_sampling.py tests/tf/utils/test_batch.py .... tests/torch/test_dataset.py ......... tests/torch/test_public_api.py . tests/torch/block/test_base.py .... tests/torch/block/test_mlp.py . tests/torch/features/test_continuous.py .. tests/torch/features/test_embedding.py .............. tests/torch/features/test_tabular.py .... tests/torch/model/test_head.py ............ tests/torch/model/test_model.py .. tests/torch/tabular/test_aggregation.py ........ tests/torch/tabular/test_tabular.py ... tests/torch/tabular/test_transformations.py ....... tests/utils/test_schema_utils.py ...........

[  0%]
 [  1%]
 [  3%]
 [  3%]
 [  4%]
 [  8%]
 [ 11%]
 [ 11%]
 [ 14%]
 [ 16%]
 [ 16%]
 [ 23%]
 [ 25%]
 [ 25%]
 [ 26%]
 [ 28%]
 [ 30%]
 ..                [ 31%]
 [ 33%]
 [ 34%]
 [ 34%]
 .                [ 34%]
 [ 34%]
 [ 34%]
 [ 35%]
 [ 36%]
 [ 38%]
 [ 40%]
 [ 43%]
 [ 48%]
 [ 50%]
 .................               [ 54%]
 [ 54%]
 [ 59%]
 [ 63%]
 [ 64%]
 [ 65%]
 ......................       [ 70%]
 [ 71%]
 ....................          [ 75%]
 [ 76%]
 [ 79%]
 [ 79%]
 [ 80%]
 [ 80%]
 [ 80%]
 [ 84%]
 [ 85%]
 [ 87%]
 [ 88%]
 [ 90%]
 [ 90%]
 [ 92%]
 .....................        [100%]
=============================== warnings summary ===============================

../../../.local/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:22

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:22: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

import imp
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:23

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:23: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:24

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:24: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:25

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:25: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:28

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:28: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

if hasattr(pil_image, 'HAMMING'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:29

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:29: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

_PIL_INTERPOLATION_METHODS['hamming'] = pil_image.HAMMING
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:30

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:30: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

if hasattr(pil_image, 'BOX'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:31

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:31: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

_PIL_INTERPOLATION_METHODS['box'] = pil_image.BOX
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:33

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:33: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

if hasattr(pil_image, 'LANCZOS'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:34

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:34: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

_PIL_INTERPOLATION_METHODS['lanczos'] = pil_image.LANCZOS
../../../../../usr/lib/python3/dist-packages/requests/init.py:89

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.9) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:228

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:228: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

interpolation: int = Image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:296

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:296: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

interpolation: int = Image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:312

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:312: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

interpolation: int = Image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:329

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:329: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

interpolation: int = Image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/io/image.py:11

/usr/local/lib/python3.8/dist-packages/torchvision/io/image.py:11: UserWarning: Failed to load image Python extension: libtorch_cuda_cu.so: cannot open shared object file: No such file or directory

warn(f"Failed to load image Python extension: {e}")
merlin/models/tf/models/base.py:3

/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py:3: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working

from collections import Sequence as SequenceCollection
tests/datasets/test_ecommerce.py::test_synthetic_aliccp_raw_data

tests/tf/test_dataset.py::test_tf_drp_reset[100-True-10]

tests/tf/test_dataset.py::test_tf_drp_reset[100-True-9]

tests/tf/test_dataset.py::test_tf_drp_reset[100-True-8]

tests/tf/test_dataset.py::test_tf_drp_reset[100-False-10]

tests/tf/test_dataset.py::test_tf_drp_reset[100-False-9]

tests/tf/test_dataset.py::test_tf_drp_reset[100-False-8]

tests/tf/test_dataset.py::test_tf_catname_ordering

tests/tf/test_dataset.py::test_tf_map

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:1253: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(
tests/tf/blocks/core/test_index.py: 3 warnings

tests/tf/models/test_retrieval.py: 28 warnings

tests/tf/prediction_tasks/test_next_item.py: 55 warnings

tests/tf/utils/test_batch.py: 2 warnings

/tmp/tmptm_h9mpa.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead

ag__.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.1]

tests/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.3]

tests/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.5]

tests/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.7]

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/backend.py:6089: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.

warnings.warn('tf.keras.backend.random_binomial is deprecated, '
tests/torch/block/test_mlp.py::test_mlp_block

/var/jenkins_home/workspace/merlin_models/models/tests/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ../torch/csrc/utils/tensor_new.cpp:201.)

return {key: torch.tensor(value) for key, value in data.items()}
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

=========== 422 passed, 9 skipped, 119 warnings in 680.40s (0:11:20) ===========

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_models] $ /bin/bash /tmp/jenkins4793375207947725838.sh

github-actions · 2022-05-04T22:30:44Z

Documentation preview

https://nvidia-merlin.github.io/models/review/pr-413

nvidia-merlin-bot · 2022-05-04T22:38:31Z

Click to view CI Results

GitHub pull request #413 of commit 4b5f4736d9261afca760666cf3bfa791ef8204d3, no merge conflicts.
Running as SYSTEM
Setting status of 4b5f4736d9261afca760666cf3bfa791ef8204d3 to PENDING with url https://10.20.13.93:8080/job/merlin_models/240/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/413/*:refs/remotes/origin/pr/413/* # timeout=10
 > git rev-parse 4b5f4736d9261afca760666cf3bfa791ef8204d3^{commit} # timeout=10
Checking out Revision 4b5f4736d9261afca760666cf3bfa791ef8204d3 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 4b5f4736d9261afca760666cf3bfa791ef8204d3 # timeout=10
Commit message: "Merge branch 'main' into fix_shuffle"
 > git rev-list --no-walk d6b4dc1d3355773848c9ac71db1827ca8f55239e # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins261961916912900739.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.4.0)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.0)
Requirement already satisfied: traitlets>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (5.1.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.4.0)
Requirement already satisfied: jupyter-core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.10.0)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.15.3)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.7.1)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (21.4.0)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: pyzmq>=22.3 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (22.3.0)
Requirement already satisfied: tornado>=6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.1)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.0)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 429 items / 2 skipped
tests/datasets/test_advertising.py .s                                    [  0%]

tests/datasets/test_ecommerce.py ..sss                                   [  1%]

tests/datasets/test_entertainment.py ....sss                             [  3%]

tests/datasets/test_social.py .                                          [  3%]

tests/datasets/test_synthetic.py .....                                   [  4%]

tests/tf/test_core.py ................                                   [  8%]

tests/tf/test_dataset.py ..............                                  [ 11%]

tests/tf/test_public_api.py .                                            [ 11%]

tests/tf/blocks/test_cross.py ............                               [ 14%]

tests/tf/blocks/test_dlrm.py ........                                    [ 16%]

tests/tf/blocks/test_interactions.py .                                   [ 16%]

tests/tf/blocks/test_mlp.py .............................                [ 23%]

tests/tf/blocks/core/test_aggregation.py .........                       [ 25%]

tests/tf/blocks/core/test_base.py .                                      [ 25%]

tests/tf/blocks/core/test_index.py ...                                   [ 26%]

tests/tf/blocks/core/test_masking.py .......                             [ 28%]

tests/tf/blocks/core/test_transformations.py ...........                 [ 30%]

tests/tf/blocks/retrieval/test_matrix_factorization.py ..                [ 31%]

tests/tf/blocks/retrieval/test_two_tower.py ...........                  [ 33%]

tests/tf/examples/test_01_getting_started.py .                           [ 34%]

tests/tf/examples/test_02_dataschema.py .                                [ 34%]

tests/tf/examples/test_03_exploring_different_models.py .                [ 34%]

tests/tf/examples/test_04_export_ranking_models.py .                     [ 34%]

tests/tf/examples/test_05_export_retrieval_model.py .                    [ 34%]

tests/tf/examples/test_06_advanced_own_architecture.py .                 [ 35%]

tests/tf/features/test_continuous.py .....                               [ 36%]

tests/tf/features/test_embedding.py ..........                           [ 38%]

tests/tf/features/test_tabular.py .......                                [ 40%]

tests/tf/layers/test_queue.py ..............                             [ 43%]

tests/tf/losses/test_losses.py .......................                   [ 48%]

tests/tf/metrics/test_metrics_popularity.py .....                        [ 50%]

tests/tf/metrics/test_metrics_ranking.py .................               [ 54%]

tests/tf/models/test_benchmark.py .                                      [ 54%]

tests/tf/models/test_ranking.py ......................                   [ 59%]

tests/tf/models/test_retrieval.py ..................                     [ 63%]

tests/tf/prediction_tasks/test_classification.py ..                      [ 64%]

tests/tf/prediction_tasks/test_multi_task.py .......                     [ 65%]

tests/tf/prediction_tasks/test_next_item.py ......................       [ 70%]

tests/tf/prediction_tasks/test_regression.py ..                          [ 71%]

tests/tf/prediction_tasks/test_sampling.py ....................          [ 75%]

tests/tf/utils/test_batch.py ....                                        [ 76%]

tests/torch/test_dataset.py .........                                    [ 79%]

tests/torch/test_public_api.py .                                         [ 79%]

tests/torch/block/test_base.py ....                                      [ 80%]

tests/torch/block/test_mlp.py .                                          [ 80%]

tests/torch/features/test_continuous.py ..                               [ 80%]

tests/torch/features/test_embedding.py ..............                    [ 84%]

tests/torch/features/test_tabular.py ....                                [ 85%]

tests/torch/model/test_head.py ............                              [ 87%]

tests/torch/model/test_model.py ..                                       [ 88%]

tests/torch/tabular/test_aggregation.py ........                         [ 90%]

tests/torch/tabular/test_tabular.py ...                                  [ 90%]

tests/torch/tabular/test_transformations.py .......                      [ 92%]

tests/utils/test_schema_utils.py ................................        [100%]
=============================== warnings summary ===============================

../../../.local/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:22

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:22: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

import imp
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:23

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:23: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

'nearest': pil_image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:24

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:24: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

'bilinear': pil_image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:25

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:25: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

'bicubic': pil_image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:28

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:28: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

if hasattr(pil_image, 'HAMMING'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:29

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:29: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

_PIL_INTERPOLATION_METHODS['hamming'] = pil_image.HAMMING
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:30

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:30: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

if hasattr(pil_image, 'BOX'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:31

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:31: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

_PIL_INTERPOLATION_METHODS['box'] = pil_image.BOX
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:33

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:33: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

if hasattr(pil_image, 'LANCZOS'):
../../../../../usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:34

/usr/local/lib/python3.8/dist-packages/keras_preprocessing/image/utils.py:34: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

_PIL_INTERPOLATION_METHODS['lanczos'] = pil_image.LANCZOS
../../../../../usr/lib/python3/dist-packages/requests/init.py:89

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.9) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:228

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:228: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

interpolation: int = Image.BILINEAR,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:296

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:296: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

interpolation: int = Image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:312

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:312: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

interpolation: int = Image.NEAREST,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:329

/usr/local/lib/python3.8/dist-packages/torchvision/transforms/functional_pil.py:329: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

interpolation: int = Image.BICUBIC,
../../../../../usr/local/lib/python3.8/dist-packages/torchvision/io/image.py:11

/usr/local/lib/python3.8/dist-packages/torchvision/io/image.py:11: UserWarning: Failed to load image Python extension: libtorch_cuda_cu.so: cannot open shared object file: No such file or directory

warn(f"Failed to load image Python extension: {e}")
merlin/models/tf/models/base.py:3

/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py:3: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working

from collections import Sequence as SequenceCollection
tests/datasets/test_ecommerce.py::test_synthetic_aliccp_raw_data

tests/tf/test_dataset.py::test_tf_drp_reset[100-True-10]

tests/tf/test_dataset.py::test_tf_drp_reset[100-True-9]

tests/tf/test_dataset.py::test_tf_drp_reset[100-True-8]

tests/tf/test_dataset.py::test_tf_drp_reset[100-False-10]

tests/tf/test_dataset.py::test_tf_drp_reset[100-False-9]

tests/tf/test_dataset.py::test_tf_drp_reset[100-False-8]

tests/tf/test_dataset.py::test_tf_catname_ordering

tests/tf/test_dataset.py::test_tf_map

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:1253: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(
tests/tf/blocks/core/test_index.py: 3 warnings

tests/tf/models/test_retrieval.py: 28 warnings

tests/tf/prediction_tasks/test_next_item.py: 55 warnings

tests/tf/utils/test_batch.py: 2 warnings

/tmp/tmpr7qoc4jy.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead

ag__.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)
tests/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.1]

tests/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.3]

tests/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.5]

tests/tf/blocks/core/test_transformations.py::test_stochastic_swap_noise[0.7]

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/backend.py:6089: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.

warnings.warn('tf.keras.backend.random_binomial is deprecated, '
tests/torch/block/test_mlp.py::test_mlp_block

/var/jenkins_home/workspace/merlin_models/models/tests/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ../torch/csrc/utils/tensor_new.cpp:201.)

return {key: torch.tensor(value) for key, value in data.items()}
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

=========== 422 passed, 9 skipped, 119 warnings in 680.86s (0:11:20) ===========

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_models] $ /bin/bash /tmp/jenkins4375939871001411706.sh

jperez999 approved these changes May 4, 2022

View reviewed changes

Merge branch 'main' into fix_shuffle

4b5f473

jperez999 merged commit e3fe59c into main May 4, 2022

benfred deleted the fix_shuffle branch May 4, 2022 22:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use shuffle_df from merlin-core#413

Use shuffle_df from merlin-core#413
jperez999 merged 2 commits intomainfrom
fix_shuffle

benfred commented May 4, 2022

Uh oh!

nvidia-merlin-bot commented May 4, 2022

Uh oh!

github-actions bot commented May 4, 2022

Uh oh!

nvidia-merlin-bot commented May 4, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

benfred commented May 4, 2022

Uh oh!

nvidia-merlin-bot commented May 4, 2022

Uh oh!

github-actions bot commented May 4, 2022

Documentation preview

Uh oh!

nvidia-merlin-bot commented May 4, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants