Limiting tests: reducing the time of the news recommendation GPU notebooks

[miguelgfierro]

Description

Reducing the test time of the limiting tests. I reduced the epochs and increased the batch size

Related Issues

related to #995 (comment)

Checklist:

• I have followed the contribution guidelines and code style for this project.
• I have added tests covering my contributions.
• I have updated the documentation accordingly.
• This PR is being made to staging branch and not to main branch.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[miguelgfierro]

@pradnyeshjoshi, in a VM could you please run:

git pull
git checkout limiting_tests
pytest tests/integration/examples/test_notebooks_gpu.py::test_nrms_quickstart_integration --durations
pytest tests/integration/examples/test_notebooks_gpu.py::test_naml_quickstart_integration --durations
pytest tests/integration/examples/test_notebooks_gpu.py::test_lstur_quickstart_integration --durations
pytest tests/integration/examples/test_notebooks_gpu.py::test_lstur_quickstart_integration --durations

There are two errors that might be expected:

1. OOM because the batch size is too high
2. errors in the metrics

[pradnyeshjoshi]

@pradnyeshjoshi, in a VM could you please run:

git pull
git checkout limiting_tests
pytest tests/integration/examples/test_notebooks_gpu.py::test_nrms_quickstart_integration --durations
pytest tests/integration/examples/test_notebooks_gpu.py::test_naml_quickstart_integration --durations
pytest tests/integration/examples/test_notebooks_gpu.py::test_lstur_quickstart_integration --durations
pytest tests/integration/examples/test_notebooks_gpu.py::test_lstur_quickstart_integration --durations

There are two errors that might be expected:

1. OOM because the batch size is too high
2. errors in the metrics

pytest tests/integration/examples/test_notebooks_gpu.py::test_nrms_quickstart_integration --durations=10
==================================================================== test session starts =====================================================================
platform linux -- Python 3.7.12, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/pradjoshi/recommenders, configfile: tox.ini
plugins: hypothesis-6.31.5, cov-3.0.0
collected 1 item                                                                                                                                             

tests/integration/examples/test_notebooks_gpu.py .                                                                                                     [100%]/home/pradjoshi/anaconda3/envs/reco-env/lib/python3.7/site-packages/coverage/report.py:87: CoverageWarning: Couldn't parse '/home/pradjoshi/recommenders/recommenders/evaluation/tf_evaluation.py': No source for code: '/home/pradjoshi/recommenders/recommenders/evaluation/tf_evaluation.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")


====================================================================== warnings summary ======================================================================
../anaconda3/envs/reco-env/lib/python3.7/site-packages/ansiwrap/core.py:6
  /home/pradjoshi/anaconda3/envs/reco-env/lib/python3.7/site-packages/ansiwrap/core.py:6: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    import imp

../anaconda3/envs/reco-env/lib/python3.7/site-packages/papermill/iorw.py:50
  /home/pradjoshi/anaconda3/envs/reco-env/lib/python3.7/site-packages/papermill/iorw.py:50: FutureWarning: pyarrow.HadoopFileSystem is deprecated as of 2.0.0, please use pyarrow.fs.HadoopFileSystem instead.
    from pyarrow import HadoopFileSystem

tests/integration/examples/test_notebooks_gpu.py: 4219 warnings
  /home/pradjoshi/anaconda3/envs/reco-env/lib/python3.7/site-packages/jsonschema/validators.py:197: DeprecationWarning: Passing a schema to Validator.iter_errors is deprecated and will be removed in a future release. Call validator.evolve(schema=new_schema).iter_errors(...) instead.
    DeprecationWarning,

tests/integration/examples/test_notebooks_gpu.py::test_nrms_quickstart_integration[5-64-42-demo-expected_values0]
  /home/pradjoshi/anaconda3/envs/reco-env/lib/python3.7/site-packages/traitlets/config/configurable.py:85: DeprecationWarning: Passing unrecognized arguments to super(PapermillNotebookClient).__init__(input_path='/home/pradjoshi/recommenders/examples/00_quick_start/nrms_MIND.ipynb').
  object.__init__() takes exactly one argument (the instance to initialize)
  This is deprecated in traitlets 4.2.This error will be raised in a future release of traitlets.
    super(Configurable, self).__init__(**kwargs)

-- Docs: https://docs.pytest.org/en/stable/warnings.html
------------------------------------------ generated xml file: /home/pradjoshi/recommenders/junit/test-results.xml -------------------------------------------

---------- coverage: platform linux, python 3.7.12-final-0 -----------
Name                                                                     Stmts   Miss  Cover
--------------------------------------------------------------------------------------------
recommenders/datasets/amazon_reviews.py                                    359    339     6%
recommenders/datasets/cosmos_cli.py                                         31     31     0%
recommenders/datasets/covid_utils.py                                        40      4    90%
recommenders/datasets/criteo.py                                             53     23    57%
recommenders/datasets/download_utils.py                                     50      5    90%
recommenders/datasets/mind.py                                              214    214     0%
recommenders/datasets/movielens.py                                         226     93    59%
recommenders/datasets/pandas_df_utils.py                                   137     18    87%
recommenders/datasets/python_splitters.py                                   57      5    91%
recommenders/datasets/spark_splitters.py                                    49     49     0%
recommenders/datasets/sparse.py                                             63      6    90%
recommenders/datasets/split_utils.py                                        49     12    76%
recommenders/datasets/wikidata.py                                           75     40    47%
recommenders/evaluation/python_evaluation.py                               269     17    94%
recommenders/evaluation/spark_evaluation.py                                224    224     0%
recommenders/models/deeprec/DataModel/ImplicitCF.py                         88     75    15%
recommenders/models/deeprec/deeprec_utils.py                               180     97    46%
recommenders/models/deeprec/io/dkn_item2item_iterator.py                    57     57     0%
recommenders/models/deeprec/io/dkn_iterator.py                             153    153     0%
recommenders/models/deeprec/io/iterator.py                                 105     91    13%
recommenders/models/deeprec/io/nextitnet_iterator.py                       119    119     0%
recommenders/models/deeprec/io/sequential_iterator.py                      223    223     0%
recommenders/models/deeprec/models/base_model.py                           312    312     0%
recommenders/models/deeprec/models/dkn.py                                  175    175     0%
recommenders/models/deeprec/models/dkn_item2item.py                         48     48     0%
recommenders/models/deeprec/models/graphrec/lightgcn.py                    170    170     0%
recommenders/models/deeprec/models/sequential/asvd.py                       16     16     0%
recommenders/models/deeprec/models/sequential/caser.py                      42     42     0%
recommenders/models/deeprec/models/sequential/gru4rec.py                    28     28     0%
recommenders/models/deeprec/models/sequential/nextitnet.py                  61     61     0%
recommenders/models/deeprec/models/sequential/rnn_cell_implement.py        282    282     0%
recommenders/models/deeprec/models/sequential/sequential_base_model.py     149    149     0%
recommenders/models/deeprec/models/sequential/sli_rec.py                    49     49     0%
recommenders/models/deeprec/models/sequential/sum.py                        57     57     0%
recommenders/models/deeprec/models/sequential/sum_cells.py                 143    143     0%
recommenders/models/deeprec/models/xDeepFM.py                              208    208     0%
recommenders/models/fastai/fastai_utils.py                                  31     31     0%
recommenders/models/geoimc/geoimc_algorithm.py                              69     64     7%
recommenders/models/geoimc/geoimc_data.py                                   92     56    39%
recommenders/models/geoimc/geoimc_predict.py                                40     24    40%
recommenders/models/geoimc/geoimc_utils.py                                  13      8    38%
recommenders/models/lightfm/lightfm_utils.py                                72     27    62%
recommenders/models/ncf/dataset.py                                         249    168    33%
recommenders/models/ncf/ncf_singlenode.py                                  131    128     2%
recommenders/models/newsrec/io/mind_all_iterator.py                        262    261     1%
recommenders/models/newsrec/io/mind_iterator.py                            185     15    92%
recommenders/models/newsrec/models/base_model.py                           166     44    73%
recommenders/models/newsrec/models/layers.py                               118     69    42%
recommenders/models/newsrec/models/lstur.py                                 70     70     0%
recommenders/models/newsrec/models/naml.py                                 119    119     0%
recommenders/models/newsrec/models/npa.py                                   68     68     0%
recommenders/models/newsrec/newsrec_utils.py                                73     21    71%
recommenders/models/rbm/rbm.py                                             183    116    37%
recommenders/models/rlrmc/RLRMCalgorithm.py                                127    124     2%
recommenders/models/rlrmc/RLRMCdataset.py                                   57     49    14%
recommenders/models/rlrmc/conjugate_gradient_ms.py                         118    118     0%
recommenders/models/sar/sar_singlenode.py                                  159     60    62%
recommenders/models/sasrec/model.py                                        331    331     0%
recommenders/models/sasrec/sampler.py                                       48     48     0%
recommenders/models/sasrec/ssept.py                                         85     85     0%
recommenders/models/sasrec/util.py                                          71     71     0%
recommenders/models/surprise/surprise_utils.py                              29     29     0%
recommenders/models/tfidf/tfidf_utils.py                                   145     34    77%
recommenders/models/vae/multinomial_vae.py                                 179    179     0%
recommenders/models/vae/standard_vae.py                                    175    175     0%
recommenders/models/vowpal_wabbit/vw.py                                     67      3    96%
recommenders/models/wide_deep/wide_deep_utils.py                            31     31     0%
recommenders/tuning/nni/ncf_training.py                                     95     95     0%
recommenders/tuning/nni/ncf_utils.py                                        27      4    85%
recommenders/tuning/nni/nni_utils.py                                        71     11    85%
recommenders/tuning/nni/svd_training.py                                    101    101     0%
recommenders/utils/general_utils.py                                         13      4    69%
recommenders/utils/gpu_utils.py                                             69     56    19%
recommenders/utils/notebook_memory_management.py                            50     50     0%
recommenders/utils/notebook_utils.py                                        16      4    75%
recommenders/utils/plot.py                                                  34      1    97%
recommenders/utils/python_utils.py                                          40      2    95%
recommenders/utils/spark_utils.py                                           26     18    31%
recommenders/utils/tf_utils.py                                             128    125     2%
--------------------------------------------------------------------------------------------
TOTAL                                                                     9095   6732    26%

38 files skipped due to complete coverage.
Coverage XML written to file coverage.xml

==================================================================== slowest 10 durations ====================================================================
1841.90s call     tests/integration/examples/test_notebooks_gpu.py::test_nrms_quickstart_integration[5-64-42-demo-expected_values0]

(2 durations < 0.005s hidden.  Use -vv to show these durations.)
======================================================= 1 passed, 4222 warnings in 1845.26s (0:30:45) ========================================================

pytest tests/integration/examples/test_notebooks_gpu.py::test_naml_quickstart_integration --durations=10
==================================================================== test session starts =====================================================================
platform linux -- Python 3.7.12, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/pradjoshi/recommenders, configfile: tox.ini
plugins: hypothesis-6.31.5, cov-3.0.0
collected 1 item                                                                                                                                             

tests/integration/examples/test_notebooks_gpu.py .                                                                                                     [100%]/home/pradjoshi/anaconda3/envs/reco-env/lib/python3.7/site-packages/coverage/report.py:87: CoverageWarning: Couldn't parse '/home/pradjoshi/recommenders/recommenders/evaluation/tf_evaluation.py': No source for code: '/home/pradjoshi/recommenders/recommenders/evaluation/tf_evaluation.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")


====================================================================== warnings summary ======================================================================
../anaconda3/envs/reco-env/lib/python3.7/site-packages/ansiwrap/core.py:6
  /home/pradjoshi/anaconda3/envs/reco-env/lib/python3.7/site-packages/ansiwrap/core.py:6: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    import imp

../anaconda3/envs/reco-env/lib/python3.7/site-packages/papermill/iorw.py:50
  /home/pradjoshi/anaconda3/envs/reco-env/lib/python3.7/site-packages/papermill/iorw.py:50: FutureWarning: pyarrow.HadoopFileSystem is deprecated as of 2.0.0, please use pyarrow.fs.HadoopFileSystem instead.
    from pyarrow import HadoopFileSystem

tests/integration/examples/test_notebooks_gpu.py: 8643 warnings
  /home/pradjoshi/anaconda3/envs/reco-env/lib/python3.7/site-packages/jsonschema/validators.py:197: DeprecationWarning: Passing a schema to Validator.iter_errors is deprecated and will be removed in a future release. Call validator.evolve(schema=new_schema).iter_errors(...) instead.
    DeprecationWarning,

tests/integration/examples/test_notebooks_gpu.py::test_naml_quickstart_integration[5-64-42-demo-expected_values0]
  /home/pradjoshi/anaconda3/envs/reco-env/lib/python3.7/site-packages/traitlets/config/configurable.py:85: DeprecationWarning: Passing unrecognized arguments to super(PapermillNotebookClient).__init__(input_path='/home/pradjoshi/recommenders/examples/00_quick_start/naml_MIND.ipynb').
  object.__init__() takes exactly one argument (the instance to initialize)
  This is deprecated in traitlets 4.2.This error will be raised in a future release of traitlets.
    super(Configurable, self).__init__(**kwargs)

-- Docs: https://docs.pytest.org/en/stable/warnings.html
------------------------------------------ generated xml file: /home/pradjoshi/recommenders/junit/test-results.xml -------------------------------------------

---------- coverage: platform linux, python 3.7.12-final-0 -----------
Name                                                                     Stmts   Miss  Cover
--------------------------------------------------------------------------------------------
recommenders/datasets/amazon_reviews.py                                    359    339     6%
recommenders/datasets/cosmos_cli.py                                         31     31     0%
recommenders/datasets/covid_utils.py                                        40      4    90%
recommenders/datasets/criteo.py                                             53     23    57%
recommenders/datasets/download_utils.py                                     50      5    90%
recommenders/datasets/mind.py                                              214    214     0%
recommenders/datasets/movielens.py                                         226     93    59%
recommenders/datasets/pandas_df_utils.py                                   137     18    87%
recommenders/datasets/python_splitters.py                                   57      5    91%
recommenders/datasets/spark_splitters.py                                    49     49     0%
recommenders/datasets/sparse.py                                             63      6    90%
recommenders/datasets/split_utils.py                                        49     12    76%
recommenders/datasets/wikidata.py                                           75     40    47%
recommenders/evaluation/python_evaluation.py                               269     17    94%
recommenders/evaluation/spark_evaluation.py                                224    224     0%
recommenders/models/deeprec/DataModel/ImplicitCF.py                         88     75    15%
recommenders/models/deeprec/deeprec_utils.py                               180     97    46%
recommenders/models/deeprec/io/dkn_item2item_iterator.py                    57     57     0%
recommenders/models/deeprec/io/dkn_iterator.py                             153    153     0%
recommenders/models/deeprec/io/iterator.py                                 105     91    13%
recommenders/models/deeprec/io/nextitnet_iterator.py                       119    119     0%
recommenders/models/deeprec/io/sequential_iterator.py                      223    223     0%
recommenders/models/deeprec/models/base_model.py                           312    312     0%
recommenders/models/deeprec/models/dkn.py                                  175    175     0%
recommenders/models/deeprec/models/dkn_item2item.py                         48     48     0%
recommenders/models/deeprec/models/graphrec/lightgcn.py                    170    170     0%
recommenders/models/deeprec/models/sequential/asvd.py                       16     16     0%
recommenders/models/deeprec/models/sequential/caser.py                      42     42     0%
recommenders/models/deeprec/models/sequential/gru4rec.py                    28     28     0%
recommenders/models/deeprec/models/sequential/nextitnet.py                  61     61     0%
recommenders/models/deeprec/models/sequential/rnn_cell_implement.py        282    282     0%
recommenders/models/deeprec/models/sequential/sequential_base_model.py     149    149     0%
recommenders/models/deeprec/models/sequential/sli_rec.py                    49     49     0%
recommenders/models/deeprec/models/sequential/sum.py                        57     57     0%
recommenders/models/deeprec/models/sequential/sum_cells.py                 143    143     0%
recommenders/models/deeprec/models/xDeepFM.py                              208    208     0%
recommenders/models/fastai/fastai_utils.py                                  31     31     0%
recommenders/models/geoimc/geoimc_algorithm.py                              69     64     7%
recommenders/models/geoimc/geoimc_data.py                                   92     56    39%
recommenders/models/geoimc/geoimc_predict.py                                40     24    40%
recommenders/models/geoimc/geoimc_utils.py                                  13      8    38%
recommenders/models/lightfm/lightfm_utils.py                                72     27    62%
recommenders/models/ncf/dataset.py                                         249    168    33%
recommenders/models/ncf/ncf_singlenode.py                                  131    128     2%
recommenders/models/newsrec/io/mind_all_iterator.py                        262     23    91%
recommenders/models/newsrec/io/mind_iterator.py                            185     15    92%
recommenders/models/newsrec/models/base_model.py                           166     44    73%
recommenders/models/newsrec/models/layers.py                               118     69    42%
recommenders/models/newsrec/models/lstur.py                                 70     70     0%
recommenders/models/newsrec/models/npa.py                                   68     68     0%
recommenders/models/newsrec/newsrec_utils.py                                73     17    77%
recommenders/models/rbm/rbm.py                                             183    116    37%
recommenders/models/rlrmc/RLRMCalgorithm.py                                127    124     2%
recommenders/models/rlrmc/RLRMCdataset.py                                   57     49    14%
recommenders/models/rlrmc/conjugate_gradient_ms.py                         118    118     0%
recommenders/models/sar/sar_singlenode.py                                  159     60    62%
recommenders/models/sasrec/model.py                                        331    331     0%
recommenders/models/sasrec/sampler.py                                       48     48     0%
recommenders/models/sasrec/ssept.py                                         85     85     0%
recommenders/models/sasrec/util.py                                          71     71     0%
recommenders/models/surprise/surprise_utils.py                              29     29     0%
recommenders/models/tfidf/tfidf_utils.py                                   145     34    77%
recommenders/models/vae/multinomial_vae.py                                 179    179     0%
recommenders/models/vae/standard_vae.py                                    175    175     0%
recommenders/models/vowpal_wabbit/vw.py                                     67      3    96%
recommenders/models/wide_deep/wide_deep_utils.py                            31     31     0%
recommenders/tuning/nni/ncf_training.py                                     95     95     0%
recommenders/tuning/nni/ncf_utils.py                                        27      4    85%
recommenders/tuning/nni/nni_utils.py                                        71     11    85%
recommenders/tuning/nni/svd_training.py                                    101    101     0%
recommenders/utils/general_utils.py                                         13      4    69%
recommenders/utils/gpu_utils.py                                             69     56    19%
recommenders/utils/notebook_memory_management.py                            50     50     0%
recommenders/utils/notebook_utils.py                                        16      4    75%
recommenders/utils/plot.py                                                  34      1    97%
recommenders/utils/python_utils.py                                          40      2    95%
recommenders/utils/spark_utils.py                                           26     18    31%
recommenders/utils/tf_utils.py                                             128    125     2%
--------------------------------------------------------------------------------------------
TOTAL                                                                     9095   6371    30%

39 files skipped due to complete coverage.
Coverage XML written to file coverage.xml

==================================================================== slowest 10 durations ====================================================================
3853.96s call     tests/integration/examples/test_notebooks_gpu.py::test_naml_quickstart_integration[5-64-42-demo-expected_values0]

pytest tests/integration/examples/test_notebooks_gpu.py::test_lstur_quickstart_integration --durations=10
============================================================================================================================================= test session starts ==============================================================================================================================================
platform linux -- Python 3.7.12, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/pradjoshi/recommenders, configfile: tox.ini
plugins: hypothesis-6.31.5, cov-3.0.0
collected 1 item                                                                                                                                                                                                                                                                                               

tests/integration/examples/test_notebooks_gpu.py .                                                                                                                                                                                                                                                       [100%]/home/pradjoshi/anaconda3/envs/reco-env/lib/python3.7/site-packages/coverage/report.py:87: CoverageWarning: Couldn't parse '/home/pradjoshi/recommenders/recommenders/evaluation/tf_evaluation.py': No source for code: '/home/pradjoshi/recommenders/recommenders/evaluation/tf_evaluation.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")


=============================================================================================================================================== warnings summary ===============================================================================================================================================
../anaconda3/envs/reco-env/lib/python3.7/site-packages/ansiwrap/core.py:6
  /home/pradjoshi/anaconda3/envs/reco-env/lib/python3.7/site-packages/ansiwrap/core.py:6: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    import imp

../anaconda3/envs/reco-env/lib/python3.7/site-packages/papermill/iorw.py:50
  /home/pradjoshi/anaconda3/envs/reco-env/lib/python3.7/site-packages/papermill/iorw.py:50: FutureWarning: pyarrow.HadoopFileSystem is deprecated as of 2.0.0, please use pyarrow.fs.HadoopFileSystem instead.
    from pyarrow import HadoopFileSystem

tests/integration/examples/test_notebooks_gpu.py: 3473 warnings
  /home/pradjoshi/anaconda3/envs/reco-env/lib/python3.7/site-packages/jsonschema/validators.py:197: DeprecationWarning: Passing a schema to Validator.iter_errors is deprecated and will be removed in a future release. Call validator.evolve(schema=new_schema).iter_errors(...) instead.
    DeprecationWarning,

tests/integration/examples/test_notebooks_gpu.py::test_lstur_quickstart_integration[5-64-42-demo-expected_values0]
  /home/pradjoshi/anaconda3/envs/reco-env/lib/python3.7/site-packages/traitlets/config/configurable.py:85: DeprecationWarning: Passing unrecognized arguments to super(PapermillNotebookClient).__init__(input_path='/home/pradjoshi/recommenders/examples/00_quick_start/lstur_MIND.ipynb').
  object.__init__() takes exactly one argument (the instance to initialize)
  This is deprecated in traitlets 4.2.This error will be raised in a future release of traitlets.
    super(Configurable, self).__init__(**kwargs)

-- Docs: https://docs.pytest.org/en/stable/warnings.html
------------------------------------------------------------------------------------------------------------------- generated xml file: /home/pradjoshi/recommenders/junit/test-results.xml --------------------------------------------------------------------------------------------------------------------

---------- coverage: platform linux, python 3.7.12-final-0 -----------
Name                                                                     Stmts   Miss  Cover
--------------------------------------------------------------------------------------------
recommenders/datasets/amazon_reviews.py                                    359    339     6%
recommenders/datasets/cosmos_cli.py                                         31     31     0%
recommenders/datasets/covid_utils.py                                        40      4    90%
recommenders/datasets/criteo.py                                             53     23    57%
recommenders/datasets/download_utils.py                                     50      5    90%
recommenders/datasets/mind.py                                              214    214     0%
recommenders/datasets/movielens.py                                         226     93    59%
recommenders/datasets/pandas_df_utils.py                                   137     18    87%
recommenders/datasets/python_splitters.py                                   57      5    91%
recommenders/datasets/spark_splitters.py                                    49     49     0%
recommenders/datasets/sparse.py                                             63      6    90%
recommenders/datasets/split_utils.py                                        49     12    76%
recommenders/datasets/wikidata.py                                           75     40    47%
recommenders/evaluation/python_evaluation.py                               269     17    94%
recommenders/evaluation/spark_evaluation.py                                224    224     0%
recommenders/models/deeprec/DataModel/ImplicitCF.py                         88     75    15%
recommenders/models/deeprec/deeprec_utils.py                               180     97    46%
recommenders/models/deeprec/io/dkn_item2item_iterator.py                    57     57     0%
recommenders/models/deeprec/io/dkn_iterator.py                             153    153     0%
recommenders/models/deeprec/io/iterator.py                                 105     91    13%
recommenders/models/deeprec/io/nextitnet_iterator.py                       119    119     0%
recommenders/models/deeprec/io/sequential_iterator.py                      223    223     0%
recommenders/models/deeprec/models/base_model.py                           312    312     0%
recommenders/models/deeprec/models/dkn.py                                  175    175     0%
recommenders/models/deeprec/models/dkn_item2item.py                         48     48     0%
recommenders/models/deeprec/models/graphrec/lightgcn.py                    170    170     0%
recommenders/models/deeprec/models/sequential/asvd.py                       16     16     0%
recommenders/models/deeprec/models/sequential/caser.py                      42     42     0%
recommenders/models/deeprec/models/sequential/gru4rec.py                    28     28     0%
recommenders/models/deeprec/models/sequential/nextitnet.py                  61     61     0%
recommenders/models/deeprec/models/sequential/rnn_cell_implement.py        282    282     0%
recommenders/models/deeprec/models/sequential/sequential_base_model.py     149    149     0%
recommenders/models/deeprec/models/sequential/sli_rec.py                    49     49     0%
recommenders/models/deeprec/models/sequential/sum.py                        57     57     0%
recommenders/models/deeprec/models/sequential/sum_cells.py                 143    143     0%
recommenders/models/deeprec/models/xDeepFM.py                              208    208     0%
recommenders/models/fastai/fastai_utils.py                                  31     31     0%
recommenders/models/geoimc/geoimc_algorithm.py                              69     64     7%
recommenders/models/geoimc/geoimc_data.py                                   92     56    39%
recommenders/models/geoimc/geoimc_predict.py                                40     24    40%
recommenders/models/geoimc/geoimc_utils.py                                  13      8    38%
recommenders/models/lightfm/lightfm_utils.py                                72     27    62%
recommenders/models/ncf/dataset.py                                         249    168    33%
recommenders/models/ncf/ncf_singlenode.py                                  131    128     2%
recommenders/models/newsrec/io/mind_all_iterator.py                        262     23    91%
recommenders/models/newsrec/io/mind_iterator.py                            185     15    92%
recommenders/models/newsrec/models/base_model.py                           166     44    73%
recommenders/models/newsrec/models/layers.py                               118     64    46%
recommenders/models/newsrec/models/lstur.py                                 70      4    94%
recommenders/models/newsrec/models/npa.py                                   68     68     0%
recommenders/models/newsrec/newsrec_utils.py                                73     15    79%
recommenders/models/rbm/rbm.py                                             183    116    37%
recommenders/models/rlrmc/RLRMCalgorithm.py                                127    124     2%
recommenders/models/rlrmc/RLRMCdataset.py                                   57     49    14%
recommenders/models/rlrmc/conjugate_gradient_ms.py                         118    118     0%
recommenders/models/sar/sar_singlenode.py                                  159     60    62%
recommenders/models/sasrec/model.py                                        331    331     0%
recommenders/models/sasrec/sampler.py                                       48     48     0%
recommenders/models/sasrec/ssept.py                                         85     85     0%
recommenders/models/sasrec/util.py                                          71     71     0%
recommenders/models/surprise/surprise_utils.py                              29     29     0%
recommenders/models/tfidf/tfidf_utils.py                                   145     34    77%
recommenders/models/vae/multinomial_vae.py                                 179    179     0%
recommenders/models/vae/standard_vae.py                                    175    175     0%
recommenders/models/vowpal_wabbit/vw.py                                     67      3    96%
recommenders/models/wide_deep/wide_deep_utils.py                            31     31     0%
recommenders/tuning/nni/ncf_training.py                                     95     95     0%
recommenders/tuning/nni/ncf_utils.py                                        27      4    85%
recommenders/tuning/nni/nni_utils.py                                        71     11    85%
recommenders/tuning/nni/svd_training.py                                    101    101     0%
recommenders/utils/general_utils.py                                         13      4    69%
recommenders/utils/gpu_utils.py                                             69     56    19%
recommenders/utils/notebook_memory_management.py                            50     50     0%
recommenders/utils/notebook_utils.py                                        16      4    75%
recommenders/utils/plot.py                                                  34      1    97%
recommenders/utils/python_utils.py                                          40      2    95%
recommenders/utils/spark_utils.py                                           26     18    31%
recommenders/utils/tf_utils.py                                             128    125     2%
--------------------------------------------------------------------------------------------
TOTAL                                                                     9095   6298    31%

39 files skipped due to complete coverage.
Coverage XML written to file coverage.xml

============================================================================================================================================= slowest 10 durations =============================================================================================================================================
1552.28s call     tests/integration/examples/test_notebooks_gpu.py::test_lstur_quickstart_integration[5-64-42-demo-expected_values0]

(2 durations < 0.005s hidden.  Use -vv to show these durations.)
================================================================================================================================ 1 passed, 3476 warnings in 1555.61s (0:25:55) =================================================================================================================================

[miguelgfierro]

@pradnyeshjoshi the times between your VM and the test VM vary a lot, even though the batch_size is bigger in this PR. What VM are you using?

@anargyri @laserprec there is a weird error in the coverage, any idea of what could be the problem?

[pradnyeshjoshi]

@pradnyeshjoshi the times between your VM and the test VM vary a lot, even though the batch_size is bigger in this PR. What VM are you using?

@anargyri @laserprec there is a weird error in the coverage, any idea of what could be the problem?

@miguelgfierro I'm using Standard NV6 (6 vcpus, 56 GiB memory).

[anargyri]

@pradnyeshjoshi the times between your VM and the test VM vary a lot, even though the batch_size is bigger in this PR. What VM are you using?

@anargyri @laserprec there is a weird error in the coverage, any idea of what could be the problem?

Not sure, but it has to do with switching to an AzureML VM for the tests.

[miguelgfierro]

@pradnyeshjoshi do you think we can merge this PR?

[laserprec]

@anargyri @laserprec there is a weird error in the coverage, any idea of what could be the problem?

The error is relating to merging all the code coverage reports from the various runs before sending to codecov. When merging, coverage is trying to look for the local install path of our recommenders package to finalize the coverage numbers, but such path does not exist outside of the test machine, thus the error: No source for code: '/home/azureuser/localfiles/runner/work/recommenders/recommenders/recommenders/__init__.py'.

As it turns out, codecov can merge the reports for us, I think we can try skipping the merging step and send all of the .coverage* files to codecov instead.