run_test at CircleCI fails

[lee1043]

run_tests fails in CircleCI, since from #634 and other recent PRs.

@muryanto1 Any chance if this is related to removing python 2.x?

[muryanto1]

@lee1043 There are several failures https://circleci.com/gh/PCMDI/pcmdi_metrics/1750
Some of them is due to CDAT/genutil#38.
https://circleci.com/gh/PCMDI/pcmdi_metrics/tree/use_updated_conda_recipes

some are due to flake8 failures (since there is a new flake8 package).

I do not think this is related to removal of py2.7.
We are not building py2.7 for cdat anymore.

Due you know what caused failures that say:
b' main_mod_name = getattr(main_module.spec, "name", None)'
b"AttributeError: module 'main' has no attribute 'spec'"

[muryanto1]

Notes for reproducing the failures locally on your laptop.

#
# install miniconda
#
$ mkdir -p ~/work/pcmdi/workdir
$ export WORKDIR=~/work/pcmdi/workdir
$ cd $WORKDIR

$ bash ~/work/Miniconda3-4.7.12.1-MacOSX-x86_64.sh -b -p miniconda

$ source miniconda/etc/profile.d/conda.sh
$ conda activate base

$ cd ~/work/pcmdi
$ git clone -b use_updated_conda_recipes https://github.com/pcmdi/pcmdi_metrics
$ cd pcmdi_metrics

#
# define environment variables 
#
$ export PKG_NAME=pcmdi_metrics
$ export REPO_NAME=pcmdi_metrics
$ export LAST_STABLE=1.2

$ export CIRCLE_BRANCH=use_updated_conda_recipes
$ export ENV_NAME=test_pcmdi
$ export PY_VER=3.7
$ export LIBNETCDF=nompi

$ export CONDA_PY_VER="python=$PY_VER"
$ export LIBNETCDF_VER="libnetcdf=*=${LIBNETCDF}_*"

$ export LOCAL_REPO=`pwd`
$ mkdir $WORKDIR/artifact_dir
$ export ARTIFACT_DIR=$WORKDIR/artifact_dir

#
# rerender - pick up conda-forge dependencies pinning
#
$ make conda-local-rerender conda=$WORKDIR/miniconda/bin/conda workdir=$WORKDIR last_stable=$LAST_STABLE branch=$CIRCLE_BRANCH local_repo=$LOCAL_REPO

#
# build
#
$ make conda-local-build conda=$WORKDIR/miniconda/bin/conda workdir=$WORKDIR artifact_dir=$PWD/$artifacts_dir local_repo=$LOCAL_REPO

#
# create a test environment, the test environment name is whatever you set ENV_NAME to.
#
$ make setup-tests conda=$WORKDIR/miniconda/bin/conda conda_env=$ENV_NAME extra_pkgs="$CONDA_PY_VER $LIBNETCDF_VER $COVERAGE_PKGS"

$ conda activate $ENV_NAME

#
# run all tests
#
$ python run_tests.py -v 2

# after you run run_test.py above, you can rerun individual testcases that failed:
$ nosetests -s tests/test_pmp_diurnal.py:DiurnalTest.testFourierDiurnalAllGrid 

[lee1043]

@muryanto1 thanks for the advice. I was able to rerun the failing test on my local mac:

(test_pcmdi) ml-9713657:pcmdi_metrics lee1043$ nosetests -s tests/test_pmp_diurnal.py:DiurnalTest.testFourierDiurnalAllGrid 

Log Message:

modpath  tests/diurnal/results/nc
filename_template  pr_%(model)_%(month)_%(firstyear)-%(lastyear)_diurnal_avg.nc
filename_template_LST  pr_%(model)_LocalSolarTimes.nc
LSTFILES: ['tests/diurnal/results/nc/pr_CMCC_LocalSolarTimes.nc']
TMPL pr_*_LocalSolarTimes.nc
Reading tests/diurnal/results/nc/pr_CMCC_LocalSolarTimes.nc ... pr_CMCC_LocalSolarTimes.nc
====================
CMCC
====================
Reading time series of mean diurnal cycle ...
Input shapes:  (8, 46, 72) (8, 46, 72)
Getting latitude and longitude coordinates.
Taking fast Fourier transform of the mean diurnal cycle ...
Creating output arrays ...
Calling numpy FFT function ...
(8, 46, 72)
Converting from complex-valued FFT to real-valued amplitude and phase ...
  Output:
    cycmean (46, 72)
    maxvalue (3, 46, 72)
    tmax (3, 46, 72)
"Re-decorating" Fourier harmonics with grid info, etc., ...
... and writing to netCDF.
nccreate: filename "/Users/lee1043/work/pcmdi/pcmdi_metrics/test_data/results/nc/pr_CMCC_Jul_1999-2005_tmean.nc": Permission denied
Traceback (most recent call last):
  File "/Users/lee1043/work/pcmdi/workdir/miniconda/envs/test_pcmdi/lib/python3.7/site-packages/cdms2/dataset.py", line 1275, in __init__
    _fileobj_ = Cdunif.CdunifFile(path, mode)
OSError: Unknown Error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/lee1043/work/pcmdi/workdir/miniconda/envs/test_pcmdi/bin/fourierDiurnalAllGrid.py", line 4, in <module>
    __import__('pkg_resources').run_script('pcmdi-metrics===v1.2.1-239-g9480c72', 'fourierDiurnalAllGrid.py')
  File "/Users/lee1043/work/pcmdi/workdir/miniconda/envs/test_pcmdi/lib/python3.7/site-packages/pkg_resources/__init__.py", line 666, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/Users/lee1043/work/pcmdi/workdir/miniconda/envs/test_pcmdi/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1471, in run_script
    exec(script_code, namespace, namespace)
  File "/Users/lee1043/work/pcmdi/workdir/miniconda/envs/test_pcmdi/lib/python3.7/site-packages/pcmdi_metrics-v1.2.1_239_g9480c72-py3.6.egg/EGG-INFO/scripts/fourierDiurnalAllGrid.py", line 112, in <module>
  File "/Users/lee1043/work/pcmdi/workdir/miniconda/envs/test_pcmdi/lib/python3.7/site-packages/cdms2/dataset.py", line 497, in openDataset
    return CdmsFile(path, mode, mpiBarrier=CdMpi)
  File "/Users/lee1043/work/pcmdi/workdir/miniconda/envs/test_pcmdi/lib/python3.7/site-packages/cdms2/dataset.py", line 1277, in __init__
    raise CDMSError('Cannot open file %s (%s)' % (path, err))
cdms2.error.CDMSError: Cannot open file /Users/lee1043/work/pcmdi/pcmdi_metrics/test_data/results/nc/pr_CMCC_Jul_1999-2005_tmean.nc (Unknown Error)
Checking integrity of results/nc/pr_CMCC_Jul_1999-2005_tmean.nc
F
======================================================================
FAIL: testFourierDiurnalAllGrid (test_pmp_diurnal.DiurnalTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/lee1043/work/pcmdi/pcmdi_metrics/tests/test_pmp_diurnal.py", line 56, in testFourierDiurnalAllGrid
    self.compare_nc("results/nc/pr_CMCC_Jul_1999-2005_tmean.nc")
  File "/Users/lee1043/work/pcmdi/pcmdi_metrics/tests/test_pmp_diurnal.py", line 29, in compare_nc
    self.assertTrue(os.path.exists(os.path.join("test_data",test_name)))
AssertionError: False is not true
-------------------- >> begin captured logging << --------------------
asyncio: DEBUG: Using selector: KqueueSelector
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
Ran 1 test in 1.431s

FAILED (failures=1)

[lee1043]

It was resolved by creating test_data/results/nc directory prior to run the test. It looks the "directory generation" process has been missed somehow, which prevents creating nc file as a results of the test, so it cannot be compared to the original file.

(test_pcmdi) ml-9713657:pcmdi_metrics lee1043$ pwd
/Users/lee1043/work/pcmdi/pcmdi_metrics
(test_pcmdi) ml-9713657:pcmdi_metrics lee1043$ mkdir -p test_data/results/nc
(test_pcmdi) ml-9713657:pcmdi_metrics lee1043$ nosetests -s tests/test_pmp_diurnal.py:DiurnalTest.testFourierDiurnalAllGrid 

modpath  tests/diurnal/results/nc
filename_template  pr_%(model)_%(month)_%(firstyear)-%(lastyear)_diurnal_avg.nc
filename_template_LST  pr_%(model)_LocalSolarTimes.nc
LSTFILES: ['tests/diurnal/results/nc/pr_CMCC_LocalSolarTimes.nc']
TMPL pr_*_LocalSolarTimes.nc
Reading tests/diurnal/results/nc/pr_CMCC_LocalSolarTimes.nc ... pr_CMCC_LocalSolarTimes.nc
====================
CMCC
====================
Reading time series of mean diurnal cycle ...
Input shapes:  (8, 46, 72) (8, 46, 72)
Getting latitude and longitude coordinates.
Taking fast Fourier transform of the mean diurnal cycle ...
Creating output arrays ...
Calling numpy FFT function ...
(8, 46, 72)
Converting from complex-valued FFT to real-valued amplitude and phase ...
  Output:
    cycmean (46, 72)
    maxvalue (3, 46, 72)
    tmax (3, 46, 72)
"Re-decorating" Fourier harmonics with grid info, etc., ...
... and writing to netCDF.
Checking integrity of results/nc/pr_CMCC_Jul_1999-2005_tmean.nc
Checking same variables are present as in results/nc/pr_CMCC_Jul_1999-2005_tmean.nc
Checking variable bounds_latitude is correct
Checking variable bounds_longitude is correct
Checking variable tmean is correct
Checking integrity of results/nc/pr_CMCC_Jul_1999-2005_tS.nc
Checking same variables are present as in results/nc/pr_CMCC_Jul_1999-2005_tS.nc
Checking variable bounds_latitude is correct
Checking variable bounds_longitude is correct
Checking variable tS is correct
Checking integrity of results/nc/pr_CMCC_Jul_1999-2005_S.nc
Checking same variables are present as in results/nc/pr_CMCC_Jul_1999-2005_S.nc
Checking variable bounds_latitude is correct
Checking variable bounds_longitude is correct
Checking variable S is correct
.
----------------------------------------------------------------------
Ran 1 test in 1.403s

OK

[lee1043]

I found that running all tests by python run_tests.py -v 2 commend generates jsons and ascii directories under pcmdi_metrics/test_data/results, but not nc.

@muryanto1 do you know in which part of the testing code it generates those directories?

[gleckler1]

@lee1043 Is this issue resolved?

[lee1043]

@gleckler1 yes, I am closing it. Thanks for the reminder!