Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Numerous warnings when generating composites #9

Open
gerritholl opened this issue Apr 24, 2019 · 5 comments
Open

Numerous warnings when generating composites #9

gerritholl opened this issue Apr 24, 2019 · 5 comments
Assignees
@gerritholl

Comments

@gerritholl
Copy link
Collaborator

Generating composites issues numerous warnings:

from satpy import Scene
from glob import glob
from satpy.utils import debug_on
debug_on()
fn_nwcsaf = glob("/media/nas/x21308/scratch/NWCSAF/*100000Z.nc")
fn_sev = glob("/media/nas/x21308/scratch/SEVIRI/*201904151000*")
fn_dem=["/media/nas/x21308/DEM/dem_eu_1km.tif"]
sc = Scene(filenames={"seviri_l1b_hrit": fn_sev, "nwcsaf-geo": fn_nwcsaf})
sc.load(["cmic_reff", "IR_108", "IR_087", "cmic_cot", "IR_016", "VIS006",
    "IR_120", "VIS008", "cmic_lwp", "IR_039"])
ls = sc.resample("eurol")
ls.load(["fls_night"])
ls.show("fls_night")
ls.load(["fls_day", "fls_night"])
ls.show("fls_day")

Running this gives:

/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/pyresample/spherical.py:123: RuntimeWarning: invalid value encountered in true_divide
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/algorithms.py:1446: RuntimeWarning: invalid value encountered in less
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/dask/core.py:119: RuntimeWarning: invalid value encountered in less_equal
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/dask/core.py:119: RuntimeWarning: invalid value encountered in greater_equal
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:433: RuntimeWarning: invalid value encountered in greater
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:434: RuntimeWarning: invalid value encountered in less
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:437: RuntimeWarning: invalid value encountered in greater
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:498: RuntimeWarning: invalid value encountered in greater_equal
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:499: RuntimeWarning: invalid value encountered in greater_equal
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:535: RuntimeWarning: invalid value encountered in less
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:598: RuntimeWarning: invalid value encountered in greater
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:602: RuntimeWarning: invalid value encountered in greater
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:668: RuntimeWarning: invalid value encountered in greater
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:697: RuntimeWarning: invalid value encountered in less_equal
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/algorithms.py:692: RuntimeWarning: Mean of empty slice
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:757: RuntimeWarning: invalid value encountered in greater
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:855: RuntimeWarning: invalid value encountered in greater
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:1055: RuntimeWarning: Mean of empty slice
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/scipy/optimize/_minimize.py:517: RuntimeWarning: Method BFGS cannot handle constraints nor bounds.

The last warning is printed 4385 times, but that may be related to https://bugs.python.org/issue29672 .
@gerritholl gerritholl self-assigned this Apr 24, 2019
@gerritholl
Copy link
Collaborator Author

When running python with the -W always flag, it appears many warnings are issued thousands of times. The most frequent ones:

     11 /media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/pyresample/geometry.py:1364: DeprecationWarning: 'get_proj_vectors_dask' is deprecated, please use 'get_proj_vectors' with the 'chunks' keyword argument specified.
     11 /media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/pyresample/geometry.py:916: PendingDeprecationWarning: 'x_size' is deprecated, use 'width' instead.
     11 /media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/pyresample/geometry.py:921: PendingDeprecationWarning: 'y_size' is deprecated, use 'height' instead.
     11   warnings.warn("'x_size' is deprecated, use 'width' instead.", PendingDeprecationWarning)
     11   warnings.warn("'y_size' is deprecated, use 'height' instead.", PendingDeprecationWarning)
     13 [DEBUG/MainProcess] worker exiting after 3 tasks
     22   "'get_lonlats' with the 'chunks' keyword argument specified.", DeprecationWarning)
     22 /media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/pyresample/geometry.py:1501: DeprecationWarning: 'get_lonlats_dask' is deprecated, please use 'get_lonlats' with the 'chunks' keyword argument specified.
     26 /media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__
     26   return f(*args, **kwds)
     32   **kwargs))
     32 /media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/satpy/readers/seviri_base.py:286: PendingDeprecationWarning: xarray.ufuncs will be deprecated when xarray no longer supports versions of numpy older than v1.17. Instead, use numpy ufuncs directly.
     32 /media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/xarray/core/dataarray.py:1961: PendingDeprecationWarning: xarray.ufuncs will be deprecated when xarray no longer supports versions of numpy older than v1.17. Instead, use numpy ufuncs directly.
     32   xu.log((1.0 / data) * C1 * wavenumber ** 3 + 1.0))
     35 [DEBUG/MainProcess] worker exiting after 0 tasks
     38 /media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:1055: RuntimeWarning: Mean of empty slice
     38   result = {k: np.nanmean(v) for k, v in result.items()}
     56 [DEBUG/MainProcess] worker got sentinel -- exiting
    177 /media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:697: RuntimeWarning: invalid value encountered in less_equal
    177   res = lat <= thres[self.line]
   2770   cth = np.nanmean(cthmargin)
   2770 /media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/algorithms.py:692: RuntimeWarning: Mean of empty slice
   4765 /media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/scipy/optimize/_minimize.py:517: RuntimeWarning: Method BFGS cannot handle constraints nor bounds.
   4765   RuntimeWarning)
   5793 /media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/pyresample/spherical.py:123: RuntimeWarning: invalid value encountered in true_divide
   5793   self.cart /= np.sqrt(np.einsum('...i, ...i', self.cart, self.cart))

I don't know if this is a problem.

@gerritholl
Copy link
Collaborator Author

The code causing the numerous BFGS warnings appears to be:

https://github.com/gerritholl/fogpy/blob/2c9aa9436e4924f55879d52a9de14817d370df5a/fogpy/lowwatercloud.py#L481-L488

@gerritholl
Copy link
Collaborator Author

The invalid value encountered in true_divide warning is in pyresample and triggered because the eurol projection extends to areas not covered by the EUROPE-B area that the NWCSAF files cover.

@gerritholl gerritholl mentioned this issue May 3, 2019
Open
@gerritholl
Copy link
Collaborator Author

The RuntimeWarning: Method BFGS cannot handle constraints nor bounds is one that @m4sth0 has seen before.

@gerritholl
Copy link
Collaborator Author

The RuntimeWarning: Mean of empty slice is caused by #10.

@gerritholl gerritholl mentioned this issue Jun 4, 2019
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gerritholl gerritholl

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@gerritholl