Thoughts on dropping support for Python 2.7

[andersy005]

@matt-long, @mnlevy1981, @kmpaul:

I would like to know your thoughts on dropping Python 2.7 suppport in future versions of esmlab.
I am currently working on integrating some functionality from xarray-extras namely interpolations and integrations functions (see #25) in esmlab. However, xarray-extras is only Python 3.5+ compatible. Since it is 2019 already, it is likely that the number of upstream packages that will drop support for Python 2.7 will definitely increase, making Python 2.7 support a painful exercise for us.

@mnlevy1981, I haven't forgotten the fact that cesm_postprocessing still relies on Python2.7 and that you'd like to use some features from esmlab in cesm_postprocessing. I am open to ideas on how to deal with this issue.

[mnlevy1981]

How widespread do you think the usage of xarray-extras would be? Would some of esmlab still be accessible to python 2.7 users? My preference would be for you to wait until we have a clear roadmap of what is going into esmlab / what will be used in the post processing, but on the other hand I don't know how long it will take to come up with the latter and I don't want to hold up your development indefinitely. I don't think I'll have more info by tomorrow's meeting, but what if I aim to have a more concrete idea of what we want out of esmlab by the January 17 meeting and we figure out if that's compatible with using xarray-extras in other areas of the code?

Both converting the postprocessing tools to be py3 compliant and updating the way the environment is built to provide py3 will be time-consuming tasks, but I'll do all I can to push for that to happen ASAP.

[matt-long]

In general I am in favor of dropping Python 2 support, except for this issue with the cesm_postprocessing tools.

[andersy005]

@mnlevy1981, components from xarray-extras are not really a priority for the time being. xarray-extras itself is an evolving package. So, I can postpone this integration until we figure out how to deal with Python 2.7 support. Since we want to avoid reinventing the wheel when it comes to esmlab features, this implies that we have to deal with restrictions from upstream packages esmlab ends up depending on. I have a feeling that there's probably some other packages (that don't support Python 2.7) we might end up wanting to use in the future. So, I just thought I would bring this issue so that we can discuss it.

Would some of esmlab still be accessible to python 2.7 users?

Yes, the latest version: v2019.1.0

I understand and agree with you regarding this.

Both converting the postprocessing tools to be py3 compliant and updating the way the environment is built to provide py3 will be time-consuming tasks, but I'll do all I can to push for that to happen ASAP.

Speaking of cesm_postprocessing, are there any other additional features useful in cesm_postprocessing that you would like us to add to esmlab? @matt-long, @mnlevy1981

[andersy005]

Xarray seems to be broken for:

• Python 2.7
• NumPy 1.16.0
• Xarray 0.11.2

This seems to be connected to xarray are not compatible with latest numpy

xref: https://stackoverflow.com/questions/54204247/attributeerror-numpy-ufunc-object-has-no-attribute-module-for-skimage-d

root@01294942813e:~# python
Python 2.7.15 | packaged by conda-forge | (default, Nov 28 2018, 18:42:13)
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import xarray as xr
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/conda/envs/esmlab-dev/lib/python2.7/site-packages/xarray/__init__.py", line 10, in <module>
    from .core.alignment import align, broadcast, broadcast_arrays
  File "/opt/conda/envs/esmlab-dev/lib/python2.7/site-packages/xarray/core/alignment.py", line 10, in <module>
    from . import utils
  File "/opt/conda/envs/esmlab-dev/lib/python2.7/site-packages/xarray/core/utils.py", line 16, in <module>
    from .pycompat import (
  File "/opt/conda/envs/esmlab-dev/lib/python2.7/site-packages/xarray/core/pycompat.py", line 73, in <module>
    import dask.array
  File "/opt/conda/envs/esmlab-dev/lib/python2.7/site-packages/dask/array/__init__.py", line 9, in <module>
    from .routines import (take, choose, argwhere, where, coarsen, insert,
  File "/opt/conda/envs/esmlab-dev/lib/python2.7/site-packages/dask/array/routines.py", line 256, in <module>
    @wraps(np.matmul)
  File "/opt/conda/envs/esmlab-dev/lib/python2.7/functools.py", line 33, in update_wrapper
    setattr(wrapper, attr, getattr(wrapped, attr))
AttributeError: 'numpy.ufunc' object has no attribute '__module__'

[andersy005]

@mnlevy1981, Since xarray dropped support for Python 2.7 starting with xarray v.0.11.3, I was wondering if it would be okay for us to drop Python2.7 in the next release of esmlab? Would any of your tools break if you were to pin the esmlab version to most recent, stable version (v2019.3.16)? I've started noticing that there's quite a lot of new xarray features that are not supported in recent releases of xarray. For instance, resample() on CFtimeIndex works for Python 3 only!

[matt-long]

I don't see a problem with pinning to the most recent stable version and dropping 2.7 support moving forward.

[mnlevy1981]

I'm okay with tagging the last 2.7-compatible version and only maintaining py3 compatibility moving forward... hopefully our MARBL diagnostics can drop python 2 support soon as well.