[FIX] requirements: python3.7 compatibility #25841

moylop260 · 2018-07-18T11:12:29Z

Continuation of #25783
Part of #25578

UPDATED:

Gevent

python3.7 -m pip install --user gevent==1.1.2
Collecting gevent==1.1.2
  Using cached https://files.pythonhosted.org/packages/43/8f/cb3224a0e6ab663547f45c10d0651cfd52633fde4283bf68d627084df8cc/gevent-1.1.2.tar.gz
Requirement already satisfied: greenlet>=0.4.9 in /Users/moylop260/Library/Python/3.7/lib/python/site-packages (from gevent==1.1.2) (0.4.14)
Installing collected packages: gevent
  Found existing installation: gevent 1.3.5
    Uninstalling gevent-1.3.5:
      Successfully uninstalled gevent-1.3.5
  Running setup.py install for gevent ... error
    Complete output from command /opt/local/bin/python3.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/zc/1m37yppx4z5596yfyzg385140000gn/T/pip-install-16if7_hc/gevent/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /private/var/folders/zc/1m37yppx4z5596yfyzg385140000gn/T/pip-record-d5iu4xes/install-record.txt --single-version-externally-managed --compile --user --prefix=:
...
    /opt/local/Library/Frameworks/Python.framework/Versions/3.7/include/python3.7m/pystate.h:238:15: note: 'curexc_traceback' declared here
        PyObject *curexc_traceback;
                  ^
    gevent/gevent.corecext.c:45489:13: error: no member named 'exc_type' in 'struct _ts'
        tstate->exc_type = *type;
        ~~~~~~  ^
    fatal error: too many errors emitted, stopping now [-ferror-limit=]
    6 warnings and 20 errors generated.
    error: command '/usr/bin/clang' failed with exit status 1

    ----------------------------------------
  Rolling back uninstall of gevent
Command "/opt/local/bin/python3.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/zc/1m37yppx4z5596yfyzg385140000gn/T/pip-install-16if7_hc/gevent/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /private/var/folders/zc/1m37yppx4z5596yfyzg385140000gn/T/pip-record-d5iu4xes/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /private/var/folders/zc/1m37yppx4z5596yfyzg385140000gn/T/pip-install-16if7_hc/gevent/

python3.7 -m pip install --user -U --force-reinstall gevent==1.3.5
Collecting gevent==1.3.5
  Using cached https://files.pythonhosted.org/packages/cf/f6/cf76540f405d4f6d936286b5369aa047b2844c5ce4450457fd23160d7c02/gevent-1.3.5-cp37-cp37m-macosx_10_9_x86_64.whl
Collecting greenlet>=0.4.13; platform_python_implementation == "CPython" (from gevent==1.3.5)
  Using cached https://files.pythonhosted.org/packages/5d/82/2e53a8def6f99db51992ca3a0a2448c3bbec1a9db3a7cbf7d5dad011e138/greenlet-0.4.14.tar.gz
Installing collected packages: greenlet, gevent
  Found existing installation: greenlet 0.4.14
    Uninstalling greenlet-0.4.14:
      Successfully uninstalled greenlet-0.4.14
  Running setup.py install for greenlet ... done
  Found existing installation: gevent 1.3.5
    Uninstalling gevent-1.3.5:
      Successfully uninstalled gevent-1.3.5
Successfully installed gevent-1.3.5 greenlet-0.4.14

lxml

python3.7 -m pip install --user -U --force-reinstall lxml==3.7.1
Collecting lxml==3.7.1
  Using cached https://files.pythonhosted.org/packages/c4/68/cf0ab7e26de58d14d441f19f7f9c2ab15eb109b0b2640f8b19c1da34e9e0/lxml-3.7.1.tar.gz
Installing collected packages: lxml
  Found existing installation: lxml 4.2.3
    Uninstalling lxml-4.2.3:
      Successfully uninstalled lxml-4.2.3
  Running setup.py install for lxml ... error
    Complete output from command /opt/local/bin/python3.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/zc/1m37yppx4z5596yfyzg385140000gn/T/pip-install-w0n198cm/lxml/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /private/var/folders/zc/1m37yppx4z5596yfyzg385140000gn/T/pip-record-aizd5dgp/install-record.txt --single-version-externally-managed --compile --user --prefix=:
    Building lxml version 3.7.1.
...
yzg385140000gn/T/xmlXPathInituj2g9pmx.c -o var/folders/zc/1m37yppx4z5596yfyzg385140000gn/T/xmlXPathInituj2g9pmx.o
    /var/folders/zc/1m37yppx4z5596yfyzg385140000gn/T/xmlXPathInituj2g9pmx.c:2:1: warning: type specifier missing, defaults to 'int' [-Wimplicit-int]
    main (int argc, char **argv) {
    ^
    1 warning generated.
    cc var/folders/zc/1m37yppx4z5596yfyzg385140000gn/T/xmlXPathInituj2g9pmx.o -L/opt/local/lib -lxml2 -o a.out
    error: command '/usr/bin/clang' failed with exit status 1

    ----------------------------------------
  Rolling back uninstall of lxml
Command "/opt/local/bin/python3.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/zc/1m37yppx4z5596yfyzg385140000gn/T/pip-install-w0n198cm/lxml/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /private/var/folders/zc/1m37yppx4z5596yfyzg385140000gn/T/pip-record-aizd5dgp/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /private/var/folders/zc/1m37yppx4z5596yfyzg385140000gn/T/pip-install-w0n198cm/lxml/

python3.7 -m pip install --user -U --force-reinstall lxml==4.2.3
Collecting lxml==4.2.3
  Using cached https://files.pythonhosted.org/packages/54/a6/43be8cf1cc23e3fa208cab04ba2f9c3b7af0233aab32af6b5089122b44cd/lxml-4.2.3.tar.gz
Installing collected packages: lxml
  Found existing installation: lxml 4.2.3
    Uninstalling lxml-4.2.3:
      Successfully uninstalled lxml-4.2.3
  Running setup.py install for lxml ... done
Successfully installed lxml-4.2.3

Greenlet

python3.7 -m pip install --user -U --force-reinstall greenlet==0.4.10
Collecting greenlet==0.4.10
  Using cached https://files.pythonhosted.org/packages/f2/3f/09412b656067f280cf017ce5b6465e6339089129212425111117be5557d9/greenlet-0.4.10.tar.gz
gevent 1.3.5 has requirement greenlet>=0.4.13; platform_python_implementation == "CPython", but you'll have greenlet 0.4.10 which is incompatible.
Installing collected packages: greenlet
  Found existing installation: greenlet 0.4.14
    Uninstalling greenlet-0.4.14:
      Successfully uninstalled greenlet-0.4.14
  Running setup.py install for greenlet ... error
    Complete output from command /opt/local/bin/python3.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/zc/1m37yppx4z5596yfyzg385140000gn/T/pip-install-8ppkrc4r/greenlet/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /private/var/folders/zc/1m37yppx4z5596yfyzg385140000gn/T/pip-record-5lwb2_6v/install-record.txt --single-version-externally-managed --compile --user --prefix=:
....
/opt/local/Library/Frameworks/Python.framework/Versions/3.7/include/python3.7m/pystate.h:238:15: note: 'curexc_traceback' declared here
        PyObject *curexc_traceback;
                  ^
    6 errors generated.
    error: command '/usr/bin/clang' failed with exit status 1

    ----------------------------------------
  Rolling back uninstall of greenlet
Command "/opt/local/bin/python3.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/zc/1m37yppx4z5596yfyzg385140000gn/T/pip-install-8ppkrc4r/greenlet/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /private/var/folders/zc/1m37yppx4z5596yfyzg385140000gn/T/pip-record-5lwb2_6v/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /private/var/folders/zc/1m37yppx4z5596yfyzg385140000gn/T/pip-install-8ppkrc4r/greenlet/

python3.7 -m pip install --user -U --force-reinstall gevent==1.3.5
Collecting gevent==1.3.5
  Using cached https://files.pythonhosted.org/packages/cf/f6/cf76540f405d4f6d936286b5369aa047b2844c5ce4450457fd23160d7c02/gevent-1.3.5-cp37-cp37m-macosx_10_9_x86_64.whl
Collecting greenlet>=0.4.13; platform_python_implementation == "CPython" (from gevent==1.3.5)
  Using cached https://files.pythonhosted.org/packages/5d/82/2e53a8def6f99db51992ca3a0a2448c3bbec1a9db3a7cbf7d5dad011e138/greenlet-0.4.14.tar.gz
Installing collected packages: greenlet, gevent
  Found existing installation: greenlet 0.4.14
    Uninstalling greenlet-0.4.14:
      Successfully uninstalled greenlet-0.4.14
  Running setup.py install for greenlet ... done
  Found existing installation: gevent 1.3.5
    Uninstalling gevent-1.3.5:
      Successfully uninstalled gevent-1.3.5
Successfully installed gevent-1.3.5 greenlet-0.4.14

Continuation of odoo#25783 Part of odoo#25578

pedrobaeza · 2018-07-18T11:14:39Z

I don't think this is going to be accepted, as they use the library versions of latest Debian stable. Are they needed because the libraries versions are not compatible with Python 3.7?

d-fence · 2018-07-18T11:44:34Z

I'm sorry to say that @pedrobaeza is true.

python3-gevent is 1.1.2 in stretch and only 1.3.4 in buster
python3-greenlet is 0.4.11 in stretch
python3-lxml is 3.7.1 in stretch

python3-yaml may probably be removed.

So, it's true that we try to follow Debian stable python3-* packages version as smuch as possible.

The good news is that python3.7 will be available in Buster.

moylop260 · 2018-07-18T11:58:48Z

@d-fence

We can use the same case like lxml

lxml==3.7.1 ; sys_platform != 'win32' and python_version < '3.7'
lxml==4.2.3 ; sys_platform != 'win32' and python_version >= '3.7'

Notice the ... and python_version >= '3.7'
This updated version is just used if you are using python3.7 instead of show the error of the main description.

If you are using the official python3.5 version then you will use the official lxml too.

moylop260 · 2018-07-18T12:01:04Z

I mean,

diff --git a/requirements.txt b/requirements.txt
index 27127c70862..6919450cba8 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -3,11 +3,14 @@ decorator==4.0.10
 docutils==0.12
 ebaysdk==2.1.5
 feedparser==5.2.1
-gevent==1.1.2 ; sys_platform != 'win32'
-greenlet==0.4.10
+gevent==1.1.2 ; sys_platform != 'win32' and python_version < '3.7'
+gevent==1.3.5 ; sys_platform != 'win32' and python_version >= '3.7'
+greenlet==0.4.10 ; python_version < '3.7'
+greenlet==0.4.14 ; python_version >= '3.7'
 html2text==2016.9.19
 Jinja2==2.8
-lxml==3.7.1 ; sys_platform != 'win32'
+lxml==3.7.1 ; sys_platform != 'win32' and python_version < '3.7'
+lxml==4.2.3 ; sys_platform != 'win32' and python_version >= '3.7'
 lxml ; sys_platform == 'win32'
 Mako==1.0.4
 MarkupSafe==0.23

d-fence · 2018-07-18T12:04:35Z

Looks good to me, I closed a bit quickly, sorry.

moylop260 · 2018-07-18T12:19:14Z

In fact, using docker with debian:buster is not working the current version of greenlet then they will need to bump the version in order to be compatible with python3.7 too.

I just confirm the following versions are valid for buster and python3.7:

lxml==4.2.3
gevent==1.3.4

The greenlet official for buster is greenlet==0.4.12 but is failing minimal required for py3.7 is greenlet==0.4.13

With this information the final requirements should be:

diff --git a/requirements.txt b/requirements.txt
index 27127c70862..96736e9917d 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -3,11 +3,14 @@ decorator==4.0.10
 docutils==0.12
 ebaysdk==2.1.5
 feedparser==5.2.1
-gevent==1.1.2 ; sys_platform != 'win32'
-greenlet==0.4.10
+gevent==1.1.2 ; sys_platform != 'win32' and python_version < '3.7'
+gevent==1.3.4 ; sys_platform != 'win32' and python_version >= '3.7'
+greenlet==0.4.10 ; python_version < '3.7'
+greenlet==0.4.13 ; python_version >= '3.7'
 html2text==2016.9.19
 Jinja2==2.8
-lxml==3.7.1 ; sys_platform != 'win32'
+lxml==3.7.1 ; sys_platform != 'win32' and python_version < '3.7'
+lxml==4.2.3 ; sys_platform != 'win32' and python_version >= '3.7'
 lxml ; sys_platform == 'win32'
 Mako==1.0.4
 MarkupSafe==0.23

Since af9d6b8 and 5a57c29 , python3.7 can be used to run Odoo. This commit ensures that python3.7 compatibles versions of gevent, greenlet and lxml will be installed without breaking compatibility with Debian Stretch packages versions when using the Debian packaged Python. Closes #25841

d-fence · 2018-07-18T13:12:06Z

Merged here: 27c017a
Thanks for the contribution.

Fixed from odoo/odoo#25841

moylop260 · 2018-07-20T12:02:57Z

Hi @d-fence
Thanks for merging

I'm using a travis build with python3.7 and I can confirm that installing PyYAML==3.12 raise an error using python3.7 (Notice the red result of the travis build).

But installing PyYAML==3.13 the error is fixed (Notice the green result of the travis build).

Could you apply the following patch, please?

iff --git a/requirements.txt b/requirements.txt
index 96736e9917d..ceee9cc8cd9 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -29,7 +29,8 @@ pyserial==3.1.1
 python-dateutil==2.5.3
 pytz==2016.7
 pyusb==1.0.0
-PyYAML==3.12
+PyYAML==3.12 ; python_version < '3.7'
+PyYAML==3.13 ; python_version >= '3.7'
 qrcode==5.3
 reportlab==3.3.0
 requests==2.11.1

* [REF] odoo-shippable: Add link symbolic python3.7-dev virtual environment for compatibility with travis.yml https://docs.travis-ci.com/user/languages/python/#Specifying-Python-versions * [REF] odoo-shippable: Remove custom version for 3.7 Fixed from odoo/odoo#25841

moylop260 · 2018-08-24T22:59:47Z

FYI I just have created the following PR with the last package for full compatibility in python3.7:

[FIX] requirements: improve python3.7 compatibility #26587

yajo · 2020-05-27T11:01:18Z

@moylop260 haven't you experienced gevent cpu leaks, exposing the symptoms in #50861, whlie using these pinned versions of gevent combined with python 3.7?

moylop260 · 2020-05-27T17:26:05Z

Hi @yajo
Sorry, I'm not using python3.7 in production environments.
We are using py3.7 just for I+D purposes in test environments.

I haven't read your issue before...
I will read it

[FIX] requirements: python3.7 compatibility

876af52

Continuation of odoo#25783 Part of odoo#25578

mart-e requested a review from d-fence July 18, 2018 11:24

d-fence closed this Jul 18, 2018

moylop260 added a commit to vauxoo-dev/docker-odoo-image that referenced this pull request Jul 20, 2018

[REF] odoo-shippable: Remove custom version for 3.7

aea2799

Fixed from odoo/odoo#25841

moylop260 mentioned this pull request Jul 20, 2018

[REF] odoo-shippable: py3.7 stuffs Vauxoo/docker-odoo-image#321

Merged

This was referenced Jul 20, 2018

[REF] .travis.yml: Use python 3.7 Vauxoo/maintainer-quality-tools#284

Merged

[REF] .travis.yml: Use python 3.7 OCA/maintainer-quality-tools#562

Merged

moylop260 referenced this pull request in OCA/maintainer-quality-tools Jul 20, 2018

[REF] travis_install_nightly: Installing pyyaml compatible with py3.7

01d0816

moylop260 mentioned this pull request Jul 24, 2018

Bump PyYAML version #25937

Closed

moylop260 mentioned this pull request Aug 7, 2018

[11.0] Unable to install PyYAML==3.12 on newest Arch Linux (Python 3.7.0, GCC 8.2.0) #26197

Closed

moylop260 mentioned this pull request Aug 24, 2018

[FIX] requirements: improve python3.7 compatibility #26587

Merged

moylop260 mentioned this pull request Sep 13, 2018

[ADD] Add Support for odoo 12.0 Vauxoo/docker-odoo-image#325

Merged

moylop260 mentioned this pull request Jul 29, 2020

[REF] .travis.yml,etc: Deprecate Python 2.7 & 3.4 Vauxoo/travis2docker#131

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FIX] requirements: python3.7 compatibility #25841

[FIX] requirements: python3.7 compatibility #25841

moylop260 commented Jul 18, 2018 •

edited

pedrobaeza commented Jul 18, 2018

d-fence commented Jul 18, 2018

moylop260 commented Jul 18, 2018 •

edited

moylop260 commented Jul 18, 2018

d-fence commented Jul 18, 2018

moylop260 commented Jul 18, 2018 •

edited

d-fence commented Jul 18, 2018

moylop260 commented Jul 20, 2018 •

edited

moylop260 commented Aug 24, 2018

yajo commented May 27, 2020

moylop260 commented May 27, 2020

[FIX] requirements: python3.7 compatibility #25841

[FIX] requirements: python3.7 compatibility #25841

Conversation

moylop260 commented Jul 18, 2018 • edited

Gevent

lxml

Greenlet

pedrobaeza commented Jul 18, 2018

d-fence commented Jul 18, 2018

moylop260 commented Jul 18, 2018 • edited

moylop260 commented Jul 18, 2018

d-fence commented Jul 18, 2018

moylop260 commented Jul 18, 2018 • edited

d-fence commented Jul 18, 2018

moylop260 commented Jul 20, 2018 • edited

moylop260 commented Aug 24, 2018

yajo commented May 27, 2020

moylop260 commented May 27, 2020

moylop260 commented Jul 18, 2018 •

edited

moylop260 commented Jul 18, 2018 •

edited

moylop260 commented Jul 18, 2018 •

edited

moylop260 commented Jul 20, 2018 •

edited