New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pycparser-2.14-py2.py3-none-any.whl causes AssertionError: sorry, but this version only supports 100 named groups #147

Closed
smtakeda opened this Issue Oct 2, 2016 · 18 comments

Comments

Projects
None yet
@smtakeda

smtakeda commented Oct 2, 2016

Looks like pycparser-2.14-py2.py3-none-any.whl is broken or something misconfigured, while pycparser-2.14.tar.gz is ok. After installing a Snowflake Connector for Python, a connector for Snowflake DB, this error happen:

>pip install -U snowflake-connector-python
>python -c "import snowflake.connector"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/t/lib/python2.7/site-packages/snowflake/connector/__init__.py", line 21, in <module>
    from .connection import SnowflakeConnection
  File "/tmp/t/lib/python2.7/site-packages/snowflake/connector/connection.py", line 16, in <module>
    from .cursor import SnowflakeCursor
  File "/tmp/t/lib/python2.7/site-packages/snowflake/connector/cursor.py", line 30, in <module>
    from .file_transfer_agent import (SnowflakeFileTransferAgent)
  File "/tmp/t/lib/python2.7/site-packages/snowflake/connector/file_transfer_agent.py", line 29, in <module>
    from .s3_util import (SnowflakeS3FileEncryptionMaterial, SnowflakeS3Util,
  File "/tmp/t/lib/python2.7/site-packages/snowflake/connector/s3_util.py", line 25, in <module>
    from Crypto.Cipher import AES
  File "/tmp/t/lib/python2.7/site-packages/Crypto/Cipher/__init__.py", line 78, in <module>
    from Crypto.Cipher._mode_ecb import _create_ecb_cipher
  File "/tmp/t/lib/python2.7/site-packages/Crypto/Cipher/_mode_ecb.py", line 29, in <module>
    from Crypto.Util._raw_api import (load_pycryptodome_raw_lib,
  File "/tmp/t/lib/python2.7/site-packages/Crypto/Util/_raw_api.py", line 89, in <module>
    Array = ffi.new("char[1]").__class__.__bases__
  File "/tmp/t/lib/python2.7/site-packages/cffi/api.py", line 248, in new
    cdecl = self._typeof(cdecl)
  File "/tmp/t/lib/python2.7/site-packages/cffi/api.py", line 168, in _typeof
    result = self._typeof_locked(cdecl)
  File "/tmp/t/lib/python2.7/site-packages/cffi/api.py", line 153, in _typeof_locked
    type = self._parser.parse_type(cdecl)
  File "/tmp/t/lib/python2.7/site-packages/cffi/cparser.py", line 448, in parse_type
    return self.parse_type_and_quals(cdecl)[0]
  File "/tmp/t/lib/python2.7/site-packages/cffi/cparser.py", line 451, in parse_type_and_quals
    ast, macros = self._parse('void __dummy(\n%s\n);' % cdecl)[:2]
  File "/tmp/t/lib/python2.7/site-packages/cffi/cparser.py", line 260, in _parse
    ast = _get_parser().parse(csource)
  File "/tmp/t/lib/python2.7/site-packages/cffi/cparser.py", line 40, in _get_parser
    _parser_cache = pycparser.CParser()
  File "/tmp/t/lib/python2.7/site-packages/pycparser/c_parser.py", line 87, in __init__
    outputdir=taboutputdir)
  File "/tmp/t/lib/python2.7/site-packages/pycparser/c_lexer.py", line 66, in build
    self.lexer = lex.lex(object=self, **kwargs)
  File "/tmp/t/lib/python2.7/site-packages/pycparser/ply/lex.py", line 911, in lex
    lexobj.readtab(lextab, ldict)
  File "/tmp/t/lib/python2.7/site-packages/pycparser/ply/lex.py", line 233, in readtab
    titem.append((re.compile(pat, lextab._lexreflags | re.VERBOSE), _names_to_funcs(func_name, fdict)))
  File "/tmp/t/lib/python2.7/re.py", line 194, in compile
    return _compile(pattern, flags)
  File "/tmp/t/lib/python2.7/re.py", line 249, in _compile
    p = sre_compile.compile(pattern, flags)
  File "/tmp/t/lib/python2.7/sre_compile.py", line 583, in compile
    "sorry, but this version only supports 100 named groups"
AssertionError: sorry, but this version only supports 100 named groups

But if I install it from source pycparser-2.14.tar.gz, it works

>pip install -U ~/Downloads/pycparser-2.14.tar.gz
Processing /Users/stakeda/Downloads/pycparser-2.14.tar.gz
Building wheels for collected packages: pycparser
  Running setup.py bdist_wheel for pycparser ... done
  Stored in directory: /Users/stakeda/Library/Caches/pip/wheels/6e/f1/96/de2b8478c77d89fe540a5709a70e2cdc4e7331a3543cada3c1
Successfully built pycparser
Installing collected packages: pycparser
  Found existing installation: pycparser 2.13
    Uninstalling pycparser-2.13:
      Successfully uninstalled pycparser-2.13
Successfully installed pycparser-2.14
>python -c "import snowflake.connector"
>

Could you rebuild a new wheel or remove it? Thanks.

@smtakeda

This comment has been minimized.

Show comment
Hide comment
@smtakeda

smtakeda Oct 2, 2016

Both 2.7.10 and 3.4.3 on Linux (Linux-3.15.1-1.el6.elrepo.x86_64-x86_64-with-centos-6.5-Final) has this issue.

smtakeda commented Oct 2, 2016

Both 2.7.10 and 3.4.3 on Linux (Linux-3.15.1-1.el6.elrepo.x86_64-x86_64-with-centos-6.5-Final) has this issue.

@mriedem

This comment has been minimized.

Show comment
Hide comment
@icereval

This comment has been minimized.

Show comment
Hide comment
@icereval

icereval Oct 3, 2016

Cryptography library is also affected via cffi import.

icereval commented Oct 3, 2016

Cryptography library is also affected via cffi import.

indygreg added a commit to indygreg/python-zstandard that referenced this issue Oct 3, 2016

Pin pycparser by Git commit
As a (hopefully temporary) workaround to
eliben/pycparser#147
@indygreg

This comment has been minimized.

Show comment
Hide comment
@indygreg

indygreg Oct 3, 2016

As a temporary workaround, you can point pip at the Git repo to obtain the sources. The following will point to the commit with the 2.14 tag: git+https://github.com/eliben/pycparser@7f6b3404da3d3184560b04492b64acf5c0ca4ce2#egg=pycparser

indygreg commented Oct 3, 2016

As a temporary workaround, you can point pip at the Git repo to obtain the sources. The following will point to the commit with the 2.14 tag: git+https://github.com/eliben/pycparser@7f6b3404da3d3184560b04492b64acf5c0ca4ce2#egg=pycparser

@bcoover

This comment has been minimized.

Show comment
Hide comment
@bcoover

bcoover Oct 3, 2016

Same issue here, took down an auto-scaling cluster of docker containers on AWS, argh...
That github link did not work, but I pointed to release tag and it works fine as:
pip install git+https://github.com/eliben/pycparser@release_v2.14

bcoover commented Oct 3, 2016

Same issue here, took down an auto-scaling cluster of docker containers on AWS, argh...
That github link did not work, but I pointed to release tag and it works fine as:
pip install git+https://github.com/eliben/pycparser@release_v2.14

@indygreg

This comment has been minimized.

Show comment
Hide comment
@indygreg

indygreg Oct 3, 2016

Another workaround is to add --no-binary pycparser to your requirements file or add said argument to your pip command. That's arguably better, as it will revert to the old behavior (of downloading a .tar.gz from PyPI) and it won't require a Git install.

indygreg commented Oct 3, 2016

Another workaround is to add --no-binary pycparser to your requirements file or add said argument to your pip command. That's arguably better, as it will revert to the old behavior (of downloading a .tar.gz from PyPI) and it won't require a Git install.

openstack-gerrit pushed a commit to openstack/kolla that referenced this issue Oct 3, 2016

Fix pycparser wheel package issue
pycparser-2.14-py2.py3-none-any.whl is broken or something
misconfigured, while pycparser-2.14.tar.gz is ok.  Revert this PS when
the bug is fixed[0]

[0] eliben/pycparser#147

TrivialFix

Change-Id: If719a5fdfbcedfe4770354b18dc47cc94e90dc3d
@alexis-via

This comment has been minimized.

Show comment
Hide comment
@alexis-via

alexis-via Oct 3, 2016

Odoo and OCB are also impacted by this bug. All our travis builds fail: https://travis-ci.org/OCA/OCB/builds/164508458

alexis-via commented Oct 3, 2016

Odoo and OCB are also impacted by this bug. All our travis builds fail: https://travis-ci.org/OCA/OCB/builds/164508458

gurneyalex added a commit to gurneyalex/maintainer-quality-tools that referenced this issue Oct 3, 2016

gurneyalex added a commit to gurneyalex/maintainer-quality-tools that referenced this issue Oct 3, 2016

gurneyalex added a commit to gurneyalex/maintainer-quality-tools that referenced this issue Oct 3, 2016

gurneyalex added a commit to gurneyalex/maintainer-quality-tools that referenced this issue Oct 3, 2016

openstack-gerrit pushed a commit to openstack/openstack-ansible-tests that referenced this issue Oct 3, 2016

Work around bad pycparser wheel
As detailed in [1] the current pycparser fails to install.
This works around the issue by forcing an installation from
source instead of the wheel.

[1] eliben/pycparser#147

Change-Id: I7fd38c4b5c675a8f6f38499c2cdf496a4e99f02d

openstack-gerrit pushed a commit to openstack/tripleo-quickstart that referenced this issue Oct 3, 2016

Yolanda Robla Mota
Temporarily pin pycparser
It fails due to eliben/pycparser#147

Change-Id: Ibdcc51f69e15c59053d490c3d6581cb49b98b077

artem-panchenko added a commit to artem-panchenko/calico that referenced this issue Oct 3, 2016

Fix 'pycparser' module installation issue
The version '2.14' is broken, see [0] for details.

[0] eliben/pycparser#147

Change-Id: I0fca7dcce7da20c0f312310406a8622272483b58

artem-panchenko added a commit to artem-panchenko/calico that referenced this issue Oct 3, 2016

Fix 'pycparser' module installation issue
The version '2.14' is broken, see [0] for details.

[0] eliben/pycparser#147

Change-Id: I0fca7dcce7da20c0f312310406a8622272483b58

FedericoRessi pushed a commit to FedericoRessi/networking-odl-ws that referenced this issue Oct 3, 2016

artem-panchenko added a commit to artem-panchenko/calico that referenced this issue Oct 3, 2016

Fix 'pycparser' module installation issue
The version '2.14' is broken, see [0] for details.

[0] eliben/pycparser#147

Change-Id: I0fca7dcce7da20c0f312310406a8622272483b58

minrk added a commit to minrk/certbot that referenced this issue Oct 3, 2016

disallow binary (wheel) install for pycparser
pycparser has uploaded a wheel, which is currently causing certbot-auto [to fail](https://community.letsencrypt.org/t/certbot-auto-fails-while-setting-up-virtual-environment-complains-about-package-hashes/20529) due to a hash mismatch:

    THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE

because it's downloading the wheel instead of the tarball whose hash is checked. pycparser appears to also have [an issue](eliben/pycparser#147) with the wheel, so rather than add the wheel's hash to the accepted hashes, disallow it, forcing download of the tarball.
@nkarpin

This comment has been minimized.

Show comment
Hide comment
@nkarpin

nkarpin commented Oct 3, 2016

Devstack deployment is broken on Murano CI by this issue also https://murano-ci.mirantis.com/jenkins/job/gate-murano-dashboard-ubuntu-glare-nv/333/console

openstack-gerrit pushed a commit to openstack/openstack-ansible-tests that referenced this issue Oct 3, 2016

Work around bad pycparser wheel
As detailed in [1] the current pycparser fails to install.
This works around the issue by forcing an installation from
source instead of the wheel.

[1] eliben/pycparser#147

Change-Id: I7fd38c4b5c675a8f6f38499c2cdf496a4e99f02d
(cherry picked from commit 2d1fab6)

minrk added a commit to minrk/certbot that referenced this issue Oct 3, 2016

disallow binary (wheel) install for pycparser
pycparser has uploaded a wheel, which is currently causing certbot-auto [to fail](https://community.letsencrypt.org/t/certbot-auto-fails-while-setting-up-virtual-environment-complains-about-package-hashes/20529) due to a hash mismatch:

    THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE

because it's downloading the wheel instead of the tarball whose hash is checked. pycparser appears to also have [an issue](eliben/pycparser#147) with the wheel, so rather than add the wheel's hash to the accepted hashes, disallow it, forcing download of the tarball.

hugovk added a commit to hugovk/Pillow that referenced this issue Oct 3, 2016

rdoproject pushed a commit to rdo-infra/weirdo that referenced this issue Oct 3, 2016

Pin pycparser until upstream issue is resolved
pycparser 2.14 is broken due to
eliben/pycparser#147

Change-Id: I3ed3e1e9aaa359548eb2d6da8ec661c5fc8b9d14

openstack-gerrit added a commit to openstack/openstack that referenced this issue Oct 3, 2016

Updated openstack/openstack
Project: openstack/requirements  ef3a8d23c2fd6b2bcfb66e090bf3d45737bb54d1

Revert "Downgrade pycparser to 2.13"

This reverts commit 45c9e922734396bc22a466f316161046e92495a6.

eliben/pycparser#147 is fixed by
removing the bad wheel from pypi.

Change-Id: Id13203ef802cc4e9cc35e8e092a0b08303fcb44d
Related-Bug: #1629830

openstack-gerrit pushed a commit to openstack/requirements that referenced this issue Oct 3, 2016

Revert "Downgrade pycparser to 2.13"
This reverts commit 45c9e92.

eliben/pycparser#147 is fixed by
removing the bad wheel from pypi.

Change-Id: Id13203ef802cc4e9cc35e8e092a0b08303fcb44d
Related-Bug: #1629830

openstack-gerrit added a commit to openstack/openstack that referenced this issue Oct 3, 2016

Updated openstack/openstack
Project: openstack/requirements  ef3a8d23c2fd6b2bcfb66e090bf3d45737bb54d1

Revert "Downgrade pycparser to 2.13"

This reverts commit 45c9e922734396bc22a466f316161046e92495a6.

eliben/pycparser#147 is fixed by
removing the bad wheel from pypi.

Change-Id: Id13203ef802cc4e9cc35e8e092a0b08303fcb44d
Related-Bug: #1629830
@vstinner

This comment has been minimized.

Show comment
Hide comment
@vstinner

vstinner Oct 3, 2016

It's funny to see so many CI failing after a new release of a Python component :-) It means that more and more code is well tested and that bugs are spotted more quickly. It's a good thing, right?

vstinner commented Oct 3, 2016

It's funny to see so many CI failing after a new release of a Python component :-) It means that more and more code is well tested and that bugs are spotted more quickly. It's a good thing, right?

@bmw

This comment has been minimized.

Show comment
Hide comment
@bmw

bmw Oct 4, 2016

Is the plan now to build a new wheel or skip adding a wheel entirely? If adding a new wheel, will you bump the version number of pycparsing or keep it the same?

Asking because my project currently pins pycparser==2.14 and has pip verify the hash. The addition of a wheel (which pip prefers over sdists) caused this setup to break. Just want to know what the plan is going forward so I can accommodate accordingly.

bmw commented Oct 4, 2016

Is the plan now to build a new wheel or skip adding a wheel entirely? If adding a new wheel, will you bump the version number of pycparsing or keep it the same?

Asking because my project currently pins pycparser==2.14 and has pip verify the hash. The addition of a wheel (which pip prefers over sdists) caused this setup to break. Just want to know what the plan is going forward so I can accommodate accordingly.

@crajaganesh86

This comment has been minimized.

Show comment
Hide comment
@crajaganesh86

crajaganesh86 Oct 4, 2016

May be useful for someone...Our Devstack also failed because of this bug. We just downgraded pycparser to 2.13 in requirement.txt and disabled 'tempest' service in local.conf. After this, our Devstack installation was successful.

crajaganesh86 commented Oct 4, 2016

May be useful for someone...Our Devstack also failed because of this bug. We just downgraded pycparser to 2.13 in requirement.txt and disabled 'tempest' service in local.conf. After this, our Devstack installation was successful.

openstack-gerrit pushed a commit to openstack/openstack-ansible-tests that referenced this issue Oct 4, 2016

Remove workaround for bad pycparser wheel
This reverts https://review.openstack.org/381007 now that the
root cause of the issue [1] has been resolved, and that the
upper-constraints for OpenStack have been updated [2] to work
around the issue.

It also adds a clearer explanation for the reason why
libvirt-python is not installed via a wheel in tests.

[1] eliben/pycparser#147
[2] https://review.openstack.org/381011

Change-Id: I2a4e75d952d4218fa7a0847e6fbf8f6afc15a65b
@eliben

This comment has been minimized.

Show comment
Hide comment
@eliben

eliben Oct 4, 2016

Owner

@bmw: I don't have an objection to having wheels, but this experience also teaches me I have to be more careful with changes to pycparser (especially "release" changes like pushing stuff to PyPi). To be honest I haven't yet heard a very strong reason to add a wheel. pycparser is a pure-Python package, so the advantages of having a wheel seem slim to me, and I haven't heard many complaints about it so far.

So I'm inclined to just keep things as they are and not bump anything in the near future.

Owner

eliben commented Oct 4, 2016

@bmw: I don't have an objection to having wheels, but this experience also teaches me I have to be more careful with changes to pycparser (especially "release" changes like pushing stuff to PyPi). To be honest I haven't yet heard a very strong reason to add a wheel. pycparser is a pure-Python package, so the advantages of having a wheel seem slim to me, and I haven't heard many complaints about it so far.

So I'm inclined to just keep things as they are and not bump anything in the near future.

@indygreg

This comment has been minimized.

Show comment
Hide comment
@indygreg

indygreg Oct 4, 2016

I'd encourage you to post a wheel on the next release. They are the future of Python packaging. And installing source wheels is slightly faster than a tar.gz source distribution. The difference actually matters at scale. Firefox's CI cumulatively spends several hours a day effectively doing busy work around virtualenv setup because running setup.py (and applying wheels) is non-instantaneous. A few seconds multiplied by hundreds of thousands of test jobs per day matters :)

indygreg commented Oct 4, 2016

I'd encourage you to post a wheel on the next release. They are the future of Python packaging. And installing source wheels is slightly faster than a tar.gz source distribution. The difference actually matters at scale. Firefox's CI cumulatively spends several hours a day effectively doing busy work around virtualenv setup because running setup.py (and applying wheels) is non-instantaneous. A few seconds multiplied by hundreds of thousands of test jobs per day matters :)

@eliben

This comment has been minimized.

Show comment
Hide comment
@eliben

eliben Oct 4, 2016

Owner

@indygreg surely you wouldn't rely on package authors to publish wheels then, but have something of your own where you create wheels for projects that don't have them. Otherwise spending so much cycles doesn't sound like a good idea. Or is every project out there except pycparser already publishing wheels?

Owner

eliben commented Oct 4, 2016

@indygreg surely you wouldn't rely on package authors to publish wheels then, but have something of your own where you create wheels for projects that don't have them. Otherwise spending so much cycles doesn't sound like a good idea. Or is every project out there except pycparser already publishing wheels?

@mriedem

This comment has been minimized.

Show comment
Hide comment
@mriedem

mriedem Oct 5, 2016

I can say that OpenStack has been publishing wheels for it's many packages for awhile now.

mriedem commented Oct 5, 2016

I can say that OpenStack has been publishing wheels for it's many packages for awhile now.

openstack-gerrit pushed a commit to openstack/openstack-ansible-tests that referenced this issue Oct 5, 2016

Remove workaround for bad pycparser wheel
This reverts https://review.openstack.org/381007 now that the
root cause of the issue [1] has been resolved, and that the
upper-constraints for OpenStack have been updated [2] to work
around the issue.

It also adds a clearer explanation for the reason why
libvirt-python is not installed via a wheel in tests.

[1] eliben/pycparser#147
[2] https://review.openstack.org/381011

Change-Id: I2a4e75d952d4218fa7a0847e6fbf8f6afc15a65b
(cherry picked from commit 783709a)
@hugovk

This comment has been minimized.

Show comment
Hide comment
@hugovk

hugovk Oct 5, 2016

Contributor

surely you wouldn't rely on package authors to publish wheels then, but have something of your own where you create wheels for projects that don't have them. Otherwise spending so much cycles doesn't sound like a good idea.

If package authors publish wheels, there's no need for package users to create their own special wheels for their CIs, it reduces duplicated effort for those who want them, and end users also benefit.

Or is every project out there except pycparser already publishing wheels?

There's a big benefit in wheels for Python projects needing to compile C, especially for Windows where setting up a dev environment with a compiler can be a real problem. But for pure Python, source-only isn't generally such a big problem.

Heres's some benefits, and stats on the most-downloaded packages on PyPI (which includes pycparser): 226/360 use wheels. http://pythonwheels.com/


On our CI, pycparser is installed as part of pip install cffi. Different Python versions had different combinations of wheel and source available. Here's how many seconds the combined download and installation operation took.

cffi and pycparser wheels:

0.80
0.92
0.89
0.93

cffi wheel and pycparser source:

2.21
1.97
1.51

cffi source and pycparser wheel:

4.34
4.69
5.76

cffi and pycparser source:

7.30
6.27
5.27
5.61

Contributor

hugovk commented Oct 5, 2016

surely you wouldn't rely on package authors to publish wheels then, but have something of your own where you create wheels for projects that don't have them. Otherwise spending so much cycles doesn't sound like a good idea.

If package authors publish wheels, there's no need for package users to create their own special wheels for their CIs, it reduces duplicated effort for those who want them, and end users also benefit.

Or is every project out there except pycparser already publishing wheels?

There's a big benefit in wheels for Python projects needing to compile C, especially for Windows where setting up a dev environment with a compiler can be a real problem. But for pure Python, source-only isn't generally such a big problem.

Heres's some benefits, and stats on the most-downloaded packages on PyPI (which includes pycparser): 226/360 use wheels. http://pythonwheels.com/


On our CI, pycparser is installed as part of pip install cffi. Different Python versions had different combinations of wheel and source available. Here's how many seconds the combined download and installation operation took.

cffi and pycparser wheels:

0.80
0.92
0.89
0.93

cffi wheel and pycparser source:

2.21
1.97
1.51

cffi source and pycparser wheel:

4.34
4.69
5.76

cffi and pycparser source:

7.30
6.27
5.27
5.61

openstack-gerrit pushed a commit to openstack/rally that referenced this issue Oct 12, 2016

Remove pycparser restriction from requirements
The assertion error mentioned to justify the restriction is
not actually due to this version of the python package, but
instead due to an improperly prepared wheel which was
uploaded to pypi. This wheel has now been removed and the
package works properly.

Reference: eliben/pycparser#147

This patch removes the restriction so that Rally's
requirements are back in line with the OpenStack upper
constraint for pycparser.

Change-Id: I66ef46ad4c1511f39817270f59c7c1511645f2eb
Related-Bug: #1629830
Related-Bug: #1629726

mpeuster added a commit to mpeuster/son-cli that referenced this issue Oct 18, 2016

asimonet pushed a commit to asimonet/rally that referenced this issue Nov 4, 2016

Remove pycparser restriction from requirements
The assertion error mentioned to justify the restriction is
not actually due to this version of the python package, but
instead due to an improperly prepared wheel which was
uploaded to pypi. This wheel has now been removed and the
package works properly.

Reference: eliben/pycparser#147

This patch removes the restriction so that Rally's
requirements are back in line with the OpenStack upper
constraint for pycparser.

Change-Id: I66ef46ad4c1511f39817270f59c7c1511645f2eb
Related-Bug: #1629830
Related-Bug: #1629726

Fahad-Alsaidi added a commit to Fahad-Alsaidi/Pillow that referenced this issue Dec 13, 2016

Fahad-Alsaidi added a commit to Fahad-Alsaidi/Pillow that referenced this issue Dec 13, 2016

lasley added a commit to LasLabs/maintainer-quality-tools that referenced this issue Jun 18, 2017

Gimpneek added a commit to NeovaHealth/maintainer-quality-tools that referenced this issue Sep 8, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment