Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vine is not installed as part of the celery's requirements #3547

Closed
1 of 2 tasks
KaySackey opened this issue Nov 5, 2016 · 21 comments
Closed
1 of 2 tasks

vine is not installed as part of the celery's requirements #3547

KaySackey opened this issue Nov 5, 2016 · 21 comments

Comments

@KaySackey
Copy link

KaySackey commented Nov 5, 2016

Checklist

  • I have included the output of celery -A proj report in the issue.
    (if you are not able to do this, then at least specify the Celery
    version affected).
  • I have verified that the issue exists against the master branch of Celery.

Steps to reproduce

pip install celery

Then in python:

from celery import Celery

Expected behavior

Celery running fine.

Actual behavior

  File "/root/.virtualenvs/ceres/lib/python2.7/site-packages/celery/__init__.py", line 149, in <module>
    from . import local  # noqa
  File "/root/.virtualenvs/ceres/lib/python2.7/site-packages/celery/local.py", line 15, in <module>
    from .five import bytes_if_py2, items, string, string_t
  File "/root/.virtualenvs/ceres/lib/python2.7/site-packages/celery/five.py", line 5, in <module>
    import vine.five
ImportError: No module named vine.five
@malinoff
Copy link
Contributor

malinoff commented Nov 7, 2016

Just tried to reproduce on a fresh virtual environment:

$ pip install celery
Collecting celery
  Downloading celery-4.0.0-py2.py3-none-any.whl (395kB)
    100% |████████████████████████████████| 399kB 1.7MB/s
Collecting kombu<5.0,>=4.0 (from celery)
  Downloading kombu-4.0.0-py2.py3-none-any.whl (178kB)
    100% |████████████████████████████████| 184kB 2.8MB/s
Collecting pytz>dev (from celery)
  Downloading pytz-2016.7-py2.py3-none-any.whl (480kB)
    100% |████████████████████████████████| 481kB 2.0MB/s
Collecting billiard<3.6.0,>=3.5.0.2 (from celery)
  Downloading billiard-3.5.0.2-py3-none-any.whl (102kB)
    100% |████████████████████████████████| 102kB 3.0MB/s
Collecting amqp<3.0,>=2.1.1 (from kombu<5.0,>=4.0->celery)
  Downloading amqp-2.1.1-py2.py3-none-any.whl (48kB)
    100% |████████████████████████████████| 51kB 3.1MB/s
Collecting vine>=1.1.3 (from amqp<3.0,>=2.1.1->kombu<5.0,>=4.0->celery)
  Downloading vine-1.1.3-py2.py3-none-any.whl
Installing collected packages: vine, amqp, kombu, pytz, billiard, celery
Successfully installed amqp-2.1.1 billiard-3.5.0.2 celery-4.0.0 kombu-4.0.0 pytz-2016.7 vine-1.1.3

As you can see, vine is installing correctly.

@malinoff malinoff closed this as completed Nov 7, 2016
@ConorMcGee
Copy link

I'm experiencing this now installing celery 4.0.2 in a Debian:jessie Docker container.

What's happening for me is that certain dependencies are not being installed when I include celery as a dependency of a Django project (in the project's setup.py). When I pip install that project (with celery just added to setup.py):
Successfully installed billiard-3.5.0.2 celery-4.0.2 pytz-2016.10
Then when I subsequently pip install celery, I get:
Installing collected packages: vine, amqp, kombu

@nmcalabroso
Copy link

nmcalabroso commented Jan 17, 2017

Just happened to me. We're building with tox.

[2]   File "/home/jenkins/workspace/Warrior/wsomanager-rest-staging@4/.tox/test/local/lib/python2.7/site-packages/celery/__init__.py", line 149, in <module>
[2]     from . import local  # noqa
[2]   File "/home/jenkins/workspace/Warrior/wsomanager-rest-staging@4/.tox/test/local/lib/python2.7/site-packages/celery/local.py", line 15, in <module>
[2]     from .five import bytes_if_py2, items, string, string_t
[2]   File "/home/jenkins/workspace/Warrior/wsomanager-rest-staging@4/.tox/test/local/lib/python2.7/site-packages/celery/five.py", line 5, in <module>
[2]     import vine.five
[2] ImportError: No module named vine.five

setup.py

...
install_requires=[
    ...
    'celery==4.0.2',
    ...
]
...

@ConorMcGee
Copy link

@malinoff Do you not think this worth re-opening?

@malinoff
Copy link
Contributor

I'm willing to re-open and fix this issue, but only if I get the exact steps to reproduce,

@ConorMcGee
Copy link

Ok, I did some more digging and in our case the problem was actually that we had been specifying an older version of kombu in our setup.py before we added celery, and due to a limitation in pip (pypa/pip#775), that conflict isn't detected and the older version was being installed instead of the one specified by celery. Sorry about that!
Hopefully that's of some help to you too @nmcalabroso?

@tgehrs
Copy link

tgehrs commented Mar 3, 2017

running pip check is a nice shortcut to finding any conflicted versions based on the limitation of pip referenced by @mcgeeco

@tclancy
Copy link

tclancy commented Mar 10, 2017

I ran into a similar problem by specifying an out-of-date version of amqp.

@dejlek
Copy link
Contributor

dejlek commented Mar 13, 2017

This "problem" typically happens when people pin dependencies to some old package... pip check is the way to find this problem and fix it.

@tclancy
Copy link

tclancy commented Mar 13, 2017

Sorry, I didn't mean to indicate there was an issue here, just that I found the solution by reading this ticket and wanted to include the package that was causing my problem to help alleviate any future searcher's troubles.

@xtian777x
Copy link

I'm getting this error in a Raspberry Pi 3. I guess it's consequence of the errors I get when installing. Doesn't install completely on a RPi3

@Allan-Nava
Copy link

I got this error:

celery init v10.1.
Using config script: /etc/default/celeryd
Traceback (most recent call last):
  File "/usr/local/bin/celery", line 7, in <module>
    from celery.__main__ import main
  File "/usr/local/lib/python3.5/dist-packages/celery/__init__.py", line 150, in <module>
    from . import local  # noqa
  File "/usr/local/lib/python3.5/dist-packages/celery/local.py", line 17, in <module>
    from .five import PY3, bytes_if_py2, items, string, string_t
  File "/usr/local/lib/python3.5/dist-packages/celery/five.py", line 7, in <module>
    import vine.five
ImportError: No module named 'vine'

@Kurara
Copy link

Kurara commented Sep 16, 2020

Hello,

today I had this issue with celery version 4.3.0

The problem was that celery was installing vine version 4.0.0 I think, but it works with version 1.3.0

@thedrow
Copy link
Member

thedrow commented Sep 17, 2020

Please upgrade to 4.3.1.

@sagargg-zz
Copy link

sagargg-zz commented Sep 23, 2020

Upgrade to 4.3.1. worked for me.

hroberts added a commit to mediacloud/backend that referenced this issue Sep 25, 2020
@ErrorInPersona
Copy link

This happened installing 5.0.0 on a fresh env as well

@thedrow
Copy link
Member

thedrow commented Oct 1, 2020

Please try again, I've yanked the offending Kombu versions.

krassowski added a commit to reimandlab/ActiveDriverDB that referenced this issue Oct 3, 2020
@ErrorInPersona
Copy link

Please try again, I've yanked the offending Kombu versions.

Thanks. I can´t find your commit. Installing the dev version from master does not resolve this issue for me.

@thedrow
Copy link
Member

thedrow commented Oct 3, 2020

It doesn't???
Could something be wrong with your pip version?

@ErrorInPersona
Copy link

So, now it worked. However it now fails due to "No module named 'celery.five'"

@thedrow
Copy link
Member

thedrow commented Oct 4, 2020

See #6369.

openstack-mirroring pushed a commit to openstack/keystone that referenced this issue Nov 2, 2020
The opensuse images tend to be less stable, and the opensuse nodeset was
moved to openSUSE 15.2 which does not work properly with devstack for
the train and lower branches. Ussuri works for the moment, and Victoria+
have been moved to Ubuntu, so this patch is targeted for stable/train
and prior to disable voting for opensuse-based jobs.

To fix the gate, patch also pulls in this backport:

Update amqp lower constraint

Newer versions of kombu rely on updated versions of amqp [0]. Keystone's
lower-constraints job is failing consistently across multiple branches
because it's referencing amqp 2.2.2 when we should be relying on a newer
version (5.0.0+).

You can verify this by using `pip3 check` with the lower-constraints tox
environment.

Without this patch, you'll likely see the test_notifications tests fail
with something like:

  ModuleNotFoundError: No module named 'vine.five'

Which spirals into subsequent failures mocking messaging drivers.

[0] celery/celery#3547

(cherry picked from commit c495defc2bd4f2ef5a38d2a9e4d5aac5653fb406)
(cherry picked from commit 12891a039fb1dae7ba22884cf78ad7d1faa8569d)
(cherry picked from commit d435a915160a9876b58f2a04696d9430790e5cdc)

Change-Id: I2895300556b83e943ce5e5fd72e4b86317a306a9
openstack-mirroring pushed a commit to openstack/keystone that referenced this issue Nov 3, 2020
Newer versions of kombu rely on updated versions of amqp [0]. Keystone's
lower-constraints job is failing consistently across multiple branches
because it's referencing amqp 2.2.2 when we should be relying on a newer
version (5.0.0+).

You can verify this by using `pip3 check` with the lower-constraints tox
environment.

Without this patch, you'll likely see the test_notifications tests fail
with something like:

  ModuleNotFoundError: No module named 'vine.five'

Which spirals into subsequent failures mocking messaging drivers.

This commit also fixes the lower-constraint for PyMySQL.

keystone does not have any lower constraint for PyMySQL so the
latest version 0.10.0 is picked by the job which is failing [1]
In OpenStack, PyMySQL's upper constraint is 0.9.3. Version 0.10.0 is
not tested [2]. Let's add PyMySQL lower constraint also so that we test
lower-constraint job with correct lower version.

Both of these fixes were proposed separate reviews in newer branches,
but due to a circular failure, they're being proposed together. The
amqp fix fails lower-constraints jobs because the PyMySQL issue and vice
versa.

[0] celery/celery#3547
[1]https://zuul.opendev.org/t/openstack/build/3077d96f4fff4b7985cb763d0635d471/log/job-output.txt#621
[2]https://github.com/openstack/requirements/blob/master/upper-constraints.txt#L384

Change-Id: I3834b3b34641c006c70614d5331d292c41f8a346
Closes-Bug: #1888886
(cherry picked from commit c495defc2bd4f2ef5a38d2a9e4d5aac5653fb406)
(cherry picked from commit 12891a039fb1dae7ba22884cf78ad7d1faa8569d)
(cherry picked from commit b7c3458b6f11ade0ce54889ae5f782fbac4a9a2e)
(cherry picked from commit 255b6cdd70db654ce8112cef0b86361c34d9993b)
(cherry picked from commit 3de085b)
(cherry picked from commit 300e79e)
(cherry picked from commit fb7d545)
(cherry picked from commit 32967a8105efd2a0934ab123eb10f33df7f581c5)
thenav56 added a commit to posm/posm-replay-server that referenced this issue Feb 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests