[pypy] Beat errors on startup with "TypeError: expected string"

[joshfriend]

Checklist

• I have included the output of celery -A proj report in the issue.

software -> celery:4.1.0 (latentcall) kombu:4.1.0 py:2.7.13
            billiard:3.5.0.3 py-amqp:2.2.2
platform -> system:Darwin arch:64bit imp:PyPy
loader   -> celery.loaders.default.Loader
settings -> transport:amqp results:disabled

• I have verified that the issue exists against the master branch of Celery.

Steps to reproduce

Celery is passing the beat schedule file name as unicode (u'celerybeat-schedule') to the shelve module, which expects a str:

Removing corrupted schedule file u'celerybeat-schedule': TypeError('expected string',)
Traceback (most recent call last):
  File "/Users/josh/.venv/site-packages/celery/beat.py", line 425, in setup_schedule
    self._store = self._open_schedule()
  File "/Users/josh/.venv/site-packages/celery/beat.py", line 415, in _open_schedule
    return self.persistence.open(self.schedule_filename, writeback=True)
  File "/Users/josh/.pyenv/versions/pypy2.7-5.8.0/lib-python/2.7/shelve.py", line 243, in open
    return DbfilenameShelf(filename, flag, protocol, writeback)
  File "/Users/josh/.pyenv/versions/pypy2.7-5.8.0/lib-python/2.7/shelve.py", line 227, in __init__
    Shelf.__init__(self, anydbm.open(filename, flag), protocol, writeback)
  File "/Users/josh/.pyenv/versions/pypy2.7-5.8.0/lib-python/2.7/anydbm.py", line 85, in open
    return mod.open(file, flag, mode)
  File "/Users/josh/.pyenv/versions/pypy2.7-5.8.0/lib_pypy/dbm.py", line 160, in open
    raise TypeError("expected string")
TypeError: expected string

Downgrading to 3.1.25 fixes the issue. No 4.x release that I tried (or master) will work.

[thedrow]

This might be a PyPy bug since the same condition is used in PyPy3.
str in Python 3 is a UTF-8 encoded string.
We can work around this bug but I'd rather you reported this upstream.

[joshfriend]

That's the conclusion I eventually came to as well, though I had forgotten to report to PyPy.

A fix in celery itself would still be helpful since Heroku does not like to update it's versions of PyPy offered in the python buildpack often (5.8.0 is the latest currently offered).

[thedrow]

I'll try to get to it this weekend.

[joshfriend]

I've filed a bug upstream with PyPy

[thedrow]

I'm unable to reproduce this bug with PyPy 5.8.0. I've written the following test:

    def test_setup_schedule_real_shelve_unicode_filename(self, tmpdir):
        s = beat.PersistentScheduler(app=self.app, schedule_filename=unicode(tmpdir.join('schedule')))
        s.setup_schedule()

Unfortunately I can't get this to fail on my machine. Any instructions would be beneficial towards providing a proper workaround for this problem.

[joshfriend]

Thats really odd. I checked out the source code, added that exact test case to test_beat.py and it failed with exactly the error I got in my application.

I'm on macOS 10.13.3 if that makes a difference. My pythons are installed via pyenv

Here's the test failure from tox pypy:

=== FAILURES ===
___ test_PersistentScheduler.test_setup_schedule_real_shelve_unicode_filename ____

self = <t.unit.app.test_beat.test_PersistentScheduler instance at 0x000000010adb8420>
tmpdir = local('/private/var/folders/gb/vfm9q6ms7lz3p9743myfpq580000gn/T/pytest-of-josh/pytest-0/test_setup_schedule_real_shelv0')

    def test_setup_schedule_real_shelve_unicode_filename(self, tmpdir):
>       s = beat.PersistentScheduler(app=self.app, schedule_filename=unicode(tmpdir.join('schedule')))

t/unit/app/test_beat.py:561:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
celery/beat.py:458: in __init__
    Scheduler.__init__(self, *args, **kwargs)
celery/beat.py:225: in __init__
    self.setup_schedule()
celery/beat.py:484: in setup_schedule
    self._store = self._destroy_open_corrupted_schedule(exc)
celery/beat.py:472: in _destroy_open_corrupted_schedule
    return self._open_schedule()
celery/beat.py:466: in _open_schedule
    return self.persistence.open(self.schedule_filename, writeback=True)
../../../.pyenv/versions/pypy2.7-5.8.0/lib-python/2.7/shelve.py:243: in open
    return DbfilenameShelf(filename, flag, protocol, writeback)
../../../.pyenv/versions/pypy2.7-5.8.0/lib-python/2.7/shelve.py:227: in __init__
    Shelf.__init__(self, anydbm.open(filename, flag), protocol, writeback)
../../../.pyenv/versions/pypy2.7-5.8.0/lib-python/2.7/anydbm.py:85: in open
    return mod.open(file, flag, mode)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

filename = '/private/var/folders/gb/vfm9q6ms7lz3p9743myfpq580000gn/T/pytest-of-josh/pytest-0/test_setup_schedule_real_shelv0/schedule', flag = 'c', mode = 438

    def open(filename, flag='r', mode=0666):
        "open a DBM database"
        if not isinstance(filename, str):
>           raise TypeError("expected string")
E           TypeError: expected string

../../../.pyenv/versions/pypy2.7-5.8.0/lib_pypy/dbm.py:160: TypeError

I can also replicate by creating a virtualenv of pypy2.7-5.8.0 with Celery 4.1.0 installed and running this inside of it:

#!/usr/bin/env python
from __future__ import unicode_literals
from celery import Celery, beat

app = Celery('tasks', broker='pyamqp://guest@localhost//')
s = beat.PersistentScheduler(app=app, schedule_filename='celerybeat-schedule')
s.setup_schedule()

Is there any other info I can provide for you that might be helpful?

[thedrow]

Maybe it's because you are using Mac so dbm is selected over other mechanisms.

[thedrow]

I'm wondering if this could be reproduced in Travis-CI using their MacOS builds.
@joshfriend Care to try?

[auvipy]

closing as per #4559 (comment)