UnicodeDecodeError: 'utf8' codec can't decode byte 0xf5 in position 0: invalid start byte #2039
Comments
|
I have the same problem... :( amqp==1.4.5 (celery)vagrant@debian-squeeze-amd64:/vagrant$ celery -A tasks worker The pickle serializer is a security concern as it may give attackers If you depend on pickle then you should set a setting to disable this You must only enable the serializers that you will actually use. warnings.warn(CDeprecationWarning(W_PICKLE_DEPRECATED)) -------------- celery@debian-squeeze-amd64 v3.1.11 (Cipater)
[2014-05-15 10:46:24,298: ERROR/MainProcess] Unrecoverable error: UnicodeDecodeError('utf8', '\xf5', 0, 1, 'invalid start byte') |
|
I have the same problem... :( amqp==1.4.5 Python 2.6.6 Distributor ID: Debian (celery)vagrant@debian-squeeze-amd64:/vagrant$ celery -A tasks worker The pickle serializer is a security concern as it may give attackers If you depend on pickle then you should set a setting to disable this You must only enable the serializers that you will actually use. warnings.warn(CDeprecationWarning(W_PICKLE_DEPRECATED)) -------------- celery@debian-squeeze-amd64 v3.1.11 (Cipater)
[2014-05-15 10:46:24,298: ERROR/MainProcess] Unrecoverable error: UnicodeDecodeError('utf8', '\xf5', 0, 1, 'invalid start byte') |
|
This means the connection is blocked, probably because of low memory/disk space alerts. See the rabbitmq logs. Not quite sure why the argument to |
|
@Federico-Comesana @hebertjulio Hi guys! I'm having same issue. Did you guys fixed that? |
|
For me this is not related to the connection being blocked, because ignoring utf-8 errors means it can connect and execute tasks (or I'm missing the issue, perhaps that Anyway, replace |
I'm getting started with Celery.
Y execute on shell (virt_env activated) command: python manage.py celeryd --verbosity=2 --loglevel=DEBUG
It's seems start connection but fails with error...
I have:
OS: Debian 6
Python 2.6
Django 1.6.1
My PIP Freeze:
Django==1.6.1
MySQL-python==1.2.4b4
PIL==1.1.7
South==0.8.4
amqp==1.4.5
amqplib==1.0.2
anyjson==0.3.3
billiard==3.3.0.17
celery==3.1.11
defusedxml==0.4.1
django-bakery==0.2.2
django-celery==3.1.10
django-debug-toolbar==1.1
django-debug-toolbar-template-timings==0.6.1
django-password-reset==0.6
django-social-auth==0.7.28
djangorestframework==2.3.13
gdata==2.0.18
hachoir-core==1.3.3
hachoir-metadata==1.3.3
hachoir-parser==1.3.4
httplib2==0.8
importlib==1.0.3
kombu==3.0.16
mutagen==1.22
oauth2==1.5.211
oauthlib==0.6.1
ordereddict==1.1
python-memcached==1.53
python-openid==2.2.5
python-twitter==1.3.1
pytz==2014.2
rarfile==2.6
requests==2.1.0
requests-oauthlib==0.4.0
simplejson==3.4.1
sorl-thumbnail==11.12
sqlparse==0.1.11
twython==3.1.2
wsgiref==0.1.2
My Traceback:
[2014-05-14 21:20:09,426: DEBUG/MainProcess] | Worker: Preparing bootsteps.
[2014-05-14 21:20:09,430: DEBUG/MainProcess] | Worker: Building graph...
[2014-05-14 21:20:09,431: DEBUG/MainProcess] | Worker: New boot order: {StateDB, Timer, Hub, Queues (intra), Pool, Autoscaler, Autoreloader, Beat, Consumer}
[2014-05-14 21:20:09,439: DEBUG/MainProcess] | Consumer: Preparing bootsteps.
[2014-05-14 21:20:09,439: DEBUG/MainProcess] | Consumer: Building graph...
[2014-05-14 21:20:09,447: DEBUG/MainProcess] | Consumer: New boot order: {Connection, Events, Mingle, Gossip, Tasks, Control, Heart, Agent, event loop}
[2014-05-14 21:20:09,448: WARNING/MainProcess] /home/federico/overactive/lib/python2.6/site-packages/celery/apps/worker.py:161: CDeprecationWarning:
Starting from version 3.2 Celery will refuse to accept pickle by default.
The pickle serializer is a security concern as it may give attackers
the ability to execute any command. It's important to secure
your broker from unauthorized access when using pickle, so we think
that enabling pickle should require a deliberate action and not be
the default choice.
If you depend on pickle then you should set a setting to disable this
warning and to be sure that everything will continue working
when you upgrade to Celery 3.2::
You must only enable the serializers that you will actually use.
warnings.warn(CDeprecationWarning(W_PICKLE_DEPRECATED))
-------------- celery@debian v3.1.11 (Cipater)
---- **** -----
--- * *** * -- Linux-2.6.32-5-amd64-x86_64-with-debian-6.0.9
-- * - **** ---
-- ******* ----
--- ***** ----- [queues]
-------------- .> celery exchange=celery(direct) key=celery
[tasks]
. celery.backend_cleanup
. celery.chain
. celery.chord
. celery.chord_unlock
. celery.chunks
. celery.group
. celery.map
. celery.starmap
[2014-05-14 21:20:09,451: DEBUG/MainProcess] | Worker: Starting Hub
[2014-05-14 21:20:09,452: DEBUG/MainProcess] ^-- substep ok
[2014-05-14 21:20:09,452: DEBUG/MainProcess] | Worker: Starting Pool
[2014-05-14 21:20:09,464: DEBUG/MainProcess] ^-- substep ok
[2014-05-14 21:20:09,478: DEBUG/MainProcess] | Worker: Starting Consumer
[2014-05-14 21:20:09,479: DEBUG/MainProcess] | Consumer: Starting Connection
[2014-05-14 21:20:09,525: DEBUG/MainProcess] Start from server, version: 8.0, properties: {u'platform': u'Erlang/OTP', u'product': u'RabbitMQ', u'version': u'1.8.1', u'copyright': u'Copyright (C) 2007-2010 LShift Ltd., Cohesive Financial Technologies LLC., and Rabbit Technologies Ltd.', u'information': u'Licensed under the MPL. See http://www.rabbitmq.com/'}, mechanisms: [u'PLAIN', u'AMQPLAIN'], locales: [u'en_US']
[2014-05-14 21:20:09,526: DEBUG/MainProcess] Open OK!
[2014-05-14 21:20:09,527: INFO/MainProcess] Connected to amqp://guest@127.0.0.1:5672//
[2014-05-14 21:20:09,527: DEBUG/MainProcess] ^-- substep ok
[2014-05-14 21:20:09,531: DEBUG/MainProcess] | Consumer: Starting Events
[2014-05-14 21:20:09,584: DEBUG/MainProcess] Start from server, version: 8.0, properties: {u'platform': u'Erlang/OTP', u'product': u'RabbitMQ', u'version': u'1.8.1', u'copyright': u'Copyright (C) 2007-2010 LShift Ltd., Cohesive Financial Technologies LLC., and Rabbit Technologies Ltd.', u'information': u'Licensed under the MPL. See http://www.rabbitmq.com/'}, mechanisms: [u'PLAIN', u'AMQPLAIN'], locales: [u'en_US']
[2014-05-14 21:20:09,586: DEBUG/MainProcess] Open OK!
[2014-05-14 21:20:09,586: DEBUG/MainProcess] using channel_id: 1
[2014-05-14 21:20:09,587: DEBUG/MainProcess] Channel open
[2014-05-14 21:20:09,587: DEBUG/MainProcess] ^-- substep ok
[2014-05-14 21:20:09,588: DEBUG/MainProcess] | Consumer: Starting Mingle
[2014-05-14 21:20:09,588: INFO/MainProcess] mingle: searching for neighbors
[2014-05-14 21:20:09,589: DEBUG/MainProcess] using channel_id: 1
[2014-05-14 21:20:09,594: DEBUG/MainProcess] Channel open
[2014-05-14 21:20:09,610: ERROR/MainProcess] Unrecoverable error: UnicodeDecodeError('utf8', '\xf5', 0, 1, 'invalid start byte')
Traceback (most recent call last):
File "/home/federico/overactive/lib/python2.6/site-packages/celery/worker/init.py", line 206, in start
self.blueprint.start(self)
File "/home/federico/overactive/lib/python2.6/site-packages/celery/bootsteps.py", line 123, in start
step.start(parent)
File "/home/federico/overactive/lib/python2.6/site-packages/celery/bootsteps.py", line 373, in start
return self.obj.start()
File "/home/federico/overactive/lib/python2.6/site-packages/celery/worker/consumer.py", line 278, in start
blueprint.start(self)
File "/home/federico/overactive/lib/python2.6/site-packages/celery/bootsteps.py", line 123, in start
step.start(parent)
File "/home/federico/overactive/lib/python2.6/site-packages/celery/worker/consumer.py", line 569, in start
replies = I.hello(c.hostname, revoked._data) or {}
File "/home/federico/overactive/lib/python2.6/site-packages/celery/app/control.py", line 111, in hello
return self._request('hello', from_node=from_node, revoked=revoked)
File "/home/federico/overactive/lib/python2.6/site-packages/celery/app/control.py", line 70, in _request
timeout=self.timeout, reply=True,
File "/home/federico/overactive/lib/python2.6/site-packages/celery/app/control.py", line 306, in broadcast
limit, callback, channel=channel,
File "/home/federico/overactive/lib/python2.6/site-packages/kombu/pidbox.py", line 300, in _broadcast
channel=chan)
File "/home/federico/overactive/lib/python2.6/site-packages/kombu/pidbox.py", line 336, in _collect
with consumer:
File "/home/federico/overactive/lib/python2.6/site-packages/kombu/messaging.py", line 396, in enter
self.consume()
File "/home/federico/overactive/lib/python2.6/site-packages/kombu/messaging.py", line 445, in consume
self._basic_consume(T, no_ack=no_ack, nowait=False)
File "/home/federico/overactive/lib/python2.6/site-packages/kombu/messaging.py", line 567, in _basic_consume
no_ack=no_ack, nowait=nowait)
File "/home/federico/overactive/lib/python2.6/site-packages/kombu/entity.py", line 611, in consume
nowait=nowait)
File "/home/federico/overactive/lib/python2.6/site-packages/amqp/channel.py", line 1787, in basic_consume
(60, 21), # Channel.basic_consume_ok
File "/home/federico/overactive/lib/python2.6/site-packages/amqp/abstract_channel.py", line 67, in wait
self.channel_id, allowed_methods)
File "/home/federico/overactive/lib/python2.6/site-packages/amqp/connection.py", line 270, in _wait_method
self.wait()
File "/home/federico/overactive/lib/python2.6/site-packages/amqp/abstract_channel.py", line 69, in wait
return self.dispatch_method(method_sig, args, content)
File "/home/federico/overactive/lib/python2.6/site-packages/amqp/abstract_channel.py", line 87, in dispatch_method
return amqp_method(self, args)
File "/home/federico/overactive/lib/python2.6/site-packages/amqp/connection.py", line 530, in _blocked
reason = args.read_shortstr()
File "/home/federico/overactive/lib/python2.6/site-packages/amqp/serialization.py", line 120, in read_shortstr
return self.input.read(slen).decode('utf-8')
File "/home/federico/overactive/lib/python2.6/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xf5 in position 0: invalid start byte
[2014-05-14 21:20:09,629: DEBUG/MainProcess] | Worker: Closing Hub...
[2014-05-14 21:20:09,629: DEBUG/MainProcess] | Worker: Closing Pool...
[2014-05-14 21:20:09,629: DEBUG/MainProcess] | Worker: Closing Consumer...
[2014-05-14 21:20:09,630: DEBUG/MainProcess] | Worker: Stopping Consumer...
[2014-05-14 21:20:09,630: DEBUG/MainProcess] | Worker: Stopping Pool...
[2014-05-14 21:20:10,642: DEBUG/MainProcess] | Worker: Stopping Hub...
[2014-05-14 21:20:10,642: DEBUG/MainProcess] | Consumer: Shutdown Heart...
[2014-05-14 21:20:10,642: DEBUG/MainProcess] | Consumer: Shutdown Control...
[2014-05-14 21:20:10,643: DEBUG/MainProcess] | Consumer: Shutdown Tasks...
[2014-05-14 21:20:10,643: DEBUG/MainProcess] | Consumer: Shutdown Gossip...
[2014-05-14 21:20:10,643: DEBUG/MainProcess] | Consumer: Shutdown Events...
[2014-05-14 21:20:10,644: DEBUG/MainProcess] Closed channel #1
[2014-05-14 21:20:10,645: WARNING/MainProcess] Traceback (most recent call last):
[2014-05-14 21:20:10,645: WARNING/MainProcess] File "manage.py", line 10, in
[2014-05-14 21:20:10,645: WARNING/MainProcess] execute_from_command_line(sys.argv)
[2014-05-14 21:20:10,646: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/Django-1.6.1-py2.6.egg/django/core/management/init.py", line 399, in execute_from_command_line
[2014-05-14 21:20:10,646: WARNING/MainProcess] utility.execute()
[2014-05-14 21:20:10,646: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/Django-1.6.1-py2.6.egg/django/core/management/init.py", line 392, in execute
[2014-05-14 21:20:10,647: WARNING/MainProcess] self.fetch_command(subcommand).run_from_argv(self.argv)
[2014-05-14 21:20:10,647: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/djcelery/management/base.py", line 77, in run_from_argv
[2014-05-14 21:20:10,647: WARNING/MainProcess] return super(CeleryCommand, self).run_from_argv(argv)
[2014-05-14 21:20:10,647: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/Django-1.6.1-py2.6.egg/django/core/management/base.py", line 242, in run_from_argv
[2014-05-14 21:20:10,648: WARNING/MainProcess] self.execute(_args, *_options.dict)
[2014-05-14 21:20:10,648: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/djcelery/management/base.py", line 70, in execute
[2014-05-14 21:20:10,648: WARNING/MainProcess] super(CeleryCommand, self).execute(_args, *_options)
[2014-05-14 21:20:10,649: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/Django-1.6.1-py2.6.egg/django/core/management/base.py", line 285, in execute
[2014-05-14 21:20:10,649: WARNING/MainProcess] output = self.handle(_args, _options)
[2014-05-14 21:20:10,649: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/djcelery/management/commands/celeryd.py", line 26, in handle
[2014-05-14 21:20:10,649: WARNING/MainProcess] worker.run(_options)
[2014-05-14 21:20:10,650: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/celery/bin/worker.py", line 212, in run
[2014-05-14 21:20:10,651: WARNING/MainProcess] state_db=self.node_format(state_db, hostname), *_kwargs
[2014-05-14 21:20:10,651: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/celery/worker/init.py", line 211, in start
[2014-05-14 21:20:10,651: WARNING/MainProcess] self.stop()
[2014-05-14 21:20:10,652: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/celery/worker/init.py", line 252, in stop
[2014-05-14 21:20:10,652: WARNING/MainProcess] self._shutdown(warm=True)
[2014-05-14 21:20:10,652: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/celery/worker/init.py", line 266, in _shutdown
[2014-05-14 21:20:10,652: WARNING/MainProcess] self.blueprint.stop(self, terminate=not warm)
[2014-05-14 21:20:10,653: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/celery/bootsteps.py", line 183, in stop
[2014-05-14 21:20:10,653: WARNING/MainProcess] self.on_stopped()
[2014-05-14 21:20:10,653: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/celery/worker/init.py", line 164, in on_stopped
[2014-05-14 21:20:10,653: WARNING/MainProcess] self.consumer.shutdown()
[2014-05-14 21:20:10,654: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/celery/worker/consumer.py", line 305, in shutdown
[2014-05-14 21:20:10,654: WARNING/MainProcess] self.blueprint.shutdown(self)
[2014-05-14 21:20:10,654: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/celery/worker/consumer.py", line 170, in shutdown
[2014-05-14 21:20:10,655: WARNING/MainProcess] self.send_all(parent, 'shutdown')
[2014-05-14 21:20:10,655: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/celery/bootsteps.py", line 155, in send_all
[2014-05-14 21:20:10,655: WARNING/MainProcess] fun(parent, _args)
[2014-05-14 21:20:10,655: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/celery/worker/consumer.py", line 531, in shutdown
[2014-05-14 21:20:10,655: WARNING/MainProcess] self._close(c)
[2014-05-14 21:20:10,656: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/celery/worker/consumer.py", line 525, in _close
[2014-05-14 21:20:10,659: WARNING/MainProcess] ignore_errors(c, dispatcher.connection.close)
[2014-05-14 21:20:10,659: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/kombu/common.py", line 273, in ignore_errors
[2014-05-14 21:20:10,660: WARNING/MainProcess] return fun(_args, **kwargs)
[2014-05-14 21:20:10,660: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/kombu/connection.py", line 334, in release
[2014-05-14 21:20:10,660: WARNING/MainProcess] self._close()
[2014-05-14 21:20:10,660: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/kombu/connection.py", line 302, in _close
[2014-05-14 21:20:10,661: WARNING/MainProcess] self._do_close_self()
[2014-05-14 21:20:10,661: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/kombu/connection.py", line 295, in _do_close_self
[2014-05-14 21:20:10,661: WARNING/MainProcess] self.transport.close_connection(self._connection)
[2014-05-14 21:20:10,661: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/kombu/transport/pyamqp.py", line 122, in close_connection
[2014-05-14 21:20:10,662: WARNING/MainProcess] connection.close()
[2014-05-14 21:20:10,663: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/amqp/connection.py", line 461, in close
[2014-05-14 21:20:10,663: WARNING/MainProcess](10, 51), # Connection.close_ok
[2014-05-14 21:20:10,663: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/amqp/abstract_channel.py", line 67, in wait
[2014-05-14 21:20:10,664: WARNING/MainProcess] self.channel_id, allowed_methods)
[2014-05-14 21:20:10,664: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/amqp/connection.py", line 270, in _wait_method
[2014-05-14 21:20:10,665: WARNING/MainProcess] self.wait()
[2014-05-14 21:20:10,665: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/amqp/abstract_channel.py", line 69, in wait
[2014-05-14 21:20:10,665: WARNING/MainProcess] return self.dispatch_method(method_sig, args, content)
[2014-05-14 21:20:10,665: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/amqp/abstract_channel.py", line 87, in dispatch_method
[2014-05-14 21:20:10,665: WARNING/MainProcess] return amqp_method(self, args)
[2014-05-14 21:20:10,666: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/amqp/connection.py", line 530, in _blocked
[2014-05-14 21:20:10,666: WARNING/MainProcess] reason = args.read_shortstr()
[2014-05-14 21:20:10,666: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/site-packages/amqp/serialization.py", line 120, in read_shortstr
[2014-05-14 21:20:10,667: WARNING/MainProcess] return self.input.read(slen).decode('utf-8')
[2014-05-14 21:20:10,667: WARNING/MainProcess] File "/home/federico/overactive/lib/python2.6/encodings/utf_8.py", line 16, in decode
[2014-05-14 21:20:10,667: WARNING/MainProcess] return codecs.utf_8_decode(input, errors, True)
[2014-05-14 21:20:10,668: WARNING/MainProcess] UnicodeDecodeError
[2014-05-14 21:20:10,668: WARNING/MainProcess] :
[2014-05-14 21:20:10,668: WARNING/MainProcess] 'utf8' codec can't decode byte 0xf5 in position 0: invalid start byte
[2014-05-14 21:20:10,668: DEBUG/MainProcess] removing tasks from inqueue until task handler finished
The text was updated successfully, but these errors were encountered: