Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invalid Syntax (celery=4.1.0, kombu=4.1.0, amqp=2.2.2) #841

Closed
acknowledgeHim opened this issue Mar 27, 2018 · 6 comments
Closed

Invalid Syntax (celery=4.1.0, kombu=4.1.0, amqp=2.2.2) #841

acknowledgeHim opened this issue Mar 27, 2018 · 6 comments

Comments

@acknowledgeHim
Copy link

acknowledgeHim commented Mar 27, 2018

I have python 3.7 and am trying to start up celery with the command below (I included the full stack trace as well). I have all this working in a python 3.5 but am upgrading everything then moving over my site to this updated server. Any help is greatly appreciated.

I've tried downgrading and using kombu=3.0.35 but got a different set of errors. From what I gather on searches on the internet, it appears that python 3.7 has changed the async.

(virtual_env) celery -A myproject worker --loglevel=INFO --concurrency=5
Traceback (most recent call last):
File bin/celery", line 11, in
sys.exit(main())
File "lib/python3.7/site-packages/celery/main.py", line 13, in main
from celery.bin.celery import main as _main
File "lib/python3.7/site-packages/celery/bin/init.py", line 2, in
from .base import Option
File "lib/python3.7/site-packages/celery/bin/base.py", line 17, in
from celery import VERSION_BANNER, Celery, maybe_patch_concurrency
File "", line 1032, in _handle_fromlist
File "lib/python3.7/site-packages/celery/local.py", line 509, in getattr
module = import(self._object_origins[name], None, None, [name])
File "lib/python3.7/site-packages/celery/app/init.py", line 5, in
from celery import _state
File "lib/python3.7/site-packages/celery/_state.py", line 15, in
from celery.utils.threads import LocalStack
File "lib/python3.7/site-packages/celery/utils/init.py", line 10, in
from .nodenames import worker_direct, nodename, nodesplit
File "lib/python3.7/site-packages/celery/utils/nodenames.py", line 7, in
from kombu.entity import Exchange, Queue
File "lib/python3.7/site-packages/kombu/entity.py", line 6, in
from .abstract import MaybeChannelBound, Object
File "lib/python3.7/site-packages/kombu/abstract.py", line 6, in
from .connection import maybe_channel
File "lib/python3.7/site-packages/kombu/connection.py", line 893
return self.transport.implements.async
^
SyntaxError: invalid syntax

@clokep
Copy link

clokep commented Mar 27, 2018

I believe this was fixed by #839. (This is a duplicate of #742.)

@auvipy
Copy link
Member

auvipy commented Mar 27, 2018

wait for new release of kombu

@auvipy auvipy closed this as completed Mar 27, 2018
@acknowledgeHim
Copy link
Author

@auvipy do you know when the new release of kombu will be released to fix this issue. From clokep's link it appears it was a pull request to maybe fix this, so is it just a matter of waiting for it to be merged with the master and if so when do you think that will happen?
Thanks!

@clokep
Copy link

clokep commented Mar 28, 2018

#839 was already merged, so it's just waiting for release. I can't speak to when that will be though!

@mgan59
Copy link

mgan59 commented Jan 30, 2019

Stumbled onto this issue setting up a new max... Brew defaults to python3.7 out of the box and installing anything below 3.7 is a challenge. What is the movement on getting this release out?

@melarish
Copy link

melarish commented Feb 5, 2019

@mgan59 Kombu version 4.2.0 has the fix - worked for me on Mac + Python 3.7

jwalgran added a commit to OpenTreeMap/otm-core that referenced this issue Jan 3, 2020
Django 1.11.17 is the minimum version required to run under Python 3.
https://docs.djangoproject.com/en/1.11/faq/install/#what-python-version-can-i-use-with-django

`functools32` was a backport of a Python 3 included library.

The 4.2.0 version of kombu fixes a syntax error under Python 3.
celery/kombu#841

wsgiref is part of the Python 3 standard library.
hectcastro pushed a commit to OpenTreeMap/otm-core that referenced this issue Jan 9, 2020
Django 1.11.17 is the minimum version required to run under Python 3.
https://docs.djangoproject.com/en/1.11/faq/install/#what-python-version-can-i-use-with-django

`functools32` was a backport of a Python 3 included library.

The 4.2.0 version of kombu fixes a syntax error under Python 3.
celery/kombu#841

wsgiref is part of the Python 3 standard library.
hectcastro pushed a commit to OpenTreeMap/otm-core that referenced this issue Jan 9, 2020
Django 1.11.17 is the minimum version required to run under Python 3.
https://docs.djangoproject.com/en/1.11/faq/install/#what-python-version-can-i-use-with-django

`functools32` was a backport of a Python 3 included library.

The 4.2.0 version of kombu fixes a syntax error under Python 3.
celery/kombu#841

wsgiref is part of the Python 3 standard library.
knightjoel added a commit to knightjoel/zpark that referenced this issue Nov 24, 2020
cwardcode pushed a commit to cwardcode/otm-core that referenced this issue Jan 2, 2021
* Update requirements for Python 3 compatibility

Django 1.11.17 is the minimum version required to run under Python 3.
https://docs.djangoproject.com/en/1.11/faq/install/#what-python-version-can-i-use-with-django

`functools32` was a backport of a Python 3 included library.

The 4.2.0 version of kombu fixes a syntax error under Python 3.
celery/kombu#841

wsgiref is part of the Python 3 standard library.

* FIXUP replace py2 backport of modgrammer with latest py3 version

* Adjust hashbang for manage.py to use Python 3

* Use 2to3 to upgrade to Python 3 compatible syntax

These changes were made by running the following command from the project root:

```
2to3 -w -n --no-diffs .
```

The resulting changes have lint errors and broken functionality but we are
committing them separately to allow distinguishing between changes made by the
automated upgrade tooling and manual fixes that were required after the fact.

* Remove extra blank lines created by 2to3

Programmatically removed with the following command

autopep8 --in-place --recursive . --select E303

https://github.com/hhatto/autopep8/

* Fix variable references in list comprehension for Python 3

List comprehensions in Python 3 create a new functional scope. As a result in
the case where we are using a list comprehension when defining a class-level
member we do not have access to the other class-level members within the
comprehension. We have implemented a workaround that uses an immediately
executed lambda to bind the member variables in the new scope.

* Replace bytes literal with strings when using modgrammer in Python 3

"cannot use a string pattern on a bytes-like object" exceptions were being
raised by code using modgrammer which expects string objects under Python 3.

* Avoid DotDict exceptions due to Python 3 change in hasattr handling

Django 1.11.17 calls `hasattr(a_dotdict_field, 'resolve_expression')`.
https://github.com/django/django/blob/121115d2c291b3969ac00ca62253f23513481739/django/db/models/sql/compiler.py#L990

The Python 3 documentation says that this ends up calling `getattr` and
returning False if an `AttributeError` is raised
https://docs.python.org/3/library/functions.html#hasattr

For dictionaries `getattr` ends up calling `__getitem__`. Our implementation of
`__getitem__` correctly raises KeyError, and we were relying on a bug in Python
2 where any exception raised during a `hasattr` check would result in False. Now
that Python 3 explicitly checks for `AttributeError` we need this workaround

The tradeoff is that we now return `AttributeError` instead of `KeyError` when
attempting to access a non-existent key. If we rely on catching `KeyError`
somewhere in our application we can fix it there.

* Fix undefined variable flake8 errors

It appears that these lines may have been functioning properly by accident,
relying on the fact that in Python 2 list comprehensions did not have their own
function scope.

* Fix multiple imports on one line flake 8 error

Programmatically fixed with the following command

autopep8 --in-place --recursive . --select E401

https://github.com/hhatto/autopep8/

We also removed a duplicate set of imports from ecobackend.py

* Fix flake8 line length errors after 2to3

We shorten and rearrange statements as needed to return to having all lines
under 80 characters.

* Replace reference to file type with a call to open

Python 3 removed the built in `file` type, which was semantically equivalent to
`open` and what we are required to use in Python 3.

https://stackoverflow.com/a/32131309

* Fix flake8 indentation errors after 2to3

* Add migrations as a result of 2to3 converting choices to a list

After running 2to3 the choices defined for the `ExportJob.status` and
`Instance.itree_region_default` fields were wrapped with a call to `list()`.
This resulted in the order of the choices changing. Any change to the choices
defined for a field requires a migration.

* Update requirements for Python 3 compatibility

Django 1.11.17 is the minimum version required to run under Python 3.
https://docs.djangoproject.com/en/1.11/faq/install/#what-python-version-can-i-use-with-django

`functools32` was a backport of a Python 3 included library.

The 4.2.0 version of kombu fixes a syntax error under Python 3.
celery/kombu#841

wsgiref is part of the Python 3 standard library.

* Adjust hashbang for manage.py to use Python 3

Ensure that manage.py makes use of the Python 3.7 binary, which is
distinct from the python (Python 2.7) and python3 (Python 3.6)
binaries.

* Clarify usage of bytes vs. string

One major change in Python3 is a clean divorce of unicode and binary data. Strings are unicode and bytes are binary. In python2 they could sometimes be handled interchangeably. Now one must explicitly convert between the two data formats, encoding to strings to bytes or decoding bytes to strings.

- Django's HttpResponse.content is explicitly a bytestring
- Use BytesIO instead of StringIO to prepare request data
- Basic auth requires b64 encoding of byte credentials. The result of this is several new layers of de/encoding and unpacking are needed to verify the Basic auth credentials.

* Update deps to Py3.7-compatible versions

Specifically, Celery and Django-tinsel required version bumps. See: celery/celery#5049 and azavea/django-tinsel#9

* Comply with PEP 352

The message attribute (and all attributes, other than args) were removed off the Exception base class between Py2 and Py3.

* Handle CSV data de/encoding

* Fix incorrect coercion to string method

D/encoding worked differently and more loosely in py2. Now decoding converts bytes to unicode, vice versa for encoding. Also, fix a duplicate conditional expression that captures strings to capture integers too.

* Encode image data in bytes not string

Refs https://github.com/OpenTreeMap/otm-cloud/issues/505

* Upgrade modgrammar behavior for v.10

Some breaking changes in usage in modgrammar v.9 => v.10. Whitespace cleaning is invoked through a different class property (bool: grammar_whitespace => str: grammar_whitespace_mode) and  stripping is no longer done by default. The parse_string method has strictly defined behavior, prefer to use parse_text with args set.

Modgrammar is no longer maintained (or so it seems) and the latest / most helpful documentation is this google group post: https://groups.google.com/forum/#!msg/modgrammar/3CsPBb_UCck/kKr-0YmwtJ8J

* Handle image data as bytes not unicode

* Encode all strings to bytes before hashing

* Format request params as unicode strings

* Repackage bytestream into string stream for csv

A CSV by python definition is an iterator object that returns a string. A python3 string is defined as unicode and not binary, explicitly, so we have to be explicit about passing string data to csv generators.
A downside to this solution is we may now have 2 copies of the entire file in memory while it is being sliced into individual database rows but this should probably be ok given the expected use cases and available computing resources.

* Remove obsolete unicode literal

Strings are implicitly unicode in py3

* update reqs

Co-authored-by: Justin Walgran <jwalgran@azavea.com>
Co-authored-by: Hector Castro <hectcastro@gmail.com>
Co-authored-by: Jenny Fung <fungjj92@gmail.com>
Co-authored-by: Jenny Fung <fungjj92@users.noreply.github.com>
openstack-mirroring pushed a commit to openstack/openstack that referenced this issue Jan 5, 2021
* Update blazar from branch 'master'
  to 4f642c877f46646b22602ea495d9e265c7887c62
  - Fix lower-constraints job
    
    * Move upper constraints from install_command to testenv deps, so it
      doesn't get inherited by the lower-constraints job. This was causing
      the wrong set of constraints to be used.
    * Update MarkupSafe to 1.1.1 because of an incompatibility with recent
      setuptools: pallets/markupsafe#116
    * Update PyYAML to 5.1 to fix build on some platforms such as macOS
    * Update oslo.db to 4.40.0 to fix issues with async keyword:
      https://opendev.org/openstack/oslo.db/commit/df6bf34
    * Update oslo.utils to 3.37.0 for uuidsentinel:
      https://opendev.org/openstack/oslo.utils/commit/63d7649
    * Update kombu to 4.2.0 to fix issues with async:
      celery/kombu#841
    * Remove kombu from requirements since we are not using it directly
    * Update Flask to 1.0.2 due to various API changes
    * Update oslo.service to 1.34.0 for add_timer_args:
      https://opendev.org/openstack/oslo.service/commit/750b51c
    * Update oslo.context to 2.21.0 for system_scope:
      https://opendev.org/openstack/oslo.context/commit/1a40b3d
    * Move oslo.context from test-requirements to requirements since we are
      also using it in non-test code
    * Update keystoneauth1 to 3.13.0 for rate_semaphore:
      https://opendev.org/openstack/keystoneauth/commit/0993471
    * Update alembic to 0.9.6: oslo-db 4.40.0 depends on alembic>=0.9.6
    
    Change-Id: I4224c142d7481a79d4cc5e704436a6002b0d884f
openstack-mirroring pushed a commit to openstack/blazar that referenced this issue Jan 5, 2021
* Move upper constraints from install_command to testenv deps, so it
  doesn't get inherited by the lower-constraints job. This was causing
  the wrong set of constraints to be used.
* Update MarkupSafe to 1.1.1 because of an incompatibility with recent
  setuptools: pallets/markupsafe#116
* Update PyYAML to 5.1 to fix build on some platforms such as macOS
* Update oslo.db to 4.40.0 to fix issues with async keyword:
  https://opendev.org/openstack/oslo.db/commit/df6bf34
* Update oslo.utils to 3.37.0 for uuidsentinel:
  https://opendev.org/openstack/oslo.utils/commit/63d7649
* Update kombu to 4.2.0 to fix issues with async:
  celery/kombu#841
* Remove kombu from requirements since we are not using it directly
* Update Flask to 1.0.2 due to various API changes
* Update oslo.service to 1.34.0 for add_timer_args:
  https://opendev.org/openstack/oslo.service/commit/750b51c
* Update oslo.context to 2.21.0 for system_scope:
  https://opendev.org/openstack/oslo.context/commit/1a40b3d
* Move oslo.context from test-requirements to requirements since we are
  also using it in non-test code
* Update keystoneauth1 to 3.13.0 for rate_semaphore:
  https://opendev.org/openstack/keystoneauth/commit/0993471
* Update alembic to 0.9.6: oslo-db 4.40.0 depends on alembic>=0.9.6

Change-Id: I4224c142d7481a79d4cc5e704436a6002b0d884f
tzinckgraf added a commit to sustainablejc/otm-core that referenced this issue Apr 13, 2021
commit 85e0494
Author: Tom Z <tzinckgraf@gmail.com>
Date:   Mon Apr 12 00:06:19 2021 -0400

    feature/api-python-3 - additional final features

commit 9f3bd4e
Author: Tom Z <tzinckgraf@gmail.com>
Date:   Sun Apr 11 23:43:41 2021 -0400

    feature/api-python-3 - final pieces of this feature

commit f4866de
Author: Tom Z <tzinckgraf@gmail.com>
Date:   Fri Apr 9 20:18:09 2021 -0400

    feature/api-python-3 - some additional fixes

commit 7c3de56
Author: Tom Z <tzinckgraf@gmail.com>
Date:   Sat Apr 3 23:15:48 2021 -0400

    feature/api-python-3 - update the map page with new layers and finishing
    touches on other pieces

commit bdb8183
Author: Tom Z <tzinckgraf@gmail.com>
Date:   Sat Mar 27 10:34:51 2021 -0400

    feature/api-python-3 - validation errors are working

commit 412ea36
Author: Tom Z <tzinckgraf@gmail.com>
Date:   Sun Mar 21 21:35:57 2021 -0400

    feature/api-python-3 - fixed up the creation. various fixes to the UI.

commit ed4d9bc
Author: Tom Z <tzinckgraf@gmail.com>
Date:   Fri Mar 19 17:14:04 2021 -0400

    feature/api-python-3 - more fixes

commit d1dd629
Author: Tom Z <tzinckgraf@gmail.com>
Date:   Sat Mar 13 13:34:59 2021 -0500

    feature/api-python-3 - fixes to the pagination and a sample for creation

commit c451010
Merge: fc1765d 13d6c14
Author: Tom Z <tzinckgraf@gmail.com>
Date:   Sun Mar 7 21:48:28 2021 -0500

    Merge branch 'feature/api-python-3' of github.com:sustainablejc/otm-core into feature/api-python-3

commit fc1765d
Author: Tom Z <tzinckgraf@gmail.com>
Date:   Sun Mar 7 21:34:50 2021 -0500

    feature/api-python-3 - updated bootstrap and everything associated with
    it

commit 13d6c14
Author: Tom Z <tzinckgraf@gmail.com>
Date:   Sun Feb 28 21:48:17 2021 -0500

    fixed the Dockerfile

commit e0b4e92
Author: Tom Z <tzinckgraf@gmail.com>
Date:   Sun Feb 28 18:43:18 2021 -0500

    feature/api-python-3 - added common trees. refactored some code to move
    it to easier places

commit 7c88c3c
Author: Tom Z <tzinckgraf@gmail.com>
Date:   Sat Feb 27 11:05:27 2021 -0500

    feature/api-python-3 - the api features combined with python 3

commit c7e20ca
Merge: e06bae0 8ee5891
Author: Tom Z <tzinckgraf@gmail.com>
Date:   Tue Feb 23 20:43:41 2021 -0500

    Merge branch 'python3' into feature/api

commit e06bae0
Author: Tom Z <tzinckgraf@gmail.com>
Date:   Sat Feb 20 17:05:37 2021 -0500

    feature/api - fixed the styling to match the current OTM. the create
    tree feature almost works

commit 8935e23
Author: Tom Z <tzinckgraf@gmail.com>
Date:   Mon Feb 15 22:40:30 2021 -0500

    feature/api - frontend working with grids and the sidebar

commit 8ee5891
Author: Tom Z <tzinckgraf@gmail.com>
Date:   Sun Jan 24 19:49:43 2021 -0500

    python3 - fixed minor issues, getting  closer to all test cases

commit 15e5b8f
Author: Tom Z <tzinckgraf@gmail.com>
Date:   Mon Jan 18 15:21:51 2021 -0500

    feature/api - included the build_all script

commit 9d5c0b1
Author: Tom Z <tzinckgraf@gmail.com>
Date:   Sun Jan 17 17:36:27 2021 -0500

    feature/api - added the tree popup

commit 4c2691c
Author: Tom Z <tzinckgraf@gmail.com>
Date:   Sat Jan 16 12:47:53 2021 -0500

    feature/api - a working instance of a UTF Grid

commit ad1fc56
Author: Tom Z <tzinckgraf@gmail.com>
Date:   Sat Jan 9 10:45:05 2021 -0500

    feature/api - added a bunch of fixes to Webpack to upgrade everything.

    Webpack is now v4

    This required some changes to the reverse library, as I could not get
    the export to work, so instead I updated the calls using reverse.

commit 4282184
Author: Tom Z <tzinckgraf@gmail.com>
Date:   Fri Jan 1 14:24:51 2021 -0500

    feature/api - upgrade to webpack

commit fbd1299
Author: Tom Z <tzinckgraf@gmail.com>
Date:   Sun Dec 27 16:05:54 2020 -0500

    feature/api - routes and users page

commit f745822
Author: Tom Z <tzinckgraf@gmail.com>
Date:   Sun Nov 15 00:02:39 2020 -0500

    python3 - more test cases working

commit a6e5a8b
Merge: c1fb9fb d3b4fb9
Author: Jenny Fung <fungjj92@users.noreply.github.com>
Date:   Thu Feb 6 14:16:07 2020 -0500

    Merge pull request OpenTreeMap#3288 from OpenTreeMap/jf/fix-tests-2to3

    First pass at fixing tests from Py2->3 upgrade

commit d3b4fb9
Merge: d85cd29 b8a632a
Author: Jenny Fung <fungjj92@users.noreply.github.com>
Date:   Thu Feb 6 14:14:23 2020 -0500

    Merge pull request OpenTreeMap#3290 from OpenTreeMap/jf/fix-tests-manage-treemap

    Fix remaining broken tests in manage_treemap Django app

commit b8a632a
Author: Jenny Fung <fungjj92@gmail.com>
Date:   Tue Feb 4 13:22:14 2020 -0500

    Remove obsolete unicode literal

    Strings are implicitly unicode in py3

commit db54d58
Author: Jenny Fung <fungjj92@gmail.com>
Date:   Tue Feb 4 13:21:11 2020 -0500

    Repackage bytestream into string stream for csv

    A CSV by python definition is an iterator object that returns a string. A python3 string is defined as unicode and not binary, explicitly, so we have to be explicit about passing string data to csv generators.
    A downside to this solution is we may now have 2 copies of the entire file in memory while it is being sliced into individual database rows but this should probably be ok given the expected use cases and available computing resources.

commit 0f2c3f0
Author: Jenny Fung <fungjj92@gmail.com>
Date:   Mon Feb 3 17:01:34 2020 -0500

    Format request params as unicode strings

commit c011701
Author: Jenny Fung <fungjj92@gmail.com>
Date:   Fri Jan 24 12:44:19 2020 -0500

    Encode all strings to bytes before hashing

commit ed3282c
Author: Jenny Fung <fungjj92@gmail.com>
Date:   Fri Jan 24 12:30:27 2020 -0500

    Handle image data as bytes not unicode

commit e6a731e
Author: Jenny Fung <fungjj92@gmail.com>
Date:   Fri Jan 24 09:25:02 2020 -0500

    Upgrade modgrammar behavior for v.10

    Some breaking changes in usage in modgrammar v.9 => v.10. Whitespace cleaning is invoked through a different class property (bool: grammar_whitespace => str: grammar_whitespace_mode) and  stripping is no longer done by default. The parse_string method has strictly defined behavior, prefer to use parse_text with args set.

    Modgrammar is no longer maintained (or so it seems) and the latest / most helpful documentation is this google group post: https://groups.google.com/forum/#!msg/modgrammar/3CsPBb_UCck/kKr-0YmwtJ8J

commit b79a409
Author: Jenny Fung <fungjj92@gmail.com>
Date:   Thu Jan 23 13:01:12 2020 -0500

    Encode image data in bytes not string

    Refs https://github.com/OpenTreeMap/otm-cloud/issues/505

commit d85cd29
Author: Jenny Fung <fungjj92@gmail.com>
Date:   Wed Jan 22 09:54:03 2020 -0500

    Fix incorrect coercion to string method

    D/encoding worked differently and more loosely in py2. Now decoding converts bytes to unicode, vice versa for encoding. Also, fix a duplicate conditional expression that captures strings to capture integers too.

commit b386116
Author: Jenny Fung <fungjj92@gmail.com>
Date:   Wed Jan 22 09:21:04 2020 -0500

    Handle CSV data de/encoding

commit 6fa64bf
Author: Jenny Fung <fungjj92@gmail.com>
Date:   Tue Jan 21 18:42:00 2020 -0500

    Comply with PEP 352

    The message attribute (and all attributes, other than args) were removed off the Exception base class between Py2 and Py3.

commit bc961a8
Author: Jenny Fung <fungjj92@gmail.com>
Date:   Tue Jan 21 15:22:25 2020 -0500

    Update deps to Py3.7-compatible versions

    Specifically, Celery and Django-tinsel required version bumps. See: celery/celery#5049 and azavea/django-tinsel#9

commit a1a3b6d
Author: Jenny Fung <fungjj92@gmail.com>
Date:   Fri Jan 10 09:55:35 2020 -0500

    Clarify usage of bytes vs. string

    One major change in Python3 is a clean divorce of unicode and binary data. Strings are unicode and bytes are binary. In python2 they could sometimes be handled interchangeably. Now one must explicitly convert between the two data formats, encoding to strings to bytes or decoding bytes to strings.

    - Django's HttpResponse.content is explicitly a bytestring
    - Use BytesIO instead of StringIO to prepare request data
    - Basic auth requires b64 encoding of byte credentials. The result of this is several new layers of de/encoding and unpacking are needed to verify the Basic auth credentials.

commit c1fb9fb
Merge: f63cfb9 ccef808
Author: Justin Walgran <jwalgran@azavea.com>
Date:   Fri Jan 10 14:40:03 2020 -0700

    First stage of the application-level Python 2 to Python 3 conve… (OpenTreeMap#3285)

    First stage of the application-level Python 2 to Python 3 conversion

commit f63cfb9
Author: Hector Castro <hectcastro@gmail.com>
Date:   Tue Jan 7 15:44:52 2020 -0500

    Adjust hashbang for manage.py to use Python 3

    Ensure that manage.py makes use of the Python 3.7 binary, which is
    distinct from the python (Python 2.7) and python3 (Python 3.6)
    binaries.

commit c27a86b
Author: Justin Walgran <jwalgran@azavea.com>
Date:   Fri Jan 3 11:53:55 2020 -0700

    Update requirements for Python 3 compatibility

    Django 1.11.17 is the minimum version required to run under Python 3.
    https://docs.djangoproject.com/en/1.11/faq/install/#what-python-version-can-i-use-with-django

    `functools32` was a backport of a Python 3 included library.

    The 4.2.0 version of kombu fixes a syntax error under Python 3.
    celery/kombu#841

    wsgiref is part of the Python 3 standard library.

commit ccef808
Author: Justin Walgran <jwalgran@azavea.com>
Date:   Wed Jan 8 12:36:05 2020 -0700

    Add migrations as a result of 2to3 converting choices to a list

    After running 2to3 the choices defined for the `ExportJob.status` and
    `Instance.itree_region_default` fields were wrapped with a call to `list()`.
    This resulted in the order of the choices changing. Any change to the choices
    defined for a field requires a migration.

commit 545e383
Author: Justin Walgran <jwalgran@azavea.com>
Date:   Mon Jan 6 16:35:05 2020 -0700

    Fix flake8 indentation errors after 2to3

commit b96f11d
Author: Justin Walgran <jwalgran@azavea.com>
Date:   Mon Jan 6 16:26:11 2020 -0700

    Replace reference to file type with a call to open

    Python 3 removed the built in `file` type, which was semantically equivalent to
    `open` and what we are required to use in Python 3.

    https://stackoverflow.com/a/32131309

commit e5a7b47
Author: Justin Walgran <jwalgran@azavea.com>
Date:   Mon Jan 6 16:04:28 2020 -0700

    Fix flake8 line length errors after 2to3

    We shorten and rearrange statements as needed to return to having all lines
    under 80 characters.

commit 0afa0fb
Author: Justin Walgran <jwalgran@azavea.com>
Date:   Mon Jan 6 15:38:18 2020 -0700

    Fix multiple imports on one line flake 8 error

    Programmatically fixed with the following command

    autopep8 --in-place --recursive . --select E401

    https://github.com/hhatto/autopep8/

    We also removed a duplicate set of imports from ecobackend.py

commit ef8a6f2
Author: Justin Walgran <jwalgran@azavea.com>
Date:   Mon Jan 6 15:02:30 2020 -0700

    Fix undefined variable flake8 errors

    It appears that these lines may have been functioning properly by accident,
    relying on the fact that in Python 2 list comprehensions did not have their own
    function scope.

commit 335f284
Author: Justin Walgran <jwalgran@azavea.com>
Date:   Mon Jan 6 14:21:58 2020 -0700

    Avoid DotDict exceptions due to Python 3 change in hasattr handling

    Django 1.11.17 calls `hasattr(a_dotdict_field, 'resolve_expression')`.
    https://github.com/django/django/blob/121115d2c291b3969ac00ca62253f23513481739/django/db/models/sql/compiler.py#L990

    The Python 3 documentation says that this ends up calling `getattr` and
    returning False if an `AttributeError` is raised
    https://docs.python.org/3/library/functions.html#hasattr

    For dictionaries `getattr` ends up calling `__getitem__`. Our implementation of
    `__getitem__` correctly raises KeyError, and we were relying on a bug in Python
    2 where any exception raised during a `hasattr` check would result in False. Now
    that Python 3 explicitly checks for `AttributeError` we need this workaround

    The tradeoff is that we now return `AttributeError` instead of `KeyError` when
    attempting to access a non-existent key. If we rely on catching `KeyError`
    somewhere in our application we can fix it there.

commit adf4233
Author: Justin Walgran <jwalgran@azavea.com>
Date:   Mon Jan 6 09:47:10 2020 -0700

    Replace bytes literal with strings when using modgrammer in Python 3

    "cannot use a string pattern on a bytes-like object" exceptions were being
    raised by code using modgrammer which expects string objects under Python 3.

commit 506306f
Author: Justin Walgran <jwalgran@azavea.com>
Date:   Mon Jan 6 09:43:05 2020 -0700

    Fix variable references in list comprehension for Python 3

    List comprehensions in Python 3 create a new functional scope. As a result in
    the case where we are using a list comprehension when defining a class-level
    member we do not have access to the other class-level members within the
    comprehension. We have implemented a workaround that uses an immediately
    executed lambda to bind the member variables in the new scope.

commit bb20099
Author: Justin Walgran <jwalgran@azavea.com>
Date:   Mon Jan 6 14:51:51 2020 -0700

    Remove extra blank lines created by 2to3

    Programmatically removed with the following command

    autopep8 --in-place --recursive . --select E303

    https://github.com/hhatto/autopep8/

commit fe25f7b
Author: Justin Walgran <jwalgran@azavea.com>
Date:   Fri Jan 3 14:12:16 2020 -0700

    Use 2to3 to upgrade to Python 3 compatible syntax

    These changes were made by running the following command from the project root:

    ```
    2to3 -w -n --no-diffs .
    ```

    The resulting changes have lint errors and broken functionality but we are
    committing them separately to allow distinguishing between changes made by the
    automated upgrade tooling and manual fixes that were required after the fact.

commit 2040c21
Author: Hector Castro <hectcastro@gmail.com>
Date:   Tue Jan 7 15:44:52 2020 -0500

    Adjust hashbang for manage.py to use Python 3

commit fcd6fb6
Author: Justin Walgran <jwalgran@azavea.com>
Date:   Mon Jan 6 09:35:05 2020 -0700

    FIXUP replace py2 backport of modgrammer with latest py3 version

commit 6a29266
Author: Justin Walgran <jwalgran@azavea.com>
Date:   Fri Jan 3 11:53:55 2020 -0700

    Update requirements for Python 3 compatibility

    Django 1.11.17 is the minimum version required to run under Python 3.
    https://docs.djangoproject.com/en/1.11/faq/install/#what-python-version-can-i-use-with-django

    `functools32` was a backport of a Python 3 included library.

    The 4.2.0 version of kombu fixes a syntax error under Python 3.
    celery/kombu#841

    wsgiref is part of the Python 3 standard library.
AnishReddyRavula pushed a commit to ChameleonCloud/blazar that referenced this issue Jun 26, 2023
* Move upper constraints from install_command to testenv deps, so it
  doesn't get inherited by the lower-constraints job. This was causing
  the wrong set of constraints to be used.
* Update MarkupSafe to 1.1.1 because of an incompatibility with recent
  setuptools: pallets/markupsafe#116
* Update PyYAML to 5.1 to fix build on some platforms such as macOS
* Update oslo.db to 4.40.0 to fix issues with async keyword:
  https://opendev.org/openstack/oslo.db/commit/df6bf34
* Update oslo.utils to 3.37.0 for uuidsentinel:
  https://opendev.org/openstack/oslo.utils/commit/63d7649
* Update kombu to 4.2.0 to fix issues with async:
  celery/kombu#841
* Remove kombu from requirements since we are not using it directly
* Update Flask to 1.0.2 due to various API changes
* Update oslo.service to 1.34.0 for add_timer_args:
  https://opendev.org/openstack/oslo.service/commit/750b51c
* Update oslo.context to 2.21.0 for system_scope:
  https://opendev.org/openstack/oslo.context/commit/1a40b3d
* Move oslo.context from test-requirements to requirements since we are
  also using it in non-test code
* Update keystoneauth1 to 3.13.0 for rate_semaphore:
  https://opendev.org/openstack/keystoneauth/commit/0993471
* Update alembic to 0.9.6: oslo-db 4.40.0 depends on alembic>=0.9.6

Change-Id: I4224c142d7481a79d4cc5e704436a6002b0d884f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants