Skip to content

Commit

Permalink
Add protobuf pin to dagster (#8078)
Browse files Browse the repository at this point in the history
Summary:
protobuf 4 came out a couple of hours ago and gives an error on startup for dagster:

```
(dagster-3.8.9) dgibson@Daniels-MacBook-Pro dagster % dagster
Traceback (most recent call last):
  File "/Users/dgibson/.pyenv/versions/dagster-3.8.9/bin/dagster", line 33, in <module>
    sys.exit(load_entry_point('dagster', 'console_scripts', 'dagster')())
  File "/Users/dgibson/.pyenv/versions/dagster-3.8.9/bin/dagster", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/Users/dgibson/.pyenv/versions/3.8.9/lib/python3.8/importlib/metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "/Users/dgibson/.pyenv/versions/3.8.9/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/dgibson/dagster/python_modules/dagster/dagster/__init__.py", line 227, in <module>
    from dagster.core.launcher import DefaultRunLauncher
  File "/Users/dgibson/dagster/python_modules/dagster/dagster/core/launcher/__init__.py", line 1, in <module>
    from .base import (
  File "/Users/dgibson/dagster/python_modules/dagster/dagster/core/launcher/base.py", line 8, in <module>
    from dagster.core.workspace.workspace import IWorkspace
  File "/Users/dgibson/dagster/python_modules/dagster/dagster/core/workspace/__init__.py", line 1, in <module>
    from .context import IWorkspaceProcessContext, WorkspaceProcessContext
  File "/Users/dgibson/dagster/python_modules/dagster/dagster/core/workspace/context.py", line 16, in <module>
    from dagster.core.host_representation import (
  File "/Users/dgibson/dagster/python_modules/dagster/dagster/core/host_representation/__init__.py", line 50, in <module>
    from .repository_location import (
  File "/Users/dgibson/dagster/python_modules/dagster/dagster/core/host_representation/repository_location.py", line 9, in <module>
    from dagster.api.list_repositories import sync_list_repositories_grpc
  File "/Users/dgibson/dagster/python_modules/dagster/dagster/api/list_repositories.py", line 6, in <module>
    from dagster.grpc.types import ListRepositoriesResponse
  File "/Users/dgibson/dagster/python_modules/dagster/dagster/grpc/__init__.py", line 11, in <module>
    from .client import DagsterGrpcClient, ephemeral_grpc_api_client
  File "/Users/dgibson/dagster/python_modules/dagster/dagster/grpc/client.py", line 9, in <module>
    from grpc_health.v1 import health_pb2
  File "/Users/dgibson/.pyenv/versions/3.8.9/envs/dagster-3.8.9/lib/python3.8/site-packages/grpc_health/v1/health_pb2.py", line 34, in <module>
    _descriptor.EnumValueDescriptor(
  File "/Users/dgibson/.pyenv/versions/3.8.9/envs/dagster-3.8.9/lib/python3.8/site-packages/google/protobuf/descriptor.py", line 755, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
```
  • Loading branch information
gibsondan committed May 26, 2022
1 parent 1e411ad commit 284e912
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,6 @@ grpcio-health-checking<1.44.0
# 5.2+ stops pulling in `ipython_genutils`, on which the old version of `nbconvert` we use
# implicitly depends. Can remove this pin when/if dagit cap on nbconvert is lifted.
nbformat<=5.1.3

# protobuf 4 retroactively breaks old versions of dagster
protobuf>=3.13.0,<4
2 changes: 1 addition & 1 deletion python_modules/dagster/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ def get_version() -> str:
"packaging>=20.9",
"pendulum",
"pep562",
"protobuf>=3.13.0", # ensure version we require is >= that with which we generated the proto code (set in dev-requirements)
"protobuf>=3.13.0,<4", # ensure version we require is >= that with which we generated the proto code (set in dev-requirements)
"python-dateutil",
"pytz",
"requests",
Expand Down

0 comments on commit 284e912

Please sign in to comment.