ImportError: cannot import name 'Iterable' from 'collections' while performing update superset

[lethehoa]

Bug description

I tried to update superset to version 4.1.1
Those step that I did:

1. Change version in docker-compose file
2. superset db upgrade (in worker container)
   Stuck in step 2 with those error

INFO [alembic] Total rows to be processed for tab_state: 0
INFO [alembic] Total rows to be processed for table_schema: 0
Traceback (most recent call last):
File "/usr/local/bin/superset", line 8, in
sys.exit(superset())
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/flask/cli.py", line 357, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/flask_migrate/cli.py", line 149, in upgrade
_upgrade(directory, revision, sql, tag, x_arg)
File "/usr/local/lib/python3.10/site-packages/flask_migrate/init.py", line 98, in wrapped
f(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/flask_migrate/init.py", line 185, in upgrade
command.upgrade(config, revision, sql=sql, tag=tag)
File "/usr/local/lib/python3.10/site-packages/alembic/command.py", line 403, in upgrade
script.run_env()
File "/usr/local/lib/python3.10/site-packages/alembic/script/base.py", line 583, in run_env
util.load_python_file(self.dir, "env.py")
File "/usr/local/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 95, in load_python_file
module = load_module_py(module_id, path)
File "/usr/local/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 113, in load_module_py
spec.loader.exec_module(module) # type: ignore
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "/app/superset/extensions/../migrations/env.py", line 142, in
run_migrations_online()
File "/app/superset/extensions/../migrations/env.py", line 133, in run_migrations_online
context.run_migrations()
File "", line 8, in run_migrations
File "/usr/local/lib/python3.10/site-packages/alembic/runtime/environment.py", line 948, in run_migrations
self.get_context().run_migrations(**kw)
File "/usr/local/lib/python3.10/site-packages/alembic/runtime/migration.py", line 627, in run_migrations
step.migration_fn(**kw)
File "/app/superset/migrations/versions/2024-05-01_10-52_58d051681a3b_add_catalog_perm_to_tables.py", line 47, in upgrade
upgrade_catalog_perms(engines={"postgresql"})
File "/app/superset/migrations/shared/catalogs.py", line 374, in upgrade_catalog_perms
db_engine_spec = database.db_engine_spec
File "/app/superset/models/core.py", line 872, in db_engine_spec
return self.get_db_engine_spec(url)
File "/app/superset/models/core.py", line 881, in get_db_engine_spec
driver = url.get_driver_name()
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/url.py", line 625, in get_driver_name
return self.get_dialect().driver
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/url.py", line 680, in get_dialect
entrypoint = self._get_entrypoint()
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/engine/url.py", line 662, in _get_entrypoint
cls = registry.load(name)
File "/usr/local/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 341, in load
return impl.load()
File "/usr/local/lib/python3.10/site-packages/importlib_metadata/init.py", line 184, in load
module = import_module(match.group('module'))
File "/usr/local/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 992, in _find_and_load_unlocked
File "", line 241, in _call_with_frames_removed
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "/usr/local/lib/python3.10/site-packages/gsheetsdb/init.py", line 1, in
from gsheetsdb.db import connect
File "/usr/local/lib/python3.10/site-packages/gsheetsdb/db.py", line 11, in
from gsheetsdb.query import execute
File "/usr/local/lib/python3.10/site-packages/gsheetsdb/query.py", line 11, in
from moz_sql_parser import parse as parse_sql
File "/usr/local/lib/python3.10/site-packages/moz_sql_parser/init.py", line 15, in
from moz_sql_parser.sql_parser import SQLParser, scrub_literal, scrub
File "/usr/local/lib/python3.10/site-packages/moz_sql_parser/sql_parser.py", line 12, in
from mo_parsing.engine import Engine
File "/usr/local/lib/python3.10/site-packages/mo_parsing/init.py", line 60, in
from mo_parsing.infix import LEFT_ASSOC, RIGHT_ASSOC, infixNotation
File "/usr/local/lib/python3.10/site-packages/mo_parsing/infix.py", line 4, in
from collections import Iterable
ImportError: cannot import name 'Iterable' from 'collections' (/usr/local/lib/python3.10/collections/init.py)
missing expected call export("mo_parsing.core", regex_parameters)
Exception in thread Thread-3 (worker):
Traceback (most recent call last):
File "/usr/local/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.10/threading.py", line 953, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.10/site-packages/mo_imports/init.py", line 204, in worker
_error("Missing export() calls")
File "/usr/local/lib/python3.10/site-packages/mo_imports/init.py", line 211, in _error
raise Exception(description)
Exception: Missing export() calls

As I research so far, in python ver 3.10. They remove the Iterable abstract class from collections, and this error above can be fix by:
import Iterable from collections.abc

Screenshots/recordings

Superset version

4.1.1

Python version

3.10

Node version

I don't know

Browser

Chrome

Additional context

No response

Checklist

• I have searched Superset docs and Slack and didn't find a solution to my problem.
• I have searched the GitHub issue tracker and didn't find a similar bug report.
• I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

[dosubot]

To resolve the ImportError related to 'Iterable' from 'collections' when updating Superset to version 4.1.1 using Python 3.10, you should modify the import statement in the mo_parsing/infix.py file. Change the import from collections to collections.abc as follows:

# Change this import
# from collections import Iterable

# To this import
from collections.abc import Iterable

This change is necessary because, in Python 3.10, Iterable and other abstract base classes were moved from collections to collections.abc ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[rusackas]

What version of Superset are you upgrading from? You may need to install additional dependencies before running the db upgrade. Please refer to the upgrade guide if you haven't already: https://superset.apache.org/docs/installation/upgrading-superset/

[lethehoa]

I'm upgrade from 2.1.0 to 4.1.1
I'm running Superset on docker, so I perform: superset db upgrade & superset init, still got the error.
Btw, I did workaround by these command (Build new image with base superset):

sed -i "s|from collections|from collections.abc|g" /usr/local/lib/python3.10/site-packages/mo_parsing/results.py;
sed -i "s|from collections|from collections.abc|g" /usr/local/lib/python3.10/site-packages/mo_parsing/helpers.py;

Not sure if those working in longterm but solve my problem

What version of Superset are you upgrading from? You may need to install additional dependencies before running the db upgrade. Please refer to the upgrade guide if you haven't already: https://superset.apache.org/docs/installation/upgrading-superset/

[hainenber]

I saw there's a reference to gsheetsdb and try to check this term in the codebase. The result is this notice in UPGRADING.md

In the end, glad that you managed to get to the latest Superset! 👍

I'll be closing this issue now since the original issue is resolved. Hopefully my and Evan's comment will help some future folks with their upgrading 🙏