pypi installation vs. dependencies

[maederm]

Hi

I'm using Superset using the pypi installation method. It makes it easier to add DB drivers and other dependencies.

However, it seems that some workarounds are needed for it to work, which I wouldn’t need with the Docker method. As of 6.1.0, I needed to add these extra dependencies which I assume should live in the project's pyproject.toml.

Additional entries required for 6.1.0:

# Workaround for https://github.com/apache/superset/issues/40962
cachetools = ">=5.3.0,<7"
# Workaround for missing apache-superset dependency (didn't find an issue)
rich = "^15.0.0"
# Workaround for https://github.com/apache/superset/discussions/40172#discussioncomment-16980371
flask-sqlalchemy = "^2.5.1"

It seems that I can't expect the dependencies in pyproject.toml to be up to date. Is that assumption correct?
Should I use something like uv pip compile with the base.txt instead?
Or is this something that should work, and could I contribute a fix

[dosubot[bot]]

Hi @maederm! I'm Dosu and I'm helping the Apache Superset team.

Your observation is correct — there is a real gap between Docker and PyPI installations. The root cause is that Docker builds install from requirements/base.txt [1], which is a compiled lockfile with all transitive dependencies pinned to tested versions, while pip install apache-superset uses only pyproject.toml with version ranges and relies on dynamic resolution of transitive dependencies [2].

Here's what's happening with your three workarounds:

• cachetools — This is a confirmed bug (cachetools is a Superset requirement but is not explicitly required #40962). Superset directly imports cachetools (in aws_iam.py), but it was only pulled in transitively via google-auth. When google-auth v2.53.0+ dropped cachetools from its dependencies, PyPI installs broke. Docker still works because base.txt pins google-auth==2.43.0 which still includes it [3]. This should be added as an explicit dependency in pyproject.toml.

• flask-sqlalchemy — The pyproject.toml doesn't pin this directly; it comes in via flask-appbuilder and flask-migrate [4]. PyPI resolution can pull in v3.x, which has breaking API changes. Docker pins it to 2.5.1.

• rich — This is a transitive dependency via flask-limiter [5] and isn't declared directly in pyproject.toml.

To answer your questions:

1. Is pyproject.toml expected to be incomplete? Not intentionally, but in practice it only declares top-level dependencies with loose ranges, and the project is primarily tested against the pinned base.txt versions. So yes, PyPI installs can drift into untested territory.

2. Should you use uv pip compile with base.txt? Using requirements/base.txt directly (or compiling from it) would give you the same pinned environment Docker uses — that's the most reliable approach for PyPI-based installs today.

3. Can you contribute a fix? Absolutely — adding cachetools as an explicit dependency in pyproject.toml (as suggested in cachetools is a Superset requirement but is not explicitly required #40962) and tightening the flask-sqlalchemy version constraint would be welcome contributions. These are legitimate gaps that affect PyPI users.

To reply, just mention @dosu.

---

Share context across your team and agents. Try Dosu.