Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AttributeError on superset db upgrade #7514

Closed
KilimAnnejaro opened this issue May 15, 2019 · 15 comments

Comments

Projects
None yet
3 participants
@KilimAnnejaro
Copy link
Contributor

commented May 15, 2019

Hi, when I tried to run superset db upgrade I got the error "AttributeError: module 'sqlalchemy.util' has no attribute 'deprecated_params'". I already tried downgrading my version of sqlalchemy but it didn't help.

Screenshots

Screenshot from 2019-05-15 12-11-39

How to reproduce the bug

Run installation commands from https://superset.incubator.apache.org/installation.html, except with pip3 rather than pip.

Environment

(please complete the following information):

  • superset version: Most recent
  • python version: Python 2.7.15 :: Anaconda, Inc. (Note: I am using pip3 and python3 to get around version 2 req's.)
    UPDATE: python version now Python 3.6.7 :: Anaconda, Inc. -- still getting the same error.
  • node.js version: v11.6.0
  • npm version: 6.5.0-next.0

Checklist

Make sure these boxes are checked before submitting your issue - thank you!

  • [ x ] I have checked the superset logs for python stacktraces and included it here as text if there are any.
  • [ x ] I have reproduced the issue with at least the latest released version of superset.
  • [ x ] I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context

Add any other context about the problem here.

@issue-label-bot issue-label-bot bot added the #bug label May 15, 2019

@issue-label-bot

This comment has been minimized.

Copy link

commented May 15, 2019

Issue-Label Bot is automatically applying the label #bug to this issue, with a confidence of 0.88. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

@villebro

This comment has been minimized.

Copy link
Contributor

commented May 15, 2019

Is py3.6 installed via Anaconda?

@KilimAnnejaro

This comment has been minimized.

Copy link
Contributor Author

commented May 15, 2019

It wasn't at first, and when I ran conda install python3 I got the following error:
PackagesNotFoundError: The following packages are not available from current channels:

  • python3

I also ran the command conda install python==3.6.7, which did update the version, but I was still getting the deprecated params issue.

@elukey

This comment has been minimized.

Copy link
Contributor

commented May 16, 2019

@KilimAnnejaro what sqlalchemy version have you tried? Also, are you testing superset 0.32 or the latest master? pip freeze's output would also probably help to have in here. I think that we could easily find when sqlalchemy.utils removed the "deprecated_params" and see what version is best?

EDIT: I think that sqlalchemy >= 1.3.0 is needed.

@KilimAnnejaro

This comment has been minimized.

Copy link
Contributor Author

commented May 16, 2019

This is my pip freeze output:

alembic==1.0.10
amqp==2.4.2
apispec==1.3.2
asn1crypto==0.24.0
attrs==19.1.0
Babel==2.6.0
billiard==3.6.0.0
bleach==3.1.0
boto3==1.4.7
botocore==1.7.48
cchardet==2.1.4
celery==4.3.0
certifi==2019.3.9
cffi==1.12.3
chardet==3.0.4
click==6.7
colorama==0.3.9
conda==4.6.14
contextlib2==0.5.5
croniter==0.3.30
cryptography==2.6.1
decorator==4.4.0
defusedxml==0.6.0
docutils==0.14
et-xmlfile==1.0.1
Flask==0.12.4
Flask-AppBuilder==2.0.0
Flask-Babel==0.12.2
Flask-Caching==1.7.1
Flask-Compress==1.4.0
Flask-JWT-Extended==3.18.1
Flask-Login==0.4.1
Flask-Migrate==2.4.0
Flask-OpenID==1.2.5
Flask-SQLAlchemy==2.4.0
Flask-WTF==0.14.2
flower==0.9.3
future==0.16.0
geographiclib==1.49
geopy==1.19.0
gunicorn==19.9.0
humanize==0.5.1
idna==2.8
ijson==2.3
isodate==0.6.0
itsdangerous==1.1.0
jdcal==1.4.1
Jinja2==2.10.1
jmespath==0.9.4
jsonlines==1.2.0
jsonschema==3.0.1
kombu==4.5.0
linear-tsv==1.1.0
Mako==1.0.9
Markdown==3.1
MarkupSafe==1.1.1
marshmallow==2.19.2
marshmallow-enum==1.4.1
marshmallow-sqlalchemy==0.16.2
mkl-fft==1.0.12
mkl-random==1.0.2
numpy==1.16.3
openpyxl==2.4.11
pandas==0.24.2
parsedatetime==2.4
pathlib2==2.3.3
polyline==1.3.2
prison==0.1.0
psycopg2==2.7.7
py==1.8.0
pycosat==0.6.3
pycparser==2.19
pydruid==0.5.2
PyHive==0.6.1
PyJWT==1.7.1
pyOpenSSL==19.0.0
pyrsistent==0.15.1
PySocks==1.6.8
python-dateutil==2.8.0
python-editor==1.0.4
python-geohash==0.8.5
python3-openid==3.1.0
pytz==2019.1
PyYAML==5.1
requests==2.21.0
retry==0.9.2
rfc3986==1.3.1
ruamel-yaml==0.15.46
s3transfer==0.1.13
sasl==0.2.1
selenium==3.141.0
simplejson==3.16.0
six==1.12.0
SQLAlchemy==1.3.3
SQLAlchemy-Utils==0.33.11
sqlparse==0.3.0
superset==0.28.1
tableschema==1.4.1
tabulator==1.20.0
thrift==0.11.0
thrift-sasl==0.3.0
tornado==5.1.1
unicodecsv==0.14.1
Unidecode==1.0.23
urllib3==1.24.3
vine==1.3.0
webencodings==0.5.1
Werkzeug==0.15.2
WTForms==2.2.1
WTForms-JSON==0.3.3
xlrd==1.2.0

I'm testing the latest master.

Thank you!

@elukey

This comment has been minimized.

Copy link
Contributor

commented May 16, 2019

@KilimAnnejaro I just noticed that marshmallow-sqlalchemy is at the top of the stacktrace, maybe you'd need to use a more recent version? I can see this one in master's requirements.txt: https://github.com/apache/incubator-superset/blob/master/requirements.txt#L53

@KilimAnnejaro

This comment has been minimized.

Copy link
Contributor Author

commented May 16, 2019

I think I'm using the same version as in the requirements.txt? But at any rate I just tried pip install marshmallow-sqlalchemy and the Attribute error didn't change

@elukey

This comment has been minimized.

Copy link
Contributor

commented May 16, 2019

You are totally right, confused some package names. Really strange then..
The error seems coming from sqlachemy itself, maybe it could be worth to test another version of sqlalchemy? https://pypi.org/project/SQLAlchemy/1.3.3/#history

@KilimAnnejaro

This comment has been minimized.

Copy link
Contributor Author

commented May 16, 2019

I tried that and got this error: sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Try adding an explicit ON clause to help resolve the ambiguity.

So I used the fix from #6977 and got the new error "ModuleNotFoundError: No module named 'sqlalchemy'". Which feels really strange to me, since when I run pip freeze I get this output:

alembic==1.0.10
amqp==2.4.2
apispec==1.3.2
asn1crypto==0.24.0
attrs==19.1.0
Babel==2.6.0
billiard==3.6.0.0
bleach==3.1.0
boto3==1.4.7
botocore==1.7.48
cchardet==2.1.4
celery==4.3.0
certifi==2019.3.9
cffi==1.12.3
chardet==3.0.4
click==6.7
colorama==0.3.9
conda==4.6.14
contextlib2==0.5.5
croniter==0.3.30
cryptography==2.6.1
decorator==4.4.0
defusedxml==0.6.0
docutils==0.14
et-xmlfile==1.0.1
Flask==0.12.4
Flask-AppBuilder==2.0.0
Flask-Babel==0.12.2
Flask-Caching==1.7.1
Flask-Compress==1.4.0
Flask-JWT-Extended==3.18.1
Flask-Login==0.4.1
Flask-Migrate==2.4.0
Flask-OpenID==1.2.5
Flask-SQLAlchemy==2.4.0
Flask-WTF==0.14.2
flower==0.9.3
future==0.16.0
geographiclib==1.49
geopy==1.19.0
gunicorn==19.9.0
humanize==0.5.1
idna==2.8
ijson==2.3
isodate==0.6.0
itsdangerous==1.1.0
jdcal==1.4.1
Jinja2==2.10.1
jmespath==0.9.4
jsonlines==1.2.0
jsonschema==3.0.1
kombu==4.5.0
linear-tsv==1.1.0
Mako==1.0.9
Markdown==3.1
MarkupSafe==1.1.1
marshmallow==2.19.2
marshmallow-enum==1.4.1
marshmallow-sqlalchemy==0.16.2
mkl-fft==1.0.12
mkl-random==1.0.2
numpy==1.16.3
openpyxl==2.4.11
pandas==0.24.2
parsedatetime==2.4
pathlib2==2.3.3
polyline==1.3.2
prison==0.1.0
psycopg2==2.7.7
py==1.8.0
pycosat==0.6.3
pycparser==2.19
pydruid==0.5.2
PyHive==0.6.1
PyJWT==1.7.1
pyOpenSSL==19.0.0
pyrsistent==0.15.1
PySocks==1.6.8
python-dateutil==2.8.0
python-editor==1.0.4
python-geohash==0.8.5
python3-openid==3.1.0
pytz==2019.1
PyYAML==5.1
requests==2.21.0
retry==0.9.2
rfc3986==1.3.1
ruamel-yaml==0.15.46
s3transfer==0.1.13
sasl==0.2.1
selenium==3.141.0
simplejson==3.16.0
six==1.12.0
SQLAlchemy==1.3.2
SQLAlchemy-Utils==0.33.11
sqlparse==0.3.0
superset==0.28.1
tableschema==1.4.1
tabulator==1.20.0
thrift==0.11.0
thrift-sasl==0.3.0
tornado==5.1.1
unicodecsv==0.14.1
Unidecode==1.0.23
urllib3==1.24.3
vine==1.3.0
webencodings==0.5.1
Werkzeug==0.15.2
WTForms==2.2.1
WTForms-JSON==0.3.3
xlrd==1.2.0

@KilimAnnejaro

This comment has been minimized.

Copy link
Contributor Author

commented May 16, 2019

I think I should note that when I run python in interactive mode and try to 'import sqlalchemy', I get the error initially described in this issue ("AttributeError: module 'sqlalchemy.util' has no attribute 'deprecated_params'"). So I think the problem here is definitely sqlalchemy, and not so much superset.

@elukey

This comment has been minimized.

Copy link
Contributor

commented May 16, 2019

Have you also tried with sqlalchemy==1.2.18 ?

@KilimAnnejaro

This comment has been minimized.

Copy link
Contributor Author

commented May 16, 2019

Okay, I fixed this issue.

I was developing outside the virtual environment recommended by the installation instructions. When I did this, I couldn't install a sufficiently low version of sqlalchemy. Once I got my venv environment to work, the issue was fixed.

Thanks to everyone for their help!

@villebro

This comment has been minimized.

Copy link
Contributor

commented May 16, 2019

hmm, you seem to have superset 0.28.1 installed via pip. I'm fairly certain that doesn't work with sqlalchemy 1.3. I would recommend making a fresh venv with eg. pipenv and stay away from conda etc and make sure you install 0.32rc2, which works much better with recent deps.

@KilimAnnejaro

This comment has been minimized.

Copy link
Contributor Author

commented May 16, 2019

@villebro that's what I did and it worked

@villebro

This comment has been minimized.

Copy link
Contributor

commented May 16, 2019

haha I seem to have typed that at the same time as you. happy to hear things worked out 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.