Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Adding configuration_method column to Database Model #14433

Merged
merged 6 commits into from
May 14, 2021

Conversation

AAfghahi
Copy link
Member

@AAfghahi AAfghahi commented Apr 30, 2021

SUMMARY

We are adding a new enum column to the Database model that identifies which method was used in its' creation or updating.

The logic for this is written here: #14451

Reason

This column is necessary so that front end components know what information to present to the user when they are editing a database.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TEST PLAN

Coming Soon.

ADDITIONAL INFORMATION

  • Has associated issue:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@AAfghahi AAfghahi requested a review from a team as a code owner April 30, 2021 15:14
@craig-rueda
Copy link
Member

I only see a single migration file. Where is the enum defined, etc? 🤔

@AAfghahi
Copy link
Member Author

I only see a single migration file. Where is the enum defined, etc? 🤔

Hey @craig-rueda! I am working on all of that in another PR, didn't want to add anything more to a migration. I'll send you a link when it is done!

@craig-rueda
Copy link
Member

Why not just combine the two? That keeps your relevant changes together, making things easier to reason about, test, etc.

@github-actions
Copy link
Contributor

github-actions bot commented May 3, 2021

⚠️ @AAfghahi Your base branch master has just also updated superset/migrations.

Please consider rebasing your branch to avoid db migration conflicts.

Copy link
Member

@betodealmeida betodealmeida left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@craig-rueda we've been doing the migration scripts on separate PRs because it makes it easier to cherry pick them if someone needs to cherry-pick a subsequent PR with a downstream migration (ie, we need to cherry pick a PR that has a migration that revises this one). Since this PR only adds a column it can be cherry-picked safely and the logic can be left out.

Additionally, we only merge this PR once the accompanying PR with the logic has also been approved and is ready to merge.

def upgrade():
with op.batch_alter_table("dbs") as batch_op:
batch_op.add_column(
sa.Column("save_option", sa.VARCHAR(255), server_default="SQL_ALCHEMY")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think save_option is not a super informative nam), something like configuration_method would be better. And let's use SQLALCHEMY_URI as the default.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds good, I appreciate you helping me with naming conventions, it is something that I definitely struggle with.

@betodealmeida
Copy link
Member

@AAfghahi it might be better leaving this PR as draft until the one with the logic is implemented and ready to merge. Then you can also link them to each other in the description.

@AAfghahi AAfghahi marked this pull request as draft May 3, 2021 15:51
@AAfghahi
Copy link
Member Author

AAfghahi commented May 3, 2021

@AAfghahi it might be better leaving this PR as draft until the one with the logic is implemented and ready to merge. Then you can also link them to each other in the description.

converted!

@AAfghahi AAfghahi marked this pull request as ready for review May 6, 2021 21:12
@AAfghahi AAfghahi changed the title feat: Adding save_option column to Database Model feat: Adding configuration_method column to Database Model May 6, 2021
@codecov
Copy link

codecov bot commented May 6, 2021

Codecov Report

Merging #14433 (d6d7bdc) into master (1df9384) will increase coverage by 0.16%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #14433      +/-   ##
==========================================
+ Coverage   77.15%   77.31%   +0.16%     
==========================================
  Files         958      959       +1     
  Lines       48241    48393     +152     
  Branches     5636     5636              
==========================================
+ Hits        37219    37416     +197     
+ Misses      10821    10776      -45     
  Partials      201      201              
Flag Coverage Δ
hive 81.09% <ø> (+0.29%) ⬆️
mysql 81.36% <ø> (+0.30%) ⬆️
postgres 81.39% <ø> (+0.29%) ⬆️
presto 81.08% <ø> (+0.28%) ⬆️
python 81.91% <ø> (+0.28%) ⬆️
sqlite 81.00% <ø> (+0.30%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/connectors/base/models.py 88.03% <0.00%> (-2.87%) ⬇️
superset/reports/logs/api.py 94.11% <0.00%> (-1.34%) ⬇️
superset/exceptions.py 98.70% <0.00%> (-1.30%) ⬇️
superset/db_engine_specs/presto.py 89.89% <0.00%> (-0.43%) ⬇️
superset/app.py 81.31% <0.00%> (-0.07%) ⬇️
superset/views/core.py 75.48% <0.00%> (-0.04%) ⬇️
superset/errors.py 100.00% <0.00%> (ø)
superset/constants.py 100.00% <0.00%> (ø)
superset/utils/urls.py 100.00% <0.00%> (ø)
superset/examples/country_map.py 26.31% <0.00%> (ø)
... and 22 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1df9384...d6d7bdc. Read the comment docs.

with op.batch_alter_table("dbs") as batch_op:
batch_op.add_column(
sa.Column(
"configuration_method", sa.VARCHAR(255), server_default="SQLALCHEMY_URI"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this column nullable?

Copy link
Member Author

@AAfghahi AAfghahi May 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, this is a good question. I think that it should not be nullable, because it is something assigned at creation.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree.. any existing rows should default to the current method which is the sqlalchemy uri.

@@ -99,6 +100,11 @@ class CssTemplate(Model, AuditMixinNullable):
css = Column(Text, default="")


class ConfigurationMethod(str, enum.Enum):
SQLALCHEMY_URI = "sqlalchemy_form"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we make the var equal to the value

Suggested change
SQLALCHEMY_URI = "sqlalchemy_form"
SQLALCHEMY_FORM= "sqlalchemy_form"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done, pushing up now.

@hughhhh
Copy link
Member

hughhhh commented May 14, 2021

LGTM, but will let @betodealmeida do the approval

import sqlalchemy as sa
from alembic import op

from superset.models.core import ConfigurationMethod
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's hard code this to ensure the migration script doesn't get broken if someone changes the enum in the future.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, I wasn't sure what method would be preferable, to hard code or to import, looking through the versions I saw a little bit of both.

@betodealmeida betodealmeida merged commit b064cc1 into apache:master May 14, 2021
@jmistry
Copy link
Contributor

jmistry commented May 17, 2021

Hi I think this change may have introduced a bug on the front end, i get this error when adding a new database
"An error occurred while creating databases: (configuration_method) Missing data for required field."

@AAfghahi
Copy link
Member Author

hey, I think that issue is set to be fixed via this: #14583

which creates a new database edit. @betodealmeida @eschutho I can make this field optional till then?

@AAfghahi
Copy link
Member Author

@jmistry I pushed up a change to make it optional for the time being: #14668

cccs-RyanS pushed a commit to CybercentreCanada/superset that referenced this pull request Dec 17, 2021
)

* db migration for dbs

* changing naming conventions

* added in new migration

* made server default an enum

* added enum to migration file

* removed enum import
QAlexBall pushed a commit to QAlexBall/superset that referenced this pull request Dec 29, 2021
)

* db migration for dbs

* changing naming conventions

* added in new migration

* made server default an enum

* added enum to migration file

* removed enum import
cccs-rc pushed a commit to CybercentreCanada/superset that referenced this pull request Mar 6, 2024
)

* db migration for dbs

* changing naming conventions

* added in new migration

* made server default an enum

* added enum to migration file

* removed enum import
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.3.0 labels Mar 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/M 🚢 1.3.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants