Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Adding logging for datasource/save requests #14476

Merged

Conversation

michellethomas
Copy link
Contributor

@michellethomas michellethomas commented May 4, 2021

SUMMARY

I'd like to log requests to the /datasource/save endpoint so that we know when users alter a datasource.

(Edit: I was previously concerned about logging large data blobs in the json field but the size of this seems in line with other large payloads we are logging like /save_dash, the database like mysql will truncate large blobs). The data that automatically gets logged in the json column can be large (it includes all of the data for the datasource that is getting saved so if the datasource has a lot of columns they will all get logged).

TEST PLAN

Edited a datasource from the /tablemodelview/list/ and explore pages, confirmed logs in the logs table.

ADDITIONAL INFORMATION

  • Has associated issue:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@john-bodley @etr2460

@michellethomas michellethomas changed the title Adding logging for datasource/save requests chore: Adding logging for datasource/save requests May 4, 2021
@michellethomas
Copy link
Contributor Author

I just checked the logs and looks like mysql will truncate at 65535 characters, the /save_dash endpoint is also large so this seems in line with what we are already logging.

@michellethomas michellethomas marked this pull request as ready for review May 5, 2021 22:00
@codecov
Copy link

codecov bot commented May 5, 2021

Codecov Report

Merging #14476 (39d212c) into master (b4e527a) will increase coverage by 0.03%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #14476      +/-   ##
==========================================
+ Coverage   76.91%   76.95%   +0.03%     
==========================================
  Files         955      955              
  Lines       48057    48108      +51     
  Branches     6034     6034              
==========================================
+ Hits        36965    37023      +58     
+ Misses      10894    10887       -7     
  Partials      198      198              
Flag Coverage Δ
hive 80.80% <100.00%> (+0.05%) ⬆️
mysql 81.06% <100.00%> (+0.05%) ⬆️
postgres 81.09% <100.00%> (+0.05%) ⬆️
presto 80.80% <100.00%> (+0.06%) ⬆️
python 81.63% <100.00%> (+0.06%) ⬆️
sqlite 80.70% <100.00%> (+0.05%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/views/datasource.py 88.52% <100.00%> (+0.19%) ⬆️
superset/views/core.py 75.51% <0.00%> (-0.14%) ⬇️
superset/models/core.py 89.13% <0.00%> (ø)
superset/db_engine_specs/base.py 88.09% <0.00%> (ø)
superset/databases/schemas.py 99.52% <0.00%> (+<0.01%) ⬆️
superset/db_engine_specs/postgres.py 96.93% <0.00%> (+0.03%) ⬆️
superset/models/slice.py 86.03% <0.00%> (+0.07%) ⬆️
superset/charts/api.py 81.92% <0.00%> (+0.10%) ⬆️
superset/utils/core.py 89.00% <0.00%> (+0.22%) ⬆️
superset/sql_lab.py 81.90% <0.00%> (+0.33%) ⬆️
... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b4e527a...39d212c. Read the comment docs.

Copy link
Member

@etr2460 etr2460 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@michellethomas michellethomas merged commit 77bc51f into apache:master May 10, 2021
cccs-RyanS pushed a commit to CybercentreCanada/superset that referenced this pull request Dec 17, 2021
Co-authored-by: michelle_thomas <michelle.thomas@airbnb.com>
QAlexBall pushed a commit to QAlexBall/superset that referenced this pull request Dec 29, 2021
Co-authored-by: michelle_thomas <michelle.thomas@airbnb.com>
cccs-rc pushed a commit to CybercentreCanada/superset that referenced this pull request Mar 6, 2024
Co-authored-by: michelle_thomas <michelle.thomas@airbnb.com>
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.3.0 labels Mar 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/XS 🚢 1.3.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants