Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: global logs context #26418

Merged
merged 4 commits into from
Jan 16, 2024

Conversation

eschutho
Copy link
Member

@eschutho eschutho commented Jan 6, 2024

SUMMARY

This is the first PR for #26019 [SIP-109] Global logs context.
It introduces a decorator logs_context that will add permitted values to a global variable logs_context that can later be retrieved for logging information when a larger context is needed than what is available to the method that is calling the logging function. I included an example for slack notifications, but there are more examples in my POC/draft pr #25920 that has more examples of how we can use this decorator.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@eschutho eschutho changed the title Elizabeth/global logs context feat: global logs context Jan 6, 2024
@eschutho eschutho force-pushed the elizabeth/global-logs-context branch 2 times, most recently from c88d248 to 5d40fd9 Compare January 6, 2024 01:10
Copy link

codecov bot commented Jan 6, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (6f6c37e) 69.14% compared to head (329f684) 69.08%.
Report is 24 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #26418      +/-   ##
==========================================
- Coverage   69.14%   69.08%   -0.07%     
==========================================
  Files        1946     1946              
  Lines       75990    76041      +51     
  Branches     8479     8479              
==========================================
- Hits        52544    52531      -13     
- Misses      21267    21331      +64     
  Partials     2179     2179              
Flag Coverage Δ
hive 53.67% <46.15%> (-0.01%) ⬇️
mysql 78.07% <88.46%> (+0.01%) ⬆️
postgres ?
presto 53.62% <46.15%> (-0.01%) ⬇️
python 82.71% <100.00%> (-0.16%) ⬇️
sqlite ?
unit 56.00% <88.46%> (+0.19%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@eschutho eschutho force-pushed the elizabeth/global-logs-context branch from 5d40fd9 to 3f0742d Compare January 8, 2024 17:32
@eschutho eschutho marked this pull request as ready for review January 8, 2024 17:35
@eschutho eschutho force-pushed the elizabeth/global-logs-context branch 2 times, most recently from dcc0967 to 6988b1d Compare January 11, 2024 00:30
@john-bodley john-bodley self-requested a review January 11, 2024 18:00
Comment on lines 79 to 86
"slice_id",
"dashboard_id",
"execution_id",
"report_schedule_id",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious, why do we have a list of allowed keys? Is to prevent logging self?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's to prevent people from logging any unnecessary information or to avoid antipatterns. There's prob more context in the SIP.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a comment in the code for anyone else who may have the same question.

Comment on lines 106 to 107
if key in locals()["kwargs"]:
context_data = {key: locals()["kwargs"][key]}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should be able to use kwargs directly here, no? Instead of locals()["kwargs"].

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! yep, that worked.

from superset.reports.notifications.slack import SlackNotification, WebClient

execution_id = uuid.uuid4()
g.logs_context = {"execution_id": execution_id}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you patch superset.utils.decorators.g instead of relying on the actual g? Something like this:

patch("superset.utils.decorator.g.logs_context", new_callable=dict)

Or maybe patch in the origin:

patch("flask.g.logs_context", new_callable=dict)

Ditto in test_context_decorator below.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. I updated this too.

@eschutho eschutho force-pushed the elizabeth/global-logs-context branch 3 times, most recently from 1cc6846 to 3df2191 Compare January 12, 2024 20:14
@eschutho eschutho force-pushed the elizabeth/global-logs-context branch from 3df2191 to 519ea97 Compare January 12, 2024 20:23
@eschutho eschutho force-pushed the elizabeth/global-logs-context branch from bc2b552 to 329f684 Compare January 13, 2024 00:34
Copy link
Member

@betodealmeida betodealmeida left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sagan-awesome

@eschutho eschutho merged commit aaa4a7b into apache:master Jan 16, 2024
33 checks passed
@eschutho eschutho deleted the elizabeth/global-logs-context branch January 16, 2024 22:44
sadpandajoe pushed a commit to preset-io/superset that referenced this pull request Jan 26, 2024
sfirke pushed a commit to sfirke/superset that referenced this pull request Mar 22, 2024
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 4.0.0 labels Apr 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/L 🚢 4.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants