Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add provision to disable tracking event names from a source for reporting #3632

Merged
merged 7 commits into from
Jul 18, 2023

Conversation

satishrudderstack
Copy link
Contributor

@satishrudderstack satishrudderstack commented Jul 14, 2023

Description

  • One rETL source had numbers in event_name field. These were all user_ids. This causes cardinality increase in reporting
  • In this PR, we are adding a functionality to disable event_name tracking for reporting from source

Notion Ticket

https://www.notion.so/rudderstacks/Reports-Flush-Issue-from-Rudder-Server-Instance-9-on-prousmtusmt-c9592bd2b2b74d05a9e083ad8baa7988

Security

  • The code changed/added as part of this pull request won't create any security issues with how the software is being used.

@codecov
Copy link

codecov bot commented Jul 14, 2023

Codecov Report

Patch coverage: 78.94% and project coverage change: +0.01 🎉

Comparison is base (26c1791) 68.09% compared to head (8c452ec) 68.11%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3632      +/-   ##
==========================================
+ Coverage   68.09%   68.11%   +0.01%     
==========================================
  Files         318      318              
  Lines       50364    50371       +7     
==========================================
+ Hits        34295    34308      +13     
+ Misses      13848    13840       -8     
- Partials     2221     2223       +2     
Impacted Files Coverage Δ
enterprise/reporting/reporting.go 31.01% <78.94%> (+0.18%) ⬆️

... and 4 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

Comment on lines 75 to 79
sourcesWithEventNameTrackingDisabledEnv := config.GetString("SOURCES_WITH_EVENT_NAME_TRACKING_DISABLED", "")
sourcesWithEventNameTrackingDisabled := []string{}
if len(sourcesWithEventNameTrackingDisabledEnv) > 0 {
sourcesWithEventNameTrackingDisabled = strings.Split(sourcesWithEventNameTrackingDisabledEnv, ",")
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GetStringSlice can be used instead.

Suggested change
sourcesWithEventNameTrackingDisabledEnv := config.GetString("SOURCES_WITH_EVENT_NAME_TRACKING_DISABLED", "")
sourcesWithEventNameTrackingDisabled := []string{}
if len(sourcesWithEventNameTrackingDisabledEnv) > 0 {
sourcesWithEventNameTrackingDisabled = strings.Split(sourcesWithEventNameTrackingDisabledEnv, ",")
}
sourcesWithEventNameTrackingDisabled := config.GetStringSlice("SOURCES_WITH_EVENT_NAME_TRACKING_DISABLED", "")

Comment on lines 591 to 594
if r.IsEventNameTrackingDisabledForSource(metric.ConnectionDetails.SourceID) {
metric = transformMetricForEventNameTrackingDisabled(metric)
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few suggestions that are somewhat idiomatic and a bit subjective:

  • I would use slices.Contains, for consistency and readability.
  • I would not create new functions since the logic they are encapsulating is so simple, one line each assuming slices.Contains is used.
Suggested change
if r.IsEventNameTrackingDisabledForSource(metric.ConnectionDetails.SourceID) {
metric = transformMetricForEventNameTrackingDisabled(metric)
}
if slices.Contains(sourcesWithEventNameTrackingDisabled, metric.ConnectionDetails.SourceID) {
metric.StatusDetail.EventName = metric.StatusDetail.EventType
}

@satishrudderstack
Copy link
Contributor Author

@lvrach addressed the review comments

@satishrudderstack satishrudderstack merged commit d80fee2 into master Jul 18, 2023
36 checks passed
@satishrudderstack satishrudderstack deleted the chore.eventname-reporting branch July 18, 2023 10:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants