Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: added flags for event audit #3859

Merged
merged 16 commits into from
Sep 28, 2023
Merged

feat: added flags for event audit #3859

merged 16 commits into from
Sep 28, 2023

Conversation

kanishkkatara
Copy link
Contributor

@kanishkkatara kanishkkatara commented Sep 13, 2023

Description

Fetching event audit enabled flag from workspace config and using it in processor to send events to esch_jobs table

Linear Ticket

https://linear.app/rudderstack/issue/DAT-285/add-ability-for-customer-to-self-serve-the-enablement-disablement-of

Security

  • The code changed/added as part of this pull request won't create any security issues with how the software is being used.

processor/processor.go Outdated Show resolved Hide resolved
Copy link
Contributor

@atzoum atzoum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

eventAuditEnabled needs to be a map[string]bool protected by configSubscriberLock

@codecov
Copy link

codecov bot commented Sep 15, 2023

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (36cdb44) 69.21% compared to head (0f7716b) 69.21%.
Report is 2 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3859      +/-   ##
==========================================
- Coverage   69.21%   69.21%   -0.01%     
==========================================
  Files         357      357              
  Lines       53132    53145      +13     
==========================================
+ Hits        36776    36782       +6     
- Misses      14041    14046       +5     
- Partials     2315     2317       +2     
Files Coverage Δ
backend-config/backend-config.go 84.06% <100.00%> (+0.06%) ⬆️
backend-config/replay_types.go 100.00% <ø> (ø)
backend-config/types.go 88.88% <ø> (ø)
jobsdb/jobsdb.go 73.37% <100.00%> (+0.14%) ⬆️
processor/processor.go 87.42% <100.00%> (+0.02%) ⬆️
services/streammanager/kafka/kafkamanager.go 76.96% <93.75%> (+0.33%) ⬆️

... and 12 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

processor/processor.go Outdated Show resolved Hide resolved
processor/processor.go Outdated Show resolved Hide resolved
processor/processor.go Outdated Show resolved Hide resolved
@@ -121,7 +120,8 @@ func (c *ConfigT) DestinationsMap() map[string]*DestinationT {
}

type Settings struct {
DataRetention DataRetention `json:"dataRetention"`
DataRetention DataRetention `json:"dataRetention"`
EventAuditEnabled bool `json:"eventAuditEnabled"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have we verified that data is unmarshalling into this struct automatically ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

@@ -156,6 +156,8 @@ func filterProcessorEnabledDestinations(config ConfigT) ConfigT {
source.Destinations = destinations
modifiedConfig.Sources = append(modifiedConfig.Sources, source)
}
modifiedConfig.Settings.DataRetention.DisableReportingPII = config.Settings.DataRetention.DisableReportingPII
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't we just use the whole Settings struct instead of selectively picking fields??

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DataRetention has other properties that will get overridden, thats why didn't do this

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not including workspace settings in the TopicProcessConfig looks like a bug to me, cc @cisse21, @BonapartePC

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah true.... this looks like a bug which we had.

@kanishkkatara kanishkkatara merged commit 890ca68 into master Sep 28, 2023
35 checks passed
@kanishkkatara kanishkkatara deleted the feat/event-audit-flag branch September 28, 2023 07:26
atzoum pushed a commit that referenced this pull request Sep 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants