Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] Implement normalization of ruleSource for API responses #188631

Merged
merged 6 commits into from
Jul 22, 2024

Conversation

jpdjere
Copy link
Contributor

@jpdjere jpdjere commented Jul 18, 2024

Fixes: #180140

Summary

  • Implements normalization ofrule_source for API responses
  • rule_source field in API responses is calculated out of the immutable and ruleSource fields.

For maintainers

@jpdjere jpdjere added release_note:skip Skip the PR/issue when compiling release notes Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Detection Rule Management Security Detection Rule Management Team Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules v8.16.0 labels Jul 18, 2024
@jpdjere jpdjere self-assigned this Jul 18, 2024
@jpdjere jpdjere requested a review from a team as a code owner July 18, 2024 10:27
@jpdjere jpdjere requested a review from nikitaindik July 18, 2024 10:27
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

Comment on lines 15 to 32
export const normalizeRuleSource = ({
immutable,
ruleSource,
}: NormalizeRuleSourceParams): RuleSource => {
if (!ruleSource) {
const normalizedRuleSource = immutable
? {
type: 'external',
isCustomized: false,
}
: {
type: 'internal',
};

return convertObjectKeysToSnakeCase(normalizedRuleSource) as RuleSource;
}
return convertObjectKeysToSnakeCase(ruleSource) as RuleSource;
};
Copy link
Contributor Author

@jpdjere jpdjere Jul 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xcrzx I remember you mentioning handling possible data inconsistencies. For example, data in ES being for whatever reason:

{
    "immutable": false,
    "ruleSource": {
        "type": "external",
        "isCustomized": true
    }
}

It's hard to me to think where these inconsitencies might arise from, but do you think it makes sense to rely always on immutable to calculate rule_source?

In the case above, calculating rule_source to be:

{
        "type": "internal"
}

Or if in ES data looks like:

{
    "immutable": true,
    "ruleSource": {
        "type": "internal",
    }
}

calculating rule_source to be:

{
        "type": "external",
        "is_customized": false
}

WDYT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think what you've implemented is correct. Relying on immutable in responses is not always possible because we also need to return isCustomized, which is calculated on writes. If the ruleSource field is available, we can return it as is. There's synchronization logic implemented on write, so ruleSource should always match immutable if present.

@xcrzx xcrzx requested review from xcrzx and removed request for nikitaindik July 18, 2024 13:09
Comment on lines 42 to 45
rule_source: normalizeRuleSource({
immutable: params.immutable,
ruleSource: params.ruleSource,
}),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should not be normalizing data in case converters, as these functions are solely responsible for converting from camel case to snake case and vice versa. I think what you need is a converter from the alerting rule type to the rule response type.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, thanks.

Created a normalization function to all params, applied in internalRuleToAPIResponse.

@jpdjere jpdjere requested review from a team as code owners July 18, 2024 14:18
@jpdjere jpdjere requested a review from e40pud July 18, 2024 14:18
@jpdjere jpdjere requested a review from xcrzx July 18, 2024 14:52
@jpdjere
Copy link
Contributor Author

jpdjere commented Jul 18, 2024

Hi @e40pud. Small change to a file owned by the DE triggered your review request 🙏 Thanks!

x-pack/plugins/security_solution/server/lib/detection_engine/rule_schema/model/rule_schemas.mock.ts

@elasticmachine
Copy link
Contributor

⏳ Build in-progress, with failures

Failed CI Steps

Test Failures

  • [job] [logs] Jest Tests #12 / useSetupTechnology should update new agent policy and selected policy tab when setup technology is agent-based

History

cc @jpdjere

Copy link
Contributor

@xcrzx xcrzx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@jpdjere jpdjere enabled auto-merge (squash) July 22, 2024 16:04
@kibana-ci
Copy link
Collaborator

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

✅ unchanged

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @jpdjere

@jpdjere jpdjere merged commit 232a166 into elastic:main Jul 22, 2024
44 checks passed
@kibanamachine kibanamachine added the backport:skip This commit does not require backporting label Jul 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules release_note:skip Skip the PR/issue when compiling release notes Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v8.16.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Security Solution] Implement normalization on read for rule_source and immutable fields
6 participants