Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] Implement prebuilt rule upgrade/_review API endpoint #148183

Closed
1 of 2 tasks
Tracked by #174167
banderror opened this issue Dec 29, 2022 · 2 comments · Fixed by #148392 or #154571
Closed
1 of 2 tasks
Tracked by #174167

[Security Solution] Implement prebuilt rule upgrade/_review API endpoint #148183

banderror opened this issue Dec 29, 2022 · 2 comments · Fixed by #148392 or #154571
Assignees
Labels
8.8 candidate Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc.

Comments

@banderror
Copy link
Contributor

banderror commented Dec 29, 2022

Epic: https://github.com/elastic/security-team/issues/1974 (internal)
Related to: #144060, #148181

Summary

Based on the API design made in #144060, implement the new rule upgrade/_review API endpoint and hide it behind the feature flag. See the draft implementation in the POC.

POST /internal/detection_engine/prebuilt_rules/upgrade/_review

Response body:

export interface ReviewRuleUpgradeResponseBody {
  status_code: number;
  message: string;
  attributes: {
    /** Aggregated info about all rules available for upgrade */
    stats: RuleUpgradeStatsForReview;
    /** Info about individual rules: one object per each rule available for upgrade */
    rules: RuleUpgradeInfoForReview[];
  };
}

export interface RuleUpgradeStatsForReview {
  /** Number of installed prebuilt rules available for upgrade (stock + customized) */
  num_rules_to_upgrade: number;
  /** Number of installed prebuilt rules available for upgrade which are stock (non-customized) */
  num_stock_rules_to_upgrade: number;
  /** Number of installed prebuilt rules available for upgrade which are customized by the user */
  num_customized_rules_to_upgrade: number;
  /** A union of all tags of all rules available for upgrade */
  tags: RuleTagArray;
  /** A union of all fields "to be upgraded" across all the rules available for upgrade. An array of field names. */
  fields: string[];
}

export interface RuleUpgradeInfoForReview {
  id: RuleObjectId;
  rule_id: RuleSignatureId;
  rule: DiffableRule;
  diff: {
    fields: {
      name?: ThreeWayDiff<RuleName>;
      description?: ThreeWayDiff<RuleDescription>;
      // etc; only fields that have some changes or conflicts will be returned
    };
    has_conflict: boolean;
  };
}

Todo

  • Initial implementation (PR)
  • Implement RuleUpgradeStatsForReview.tags
@banderror banderror added triage_needed Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Detection Rule Management Security Detection Rule Management Team Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules labels Dec 29, 2022
@banderror banderror self-assigned this Dec 29, 2022
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@banderror banderror changed the title [Security Solution] Implement the new rule upgrade/_preview API endpoint [Security Solution] Implement the new rule upgrade/_review API endpoint Dec 29, 2022
@banderror banderror changed the title [Security Solution] Implement the new rule upgrade/_review API endpoint [Security Solution] Implement the new prebuilt rule upgrade/_review API endpoint Dec 29, 2022
@banderror banderror changed the title [Security Solution] Implement the new prebuilt rule upgrade/_review API endpoint [Security Solution] Implement prebuilt rule upgrade/_review API endpoint Dec 29, 2022
banderror added a commit that referenced this issue Mar 8, 2023
… initial implementation (#148392)

**Addresses:** #148181,
#148182,
#148185
**Partially addresses:**
#148183,
#148189

## Summary

Based on the [POC](#144060), this
PR adds 4 endpoints for the new upgrade and installation workflows for
prebuilt rules:

- `GET /internal/detection_engine/prebuilt_rules/status`
- `POST /internal/detection_engine/prebuilt_rules/upgrade/_review`
- `POST /internal/detection_engine/prebuilt_rules/installation/_review`
- `POST /internal/detection_engine/prebuilt_rules/_generate_assets`
(temporary helper endpoint for development and testing)

The new endpoints are hidden behind a feature flag and can be enabled by
the following config setting:

```yaml
xpack.securitySolution.enableExperimental: ['prebuiltRulesNewUpgradeAndInstallationWorkflowsEnabled']
```

## In the next episodes

Will be done later in follow-up PRs:

- Implementation of some additional response properties for the
`upgrade/_review` endpoint:
  - #148183
- Making base versions optional for diff calculation (we need to support
this in order to be able to still show diffs for rule assets coming from
packages without historical versions):
  - #148189
- Further development of the diff algorithm:
  - #148191
- Test coverage:
  - #148192
sloanelybutsurely pushed a commit to sloanelybutsurely/kibana that referenced this issue Mar 8, 2023
… initial implementation (elastic#148392)

**Addresses:** elastic#148181,
elastic#148182,
elastic#148185
**Partially addresses:**
elastic#148183,
elastic#148189

## Summary

Based on the [POC](elastic#144060), this
PR adds 4 endpoints for the new upgrade and installation workflows for
prebuilt rules:

- `GET /internal/detection_engine/prebuilt_rules/status`
- `POST /internal/detection_engine/prebuilt_rules/upgrade/_review`
- `POST /internal/detection_engine/prebuilt_rules/installation/_review`
- `POST /internal/detection_engine/prebuilt_rules/_generate_assets`
(temporary helper endpoint for development and testing)

The new endpoints are hidden behind a feature flag and can be enabled by
the following config setting:

```yaml
xpack.securitySolution.enableExperimental: ['prebuiltRulesNewUpgradeAndInstallationWorkflowsEnabled']
```

## In the next episodes

Will be done later in follow-up PRs:

- Implementation of some additional response properties for the
`upgrade/_review` endpoint:
  - elastic#148183
- Making base versions optional for diff calculation (we need to support
this in order to be able to still show diffs for rule assets coming from
packages without historical versions):
  - elastic#148189
- Further development of the diff algorithm:
  - elastic#148191
- Test coverage:
  - elastic#148192
bmorelli25 pushed a commit to bmorelli25/kibana that referenced this issue Mar 10, 2023
… initial implementation (elastic#148392)

**Addresses:** elastic#148181,
elastic#148182,
elastic#148185
**Partially addresses:**
elastic#148183,
elastic#148189

## Summary

Based on the [POC](elastic#144060), this
PR adds 4 endpoints for the new upgrade and installation workflows for
prebuilt rules:

- `GET /internal/detection_engine/prebuilt_rules/status`
- `POST /internal/detection_engine/prebuilt_rules/upgrade/_review`
- `POST /internal/detection_engine/prebuilt_rules/installation/_review`
- `POST /internal/detection_engine/prebuilt_rules/_generate_assets`
(temporary helper endpoint for development and testing)

The new endpoints are hidden behind a feature flag and can be enabled by
the following config setting:

```yaml
xpack.securitySolution.enableExperimental: ['prebuiltRulesNewUpgradeAndInstallationWorkflowsEnabled']
```

## In the next episodes

Will be done later in follow-up PRs:

- Implementation of some additional response properties for the
`upgrade/_review` endpoint:
  - elastic#148183
- Making base versions optional for diff calculation (we need to support
this in order to be able to still show diffs for rule assets coming from
packages without historical versions):
  - elastic#148189
- Further development of the diff algorithm:
  - elastic#148191
- Test coverage:
  - elastic#148192
@xcrzx xcrzx reopened this Mar 13, 2023
@banderror banderror assigned xcrzx and unassigned banderror Mar 22, 2023
nkhristinin pushed a commit that referenced this issue Mar 22, 2023
… initial implementation (#148392)

**Addresses:** #148181,
#148182,
#148185
**Partially addresses:**
#148183,
#148189

## Summary

Based on the [POC](#144060), this
PR adds 4 endpoints for the new upgrade and installation workflows for
prebuilt rules:

- `GET /internal/detection_engine/prebuilt_rules/status`
- `POST /internal/detection_engine/prebuilt_rules/upgrade/_review`
- `POST /internal/detection_engine/prebuilt_rules/installation/_review`
- `POST /internal/detection_engine/prebuilt_rules/_generate_assets`
(temporary helper endpoint for development and testing)

The new endpoints are hidden behind a feature flag and can be enabled by
the following config setting:

```yaml
xpack.securitySolution.enableExperimental: ['prebuiltRulesNewUpgradeAndInstallationWorkflowsEnabled']
```

## In the next episodes

Will be done later in follow-up PRs:

- Implementation of some additional response properties for the
`upgrade/_review` endpoint:
  - #148183
- Making base versions optional for diff calculation (we need to support
this in order to be able to still show diffs for rule assets coming from
packages without historical versions):
  - #148189
- Further development of the diff algorithm:
  - #148191
- Test coverage:
  - #148192
xcrzx added a commit that referenced this issue Apr 19, 2023
saarikabhasi pushed a commit to saarikabhasi/kibana that referenced this issue Apr 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
8.8 candidate Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc.
Projects
None yet
3 participants