[ResponseOps][Connectors] Add support of additional fields for ServiceNow ITSM and SecOps #184023

cnasikas · 2024-05-22T14:05:07Z

Summary

This PR adds support for additional fields for the ServiceNow ITSM and SecOps connector. The additional fields will not be available to the recovered action.

Testing

Verify that:

Existing rules with ITSM and SecOps configured continue working as expected.
Can create rules with an ITSM action and set some additional fields supported by ITSM. You can find the available in the Elastic transformation map inside ServiceNow.
The "additional fields" verification in the UI is working as expected.
The "additional fields" are not shown when you set a recovered action.

Fixes: #183609

Checklist

Delete any items that are not applicable to this PR.

Unit or functional tests were updated or added to match the most common scenarios

For maintainers

This was checked for breaking API changes and was labeled appropriately

Release notes

Pass any field to ServiceNow using the ServiceNow ITSM and SecOps connectors with a JSON field called "additional fields".

cnasikas · 2024-05-24T10:51:52Z

/ci

elasticmachine · 2024-05-27T13:58:00Z

Pinging @elastic/response-ops (Team:ResponseOps)

…fix'

…nto itsm_additional_fields

JiaweiWu

Code looks pretty good, just some nitpicks. Will test the behaviour tomorrow.

x-pack/plugins/stack_connectors/server/connector_types/lib/validators.ts

x-pack/plugins/stack_connectors/server/connector_types/lib/servicenow/utils.ts

x-pack/plugins/stack_connectors/server/connector_types/lib/servicenow/schema.ts

JiaweiWu · 2024-05-28T01:04:42Z

x-pack/plugins/stack_connectors/public/connector_types/servicenow_sir/types.ts

+  we need to store the current state. To match with the data structure define in the backend, we make sure users can't
+  send the form while not matching the original object structure. */
+  subActionParams: ExecutorSubActionPushParamsSIR & {
+    incident: { additional_fields: string | null };


what type is this? in the schema file it says additional_fields is a Record<string, any>, here its a string | null

Good question. The frontend sends a string (JSON.stringify(additional_fields)) and the backend parses the string converts it to an object and then validates it. The backend will always have a record (object) after the validation. This is happening automatically by the kbn-schema. This is how all connectors work so far including the webhook connector. An alternative would be to do the parse + validation on the front end and send always a record. Would you like me to do something like that? Any other ideas?

JiaweiWu

I was able to verify on the UI side, everything is working as expected. The additional field is validated correctly and will disappear when alert triggers on recover.

However I triggers an ITSM action with an alert and was not able to see any incidents. I think it's probably because I configured something incorrectly. I used the information you provided in slack. Let me know if you need more information.

cnasikas · 2024-05-30T12:28:27Z

@elasticmachine merge upstream

…nto itsm_additional_fields

maryam-saeidi

Test change LGTM!

cnasikas · 2024-06-03T20:15:15Z

@elasticmachine merge upstream

…nto itsm_additional_fields

cnasikas · 2024-06-12T14:49:39Z

I added a feature flag to accommodate the intermediate release process in 6a593e9 (#184023).

Testing

Verify that you cannot add additional fields to an ITSM and SecOps action in rules from the UI.
Create a rule using the API with an ITSM and SecOps action where the additional fields field is set. Verify that you can see the additional fields in the UI.

cnasikas · 2024-06-13T08:12:52Z

@elasticmachine merge upstream

adcoelho · 2024-06-13T08:57:23Z

Verify that you cannot add additional fields to an ITSM and SecOps action in rules from the UI.

Create a rule using the API with an ITSM and SecOps action where the additional fields field is set.

Verify that you can see the additional fields in the UI.

I tested the above and everything is working as expected.

I had an existing Rule(from a previous version of this PR) with ITSM and SecOps actions where additional_fields were defined.

If I edit this rule I can see additional fields in the UI
If I create a new action the additional fields are not visible
Testing connectors also does not display the additional fields

kibana-ci · 2024-06-13T09:16:19Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 2f79482
Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-184023-2f79482550a1
Observability Deployment

Failed CI Steps

FTR Configs #58

Test Failures

[job] [logs] FTR Configs #58 / Dev Tools Search Profiler Editor supports pre-configured search query

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`stackConnectors`	258	261	+3

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`stackConnectors`	560.7KB	567.4KB	+6.7KB
`uptime`	467.2KB	467.2KB	+23.0B
total			+6.7KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`stackConnectors`	52.6KB	53.4KB	+854.0B

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @cnasikas

…al fields (#186949) ## Summary In #184023, we introduced the "additional fields" field for ServiceNow ITSM and SecOps. The field was under a feature flag to follow the intermediate release process. This PR reverts commit 6a593e9 to remove the feature flag. ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

cnasikas added 6 commits May 22, 2024 16:02

Share Jira validators

1439aba

Use shared validators for ITSM and SIR

abf152e

Create additional fields component

f367b32

Create validateJSON util function

ee8c47d

Use additional fields component in ITSM and SIR

88d4ad2

Validate additional fields using validateJSON in ITSM and SIR

b85060c

cnasikas added release_note:enhancement Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) Feature:Actions/ConnectorTypes Issues related to specific Connector Types on the Actions Framework v8.15.0 labels May 22, 2024

cnasikas self-assigned this May 22, 2024

cnasikas added 2 commits May 23, 2024 18:47

Merge branch 'main' into itsm_additional_fields

835a290

Add tests

742cc75

Merge branch 'main' into itsm_additional_fields

3defe90

cnasikas marked this pull request as ready for review May 27, 2024 13:57

cnasikas requested a review from a team as a code owner May 27, 2024 13:57

cnasikas and others added 7 commits May 27, 2024 17:03

Improve helptext

b466ac6

[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…

1e897fd

…fix'

Fix bug where the errors was showing twice

432c0ed

Merge branch 'itsm_additional_fields' of github.com:cnasikas/kibana i…

21924ac

…nto itsm_additional_fields

Hide additional fields from recovered action

4f7fb22

Fix i18n

517079b

Fix types

829e196

JiaweiWu reviewed May 28, 2024

View reviewed changes

cnasikas added 3 commits May 28, 2024 11:45

PR feedback

0911772

Fix tests

5b9ad7b

Merge branch 'main' into itsm_additional_fields

b326d8f

JiaweiWu approved these changes May 29, 2024

View reviewed changes

smith added the apm:review label May 29, 2024

kibanamachine and others added 4 commits May 30, 2024 08:28

Merge branch 'main' into itsm_additional_fields

7a9451b

Merge branch 'main' into itsm_additional_fields

aa71f73

Fix test

65df354

Merge branch 'itsm_additional_fields' of github.com:cnasikas/kibana i…

aecce0f

…nto itsm_additional_fields

cnasikas requested a review from a team as a code owner May 30, 2024 13:54

crespocarlos approved these changes May 30, 2024

View reviewed changes

maryam-saeidi approved these changes May 30, 2024

View reviewed changes

kibanamachine and others added 6 commits June 3, 2024 16:15

Merge branch 'main' into itsm_additional_fields

1259aa5

Merge branch 'main' into itsm_additional_fields

8faa537

Merge branch 'main' into itsm_additional_fields

854a83a

Merge branch 'main' into itsm_additional_fields

95813fd

Add feature flag for the intermediate release process

6a593e9

Merge branch 'itsm_additional_fields' of github.com:cnasikas/kibana i…

896f3fa

…nto itsm_additional_fields

Fix tests

21909ec

Merge branch 'main' into itsm_additional_fields

2f79482

cnasikas enabled auto-merge (squash) June 13, 2024 09:07

cnasikas merged commit 75f3af5 into elastic:main Jun 13, 2024
36 of 37 checks passed

cnasikas deleted the itsm_additional_fields branch June 13, 2024 10:03

kibanamachine added the backport:skip This commit does not require backporting label Jun 13, 2024

cnasikas mentioned this pull request Jun 26, 2024

[ResponseOps][Connectors] Remove feature flag for ServiceNow additional fields #186949

Merged

1 task

This was referenced Oct 1, 2024

[Actions] ServiceNow implement support for adding custom incident fields. #78237

Closed

Handle additional ServiceNow fields in UI elastic/uptime#300

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ResponseOps][Connectors] Add support of additional fields for ServiceNow ITSM and SecOps #184023

[ResponseOps][Connectors] Add support of additional fields for ServiceNow ITSM and SecOps #184023

cnasikas commented May 22, 2024 •

edited

Loading

cnasikas commented May 24, 2024

elasticmachine commented May 27, 2024

JiaweiWu left a comment

JiaweiWu May 28, 2024

cnasikas May 28, 2024 •

edited

Loading

JiaweiWu left a comment

cnasikas commented May 30, 2024

maryam-saeidi left a comment

cnasikas commented Jun 3, 2024

cnasikas commented Jun 12, 2024 •

edited

Loading

cnasikas commented Jun 13, 2024

adcoelho commented Jun 13, 2024

kibana-ci commented Jun 13, 2024 •

edited

Loading

[ResponseOps][Connectors] Add support of additional fields for ServiceNow ITSM and SecOps #184023

[ResponseOps][Connectors] Add support of additional fields for ServiceNow ITSM and SecOps #184023

Conversation

cnasikas commented May 22, 2024 • edited Loading

Summary

Testing

Checklist

For maintainers

Release notes

cnasikas commented May 24, 2024

elasticmachine commented May 27, 2024

JiaweiWu left a comment

Choose a reason for hiding this comment

JiaweiWu May 28, 2024

Choose a reason for hiding this comment

cnasikas May 28, 2024 • edited Loading

Choose a reason for hiding this comment

JiaweiWu left a comment

Choose a reason for hiding this comment

cnasikas commented May 30, 2024

maryam-saeidi left a comment

Choose a reason for hiding this comment

cnasikas commented Jun 3, 2024

cnasikas commented Jun 12, 2024 • edited Loading

Testing

cnasikas commented Jun 13, 2024

adcoelho commented Jun 13, 2024

kibana-ci commented Jun 13, 2024 • edited Loading

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

Module Count

Async chunks

Page load bundle

History

cnasikas commented May 22, 2024 •

edited

Loading

cnasikas May 28, 2024 •

edited

Loading

cnasikas commented Jun 12, 2024 •

edited

Loading

kibana-ci commented Jun 13, 2024 •

edited

Loading