Add Torq Connector #149405

natexcvi · 2023-01-24T13:03:12Z

Summary

Add a new action type for Torq which triggers Torq workflows.

This is a re-do of #139635 ...

Checklist

Delete any items that are not applicable to this PR.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
Any UI touched in this PR is usable by keyboard only (learn more about keyboard accessibility)
Any UI touched in this PR does not create any new axe failures (run axe in browser: FF, Chrome)

Risk Matrix

Delete this section if it is not applicable to this PR.

Before closing this PR, invite QA, stakeholders, and other developers to identify risks that should be tested prior to the change/feature release.

When forming the risk matrix, consider some of the following examples and how they may potentially impact the change:

Risk	Probability	Severity	Mitigation/Notes
Multiple Spaces—unexpected behavior in non-default Kibana Space.	Low	High	Integration tests will verify that all features are still supported in non-default Kibana Space and when user switches between spaces.
Multiple nodes—Elasticsearch polling might have race conditions when multiple Kibana nodes are polling for the same tasks.	High	Low	Tasks are idempotent, so executing them multiple times will not result in logical error, but will degrade performance. To test for this case we add plenty of unit tests around this logic and document manual testing procedure.
Code should gracefully handle cases when feature X or plugin Y are disabled.	Medium	High	Unit tests will verify that any feature flag or plugin combination still results in our service operational.
See more potential risk examples

For maintainers

This was checked for breaking API changes and was labeled appropriately

…q_connector

Add Torq Connector

pmuellr

I think we need to change the renderParameterTemplate bit to use json and not none, but not completely sure. Changing the tests to use an invalid character in a variable should indicate whether it's needed or not.

Other than, LGTM.

pmuellr · 2023-01-27T21:49:42Z

x-pack/plugins/stack_connectors/server/connector_types/torq/index.ts

+): ActionParamsType {
+  if (!params.body) return params;
+  return {
+    body: renderMustacheString(params.body, variables, 'none'),


hmmm ... I think this needs to be 'json' and not 'none'. For example, if the character " is embedded in one of the variable strings, and interpolated via mustache into a property value, the " won't be escaped and the JSON won't be valid. Using 'json' will escape characters that need to be escaped for JSON.

It would actually be good to create a test with a variable whose value contains a ", to ensure the result is valid JSON. I think it can go in x-pack/plugins/stack_connectors/server/connector_types/torq/index.test.ts, probably just add another variable in that test ...

I think I found a proper solution for this by using the renderMustacheObject method. Also adjusted the test as you suggested to confirm we get the best of both worlds: objects are rendered without JSON escaping and strings are JSON-escaped. LMKWYT

pmuellr · 2023-01-27T21:59:57Z

x-pack/plugins/stack_connectors/public/connector_types/torq/torq_connectors.tsx

+    };
+    if (!isUrl(value)) return error;
+    const hostname = new URL(value).hostname;
+    return hostname === 'hooks.torq.io' ? undefined : error;


As a future thought, since the UX doesn't seem to allow ANYTHING but the torq URL (for example, I can't use localhost here, but it appears the server API supports it for testing), I wonder if we should at least pre-fill the value - or perhaps we don't even need it at all.

This is a good point. When creating an Elastic integration on Torq, the user can copy the full webhook URL and paste it on Kibana, so in most cases a pre-filled value will be overwritten. Do you think we should pre-fill the prefix anyway, to give a hint as for what the URL should look like? My only concern is that the pre-filed value will make the validator pass even if the user fails to update it with their actual URL.

Let's leave it blank then. We can polish it later, if required.

natexcvi · 2023-01-29T16:25:47Z

@elasticmachine merge upstream

natexcvi · 2023-01-30T15:35:28Z

@elasticmachine merge upstream

pmuellr

LGTM, thanks for the param rendering change and test!

natexcvi · 2023-01-30T17:19:10Z

@elasticmachine merge upstream

natexcvi · 2023-01-30T17:43:18Z

LGTM, thanks for the param rendering change and test!

Thank you @pmuellr! What are the next steps for merging?

pmuellr · 2023-01-30T19:31:26Z

Thank you @pmuellr! What are the next steps for merging?

I believe @semd will be driving that.

semd · 2023-01-31T08:39:17Z

buildkite test this

semd · 2023-01-31T11:52:46Z

@elasticmachine, run elasticsearch-ci/docs

semd · 2023-01-31T13:53:36Z

docs/management/connectors/index.asciidoc

@@ -15,4 +15,6 @@ include::action-types/tines.asciidoc[leveloffset=+1]
 include::action-types/webhook.asciidoc[]
 include::action-types/cases-webhook.asciidoc[leveloffset=+1]
 include::action-types/xmatters.asciidoc[]
+include::action-types/torq.asciidoc[]
+include::action-types/tines.asciidoc[]


Hi @natexcvi the elasticsearch-ci/docs check is failing because the Tines docs are already included, we are including them twice, the duplicity may have been caused by some merge.
Can you remove this line 19? Thanks

@semd just removed it

semd · 2023-01-31T14:16:10Z

@elasticmachine, run elasticsearch-ci/docs

semd · 2023-01-31T14:18:39Z

buildkite test this

kibana-ci · 2023-01-31T15:25:05Z

💚 Build Succeeded

Buildkite Build
Commit: b081a6d

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`stackConnectors`	174	180	+6

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`stackConnectors`	415.5KB	419.3KB	+3.7KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`stackConnectors`	24.8KB	28.0KB	+3.2KB

Unknown metric groups

async chunk count

id	before	after	diff
`stackConnectors`	59	62	+3

ESLint disabled line counts

id	before	after	diff
`stackConnectors`	78	81	+3

Total ESLint disabled count

id	before	after	diff
`stackConnectors`	82	85	+3

History

💚 Build #104107 succeeded b6f10c7
💚 Build #104070 succeeded b6f10c7
💔 Build #103530 failed 29b3902
💔 Build #102950 failed 9293ad8
💛 Build #102694 was flaky 082f3cd
💔 Build #102354 failed a734b42

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

semd · 2023-01-31T15:27:26Z

Done! thanks for the contribution @natexcvi

## Summary Add a new action type for Torq which triggers Torq workflows. This is a re-do of elastic#139635 ... ### Checklist Delete any items that are not applicable to this PR. - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] Any UI touched in this PR is usable by keyboard only (learn more about [keyboard accessibility](https://webaim.org/techniques/keyboard/)) - [x] Any UI touched in this PR does not create any new axe failures (run axe in browser: [FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/), [Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US)) ### Risk Matrix Delete this section if it is not applicable to this PR. Before closing this PR, invite QA, stakeholders, and other developers to identify risks that should be tested prior to the change/feature release. When forming the risk matrix, consider some of the following examples and how they may potentially impact the change: | Risk | Probability | Severity | Mitigation/Notes | |---------------------------|-------------|----------|-------------------------| | Multiple Spaces—unexpected behavior in non-default Kibana Space. | Low | High | Integration tests will verify that all features are still supported in non-default Kibana Space and when user switches between spaces. | | Multiple nodes—Elasticsearch polling might have race conditions when multiple Kibana nodes are polling for the same tasks. | High | Low | Tasks are idempotent, so executing them multiple times will not result in logical error, but will degrade performance. To test for this case we add plenty of unit tests around this logic and document manual testing procedure. | | Code should gracefully handle cases when feature X or plugin Y are disabled. | Medium | High | Unit tests will verify that any feature flag or plugin combination still results in our service operational. | | [See more potential risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) | ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: orihoogi <ohoogi@gmail.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

stephmilovic · 2023-02-06T17:33:40Z

@semd @MadameSheema can y'all please meet to discuss a test plan for Torq?

## Summary Add a new action type for Torq which triggers Torq workflows. This is a re-do of elastic#139635 ... ### Checklist Delete any items that are not applicable to this PR. - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] Any UI touched in this PR is usable by keyboard only (learn more about [keyboard accessibility](https://webaim.org/techniques/keyboard/)) - [x] Any UI touched in this PR does not create any new axe failures (run axe in browser: [FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/), [Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US)) ### Risk Matrix Delete this section if it is not applicable to this PR. Before closing this PR, invite QA, stakeholders, and other developers to identify risks that should be tested prior to the change/feature release. When forming the risk matrix, consider some of the following examples and how they may potentially impact the change: | Risk | Probability | Severity | Mitigation/Notes | |---------------------------|-------------|----------|-------------------------| | Multiple Spaces—unexpected behavior in non-default Kibana Space. | Low | High | Integration tests will verify that all features are still supported in non-default Kibana Space and when user switches between spaces. | | Multiple nodes—Elasticsearch polling might have race conditions when multiple Kibana nodes are polling for the same tasks. | High | Low | Tasks are idempotent, so executing them multiple times will not result in logical error, but will degrade performance. To test for this case we add plenty of unit tests around this logic and document manual testing procedure. | | Code should gracefully handle cases when feature X or plugin Y are disabled. | Medium | High | Unit tests will verify that any feature flag or plugin combination still results in our service operational. | | [See more potential risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) | ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: orihoogi <ohoogi@gmail.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

natexcvi and others added 30 commits August 1, 2022 14:56

Initial implementation

dd24332

Further impl and tests

9a78eea

Add default body

5bcf482

Merge branch 'elastic:main' into add_torq_connector

aeb4e42

Change texts + fixes

fb6b0aa

Merge remote-tracking branch 'origin/add_torq_connector' into add_tor…

c8fbbbc

…q_connector

Merge remote-tracking branch 'origin/main' into add_torq_connector

687b5f6

Type fixes, translations and tests

4fdba89

More fixes

432d6da

Add default payload and docs

9bc112f

Fix action result debug log

e578e4d

Merge branch 'elastic:main' into add_torq_connector

624083e

Revert dependency change

7929e79

Merge remote-tracking branch 'origin/add_torq_connector' into add_tor…

30416b7

…q_connector

Fixes & refactoring

cf50afa

Update server tests

8251c58

Remove debug log

c1bfcaa

Remove TODO

c61054d

Remove unused code

04907a0

Remove TODO

c61ab80

Further CR fixes

4685ece

Fix test

7aead3d

Remove unneeded dep

3e4225b

Merge pull request #1 from natexcvi/add_torq_connector

bfccdd5

Add Torq Connector

Merge branch 'elastic:main' into main

67be0c6

Merge branch 'main' into main

2c139b0

Merge branch 'main' into main

a10240b

Merge branch 'main' into main

3751644

Merge branch 'main' into main

157b0de

Merge branch 'main' into main

c1a7966

pmuellr requested changes Jan 27, 2023

View reviewed changes

natexcvi added 2 commits January 29, 2023 18:20

Merge branch 'main' into main

2a54fa9

Fix templating and add tests

227d150

Merge branch 'main' into main

ff64c33

natexcvi requested a review from pmuellr January 29, 2023 17:06

Merge branch 'main' into main

48ce6f3

pmuellr approved these changes Jan 30, 2023

View reviewed changes

Merge branch 'main' into main

320f4ea

Merge branch 'main' into main

b6f10c7

semd reviewed Jan 31, 2023

View reviewed changes

natexcvi added 2 commits January 31, 2023 16:07

Fix docs index

e739de1

Merge branch 'main' into main

b081a6d

semd merged commit 92cb000 into elastic:main Jan 31, 2023

kibanamachine added the v8.7.0 label Jan 31, 2023

nastasha-solomon mentioned this pull request Mar 9, 2023

Torq connector as a new action connector elastic/security-docs#3050

Closed

lcawl mentioned this pull request Aug 31, 2023

[OAS] Add placeholders for Torq and D3 Security connectors #165283

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Torq Connector #149405

Add Torq Connector #149405

natexcvi commented Jan 24, 2023 •

edited by pmuellr

pmuellr left a comment

pmuellr Jan 27, 2023

natexcvi Jan 29, 2023

pmuellr Jan 27, 2023

natexcvi Jan 29, 2023

pmuellr Jan 30, 2023

natexcvi commented Jan 29, 2023

natexcvi commented Jan 30, 2023

pmuellr left a comment

natexcvi commented Jan 30, 2023

natexcvi commented Jan 30, 2023

pmuellr commented Jan 30, 2023

semd commented Jan 31, 2023

semd commented Jan 31, 2023

semd Jan 31, 2023

natexcvi Jan 31, 2023

semd commented Jan 31, 2023

semd commented Jan 31, 2023

kibana-ci commented Jan 31, 2023

async chunk count

ESLint disabled line counts

Total ESLint disabled count

semd commented Jan 31, 2023

stephmilovic commented Feb 6, 2023

Add Torq Connector #149405

Add Torq Connector #149405

Conversation

natexcvi commented Jan 24, 2023 • edited by pmuellr

Summary

Checklist

Risk Matrix

For maintainers

pmuellr left a comment

Choose a reason for hiding this comment

pmuellr Jan 27, 2023

Choose a reason for hiding this comment

natexcvi Jan 29, 2023

Choose a reason for hiding this comment

pmuellr Jan 27, 2023

Choose a reason for hiding this comment

natexcvi Jan 29, 2023

Choose a reason for hiding this comment

pmuellr Jan 30, 2023

Choose a reason for hiding this comment

natexcvi commented Jan 29, 2023

natexcvi commented Jan 30, 2023

pmuellr left a comment

Choose a reason for hiding this comment

natexcvi commented Jan 30, 2023

natexcvi commented Jan 30, 2023

pmuellr commented Jan 30, 2023

semd commented Jan 31, 2023

semd commented Jan 31, 2023

semd Jan 31, 2023

Choose a reason for hiding this comment

natexcvi Jan 31, 2023

Choose a reason for hiding this comment

semd commented Jan 31, 2023

semd commented Jan 31, 2023

kibana-ci commented Jan 31, 2023

💚 Build Succeeded

Metrics [docs]

Module Count

Async chunks

Page load bundle

async chunk count

ESLint disabled line counts

Total ESLint disabled count

History

semd commented Jan 31, 2023

stephmilovic commented Feb 6, 2023

natexcvi commented Jan 24, 2023 •

edited by pmuellr