Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Transforms: Adds docs_per_second to transform edit form. #65365

Merged
merged 1 commit into from
May 6, 2020

Conversation

walterra
Copy link
Contributor

@walterra walterra commented May 5, 2020

Summary

Fixes #62740.

Adds an option to edit the docs_per_second option for throttling for transforms.

transform-throttling

Checklist

Delete any items that are not applicable to this PR.

For maintainers

@elasticmachine
Copy link
Contributor

Pinging @elastic/ml-ui (:ml)

Copy link
Contributor

@alvarezmelissa87 alvarezmelissa87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code LGTM ⚡

@kibanamachine
Copy link
Contributor

💛 Build succeeded, but was flaky


Test Failures

Kibana Pipeline / kibana-xpack-agent / Firefox XPack UI Functional Tests.x-pack/test/functional/apps/infra/home_page·ts.InfraOps app Home page without metrics present renders an empty data prompt

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

[00:00:00]       │
[00:21:10]         └-: InfraOps app
[00:21:10]           └-> "before all" hook
[00:21:10]           └-: Home page
[00:21:10]             └-> "before all" hook
[00:21:10]             └-> "before all" hook
[00:21:10]               │ info [empty_kibana] Loading "mappings.json"
[00:21:11]               │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-ubuntu-18-tests-xl-1588710411338436204] [.kibana_2/GWOYbIGQTXOntyjcqUpldw] deleting index
[00:21:11]               │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-ubuntu-18-tests-xl-1588710411338436204] [.kibana_1/dAdVuRtPT6GmVKCHLP8Jlg] deleting index
[00:21:11]               │ info [empty_kibana] Deleted existing index [".kibana_2",".kibana_1"]
[00:21:11]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-18-tests-xl-1588710411338436204] applying create index request using v1 templates []
[00:21:11]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-18-tests-xl-1588710411338436204] [.kibana] creating index, cause [api], templates [], shards [1]/[1], mappings [_doc]
[00:21:11]               │ info [empty_kibana] Created index ".kibana"
[00:21:11]               │ debg [empty_kibana] ".kibana" settings {"index":{"number_of_replicas":"1","number_of_shards":"1"}}
[00:21:11]               │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-ubuntu-18-tests-xl-1588710411338436204] [.kibana/Bs61m-bDReue3ebm_4719Q] update_mapping [_doc]
[00:21:11]               │ debg Migrating saved objects
[00:21:11]               │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-ubuntu-18-tests-xl-1588710411338436204] [.kibana/Bs61m-bDReue3ebm_4719Q] update_mapping [_doc]
[00:21:12]               │ proc [kibana]   log   [21:26:34.073] [info][savedobjects-service] Creating index .kibana_2.
[00:21:12]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-18-tests-xl-1588710411338436204] applying create index request using v1 templates []
[00:21:12]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-18-tests-xl-1588710411338436204] [.kibana_2] creating index, cause [api], templates [], shards [1]/[1], mappings [_doc]
[00:21:12]               │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-ubuntu-18-tests-xl-1588710411338436204] updating number_of_replicas to [0] for indices [.kibana_2]
[00:21:12]               │ proc [kibana]   log   [21:26:34.201] [info][savedobjects-service] Reindexing .kibana to .kibana_1
[00:21:12]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-18-tests-xl-1588710411338436204] applying create index request using v1 templates []
[00:21:12]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-18-tests-xl-1588710411338436204] [.kibana_1] creating index, cause [api], templates [], shards [1]/[1], mappings [_doc]
[00:21:12]               │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-ubuntu-18-tests-xl-1588710411338436204] updating number_of_replicas to [0] for indices [.kibana_1]
[00:21:12]               │ info [o.e.t.LoggingTaskListener] [kibana-ci-immutable-ubuntu-18-tests-xl-1588710411338436204] 16279 finished with response BulkByScrollResponse[took=29.9ms,timed_out=false,sliceId=null,updated=0,created=1,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[00:21:12]               │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-ubuntu-18-tests-xl-1588710411338436204] [.kibana/Bs61m-bDReue3ebm_4719Q] deleting index
[00:21:12]               │ proc [kibana]   log   [21:26:34.617] [info][savedobjects-service] Migrating .kibana_1 saved objects to .kibana_2
[00:21:12]               │ proc [kibana]   log   [21:26:34.632] [info][savedobjects-service] Pointing alias .kibana to .kibana_2.
[00:21:12]               │ proc [kibana]   log   [21:26:34.693] [info][savedobjects-service] Finished in 621ms.
[00:21:12]               │ debg applying update to kibana config: {"accessibility:disableAnimations":true,"dateFormat:tz":"UTC"}
[00:21:13]               │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-ubuntu-18-tests-xl-1588710411338436204] [.kibana_2/W-hBbfFYRMy7FfcDfqkuuA] update_mapping [_doc]
[00:21:13]             └-: without metrics present
[00:21:13]               └-> "before all" hook
[00:21:13]               └-> "before all" hook
[00:21:13]                 │ info [infra/metrics_and_logs] Unloading indices from "mappings.json"
[00:21:13]                 │ info [infra/metrics_and_logs] Unloading indices from "data.json.gz"
[00:21:14]               └-> renders an empty data prompt
[00:21:14]                 └-> "before each" hook: global before each
[00:21:14]                 │ debg navigating to infraOps url: http://localhost:6111/app/metrics
[00:21:14]                 │ debg navigate to: http://localhost:6111/app/metrics
[00:21:14]                 │ debg browser[log] "^ A single error about an inline script not firing due to content security policy is expected!"
[00:21:15]                 │ debg ... sleep(700) start
[00:21:15]                 │ debg ... sleep(700) end
[00:21:15]                 │ debg returned from get, calling refresh
[00:21:27]                 │ debg browser[log] "^ A single error about an inline script not firing due to content security policy is expected!"
[00:21:28]                 │ debg currentUrl = http://localhost:6111/app/metrics
[00:21:28]                 │          appUrl = http://localhost:6111/app/metrics
[00:21:28]                 │ debg TestSubjects.find(kibanaChrome)
[00:21:28]                 │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=60000
[00:21:49]                 │ debg browser[info] "INFO: 2020-05-05T21:27:11Z
[00:21:49]                 │        Adding connection to http://localhost:6111/elasticsearch
[00:21:49]                 │
[00:21:49]                 │      "
[00:21:50]                 │ debg ... sleep(501) start
[00:21:50]                 │ debg ... sleep(501) end
[00:21:50]                 │ debg in navigateTo url = http://localhost:6111/app/metrics
[00:21:50]                 │ debg TestSubjects.exists(statusPageContainer)
[00:21:50]                 │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="statusPageContainer"]') with timeout=2500
[00:21:53]                 │ debg --- retry.tryForTime error: [data-test-subj="statusPageContainer"] is not displayed
[00:21:54]                 │ debg TestSubjects.find(noMetricsIndicesPrompt)
[00:21:54]                 │ debg Find.findByCssSelector('[data-test-subj="noMetricsIndicesPrompt"]') with timeout=10000
[00:22:07]                 │ info Taking screenshot "/dev/shm/workspace/kibana/x-pack/test/functional/screenshots/failure/InfraOps app Home page without metrics present renders an empty data prompt.png"
[00:22:07]                 │ info Current URL is: http://localhost:6111/app/metrics
[00:22:07]                 │ info Saving page source to: /dev/shm/workspace/kibana/x-pack/test/functional/failure_debug/html/InfraOps app Home page without metrics present renders an empty data prompt.html
[00:22:07]                 └- ✖ fail: "InfraOps app Home page without metrics present renders an empty data prompt"
[00:22:07]                 │

Stack Trace

{ TimeoutError: Waiting for element to be located By(css selector, [data-test-subj="noMetricsIndicesPrompt"])
Wait timed out after 12996ms
    at /dev/shm/workspace/kibana/node_modules/selenium-webdriver/lib/webdriver.js:842:17
    at process._tickCallback (internal/process/next_tick.js:68:7) name: 'TimeoutError', remoteStacktrace: '' }

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@walterra
Copy link
Contributor Author

walterra commented May 6, 2020

@elasticmachine merge upstream

@elasticmachine
Copy link
Contributor

user doesn't have permission to update head repository

@walterra
Copy link
Contributor Author

walterra commented May 6, 2020

jenkins retest

Copy link
Contributor

@peteharverson peteharverson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of comments for follow-ups, but otherwise LGTM.

'xpack.transform.transformList.editFlyoutFormDocsPerSecondHelptext',
{
defaultMessage:
'To enable throttling, set a limit of documents per second of input documents.',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One for a follow-up - is there any guidance we can give as to what is a sensible value here?

return {
...state,
errorMessages: state.isOptional && value.length === 0 ? [] : validate[state.validator](value),
errorMessages:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the update fails, a toast notification is correctly displayed. But when I dismiss the toast, the flyout closes, not allowing the user to correct any mistakes. The flyout should remain open after the toast error is dismissed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good find, will be solved in a follow-up.

@walterra walterra merged commit 8a65950 into elastic:master May 6, 2020
@walterra walterra deleted the ml-transforms-throttling branch May 6, 2020 09:57
walterra added a commit to walterra/kibana that referenced this pull request May 6, 2020
Adds an option to edit the docs_per_second option for throttling for transforms.
@walterra walterra added the v7.9.0 label May 6, 2020
gmmorris added a commit to gmmorris/kibana that referenced this pull request May 6, 2020
* master: (72 commits)
  add tsvb tests to Firefox suite (elastic#65425)
  Fix flaky ServerMetricsCollector integration test (elastic#65420)
  [APM] Custom links section inside the Actions menu is showing outside of the menu (elastic#65428)
  [ML] Adds docs_per_second to transform edit form. (elastic#65365)
  update apm index pattern (elastic#65424)
  add direct build command (elastic#65431)
  [ML] Adding daily_model_snapshot_retention_after_days to types and schemas (elastic#65417)
  [chore] Improve request cancelation handling in vis embeddable (elastic#65057)
  [Alerting] migrates acceptance and functional test fixtures to KP (elastic#64888)
  [ML] Fixes reordering in view by selection when overall cell selected (elastic#65290)
  Additional branding updates (elastic#64712)
  Remove redundant formatting of percentage column (elastic#64948)
  [SIEM][CASE] Configuration pages UI redesign (elastic#65355)
  New nav (elastic#64018)
  [Ingest pipelines] Address copy feedback (elastic#65175)
  bug fixing (elastic#65387)
  skip whole suite blocking snapshots (elastic#65377)
  add related event generation to ancestor nodes (fixes a bug) (elastic#64950)
  [Canvas] move files from legacy/plugins to plugins (elastic#65283)
  [SIEM] template timeline UI (elastic#64439)
  ...
walterra added a commit that referenced this pull request May 6, 2020
Adds an option to edit the docs_per_second option for throttling for transforms.
walterra added a commit that referenced this pull request May 6, 2020
Adds an option to edit the docs_per_second option for throttling for transforms.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[ML] Transforms: add throttling UI support
5 participants