Merged master into https://github.com/elastic/kibana/pull/14036 #1

spalger · 2017-09-20T17:16:26Z

No description provided.

* Remove redundant kbn-accessible-click * Make history in console accessible, fix elastic#13937

* adding tabify response handler * tabify should be the default response handler * asAggResultConfig should be configurable for BWC * update table to work with new tabify response handler * region map needs to define responseHandler: none * fixing tests * markdown should set responseHandler to none

* [DOCS] Add links to X-Pack release notes * [DOCS] Added breaking changes for X-Pack

* [babel] create babel-preset and babel-register modules * [babel-preset] add comments to babel-preset-env.target choices * [babel-preset] use more normal BUILT_WITH_BABEL signal * [babel-register] extended comment about filtering ./src in dist * [babel-preset] stage of class properties has changed, sorta

* react editor example * ensure props are not updated * use new stageEditorParams method to stage parameter changes * make component stateless * use terms_vis_editor component * get add button to work * update vis controller to display terms input controls * update componenent when query bar updates * add functional test * lay ground work for different control types in single visulization * make editors for range and text controls * text control * implement type ahead suggestor for text control * add range slider * some CSS work * add submit button, move control init functionallity under control_factory * add custom options for control types * provide buttons to move controls up and down * Make ControlEditor component and clean up styling of editor * styling work * multi select for terms dropdown control * add option to disable filter staging, only enable submit button when filters are staged * clean up range styling * rename top level vis folder * cleanup * move control type select out of each control editor * dark theme styling * use ui/public/filter_manager/lib/phrases.js to build phrases filter, add tests to range filter manager * use savedObjectsClient to get index patterns * remove text control and add id to controls for react tracking * ensure fields get updated when index pattern changes * update PropTypes for react 15.6.1 * update to latest react-select to avoid isMounted deprecation warnings * fix input controls functional test * rename termsControl to listControl to be more generic * add function test for clear button, refactor directory structure * functional tests for updateFiltersOnChange is true * fix react-select clipping problem in dashboard * try clicking option instead of pressing enter to set react-select value in functional tests * react-select css * clean up control_editor component, make ListControlEditor component be function * add jest test for vis_editor component and accessibility * add decimal places option to range slider * add jest test for InputControlVis component * add default to switch blocks, split editor into seperate tabs, use shallow in snapshot tests * fix race condition in field_select, update index_pattern_select to fetch indexPatterns on each filter * clean up control initialization * use htmlIdGenerator to avoid html element id conflicts * update functional test to support new editor tabs * finish jest tests for sub componenets * mark vis as experimental, refactor buttons for better usability * fix bug in list control where unable to select options containing numbers and options containing commas. Truncate display of long list options * fix chart types functional test * fix jest tests, add margin to action buttons * remove binds from render functions * experement with native input range sliders * Revert "experement with native input range sliders" This reverts commit aed599e. * Use Promise.resolve in tests and replace _createRequest with searchSource.fetch * add inputs to range control

…mponents-debug-info

* Prepare control flow to use embeddable factories in add panel * Rewrite saved object finder and add tests * Fix usages of new saved object finder * fix test failures * fix some functional tests and re-introduce makeUrl * fix tests * remove direct hrefs in saved_object_lists * PR review fixes * update snapshot * overwrite width of viz dialog * Update src/legacy/core_plugins/kibana/public/dashboard/top_nav/add_panel.js Co-Authored-By: flash1293 <email@johannes-reuter.de> * Update src/legacy/core_plugins/kibana/public/discover/embeddable/search_embeddable_factory.ts Co-Authored-By: flash1293 <email@johannes-reuter.de> * Update src/legacy/core_plugins/kibana/public/discover/top_nav/open_search_panel.js Co-Authored-By: flash1293 <email@johannes-reuter.de> * Update src/legacy/core_plugins/kibana/public/visualize/wizard/search_selection/search_selection.tsx Co-Authored-By: flash1293 <email@johannes-reuter.de> * Update src/legacy/core_plugins/kibana/public/visualize/wizard/search_selection/search_selection.tsx Co-Authored-By: flash1293 <email@johannes-reuter.de> * Update src/legacy/core_plugins/kibana/public/visualize/wizard/search_selection/search_selection.tsx Co-Authored-By: flash1293 <email@johannes-reuter.de> * fix tests * review fixes #1 * review fixes #2 * dont use classname in functional test * remove call to action button prop * align buttons correctly * fix tests * remove debugging statement * Update src/legacy/core_plugins/kibana/public/dashboard/top_nav/add_panel.js Co-Authored-By: flash1293 <email@johannes-reuter.de> * Update src/legacy/core_plugins/kibana/public/discover/top_nav/open_search_panel.js Co-Authored-By: flash1293 <email@johannes-reuter.de> * review fixes #3 * improve filter behavior and enable it for search wizard * adjust functional tests for new filter behavior * Change translation id due to string change * Update Jest snapshot

* Introduce new convention for config definition. We need to define a way to acquire configuration schema as a part of plugin definition. Having schema we can split steps of config validation and plugin instantiation. * Discover plugins, read their schema and validate the config. Config validation finished before core services and plugins read from it. That allows us to fail fast and have predictable validation results. * Instantiate plugins using DiscoveredPluginsDefinitions. * Update tests for new API. * test server is not created if config validation fails * move plugin discovery to plugin service pre-setup stage. Set validation schemes in ConfigService.preSetup stage. * fix eslint problem * generate docs * address Rudolfs comments * separate core services and plugins validation * rename files for consistency * address comments for root.js * address comments #1 * useSchema everywhere for consistency. get rid of validateAll * plugin system runs plugin config validation * rename configDefinition * move plugin schema registration in plugins plugins service plugins system is not setup when kibana is run in optimizer mode, so config keys aren't marked as applied. * cleanup * update docs * address comments

* unify modifyUrl on client and server * create BasePath as a separate entity on server * use BasePath class in http server * use BasePath a separate entity on client * use BasePath class on Http service on the client * switch client code to the new api * improve setver http service mocks * address comments #1 * address comments #2 * update docs * add comment why we define own typings

@eli

* New and Legacy platforms share http server instance. Required to use a common security interceptor for incoming http requests * generate docs * remove excessive contract method * add test for New platform compatibility * address comments part #1 * log server running only for http server * fix test. mutate hapi request headers for BWC with legacy * return 503 on start * address @eli comments * address @joshdover comments

@lukeelmers

* kbn top nav in discover * New top nav in dashboard and vis editor * change kbn top nav directive * Stop using template feature of kbn top nav * Combined css * console top nav * Removed unnecessary use of kbn-top-nav in dev tools app (wrapping tabs) * Changed console menu to new directive * CSS adjustments + functional implementation of top nav menu item * Fixed DevTools top nav toggle * Optional show-border * timelion menu works * Improve filter sorting logic to be more stable * Support showing filter bar as part of top nav * Use search bar in top nav in discover and maps Support search bar with no filter bar (TS) * tmp revert * Align search bar settings accross apps * Reverted change in ML, removed unused css * Added data test subj to top nav menu items * Watch config by value * Some directive magic to make sure that disabledButtons functions are watched correctly. * Fix missing controls from new top nav in some visualizations * showAutoRefreshOnly in input controls visualization * Fixed inspector disabled test * Fix dashboard action links rewrite top nav menu item to functional style * snapshots * Fixed maps filter bar * Remvoed comment * Update Query Bar defaults * Top nav menu item tests * Moved storage instantiation to angular directive * SearchBar jest tests * Query bar additional tests * Top nav menu tests * Pass store into top nav menu correctly * watch store by reference * Added not null assertion * Make index patterns optional (for timepicker only setup) * QueryBar tests again * Search bar tests * Top nav tests * Top nav menu tests * removed unnecessary ui/new_platform mock * Moved discover result count away from top nav * Moved and improved top nav menu css * remove unused translations * Move timepicker (to be deprecated) into old kbn_top_nav * Deleted search-bar and query-bar directives! * moved search bar to kibana_react (it's a generic react component, not a service) * translations * Moved superDatePicker directive to kbn_top_nav (to be deprecated) Deleted unused react_component directives call-out and tool-bar-search-box * Use index patterns from data plugin itself instead of importing from ui/public * Pass uiSettings from top level of data plugin * Pass saved objects client from top level * query bar input to rely on provided localStorage (?) * import QueryBarInput correctly from vis ediror * Query bar input tests - uiSettings * Query bar - uiSettings tests * import QueryBar from data plugin correctly * Use provider to pass config to TSVB query bar inputs * Add query bar input parms to agg types * Standardize shape data plugin and use it to pass uiSettings in. * Removed unnecessary mocks * Fixed test typo * Code review part 1 * Code review part 2 - clean up top nav menu item run interface * Give top nav items better keys * Simplified top nav construction logic in editor * Fixed mock of search bar in top nav test * Moved filter trigger button rendering to helper function * Remove responsiveness from top nav items * vertical align for cases where showSearchBarInline is true (i.e. only menu and time picker in the same row) * Export TopNavMenuData type * Removed unused name attribute in top nav. Use app-name instead. * Minor merge fixes * Update data plugin shape * type fixes * fix core start import * Fixed tests * fix another test mock * Brought back setup.ts for backward compatibility. Chnaged to relative imports * bad import from data plugin * Yet another import :-( * Code revire #1 @lukeelmers * create wrapper for QueryBarInput * use IndexPatterns type * import order * Code review fixes * inject uiSettings into filter-bar directive * Fix some type errors * Fixes * Filter bar context * ts CoreSetupContextProvider * Moved coreCOntext to vis_editor constructor * Reverted filter bar context * Removed editor html ng-show * use saved objects contract + check uiSettings

…#53345)

…lastic#54651)

* add authRequred: 'optional' * expose auth status via request context * update security plugin to use notHandled auth outcome * capabilities service uses optional auth * update tests * attach security headers only to unauthorised response * add isAuthenticated tests for 'optional' auth mode * security plugin relies on http.auth.isAuthenticated to calc capabilities * generate docs * reword test suit names * update tests * update test checking isAuth on optional auth path * address Oleg comments * add test for auth: try * fix * pass isAuthenticted as boolean via context * remove response header from notHandled * update docs * add redirected for auth interceptor * security plugin uses t.redirected to be compat with auth: optional * update docs * require location header in the interface * address comments #1 * declare isAuthenticated on KibanaRequest * remove auth.isAuthenticated from scope * update docs * remove unnecessary comment * do not fail on FakrRequest * small improvements

…lastic#65159) * Remove stray rules that should've been deleted * Update rule.ts and tests * Remove deleted prebuilt rules from cypress ES archive (#1)

…lastic#65160) * Remove stray rules that should've been deleted * Update rule.ts and tests * Remove deleted prebuilt rules from cypress ES archive (#1)

* Remove stray rules that should've been deleted * Update rule.ts and tests * Remove deleted prebuilt rules from cypress ES archive (#1)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* UI Polish for Session viewer frame + buttons * UI Update for Session Viewer + Toolbar * added Left Borders on Details panel + Fix Details panel button behaviour * Details Panel UI Polish + PR Comments * Find results interaction UI Polish, Def/Min/Max width for details panel, Timestamp bug fix * more conflict fixes * removed unused variable * fix for failed checks on buildkite #1 * Addressing PR comments * PR Comments + Search bar UI bug fix * pr comments Co-authored-by: Karl Godard <karl.godard@elastic.co>

* Updated EUI to version 67.1.2. Updated instaces of ButtonColor from EUI to EuiButtonColor. * Updated to EuiCard instances that utilize the betaBadgeProps object to return an empty string instead of undefined when the label is unavailable * Removed two instances of the deprecated internetExplorerOnly() mixin * Updated two instances of the ButtonColor import to EuiButtonColor as is was renamed in PR elastic#6150 * Updated snapshots in Jest Test Suite #1 to account for EuiButton and EuiCard Emotion conversions. Updated snapshots for EuiTooltip as it now contains the new EuiToolTipAnchor component that replaced the tooltip anchor styles * Updated snapshots in Jest Test Suite #2 to account forEuiButton, EuiDescriptionList, EuiButtonIcon, and EuiBadge Emotion conversions. * Updated snapshots in Jest Test Suite #3 to account for EuiDescriptionList, EuiButton, and EuiBadge Emotion conversions. Updated snapshots for EuiTooltip as if now contains the new EuiTooltipAnchor component that replaced the tooltop anchor styles * Updated snapshots in Jest Test Suite #4 to account for EuiButton Emotion conversion. * Updated snapshots in Jest Test Suite #5 to account for EuiButton Emotion conversion. * Updated snapshots in Jest Test Suite #8 to account for EuiButtonIcon and EuiButton Emotion conversions. Updated snapshots for EuiTooltip as it now contains the new EuiTooltipAnchor component that replaced the tooltip anchor styles. * Updated snapshots in Jest Test Suite #9 to account for EuiFlyout and EuiButton Emotion conversions. * Updated snapshots in Jest Test Suite #10 to account for EuiButton, EuiBadge, EuiButtonIcon, and EuiCard Emotion conversions. Updated snapshots for EuiToolTtip as it now contains the new EuiTooltipAnchor component that replaced the tooltip anchor styles * Updated instances of EuiButtonIconColor to use EuiButtonIconProps['color'] as it was removed in PR elastic#6150 * Updated tests that target EuiButton to simulate click events to target a generic button to prevent undefined click event errors * Updated snapshots in Jest Test Suite #1 to account for EuiButton and EuiCard Emotion conversions * Added the EuiFlyout mixins and variables to Lens Sass file as EuiFlyout has been converted to Emotion and the Sass styles are no longer available in EUI * Added the EuiCallOutTypes variable to Step Progress Sass file as EuiCallOut has been converted to Emotion and the Sass styles are no longer available in EUI * Updated snapshots in Jest Test Suite #2 to account for recent Emotion conversions. Updated snapshots in server_status.test.tsx to render EuiBadge before checking the snapshots to reduce the snapshot churn caused by Emotion. Updated tests that target EuiButton to simulate click events to target a generic button to prevent undefined click event errors * [CI] Auto-commit changed files from 'node scripts/precommit_hook.js --ref HEAD~1..HEAD --fix' * Added imports for the added flyout mixin. Removed references to EuiCallOut mixin as the component has been converted to Emotion and is no longer available for use. * Updated unit tests and snapshots in Jest Test Suite #10. Updated snaphshots to account for EuiBadge, EuiDescriptionList, EuiFlyout, and EuiCard Emotion conversions. Updated snapshots for EuiTooltip as it now contains the new EuiTooltipAnchor component that replaced the tooltip anchor styles. Updated tests that target EuiButton to simulate click events to target a generic button element to prevent undefined click event errors * Updated unit tests in Jest Test Suite #11 that target EuiButton to simulate click events to target a generic button to prevent undefined click event errors * Updated unit tests in Jest Test Suite #12 by updating tests that target EuiButton to simulate click events. Instead, these tests now target a generic button element to prevent undefined click event errors * Updated unit tests in Jest Test Suite #1 by updating tests that target EuiButton to simulate click events. Instead, these tests now target a generic button element to prevent undefined click event errors * Updated unit tests in Jest Test Suite #2 by updating tests that use EuiButton to simulate click events. Instead, these test have been updated to target a button element to prevent undefined click event errors. * [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix' * Updated reference to mixins Sass file. Updated snapshots for Jest Test Suite #5 to account for EuiButton Emotion conversion. Updated unit tests that target EuiButton to simulate click events. These tests have been updated to target a button element to prevent undefined click event errors * Updated unit tests in Jest Test Suites 3, 7, 8, 13, and 14. Updated snapshot to account for EuiButton Emotion conversion. Updated tests that target EuiButton to simulate click events. These tests now target a generic button element to prevent undefined click event errors. Updated a few snapshots by adding .render() before checking the snapshot. This will prevent large snapshots coming from recent Emotion conversions * Updated snapshots in Jest Test Suite #10 to account for the recent EuiButton Emotion conversion * Updated unit tests in Jest Test Suite #2 by editing tests that target EuiButton to simulate click events. These tests now target a button element in order to prevent undefinde click event errors * Updated snapshots in Jest Test Suite #10 to account for EuiButton and EuiDescriptionList Emotion conversions * Updated test cases in Jest Test Suites 3, 7, and 8. Updated snapshots to account for EuiButton and EuiPagination Emotion conversions. Updated tests that target EuiButton to simulate click events. These tests now target a button element to prevent undefined click errors * Updated test cases in Jest Test Suite 14. Updated snapshots to account for EuiButton Emotion conversion. Opted to use .render() when updating a few snapshots to reduce the large length of snapshots caused by Emotion * [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix' * Revised a change to betaBadgeProps to ensure that the label is available. If not, the value for the badge with be set to undefined. * Resolved two linting errors * Resolved two linting errors * Updated Jest unit tests in various suites. Updated snapshots to account for EuiButton Emotion conversion. Updated snapshots for EuiTooltip as it now contains the new EuiTooltipAnchor component that replaced the tooltip anchor styles. * Updated EuiFlyout in query_flyout.tsx to remove the onClick function from maskProps as it is no longer available. Updated this flyout to use ownFocus and not to close when the overlay mask is clicked. * Removed the use of EuiButtonIconColor in favor of EuiButtonIconProps['color'] * [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix' * Updated Cypress test looking for strict equality on EuiPaginationButton class names to match a substring of the Emotion generated class name * Removed unneeded debugging code. Updated snapshots for various test suites to account for the recent EuiButton Emotion conversion * Updated a few EuiButton, EuiButtonEmpty, and EuiText components that set the color as ghost. The ghost color mode has been deprecated as of PR elastic#6150. These components now are wrapped in EuiThemeProvider with a dark colorMode to create the previous ghost color. * Resolved TS error with EuiCard betaBadgeProps * [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix' * Remove references to now-removed EuiFlyout CSS classes/vars * Remove now-removed euiBadge className references - Convert directly to EuiBadge instead of using CSS - Remove confusing and now-possibly-irrelevant CSS badge overrides - left/right icons are now set via JSX and not via flex-direction * Pre-emptively fix various euiOverlayMask CSS overrides - this data attr isn't technically in yet but will be once elastic/eui#6289 merges - at the very least this isn't breaking any more than it currently already is! * Update to v67.1.3 * v67.1.4 * Resolved test failing test case in Security/Manage/Blocklist. The test did not remove focus from the last combo box in the form, which didn't allow the disbaled attribute to be removed from the flyout submit button. I've updated the mock file for Blocklist to return focus to the first form element in the flyout to allow the disabled attribute to be removed. * Updated snapshots to account for the recent EuiText Emotion conversion * Fix Log's custom tooltips relying on EuiTooltip classNames that no longer exist * Fix Vega vis custom tooltips relying on EuiTooltip classNames that no longer exist - this one is trickier than Log's as it's not using React, so we need to use Emotion's Global to set a static className * Convert remaining vega_vis.scss to Emotion - as an example of how other global + non global styles could be handled in the future * Fix references to removed `euiPaginationButton-isActive` className - use aria-current attribute instead * Added missing EuiFlyoutAnimation keyframes for EuiFlyout. This resolved test that failed because they used onAnimationEnd because the FlyoutAnimation could not be found. * Reolved Jest Tests in suites 1 and 5. Updated snapshots to account for the recent EuiButton Emotion conversion. Updated snapshots for EuiToolTip as it now contains the new EuiToolTipAnchor component that replaced the tooltip anchor styles. * iterate on rules_list.test.tsx * bump eui to v67.1.5 * Updatde snapshots for jest test suites to account for the recent EuiButton, EuiOverlayMask, EuiTooltip, and EuiBadge Emotion conversions * Resolved failing security test by updating the target element for CONNECTOR_TITLE. EuiCard has recently been converted to Emotion and the card title is no longer wrapper in a span. * Resolved failing test case in Runtime Fields. The modify runtime field test was failing because the combobox responsbible for adding and updating scripts was not appearing. The textbox did not appear because the shared setFieldScript function targets and toggles the script textbox when opening the flyout. When a runtime field is being modified, the toggle is already active and using the shared function will trigger the toggle again (losing access to the script textbox). Also resolved an issue that prevented the warning EuiCallout to appear when changing the type of a runtime field from its original type. Resolved this by adding an enter keypress at the end of setFieldType function to confirm the type selection, thus triggering the EuiCallout * Resolved two tests that were failing in Lens. These test were failing because they were checking for equality in class names that no longer exist within EuiButtonGroup as it was recently converted to Emotion. These tests were updated to check for a substring of the new and longer class name * Quick fix in test case failing because of misspelling in data-test-sub * Updated snapshot for Jest test case as EuiButton as recently been converted to Emotion * Removed console.log statement. Oops! * Resolved a failing test case in Lens. They were failing because they were checking for equality in class names that no longer exist within EuiButtonGroup as it was recently converted to Emotion. These tests were updated to check for a substring of the new and longer class name. Updated a Security test case by giving a target button the data-test-subj attribute for easier querying * Removed reference to EuiFlyout mixin as it has been converted to Emotion. Updated the reference to an interal copy of EuiFlyout styles * Corrected spelling error in EuiFlyout animation in Lens app * Update EUI with latest backport * Update button snapshots * fix another button snapshot * More snapshot fixes * [EuiButton][Security] Fix button relying on now-removed `euiButton__text` CSS - replace removed CSS with `eui-textTruncate` util instead - combine/DRY out unnecessary span - was affecting min-width of truncation util + increase screenshot diff limit - this was smaller than updating the actual baseline screenshots for whatever reason (likely render diff between local and CI) * Fix remaining Jest tests affected by Emotion conversions - because Emotion creates its own wrapper, `.first()` can no longer be used - prefer `.last()` instead * Fix Jest test affected by EuiButton Emotion conversion + removed modifier class - targeting the native DOM node + filtering by disabled true/false gets us back to the 'correct' lengths * Fix + improve flyout test - `.last()` changes to account for EuiButton Emotion conversion is needed, but the last onClose assertion still fails due to us having modified inputs, and the confirm modal being displayed - split test into two separate tests - one testing the onClose call, and the other testing the confirm modal * derpin * Skip rules_list Jest suite * Update new EuiButton snapshot * Upgraded EUI version to 67.1.7 * [EuiCard] Update snapshots * [EuiPopover] Update snapshots * [QA] Fix missing Vega warn/error message colors ;_; * [CI] Auto-commit changed files from 'node scripts/generate codeowners' * Fix Lens kbnToolbarButton regressions - Caused by flattening of EUI button CSS specificity - background-color was previously relying on isDisabled CSS specificity to override its #fff color - `text` color modifier & `!important` is no longer needed and overrides Emotion CSS flatly - isDisabled class is no longer needed - euiButton no longer sets `pointer-events: none` on disabled buttons (fixes tooltip bug in webkit as well) * Backport EUI 67.1.8 fixes * Update EuiCard snapshots * Fix EuiModal form wrapper causing overflow issues - see https://elastic.github.io/eui/#/layout/modal#forms-in-a-modal * Workaround for `.kbnOverlayMountWrapper` mount point causing overflow issues - not sure what all is using this modal service to be honest, but the wrapper is causing issues with the modal layout, this fixes overflow issues but will not fix any mask-image issues as a result * more snapshot updates * EuiButton - added textProps to EuiButton to prevent very long button names from spilling over outside of the container * EuiButton - Update EuiButton related snapshots. Updated tests that target EuiButton directly to use a data-telementary-id for more specific element querying required by Emotion * QA - Removed unnecessary comment in code * Temporary fix for EuiCard[selectable][layout=horizontal] instances on security solutions' rule page * Temporary fix for EuiCard[selectable][layout=horizontal] instances on osquery live query and canvas's datasource selector * [CI] Auto-commit changed files from 'node scripts/precommit_hook.js --ref HEAD~1..HEAD --fix' * Fix CSS specificity, where canvas's solutionToolbarButton's background-color now takes precedence over EuiButton's primary styles * Removed update to search_marker_tooltip that removed the euiTooltip styles and replaced then with Emotion styling. Added EuiTooltip Sass styles for the component to rely on to test for a styling bug that is causing the tooltip and the tooltip arrow to be out of sync with each other. * Lint Sass file * Lint Sass file * Removed overflow:hidden style from .vgaVis_view as it was causing euiScrollStyles not to present the scroll bars in Vega Vis * Remove typo from EuiButton textProps object. 'className' should not have been included in the actual class name * Revert tooltip Sass This reverts commit 20e6ead, a5cd2de, and c605cbd * Fix Emotion tooltip arrows Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Constance Chen <constance.chen@elastic.co> Co-authored-by: Chandler Prall <chandler.prall@elastic.co>

## Summary Fixes elastic#144161 As discussed [here](elastic#144161 (comment)), the existing implementation of update tags doesn't work well with real agents, as there are many conflicts with checkin, even when trying to add/remove one tag. Refactored the logic to make retries more efficient: - Instead of aborting the whole bulk action on conflicts, changed the conflict strategy to 'proceed'. This means, if an action of 50k agents has 1k conflicts, not all 50k is retried, but only the 1k conflicts, this makes it less likely to conflict on retry. - Because of this, on retry we have to know which agents don't yet have the tag added/removed. For this, added an additional filter to the `updateByQuery` request. Only adding the filter if there is exactly one `tagsToAdd` or one `tagsToRemove`. This is the main use case from the UI, and handling other cases would complicate the logic more (each additional tag to add/remove would result in another OR query, which would match more agents, making conflicts more likely). - Added this additional query on the initial request as well (not only retries) to save on unnecessary work e.g. if the user tries to add a tag on 50k agents, but 48k already have it, it is enough to update the remaining 2k agents. - This improvement has the effect that 'Agent activity' shows the real updated agent count, not the total selected. I think this is not really a problem for update tags. - Cleaned up some of the UI logic, because the conflicts are fully handled now on the backend. - Locally I couldn't reproduce the conflict with agent checkins, even with 1k horde agents. I'll try to test in cloud with more real agents. To verify: - Enroll 50k agents (I used 50k with create_agents script, and 1k with horde). Enroll 50k with horde if possible. - Select all on UI and try to add/remove one or more tags - Expect the changes to propagate quickly (up to 1m). It might take a few refreshes to see the result on agent list and tags list, because the UI polls the agents every 30s. It is expected that the tags list temporarily shows incorrect data because the action is async. E.g. removed `test3` tag and added `add` tag quickly: <img width="1776" alt="image" src="https://user-images.githubusercontent.com/90178898/207824481-411f0f70-d7e8-42a6-b73f-ed80e77b7700.png"> <img width="422" alt="image" src="https://user-images.githubusercontent.com/90178898/207824550-582d43fc-87db-45e1-ba58-15915447fefd.png"> The logs show the details of how many `version_conflicts` were there, and it decreased with retries. ``` [2022-12-15T10:32:12.937+01:00][INFO ][plugins.fleet] Running action asynchronously, actionId: 90acd54-19ac-4738-b3d3-db32789233de, total agents: 52000 [2022-12-15T10:32:12.981+01:00][INFO ][plugins.fleet] Scheduling task fleet:update_agent_tags:retry:check:90acd541-19ac-4738-b3d3-db32789233de [2022-12-15T10:32:16.477+01:00][INFO ][plugins.fleet] Running action asynchronously, actionId: 29e9da70-7194-4e52-8004-2c1b19f6dfd5, total agents: 52000 [2022-12-15T10:32:16.537+01:00][INFO ][plugins.fleet] Scheduling task fleet:update_agent_tags:retry:check:29e9da70-7194-4e52-8004-2c1b19f6dfd5 [2022-12-15T10:32:22.893+01:00][DEBUG][plugins.fleet] {"took":9886,"timed_out":false,"total":52000,"updated":41143,"deleted":0,"batches":52,"version_conflicts":10857,"noops":0,"retries":{"bulk":0,"search":0},"throttled_millis":0,"requests_per_second":-1,"throttled_until_millis":0,"failures":[]} [2022-12-15T10:32:26.066+01:00][DEBUG][plugins.fleet] {"took":9518,"timed_out":false,"total":52000,"updated":25755,"deleted":0,"batches":52,"version_conflicts":26245,"noops":0,"retries":{"bulk":0,"search":0},"throttled_millis":0,"requests_per_second":-1,"throttled_until_millis":0,"failures":[]} [2022-12-15T10:32:27.401+01:00][ERROR][plugins.fleet] Action failed: version conflict of 10857 agents [2022-12-15T10:32:27.461+01:00][INFO ][plugins.fleet] Scheduling task fleet:update_agent_tags:retry:90acd541-19ac-4738-b3d3-db32789233de [2022-12-15T10:32:27.462+01:00][INFO ][plugins.fleet] Retrying in task: fleet:update_agent_tags:retry:90acd541-19ac-4738-b3d3-db32789233de [2022-12-15T10:32:29.274+01:00][ERROR][plugins.fleet] Action failed: version conflict of 26245 agents [2022-12-15T10:32:29.353+01:00][INFO ][plugins.fleet] Scheduling task fleet:update_agent_tags:retry:29e9da70-7194-4e52-8004-2c1b19f6dfd5 [2022-12-15T10:32:29.353+01:00][INFO ][plugins.fleet] Retrying in task: fleet:update_agent_tags:retry:29e9da70-7194-4e52-8004-2c1b19f6dfd5 [2022-12-15T10:32:31.480+01:00][INFO ][plugins.fleet] Running bulk action retry task [2022-12-15T10:32:31.481+01:00][DEBUG][plugins.fleet] Retry #1 of task fleet:update_agent_tags:retry:90acd541-19ac-4738-b3d3-db32789233de [2022-12-15T10:32:31.481+01:00][INFO ][plugins.fleet] Running action asynchronously, actionId: 90acd54-19ac-4738-b3d3-db32789233de, total agents: 52000 [2022-12-15T10:32:31.481+01:00][INFO ][plugins.fleet] Completed bulk action retry task [2022-12-15T10:32:31.485+01:00][INFO ][plugins.fleet] Scheduling task fleet:update_agent_tags:retry:check:90acd541-19ac-4738-b3d3-db32789233de [2022-12-15T10:32:33.841+01:00][DEBUG][plugins.fleet] {"took":2347,"timed_out":false,"total":10857,"updated":9857,"deleted":0,"batches":11,"version_conflicts":1000,"noops":0,"retries":{"bulk":0,"search":0},"throttled_millis":0,"requests_per_second":-1,"throttled_until_millis":0,"failures":[]} [2022-12-15T10:32:34.556+01:00][INFO ][plugins.fleet] Running bulk action retry task [2022-12-15T10:32:34.557+01:00][DEBUG][plugins.fleet] Retry #1 of task fleet:update_agent_tags:retry:29e9da70-7194-4e52-8004-2c1b19f6dfd5 [2022-12-15T10:32:34.557+01:00][INFO ][plugins.fleet] Running action asynchronously, actionId: 29e9da70-7194-4e52-8004-2c1b19f6dfd5, total agents: 52000 [2022-12-15T10:32:34.557+01:00][INFO ][plugins.fleet] Completed bulk action retry task [2022-12-15T10:32:34.560+01:00][INFO ][plugins.fleet] Scheduling task fleet:update_agent_tags:retry:check:29e9da70-7194-4e52-8004-2c1b19f6dfd5 [2022-12-15T10:32:35.388+01:00][ERROR][plugins.fleet] Retry #1 of task fleet:update_agent_tags:retry:90acd541-19ac-4738-b3d3-db32789233de failed: version conflict of 1000 agents [2022-12-15T10:32:35.468+01:00][INFO ][plugins.fleet] Scheduling task fleet:update_agent_tags:retry:90acd541-19ac-4738-b3d3-db32789233de [2022-12-15T10:32:35.468+01:00][INFO ][plugins.fleet] Retrying in task: fleet:update_agent_tags:retry:90acd541-19ac-4738-b3d3-db32789233de {"took":5509,"timed_out":false,"total":26245,"updated":26245,"deleted":0,"batches":27,"version_conflicts":0,"noops":0,"retries":{"bulk":0,"search":0},"throttled_millis":0,"requests_per_second":-1,"throttled_until_millis":0,"failures":[]} [2022-12-15T10:32:42.722+01:00][INFO ][plugins.fleet] processed 26245 agents, took 5509ms [2022-12-15T10:32:42.723+01:00][INFO ][plugins.fleet] Removing task fleet:update_agent_tags:retry:check:29e9da70-7194-4e52-8004-2c1b19f6dfd5 [2022-12-15T10:32:46.705+01:00][INFO ][plugins.fleet] Running bulk action retry task [2022-12-15T10:32:46.706+01:00][DEBUG][plugins.fleet] Retry #2 of task fleet:update_agent_tags:retry:90acd541-19ac-4738-b3d3-db32789233de [2022-12-15T10:32:46.707+01:00][INFO ][plugins.fleet] Running action asynchronously, actionId: 90acd54-19ac-4738-b3d3-db32789233de, total agents: 52000 [2022-12-15T10:32:46.707+01:00][INFO ][plugins.fleet] Completed bulk action retry task [2022-12-15T10:32:46.711+01:00][INFO ][plugins.fleet] Scheduling task fleet:update_agent_tags:retry:check:90acd541-19ac-4738-b3d3-db32789233de [2022-12-15T10:32:47.099+01:00][DEBUG][plugins.fleet] {"took":379,"timed_out":false,"total":1000,"updated":1000,"deleted":0,"batches":1,"version_conflicts":0,"noops":0,"retries":{"bulk":0,"search":0},"throttled_millis":0,"requests_per_second":-1,"throttled_until_millis":0,"failures":[]} [2022-12-15T10:32:47.623+01:00][INFO ][plugins.fleet] processed 1000 agents, took 379ms [2022-12-15T10:32:47.623+01:00][INFO ][plugins.fleet] Removing task fleet:update_agent_tags:retry:check:90acd541-19ac-4738-b3d3-db32789233de ``` ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

## Summary ### This PR enables user roles testing in FTR We use SAML authentication to get session cookie for user with the specific role. The cookie is cached on FTR service side so we only make SAML auth one time per user within FTR config run. For Kibana CI service relies on changes coming in elastic#170852 In order to run FTR tests locally against existing MKI project: - add `.ftr/role_users.json` in Kibana root dir ``` { "viewer": { "email": "...", "password": "..." }, "developer": { "email": "...", "password": "..." } } ``` - set Cloud hostname (!not project hostname!) with TEST_CLOUD_HOST_NAME, e.g. `export TEST_CLOUD_HOST_NAME=console.qa.cld.elstc.co` ### How to use: - functional tests: ``` const svlCommonPage = getPageObject('svlCommonPage'); before(async () => { // login with Viewer role await svlCommonPage.loginWithRole('viewer'); // you are logged in in browser and on project home page, start the test }); it('has project header', async () => { await svlCommonPage.assertProjectHeaderExists(); }); ``` - API integration tests: ``` const svlUserManager = getService('svlUserManager'); const supertestWithoutAuth = getService('supertestWithoutAuth'); let credentials: { Cookie: string }; before(async () => { // get auth header for Viewer role credentials = await svlUserManager.getApiCredentialsForRole('viewer'); }); it('returns full status payload for authenticated request', async () => { const { body } = await supertestWithoutAuth .get('/api/status') .set(credentials) .set('kbn-xsrf', 'kibana'); expect(body.name).to.be.a('string'); expect(body.uuid).to.be.a('string'); expect(body.version.number).to.be.a('string'); }); ``` Flaky-test-runner: #1 https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4081 #2 https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4114 --------- Co-authored-by: Robert Oskamp <traeluki@gmail.com> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>

## Summary Set `security.session.cleanupInterval` to 5h for session concurrency test. ### **Prerequisites** - Task for session cleanup with [default schedule set to 1h](https://github.com/elastic/kibana/blob/main/x-pack/plugins/security/server/config.ts#L222). - Task polling interval is set to [3000ms](https://github.com/elastic/kibana/blob/main/x-pack/plugins/task_manager/server/config.ts#L13). - We override `scheduledAt` once we make a request in [runCleanupTaskSoon](https://github.com/elastic/kibana/blob/main/x-pack/test/security_api_integration/tests/session_concurrent_limit/cleanup.ts#L145). ### **Hypothesis** Taking into consideration that: - `session_cleanup` task is not the only one scheduled during test run. - There is sort of an exponential backoff implemented for task polling if there are too many retries. - Clock jitter. I had a hypothesis that if our whole test run exceeds 1h or polling interval gets adjusted because of retries we might end up executing the scheduled cleanup before we trigger `runCleanupTaskSoon` (this is there we drop 1 session already). ### **FTR runs (x55 each)** - `cleanupInterval` set to 5h: [#1](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5986) :green_circle:, [#2](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5987) :green_circle: - `cleanupInterval` set to default 1h: [#1](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5983) :green_circle:, [#2](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5982) :red_circle: (has 2 failures out of 55) ### Checklist - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) __Fixes: https://github.com/elastic/kibana/issues/149091__

thomasneirynck and others added 16 commits September 18, 2017 12:42

Allow panning beyond dateline (elastic#13955)

e8b43b5

Update known-plugins.asciidoc (elastic#13701)

93a565c

[aggTypes/paramTypes] naming cleanup (elastic#14020)

199e5c3

Dev Tools console is now keyboard accessible (elastic#13946)

821bd96

* Remove redundant kbn-accessible-click * Make history in console accessible, fix elastic#13937

Bump chromedriver (elastic#14053)

d49692f

Allow custom timeouts on click (elastic#14042)

c3c55de

docs: add dendogram to known-plugins

bdcafa0

adding additional vis unit tests (elastic#14031)

9346b49

Add links to X-Pack release notes (elastic#13630)

26bccb0

* [DOCS] Add links to X-Pack release notes * [DOCS] Added breaking changes for X-Pack

[DOCS] Added index-shared6.asciidoc

5879c7f

[DOCS] Added index-shared6 Remove index-shared11

612385e

[ftr/config] prevent mochaOpts and timeouts conflict (elastic#14060)

1f5835c

Merge branch 'master' of github.com:elastic/kibana into add-styled-co…

94235d9

…mponents-debug-info

sorenlouv merged commit e6cf466 into sorenlouv:add-styled-components-debug-info Sep 20, 2017

spalger deleted the add-styled-components-debug-info branch October 18, 2019 17:47

elasticmachine pushed a commit that referenced this pull request Dec 18, 2019

[SIEM] Fix eslint errors from jsx-no-bind #1 (elastic#52856)

022e6c4

sorenlouv pushed a commit that referenced this pull request Dec 23, 2019

[SIEM] Fix eslint errors from jsx-no-bind #1 (elastic#52856) (elastic…

f0ae9f4

…#53345)

sorenlouv pushed a commit that referenced this pull request Feb 6, 2020

[SIEM] Detection Engine Create Rule Design Review #1 (elastic#54442) (e…

93093a1

…lastic#54651)

sorenlouv pushed a commit that referenced this pull request May 13, 2020

[SIEM] Remove forgotten rules that weren't deleted (elastic#64974)

9db27db

* Remove stray rules that should've been deleted * Update rule.ts and tests * Remove deleted prebuilt rules from cypress ES archive (#1)

sorenlouv pushed a commit that referenced this pull request Sep 21, 2020

Round #1 of removing frozen phase UI and server code (elastic#77877)

01ef387

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Merged master into https://github.com/elastic/kibana/pull/14036 #1

Merged master into https://github.com/elastic/kibana/pull/14036 #1

spalger commented Sep 20, 2017

Merged master into https://github.com/elastic/kibana/pull/14036 #1

Merged master into https://github.com/elastic/kibana/pull/14036 #1

Conversation

spalger commented Sep 20, 2017