Skip to content

Commit

Permalink
Merge branch 'main' into siem-explore-issue-124499-fix
Browse files Browse the repository at this point in the history
  • Loading branch information
kibanamachine committed Mar 3, 2022
2 parents 1e1ee54 + 8e7a792 commit dee55f3
Show file tree
Hide file tree
Showing 320 changed files with 6,612 additions and 2,396 deletions.
2 changes: 2 additions & 0 deletions .buildkite/pipelines/bazel_cache.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
steps:
- label: ':pipeline: Create pipeline with priority'
agents:
queue: kibana-default
concurrency_group: bazel_macos
concurrency: 1
concurrency_method: eager
Expand Down
2 changes: 2 additions & 0 deletions .buildkite/pipelines/es_snapshots/promote.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,5 @@ steps:
required: true
- label: Promote Snapshot
command: .buildkite/scripts/steps/es_snapshots/promote.sh
agents:
queue: kibana-default
6 changes: 6 additions & 0 deletions .buildkite/pipelines/es_snapshots/verify.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ steps:
- command: .buildkite/scripts/lifecycle/pre_build.sh
label: Pre-Build
timeout_in_minutes: 10
agents:
queue: kibana-default

- wait

Expand Down Expand Up @@ -85,6 +87,8 @@ steps:
- command: .buildkite/scripts/steps/es_snapshots/trigger_promote.sh
label: Trigger promotion
timeout_in_minutes: 10
agents:
queue: kibana-default
depends_on:
- default-cigroup
- default-cigroup-docker
Expand All @@ -98,3 +102,5 @@ steps:
- command: .buildkite/scripts/lifecycle/post_build.sh
label: Post-Build
timeout_in_minutes: 10
agents:
queue: kibana-default
3 changes: 3 additions & 0 deletions .buildkite/pipelines/flaky_tests/pipeline.js
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,9 @@ const pipeline = {
{
command: '.buildkite/pipelines/flaky_tests/runner.sh',
label: 'Create pipeline',
agents: {
queue: 'kibana-default',
},
},
],
};
Expand Down
4 changes: 4 additions & 0 deletions .buildkite/pipelines/hourly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ steps:
- command: .buildkite/scripts/lifecycle/pre_build.sh
label: Pre-Build
timeout_in_minutes: 10
agents:
queue: kibana-default

- wait

Expand Down Expand Up @@ -174,3 +176,5 @@ steps:
- command: .buildkite/scripts/lifecycle/post_build.sh
label: Post-Build
timeout_in_minutes: 10
agents:
queue: kibana-default
4 changes: 4 additions & 0 deletions .buildkite/pipelines/on_merge.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ steps:
- command: .buildkite/scripts/lifecycle/pre_build.sh
label: Pre-Build
timeout_in_minutes: 10
agents:
queue: kibana-default

- wait

Expand Down Expand Up @@ -34,3 +36,5 @@ steps:
- command: .buildkite/scripts/lifecycle/post_build.sh
label: Post-Build
timeout_in_minutes: 10
agents:
queue: kibana-default
23 changes: 13 additions & 10 deletions .buildkite/pipelines/performance/daily.yml
Original file line number Diff line number Diff line change
@@ -1,25 +1,27 @@
steps:
- block: ":gear: Performance Tests Configuration"
prompt: "Fill out the details for performance test"
- block: ':gear: Performance Tests Configuration'
prompt: 'Fill out the details for performance test'
fields:
- text: ":arrows_counterclockwise: Iterations"
key: "performance-test-iteration-count"
hint: "How many times you want to run tests? "
- text: ':arrows_counterclockwise: Iterations'
key: 'performance-test-iteration-count'
hint: 'How many times you want to run tests? '
required: true
if: build.env('PERF_TEST_COUNT') == null

- label: ":male-mechanic::skin-tone-2: Pre-Build"
- label: ':male-mechanic::skin-tone-2: Pre-Build'
command: .buildkite/scripts/lifecycle/pre_build.sh
agents:
queue: kibana-default

- wait

- label: ":factory_worker: Build Kibana Distribution and Plugins"
- label: ':factory_worker: Build Kibana Distribution and Plugins'
command: .buildkite/scripts/steps/build_kibana.sh
agents:
queue: c2-16
key: build

- label: ":muscle: Performance Tests with Playwright config"
- label: ':muscle: Performance Tests with Playwright config'
command: .buildkite/scripts/steps/functional/performance_playwright.sh
agents:
queue: c2-16
Expand All @@ -28,6 +30,7 @@ steps:
- wait: ~
continue_on_failure: true

- label: ":male_superhero::skin-tone-2: Post-Build"
- label: ':male_superhero::skin-tone-2: Post-Build'
command: .buildkite/scripts/lifecycle/post_build.sh

agents:
queue: kibana-default
17 changes: 0 additions & 17 deletions .buildkite/pipelines/pull_request.yml

This file was deleted.

2 changes: 2 additions & 0 deletions .buildkite/pipelines/pull_request/base.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ steps:
- command: .buildkite/scripts/lifecycle/pre_build.sh
label: Pre-Build
timeout_in_minutes: 10
agents:
queue: kibana-default

- wait

Expand Down
2 changes: 2 additions & 0 deletions .buildkite/pipelines/pull_request/post_build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,5 @@ steps:

- command: .buildkite/scripts/lifecycle/post_build.sh
label: Post-Build
agents:
queue: kibana-default
2 changes: 2 additions & 0 deletions .buildkite/pipelines/purge_cloud_deployments.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,5 @@ steps:
- command: .buildkite/scripts/steps/cloud/purge.sh
label: Purge old cloud deployments
timeout_in_minutes: 10
agents:
queue: kibana-default
2 changes: 2 additions & 0 deletions .buildkite/pipelines/update_demo_env.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ steps:
- command: .buildkite/scripts/steps/demo_env/es_and_init.sh
label: Initialize Environment and Deploy ES
timeout_in_minutes: 10
agents:
queue: kibana-default

- command: .buildkite/scripts/steps/demo_env/kibana.sh
label: Build and Deploy Kibana
Expand Down
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -95,4 +95,3 @@ fleet-server-*
elastic-agent.yml
fleet-server.yml

/x-pack/plugins/fleet/server/bundled_packages
1 change: 1 addition & 0 deletions .i18nrc.json
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@
"uiActions": "src/plugins/ui_actions",
"uiActionsExamples": "examples/ui_action_examples",
"usageCollection": "src/plugins/usage_collection",
"utils": "packages/kbn-securitysolution-utils/src",
"visDefaultEditor": "src/plugins/vis_default_editor",
"visTypeHeatmap": "src/plugins/vis_types/heatmap",
"visTypeMarkdown": "src/plugins/vis_type_markdown",
Expand Down
2 changes: 1 addition & 1 deletion docs/api/saved-objects/resolve_import_errors.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ To resolve errors, you can:
==== Path parameters

`space_id`::
(Optional, string) An identifier for the <<xpack-spaces,space>>. When `space_id` is unspecfied in the URL, the default space is used.
(Optional, string) An identifier for the <<xpack-spaces,space>>. When `space_id` is unspecified in the URL, the default space is used.

[[saved-objects-api-resolve-import-errors-query-params]]
==== Query parameters
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ Execute the <<spaces-api-copy-saved-objects,copy saved objects to space API>>, w
`id`::::
(Required, string) The saved object ID.
`overwrite`::::
(Required, boolean) When set to `true`, the saved object from the source space (desigated by the <<spaces-api-resolve-copy-saved-objects-conflicts-path-params, `space_id` path parameter>>) overwrites the conflicting object in the destination space. When set to `false`, this does nothing.
(Required, boolean) When set to `true`, the saved object from the source space (designated by the <<spaces-api-resolve-copy-saved-objects-conflicts-path-params, `space_id` path parameter>>) overwrites the conflicting object in the destination space. When set to `false`, this does nothing.
`destinationId`::::
(Optional, string) Specifies the destination ID that the copied object should have, if different from the current ID.
`ignoreMissingReferences`:::
Expand Down
2 changes: 1 addition & 1 deletion docs/api/upgrade-assistant/default-field.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ GET /api/upgrade_assistant/add_query_default_field/myIndex
// KIBANA

<1> A required array of {es} field types that generate the list of fields.
<2> An optional array of additional field names, dot-deliminated.
<2> An optional array of additional field names, dot-delimited.

To add the `index.query.default_field` index setting to the specified index, {kib} generates an array of all fields from the index mapping.
The fields contain the types specified in `fieldTypes`. {kib} appends any other fields specified in `otherFields` to the array of default fields.
Expand Down
2 changes: 1 addition & 1 deletion docs/apm/service-maps.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ image:apm/images/red-service.png[APM red service]:: Max anomaly score **≥75**.
[role="screenshot"]
image::apm/images/apm-service-map-anomaly.png[Example view of anomaly scores on service maps in the APM app]

If an anomaly has been detected, click *view anomalies* to view the anomaly detection metric viewier in the Machine learning app.
If an anomaly has been detected, click *view anomalies* to view the anomaly detection metric viewer in the Machine learning app.
This time series analysis will display additional details on the severity and time of the detected anomalies.

To learn how to create a machine learning job, see <<machine-learning-integration,machine learning integration>>.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -221,7 +221,7 @@ These are the contracts exposed by the core services for each lifecycle:
[cols=",,",options="header",]
|===
|lifecycle |server contract|browser contract
|_contructor_
|_constructor_
|{kib-repo}blob/{branch}/docs/development/core/server/kibana-plugin-core-server.plugininitializercontext.md[PluginInitializerContext]
|{kib-repo}blob/{branch}/docs/development/core/public/kibana-plugin-core-public.plugininitializercontext.md[PluginInitializerContext]

Expand Down
2 changes: 1 addition & 1 deletion docs/developer/best-practices/typescript.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ Additionally, in order to migrate into project refs, you also need to make sure
],
"references": [
{ "path": "../../core/tsconfig.json" },
// add references to other TypeScript projects your plugin dependes on
// add references to other TypeScript projects your plugin depends on
]
}
----
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -137,4 +137,4 @@ If you only want to run the build once you can run:
node scripts/build_kibana_platform_plugins --validate-limits --focus {pluginId}
-----------

This command needs to apply production optimizations to get the right sizes, which means that the optimizer will take significantly longer to run and on most developmer machines will consume all of your machines resources for 20 minutes or more. If you'd like to multi-task while this is running you might need to limit the number of workers using the `--max-workers` flag.
This command needs to apply production optimizations to get the right sizes, which means that the optimizer will take significantly longer to run and on most developer machines will consume all of your machines resources for 20 minutes or more. If you'd like to multi-task while this is running you might need to limit the number of workers using the `--max-workers` flag.
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ node scripts/docs.js --open

REST APIs should be documented using the following recommended formats:

* https://raw.githubusercontent.com/elastic/docs/master/shared/api-ref-ex.asciidoc[API doc templaate]
* https://raw.githubusercontent.com/elastic/docs/master/shared/api-ref-ex.asciidoc[API doc template]
* https://raw.githubusercontent.com/elastic/docs/master/shared/api-definitions-ex.asciidoc[API object definition template]

[discrete]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ image::images/job_view.png[Jenkins job view showing a test failure]
1. *Git Changes:* the list of commits that were in this build which weren't in the previous build. For Pull Requests this list is calculated by comparing against the most recent Pull Request which was tested, it is not limited to build for this specific Pull Request, so it's not very useful.
2. *Test Results:* A link to the test results screen, and shortcuts to the failed tests. Functional tests capture and store the log output from each specific test, and make it visible at these links. For other test runners only the error message is visible and log output must be tracked down in the *Pipeline Steps*.
3. *Google Cloud Storage (GCS) Upload Report:* Link to the screen which lists out the artifacts uploaded to GCS during this job execution.
4. *Pipeline Steps:*: A breakdown of the pipline that was executed, along with individual log output for each step in the pipeline.
4. *Pipeline Steps:*: A breakdown of the pipeline that was executed, along with individual log output for each step in the pipeline.

[discrete]
=== Viewing ciGroup/test Logs
Expand Down
26 changes: 26 additions & 0 deletions docs/settings/enterprise-search-settings.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
[role="xpack"]
[[enterprise-search-settings-kb]]
=== Enterprise Search settings in {kib}
++++
<titleabbrev>Enterprise Search settings</titleabbrev>
++++

On Elastic Cloud, you do not need to configure any settings to use Enterprise Search in {kib}. It is enabled by default. On self-managed installations, you must configure `enterpriseSearch.host`.

`enterpriseSearch.host`::
The http(s) URL of your Enterprise Search instance. For example, in a local self-managed setup,
set this to `http://localhost:3002`. Authentication between {kib} and the Enterprise Search host URL,
such as via OAuth, is not supported. You can also
{enterprise-search-ref}/configure-ssl-tls.html#configure-ssl-tls-in-kibana[configure {kib} to trust
your Enterprise Search TLS certificate authority].


`enterpriseSearch.accessCheckTimeout`::
When launching the Enterprise Search UI, the maximum number of milliseconds for {kib} to wait
for a response from Enterprise Search
before considering the attempt failed and logging a warning.
Default: 5000.

`enterpriseSearch.accessCheckTimeoutWarning`::
When launching the Enterprise Search UI, the maximum number of milliseconds for {kib} to wait for a response from
Enterprise Search before logging a warning. Default: 300.
1 change: 1 addition & 0 deletions docs/setup/configuring-reporting.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ If you are using Ubuntu/Debian systems, install the following packages:

* `fonts-liberation`
* `libfontconfig1`
* `libnss3`

If the system is missing dependencies, *Reporting* fails in a non-deterministic way. {kib} runs a self-test at server startup, and
if it encounters errors, logs them in the Console. The error message does not include
Expand Down
2 changes: 1 addition & 1 deletion docs/setup/connect-to-elasticsearch.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ https://www.elastic.co/guide/en/elasticsearch/client/index.html[{es} Client docu

If you are running {kib} on our hosted {es} Service,
click *View deployment details* on the *Integrations* view
to verify your {es} endpoint and Cloud ID, and create API keys for integestion.
to verify your {es} endpoint and Cloud ID, and create API keys for integration.

[float]
=== Add sample data
Expand Down
6 changes: 2 additions & 4 deletions docs/setup/settings.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -282,9 +282,6 @@ on the {kib} index at startup. {kib} users still need to authenticate with
that the {kib} server uses to perform maintenance on the {kib} index at startup. This setting
is an alternative to `elasticsearch.username` and `elasticsearch.password`.

| `enterpriseSearch.host`
| The http(s) URL of your Enterprise Search instance. For example, in a local self-managed setup, set this to `http://localhost:3002`. Authentication between Kibana and the Enterprise Search host URL, such as via OAuth, is not supported. You can also {enterprise-search-ref}/configure-ssl-tls.html#configure-ssl-tls-in-kibana[configure Kibana to trust your Enterprise Search TLS certificate authority].

| `interpreter.enableInVisualize`
| Enables use of interpreter in Visualize. *Default: `true`*

Expand Down Expand Up @@ -719,15 +716,16 @@ Valid locales are: `en`, `zh-CN`, `ja-JP`. *Default: `en`*
include::{kib-repo-dir}/settings/alert-action-settings.asciidoc[]
include::{kib-repo-dir}/settings/apm-settings.asciidoc[]
include::{kib-repo-dir}/settings/banners-settings.asciidoc[]
include::{kib-repo-dir}/settings/enterprise-search-settings.asciidoc[]
include::{kib-repo-dir}/settings/fleet-settings.asciidoc[]
include::{kib-repo-dir}/settings/i18n-settings.asciidoc[]
include::{kib-repo-dir}/settings/logging-settings.asciidoc[]
include::{kib-repo-dir}/settings/logs-ui-settings.asciidoc[]
include::{kib-repo-dir}/settings/infrastructure-ui-settings.asciidoc[]
include::{kib-repo-dir}/settings/monitoring-settings.asciidoc[]
include::{kib-repo-dir}/settings/reporting-settings.asciidoc[]
include::secure-settings.asciidoc[]
include::{kib-repo-dir}/settings/search-sessions-settings.asciidoc[]
include::secure-settings.asciidoc[]
include::{kib-repo-dir}/settings/security-settings.asciidoc[]
include::{kib-repo-dir}/settings/spaces-settings.asciidoc[]
include::{kib-repo-dir}/settings/task-manager-settings.asciidoc[]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ Scaling {kib} instances horizontally requires a higher degree of coordination, w
A recommended strategy is to follow these steps:

1. Produce a <<task-manager-rough-throughput-estimation,rough throughput estimate>> as a guide to provisioning as many {kib} instances as needed. Include any growth in tasks that you predict experiencing in the near future, and a buffer to better address ad-hoc tasks.
2. After provisioning a deployment, assess whether the provisioned {kib} instances achieve the required throughput by evaluating the <<task-manager-health-monitoring>> as described in <<task-manager-theory-insufficient-throughput, Insufficient throughtput to handle the scheduled workload>>.
2. After provisioning a deployment, assess whether the provisioned {kib} instances achieve the required throughput by evaluating the <<task-manager-health-monitoring>> as described in <<task-manager-theory-insufficient-throughput, Insufficient throughput to handle the scheduled workload>>.
3. If the throughput is insufficient, and {kib} instances exhibit low resource usage, incrementally scale vertically while <<kibana-page,monitoring>> the impact of these changes.
4. If the throughput is insufficient, and {kib} instances are exhibiting high resource usage, incrementally scale horizontally by provisioning new {kib} instances and reassess.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -412,7 +412,7 @@ This assessment is based on the following:

* Comparing the `last_successful_poll` to the `timestamp` (value of `2021-02-16T11:38:10.077Z`) at the root, where you can see the last polling cycle took place 1 second before the monitoring stats were exposed by the health monitoring API.
* Comparing the `last_polling_delay` to the `timestamp` (value of `2021-02-16T11:38:10.077Z`) at the root, where you can see the last polling cycle delay took place 2 days ago, suggesting {kib} instances are not conflicting often.
* The `p50` of the `duration` shows that at least 50% of polling cycles take, at most, 13 millisconds to complete.
* The `p50` of the `duration` shows that at least 50% of polling cycles take, at most, 13 milliseconds to complete.
* Evaluating the `result_frequency_percent_as_number`:
** 80% of the polling cycles completed without claiming any tasks (suggesting that there aren't any overdue tasks).
** 20% completed with Task Manager claiming tasks that were then executed.
Expand Down Expand Up @@ -508,7 +508,7 @@ For details on achieving higher throughput by adjusting your scaling strategy, s
Tasks run for too long, overrunning their schedule

*Diagnosis*:
The <<task-manager-theory-insufficient-throughput,Insufficient throughtput to handle the scheduled workload>> theory analyzed a hypothetical scenario where both drift and load were unusually high.
The <<task-manager-theory-insufficient-throughput,Insufficient throughput to handle the scheduled workload>> theory analyzed a hypothetical scenario where both drift and load were unusually high.

Suppose an alternate scenario, where `drift` is high, but `load` is not, such as the following:

Expand Down Expand Up @@ -688,7 +688,7 @@ Keep in mind that these stats give you a glimpse at a moment in time, and even t
[[task-manager-health-evaluate-the-workload]]
===== Evaluate the Workload

Predicting the required throughput a deplyment might need to support Task Manager is difficult, as features can schedule an unpredictable number of tasks at a variety of scheduled cadences.
Predicting the required throughput a deployment might need to support Task Manager is difficult, as features can schedule an unpredictable number of tasks at a variety of scheduled cadences.

<<task-manager-health-monitoring>> provides statistics that make it easier to monitor the adequacy of the existing throughput.
By evaluating the workload, the required throughput can be estimated, which is used when following the Task Manager <<task-manager-scaling-guidance>>.
Expand Down
Loading

0 comments on commit dee55f3

Please sign in to comment.