Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failing ES promotion: x-pack/test/api_integration/apis/ml/indices/field_caps.ts #182514

Closed
jbudz opened this issue May 2, 2024 · 4 comments · Fixed by #182588
Closed

Failing ES promotion: x-pack/test/api_integration/apis/ml/indices/field_caps.ts #182514

jbudz opened this issue May 2, 2024 · 4 comments · Fixed by #182588
Assignees
@walterra
Labels
failed-es-promotion :ml

Comments

@jbudz
Copy link
Member

jbudz commented May 2, 2024

X-Pack API Integration Tests - ml
x-pack/test/api_integration/apis/ml/indices/field_caps.ts

Machine Learning system field_caps all fields in index

Buildkite Job
https://buildkite.com/elastic/kibana-elasticsearch-snapshot-verify/builds/3975#018f3978-acfe-4877-ac2e-febcf4316e0b

Error: Expected number of fields to be 21, but got 22
    at Assertion.assert (expect.js:100:11)
    at Assertion.eql (expect.js:244:8)
    at Context.<anonymous> (field_caps.ts:64:31)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at Object.apply (wrap_function.js:73:16) {
  actual: '22',
  expected: '21',
  showDiff: true

This failure is preventing the promotion of the current Elasticsearch snapshot.

For more information on the Elasticsearch snapshot process including how to reproduce using the unverified ES build please read the failed promotion annotation. Other important information can be found at:
@elasticmachine
Copy link
Contributor

Pinging @elastic/ml-ui (:ml)

jbudz added a commit that referenced this issue May 2, 2024
@jbudz
skip suite failing es promotion (#182514)
d824b2a
@jbudz
Copy link
Member Author

jbudz commented May 2, 2024

Skipped

main: d824b2a

@walterra
Copy link
Contributor

walterra commented May 3, 2024

Removing the blocker label. ES fields caps returns an additional metadata field now which makes the tests fail. The UI which uses this functionality will still work.

@walterra walterra removed the blocker label May 3, 2024
@walterra walterra mentioned this issue May 3, 2024
Merged
2 tasks
jbudz added a commit that referenced this issue May 6, 2024
@jbudz
skip suite failing es forward compatibility (#182514)
62f13e6
@jbudz
Copy link
Member Author

jbudz commented May 6, 2024

Also failing the 7.17 / ES 8.15 forward compatibility test - https://buildkite.com/elastic/kibana-7-dot-17-es-8-dot-15-forward-compatibility/builds/18

Skipped 7.17: 62f13e6

jbudz added a commit that referenced this issue May 6, 2024
@jbudz
skip suite failing es forward compatibility (#182514)
01cdef1
jbudz added a commit to jbudz/kibana that referenced this issue May 6, 2024
@jbudz
Revert "skip suite failing es forward compatibility (elastic#182514)"
d2e7748 
This reverts commit 01cdef1.
walterra added a commit that referenced this issue May 10, 2024
@walterra
[ML] Anomaly Detection: Use data view esTypes instead of custom fie…
71e7f5a 
…ld caps wrapper endpoint. (#182588)

## Summary

Fixes #182514.

The links menu for the anomalies table used a custom field caps wrapper
API endpoint to identify the type of the categorization field. This PR
changes this to use the data view API instead to use `esTypes` for the
same check. This allows us to remove the custom field caps API endpoint
completely.

Removes the route `POST /internal/ml/indices/field_caps` as it is no
longer required by the ml plugin.

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
@walterra walterra mentioned this issue May 10, 2024
Merged
2 tasks
walterra added a commit that referenced this issue May 10, 2024
@walterra
[ML] Anomaly Detection: Fix API integration tests for field caps. (#1…
8f04ab6 
…83110)

## Summary

Fixes #182837.
Fixes #182514.

The number of fields returned by the field caps API is different across
ES versions in forward compatibility tests. In ES 8.15.0, the
`_ignored_source` field was added
(elastic/elasticsearch#107567). This fixes the
API integration test for field caps to assert the correct number of
fields across versions. Note that in Kibana `8.15` we refactored away
from using fields caps directly in this way and removed the
corresponding API endpoint and tests
(#182588), that's why there's this
dedicated `7.17` PR to just fix the assertions on the existing test.

To test this locally, the following commands for the functional tests
server and runner can be used to run the tests in different forward
compatibility scenarios:

```
# 7.17 tests server
node scripts/functional_tests_server.js --config x-pack/test/api_integration/config.ts
# 7.17 tests runner
node scripts/functional_test_runner --config x-pack/test/api_integration/config.ts

# 8.14 tests server
ES_SNAPSHOT_MANIFEST="https://storage.googleapis.com/kibana-ci-es-snapshots-daily/8.14.0/manifest-latest-verified.json" node scripts/functional_tests_server.js
# 8.14 tests runner
node scripts/functional_test_runner --config x-pack/test/api_integration/config.ts --es-version=8.14.0-SNAPSHOT

# 8.15 tests server
ES_SNAPSHOT_MANIFEST="https://storage.googleapis.com/kibana-ci-es-snapshots-daily/8.15.0/manifest-latest-verified.json" node scripts/functional_tests_server.js
# 8.15 tests runner
node scripts/functional_test_runner --config x-pack/test/api_integration/config.ts --es-version=8.15.0-SNAPSHOT
```

Note in `7.17` the API integration tests are not split up yet into
several configs so the commands above will run ALL Kibaan API
integration tests.

The command to run the tests server for a specific ES version is also
shared in the buildkite reports, for example:
https://buildkite.com/elastic/kibana-7-dot-17-es-8-dot-15-forward-compatibility/builds/20#annotation-es-snapshot-manifest

The versions the compatibility tests will currently run against can be
found here: https://github.com/elastic/kibana/blob/main/versions.json

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
adelisle pushed a commit to Makila-AI/kibana that referenced this issue Aug 5, 2024
@jbudz @adelisle
skip suite failing es forward compatibility (elastic#182514)
1de4e44
adelisle pushed a commit to Makila-AI/kibana that referenced this issue Aug 5, 2024
@walterra @adelisle
[ML] Anomaly Detection: Fix API integration tests for field caps. (el…
cd960de 
…astic#183110)

## Summary

Fixes elastic#182837.
Fixes elastic#182514.

The number of fields returned by the field caps API is different across
ES versions in forward compatibility tests. In ES 8.15.0, the
`_ignored_source` field was added
(elastic/elasticsearch#107567). This fixes the
API integration test for field caps to assert the correct number of
fields across versions. Note that in Kibana `8.15` we refactored away
from using fields caps directly in this way and removed the
corresponding API endpoint and tests
(elastic#182588), that's why there's this
dedicated `7.17` PR to just fix the assertions on the existing test.

To test this locally, the following commands for the functional tests
server and runner can be used to run the tests in different forward
compatibility scenarios:

```
# 7.17 tests server
node scripts/functional_tests_server.js --config x-pack/test/api_integration/config.ts
# 7.17 tests runner
node scripts/functional_test_runner --config x-pack/test/api_integration/config.ts

# 8.14 tests server
ES_SNAPSHOT_MANIFEST="https://storage.googleapis.com/kibana-ci-es-snapshots-daily/8.14.0/manifest-latest-verified.json" node scripts/functional_tests_server.js
# 8.14 tests runner
node scripts/functional_test_runner --config x-pack/test/api_integration/config.ts --es-version=8.14.0-SNAPSHOT

# 8.15 tests server
ES_SNAPSHOT_MANIFEST="https://storage.googleapis.com/kibana-ci-es-snapshots-daily/8.15.0/manifest-latest-verified.json" node scripts/functional_tests_server.js
# 8.15 tests runner
node scripts/functional_test_runner --config x-pack/test/api_integration/config.ts --es-version=8.15.0-SNAPSHOT
```

Note in `7.17` the API integration tests are not split up yet into
several configs so the commands above will run ALL Kibaan API
integration tests.

The command to run the tests server for a specific ES version is also
shared in the buildkite reports, for example:
https://buildkite.com/elastic/kibana-7-dot-17-es-8-dot-15-forward-compatibility/builds/20#annotation-es-snapshot-manifest

The versions the compatibility tests will currently run against can be
found here: https://github.com/elastic/kibana/blob/main/versions.json

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@walterra walterra

Labels
failed-es-promotion :ml
Projects
None yet
Milestone
No milestone
4 participants
@walterra @jbudz @elasticmachine @kibanamachine