Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Event Log] Fixing call to update index alias to hidden #122882

Merged
merged 5 commits into from
Jan 18, 2022

Conversation

ymao1
Copy link
Contributor

@ymao1 ymao1 commented Jan 12, 2022

Resolves #121298

Summary

When upgrading from <= 7.15 versions of Kibana, the event log will try to update existing event log indices to hidden. Part of this process involves setting existing index aliases to is_hidden. It turns out if you have one alias pointing to multiple indices, we need to update the alias settings in a single command instead of consecutive commands as we were doing previously.

For example, if our 7.15 event log aliases look like this:

{
	".kibana-event-log-7.15.2-000003": {
		"aliases": {
			".kibana-event-log-7.15.2": {
				"is_write_index": true
			}
		}
	},
	".kibana-event-log-7.15.2-000002": {
		"aliases": {
			".kibana-event-log-7.15.2": {
				"is_write_index": false
			}
		}
	},
	".kibana-event-log-7.15.2-000001": {
		"aliases": {
			".kibana-event-log-7.15.2": {
				"is_write_index": false
			}
		}
	}
}

Previously, we were issuing 3 calls to the update alias API, each one looking like this:

{
	actions: [{
		add: {
			index: <indexName>,
			alias: '.kibana-event-log-7.15.2',
			is_hidden: true,
		},
	}, ],
}

We actually need to be issuing 1 call to the update alias API that looks like this:

{
	"actions": [{
			"add": {
				"is_write_index": false,
				"index": ".kibana-event-log-7.15.2-000002",
				"alias": ".kibana-event-log-7.15.2",
				"is_hidden": true
			}
		},
		{
			"add": {
				"is_write_index": true,
				"index": ".kibana-event-log-7.15.2-000003",
				"alias": ".kibana-event-log-7.15.2",
				"is_hidden": true
			}
		},
		{
			"add": {
				"is_write_index": false,
				"index": ".kibana-event-log-7.15.2-000001",
				"alias": ".kibana-event-log-7.15.2",
				"is_hidden": true
			}
		}
	]
}

This PR updates to do this correctly.

To Verify

  1. Change the Event log ILM policy to max_docs: 1 (to get ILM to roll over faster). Run 7.15 (or previous version) and let some rules/actions run to generate some event log docs and get multiple event log indices.
  2. Run main using the previous data and notice that there is an error in the logs when updating the index aliases that cause the aliases not to get updated to hidden.
  3. Run this branch using the previous data and notice no errors in the logs that verify that the index aliases are all updated to hidden.

Checklist

@ymao1 ymao1 changed the title [Event Log] Adding await to asyncForEach [Event Log] Fixing call to update index alias to hidden Jan 13, 2022
@ymao1 ymao1 marked this pull request as ready for review January 13, 2022 19:56
@ymao1 ymao1 self-assigned this Jan 13, 2022
@ymao1 ymao1 added Feature:EventLog release_note:fix Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) v7.17.0 v8.0.0 v8.1.0 labels Jan 13, 2022
@elasticmachine
Copy link
Contributor

Pinging @elastic/response-ops (Team:ResponseOps)

Copy link
Contributor

@mikecote mikecote left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes LGTM! I was able to reproduce the bug on main and confirmed it going away with this PR 👍

@ymao1
Copy link
Contributor Author

ymao1 commented Jan 18, 2022

@elasticmachine merge upstream

Copy link
Member

@pmuellr pmuellr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM; this was more complicated than I expected - I thought we just needed an await before all the asyncForEach() that got added a while back! This is probably a better approach though, get them all done in one call.

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id before after diff
eventLog 5 6 +1
kibana 957 956 -1
total -0

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @ymao1

@ymao1 ymao1 added the auto-backport Deprecated: Automatically backport this PR after it's merged label Jan 18, 2022
@ymao1 ymao1 merged commit bd57aa5 into elastic:main Jan 18, 2022
@kibanamachine
Copy link
Contributor

💔 All backports failed

Status Branch Result
8.0 Backport failed because of merge conflicts

You might need to backport the following PRs to 8.0:
- Add migration support to the event log (#58010)
- Increase stability when initializing the Elasticsearch index for the event log (#57465)
- Adds event log for actions and alerting (#45081)
7.17 Backport failed because of merge conflicts

How to fix

Re-run the backport manually:

node scripts/backport --pr 122882

Questions ?

Please refer to the Backport tool documentation

ymao1 added a commit to ymao1/kibana that referenced this pull request Jan 18, 2022
* Adding await

* Changing the way alias updates are made

* Updating unit tests

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
(cherry picked from commit bd57aa5)

# Conflicts:
#	x-pack/plugins/event_log/server/es/init.ts
ymao1 added a commit to ymao1/kibana that referenced this pull request Jan 18, 2022
* Adding await

* Changing the way alias updates are made

* Updating unit tests

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
(cherry picked from commit bd57aa5)

# Conflicts:
#	x-pack/plugins/event_log/server/es/init.ts
ymao1 added a commit that referenced this pull request Jan 18, 2022
…23238)

* Adding await

* Changing the way alias updates are made

* Updating unit tests

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
(cherry picked from commit bd57aa5)

# Conflicts:
#	x-pack/plugins/event_log/server/es/init.ts
ymao1 added a commit that referenced this pull request Jan 18, 2022
… (#123237)

* [Event Log] Fixing call to update index alias to hidden (#122882)

* Adding await

* Changing the way alias updates are made

* Updating unit tests

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
(cherry picked from commit bd57aa5)

# Conflicts:
#	x-pack/plugins/event_log/server/es/init.ts

* Fixing types check
@ymao1 ymao1 deleted the event-log/hidden-again branch January 27, 2022 19:59
ogupte pushed a commit to ogupte/kibana that referenced this pull request Jan 28, 2022
* Adding await

* Changing the way alias updates are made

* Updating unit tests

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport Deprecated: Automatically backport this PR after it's merged Feature:EventLog release_note:fix Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) v7.17.0 v8.0.0 v8.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[event log] unable to set some old event log alias to hidden when underlying indices are not hidden
6 participants