Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Connector should validate 'is_write_index' flag on aliases during write operations #1197

Closed
jbaiera opened this issue Oct 2, 2018 · 0 comments

Comments

@jbaiera
Copy link
Member

jbaiera commented Oct 2, 2018

ES-Hadoop has been historically very hands off when it comes to aliases. Recently added in 6.X is the ability to specify an index within an alias to be used for accepting write operations.

We should check the write resource when the job starts up to detect if an alias is being used for the write operation, and validate that either there is only one index within it, or if multiple indices are given for an alias, that one is marked as the write index. This will allow a job to fail faster than if an alias cannot be written to at all. Additionally, we should not attempt to pin to any nodes that are hosting the shards for those writes, since the fact that an alias is used for writing means that the shards that we are writing to might change from under us at any time, making the original node pinning not as useful.

@jbaiera jbaiera closed this as completed in f86d9c5 Oct 3, 2018
jbaiera added a commit that referenced this issue Oct 3, 2018
Validate the alias for writing if it is an alias, checking for a single
index or if an index has a write index flag set.

fixes #1197
@jbaiera jbaiera added the v6.5.0 label Oct 3, 2018
@acchen97 acchen97 mentioned this issue Oct 3, 2018
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant