Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use any index specified by .watches for Watcher #39541

Merged
merged 9 commits into from Mar 5, 2019

Conversation

Projects
None yet
3 participants
@gwbrown
Copy link
Contributor

commented Mar 1, 2019

Previously, Watcher only attached its listener to indices that started
with the prefix .watches, which causes Watcher to silently fail to
schedule newly created Watches if the .watches alias is redirected to
an index that does not start with .watches.

Watcher now attaches the listener to all indices, so that Watcher can
respond to changes in which index has the .watches alias.

All tests now randomly use a concrete index which does not start
with .watches (same for .triggered_watches), as well this behavior
being explicitly tested.

Relates to #39478

Use any index specified by .watches for Watcher
Previously, Watcher only attached its listener to indices that started
with the prefix `.watches`, which causes Watcher to silently fail to
schedule newly created Watches if the `.watches` alias is redirected to
an index that does not start with `.watches`.

Watcher now attaches the listener to all indices, so that Watcher can
respond to changes in which index has the `.watches` alias.
@elasticmachine

This comment has been minimized.

Copy link
Collaborator

commented Mar 1, 2019

@gwbrown

This comment has been minimized.

Copy link
Contributor Author

commented Mar 1, 2019

I did some basic tests with Rally and this doesn't seem to noticeably impact indexing performance, but I've only done a couple of trials.

@gwbrown gwbrown requested review from hub-cap and jakelandis Mar 1, 2019

@gwbrown gwbrown marked this pull request as ready for review Mar 1, 2019

@gwbrown

This comment has been minimized.

Copy link
Contributor Author

commented Mar 2, 2019

While I've adjusted the tests to randomly use non-prefixed concrete indices for .watches and .triggered_watches, I've run :x-pack:plugin:watcher:check many times locally with this option forced on (that is, using only non-prefixed indices) to ensure that this is stable.

@gwbrown

This comment has been minimized.

Copy link
Contributor Author

commented Mar 4, 2019

It's worth noting that the behavior of Watcher when changing the alias is still the same as it was before when it was restricted to prefixed index names: If the new index pointed to by .watches contains the same watches as the old index, everything will keep working just fine. If the contents are different, Watcher will need to be restarted to force it to reload watch definitions from the index.

It's best to switch the alias with Watcher stopped for this reason, but isn't technically necessary as long as the old index and the new index have the same contents.

@jakelandis
Copy link
Contributor

left a comment

LGTM

@gwbrown gwbrown merged commit d79cd8c into elastic:master Mar 5, 2019

8 checks passed

CLA All commits in pull request signed
Details
elasticsearch-ci/1 Build finished.
Details
elasticsearch-ci/2 Build finished.
Details
elasticsearch-ci/bwc Build finished.
Details
elasticsearch-ci/default-distro Build finished.
Details
elasticsearch-ci/docbldesx Build finished.
Details
elasticsearch-ci/oss-distro-docs Build finished.
Details
elasticsearch-ci/packaging-sample Build finished.
Details

gwbrown added a commit to gwbrown/elasticsearch that referenced this pull request Mar 5, 2019

Use any index specified by .watches for Watcher (elastic#39541)
Previously, Watcher only attached its listener to indices that started
with the prefix `.watches`, which causes Watcher to silently fail to
schedule newly created Watches if the `.watches` alias is redirected to
an index that does not start with `.watches`.

Watcher now attaches the listener to all indices, so that Watcher can
respond to changes in which index has the `.watches` alias.

Also adjusts the tests to randomly use non-prefixed concrete indices for .watches and .triggered_watches.

gwbrown added a commit to gwbrown/elasticsearch that referenced this pull request Mar 5, 2019

Use any index specified by .watches for Watcher (elastic#39541)
Previously, Watcher only attached its listener to indices that started
with the prefix `.watches`, which causes Watcher to silently fail to
schedule newly created Watches if the `.watches` alias is redirected to
an index that does not start with `.watches`.

Watcher now attaches the listener to all indices, so that Watcher can
respond to changes in which index has the `.watches` alias.

Also adjusts the tests to randomly use non-prefixed concrete indices for .watches and .triggered_watches.

gwbrown added a commit to gwbrown/elasticsearch that referenced this pull request Mar 5, 2019

Use any index specified by .watches for Watcher (elastic#39541)
Previously, Watcher only attached its listener to indices that started
with the prefix `.watches`, which causes Watcher to silently fail to
schedule newly created Watches if the `.watches` alias is redirected to
an index that does not start with `.watches`.

Watcher now attaches the listener to all indices, so that Watcher can
respond to changes in which index has the `.watches` alias.

Also adjusts the tests to randomly use non-prefixed concrete indices for .watches and .triggered_watches.

gwbrown added a commit that referenced this pull request Mar 5, 2019

Use any index specified by .watches for Watcher (#39541) (#39706)
* Use any index specified by .watches for Watcher (#39541)

Previously, Watcher only attached its listener to indices that started
with the prefix `.watches`, which causes Watcher to silently fail to
schedule newly created Watches if the `.watches` alias is redirected to
an index that does not start with `.watches`.

Watcher now attaches the listener to all indices, so that Watcher can
respond to changes in which index has the `.watches` alias.

Also adjusts the tests to randomly use non-prefixed concrete indices 
for .watches and .triggered_watches.

gwbrown added a commit that referenced this pull request Mar 5, 2019

Use any index specified by .watches for Watcher (#39541) (#39707)
Previously, Watcher only attached its listener to indices that started
with the prefix `.watches`, which causes Watcher to silently fail to
schedule newly created Watches if the `.watches` alias is redirected to
an index that does not start with `.watches`.

Watcher now attaches the listener to all indices, so that Watcher can
respond to changes in which index has the `.watches` alias.

Also adjusts the tests to randomly use non-prefixed concrete indices 
for .watches and .triggered_watches.

gwbrown added a commit that referenced this pull request Mar 5, 2019

Use any index specified by .watches for Watcher (#39541) (#39708)
Previously, Watcher only attached its listener to indices that started
with the prefix `.watches`, which causes Watcher to silently fail to
schedule newly created Watches if the `.watches` alias is redirected to
an index that does not start with `.watches`.

Watcher now attaches the listener to all indices, so that Watcher can
respond to changes in which index has the `.watches` alias.

Also adjusts the tests to randomly use non-prefixed concrete indices 
for .watches and .triggered_watches.

jasontedor added a commit to jasontedor/elasticsearch that referenced this pull request Mar 6, 2019

Merge branch '6.7' into retention-lease-bwc
* 6.7: (39 commits)
  Remove beta label from CCR (elastic#39722)
  Rename retention lease setting (elastic#39719)
  Add Docker build type (elastic#39378)
  Use any index specified by .watches for Watcher (elastic#39541) (elastic#39706)
  Add documentation on remote recovery (elastic#39483)
  fix typo in synonym graph filter docs
  Removed incorrect ML YAML tests (elastic#39400)
  Improved Terms Aggregation documentation (elastic#38892)
  Fix Fuzziness#asDistance(String) (elastic#39643)
  Revert "unmute EvilLoggerTests#testDeprecatedSettings (elastic#38743)"
  Mute TokenAuthIntegTests.testExpiredTokensDeletedAfterExpiration (elastic#39690)
  Fix security index auto-create and state recovery race (elastic#39582)
  [DOCS] Sorts security APIs
  Check for .watches that wasn't upgraded properly (elastic#39609)
  Assert recovery done in testDoNotWaitForPendingSeqNo (elastic#39595)
  [DOCS] Updates API in Watcher transform context (elastic#39540)
  Fixing the custom object serialization bug in diffable utils. (elastic#39544)
  mute test
  SQL: Don't allow inexact fields for MIN/MAX (elastic#39563)
  Update release notes for 6.7.0
  ...

@jakelandis jakelandis added v7.0.0-rc2 and removed v7.0.0 labels Apr 3, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.