Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow Search API to talk to multiple elasticsearch clusters #1569

Merged
merged 1 commit into from Jun 12, 2019

Conversation

Projects
None yet
2 participants
@bilbof
Copy link
Contributor

commented Jun 4, 2019

This enables search-api to send requests to multiple
elasticsearch clusters. You can specify the clusters
you would like to query using the query param cluster.

E.g. /search.json?cluster=B will query elasticsearch6.
The default behaviour is to query cluster A, which is
currently elasticsearch5.

This is needed now because we want to upgrade from
elasticsearch version 5 to 6, by querying 5 or 6 ad hoc.

This will have the side effect that we will be able to
do a/b tests and compare indexes/clusters more easily.

All writes to one cluster should be performed across all clusters

It also returns es_cluster with search responses.

Trello: https://trello.com/c/uTsa3fpt/745.

@bilbof bilbof requested a review from barrucadu Jun 4, 2019

Show resolved Hide resolved elasticsearch.yml Outdated

@bilbof bilbof force-pushed the select_cluster branch from 0dbd105 to 00c2da3 Jun 4, 2019

Show resolved Hide resolved elasticsearch.yml Outdated

@bilbof bilbof force-pushed the select_cluster branch 2 times, most recently from 27f9a43 to 566932c Jun 10, 2019

@bilbof bilbof force-pushed the select_cluster branch from 566932c to 11532f0 Jun 10, 2019

@barrucadu
Copy link
Contributor

left a comment

We should put this on integration and exercise it a bit

bilbof added a commit to alphagov/finder-frontend that referenced this pull request Jun 10, 2019

Send cluster param to search-api
This will permit end users to specify which cluster they
would like to use.

Trello: https://trello.com/c/gRe2XQGx/796.

Should be merged after alphagov/search-api#1569
is deployed.
Allow Search API to talk to multiple elasticsearch clusters
This enables search-api to send requests to multiple
elasticsearch clusters. You can specify the clusters
you would like to query using the query param cluster.

E.g. /search.json?cluster=B will query elasticsearch6.
The default behaviour is to query cluster A, which is
currently elasticsearch5.

This is needed now because we want to upgrade from
elasticsearch version 5 to 6, by querying 5 or 6 ad hoc.

This will have the side effect that we will be able to
do a/b tests and compare indexes/clusters more easily.

All write actions on clusters should now be performed
on all active clusters in an environment.

Some rake tasks now have an optional cluster argument,
but by default rake tasks will run against all clusters.

Trello: https://trello.com/c/uTsa3fpt/745.

@bilbof bilbof force-pushed the select_cluster branch from 11532f0 to 803fb83 Jun 12, 2019

@bilbof bilbof merged commit 6e67b8c into master Jun 12, 2019

1 of 2 checks passed

continuous-integration/jenkins/publishing-e2e-tests Publishing end-to-end tests failed on Jenkins
Details
continuous-integration/jenkins/branch This commit looks good
Details

@barrucadu barrucadu deleted the select_cluster branch Jun 16, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.