Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support two Elasticsearch clusters #602

Closed
wants to merge 14 commits into from
Closed

Conversation

AndrewKostka
Copy link
Contributor

No description provided.

@rosalieper
Copy link
Contributor

Is this still a WIP?


protected function getQueryParams(): string {
return parent::getQueryParams() . '&cluster=' . $this->cluster();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AndrewKostka I'm currently looking into moving this forward and there's a high-level question I'd like to ask here: what's the reason you decided to have api be in control over which clusters the maintenance scripts are invoked for, when instead this could also be handled by our wrappers in MediaWiki themselves? E.g. by default it seems most CirrusSearch maintenance scripts already work on all known clusters by default.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the reason you decided to have api be in control over which clusters the maintenance scripts are invoked for, when instead this could also be handled by our wrappers in MediaWiki themselves?

Our current process for running maintenance scripts requires us to fire off a job in the API rather than interfacing directly with MediaWiki. If we need to run UpdateSearchIndexConfig on a new cluster or force an index update using ForceSearchIndex, these actions must/should only be run on a specific cluster. Hence, why we need to specify which cluster should be the target for a specific maintenance script. If we were to directly run maintenance scripts from MediaWiki, thus dropping the cURL dependency between the API and MediaWiki, we could choose to only update the wrappers in MediaWiki.

E.g. by default it seems most CirrusSearch maintenance scripts already work on all known clusters by default.

AFAIK the CirrusSearch maintenance scripts do not work on all clusters by default.

Maintenance tasks (update mapping, reindex, etc.) must be performed per
CirrusSearch cluster. CirrusSearch maintenance scripts all take a --cluster
option to specify the CirrusSearch cluster to operate on. When not specified
the default search cluster is used. Informational maintenance scripts that
can not change any state may choose to emit for all clusters when --cluster
is not provided.

from https://github.com/wikimedia/mediawiki-extensions-CirrusSearch/blob/REL1_39/docs/multi_cluster.txt#L56C1-L61C17

Copy link
Contributor

@m90 m90 Jul 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK the CirrusSearch maintenance scripts do not work on all clusters by default.

Interesting. The script level documentation / comments say differently, but it's probably best to be explicit in that case.

https://github.com/wbstack/mediawiki/blob/3a27dc649d171ba8fea0006563b3a504d1eea0ae/dist/extensions/CirrusSearch/maintenance/ForceSearchIndex.php#L75-L81

@m90
Copy link
Contributor

m90 commented Aug 7, 2023

Closing this in favor of #624

@m90 m90 closed this Aug 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants