ElasticStore Can't be enable by error on rebuildElasticIndex.php with before ES 6.4

[MPThLee]

Setup and configuration

• SMW version: 3.1.0
• MW version: 1.31.5
• PHP version: 7.3.11
• DB system (MySQL, Blazegraph, etc.) and version: MariaDB v15.1, ElasticSearch v5.6.16, Redis 5.0.6

Issue

I'm using CirrusSearch and Elastica before, and moved to Semantic Mediawiki on existing wiki. But ElasticStore is unable to use due to error durring rebuildElasticIndex.php

Stack trace

Produces a stack trace and/or outputs:

Settings and mappings ...
   ... updating settings and mappings ...
   ... *data* index ...
       ... closing[dba20cd88ad809a585deefd3] [no req]   Elasticsearch\Common\Exceptions\BadRequest400Exception from line 610 of /srv/http/wiki/w/extensions/Elastica/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Can't update [index.number_of_replicas] on closed indices [[smw-data-mediawiki_db-v2/kkg_RHzKT1-3vwcku7OkfA]] - can leave index in an unopenable state"}],"type":"illegal_argument_exception","reason":"Can't update [index.number_of_replicas] on closed indices [[smw-data-mediawiki_db-v2/kkg_RHzKT1-3vwcku7OkfA]] - can leave index in an unopenable state"},"status":400}
Backtrace:
#0 /srv/http/wiki/w/extensions/Elastica/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php(273): Elasticsearch\Connections\Connection->process4xxError(array, array, array)
#1 /srv/http/wiki/w/vendor/react/promise/src/FulfilledPromise.php(25): Elasticsearch\Connections\Connection->Elasticsearch\Connections\{closure}(array)
#2 /srv/http/wiki/w/vendor/guzzlehttp/ringphp/src/Future/CompletedFutureValue.php(55): React\Promise\FulfilledPromise->then(Closure, NULL, NULL)
#3 /srv/http/wiki/w/vendor/guzzlehttp/ringphp/src/Core.php(341): GuzzleHttp\Ring\Future\CompletedFutureValue->then(Closure, NULL, NULL)
#4 /srv/http/wiki/w/extensions/Elastica/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php(294): GuzzleHttp\Ring\Core::proxy(GuzzleHttp\Ring\Future\CompletedFutureArray, Closure)
#5 /srv/http/wiki/w/extensions/Elastica/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php(171): Elasticsearch\Connections\Connection->Elasticsearch\Connections\{closure}(array, Elasticsearch\Connections\Connection, Elasticsearch\Transport, array)
#6 /srv/http/wiki/w/extensions/Elastica/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Transport.php(101): Elasticsearch\Connections\Connection->performRequest(string, string, array, string, array, Elasticsearch\Transport)
#7 /srv/http/wiki/w/extensions/Elastica/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Namespaces/AbstractNamespace.php(72): Elasticsearch\Transport->performRequest(string, string, array, array, array)
#8 /srv/http/wiki/w/extensions/Elastica/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Namespaces/IndicesNamespace.php(209): Elasticsearch\Namespaces\AbstractNamespace->performRequest(Elasticsearch\Endpoints\Indices\Settings\Put)
#9 /srv/http/wiki/w/extensions/SemanticMediaWiki/src/Elastic/Connection/Client.php(406): Elasticsearch\Namespaces\IndicesNamespace->putSettings(array)
#10 /srv/http/wiki/w/extensions/SemanticMediaWiki/src/Elastic/Indexer/Rebuilder.php(389): SMW\Elastic\Connection\Client->putSettings(array)
#11 /srv/http/wiki/w/extensions/SemanticMediaWiki/src/Elastic/Indexer/Rebuilder.php(214): SMW\Elastic\Indexer\Rebuilder->setDefaultByType(string)
#12 /srv/http/wiki/w/extensions/SemanticMediaWiki/maintenance/rebuildElasticIndex.php(222): SMW\Elastic\Indexer\Rebuilder->setDefaults()
#13 /srv/http/wiki/w/extensions/SemanticMediaWiki/maintenance/rebuildElasticIndex.php(151): SMW\Maintenance\RebuildElasticIndex->otherActivities()
#14 /srv/http/wiki/w/maintenance/doMaintenance.php(94): SMW\Maintenance\RebuildElasticIndex->execute()
#15 /srv/http/wiki/w/extensions/SemanticMediaWiki/maintenance/rebuildElasticIndex.php(437): require_once(string)
#16 {main}

Steps to reproduce

Since it's administrative issue, Can not produce with sandbox

[JeroenDeDauw]

Does Elastic work without SMW enabled?

[MPThLee]

It's work with CirrusSearch. I just fixed this issue by dropping all nodes :(

[mwjames]

On 10/25/19, MPThLee ***@***.***> wrote: ### Setup and configuration - SMW version: 3.1.0 - MW version: 1.31.5 - PHP version: 7.3.11 - DB system (MySQL, Blazegraph, etc.) and version: MariaDB v15.1, ElasticSearch v5.6.16, Redis 5.0.6 ### Issue _Please describe the issue as best as you can and also fill in the following two sections if applicable._ I'm using CirrusSearch and Elastica before, and moved to Semantic Mediawiki on existing wiki. But ElasticStore is unable to use due to error durring `rebuildElasticIndex.php` ### Stack trace Produces a [stack trace](https://www.semantic-mediawiki.org/wiki/Help:Identifying_bugs) and/or outputs: ``` Settings and mappings ... ... updating settings and mappings ... ... *data* index ... ... closing[dba20cd88ad809a585deefd3] [no req] Elasticsearch\Common\Exceptions\BadRequest400Exception from line 610 of /srv/http/wiki/w/extensions/Elastica/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Can't update [index.number_of_replicas] on closed indices [[smw-data-mediawiki_db-v2/kkg_RHzKT1-3vwcku7OkfA]] - can leave index in an unopenable state"}],"type":"illegal_argument_exception","reason":"Can't update [index.number_of_replicas] on closed indices [[smw-data-mediawiki_db-v2/kkg_RHzKT1-3vwcku7OkfA]] - can leave index in an unopenable state"},"status":400} Backtrace: #0 /srv/http/wiki/w/extensions/Elastica/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php(273): Elasticsearch\Connections\Connection->process4xxError(array, array, array) #1 /srv/http/wiki/w/vendor/react/promise/src/FulfilledPromise.php(25): Elasticsearch\Connections\Connection->Elasticsearch\Connections\{closure}(array) #2 /srv/http/wiki/w/vendor/guzzlehttp/ringphp/src/Future/CompletedFutureValue.php(55): React\Promise\FulfilledPromise->then(Closure, NULL, NULL) #3 /srv/http/wiki/w/vendor/guzzlehttp/ringphp/src/Core.php(341): GuzzleHttp\Ring\Future\CompletedFutureValue->then(Closure, NULL, NULL) #4 /srv/http/wiki/w/extensions/Elastica/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php(294): GuzzleHttp\Ring\Core::proxy(GuzzleHttp\Ring\Future\CompletedFutureArray, Closure) #5 /srv/http/wiki/w/extensions/Elastica/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php(171): Elasticsearch\Connections\Connection->Elasticsearch\Connections\{closure}(array, Elasticsearch\Connections\Connection, Elasticsearch\Transport, array) #6 /srv/http/wiki/w/extensions/Elastica/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Transport.php(101): Elasticsearch\Connections\Connection->performRequest(string, string, array, string, array, Elasticsearch\Transport) #7 /srv/http/wiki/w/extensions/Elastica/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Namespaces/AbstractNamespace.php(72): Elasticsearch\Transport->performRequest(string, string, array, array, array) #8 /srv/http/wiki/w/extensions/Elastica/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Namespaces/IndicesNamespace.php(209): Elasticsearch\Namespaces\AbstractNamespace->performRequest(Elasticsearch\Endpoints\Indices\Settings\Put) #9 /srv/http/wiki/w/extensions/SemanticMediaWiki/src/Elastic/Connection/Client.php(406): Elasticsearch\Namespaces\IndicesNamespace->putSettings(array) #10 /srv/http/wiki/w/extensions/SemanticMediaWiki/src/Elastic/Indexer/Rebuilder.php(389): SMW\Elastic\Connection\Client->putSettings(array) #11 /srv/http/wiki/w/extensions/SemanticMediaWiki/src/Elastic/Indexer/Rebuilder.php(214): SMW\Elastic\Indexer\Rebuilder->setDefaultByType(string) #12 /srv/http/wiki/w/extensions/SemanticMediaWiki/maintenance/rebuildElasticIndex.php(222): SMW\Elastic\Indexer\Rebuilder->setDefaults() #13 /srv/http/wiki/w/extensions/SemanticMediaWiki/maintenance/rebuildElasticIndex.php(151): SMW\Maintenance\RebuildElasticIndex->otherActivities() #14 /srv/http/wiki/w/maintenance/doMaintenance.php(94): SMW\Maintenance\RebuildElasticIndex->execute() #15 /srv/http/wiki/w/extensions/SemanticMediaWiki/maintenance/rebuildElasticIndex.php(437): require_once(string) #16 {main} ``` ### Steps to reproduce Since it's administrative issue, Can not produe with sandbox -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: #4341 ElasticSearch v5.6.16, Redis 5.0.6 ... closing[dba20cd88ad809a585deefd3] [no req] Elasticsearch\Common\Exceptions\BadRequest400Exception from line 610 of /srv/http/wiki/w/extensions/Elastica/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Can't update [index.number_of_replicas] on closed indices

The issue is with your version of Elasticsearch. When rebuilding indices we rely on some config variables to be set before and after the operation [0]. As it turns out Elasticsearch has a bug [1, 2] which prevents us from using open/close statements on ES 5.6 which is expected to be fixed with ES 6.4. I don't really remember which settings could cause above condition but according to [1] `number_of_replicas` is one of them. So, in case you need to run ES 5.6 and are unable to upgrade, adding the following lines as a temporary fix might help you. ``` @@ -377,10 +377,17 @@ class Rebuilder { // Cannot be altered by a simple settings update and requires a complete // rebuild unset( $indexDef['settings']['number_of_shards'] ); + // ES 5.6 may cause a "Can't update [index.number_of_replicas] on closed + // indices" see elastic/elasticsearch#22993 and + // should be fixed with ES 6.4. + if ( version_compare( $this->client->getVersion(), '6.4.0', '<' ) ) { + unset( $indexDef['settings']['number_of_replicas'] ); + } ``` [0] https://github.com/SemanticMediaWiki/SemanticMediaWiki/blob/master/src/Elastic/Indexer/Rebuilder.php#L368-L403 [1] elastic/elasticsearch#22993 [2] https://github.com/SemanticMediaWiki/SemanticMediaWiki/blob/master/tests/phpunit/Integration/Maintenance/RebuildElasticIndexTest.php#L48-L56

[MPThLee]

For fix this issue in my end, just upgraded to MW 1.33 (and ES 6.8).

I think ElasticStore can be enabled with ES 5.6, as Elasticsearch: Recommended 6.1+, Tested with 5.6.6 on Help:ElasticStore.

I will open this issue. If this issue planned to won't fix, Please close this issue.

Edit: note that CirrusSearch with MW 1.31 and 1.32 still require ES 5.x.