Skip to content

Commit

Permalink
HSEARCH-2660 Rename index schema management strategy options to make …
Browse files Browse the repository at this point in the history
…them more consistent with JPA
  • Loading branch information
yrodiere authored and Sanne committed Apr 13, 2017
1 parent d86ed88 commit 8cd0bc3
Show file tree
Hide file tree
Showing 21 changed files with 111 additions and 62 deletions.
2 changes: 1 addition & 1 deletion backends/jgroups/src/test/resources/hibernate.properties
Expand Up @@ -22,5 +22,5 @@ hibernate.cache.region_prefix hibernate.test
hibernate.cache.provider_class org.hibernate.cache.HashtableCacheProvider

hibernate.search.default.elasticsearch.required_index_status YELLOW
hibernate.search.default.elasticsearch.index_schema_management_strategy RECREATE_DELETE
hibernate.search.default.elasticsearch.index_schema_management_strategy drop-and-create-and-drop
hibernate.search.default.elasticsearch.refresh_after_write true
26 changes: 16 additions & 10 deletions documentation/src/main/asciidoc/elasticsearch-integration.asciidoc
Expand Up @@ -177,28 +177,34 @@ Let's see the options for the `index_schema_management_strategy` property:
[options="header"]
|===============
|Value|Definition
|`NONE`|The index, its mappings and the analyzer definitions will not be created, deleted nor altered.
|`none`|The index, its mappings and the analyzer definitions will not be created, deleted nor altered.
Hibernate Search will **not even check** that the index already exists.
|`VALIDATE`|The index, its mappings and analyzer definitions will be checked for conflicts with Hibernate Search's metamodel.
|`validate`|The index, its mappings and analyzer definitions will be checked for conflicts with Hibernate Search's metamodel.
The index, its mappings and analyzer definitions will not be created, deleted nor altered.
|`MERGE`|The index, its mappings and analyzer definitions will be created, existing mappings will be updated if there are no conflicts.
|`update`|The index, its mappings and analyzer definitions will be created, existing mappings will be updated if there are no conflicts.
+
Caution: if analyzer definitions have to be updated, the index will be closed automatically during the update.
|`CREATE`|**The default**: an existing index will not be altered, a missing index will be created along with their mappings and analyzer definitions.
|`RECREATE`|Indexes will be deleted if existing and then created along with their mappings and analyzer definitions.
|`create`|**The default**: an existing index will not be altered, a missing index will be created along with their mappings and analyzer definitions.
|`drop-and-create`|Indexes will be deleted if existing and then created along with their mappings and analyzer definitions.
This will delete all content from the indexes!
|`RECREATE_DELETE`|Similarly to `RECREATE` but will also delete the index at shutdown. Commonly used for tests.
|`drop-and-create-and-drop`|Similar to `drop-and-create` but will also delete the index at shutdown. Commonly used for tests.
|===============
+
[CAUTION]
.Strategies in production environments
====
It is strongly recommended to use either `NONE` or `VALIDATE` in a production environment. `RECREATE` and `RECREATE_DELETE` are obviously unsuitable in this context (unless you want to reindex everything upon every startup), and `MERGE` may leave your mapping half-merged in case of conflict.
It is strongly recommended to use either `none` or `validate` in a production environment.
`drop-and-create` and `drop-and-create-and-drop` are obviously unsuitable in this context
(unless you want to reindex everything upon every startup),
and `update` may leave your mapping half-updated in case of conflict.
To be precise, if your mapping changed in an incompatible way, such as a field having its type changed, merging may be impossible. In this case, the `MERGE` strategy will prevent Hibernate Search from starting, but it may already have successfully merged another index, making a rollback difficult at best.
To be precise, if your mapping changed in an incompatible way, such as a field having its type changed,
updating the mapping may be impossible without manual intervention.
In this case, the `update` strategy will prevent Hibernate Search from starting,
but it may already have successfully updated the mappings for another index, making a rollback difficult at best.
Also, when updating analyzer definitions, Hibernate Search will stop the affected indexes during the update.
This means the `MERGE` strategy should be used with caution when multiple clients use Elasticsearch indexes managed by Hibernate Search:
This means the `update` strategy should be used with caution when multiple clients use Elasticsearch indexes managed by Hibernate Search:
those clients should be synchronized in such a way that while Hibernate Search is starting, no other client tries to use the index.
For these reasons, migrating your mapping should be considered a part of your deployment process and be planned cautiously.
Expand Down Expand Up @@ -257,7 +263,7 @@ When <<elasticsearch-scrolling,scrolling>>, the maximum duration `ScrollableResu
--
Properties prefixed with `hibernate.search.default` can be given globally as shown above and/or be given for specific indexes:

`hibernate.search.someindex.elasticsearch.index_schema_management_strategy MERGE`
`hibernate.search.someindex.elasticsearch.index_schema_management_strategy update`

This excludes properties related to the internal Elasticsearch client, which at the moment is common to every index manager (but this will change in a future version).
Excluded properties are `host`, `username`, `password`, `read_timeout`, `connection_timeout`, `max_total_connection`, `max_total_connection_per_route`, `discovery.enabled`, `discovery.refresh_interval` and `discovery.scheme`.
Expand Down
Expand Up @@ -191,10 +191,11 @@ public static final class Defaults {
/**
* Property for specifying the strategy for maintaining the Elasticsearch index.
* <p>
* The name of one of the {@link IndexSchemaManagementStrategy} constants is expected, e.g. MERGE.
* The external name of one of the {@link IndexSchemaManagementStrategy} constants is expected, e.g. 'update'
* (the external names can be retrieved programmatically using {@link IndexSchemaManagementStrategy#getExternalName}).
* <p>
* Can be given globally (e.g. {@code hibernate.search.default.elasticsearch.index_schema_management_strategy=MERGE}) or
* for specific indexes (e.g. {@code hibernate.search.someindex.elasticsearch.index_schema_management_strategy=RECREATE}).
* Can be given globally (e.g. {@code hibernate.search.default.elasticsearch.index_schema_management_strategy=update}) or
* for specific indexes (e.g. {@code hibernate.search.someindex.elasticsearch.index_schema_management_strategy=drop-and-create}).
*/
public static final String INDEX_SCHEMA_MANAGEMENT_STRATEGY = "elasticsearch.index_schema_management_strategy";

Expand Down
Expand Up @@ -6,6 +6,10 @@
*/
package org.hibernate.search.elasticsearch.cfg;

import java.util.Collections;
import java.util.HashMap;
import java.util.Map;

/**
* Strategy for creating/deleting the indexes in Elasticsearch upon session factory initialization and shut-down.
*
Expand All @@ -18,7 +22,7 @@ public enum IndexSchemaManagementStrategy {
* <p>Hibernate Search will not even check that the index actually exists.
* <p>The index schema (mapping and analyzer definitions) is not managed by Hibernate Search and is not checked.
*/
NONE,
NONE("none"),

/**
* Upon session factory initialization, existing index mappings will be checked
Expand All @@ -27,20 +31,20 @@ public enum IndexSchemaManagementStrategy {
* or if an analyzer definition is missing or differs in any way.
* <p>This strategy will not bring any change to the mappings or analyzer definitions, nor create or delete any index.
*/
VALIDATE,
VALIDATE("validate"),

/**
* Upon session factory initialization, index mappings and analyzer definitions will be merged with existing ones,
* causing an exception if a mapping to be merged is not compatible with the existing one.
* Upon session factory initialization, index mappings and analyzer definitions will be updated to match expected ones,
* causing an exception if a mapping to be updated is not compatible with the expected one.
* <p>Missing indexes will be created along with their mappings and analyzer definitions.
* Missing mappings and analyzer definitions on existing indexes will be created.
*/
MERGE,
UPDATE("update"),

/**
* Existing indexes will not be altered, missing indexes will be created along with their mappings and analyzer definitions.
*/
CREATE,
CREATE("create"),

/**
* Indexes - and all their contents - will be deleted and newly created (along with their mappings and analyzer definitions)
Expand All @@ -49,11 +53,42 @@ public enum IndexSchemaManagementStrategy {
* <p>Note that whenever a search factory is altered after initialization (i.e. new entities are mapped),
* the index will <strong>not</strong> be deleted again: new mappings will simply be added to the index.
*/
RECREATE,
DROP_AND_CREATE("drop-and-create"),

/**
* The same as {@link #RECREATE}, but indexes - and all their contents - will be deleted upon session factory
* The same as {@link #DROP_AND_CREATE}, but indexes - and all their contents - will be deleted upon session factory
* shut-down as well.
*/
RECREATE_DELETE;
DROP_AND_CREATE_AND_DROP("drop-and-create-and-drop");

private static final Map<String, IndexSchemaManagementStrategy> VALUES_BY_EXTERNAL_NAME;
static {
Map<String, IndexSchemaManagementStrategy> tmpMap = new HashMap<>();
for ( IndexSchemaManagementStrategy strategy : values() ) {
tmpMap.put( strategy.externalName, strategy );
}
VALUES_BY_EXTERNAL_NAME = Collections.unmodifiableMap( tmpMap );
}

public static IndexSchemaManagementStrategy interpretPropertyValue(String propertyValue) {
IndexSchemaManagementStrategy strategy = VALUES_BY_EXTERNAL_NAME.get( propertyValue );
if ( strategy == null ) {
throw new IllegalArgumentException( "Unrecognized property value for an index schema management strategy: '" + propertyValue
+ "'. Please use one of " + VALUES_BY_EXTERNAL_NAME.keySet() );
}
return strategy;
}

private final String externalName;

private IndexSchemaManagementStrategy(String propertyValue) {
this.externalName = propertyValue;
}

/**
* @return the name to use in configuration files.
*/
public String getExternalName() {
return externalName;
}
}
Expand Up @@ -47,6 +47,7 @@
import org.hibernate.search.indexes.spi.IndexNameNormalizer;
import org.hibernate.search.indexes.spi.ReaderProvider;
import org.hibernate.search.spi.WorkerBuildContext;
import org.hibernate.search.util.StringHelper;
import org.hibernate.search.util.configuration.impl.ConfigurationParseHelper;
import org.hibernate.search.util.logging.impl.LoggerFactory;

Expand Down Expand Up @@ -161,8 +162,13 @@ private static String getOverriddenIndexName(String indexName, Properties proper
}

private static IndexSchemaManagementStrategy getIndexManagementStrategy(Properties properties) {
String strategy = properties.getProperty( ElasticsearchEnvironment.INDEX_SCHEMA_MANAGEMENT_STRATEGY );
return strategy != null ? IndexSchemaManagementStrategy.valueOf( strategy ) : ElasticsearchEnvironment.Defaults.INDEX_SCHEMA_MANAGEMENT_STRATEGY;
String propertyValue = properties.getProperty( ElasticsearchEnvironment.INDEX_SCHEMA_MANAGEMENT_STRATEGY );
if ( StringHelper.isNotEmpty( propertyValue ) ) {
return IndexSchemaManagementStrategy.interpretPropertyValue( propertyValue );
}
else {
return ElasticsearchEnvironment.Defaults.INDEX_SCHEMA_MANAGEMENT_STRATEGY;
}
}

private static int getIndexManagementWaitTimeout(Properties properties) {
Expand Down Expand Up @@ -213,7 +219,7 @@ private static boolean getRefreshAfterWrite(Properties properties) {

@Override
public void destroy() {
if ( schemaManagementStrategy == IndexSchemaManagementStrategy.RECREATE_DELETE ) {
if ( schemaManagementStrategy == IndexSchemaManagementStrategy.DROP_AND_CREATE_AND_DROP ) {
elasticsearchService.getSchemaDropper().dropIfExisting( actualIndexName, schemaManagementExecutionOptions );
}

Expand Down Expand Up @@ -280,22 +286,22 @@ private boolean initializeIndex(Set<Class<?>> entityTypesToInitialize) {
schemaCreator.createMappings( indexMetadata, schemaManagementExecutionOptions );
}
break;
case RECREATE:
case RECREATE_DELETE:
case DROP_AND_CREATE:
case DROP_AND_CREATE_AND_DROP:
ElasticsearchSchemaDropper schemaDropper = elasticsearchService.getSchemaDropper();
schemaDropper.dropIfExisting( actualIndexName, schemaManagementExecutionOptions );
schemaCreator.createIndex( indexMetadata, schemaManagementExecutionOptions );
schemaCreator.createMappings( indexMetadata, schemaManagementExecutionOptions );
createdIndex = true;
break;
case MERGE:
case UPDATE:
createdIndex = schemaCreator.createIndexIfAbsent( indexMetadata, schemaManagementExecutionOptions );
if ( createdIndex ) {
schemaCreator.createMappings( indexMetadata, schemaManagementExecutionOptions );
}
else {
ElasticsearchSchemaMigrator schemaMigrator = elasticsearchService.getSchemaMigrator();
schemaMigrator.merge( indexMetadata, schemaManagementExecutionOptions );
schemaMigrator.migrate( indexMetadata, schemaManagementExecutionOptions );
}
break;
case VALIDATE:
Expand Down Expand Up @@ -336,13 +342,13 @@ private void reinitializeIndex(Set<Class<?>> entityTypesToInitialize) {
schemaCreator.createMappings( indexMetadata, schemaManagementExecutionOptions );
}
break;
case RECREATE:
case RECREATE_DELETE:
case DROP_AND_CREATE:
case DROP_AND_CREATE_AND_DROP:
schemaCreator.createMappings( indexMetadata, schemaManagementExecutionOptions );
break;
case MERGE:
case UPDATE:
ElasticsearchSchemaMigrator schemaMigrator = elasticsearchService.getSchemaMigrator();
schemaMigrator.merge( indexMetadata, schemaManagementExecutionOptions );
schemaMigrator.migrate( indexMetadata, schemaManagementExecutionOptions );
break;
case VALIDATE:
ElasticsearchSchemaValidator schemaValidator = elasticsearchService.getSchemaValidator();
Expand Down
Expand Up @@ -198,9 +198,9 @@ BulkRequestFailedException elasticsearchBulkRequestFailed(String request, String
SearchException elasticsearchMappingRetrievalForValidationFailed(@Cause Exception cause);

@Message(id = ES_BACKEND_MESSAGES_START_ID + 35,
value = "Could not merge mappings in index '%1$s'"
value = "Could not update mappings in index '%1$s'"
)
SearchException schemaMergeFailed(String indexName, @Cause Exception cause);
SearchException schemaUpdateFailed(String indexName, @Cause Exception cause);

@Message(id = ES_BACKEND_MESSAGES_START_ID + 36,
value = "Mapping conflict detected for field '%2$s' on entity '%1$s'."
Expand Down
Expand Up @@ -35,7 +35,7 @@ public DefaultElasticsearchSchemaMigrator(ElasticsearchSchemaAccessor schemaAcce
}

@Override
public void merge(IndexMetadata indexMetadata, ExecutionOptions executionOptions) {
public void migrate(IndexMetadata indexMetadata, ExecutionOptions executionOptions) {
String indexName = indexMetadata.getName();
IndexSettings settings = indexMetadata.getSettings();

Expand Down Expand Up @@ -72,7 +72,7 @@ public void merge(IndexMetadata indexMetadata, ExecutionOptions executionOptions
}
}
catch (SearchException e) {
throw LOG.schemaMergeFailed( indexName, e );
throw LOG.schemaUpdateFailed( indexName, e );
}
}

Expand Down
Expand Up @@ -10,23 +10,23 @@
import org.hibernate.search.exception.SearchException;

/**
* An object responsible for merging an existing index with provided metadata.
* An object responsible for updating an existing index to match provided metadata.
*
* @author Yoann Rodiere
*/
public interface ElasticsearchSchemaMigrator {

/**
* Merge metadata with the existing schema: for each mapping, merge the existing mapping
* with the expected one, throwing {@link SearchException} if an incompatible attribute
* is detected.
* Update the existing schema to match the given metadata: for each mapping,
* update the existing mappings and analyzer definitions to match the expected ones,
* throwing {@link SearchException} if an incompatible attribute is detected.
*
* <p>The index is expected to already exist.
*
* @param indexMetadata The expected index metadata.
* @param executionOptions The execution options, giving more context information.
* @throws SearchException If an error occurs.
*/
void merge(IndexMetadata indexMetadata, ExecutionOptions executionOptions);
void migrate(IndexMetadata indexMetadata, ExecutionOptions executionOptions);

}
Expand Up @@ -84,7 +84,7 @@ protected void init(Class<?>... annotatedClasses) {
Map<String, Object> settings = new HashMap<>();
settings.put(
"hibernate.search.default." + ElasticsearchEnvironment.INDEX_SCHEMA_MANAGEMENT_STRATEGY,
strategy.name()
strategy.getExternalName()
);
init( new ImmutableTestConfiguration( settings, annotatedClasses ) );
}
Expand Down
Expand Up @@ -49,7 +49,7 @@
@Category(SkipOnElasticsearch5.class)
public class Elasticsearch2SchemaMigrationIT extends SearchInitializationTestBase {

private static final String MERGE_FAILED_MESSAGE_ID = "HSEARCH400035";
private static final String UPDATE_FAILED_MESSAGE_ID = "HSEARCH400035";
private static final String MAPPING_CREATION_FAILED_MESSAGE_ID = "HSEARCH400020";
private static final String ELASTICSEARCH_REQUEST_FAILED_MESSAGE_ID = "HSEARCH400007";

Expand All @@ -64,7 +64,7 @@ protected void init(Class<?>... annotatedClasses) {
Map<String, Object> settings = new HashMap<>();
settings.put(
"hibernate.search.default." + ElasticsearchEnvironment.INDEX_SCHEMA_MANAGEMENT_STRATEGY,
IndexSchemaManagementStrategy.MERGE.name()
IndexSchemaManagementStrategy.UPDATE.getExternalName()
);
init( new ImmutableTestConfiguration( settings, annotatedClasses ) );
}
Expand Down Expand Up @@ -412,7 +412,7 @@ public void property_attribute_invalid() throws Exception {

thrown.expect(
isException( SearchException.class )
.withMessage( MERGE_FAILED_MESSAGE_ID )
.withMessage( UPDATE_FAILED_MESSAGE_ID )
.causedBy( SearchException.class )
.withMessage( MAPPING_CREATION_FAILED_MESSAGE_ID )
.causedBy( SearchException.class )
Expand Down Expand Up @@ -446,7 +446,7 @@ public void property_attribute_invalid_conflictingAnalyzer() throws Exception {

thrown.expect(
isException( SearchException.class )
.withMessage( MERGE_FAILED_MESSAGE_ID )
.withMessage( UPDATE_FAILED_MESSAGE_ID )
.causedBy( SearchException.class )
.withMessage( MAPPING_CREATION_FAILED_MESSAGE_ID )
.causedBy( SearchException.class )
Expand Down
Expand Up @@ -66,7 +66,7 @@ protected void init(Class<?>... annotatedClasses) {
Map<String, Object> settings = new HashMap<>();
settings.put(
"hibernate.search.default." + ElasticsearchEnvironment.INDEX_SCHEMA_MANAGEMENT_STRATEGY,
IndexSchemaManagementStrategy.VALIDATE.name()
IndexSchemaManagementStrategy.VALIDATE.getExternalName()
);
init( new ImmutableTestConfiguration( settings, annotatedClasses ) );
}
Expand Down
Expand Up @@ -85,7 +85,7 @@ protected void init(Class<?>... annotatedClasses) {
Map<String, Object> settings = new HashMap<>();
settings.put(
"hibernate.search.default." + ElasticsearchEnvironment.INDEX_SCHEMA_MANAGEMENT_STRATEGY,
strategy.name()
strategy.getExternalName()
);
init( new ImmutableTestConfiguration( settings, annotatedClasses ) );
}
Expand Down

0 comments on commit 8cd0bc3

Please sign in to comment.