From cafdc9d639a91b4f6aa02e9c093af9fc8e9f7803 Mon Sep 17 00:00:00 2001 From: lcawl Date: Thu, 27 Sep 2018 12:13:16 -0700 Subject: [PATCH] [DOCS] Synchronize location of Breaking Changes (#33588) --- docs/reference/index.asciidoc | 4 ++-- docs/reference/migration/migrate_6_0.asciidoc | 3 +++ .../migrate_6_0/aggregations.asciidoc | 6 +++++- .../migration/migrate_6_0/analysis.asciidoc | 5 ++++- .../migration/migrate_6_0/cat.asciidoc | 2 ++ .../migration/migrate_6_0/clients.asciidoc | 2 ++ .../migration/migrate_6_0/cluster.asciidoc | 2 ++ .../migration/migrate_6_0/docs.asciidoc | 6 +++++- .../migration/migrate_6_0/geo.asciidoc | 3 +++ .../migration/migrate_6_0/indices.asciidoc | 12 ++++++++++- .../migration/migrate_6_0/ingest.asciidoc | 3 +++ .../migration/migrate_6_0/java.asciidoc | 10 ++++++++++ .../migration/migrate_6_0/mappings.asciidoc | 6 ++++++ .../migration/migrate_6_0/packaging.asciidoc | 13 ++++++++++-- .../migration/migrate_6_0/percolator.asciidoc | 3 +++ .../migration/migrate_6_0/plugins.asciidoc | 9 +++++++++ .../migration/migrate_6_0/reindex.asciidoc | 2 ++ .../migration/migrate_6_0/rest.asciidoc | 15 ++++++++++++++ .../migration/migrate_6_0/scripting.asciidoc | 9 +++++++++ .../migration/migrate_6_0/search.asciidoc | 20 +++++++++++++++++++ .../migration/migrate_6_0/settings.asciidoc | 11 ++++++++++ .../migration/migrate_6_0/stats.asciidoc | 3 +++ docs/reference/migration/migrate_6_1.asciidoc | 3 +++ docs/reference/migration/migrate_6_2.asciidoc | 6 ++++++ docs/reference/migration/migrate_6_3.asciidoc | 14 +++++++++++++ docs/reference/migration/migrate_6_4.asciidoc | 14 +++++++++++++ .../migrate_7_0/snapshotstats.asciidoc | 2 ++ docs/reference/release-notes/6.2.asciidoc | 7 +++---- docs/reference/release-notes/6.4.asciidoc | 2 +- 29 files changed, 184 insertions(+), 13 deletions(-) diff --git a/docs/reference/index.asciidoc b/docs/reference/index.asciidoc index 8d5e4300af686..52fb022f0dc0a 100644 --- a/docs/reference/index.asciidoc +++ b/docs/reference/index.asciidoc @@ -31,8 +31,6 @@ include::setup/bootstrap-checks-xes.asciidoc[] :edit_url: include::upgrade.asciidoc[] -include::migration/index.asciidoc[] - include::api-conventions.asciidoc[] include::docs.asciidoc[] @@ -78,6 +76,8 @@ include::glossary.asciidoc[] include::release-notes/highlights.asciidoc[] +include::migration/index.asciidoc[] + include::release-notes.asciidoc[] include::redirects.asciidoc[] diff --git a/docs/reference/migration/migrate_6_0.asciidoc b/docs/reference/migration/migrate_6_0.asciidoc index f054c11692e56..e74b4ac7ed4fd 100644 --- a/docs/reference/migration/migrate_6_0.asciidoc +++ b/docs/reference/migration/migrate_6_0.asciidoc @@ -1,5 +1,8 @@ [[breaking-changes-6.0]] == Breaking changes in 6.0 +++++ +6.0 +++++ This section discusses the changes that you need to be aware of when migrating your application to Elasticsearch 6.0. diff --git a/docs/reference/migration/migrate_6_0/aggregations.asciidoc b/docs/reference/migration/migrate_6_0/aggregations.asciidoc index e4ee8b8181558..a6413cc5b421f 100644 --- a/docs/reference/migration/migrate_6_0/aggregations.asciidoc +++ b/docs/reference/migration/migrate_6_0/aggregations.asciidoc @@ -1,6 +1,8 @@ +[float] [[breaking_60_aggregations_changes]] === Aggregations changes +[float] ==== Deprecated `pattern` element of include/exclude for terms aggregations has been removed The `include` and `exclude` options of `terms` aggregations used to accept a @@ -50,6 +52,7 @@ POST /twitter/_search?size=0 // CONSOLE // TEST[setup:twitter] +[float] ==== Numeric `to` and `from` parameters in `date_range` aggregation are interpreted according to `format` now Numeric `to` and `from` parameters in `date_range` aggregations used to always be interpreted as `epoch_millis`, @@ -58,13 +61,14 @@ Now we interpret these parameters according to the `format` of the target field. If the `format` in the mappings is not compatible with the numeric input value, a compatible `format` (e.g. `epoch_millis`, `epoch_second`) must be specified in the `date_range` aggregation, otherwise an error is thrown. - +[float] ==== `global_ordinals_hash` and `global_ordinals_low_cardinality` are deprecated in the `terms` aggregation The execution hints `global_ordinals_hash` and `global_ordinals_low_cardinality` are deprecated and should be replaced by `global_ordinals` which now internally choose whether it should remap global ordinals to dense ordinals or directly use the segment ordinals. +[float] ==== `missing` is deprecated in the `composite` aggregation The `missing` option of the `composite` aggregation is deprecated, `missing_bucket` diff --git a/docs/reference/migration/migrate_6_0/analysis.asciidoc b/docs/reference/migration/migrate_6_0/analysis.asciidoc index 4316a342eba86..da084b93e2bc2 100644 --- a/docs/reference/migration/migrate_6_0/analysis.asciidoc +++ b/docs/reference/migration/migrate_6_0/analysis.asciidoc @@ -1,6 +1,8 @@ +[float] [[breaking_60_analysis_changes]] === Analysis changes +[float] ==== Synonym Token Filter In 6.0, Synonym Token Filter tokenizes synonyms with whatever @@ -11,6 +13,7 @@ and will be ignored when used in new indices. These parameters will continue to function as before when used in indices created in 5.x. +[float] ==== Limiting the length of an analyzed text during highlighting Highlighting a text that was indexed without offsets or term vectors, @@ -20,4 +23,4 @@ To protect against this, the maximum number of characters that to be analyzed wi limited to 1000000 in the next major Elastic version. For this version, by default the limit is not set. A deprecation warning will be issued when an analyzed text exceeds 1000000. The limit can be set for a particular index with the index setting -`index.highlight.max_analyzed_offset`. \ No newline at end of file +`index.highlight.max_analyzed_offset`. diff --git a/docs/reference/migration/migrate_6_0/cat.asciidoc b/docs/reference/migration/migrate_6_0/cat.asciidoc index 013c0705991cb..13e8c9f00fb2e 100644 --- a/docs/reference/migration/migrate_6_0/cat.asciidoc +++ b/docs/reference/migration/migrate_6_0/cat.asciidoc @@ -1,6 +1,8 @@ +[float] [[breaking_60_cat_changes]] === Cat API changes +[float] ==== Unbounded queue size in cat thread pool Previously if a queue size backing a thread pool was unbounded, the cat thread pool API would output an empty string in diff --git a/docs/reference/migration/migrate_6_0/clients.asciidoc b/docs/reference/migration/migrate_6_0/clients.asciidoc index 55d1675e5dda0..d3b9980dcfc86 100644 --- a/docs/reference/migration/migrate_6_0/clients.asciidoc +++ b/docs/reference/migration/migrate_6_0/clients.asciidoc @@ -1,6 +1,8 @@ +[float] [[breaking_60_clients_changes]] === Clients changes +[float] ==== Java High Level REST Client Starting from version 5.6.0 a new Java client has been released: the Java High Level REST Client. diff --git a/docs/reference/migration/migrate_6_0/cluster.asciidoc b/docs/reference/migration/migrate_6_0/cluster.asciidoc index bd070d8d1f483..27204a071a24d 100644 --- a/docs/reference/migration/migrate_6_0/cluster.asciidoc +++ b/docs/reference/migration/migrate_6_0/cluster.asciidoc @@ -1,6 +1,8 @@ +[float] [[breaking_60_cluster_changes]] === Cluster changes +[float] ==== Cluster name no longer allowed in path.data Previously the cluster name could be used in the `path.data` setting with a diff --git a/docs/reference/migration/migrate_6_0/docs.asciidoc b/docs/reference/migration/migrate_6_0/docs.asciidoc index a7e34090e9e41..7841dbe8f835e 100644 --- a/docs/reference/migration/migrate_6_0/docs.asciidoc +++ b/docs/reference/migration/migrate_6_0/docs.asciidoc @@ -1,21 +1,25 @@ +[float] [[breaking_60_docs_changes]] === Document API changes +[float] ==== version type `force` removed Document modification operations may no longer specify the `version_type` of `force` to override any previous version checks. +[float] ==== <> no longer support versions Adding a `version` to an upsert request is no longer supported. +[float] ==== `created` field removed in the Index API The `created` field has been removed in the Index API as in the `index` and `create` bulk operations. `operation` field should be used instead. - +[float] ==== `found` field removed in the Delete API The `found` field has been removed in the Delete API as in the `delete` bulk diff --git a/docs/reference/migration/migrate_6_0/geo.asciidoc b/docs/reference/migration/migrate_6_0/geo.asciidoc index 8b13059b6b923..44d32f2d1ba9a 100644 --- a/docs/reference/migration/migrate_6_0/geo.asciidoc +++ b/docs/reference/migration/migrate_6_0/geo.asciidoc @@ -1,12 +1,15 @@ +[float] [[breaking_60_geo_changes]] === Geo changes +[float] ==== Deprecated `ShapeBuilders` helper class The `ShapeBuilders` class containing static methods to create shape specific builders (e.g., `ShapeBuilders.newPoint`) has been deprecated. One must now use the builder's constructor instead (e.g., `new MultiPointBuilder()). +[float] ==== Deprecated `ShapeBuilder.parse` static method The `ShapeBuilder.parse` method has been deprecated. Use `ShapeParser.parse` instead. \ No newline at end of file diff --git a/docs/reference/migration/migrate_6_0/indices.asciidoc b/docs/reference/migration/migrate_6_0/indices.asciidoc index 76f2e9d75a81c..9f43815e4e30c 100644 --- a/docs/reference/migration/migrate_6_0/indices.asciidoc +++ b/docs/reference/migration/migrate_6_0/indices.asciidoc @@ -1,6 +1,8 @@ +[float] [[breaking_60_indices_changes]] === Indices changes +[float] ==== Index templates use `index_patterns` instead of `template` Previously templates expressed the indices that they should match using a glob @@ -28,7 +30,7 @@ PUT _template/template_2 -------------------------------------------------- // CONSOLE - +[float] ==== Shadow Replicas have been removed Shadow replicas don't see enough usage, and have been removed. This includes the @@ -38,6 +40,7 @@ following settings: - `index.shadow_replicas` - `node.add_lock_id_to_custom_path` +[float] ==== Open/Close index API allows wildcard expressions that match no indices by default The default value of the `allow_no_indices` option for the Open/Close index API @@ -45,35 +48,41 @@ has been changed from `false` to `true` so it is aligned with the behaviour of t Delete index API. As a result, Open/Close index API don't return an error by default when a provided wildcard expression doesn't match any closed/open index. +[float] ==== Delete a document Delete a document from non-existing index has been modified to not create the index. However if an external versioning is used the index will be created and the document will be marked for deletion. +[float] ==== Indices aliases api resolves indices expressions only against indices The index parameter in the update-aliases, put-alias, and delete-alias APIs no longer accepts alias names. Instead, it accepts only index names (or wildcards which will expand to matching indices). +[float] ==== Delete index api resolves indices expressions only against indices The index parameter in the delete index API no longer accepts alias names. Instead, it accepts only index names (or wildcards which will expand to matching indices). +[float] ==== Support for `+` has been removed in index expressions Omitting the `+` has the same effect as specifying it, hence support for `+` has been removed in index expressions. +[float] ==== Translog retention Translog files are now kept for up to 12 hours (by default), with a maximum size of `512mb` (default), and are no longer deleted on `flush`. This is to increase the chance of doing an operation based recovery when bringing up replicas up to speed. +[float] ==== Limit to the difference between max_size and min_size in NGramTokenFilter and NGramTokenizer To safeguard against creating too many index terms, the difference between @@ -82,6 +91,7 @@ limited to 1. This default limit can be changed with the index setting index.max_ngram_diff. Note that if the limit is exceeded a deprecation warning is logged. +[float] ==== Limit to the difference between max_size and min_size in ShingleTokenFilter To safeguard against creating too many tokens, the difference between diff --git a/docs/reference/migration/migrate_6_0/ingest.asciidoc b/docs/reference/migration/migrate_6_0/ingest.asciidoc index eba1a19ec8788..e02d7eeac0cc5 100644 --- a/docs/reference/migration/migrate_6_0/ingest.asciidoc +++ b/docs/reference/migration/migrate_6_0/ingest.asciidoc @@ -1,10 +1,13 @@ +[float] [[breaking_60_ingest_changes]] === Ingest changes +[float] ==== Timestamp meta-data field type has changed The type of the "timestamp" meta-data field has changed from `java.lang.String` to `java.util.Date`. +[float] ==== The format of the string-formatted ingest.timestamp field has changed Previously, since Elasticsearch 5.4.0, you needed to use `ingest.new_date_format` to have the diff --git a/docs/reference/migration/migrate_6_0/java.asciidoc b/docs/reference/migration/migrate_6_0/java.asciidoc index 3d9ccdf478c53..07a7fbcb7319b 100644 --- a/docs/reference/migration/migrate_6_0/java.asciidoc +++ b/docs/reference/migration/migrate_6_0/java.asciidoc @@ -1,6 +1,8 @@ +[float] [[breaking_60_java_changes]] === Java API changes +[float] ==== `setSource` methods require XContentType Previously the `setSource` methods and other methods that accepted byte/string representations of @@ -8,18 +10,21 @@ an object source did not require the XContentType to be specified. The auto-dete type is no longer used, so these methods now require the XContentType as an additional argument when providing the source in bytes or as a string. +[float] ==== `DeleteByQueryRequest` requires an explicitly set query In previous versions of Elasticsearch, delete by query requests without an explicit query were accepted, match_all was used as the default query and all documents were deleted as a result. From version 6.0.0, a `DeleteByQueryRequest` requires an explicit query be set. +[float] ==== `InternalStats` and `Stats` getCountAsString() method removed The `count` value in the stats aggregation represents a doc count that shouldn't require a formatted version. This method was deprecated in 5.4 in favour of just using `String.valueOf(getCount())` if needed +[float] ==== `ActionRequestBuilder#execute` returns `ActionFuture` rather than `ListenableActionFuture` When sending a request through the request builders e.g. client.prepareSearch().execute(), it used to @@ -28,6 +33,7 @@ be possible to call `addListener` against the returned `ListenableActionFuture`. it is not possible to associate the future with listeners. The `execute` method that accept a listener as an argument can be used instead. +[float] ==== `Terms.Order` and `Histogram.Order` classes replace by `BucketOrder` The `terms`, `histogram`, and `date_histogram` aggregation code has been refactored to use common @@ -36,17 +42,20 @@ code for ordering buckets. The `BucketOrder` class must be used instead of `Term accessing internal order instances, e.g. `BucketOrder.count(boolean)` and `BucketOrder.aggregation(String, boolean)`. Use `BucketOrder.key(boolean)` to order the `terms` aggregation buckets by `_term`. +[float] ==== `getTookInMillis()` removed in `BulkResponse`, `SearchResponse` and `TermVectorsResponse` In `BulkResponse`, `SearchResponse` and `TermVectorsResponse` `getTookInMiilis()` method has been removed in favor of `getTook` method. `getTookInMiilis()` is easily replaced by `getTook().getMillis()`. +[float] ==== `GetField` and `SearchHitField` replaced by `DocumentField` As `GetField` and `SearchHitField` have the same members, they have been unified into `DocumentField`. +[float] ==== Some Aggregation classes have moved packages The classes for the variants of the range aggregation (geo_distance, date and ip) were moved into the `org.elasticsearch.search.aggregations.bucket.range` @@ -57,6 +66,7 @@ The `org.elasticsearch.search.aggregations.bucket.terms.support` package was rem The filter aggregation classes were moved to `org.elasticsearch.search.aggregations.bucket.filter` +[float] ==== Constructor for `PercentileRanksAggregationBuilder` has changed It is now required to include the desired ranks as a non-null, non-empty array of doubles to the builder's constructor, diff --git a/docs/reference/migration/migrate_6_0/mappings.asciidoc b/docs/reference/migration/migrate_6_0/mappings.asciidoc index 76b2f407aa811..96b4b30dc8dea 100644 --- a/docs/reference/migration/migrate_6_0/mappings.asciidoc +++ b/docs/reference/migration/migrate_6_0/mappings.asciidoc @@ -1,6 +1,8 @@ +[float] [[breaking_60_mappings_changes]] === Mapping changes +[float] ==== Coercion of boolean fields Previously, Elasticsearch recognized the strings `true`, `false`, `on`, `off`, `yes`, `no`, `0`, `1` as booleans. Elasticsearch 6.0 @@ -9,6 +11,7 @@ series the previous coercion rules will continue to work on pre-6.0 indices. Thi mappings immediately. However, it is not possible to create new indices from existing index templates that violate the strict `boolean` coercion rules. +[float] ==== The `_all` meta field is now disabled by default On new mappings, the `_all` meta field that contains a copy of the text from @@ -18,18 +21,21 @@ check if `_all` is enabled/disabled and switch to executing the query across all fields if `_all` is disabled. `_all` can no longer be configured for indices created with Elasticsearch version 6.0 or later. +[float] ==== The `include_in_all` mapping parameter is now disallowed Since the ++_all++ field is now disabled by default and cannot be configured for indices created with Elasticsearch 6.0 or later, the `include_in_all` setting is now disallowed for these indices' mappings. +[float] ==== Unrecognized `match_mapping_type` options not silently ignored Previously Elasticsearch would silently ignore any dynamic templates that included a `match_mapping_type` type that was unrecognized. An exception is now thrown on an unrecognized type. +[float] ==== Validation of `locale` on date fields The `locale` option of `date` fields previously allowed almost any string values, diff --git a/docs/reference/migration/migrate_6_0/packaging.asciidoc b/docs/reference/migration/migrate_6_0/packaging.asciidoc index 404c5c8a97f09..45d760527d4be 100644 --- a/docs/reference/migration/migrate_6_0/packaging.asciidoc +++ b/docs/reference/migration/migrate_6_0/packaging.asciidoc @@ -1,6 +1,8 @@ +[float] [[breaking_60_packaging_changes]] === Packaging changes +[float] ==== Configuring custom user and group for package is no longer allowed Previously someone could configure the `$ES_USER` and `$ES_GROUP` variables to @@ -10,15 +12,16 @@ possible, the DEB and RPM packages now exclusively use the user and group should use the tarball distribution instead of the provided RPM and DEB packages. +[float] ==== `path.conf` is no longer a configurable setting Previous versions of Elasticsearch enabled setting `path.conf` as a setting. This was rather convoluted as it meant that you could start Elasticsearch with a config file that specified via `path.conf` that Elasticsearch should use another config file. Instead, to configure a custom -config directory, use the <>. +config directory, use the <>. +[float] ==== `CONF_DIR` is no longer supported Previous versions of Elasticsearch enabled using the `CONF_DIR` environment @@ -28,6 +31,7 @@ files and some scripts (it was used inconsistently). Starting in Elasticsearch `ES_PATH_CONF`, and this new environment variable is consistently used for all configuration files and scripts. +[float] ==== Default path settings are removed Previous versions of Elasticsearch enabled setting `default.path.data` and @@ -55,12 +59,14 @@ path.logs: /var/log/elasticsearch course preserve those). If you do not do this, Elasticsearch will refuse to start. +[float] ==== 32-bit is no longer maintained We previously attempted to ensure that Elasticsearch could be started on 32-bit JVM (although a bootstrap check prevented using a 32-bit JVM in production). We are no longer maintaining this attempt. +[float] ==== `-server` flag no longer filtered in the Windows service Related to the previous change that 32-bit is no longer supported, the default @@ -72,6 +78,7 @@ flag was removed from the installation of the Windows service. This means that if you have a `jvm.options` file that includes this flag you will need to remove it. +[float] ==== `ES_JVM_OPTIONS` is no longer supported The environment variable `ES_JVM_OPTIONS` that enabled a custom location for the @@ -80,6 +87,7 @@ The environment variable `ES_JVM_OPTIONS` that enabled a custom location for the support relocating the configuration files so this change merely aligns the other configuration files with the location of the `jvm.options` file. +[float] ==== `ES_INCLUDE` is no longer supported The environment variable `ES_INCLUDE` could previously be used to establish the @@ -88,6 +96,7 @@ legacy feature could be useful when there were several environment variables useful for configuring JVM options; this functionality had previously been replaced by <>. Therefore, `ES_INCLUDE` has been removed. +[float] ==== Logging configuration Previously Elasticsearch exposed a single system property (`es.logs`) that diff --git a/docs/reference/migration/migrate_6_0/percolator.asciidoc b/docs/reference/migration/migrate_6_0/percolator.asciidoc index e77816fa9062c..dd87ef8630e6f 100644 --- a/docs/reference/migration/migrate_6_0/percolator.asciidoc +++ b/docs/reference/migration/migrate_6_0/percolator.asciidoc @@ -1,10 +1,13 @@ +[float] [[breaking_60_percolator_changes]] === Percolator changes +[float] ==== Deprecated percolator and mpercolate apis have been removed Instead the `percolate` query should be used via either the search or msearch apis. +[float] ==== The percolator.map_unmapped_fields_as_string setting has been deprecated The `percolator.map_unmapped_fields_as_text` setting should be used instead. \ No newline at end of file diff --git a/docs/reference/migration/migrate_6_0/plugins.asciidoc b/docs/reference/migration/migrate_6_0/plugins.asciidoc index e782adbe88fec..62a03ce6d677f 100644 --- a/docs/reference/migration/migrate_6_0/plugins.asciidoc +++ b/docs/reference/migration/migrate_6_0/plugins.asciidoc @@ -1,11 +1,14 @@ +[float] [[breaking_60_plugins_changes]] === Plugins changes +[float] ==== Mapper attachments plugin * The mapper attachments plugin has been deprecated in elasticsearch 5.0 and is now removed. You can use {plugins}/ingest-attachment.html[ingest attachment plugin] instead. +[float] ==== S3 Repository plugin * The bucket an s3 repository is configured with will no longer be created automatically. @@ -44,6 +47,7 @@ Instead, use the new named client settings under `s3.client.CLIENT_NAME.*`. * The `path_style_access` setting has been removed. The repository will use the appropriate path style access based on the configured endpoint. +[float] ==== Azure Repository plugin * The container an azure repository is configured with will no longer be created automatically. @@ -57,11 +61,13 @@ You must set those settings per repository instead. Respectively `account`, `con `location_mode`, `chunk_size` and `compress`. See {plugins}/repository-azure-repository-settings.html#repository-azure-repository-settings[Azure Repository settings]. +[float] ==== GCS Repository plugin * The `service_account` setting has been removed. A service account json credential file must now be specified in the <>. +[float] ==== EC2 Discovery plugin * Specifying ec2 signer type has been removed, including `cloud.aws.signer` and `cloud.aws.ec2.signer`. @@ -71,11 +77,13 @@ and `cloud.aws.ec2.region`. Instead, specify the full endpoint. * All `cloud.aws.*` and `cloud.aws.ec2.*` settings have been removed. Use `discovery.ec2.*` settings instead. +[float] ==== Ignoring hidden folders Previous versions of Elasticsearch would skip hidden files and directories when scanning the plugins folder. This leniency has been removed. +[float] ==== ICU Analysis plugin The icu4j library has been upgraded to 59.1, @@ -83,6 +91,7 @@ Indices created in the previous major version will need to be reindexed in order to return correct (and correctly ordered) results, and to take advantage of new characters. +[float] ==== Plugins should not construct `Environment` instances from `Settings` Previously, plugins could construct an `Environment` instance from `Settings` to diff --git a/docs/reference/migration/migrate_6_0/reindex.asciidoc b/docs/reference/migration/migrate_6_0/reindex.asciidoc index d74a5102d91b7..262f89be4f46f 100644 --- a/docs/reference/migration/migrate_6_0/reindex.asciidoc +++ b/docs/reference/migration/migrate_6_0/reindex.asciidoc @@ -1,6 +1,8 @@ +[float] [[breaking_60_reindex_changes]] === Reindex changes +[float] ==== `size` parameter The `size` parameter can no longer be explicitly set to `-1`. If all documents are required then the `size` parameter should not be set. \ No newline at end of file diff --git a/docs/reference/migration/migrate_6_0/rest.asciidoc b/docs/reference/migration/migrate_6_0/rest.asciidoc index fbf1f0cc96a11..8663c3a990722 100644 --- a/docs/reference/migration/migrate_6_0/rest.asciidoc +++ b/docs/reference/migration/migrate_6_0/rest.asciidoc @@ -1,6 +1,8 @@ +[float] [[breaking_60_rest_changes]] === REST changes +[float] ==== Unquoted JSON In previous versions of Elasticsearch, JSON documents were allowed to contain unquoted field names. @@ -8,11 +10,13 @@ This feature was removed in the 5.x series, but a backwards-compatibility layer system property `elasticsearch.json.allow_unquoted_field_names`. This backwards-compatibility layer has been removed in Elasticsearch 6.0.0. +[float] ==== Duplicate Keys in JSON, CBOR, Yaml and Smile In previous versions of Elasticsearch, documents were allowed to contain duplicate keys. Elasticsearch 6.0.0 enforces that all keys are unique. This applies to all content types: JSON, CBOR, Yaml and Smile. +[float] ==== Content-Type Auto-detection In previous versions of Elasticsearch, having a proper Content-Type for the data in a request was not enforced. @@ -22,15 +26,18 @@ be used when parsing the data. When using the `source` query string parameter, the `source_content_type` parameter must also be specified with the media type of the source. +[float] ==== Boolean API parameters All REST APIs parameters (both request parameters and JSON body) support providing boolean "false" as the value `false` and boolean "true" as the value `true`. All other values will raise an error. +[float] ==== Analyze API changes The deprecated request parameters and plain text in request body has been removed. Define parameters in request body. +[float] ==== Support custom normalizer in Analyze API Analyze API can analyze normalizer and custom normalizer. @@ -38,28 +45,33 @@ In previous versions of Elasticsearch, Analyze API is requiring a `tokenizer` or In Elasticsearch 6.0.0, Analyze API can analyze a text as a keyword field with custom normalizer or if `char_filter`/`filter` is set and `tokenizer`/`analyzer` is not set. +[float] ==== `timestamp` and `ttl` in index requests `timestamp` and `ttl` are not accepted anymore as parameters of index/update requests. +[float] ==== Refresh requests with one or more shard failures return HTTP 500 response instead of 200 Refresh requests that are broadcast to multiple shards that can have one or more shards fail during the request now return a 500 response instead of a 200 response in the event there is at least one failure. +[float] ==== Delete by Query API requires an explicit query In previous versions of Elasticsearch, delete by query requests without an explicit query were accepted, match_all was used as the default query and all documents were deleted as a result. From version 6.0.0, delete by query requests require an explicit query. +[float] ==== DELETE document calls now implicitly create the type Running `DELETE index/type/id` now implicitly creates `type` with a default mapping if it did not exist yet. +[float] ==== Indices information APIs Previously it was possible to execute `GET /_aliases,_mappings` or `GET @@ -72,6 +84,7 @@ mappings for all indices. In order to retrieve only the mapping for an index use: `GET /myindex/_mapping` (or `_alias` for a list of aliases, or `_settings` for the settings). +[float] ==== Requests to existing endpoints with incorrect HTTP verb now return 405 responses Issuing a request to an endpoint that exists, but with an incorrect HTTP verb @@ -103,12 +116,14 @@ $ curl -v -XPOST 'localhost:9200/my_index/_settings' * Connection #0 to host localhost left intact -------------------------------------------- +[float] ==== Disallow using `_cache` and `_cache_key` The `_cache` and `_cache_key` options in queries have been deprecated since version 2.0.0 and have been ignored since then, issuing a deprecation warning. These options have now been completely removed, so using them now will throw an error. +[float] ==== IndexClosedException to return 400 status code An `IndexClosedException` is returned whenever an api that doesn't support diff --git a/docs/reference/migration/migrate_6_0/scripting.asciidoc b/docs/reference/migration/migrate_6_0/scripting.asciidoc index ac24383a89c2d..b8fab0b7d059e 100644 --- a/docs/reference/migration/migrate_6_0/scripting.asciidoc +++ b/docs/reference/migration/migrate_6_0/scripting.asciidoc @@ -1,16 +1,20 @@ +[float] [[breaking_60_scripting_changes]] === Scripting changes +[float] ==== Groovy, JavaScript, and Python languages removed The Groovy, JavaScript, and Python scripting languages were deprecated in elasticsearch 5.0 and have now been removed. Use painless instead. +[float] ==== Native scripts removed Native scripts have been removed. Instead, <>. +[float] ==== Date fields now return dates `doc.some_date_field.value` now returns ++ReadableDateTime++s instead of @@ -18,15 +22,18 @@ milliseconds since epoch as a `long`. The same is true for `doc.some_date_field[some_number]`. Use `doc.some_date_field.value.millis` to fetch the milliseconds since epoch if you need it. +[float] ==== Removed access to index internal via the `_index` variable The `_index` variable has been removed. If you used it for advanced scoring, consider writing a `Similarity` plugin. +[float] ==== Script Settings All of the existing scripting security settings have been removed. Instead they are replaced with `script.allowed_types` and `script.allowed_contexts`. +[float] ==== `lang` can no longer be specified when using a stored script as part of a request The `lang` variable can no longer be specified as part of a request that uses a stored @@ -35,6 +42,7 @@ different from a request that puts a stored script. The language of the script already been stored as part of the cluster state and an `id` is sufficient to access all of the information necessary to execute a stored script. +[float] ==== 'lang` can no longer be used when putting, getting, or deleting a stored script Stored scripts can no longer have the `lang` parameter specified as part of the url @@ -42,6 +50,7 @@ when performing PUT, GET, and DELETE actions on the `_scripts/` path. All store scripts must have a unique `id` as the namespace is only `id` now and no longer `lang` and `id`. +[float] ==== Stored search template apis removed The PUT, GET and DELETE `_search/template` apis have been removed. Store search templates with the stored scripts apis instead. diff --git a/docs/reference/migration/migrate_6_0/search.asciidoc b/docs/reference/migration/migrate_6_0/search.asciidoc index 3416c940e225e..e0fb330570578 100644 --- a/docs/reference/migration/migrate_6_0/search.asciidoc +++ b/docs/reference/migration/migrate_6_0/search.asciidoc @@ -1,6 +1,8 @@ +[float] [[breaking_60_search_changes]] === Search and Query DSL changes +[float] ==== Changes to queries * The `collect_payloads` parameter of the `span_near` query has been removed. Payloads will be @@ -99,19 +101,23 @@ the `match` query but is supported for `match_phrase` and `match_phrase_prefix`. * The deprecated multi term rewrite parameters `constant_score_auto`, `constant_score_filter` (synonyms for `constant_score`) have been removed. +[float] ==== Search shards API The search shards API no longer accepts the `type` url parameter, which didn't have any effect in previous versions. +[float] ==== Changes to the Profile API The `"time"` field showing human readable timing output has been replaced by the `"time_in_nanos"` field which displays the elapsed time in nanoseconds. The `"time"` field can be turned on by adding `"?human=true"` to the request url. It will display a rounded, human readable time value. +[float] ==== Scoring changes +[float] ===== Query normalization Query normalization has been removed. This means that the TF-IDF similarity no @@ -124,6 +130,7 @@ already integrated boosts into scores as multiplicative factors. See https://issues.apache.org/jira/browse/LUCENE-7347[`LUCENE-7347`] for more information. +[float] ===== Coordination factors Coordination factors have been removed from the scoring formula. This means that @@ -137,6 +144,7 @@ produces. BM25 is recommended instead. See https://issues.apache.org/jira/browse/LUCENE-7347[`LUCENE-7347`] for more information. +[float] ==== Fielddata on `_uid` Fielddata on `_uid` is deprecated. It is possible to switch to `_id` instead @@ -145,6 +153,7 @@ for the `random_score` function. If you really need access to the id of documents for sorting, aggregations or search scripts, the recommendation is to duplicate the id as a field in the document. +[float] ==== Highlighters The `unified` highlighter is the new default choice for highlighter. @@ -156,36 +165,43 @@ The `postings` highlighter has been removed from Lucene and Elasticsearch. The `unified` highlighter outputs the same highlighting when `index_options` is set to `offsets`. +[float] ==== `fielddata_fields` The deprecated `fielddata_fields` have now been removed. `docvalue_fields` should be used instead. +[float] ==== `docvalue_fields` `docvalue_fields` now have a default upper limit of 100 fields that can be requested. This limit can be overridden by using the `index.max_docvalue_fields_search` index setting. +[float] ==== `script_fields` `script_fields` now have a default upper limit of 32 script fields that can be requested. This limit can be overridden by using the `index.max_script_fields` index setting. +[float] ==== Inner hits The source inside a hit of inner hits keeps its full path with respect to the entire source. In prior versions the source field names were relative to the inner hit. +[float] ==== Scroll The `from` parameter can no longer be used in the search request body when initiating a scroll. The parameter was already ignored in these situations, now in addition an error is thrown. +[float] ==== Limit on from/size in top hits and inner hits The maximum number of results (`from` + `size`) that is allowed to be retrieved via inner hits and top hits has been limited to 100. The limit can be controlled via the `index.max_inner_result_window` index setting. +[float] ==== Scroll queries that use the request_cache are deprecated Setting `request_cache:true` on a query that creates a scroll ('scroll=1m`) @@ -194,6 +210,7 @@ In future versions we will return a `400 - Bad request` instead of just ignoring the hint. Scroll queries are not meant to be cached. +[float] ==== Limiting the number of terms that can be used in a Terms Query request Executing a Terms Query with a lot of terms may degrade the cluster performance, @@ -202,6 +219,7 @@ To safeguard against this, the maximum number of terms that can be used in a Terms Query request has been limited to 65536. This default maximum can be changed for a particular index with the index setting `index.max_terms_count`. +[float] ==== Invalid `_search` request body For 6.x and starting in 6.3 a deprecation warning will be printed to warn @@ -209,12 +227,14 @@ against search requests that contain extra tokens after the main object. These extra tokens were ignored by the query parser before 6.3 but the next major version will not accept invalid body anymore. +[float] ==== Context suggester without contexts The ability to query and index context enabled suggestions without contexts has been deprecated. Context enabled suggestion queries without contexts have to visit every suggestion, which degrades the search performance considerably. +[float] ==== Limiting the max number of expansion of span_multi queries `span_multi` queries will hit too many clauses failure if the number of terms that match the diff --git a/docs/reference/migration/migrate_6_0/settings.asciidoc b/docs/reference/migration/migrate_6_0/settings.asciidoc index 289e1903fb81e..478f4bbee51cc 100644 --- a/docs/reference/migration/migrate_6_0/settings.asciidoc +++ b/docs/reference/migration/migrate_6_0/settings.asciidoc @@ -1,11 +1,14 @@ +[float] [[breaking_60_settings_changes]] === Settings changes +[float] ==== Remove support for elasticsearch.json and elasticsearch.yaml configuration file The configuration file found in the Elasticsearch config directory could previously have a `.yml`, `.yaml` or `.json` extension. Only `elasticsearch.yml` is now supported. +[float] ==== Duplicate keys in configuration file In previous versions of Elasticsearch, the configuration file was allowed to @@ -32,6 +35,7 @@ node: rack: my-rack -------------------------------------------------- +[float] ==== Coercion of boolean settings Previously, Elasticsearch recognized the strings `true`, `false`, `on`, `off`, `yes`, `no`, `0`, `1` as booleans. Elasticsearch 6.0 @@ -39,12 +43,14 @@ recognizes only `true` and `false` as boolean and will throw an error otherwise. index settings on pre-6.0 indices will continue to work. Note that this does not apply to node-level settings that are stored in `elasticsearch.yml`. +[float] ==== Snapshot settings The internal setting `cluster.routing.allocation.snapshot.relocation_enabled` that allowed shards with running snapshots to be reallocated to different nodes has been removed. Enabling this setting could cause allocation issues if a shard got allocated off a node and then reallocated back to this node while a snapshot was running. +[float] ==== Store throttling settings Store throttling has been removed. As a consequence, the @@ -53,12 +59,14 @@ cluster settings and the `index.store.throttle.type` and `index.store.throttle.max_bytes_per_sec` index settings are not recognized anymore. +[float] ==== Store settings The `default` `index.store.type` has been removed. If you were using it, we advise that you simply remove it from your index settings and Elasticsearch will use the best `store` implementation for your operating system. +[float] ==== Network settings The blocking TCP client, blocking TCP server, and blocking HTTP server have been removed. @@ -70,17 +78,20 @@ The previously unused settings `transport.netty.max_cumulation_buffer_capacity`, `transport.netty.max_composite_buffer_components` and `http.netty.max_cumulation_buffer_capacity` have been removed. +[float] ==== Similarity settings The `base` similarity is now ignored as coords and query normalization have been removed. If provided, this setting will be ignored and issue a deprecation warning. +[float] ==== Script Settings All of the existing scripting security settings have been removed. Instead they are replaced with `script.allowed_types` and `script.allowed_contexts`. +[float] ==== Discovery Settings The `discovery.type` settings no longer supports the values `gce`, `aws` and `ec2`. diff --git a/docs/reference/migration/migrate_6_0/stats.asciidoc b/docs/reference/migration/migrate_6_0/stats.asciidoc index ed70d1503c4c3..d26840d482bac 100644 --- a/docs/reference/migration/migrate_6_0/stats.asciidoc +++ b/docs/reference/migration/migrate_6_0/stats.asciidoc @@ -1,11 +1,14 @@ +[float] [[breaking_60_stats_changes]] === Stats and info changes +[float] ==== Removal of `throttle_time` in the `store` stats Given that store throttling has been removed, the `store` stats do not report `throttle_time` anymore. +[float] ==== FS stats no longer reports if the disk spins Elasticsearch has defaulted to assuming that it is running on SSDs since diff --git a/docs/reference/migration/migrate_6_1.asciidoc b/docs/reference/migration/migrate_6_1.asciidoc index 579d00fa778fc..102f66005003b 100644 --- a/docs/reference/migration/migrate_6_1.asciidoc +++ b/docs/reference/migration/migrate_6_1.asciidoc @@ -1,5 +1,8 @@ [[breaking-changes-6.1]] == Breaking changes in 6.1 +++++ +6.1 +++++ This section discusses the changes that you need to be aware of when migrating your application to Elasticsearch 6.1. diff --git a/docs/reference/migration/migrate_6_2.asciidoc b/docs/reference/migration/migrate_6_2.asciidoc index 6a78363721f80..bab210cf3daf5 100644 --- a/docs/reference/migration/migrate_6_2.asciidoc +++ b/docs/reference/migration/migrate_6_2.asciidoc @@ -1,5 +1,8 @@ [[breaking-changes-6.2]] == Breaking changes in 6.2 +++++ +6.2 +++++ This section discusses the changes that you need to be aware of when migrating your application to Elasticsearch 6.2 @@ -16,6 +19,7 @@ and one permission that can be added to this policy is manager. As such, granting this permission in production mode is now forbidden via the <>. +[float] === Private temporary directory On Linux, previous versions of Elasticsearch defaulted to using `/tmp` as the @@ -27,6 +31,7 @@ existing <>, you should add the line using a private temporary directory on non-Linux systems with the same mechanism. +[float] === GC logging Starting in Elasticsearch 6.2.0, JVM <> is enabled out of @@ -55,6 +60,7 @@ otherwise if you have installed from the `tar.gz` or `zip` archive distributions and use the default logging location use `logs/gc.log`; otherwise, set the location to the absolute path to your log files. +[float] === Rollover alias A rollover request in Elasticsearch 6.2.0 or later will be aborted if the same alias is found in the matching index templates. This change is to prevent the alias diff --git a/docs/reference/migration/migrate_6_3.asciidoc b/docs/reference/migration/migrate_6_3.asciidoc index e18021d46222d..e2565acee1813 100644 --- a/docs/reference/migration/migrate_6_3.asciidoc +++ b/docs/reference/migration/migrate_6_3.asciidoc @@ -1,5 +1,8 @@ [[breaking-changes-6.3]] == Breaking changes in 6.3 +++++ +6.3 +++++ This section discusses the changes that you need to be aware of when migrating your application to Elasticsearch 6.3. @@ -10,9 +13,11 @@ your application to Elasticsearch 6.3. See also <> and <>. +[float] [[breaking_63_api_changes]] === API changes +[float] ==== Suggest stats metrics deprecated The suggest stats were previously folded into the search on the indices stats @@ -24,9 +29,11 @@ Similarly, the `suggest` index metric on the `indices` metric on the nodes stats API has provided a response containing only an empty object since 5.0.0. In 6.3.0 this metric has been deprecated. +[float] [[breaking_63_plugins_changes]] === Plugins changes +[float] ==== GCS Repository plugin The repository settings `application_name`, `connect_timeout` and `read_timeout` @@ -34,6 +41,7 @@ have been deprecated and must now be specified in the client settings instead. See {plugins}/repository-gcs-client.html#repository-gcs-client[Google Cloud Storage Client Settings]. +[float] ==== Ingest Geoip Plugin In earlier versions, database files have been stored as gzip compressed files @@ -44,6 +52,7 @@ memory. Any custom database files must also be stored uncompressed. Consequently the `database_file` property in any ingest pipelines that use the Geoip Processor must refer to the uncompressed database files as well. +[float] ==== Using the plugin installer without a TTY The Elasticsearch plugin installer (`elasticsearch-plugin install`) would @@ -55,15 +64,18 @@ accept these permissions either by keeping standard input open and attaching a TTY (i.e., using interactive mode to accept the permissions), or by passing the `--batch` flag. +[float] ==== Implementing custom realms with SPI instead of XPackExtension The legacy `XPackExtension` extension mechanism has been removed and replaced with an SPI based extension mechanism that is installed and built as an elasticsearch plugin. For more information about using SPI loaded security extensions in custom realms, see {stack-ov}/custom-realms.html[Integrating with other authentication systems]. +[float] [[breaking_63_settings_changes]] === Settings changes +[float] ==== Concurrency level of analyze requests Previously, analyze requests would run on the same thread pool as indexing @@ -79,6 +91,7 @@ probably do not see high concurrency. If you are impacted by this you can increase the size of the thread pool by using the `thread_pool.analyze.size` setting. +[float] ==== Renaming the bulk thread pool The `bulk` thread pool has been renamed to the `write` thread pool. This change @@ -95,6 +108,7 @@ place) you can start Elasticsearch with the JVM option continue to display the name of this thread pool as `bulk`. Elasticsearch will stop observing this system property in 7.0.0. +[float] ==== Enabling monitoring By default when you install {xpack}, monitoring is enabled but data collection diff --git a/docs/reference/migration/migrate_6_4.asciidoc b/docs/reference/migration/migrate_6_4.asciidoc index 2ee1c4339eed8..44fa915e365a5 100644 --- a/docs/reference/migration/migrate_6_4.asciidoc +++ b/docs/reference/migration/migrate_6_4.asciidoc @@ -1,5 +1,8 @@ [[breaking-changes-6.4]] == Breaking changes in 6.4 +++++ +6.4 +++++ This section discusses the changes that you need to be aware of when migrating your application to Elasticsearch 6.4. @@ -9,9 +12,11 @@ your application to Elasticsearch 6.4. See also <> and <>. +[float] [[breaking_64_api_changes]] === API changes +[float] ==== Field capabilities request format In the past, `fields` could be provided either as a parameter, or as part of the request @@ -19,6 +24,7 @@ body. Specifying `fields` in the request body is now deprecated, and instead the always be supplied through a request parameter. In 7.0.0, the field capabilities API will not accept `fields` supplied in the request body. +[float] [[copy-source-settings-on-resize]] ==== Copying source settings during shrink/split operations @@ -36,9 +42,11 @@ to follow up the operation with a request to adjust to the desired settings on the target index, or send the desired value of these settings with the resize operation. +[float] [[breaking_64_search_changes]] === Search and query DSL changes +[float] ==== Limiting the length of regex that can be used in a Regexp Query request <> with long string made of many operators may run into a stack overflow. @@ -46,9 +54,11 @@ To safeguard against this, the maximum length of regex that can be used in a Regexp Query request has been limited to 1000. This default maximum can be changed for a particular index with the index setting `index.max_regex_length`. +[float] [[breaking_64_scripting_changes]] === Scripting +[float] ==== Legacy stored script formats have been deprecated Stored script formats that don't use top level `script` object have been deprecated and @@ -57,9 +67,11 @@ support for these formats will be removed in the next major release. This includes `template` stored scripts format and formats that do not encapsulate the script inside a `script` json object. +[float] [[breaking_64_rest_client_changes]] === REST Client +[float] ==== Old low level ++performRequest++s deprecated The versions of `performRequest` and `performRequestAsync` that were in the low level client in 6.3 have been deprecated in favor of @@ -67,12 +79,14 @@ low level client in 6.3 have been deprecated in favor of These will allow us to add more features to the client without adding more variants of `performRequest`. +[float] ==== Old high level request methods deprecated All request methods on the high level client have been deprecated in favor of a new variant that takes `RequestOptions`. This allows users of the high level client to customize many options on individual requests that were not available otherwise. +[float] ==== HostSniffer renamed to NodeSniffer and signature changed To provide allow the `Sniffer` to pick up more metadata we changed it from sniffing ++HttpHost++s to sniffing ++Node++s, a new class introduced to contain diff --git a/docs/reference/migration/migrate_7_0/snapshotstats.asciidoc b/docs/reference/migration/migrate_7_0/snapshotstats.asciidoc index 6dbd24b13a1eb..2098eb3574ca8 100644 --- a/docs/reference/migration/migrate_7_0/snapshotstats.asciidoc +++ b/docs/reference/migration/migrate_7_0/snapshotstats.asciidoc @@ -1,3 +1,4 @@ +[float] [[breaking_70_snapshotstats_changes]] === Snapshot stats changes @@ -7,6 +8,7 @@ Snapshot stats details are provided in a new structured way: * `incremental` section for those files that actually needed to be copied over as part of the incremental snapshotting. * In case of a snapshot that's still in progress, there's also a `processed` section for files that are in the process of being copied. +[float] ==== Deprecated `number_of_files`, `processed_files`, `total_size_in_bytes` and `processed_size_in_bytes` snapshot stats properties have been removed * Properties `number_of_files` and `total_size_in_bytes` are removed and should be replaced by values of nested object `total`. diff --git a/docs/reference/release-notes/6.2.asciidoc b/docs/reference/release-notes/6.2.asciidoc index 14be7e016c9b0..9ac310c527ce8 100644 --- a/docs/reference/release-notes/6.2.asciidoc +++ b/docs/reference/release-notes/6.2.asciidoc @@ -141,10 +141,9 @@ analysis module. ({pull}30397[#30397]) === Enhancements Highlighting:: -* Limit analyzed text for highlighting (improvements) {pull}28808[#28808] (issues: {issue}16764[#16764], {issue}27934[#27934]) -{ref-64}/breaking_64_api_changes.html#copy-source-settings-on-resize[Allow copying source settings on index resize operations] ({pull}30255[#30255]) -{ref-64}/breaking_64_api_changes.html#copy-source-settings-on-resize[Allow -copying source settings on index resize operations] ({pull}30255[#30255], {pull}30404[#30404]) +* Limit analyzed text for highlighting (improvements) {pull}28808[#28808] (issues: {issue}16764[#16764], {issue}27934[#27934]) + +<> ({pull}30255[#30255],{pull}30404[#30404]) Added new "Request" object flavored request methods in the RestClient. Prefer these instead of the multi-argument versions. ({pull}29623[#29623]) diff --git a/docs/reference/release-notes/6.4.asciidoc b/docs/reference/release-notes/6.4.asciidoc index 9b29db6450ea2..66a89e33b6372 100644 --- a/docs/reference/release-notes/6.4.asciidoc +++ b/docs/reference/release-notes/6.4.asciidoc @@ -277,7 +277,7 @@ Highlighting:: Index APIs:: * Add Index UUID to `/_stats` Response {pull}31871[#31871] (issue: {issue}31791[#31791]) * add support for write index resolution when creating/updating documents {pull}31520[#31520] - * {ref-64}/breaking_64_api_changes.html#copy-source-settings-on-resize[Allow copying source settings on index resize operations] {pull}30255[#30255] (issue: {issue}28347[#28347]) + * <> {pull}30255[#30255] (issue: {issue}28347[#28347]) Ingest:: * Extend KV Processor (#31789) {pull}32232[#32232] (issue: {issue}31786[#31786])