Add simulate template composition API _index_template/_simulate_index/{name} #55686

andreidan · 2020-04-23T17:09:35Z

This adds a new api to simulate matching the given index name against the
index templates in the system.

The syntax for the new API takes the following form:

POST _index_template/_simulate_index/{index_name}
{
  "index_patterns": ["logs-*"],
  "priority": 15,
  "template": {
	"settings": {
		"number_of_shards": 3
	}
       ...
   }
}

Where the body is optional, but we support the entire body used by the
PUT _index_template/{name} api. When the body is specified we'll simulate
matching the given index against a system that'd have the given index
template together with the index templates that exist in the system.

The response, in both cases, will return the matching template's resolved
settings, mappings and aliases, together with a special field that'll print any
overlapping templates and their corresponding index patterns.

Relates to #53101

This adds a new api to simulate matching the given index name against the index templates in the system.

elasticmachine · 2020-04-23T17:09:37Z

Pinging @elastic/es-core-features (:Core/Features/Indices APIs)

andreidan · 2020-04-24T10:04:12Z

@elasticmachine update branch

dakrone · 2020-04-24T15:09:52Z

@andreidan how would you feel about removing the "Post" prefix from the actions and class names? I think it sounds a little strange to call it PostSimulateIndexTemplate* rather than just SimulateIndexTemplate*, and I'd prefer to remove the prefix.

andreidan · 2020-04-24T15:13:58Z

@andreidan how would you feel about removing the "Post" prefix from the actions and class names? I think it sounds a little strange to call it PostSimulateIndexTemplate* rather than just SimulateIndexTemplate*, and I'd prefer to remove the prefix.

Sure thing

…teAction

andreidan · 2020-04-24T15:22:12Z

@dakrone renamed the classes

andreidan · 2020-04-24T18:08:55Z

@dakrone I think this is ready for review (removed the post_ prefix from the action names too)

dakrone

Thanks for working on this Andrei! I left some comments about it

...n/java/org/elasticsearch/action/admin/indices/template/post/SimulateIndexTemplateAction.java

.../java/org/elasticsearch/action/admin/indices/template/post/SimulateIndexTemplateRequest.java

rest-api-spec/src/main/resources/rest-api-spec/api/indices.simulate_index_template.json

...java/org/elasticsearch/action/admin/indices/template/post/SimulateIndexTemplateResponse.java

...g/elasticsearch/action/admin/indices/template/post/TransportSimulateIndexTemplateAction.java

server/src/main/java/org/elasticsearch/cluster/metadata/MetadataCreateIndexService.java

server/src/main/java/org/elasticsearch/cluster/metadata/MetadataIndexTemplateService.java

Co-Authored-By: Lee Hinman <dakrone@users.noreply.github.com>

dakrone

Thanks @andreidan! I left a couple more comments, but overall looks good!

...java/org/elasticsearch/action/admin/indices/template/post/SimulateIndexTemplateResponse.java

...g/elasticsearch/action/admin/indices/template/post/TransportSimulateIndexTemplateAction.java

andreidan · 2020-04-28T16:42:28Z

@elasticmachine update branch

dakrone

LGTM, thanks for iterating on this Andrei! I left one more comment about adding a test that also uses composed_of (to make sure we resolve everything when simulating)

...-api-spec/src/main/resources/rest-api-spec/test/indices.simulate_index_template/10_basic.yml

andreidan · 2020-04-28T17:17:02Z

@elasticmachine update branch

andreidan · 2020-04-28T17:30:44Z

@elasticmachine run elasticsearch-ci/2

…/{name} (elastic#55686) This adds a new api to simulate matching the given index name against the index templates in the system. The syntax for the new API takes the following form: POST _index_template/_simulate_index/{index_name} { "index_patterns": ["logs-*"], "priority": 15, "template": { "settings": { "number_of_shards": 3 } ... } } Where the body is optional, but we support the entire body used by the PUT _index_template/{name} api. When the body is specified we'll simulate matching the given index against a system that'd have the given index template together with the index templates that exist in the system. The response, in both cases, will return the matching template's resolved settings, mappings and aliases, together with a special field that'll print any overlapping templates and their corresponding index patterns. (cherry picked from commit 1a5845e) Signed-off-by: Andrei Dan <andrei.dan@elastic.co>

…_index/{name} (#55686) (#55922) This adds a new api to simulate matching the given index name against the index templates in the system. The syntax for the new API takes the following form: POST _index_template/_simulate_index/{index_name} { "index_patterns": ["logs-*"], "priority": 15, "template": { "settings": { "number_of_shards": 3 } ... } } Where the body is optional, but we support the entire body used by the PUT _index_template/{name} api. When the body is specified we'll simulate matching the given index against a system that'd have the given index template together with the index templates that exist in the system. The response, in both cases, will return the matching template's resolved settings, mappings and aliases, together with a special field that'll print any overlapping templates and their corresponding index patterns. (cherry picked from commit 1a5845e) Signed-off-by: Andrei Dan <andrei.dan@elastic.co>

As elastic#55686 was backported the simulate index template api is no available in 7.8.

As #55686 was backported the simulate index template api is no available in 7.8.

This adds an API for simulating template composition with or without an index template. It looks like: ``` POST /_index_template/_simulate/my-template ``` To simulate a template named `my-template` that already exists, or, to simulate a template that does not already exist: ``` POST /_index_template/_simulate { "index_patterns": ["my-index"] "composed_of": ["ct1", "ct2"], } ``` This is related to elastic#55686, which adds an API to simulate composition based on an index name (hence the `_simulate_index` vs `_simulate`). This commit also adds reference documentation for both simulation APIs. Relates to elastic#53101 Resolves elastic#56390 Resolves elastic#56255

This adds an API for simulating template composition with or without an index template. It looks like: ``` POST /_index_template/_simulate/my-template ``` To simulate a template named `my-template` that already exists, or, to simulate a template that does not already exist: ``` POST /_index_template/_simulate { "index_patterns": ["my-index"] "composed_of": ["ct1", "ct2"], } ``` This is related to #55686, which adds an API to simulate composition based on an index name (hence the `_simulate_index` vs `_simulate`). This commit also adds reference documentation for both simulation APIs. Relates to #53101 Resolves #56390 Resolves #56255

…tic#56842) This adds an API for simulating template composition with or without an index template. It looks like: ``` POST /_index_template/_simulate/my-template ``` To simulate a template named `my-template` that already exists, or, to simulate a template that does not already exist: ``` POST /_index_template/_simulate { "index_patterns": ["my-index"] "composed_of": ["ct1", "ct2"], } ``` This is related to elastic#55686, which adds an API to simulate composition based on an index name (hence the `_simulate_index` vs `_simulate`). This commit also adds reference documentation for both simulation APIs. Relates to elastic#53101 Resolves elastic#56390 Resolves elastic#56255

Add _index_template/_simulate_index/{name} api

2be7e19

This adds a new api to simulate matching the given index name against the index templates in the system.

andreidan added :Data Management/Indices APIs APIs to create and manage indices and templates v8.0.0 v7.8.0 labels Apr 23, 2020

Remove unused logger

17a912e

elasticmachine and others added 5 commits April 24, 2020 06:04

Merge branch 'master' into itv2-simulate-api

3025a51

Tests

490ca72

Yaml parsing and new line

20eeea4

Fix tests

871980a

Fix version restriction

f34071d

andreidan requested a review from dakrone April 24, 2020 13:43

andreidan marked this pull request as ready for review April 24, 2020 13:52

martijnvg mentioned this pull request Apr 24, 2020

Composable Templates #53101

Closed

39 tasks

andreidan changed the title ~~Add _index_template/_simulate_index/{name} api~~ Add simulate template composition API _index_template/_simulate_index/{name} Apr 24, 2020

Fix response serialisation

8ba793a

andreidan added 2 commits April 24, 2020 16:18

Rename PostSimulateIndexTemplateAction to SimulateIndexTemplateAction

4b85534

Rename RestPostSimulateIndexTemplateAction to RestSimulateIndexTempla…

b98a59b

…teAction

Remove "post_" prefix

cbcc977

dakrone requested changes Apr 24, 2020

View reviewed changes

andreidan and others added 5 commits April 27, 2020 11:13

Drop /post from action name

3ca7e20

Co-Authored-By: Lee Hinman <dakrone@users.noreply.github.com>

Update rest api spec

a436e39

Co-Authored-By: Lee Hinman <dakrone@users.noreply.github.com>

Drop "post" from package name

604936a

Fix indentation

edc8610

Use the resolveAndValidateAliases method to resolve aliases

34d7a75

dakrone requested changes Apr 28, 2020

View reviewed changes

andreidan added 3 commits April 28, 2020 17:09

Drop dynamic keys from response

03d9b0f

Use additional map

8863d1f

Update remove indexService reason

db5782b

Merge branch 'master' into itv2-simulate-api

7a6582c

dakrone approved these changes Apr 28, 2020

View reviewed changes

...-api-spec/src/main/resources/rest-api-spec/test/indices.simulate_index_template/10_basic.yml Show resolved Hide resolved

andreidan added 2 commits April 28, 2020 18:15

Use composed_of in the simulated template

ab918fd

Assert overlapping is array

e368647

Merge branch 'master' into itv2-simulate-api

d3cac66

andreidan merged commit 1a5845e into elastic:master Apr 29, 2020

andreidan added the backport pending label Apr 29, 2020

andreidan mentioned this pull request Apr 29, 2020

[7.x] Add simulate template composition API _index_template/_simulate_index/{name} (#55686) #55922

Merged

andreidan removed the backport pending label Apr 29, 2020

andreidan added a commit to andreidan/elasticsearch that referenced this pull request Apr 29, 2020

Enable simulate API tests in 7.8

d8874ee

As elastic#55686 was backported the simulate index template api is no available in 7.8.

andreidan mentioned this pull request Apr 29, 2020

[7.x] Enable simulate API tests in 7.8 #55946

Merged

andreidan added a commit that referenced this pull request May 5, 2020

Enable simulate API tests in 7.8 (#55946)

f569405

As #55686 was backported the simulate index template api is no available in 7.8.

pugnascotia added the >feature label May 6, 2020

cjcenizal mentioned this pull request May 6, 2020

API endpoint for previewing composed result of an index template v2 #56255

Closed

dakrone mentioned this pull request May 15, 2020

Add template simulation API for simulating template composition #56842

Merged

russcam mentioned this pull request May 29, 2020

7.8.0 Meta ticket elastic/elasticsearch-net#4718

Closed

17 tasks

jakelandis added v8.0.0-alpha1 and removed v8.0.0 labels Jul 26, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add simulate template composition API _index_template/_simulate_index/{name} #55686

Add simulate template composition API _index_template/_simulate_index/{name} #55686

andreidan commented Apr 23, 2020 •

edited

elasticmachine commented Apr 23, 2020

andreidan commented Apr 24, 2020

dakrone commented Apr 24, 2020

andreidan commented Apr 24, 2020

andreidan commented Apr 24, 2020

andreidan commented Apr 24, 2020

dakrone left a comment

dakrone left a comment

andreidan commented Apr 28, 2020

dakrone left a comment

andreidan commented Apr 28, 2020

andreidan commented Apr 28, 2020

Add simulate template composition API _index_template/_simulate_index/{name} #55686

Add simulate template composition API _index_template/_simulate_index/{name} #55686

Conversation

andreidan commented Apr 23, 2020 • edited

elasticmachine commented Apr 23, 2020

andreidan commented Apr 24, 2020

dakrone commented Apr 24, 2020

andreidan commented Apr 24, 2020

andreidan commented Apr 24, 2020

andreidan commented Apr 24, 2020

dakrone left a comment

Choose a reason for hiding this comment

dakrone left a comment

Choose a reason for hiding this comment

andreidan commented Apr 28, 2020

dakrone left a comment

Choose a reason for hiding this comment

andreidan commented Apr 28, 2020

andreidan commented Apr 28, 2020

andreidan commented Apr 23, 2020 •

edited