Remove reference to mapping types #23650

jbudz · 2018-10-01T15:54:30Z

Elasticsearch is taking the next step in working towards a world with no types. In 6.x, we enforced using a single type. In 7.0 we plan to offer APIs which do not require a type at all. Relevant APIs are indexing APIs, read APIs, and management APIs for things like index templates and mapping.

The plan is to take a two-step approach:

Elasticsearch will offer both a type less API and a typed API in 7.x. Usage of the later will cause deprecation warnings.
All our shippers and Kibana need to move to use the type less API.

lukeelmers · 2019-01-10T23:01:09Z

Awhile ago I did a full run of the functional test suite and logged every request the ES client makes to the .kibana index. This was the list of the unique routes I found that are using types in the path (obviously this only includes routes with test coverage; nothing too surprising here):

  -> GET .kibana/doc/config:{version}
  -> POST .kibana/doc/config:{version}/_create
  
  -> GET .kibana/doc/dashboard:{id}
  -> POST .kibana/doc/dashboard:{id}
  -> DELETE .kibana/doc/dashboard:{id}
  
  -> POST .kibana/doc/index-pattern:{id}
  -> POST .kibana/doc/index-pattern:{id}/_create
  -> POST .kibana/doc/index-pattern:{id}/_update
  -> DELETE .kibana/doc/index-pattern:{id}
  
  -> POST .kibana/doc/search:{id}
  -> DELETE .kibana/doc/search:{id}
  
  -> POST .kibana/doc/url:{id}/_create
  
  -> POST .kibana/doc/visualization:{id}
  -> POST .kibana/doc/visualization:{id}/_create
  -> DELETE .kibana/doc/visualization:{id}

Of course I just now saw #23654, which pretty much covers all of these already. But figured I would post this info here anyway for posterity.

chrisdavies · 2019-01-11T12:09:24Z

Got a little ways down the road on this, and am currently blocked waiting for this:

elastic/elasticsearch#37285

Currently, put mapping still expects you to provide the root-level type. That PR will fix this (and other inconsistencies).

Due to elastic/elasticsearch#37285, we must set include_type_name when using the type in the request. This is a stop-gap until we complete elastic#23650 Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

Follow-up to elastic#28497 Since elastic/elasticsearch#37285 has been merged, it's aparent there are a couple more places we need to specify include_type_name as a stop-gap until elastic#23650 is completed Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

* [migrations] Fetch additional mappings with types Follow-up to #28497 Since elastic/elasticsearch#37285 has been merged, it's aparent there are a couple more places we need to specify include_type_name as a stop-gap until #23650 is completed Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co> * Updates tests Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

* [migrations] Fetch additional mappings with types Follow-up to elastic#28497 Since elastic/elasticsearch#37285 has been merged, it's aparent there are a couple more places we need to specify include_type_name as a stop-gap until elastic#23650 is completed Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co> * Updates tests Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

elasticmachine · 2019-01-18T15:09:59Z

Pinging @elastic/infrastructure-ui

elasticmachine · 2019-01-18T15:10:00Z

Pinging @elastic/stack-monitoring

elasticmachine · 2019-01-18T15:10:05Z

Pinging @elastic/kibana-app

elasticmachine · 2019-01-18T15:10:13Z

Pinging @elastic/es-ui

elasticmachine · 2019-01-18T15:11:13Z

Pinging @elastic/kibana-qa

stacey-gammon · 2019-01-18T15:12:25Z

Current list of failing tests, after the es archives were updated:

API Integration Tests.test/api_integration/apis/index_patterns/es_errors/errors·js.apis index_patterns index_patterns/service/lib index_patterns/* error handler "before all" hook
UI Functional Tests.test/functional/apps/context/index·js.context app "before all" hook
UI Functional Tests.test/functional/apps/dashboard/index·js.dashboard app using current data "before all" hook: loadCurrentData
UI Functional Tests.test/functional/apps/dashboard/index·js.dashboard app using current data "before all" hook: loadCurrentData
UI Functional Tests.test/functional/apps/dashboard/_dashboard_time_picker·js.dashboard app using legacy data dashboard time picker "before all" hook
UI Functional Tests.test/functional/apps/dashboard/_dashboard_save·js.dashboard app using legacy data dashboard save "before all" hook
UI Functional Tests.test/functional/apps/getting_started/_shakespeare·js.Getting Started Shakespeare "before all" hook
UI Functional Tests.test/functional/apps/getting_started/_shakespeare·js.Getting Started Shakespeare "after all" hook
UI Functional Tests.test/functional/apps/getting_started/index·js.Getting Started "after all" hook
UI Functional Tests.test/functional/apps/management/index·js.management "before all" hook
UI Functional Tests.test/functional/apps/management/index·js.management "after all" hook
UI Functional Tests.test/functional/apps/management/index·js.management "before all" hook
UI Functional Tests.test/functional/apps/management/index·js.management "after all" hook
UI Functional Tests.test/functional/apps/visualize/index·js.visualize app "before all" hook
UI Functional Tests.test/functional/apps/visualize/index·js.visualize app "after all" hook
UI Functional Tests.test/functional/apps/visualize/index·js.visualize app "before all" hook
UI Functional Tests.test/functional/apps/visualize/index·js.visualize app "after all" hook
UI Functional Tests.test/functional/apps/visualize/index·js.visualize app "before all" hook
UI Functional Tests.test/functional/apps/visualize/index·js.visualize app "after all" hook
UI Functional Tests.test/functional/apps/visualize/index·js.visualize app "before all" hook
UI Functional Tests.test/functional/apps/visualize/index·js.visualize app "after all" hook
X-Pack Functional Tests.test/functional/apps/monitoring/cluster/list·js.Monitoring app Cluster listing with trial license clusters "before all" hook
X-Pack Functional Tests.test/functional/apps/monitoring/cluster/list·js.Monitoring app Cluster listing with trial license clusters "after all" hook
X-Pack Functional Tests.test/functional/apps/monitoring/cluster/list·js.Monitoring app Cluster listing "after all" hook
X-Pack Functional Tests.test/functional/apps/monitoring.Monitoring app "after all" hook
X-Pack Chromium API Reporting Tests.x-pack/test/reporting/api/chromium_tests·js.chromium "before all" hook
X-Pack Functional Tests.test/functional/apps/dashboard_mode/dashboard_view_mode·js.dashboard mode Dashboard View Mode "before all" hook: initialize tests
X-Pack Functional Tests.test/functional/apps/dashboard_mode/dashboard_view_mode·js.dashboard mode Dashboard View Mode "after all" hook
X-Pack Functional Tests.test/functional/apps/dashboard_mode.dashboard mode "after all" hook
X-Pack Functional Tests.test/functional/apps/security/security·js.security app Security Login Page "before all" hook
X-Pack Functional Tests.test/functional/apps/security/security·js.security app Security Login Page "after all" hook
X-Pack Functional Tests.test/functional/apps/security/security·js.security app Security "after all" hook
X-Pack Functional Tests.test/functional/apps/security.security app "after all" hook
X-Pack API Integration Tests.x-pack/test/api_integration/apis/monitoring/apm/overview·js.apis Monitoring APM overview "before all" hook: load archive
X-Pack Chromium Functional Reporting Tests.x-pack/test/reporting/functional/reporting·js.reporting app Reporting "before all" hook: initialize tests
X-Pack Chromium Functional Reporting Tests.x-pack/test/reporting/functional/reporting·js.reporting app Reporting "after all" hook
X-Pack Chromium Functional Reporting Tests.x-pack/test/reporting/functional/index·js.reporting app "after all" hook
X-Pack Functional Tests.test/functional/apps/infra/home_page·ts.InfraOps app Home page "before all" hook
X-Pack Functional Tests.test/functional/apps/infra/home_page·ts.InfraOps app Home page "after all" hook
X-Pack Functional Tests.test/functional/apps/infra.InfraOps app "after all" hook

Pull that updates the archives: #28862
Branch that should be worked on to resolve this: https://github.com/elastic/kibana/tree/upgrade/elasticsearch-7-0

chrisdavies · 2019-01-22T12:16:19Z

The context for this issue is in an email thread, but the TL;DR; is: I had a PR open that updated all es_archives, but I closed the PR after talking to some other teams who were independently working on fixing their own tests. It seems best to let each team fix their own slice of the app, including their es_archvies so that there wasn't a big nasty merge involved.

chrisdavies · 2019-01-23T15:47:28Z

We've moved forward with this, but are currently blocked by this issue:

elastic/elasticsearch#37773

Elasticsearch is loading up index templates which create mappings with the wrong root types (e.g. 'doc' or whatever), so our tests fail when we attempt to create indices / load archives with _doc or undefined as the root type.

@chrisronline is currently working around this by deleting all index templates prior to running the test suite.

tylersmalley · 2019-01-23T16:35:58Z

To expand on this a bit, Kibana is probably only affected by a subset of the templates listed in elastic/elasticsearch#37773. Due to the templates existing, we need to create hacks to delete the templates in our functional testing to proceed (monitoring example)

jbudz added blocker Team:Operations Team label for Operations Team v7.0.0 labels Oct 1, 2018

jbudz mentioned this issue Oct 2, 2018

doc -> _doc #23654

Closed

This comment has been minimized.

Sign in to view

tylersmalley changed the title ~~Upgrade path from doc to _doc~~ Remove reference to mapping types Nov 28, 2018

droberts195 mentioned this issue Dec 11, 2018

[ML] Removal of types #26968

Closed

chrisdavies self-assigned this Jan 11, 2019

tylersmalley mentioned this issue Jan 15, 2019

[migrations] Fetch additional mappings with types #28729

Merged

chrisdavies mentioned this issue Jan 15, 2019

[WIP] ES7 upgrade-- remove doc #28757

Closed

lukeelmers mentioned this issue Jan 15, 2019

Remove nested table splits from table vis #26057

Merged

8 tasks

stacey-gammon added Team:Monitoring Stack Monitoring team Feature:Metrics UI Metrics UI feature labels Jan 18, 2019

stacey-gammon added the Team:Visualizations Visualization editors, elastic-charts and infrastructure label Jan 18, 2019

stacey-gammon added the Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more label Jan 18, 2019

stacey-gammon added the Team:QA Team label for QA Team label Jan 18, 2019

chrisronline mentioned this issue Jan 18, 2019

[Monitoring] Update es archive mappings to handle doc changes #28994

Closed

chrisronline mentioned this issue Jan 23, 2019

[Monitoring] Get monitoring tests working with mapping changes #29188

Merged

chrisdavies mentioned this issue Jan 23, 2019

Upgrade Kibana to Elasticsearch 7.0 #29184

Merged

alvarolobato added the Team:Infra Monitoring UI - DEPRECATED DEPRECATED - Label for the Infra Monitoring UI team. Use Team:obs-ux-infra_services label Jan 29, 2019

chrisdavies closed this as completed in #29184 Jan 30, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove reference to mapping types #23650

Remove reference to mapping types #23650

jbudz commented Oct 1, 2018 •

edited by tylersmalley

Loading

This comment has been minimized.

lukeelmers commented Jan 10, 2019 •

edited

Loading

chrisdavies commented Jan 11, 2019

elasticmachine commented Jan 18, 2019

elasticmachine commented Jan 18, 2019

elasticmachine commented Jan 18, 2019

elasticmachine commented Jan 18, 2019

elasticmachine commented Jan 18, 2019

stacey-gammon commented Jan 18, 2019

chrisdavies commented Jan 22, 2019

chrisdavies commented Jan 23, 2019

tylersmalley commented Jan 23, 2019

Remove reference to mapping types #23650

Remove reference to mapping types #23650

Comments

jbudz commented Oct 1, 2018 • edited by tylersmalley Loading

This comment has been minimized.

lukeelmers commented Jan 10, 2019 • edited Loading

chrisdavies commented Jan 11, 2019

elasticmachine commented Jan 18, 2019

elasticmachine commented Jan 18, 2019

elasticmachine commented Jan 18, 2019

elasticmachine commented Jan 18, 2019

elasticmachine commented Jan 18, 2019

stacey-gammon commented Jan 18, 2019

chrisdavies commented Jan 22, 2019

chrisdavies commented Jan 23, 2019

tylersmalley commented Jan 23, 2019

jbudz commented Oct 1, 2018 •

edited by tylersmalley

Loading

lukeelmers commented Jan 10, 2019 •

edited

Loading