Unable to delete patterns if the indices don't exist #9224

markwalkom · 2016-11-26T00:13:30Z

Kibana version: 5.0.1

Elasticsearch version: 5.0.1

Server OS version: Ubuntu 14.04

Browser version: Chrome

Browser OS version: OS X 10.12.1

Original install method (e.g. download page, yum, from source, etc.): apt

Description of the problem including expected versus actual behavior:
If an index pattern exists in the KB settings, but not in ES, you cannot delete that pattern.

Are we able to allow the user to go to the pattern definition, display a warning that the index doesn't exist, and at least offer the ability to delete said pattern at least?

Steps to reproduce:

Delete an index that has an existing index pattern in Kibana (via console/curl)
Try to remove that index pattern in Kibana Index Management page

Errors in browser console (if relevant):

Provide logs and/or server output (if relevant):

danielmotaleite · 2017-01-05T16:23:15Z

i also had this, had to recreate the index on ES so i could remove it :)

ycombinator · 2017-01-10T07:57:59Z

This might be related to #8373.

burnstreet · 2017-02-16T08:49:58Z

I am also having this issue, with 5.2

branchnetconsulting · 2017-05-03T15:58:20Z

I am also having this problem on 5.3

chrisronline · 2017-05-16T15:06:06Z

Hi @branchnetconsulting. I'm unable to reproduce the error in 5.3, but I'm also not really sure what the error actually is. Can you provide some more information on steps to reproduce and also what the error actually looks like?

danielmotaleite · 2017-05-16T15:48:30Z

@chrisronline It is simple:

-Create a index on ES
-Create a index pattern in kibana for that index
-Remove the index in ES
-Try to remove the pattern index in kibana.

As the index does not exist anymore, you can not select it to remove the pattern, forcing the admin to create the index again, remove pattern and then remove the index for good.

chrisronline · 2017-05-16T17:35:06Z

Thanks for the added information @danielmotaleite.

I'm still having some issues so I just want to verify something while also getting a little more information.

What version of ES and Kibana are you seeing this?
Can you attach a screenshot of the management page when you are unable to select the index pattern?

Thanks

branchnetconsulting · 2017-05-16T18:03:36Z

I saw this issue with ElasticStack 5.3.0. In Kibana, go to Management->Index Patterns. If any of the index patterns listed there have no existing indexes in Elasticsearch, then the page will not respond all to you left clicking on those patterns, making it impossible to highlight the pattern you want to delete. The only solution I am aware of is to create a matching index, at which point Kibana will let you delete the index pattern, and then you can delete the dummy index. Kevin

…

On Tue, May 16, 2017 at 1:35 PM, Chris Roberson ***@***.***> wrote: Thanks for the added information @danielmotaleite <https://github.com/danielmotaleite>. I'm still having some issues so I just want to verify something while also getting a little more information. What version of ES and Kibana are you seeing this? Can you attach a screenshot of the management page when you are unable to select the index pattern? Thanks — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#9224 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AMj33vFN8VfUPWjV-LkMAWpENt3NNpumks5r6d5kgaJpZM4K8w61> .

chrisronline · 2017-05-16T18:52:00Z

Thanks for the clarification @branchnetconsulting

I'm attempting to follow those same steps in 5.3.0 but after I manually delete the index from ES (through a curl command) and refresh Kibana, I can still see, select, and delete the index pattern. It'd helpful if I can see maybe a video or at least some screenshots of the error scenario so it might help me track down the root cause. Unfortunately, I can't find any code within Kibana that would result in the index pattern showing up in the list, but not be clickable.

branchnetconsulting · 2017-05-16T18:57:30Z

Chris, there may be some lag time involved. I recall that after creating a dummy index, it didn't actually let me delete the matching Kibana index pattern until a few minutes later, and possibly after restarting Kibana and/or Chrome. Kevin

…

On Tue, May 16, 2017 at 2:52 PM, Chris Roberson ***@***.***> wrote: Thanks for the clarification @branchnetconsulting <https://github.com/branchnetconsulting> I'm attempting to follow those same steps in 5.3.0 but after I manually delete the index from ES (through a curl command) and refresh Kibana, I can still see, select, and delete the index pattern. It'd helpful if I can see maybe a video or at least some screenshots of the error scenario so it might help me track down the root cause. Unfortunately, I can't find any code within Kibana that would result in the index pattern showing up in the list, but not be clickable. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#9224 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AMj33nyU5l4CScJWXs9ylhUhlhJtq8v7ks5r6fBegaJpZM4K8w61> .

chrisronline · 2017-05-16T19:03:55Z

@branchnetconsulting Thanks Kevin.

Just so I fully understand, you're saying that you eventually could delete the Kibana index pattern, but it just took some time before Kibana let you do it?

branchnetconsulting · 2017-05-16T19:08:42Z

Yes, I used curl to create a dummy index that matched the index pattern that previously had no matching indexes, and after a few minutes Kibana and a restart of Kibana and/or Chrome, it let me delete the Kibana index pattern.

…

On Tue, May 16, 2017 at 3:04 PM, Chris Roberson ***@***.***> wrote: @branchnetconsulting <https://github.com/branchnetconsulting> Thanks Kevin. Just so I fully understand, you're saying that you eventually could delete the Kibana index pattern, but it just took some time before Kibana let you do it? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#9224 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AMj33r4xY4K0GuhlhyU2AUekJT33puTBks5r6fMpgaJpZM4K8w61> .

chrisronline · 2017-05-16T19:09:55Z

Thanks for the clarification @branchnetconsulting.

Does this match the behavior you're seeing too @danielmotaleite?

markwalkom · 2017-05-16T20:55:40Z

I just tried this on 5.4.0 and I couldn't replicate.

danielmotaleite · 2017-05-16T22:22:55Z

I'm attempting to follow those same steps in 5.3.0 but after I manually delete the index from ES
(through a curl command) and refresh Kibana, I can still see, select, and delete the index pattern

You probably need to restart kibana and/or maybe the browser to be able to be unable to select the pattern, if you do not do that, you are still using some cached info that allow you to select it.

Just so I fully understand, you're saying that you eventually could delete the Kibana index pattern, but it just took some time before Kibana let you do it?

Yes, I used curl to create a dummy index that matched the index pattern that previously had no matching indexes,

No! you both are talking about different steps, one after the index removal, the other after the index re-creation! :)

After creating the index and the kibana pattern, if you remove the index, you probably need to restart kibana, the browser or both. After that you are unable to select the removed index pattern in kibana. As you can not select it, you can not remove it.

Then, as workaround, you have to create the index (or something that match that index pattern) and restart kibana/browser or again both, you can then select the pattern in kibana and finally remove it

chrisronline · 2017-05-17T01:57:41Z

Here is a video I took trying to reproduce the problem on 5.3.0.

https://www.youtube.com/watch?v=d1M4CNYw-BQ&feature=youtu.be

My steps:

Verify ES index exists and matching Kibana index pattern.
Delete the ES index through a curl command, and verify it doesn't exist anymore.
Restart Kibana and open a new incognito Chrome window.
Load Kibana.
Verify index pattern matches nothing on Discover section (no results found).
Load all index patterns in the Management section.
Verify the index pattern shows up in the list.
Click and load the index pattern.
Successfully delete the index pattern.

I've also tried these same steps with another existing index pattern and I'm still able to properly select and delete the orphaned index pattern.

Is there another step involved that reproduces the error?

danielmotaleite · 2017-05-17T19:32:24Z

i'm on kibana 5.3.2 and creating the index, adding the pattern, removing the index, restart kibana and browser, i can still select the pattern!! :)

But i know that is isn't fixed, as i still have one index pattern that i can not delete as the related index does not exist anymore.

So there is something else missing to trigger this issue. Try to restart the elasticsearh cluster (i can't do that on mine, as it is live) , maybe there is some index cache on the ES that can change the test? or maybe the index or pattern must be created with a older (< v5.x) elasticsearch or kibana? Or maybe the pattern must be a wildcard (logstash-*), as all my patterns have wildcards

i will wait for the nighly rotation and maintenance to see if that changes anything in my cluster.

at least i have one broken pattern, so i can do some tests there. In the browser when i press the broken pattern i get this in the browser console:

Error: No matching indices found: [index_not_found_exception] no such index, with { resource.type="index_or_alias" & 
resource.id="logstash-0016*-filebeat-nginx-error-live" & index_uuid="_na_" & index="logstash-0016*-filebeat-nginx-error-live" }
KbnError@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:63:21936
errors.IndexPatternMissingIndices@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:63:25640
handleMissingIndexPattern@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:27:14338
processQueue@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:38:23621
scheduleProcessQueue/<@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:38:23888
$eval@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:39:4607
$digest@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:39:2343
$apply@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:39:5026
done@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:37:25016
completeRequest@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:37:28702
createHttpBackend/</xhr.onload@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:37:29634
EventHandlerNonNull*createHttpBackend/<@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:37:29409
sendReq@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:37:26505
serverRequest@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:37:23333
processQueue@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:38:23621
scheduleProcessQueue/<@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:38:23888
$eval@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:39:4607
$digest@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:39:2343
$evalAsync/<@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:39:4751
completeOutstandingRequest@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:36:4607
Browser/self.defer/timeoutId<@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:36:7706
setTimeout handler*Browser/self.defer@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:36:7650
$evalAsync@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:39:4706
$QProvider/this.$get</<@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:38:22786
scheduleProcessQueue@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:38:23868
then@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:38:24896
$http@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:37:23803
AngularConnector.prototype.request@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:1:26920
CustomAngularConnector/this.request<@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:1:21735
wrapper@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:2:5945
sendReqWithConnection@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:13:1148
_.applyArgs@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:1:29624
bound@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:9:20721
Item.prototype.run@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:1:25863
drainQueue@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:1:25018
setTimeout handler*runTimeout@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:1:24003
process.nextTick@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:1:25809
_.nextTick@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:1:30113
ConnectionPool.prototype.select@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:13:13649
Transport.prototype.request@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:13:4589
exec@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:13:24024
action@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:13:20756
Mapper/self.getFieldsForIndexPattern/<@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:27:13241
processQueue@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:38:23621
scheduleProcessQueue/<@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:38:23888
$eval@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:39:4607
$digest@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:39:2343
$apply@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:39:5026
done@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:37:25016
completeRequest@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:37:28702
createHttpBackend/</xhr.onload@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:37:29634
EventHandlerNonNull*createHttpBackend/<@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:37:29409
sendReq@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:37:26505
serverRequest@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:37:23333
processQueue@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:38:23621
scheduleProcessQueue/<@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:38:23888
$eval@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:39:4607
$digest@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:39:2343
$evalAsync/<@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:39:4751
completeOutstandingRequest@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:36:4607
Browser/self.defer/timeoutId<@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:36:7706
setTimeout handler*Browser/self.defer@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:36:7650
$evalAsync@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:39:4706
$QProvider/this.$get</<@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:38:22786
scheduleProcessQueue@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:38:23868
then@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:38:24896
$http@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:37:23803
AngularConnector.prototype.request@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:1:26920
CustomAngularConnector/this.request<@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:1:21735
wrapper@https://kibana-internal.qwerty.com/bundles/commons.bundle.js?v=14849:2:5945
sendReqWithConnection@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:13:1148
_.applyArgs@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:1:29624
bound@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:9:20721
Item.prototype.run@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:1:25863
drainQueue@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:1:25018
setTimeout handler*runTimeout@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:1:24003
process.nextTick@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:1:25809
_.nextTick@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:1:30113
ConnectionPool.prototype.select@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:13:13649
Transport.prototype.request@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:13:4589
exec@https://kibana-internal.qwerty.com/bundles/kibana.bundle.js?v=14849:13:24024
  commons.bundle.js:38:11454

chrisronline · 2017-05-17T19:59:09Z

@danielmotaleite That stack trace is VERY helpful! I'm seeing some area of the code that might be causing it. I'll update when I know more

ycombinator · 2017-05-18T13:13:54Z

But i know that is isn't fixed, as i still have one index pattern that i can not delete as the related index does not exist anymore.

Hey @danielmotaleite, for this index pattern that is un-deletable, would you mind making the following request against your Elasticsearch cluster and posting its results here?

GET .kibana/index-pattern/<your-index-pattern-name>?pretty

Thanks!

danielmotaleite · 2017-05-18T13:28:08Z

Sure:

$ curl -s  -XGET 192.168.xx.xx:9200/.kibana/index-pattern/logstash-0016*-filebeat-nginx-error-live?pretty
{
  "_index" : ".kibana",
  "_type" : "index-pattern",
  "_id" : "logstash-0016*-filebeat-nginx-error-live",
  "_version" : 3,
  "found" : true,
  "_source" : {
    "title" : "logstash-0016*-filebeat-nginx-error-live",
    "timeFieldName" : "@timestamp",
    "fields" : "[{\"name\":\"input_type.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"source\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"type\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"fields.venture\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"fields.venture.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"host\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"beat.hostname.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"geoip.longitude\",\"type\":\"number\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"tags.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"offset\",\"type\":\"number\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"count\",\"type\":\"number\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"input_type\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"fields.role\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"tags\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"beat.name\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"geoip.latitude\",\"type\":\"number\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"_source\",\"type\":\"_source\",\"count\":0,\"scripted\":false,\"indexed\":false,\"analyzed\":false,\"doc_values\":false},{\"name\":\"fields.environment.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"_index\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":false,\"analyzed\":false,\"doc_values\":false},{\"name\":\"error\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"fields.type.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"geoip.ip\",\"type\":\"ip\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"error.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"fields.service\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"geoip.location\",\"type\":\"geo_point\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"@version\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"severity\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"beat.name.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"host.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"fields.type\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"type.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"fields.environment\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"fields.service.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"message\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"severity.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"beat.hostname\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":true,\"doc_values\":false},{\"name\":\"@timestamp\",\"type\":\"date\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"fields.role.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"source.raw\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":true,\"analyzed\":false,\"doc_values\":true},{\"name\":\"_id\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":false,\"analyzed\":false,\"doc_values\":false},{\"name\":\"_type\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"indexed\":false,\"analyzed\":false,\"doc_values\":false},{\"name\":\"_score\",\"type\":\"number\",\"count\":0,\"scripted\":false,\"indexed\":false,\"analyzed\":false,\"doc_values\":false}]"
  }
}

chrisronline · 2017-05-18T13:42:34Z

@danielmotaleite Thanks for providing that.

Quick question, do you recall when this index pattern was created? I have a hunch it might have been created before ES 5.0.

branchnetconsulting · 2017-05-18T14:10:51Z

All the index patterns that I could not delete were definitely created pre ES 5.0. Kevin

…

On Thu, May 18, 2017 at 9:42 AM, Chris Roberson ***@***.***> wrote: @danielmotaleite <https://github.com/danielmotaleite> Thanks for providing that. Quick question, do you recall when this index pattern was created? I have a hunch it might have been created before ES 5.0. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#9224 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AMj33lSesJA-5chcZ6Klb2gRbUqwZG6Xks5r7ErZgaJpZM4K8w61> .

danielmotaleite · 2017-05-18T15:54:01Z

the same thing here, those patterns are old, more than one year old... and not even used for a long time

That was a debug pattern for a weird bug that i had and that i forgot until i upgraded to ELK 5.0

chrisronline · 2017-05-18T16:50:58Z

Okay thanks so much for all the information @danielmotaleite and @branchnetconsulting. Please keep an eye on the PR and I'll update this ticket once the fix is ready to use.

#11885

chrisronline · 2017-05-18T19:02:07Z

As a followup, I just wanted to explain what we think happened.

The issue is the affected index patterns do not contain searchable or aggregatable fields which can happen from a couple reasons, per @Bargs

Index pattern without searchable/aggregatable is created directly in ES via an external process. This usually happens when migrating from 2.x to 5.x. If the user doesn't migrate the data indices the pattern points to, field stats will fail and Kibana will never be able to fill in the searchable/aggregatable info.

This is a newer one. If you have a geo field and you upgrade from 2.x to 5.3 your field stats request will fail due to a bug in ES. Kibana will never be able to retrieve searchable/aggregatable info.

Basically anything that causes field_stats to fail after upgrading from 2.x to 5.x will put the index pattern in a bad state. Things should hopefully be better once we switch to the field_caps API.

Hopefully this can help identify potentially stale index patterns that need their fields refreshed in your cluster(s) but the fix will now allow you to properly delete orphaned index patterns.

danielmotaleite · 2017-05-19T13:59:56Z

Thanks for the update and the fix!

by the way, what happens when one press "refresh fields list" on one of those patterns? as there is no index, i assume it would simply fail and report one error and not open a new kind of bug :)

chrisronline · 2017-05-19T14:07:50Z

@danielmotaleite That's right. The behavior will mostly match when you're seeing now, but it won't break the page. It will just report the same error in the notification section at the top.

soulesjms · 2019-03-06T18:35:50Z

Might be a bit late to the conversation but this may help those who need to manually delete index patterns.
To get the doc id of index pattern:
curl -XGET ":9200/.kibana/index-pattern/"
then:
curl -XDELETE ":9200/.kibana/index-pattern/"
then restart kibana, and index-pattern should be gone.

soulesjms · 2019-03-06T18:41:51Z

curl -XGET "host:9200/.kibana/index-pattern/index-pattern"

zazi · 2019-03-20T09:21:43Z

for kibana/es 6.3.x (and above) it should be
curl -XGET ":9200/.kibana/doc/_search?q=index-pattern.title:[INDEX-PATTERN-NAME]"
to get the doc id of the index pattern ([INDEX-PATTERN-NAME] should be replaced with the real index pattern name)
then
curl -XDELETE ":9200/.kibana/doc/[INDEX-PATTERN-ID]/"
to delete index pattern ([INDEX-PATTERN-ID] should be replaced with the real index pattern id)

a restart of kibana is not really required (but a browser refresh maybe)

markwalkom added :Management bug Fixes for quality problems that affect the customer experience labels Nov 26, 2016

w33ble added the good first issue low hanging fruit label Dec 29, 2016

w33ble mentioned this issue Dec 29, 2016

If user tries to refresh a deleted index pattern in management, Kibana should warn the user instead of throwing 404 in console. #9624

Closed

BigFunger self-assigned this Jan 6, 2017

spalger mentioned this issue Apr 4, 2017

Switch to field_caps API #11014

Closed

spalger mentioned this issue Apr 11, 2017

[IndexPatterns] Support cross cluster search #11114

Merged

chrisronline assigned chrisronline and unassigned BigFunger May 15, 2017

chrisronline mentioned this issue May 18, 2017

Prevent refresh fields error from breaking index patterns management page #11885

Merged

chrisronline closed this as completed in #11885 May 18, 2017

eherot mentioned this issue Oct 11, 2019

No obvious way to delete an index pattern that matches 0 indices #48019

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unable to delete patterns if the indices don't exist #9224

Unable to delete patterns if the indices don't exist #9224

markwalkom commented Nov 26, 2016

danielmotaleite commented Jan 5, 2017

ycombinator commented Jan 10, 2017

burnstreet commented Feb 16, 2017 •

edited

branchnetconsulting commented May 3, 2017

chrisronline commented May 16, 2017

danielmotaleite commented May 16, 2017

chrisronline commented May 16, 2017

branchnetconsulting commented May 16, 2017 via email

chrisronline commented May 16, 2017

branchnetconsulting commented May 16, 2017 via email

chrisronline commented May 16, 2017

branchnetconsulting commented May 16, 2017 via email

chrisronline commented May 16, 2017

markwalkom commented May 16, 2017

danielmotaleite commented May 16, 2017

chrisronline commented May 17, 2017

danielmotaleite commented May 17, 2017

chrisronline commented May 17, 2017

ycombinator commented May 18, 2017 •

edited

danielmotaleite commented May 18, 2017

chrisronline commented May 18, 2017

branchnetconsulting commented May 18, 2017 via email

danielmotaleite commented May 18, 2017

chrisronline commented May 18, 2017

chrisronline commented May 18, 2017

danielmotaleite commented May 19, 2017

chrisronline commented May 19, 2017

soulesjms commented Mar 6, 2019

soulesjms commented Mar 6, 2019

zazi commented Mar 20, 2019

Unable to delete patterns if the indices don't exist #9224

Unable to delete patterns if the indices don't exist #9224

Comments

markwalkom commented Nov 26, 2016

danielmotaleite commented Jan 5, 2017

ycombinator commented Jan 10, 2017

burnstreet commented Feb 16, 2017 • edited

branchnetconsulting commented May 3, 2017

chrisronline commented May 16, 2017

danielmotaleite commented May 16, 2017

chrisronline commented May 16, 2017

branchnetconsulting commented May 16, 2017 via email

chrisronline commented May 16, 2017

branchnetconsulting commented May 16, 2017 via email

chrisronline commented May 16, 2017

branchnetconsulting commented May 16, 2017 via email

chrisronline commented May 16, 2017

markwalkom commented May 16, 2017

danielmotaleite commented May 16, 2017

chrisronline commented May 17, 2017

danielmotaleite commented May 17, 2017

chrisronline commented May 17, 2017

ycombinator commented May 18, 2017 • edited

danielmotaleite commented May 18, 2017

chrisronline commented May 18, 2017

branchnetconsulting commented May 18, 2017 via email

danielmotaleite commented May 18, 2017

chrisronline commented May 18, 2017

chrisronline commented May 18, 2017

danielmotaleite commented May 19, 2017

chrisronline commented May 19, 2017

soulesjms commented Mar 6, 2019

soulesjms commented Mar 6, 2019

zazi commented Mar 20, 2019

burnstreet commented Feb 16, 2017 •

edited

ycombinator commented May 18, 2017 •

edited