Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to change the index pattern on a visualization #17542

Closed
stacey-gammon opened this issue Apr 4, 2018 · 33 comments
Closed

Ability to change the index pattern on a visualization #17542

stacey-gammon opened this issue Apr 4, 2018 · 33 comments

Comments

@stacey-gammon
Copy link
Contributor

@stacey-gammon stacey-gammon commented Apr 4, 2018

Similar to how discover does it. Rather than requiring a new visualization to be built, add a field when editing a visualization that lets you change the index pattern on it. We need to figure out how to handle errors when a visualization was built with fields from the original index that don't exist in the new one.

Splitting this particular part out of this issue: #3668 which includes both dashboards and visualizations. Changing the index pattern on a dashboard is a much different problem and is now covered by this issue: #16917

It will be helpful for us to split these issues out and see which are the most important to the community.

also related: #16831

@ReanimationXP
Copy link

@ReanimationXP ReanimationXP commented Apr 9, 2018

I believe the visualization index pattern can already be changed manually by changing its' index guid, but it's extremely non user-friendly considering the compounding issues of (a) those index pattern guids are conveniently not available in the object manager / browser, (b) there is no simple dropdown to change it from within the visualization itself, and perhaps most importantly (c) there is no way to bulk-change multiple visualizations.

@conmurray
Copy link

@conmurray conmurray commented May 29, 2018

I'll add my request here as well. Im a GOLD licence and Ill raise this with my support engineer but I wanted to pop it here aswell.

  • User created a visualization against an index called "logstash_test_metrics"
  • We then deleted "logstash_test_metrics" and the index pattern for it aswell.
  • We needed to modify the logstash conf file and so re ran the logstash conf file to re-add the logstash_test_metrics index
  • We then created the index pattern for this "logstash_test_metrics"
  • We'd go to the visualization to be told the underlying index is not there. Actually, it is but because the kibanaSavedObjectMeta.searchSourceJSON now references the index by an ID and not by the actual index name, everything is broken. I have to either recreate the visualization or I have to find the ID for the new index and copy it into the kibanaSavedObjectMeta.searchSourceJSON. This isnt really feasible when I've a LOT of visualizations that all reference the same index.

@conmurray
Copy link

@conmurray conmurray commented Jun 7, 2018

Why was this closed? Also the Switch Index on Visualization is also closed so what's the story here? I've referenced this in my Internal Elastic Support case as well.

@stacey-gammon
Copy link
Contributor Author

@stacey-gammon stacey-gammon commented Jun 7, 2018

This issue isn't closed @conmurray. Switch Index on Visualization was closed because it's a duplicate of this one. Am I overlooking something else?

Regarding your use case above, it almost seems like #16831 is a better fit for it. Just being able to change the index pattern on one visualization in the UI would still require you to open each visualization and change it. If we had an export API that included linked objects you would be able to open the file, do a bulk find/replace, then re-import the file.

@conmurray
Copy link

@conmurray conmurray commented Jun 7, 2018

Ah yes I see that. Thanks... @stacey-gammon
With respect to #16831 it doesnt seem to support my particular use case. The fact that you're using ID and not the actual index name is the problem here. Ill add this to the #16831

@PritomAhmed
Copy link

@PritomAhmed PritomAhmed commented Aug 16, 2018

+1

2 similar comments
@Razikus
Copy link

@Razikus Razikus commented Aug 24, 2018

+1

@xingfanxia
Copy link

@xingfanxia xingfanxia commented Sep 11, 2018

+1

@lambdamusic
Copy link

@lambdamusic lambdamusic commented Sep 20, 2018

+1
As a side note, I noticed the the index ID in the kibanaSavedObjectMeta does not match any of the indexes I have in elasticsearch. Looks like the ID is created by Kibana when an 'index pattern' gets generated.
Is that right? If so how what's the best way to resolve these mappings?

@yami12376
Copy link

@yami12376 yami12376 commented May 23, 2019

+1

@smerzlyakov
Copy link

@smerzlyakov smerzlyakov commented Jun 21, 2019

+1 really need this feature. Or even manually change index pattern.

@nerophon
Copy link
Contributor

@nerophon nerophon commented Jun 28, 2019

+1

@canturkisci
Copy link

@canturkisci canturkisci commented Jul 18, 2019

+1 on this please! This is tedious to recover when we have repeat deploys and many dashboards stop working due to the mismatch in index patterns. Intuitively we should be able to point a visualization to an index pattern name, and say "try this instead".

@louieamone
Copy link

@louieamone louieamone commented Jul 22, 2019

+1. It would be really great to have a more intuitive and efficient way to do this!

@duffar12
Copy link

@duffar12 duffar12 commented Aug 29, 2019

+1

1 similar comment
@aviadbi1
Copy link

@aviadbi1 aviadbi1 commented Sep 2, 2019

+1

@r33bl
Copy link

@r33bl r33bl commented Sep 17, 2019

+1

3 similar comments
@rajesh212
Copy link

@rajesh212 rajesh212 commented Oct 14, 2019

+1

@ranjitmw
Copy link

@ranjitmw ranjitmw commented Oct 14, 2019

+1

@0asp0
Copy link

@0asp0 0asp0 commented Nov 19, 2019

+1

@jkamdjou
Copy link

@jkamdjou jkamdjou commented Nov 21, 2019

For those in search of a temporary solution to restore broken visualizations due to deletion of an index pattern, the following worked for me (Kibana 7.4.2):

  1. Navigate to a broken visualization, and take note of the broken index pattern id. It will be in the URL string in the form index:'<uuid>'.
  2. Create a new index pattern from Management->Kibana->Index Patterns.
  3. Export the index pattern:
    curl -X POST 'http://localhost:5601/api/saved_objects/_export' -H "kbn-xsrf: true" -H "Content-Type: application/json" -d '{"type": "index-pattern"}' > new_index_pattern.ndjson
  4. Modify this file and change the top level id to your old index pattern id.
  5. Delete the new index pattern you just created in Kibana.
  6. Import the index pattern, which now specifies your old index pattern id:
    curl -X POST "localhost:5601/api/saved_objects/_import" -H "kbn-xsrf: true" --form file=@new_index_pattern.ndjson

Your visualizations should now work again.

Further documentation on the import/export APIs can be found here.

@craigboman
Copy link

@craigboman craigboman commented Jan 6, 2020

I don't think we need to export the index pattern in order to update a broken visualization. From the list of saved objects in advanced settings, inspect the index pattern using the "..." menu. Viewing the index pattern copy the index pattern id from the url string and paste that into the broken visualization edit window. To paste into the visualization, go to advanced settings->saved objects, then inspect the object (... menu), which should open edit search/visualization. Anytime the visualization is referencing the broken index pattern, replace it with the new index pattern id. I've done this loads of times, works great. I would consider this issue request completed.

@jkamdjou
Copy link

@jkamdjou jkamdjou commented Jan 6, 2020

I tried that before resorting to the steps I outlined above. That did not work for me, got an error.

@arsalankhalid
Copy link

@arsalankhalid arsalankhalid commented Jan 20, 2020

Because of how un-intuitive Kibana visual editors out, after some time I was able to figure out how to fetch the ID. Basically, create a "new" visual with the right index, then go to Saved Objects, click on that new visual you created - and go into "edit" or advanced edit. Where you should see a set of jsons, under references you'll see an ID which shows the new index you want. Now go into the visuals that are breaking, and paste the correct ID in the advanced edit for that visual (inside saved objects, under Management)

@csawtelle
Copy link

@csawtelle csawtelle commented Apr 16, 2020

+1

1 similar comment
@PieroValdebenito
Copy link

@PieroValdebenito PieroValdebenito commented Jun 18, 2020

+1

@piehld
Copy link

@piehld piehld commented Aug 25, 2020

An easier way to change the index pattern being used for a visualization (and across a dashboard) would be extremely valuable, especially in my use case of cloning the same dashboard (of 6+ visualizations) for several different index patterns, each representing a different microservice in our architecture.

Regarding a workaround for people experiencing a dashboard not being able to recognize or locate a particular index pattern, what I found to work the easiest was to just export the whole dashboard from the saved objects list (including all child objects), and then drag and drop that file back onto the 'Import' panel of the saved objects list. The import process will actually ask you to select the correct index pattern to use for each visualization if there are any conflicts or non-matches. Note that this will overwrite the saved objects of the "broken" dashboard you just exported.

That said, I wonder why this functionality of the import feature can't be extended to the feature that this issue is trying to address. It essentially allows you to re-set the index pattern for all visualizations of a dashboard by re-importing it, but only if pattern conflicts or non-matches arise. So, if you were to get rid of those conditions and just require that a new index pattern to be specified every time--and perhaps a new suffix to append to every saved object name, so as to not overwrite--that would accomplish what I believe a lot of users here are voting for.

@sebastian-bugajny
Copy link

@sebastian-bugajny sebastian-bugajny commented Oct 13, 2020

+1

@KrishnaChaitanya-Gopaluni
Copy link

@KrishnaChaitanya-Gopaluni KrishnaChaitanya-Gopaluni commented Oct 19, 2020

From Stack Management > Saved Objects I've replaced index id for my visualization. It worked.

@timroes
Copy link
Contributor

@timroes timroes commented Nov 26, 2020

This enhancement is implemented in Lens since the first release. Lens GA’d in 7.10 and going forward Lens is the recommended way in Kibana to create visualizations, with more chart types, data processing and integrations planned (see this Lens meta issue).

Since this feature is not a good fit for the current Visualize Editor architecture and thus would require major refactoring, we decided to not implement this outside of Lens and thus closing this issue with the implementation in Lens now.

@timroes timroes closed this Nov 26, 2020
@th0ger
Copy link

@th0ger th0ger commented Dec 7, 2020

@KrishnaChaitanya-Gopaluni I don't see Saved Objects under Stack Management. Has it been moved, removed, renamed?
Stack Management 7.10.0.

Edit: It was hidden in Manage Spaces > Features.

@KrishnaChaitanya-Gopaluni

I

@KrishnaChaitanya-Gopaluni I don't see Saved Objects under Stack Management. Has it been moved, removed, renamed?
Stack Management 7.10.0.

@th0ger My Stack Management is 7.9.3. I can see the saved objects at the bottom left. Seems like 7.10 too has one https://www.elastic.co/guide/en/kibana/current/managing-saved-objects.html

@payne747
Copy link

@payne747 payne747 commented Jan 9, 2021

+1
Am new to Kibana and just learning, but I've found a need to do this several times already with no easy method. Would very much welcome it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet