Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to change the index pattern on a visualization #17542

Closed
stacey-gammon opened this issue Apr 4, 2018 · 33 comments
Closed

Ability to change the index pattern on a visualization #17542

stacey-gammon opened this issue Apr 4, 2018 · 33 comments
Labels
Feature:Vis Editor Visualization editor issues release_note:enhancement Team:Visualizations Visualization editors, elastic-charts and infrastructure

Comments

@stacey-gammon
Copy link
Contributor

Similar to how discover does it. Rather than requiring a new visualization to be built, add a field when editing a visualization that lets you change the index pattern on it. We need to figure out how to handle errors when a visualization was built with fields from the original index that don't exist in the new one.

Splitting this particular part out of this issue: #3668 which includes both dashboards and visualizations. Changing the index pattern on a dashboard is a much different problem and is now covered by this issue: #16917

It will be helpful for us to split these issues out and see which are the most important to the community.

also related: #16831

@stacey-gammon stacey-gammon added Feature:Visualizations Generic visualization features (in case no more specific feature label is available) enhancement New value added to drive a business result labels Apr 4, 2018
@timroes timroes added :New Editor release_note:enhancement and removed enhancement New value added to drive a business result labels Apr 4, 2018
@ReanimationXP
Copy link

ReanimationXP commented Apr 9, 2018

I believe the visualization index pattern can already be changed manually by changing its' index guid, but it's extremely non user-friendly considering the compounding issues of (a) those index pattern guids are conveniently not available in the object manager / browser, (b) there is no simple dropdown to change it from within the visualization itself, and perhaps most importantly (c) there is no way to bulk-change multiple visualizations.

@conmurray
Copy link

I'll add my request here as well. Im a GOLD licence and Ill raise this with my support engineer but I wanted to pop it here aswell.

  • User created a visualization against an index called "logstash_test_metrics"
  • We then deleted "logstash_test_metrics" and the index pattern for it aswell.
  • We needed to modify the logstash conf file and so re ran the logstash conf file to re-add the logstash_test_metrics index
  • We then created the index pattern for this "logstash_test_metrics"
  • We'd go to the visualization to be told the underlying index is not there. Actually, it is but because the kibanaSavedObjectMeta.searchSourceJSON now references the index by an ID and not by the actual index name, everything is broken. I have to either recreate the visualization or I have to find the ID for the new index and copy it into the kibanaSavedObjectMeta.searchSourceJSON. This isnt really feasible when I've a LOT of visualizations that all reference the same index.

@conmurray
Copy link

Why was this closed? Also the Switch Index on Visualization is also closed so what's the story here? I've referenced this in my Internal Elastic Support case as well.

@stacey-gammon
Copy link
Contributor Author

This issue isn't closed @conmurray. Switch Index on Visualization was closed because it's a duplicate of this one. Am I overlooking something else?

Regarding your use case above, it almost seems like #16831 is a better fit for it. Just being able to change the index pattern on one visualization in the UI would still require you to open each visualization and change it. If we had an export API that included linked objects you would be able to open the file, do a bulk find/replace, then re-import the file.

@conmurray
Copy link

Ah yes I see that. Thanks... @stacey-gammon
With respect to #16831 it doesnt seem to support my particular use case. The fact that you're using ID and not the actual index name is the problem here. Ill add this to the #16831

@PritomAhmed
Copy link

+1

2 similar comments
@Razikus
Copy link

Razikus commented Aug 24, 2018

+1

@xingfanxia
Copy link

+1

@timroes timroes added the Team:Visualizations Visualization editors, elastic-charts and infrastructure label Sep 16, 2018
@lambdamusic
Copy link

+1
As a side note, I noticed the the index ID in the kibanaSavedObjectMeta does not match any of the indexes I have in elasticsearch. Looks like the ID is created by Kibana when an 'index pattern' gets generated.
Is that right? If so how what's the best way to resolve these mappings?

@timroes timroes added Feature:Vis Editor Visualization editor issues and removed Feature:Visualizations Generic visualization features (in case no more specific feature label is available) labels Oct 27, 2018
@yami12376
Copy link

+1

@smerzlyakov
Copy link

+1 really need this feature. Or even manually change index pattern.

@nerophon
Copy link
Contributor

+1

@canturkisci
Copy link

+1 on this please! This is tedious to recover when we have repeat deploys and many dashboards stop working due to the mismatch in index patterns. Intuitively we should be able to point a visualization to an index pattern name, and say "try this instead".

@louieamone
Copy link

+1. It would be really great to have a more intuitive and efficient way to do this!

@duffar12
Copy link

+1

1 similar comment
@aviadbi1
Copy link

aviadbi1 commented Sep 2, 2019

+1

@r33bl
Copy link

r33bl commented Sep 17, 2019

+1

3 similar comments
@rajesh212
Copy link

+1

@ranjitmw
Copy link

+1

@0asp0
Copy link

0asp0 commented Nov 19, 2019

+1

@jkamdjou
Copy link

For those in search of a temporary solution to restore broken visualizations due to deletion of an index pattern, the following worked for me (Kibana 7.4.2):

  1. Navigate to a broken visualization, and take note of the broken index pattern id. It will be in the URL string in the form index:'<uuid>'.
  2. Create a new index pattern from Management->Kibana->Index Patterns.
  3. Export the index pattern:
    curl -X POST 'http://localhost:5601/api/saved_objects/_export' -H "kbn-xsrf: true" -H "Content-Type: application/json" -d '{"type": "index-pattern"}' > new_index_pattern.ndjson
  4. Modify this file and change the top level id to your old index pattern id.
  5. Delete the new index pattern you just created in Kibana.
  6. Import the index pattern, which now specifies your old index pattern id:
    curl -X POST "localhost:5601/api/saved_objects/_import" -H "kbn-xsrf: true" --form file=@new_index_pattern.ndjson

Your visualizations should now work again.

Further documentation on the import/export APIs can be found here.

@craigboman
Copy link

I don't think we need to export the index pattern in order to update a broken visualization. From the list of saved objects in advanced settings, inspect the index pattern using the "..." menu. Viewing the index pattern copy the index pattern id from the url string and paste that into the broken visualization edit window. To paste into the visualization, go to advanced settings->saved objects, then inspect the object (... menu), which should open edit search/visualization. Anytime the visualization is referencing the broken index pattern, replace it with the new index pattern id. I've done this loads of times, works great. I would consider this issue request completed.

@jkamdjou
Copy link

jkamdjou commented Jan 6, 2020

I tried that before resorting to the steps I outlined above. That did not work for me, got an error.

@arsalankhalid
Copy link

Because of how un-intuitive Kibana visual editors out, after some time I was able to figure out how to fetch the ID. Basically, create a "new" visual with the right index, then go to Saved Objects, click on that new visual you created - and go into "edit" or advanced edit. Where you should see a set of jsons, under references you'll see an ID which shows the new index you want. Now go into the visuals that are breaking, and paste the correct ID in the advanced edit for that visual (inside saved objects, under Management)

@csawtelle
Copy link

+1

1 similar comment
@PieroValdebenito
Copy link

+1

@piehld
Copy link

piehld commented Aug 25, 2020

An easier way to change the index pattern being used for a visualization (and across a dashboard) would be extremely valuable, especially in my use case of cloning the same dashboard (of 6+ visualizations) for several different index patterns, each representing a different microservice in our architecture.

Regarding a workaround for people experiencing a dashboard not being able to recognize or locate a particular index pattern, what I found to work the easiest was to just export the whole dashboard from the saved objects list (including all child objects), and then drag and drop that file back onto the 'Import' panel of the saved objects list. The import process will actually ask you to select the correct index pattern to use for each visualization if there are any conflicts or non-matches. Note that this will overwrite the saved objects of the "broken" dashboard you just exported.

That said, I wonder why this functionality of the import feature can't be extended to the feature that this issue is trying to address. It essentially allows you to re-set the index pattern for all visualizations of a dashboard by re-importing it, but only if pattern conflicts or non-matches arise. So, if you were to get rid of those conditions and just require that a new index pattern to be specified every time--and perhaps a new suffix to append to every saved object name, so as to not overwrite--that would accomplish what I believe a lot of users here are voting for.

@sebastian-bugajny
Copy link

+1

@KrishnaChaitanya-Gopaluni
Copy link

From Stack Management > Saved Objects I've replaced index id for my visualization. It worked.

@timroes
Copy link
Contributor

timroes commented Nov 26, 2020

This enhancement is implemented in Lens since the first release. Lens GA’d in 7.10 and going forward Lens is the recommended way in Kibana to create visualizations, with more chart types, data processing and integrations planned (see this Lens meta issue).

Since this feature is not a good fit for the current Visualize Editor architecture and thus would require major refactoring, we decided to not implement this outside of Lens and thus closing this issue with the implementation in Lens now.

@timroes timroes closed this as completed Nov 26, 2020
@th0ger
Copy link

th0ger commented Dec 7, 2020

@KrishnaChaitanya-Gopaluni I don't see Saved Objects under Stack Management. Has it been moved, removed, renamed?
Stack Management 7.10.0.

Edit: It was hidden in Manage Spaces > Features.

@KrishnaChaitanya-Gopaluni

I

@KrishnaChaitanya-Gopaluni I don't see Saved Objects under Stack Management. Has it been moved, removed, renamed?
Stack Management 7.10.0.

@th0ger My Stack Management is 7.9.3. I can see the saved objects at the bottom left. Seems like 7.10 too has one https://www.elastic.co/guide/en/kibana/current/managing-saved-objects.html

@payne747
Copy link

payne747 commented Jan 9, 2021

+1
Am new to Kibana and just learning, but I've found a need to do this several times already with no easy method. Would very much welcome it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Vis Editor Visualization editor issues release_note:enhancement Team:Visualizations Visualization editors, elastic-charts and infrastructure
Projects
None yet
Development

No branches or pull requests