Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom drilldown links for a dashboard panel #12560

Open
alexfrancoeur opened this issue Jun 29, 2017 · 44 comments

Comments

Projects
None yet
@alexfrancoeur
Copy link
Contributor

commented Jun 29, 2017

Why this is important

The ability to drill down from one dashboard to another view creates a custom workflow for troubleshooting and analyzing data. When speaking with experienced users, this is a fairly common ask. When speaking with new users, the expectation is that the behavior is already available within Kibana.

What we have in mind

In the definition of a dashboard panel, we will allow a user to define one or multiple custom drilldown links that can be easily accessed from that dashboard panel. Clicking into this drilldown link will navigate users to the preconfigured view.

Example custom workflows

  • Dashboard for overall status of multiple data centers ➡️ Drill down to dashboard for a single data center ➡️ Drill down to a dashboard of a single server
  • Dashboard with visualization for overall HTTP responses ➡️ Drill down to saved search for HTTP responses with filters from previous dashboard
  • Operational dashboard panel ➡️ drill down to internal ticketing system

Proposed approach

Users should be able to navigate to the following:

  • Kibana saved objects
    • Dashboard
    • Visualization
    • Discover saved search
  • Absolute URLs
    • Any internal or external URL a user defines

For each drilldown navigation configured, each should have the following options:

  • Kibana saved objects
    • Option to inherit state from previous dashboard. This would include filters, query, timeframe, etc. This setting would be enabled by default but if not selected, we'd use the saved objects originally configured state
    • Allow for additional custom static filters, query or timeframe that are specific to that drilldown link. This would allow for re-use of saved objects with filter's explicitly applied only for that drilldown link
  • All drilldown links
    • Configurable navigation - direct navigation (default), open in new tab, open in new window

Initial concept

panel editing and links

  • Configuration would be done per panel and the drilldown links will be stored with the dashboard panel, not the visualization. This will allow for re-use of visualizations on multiple dashboards with different drilldown links.
  • Links will be seen only on hover as to not affect reporting or visualization views
  • A user can add as many links to a panel as they'd like
  • Some future concepts that may compliment this new capability include a "Copy panel to other dashboard" option and a standalone "navigation panel" for links defined in that dashboard.

Related issues and PR's

cc: @stacey-gammon @snide @thomasneirynck @kobelb

@trevan

This comment has been minimized.

Copy link
Contributor

commented Jun 29, 2017

It would also be nice to be able to drill down and add a dynamic filter based on a click. This would be for where I have a table/line that shows multiple servers and high level values. I'd like to click on the server and then go to a different dashboard to see details for that specific server. Looking at your mockups and description, it sounds like I would need to first click on the table/line for that server to add it as a filter, wait for the query to load (since the visualization panel will gray out), then click on these new links to drill down.

Also, it kind of fixes the click handler metric visualization but not entirely since that issue is more about getting metric visualization on par with the rest of the visualizations where you can add a filter based on the data in the visualization. These links might get it part of the way but it will still be kind of weird since most visualizations allow clicking on the data to filter while metric doesn't.

@snide

This comment has been minimized.

Copy link
Contributor

commented Jun 29, 2017

Here's another design, but for what a "Dashboard link menu" would look like. Essentially same UI as the panel drilldowns in the above design, but rather than having a visualization, the view mode for this is just a list of exposed links. Figured I'd through it in here since the concepts are so similar and will likely reuse a lot of the same code.

image

@source-ram

This comment has been minimized.

Copy link

commented Jul 11, 2017

+1

@alexfrancoeur

This comment has been minimized.

Copy link
Contributor Author

commented Jul 11, 2017

@trevan you are correct, in the proposed approach above you would need to click into a filter and then navigate to the URL. This implementation requires the least amount of change to the current users workflow. However, I think both approaches could be beneficial as having links that are unrelated to a filter may not necessarily need to be accessed via a data point. From a user experience perspective, what would your expectation be? Some thoughts below

  • Click into a data point
    • If one or multiple drilldowns defined, show tooltip ➡️ options include filter entire dashboard or navigate to new view(s) / url(s) with filter applied
  • If no drilldowns defined, don't show tools tip ➡️ filter entire dashboard
  • Click into a data point
    • If drilldown defined, immediately navigate to new view with filter applied
    • If no drilldown defined, apply filter globally

Regarding the metric visualization. If you were able to define a link with pre-defined filters, you'd technically be able to provide one or multiple drilldowns with that metric pre-filtered. However, I agree - it's not exactly the same as clicking into the metric to navigate.

@seanziee

This comment has been minimized.

Copy link

commented Sep 20, 2017

I think this is a great idea, especially because most users need to integrate kibana with several other websites/programs that their system is connected to. +1

@RomainXie

This comment has been minimized.

Copy link

commented Oct 12, 2017

We need this feature just yet.
Suggest to add a option in the dashboard design, allow a internal URL which can transfer the query string & filter to the next automatically.

@stacey-gammon

This comment has been minimized.

Copy link
Contributor

commented Oct 25, 2017

@snide - as I was implementing the mocks I came across a situation that might crop up with the way we do the panel flipping for the editing screen.

When the panel is really small, the link editing space will be really small too, and it's a little weird to make the user expand it just to edit links, and then have to resize it small again so the dashboard goes back to the way they want it.

I like the panel flipping idea, but this might be a blocker. Sounds like our other option is just using a modal?

Here are a couple screenshots that show my concern. Obviously the contents of the add/edit links panel will be a lot nicer, but you can get the idea that it will all be very squished.

screen shot 2017-10-25 at 4 38 20 pm

screen shot 2017-10-25 at 4 38 30 pm

I just thought of something else. The location of the link icon that shows up on hover is in the same spot as the location of the spy toggle. The spy toggle is internal to the visualization - it's not added by dashboard. So if we want to add a clickable panel icon on top of the actual visualization space, we can't make any assumptions about what it is hovering over (anyone can write anything to be an embeddable and put stuff there - so just moving the spy toggle may not be a sufficient solution).

We could add a footer to the panel that doesn't overlap the vis, but people might be unhappy with the added space it takes up. We could have it pop out on hover, below the visualization. Perhaps if we do that and make it "floatable" it'll work - shouldn't matter if the footer floats over a visualization below it, because the mouse is in the top visualization so the user isn't possibly trying to click something and can't get to it.

@trevan

This comment has been minimized.

Copy link
Contributor

commented Oct 25, 2017

@stacey-gammon, is it going to be possible for the same visualization to have different "links" depending on the dashboard it is in? If not, why not leave the editing of the links in the visualization editor? It feels weird to edit a visualization specific item inside of the dashboard.

Also, why not put the "link icon" in the next to the "expand panel" icon? Or always show the dropdown in the top left that shows up in edit mode and have 'expand panel' and 'links' be in that menu? You could even move the spy link there as well.

And are these links going to be dynamic per my comments above? Will I be able to click/hover/interact on a row/bar/line/etc and get a link that includes the data that I'm interacting with?

@stacey-gammon

This comment has been minimized.

Copy link
Contributor

commented Oct 25, 2017

@trevan

is it going to be possible for the same visualization to have different "links" depending on the dashboard it is in?
Yes

If not, why not leave the editing of the links in the visualization editor? It feels weird to edit a visualization specific item inside of the dashboard.
Another reason we went this route is because we wanted it to be a feature that any "embeddable", not just visualizations. For instance, we have visualizations and saved searches. Without making it part of the dashboard panel, you'd have to implement the same logic in a visualization and a saved search.

This topic of a pluggable dashboard embeddable vs a pluggable visualization type is somewhat complicated (and probably off topic for this specific feature). We have gone back and forth on whether everything should be implemented as a visualization plugin, or whether we should allow a more flexible "embeddable" plugin. We went with the later, though jury is still out on whether that's the best route. For now, with all the refactoring our visualize code was going through, it was easier to extract saved search logic out of dashboard code with a new embeddable plugin rather than implement it as a visualization plugin.

Also, why not put the "link icon" in the next to the "expand panel" icon? Or always show the dropdown in the top left that shows up in edit mode and have 'expand panel' and 'links' be in that menu? You could even move the spy link there as well.

I think this is a good option! Spy toggle is part of the embeddable so dashboard doesn't know anything about it. We could consider moving it out but it not sure we want to assume every embeddable will have a spy option.

And are these links going to be dynamic per my comments above? Will I be able to click/hover/interact on a row/bar/line/etc and get a link that includes the data that I'm interacting with?
No. You create the links up front, but after that they are static. Having the links be part of the hovering options on a visualization would be a visualization specific feature. I think this would be a useful next step, but we'd have to reconcile it with how we do filtering now.

So I think for your use case, you probably want visualization specific drill down links that are customizable on the visualization itself, and are specific to the type of visualization shown, so clicking might not filter, but would open up a new object. Perhaps we should have two, separate issues to cover both those use cases.

@trevan

This comment has been minimized.

Copy link
Contributor

commented Oct 25, 2017

@stacey-gammon, yeah I've been following the external discussions about embedded options. I also re-read the description here and see where it said this should be stored on the dashboard state instead of the visualization state.

So I think for your use case, you probably want visualization specific drill down links that are customizable on the visualization itself, and are specific to the type of visualization shown, so clicking might not filter, but would open up a new object. Perhaps we should have two, separate issues to cover both those use cases.

They don't have to be visualization specific drill down links. If you look at the "related issues" in the description, almost all of them talk about clicking on something in the visualization/search/etc that then takes them to a different dashboard/visualization/search filtered based off of what they just clicked on. These aren't links that you define before hand but links that are dynamic based off of the data currently shown in the visualization/search.

@stacey-gammon

This comment has been minimized.

Copy link
Contributor

commented Oct 26, 2017

@trevan

They don't have to be visualization specific drill down links.

I don't know if I follow this. What I mean to say is that those types of links would have to be configured during creation of the visualization. For instance your request to make a data table cell clickable to a dynamic link by inserting the value you clicked into a url template - that is something that would be very specific to data tables, and not applicable for a bar chart.

I've thought a bit about whether we could generalize this somehow at the dashboard level by having embeddables send their filtering through dashboard, which could then intercept it and rather than add a filter, it would redirect the user to a link, stuffing the filter into a url template defined at the panel level. I'm not convinced this is the right solution though because it requires more coupling of the embeddable to the dashboard, and it'd still be possible for someone to write an embeddable and apply the filter directly.

@snide
Some more questions regarding the UI/UX flow:
screen shot 2017-10-25 at 7 15 17 pm

  • What happens if I click Add link more than once? Do I get additional input lines, or should I be forced to finish, or delete the one I just started first?
  • The saved object list might be too simple. I think we need to distinguish between types (saved searches, visualizations and dashboards). Kind of like the listing pages, but showing all saved objects (except some I suppose - probably not index patterns). Also brings up a question of whether we should show saved objects added by plugins. Probably, but we need for all of them to be able to give us a way to construct a navigation url. I'm thinking the control might be too small for all our needs? We could add in a listing table minus the check boxes, but not sure how that would fit in the pop up.
  • After I click Add link but before I select a saved object, should we give the user a way to cancel and remove the newly added link? Otherwise I think they have to pick an object then remove it via the edit tool? What if there are no objects to pick, how would a user remove the line then?
@trevan

This comment has been minimized.

Copy link
Contributor

commented Oct 26, 2017

@stacey-gammon

They don't have to be visualization specific drill down links.

I don't know if I follow this. What I mean to say is that those types of links would have to be configured during creation of the visualization. For instance your request to make a data table cell clickable to a dynamic link by inserting the value you clicked into a url template - that is something that would be very specific to data tables, and not applicable for a bar chart.

I was just trying to say that if links will be per visualization per dashbard (which is what I gathered from your response), then these "dynamic" links also can be per visualization per dashboard. You would just edit them in the dashboard and then a visualization can have different dynamic links depending on the dashboard it is in. That is what it sounds like is going to happen for the "static" links that you are currently designing.

I've thought a bit about whether we could generalize this somehow at the dashboard level by having embeddables send their filtering through dashboard, which could then intercept it and rather than add a filter, it would redirect the user to a link, stuffing the filter into a url template defined at the panel level. I'm not convinced this is the right solution though because it requires more coupling of the embeddable to the dashboard, and it'd still be possible for someone to write an embeddable and apply the filter directly.

I've actually thought about adding that "intercept" ability so that plugins can hook into it and do exactly what you said.

What would really be nice is that instead of causing a filter when you click on an item (either it is a visualize table row, a search table row, a visualize bar chart, or a different embeddable thing), a context menu appears. This context menu then gives you the option of "filtering" (with both a positive and negative option), go to a link, create watch, create ml job, etc (all of these actions are defined by the embeddable or by plugins). Dashboard creators would then specify what the link templates should look like (with, preferably, easy shortcuts to other dashboards/visualizations/etc) and on click, the embeddable would request the app state and fill in the link.

@stacey-gammon

This comment has been minimized.

Copy link
Contributor

commented Oct 26, 2017

Good thoughts @trevan, I'd like to explore them more. The static links will probably remain a first step, but it sounds like most use cases will require the dynamic filtering context menu. btw, we do have plans for a "create ml job" link in the panel, so will need to expose more pluggability in the dashboard panel anyhow.

@stacey-gammon

This comment has been minimized.

Copy link
Contributor

commented Oct 28, 2017

btw @trevan - it should technically be the same amount of clicks to accomplish the same goal. E.g.

Looking at your mockups and description, it sounds like I would need to first click on the table/line for that server to add it as a filter, wait for the query to load (since the visualization panel will gray out), then click on these new links to drill down.

Because the links will be on the panel, not the embeddable, you should be able to click the drill down links before the embeddable finishes rendering. Same with if you built a "Dashboard link menu" - that won't require any network requests so the panel should finish loading nearly instantly and then dashboard level drill down links will be available immediately after a filter is clicked and applied at the dashboard level.

It might be a little bit of a clunkier UI/UX than your suggestion, but it does retain the benefit of one click filtering for people that don't want to use drill down links.

@trevan

This comment has been minimized.

Copy link
Contributor

commented Oct 28, 2017

@stacey-gammon, that is definitely clunky and isn't very intuitive (how do you tell people to make sure they add a filter before they click on the link?). You also still have the issue of sending a request to ES to rebuild the dashboard that you are just going to throw away. That increases the load on ES for little benefit.

I don't understand what you mean by the "Dashboard link menu".

@RANGERBEE

This comment has been minimized.

Copy link

commented Dec 18, 2017

For REF:

Its called "dynamic drill-downs" in Splunk. We used advanced XML , leveraging tokens within the code of the app. The next piece that is relative and missing here is called "workflow actions" in Splunk - being able to pull token data from the page and use it in a workflow URL - IE: passing to an external REST API..
Workflow actions has its own admin UI...

Cheers!

@alexfrancoeur

This comment has been minimized.

Copy link
Contributor Author

commented Dec 18, 2017

thanks for the additional info @RANGERBEE! We've taken a step back and are hoping to approach this in a dynamic sense as most users here have requested. I plan on putting together more detailed specs with mocks in the coming weeks. Will update this issue when ready.

@RANGERBEE

This comment has been minimized.

Copy link

commented Dec 18, 2017

no problem... To really complete the workflow and surpass Splunk, making available a customizable secure landing page for each Workflow action genre that allows for some JS would be seriously trick! When we get those REST API response headers they need to be handled, the user does not need to see it, so a landing page with an iframe was our solution to both handle the request/responses and forward to a proper URL then after... ;)

@Braj-120

This comment has been minimized.

Copy link

commented Jan 5, 2018

Hi @alexfrancoeur,
Any timeline on this feature? It would be a really nice feature to have.

@alexfrancoeur

This comment has been minimized.

Copy link
Contributor Author

commented Jan 8, 2018

@Braj-120 Unfortunately we do not have a specific timeframe at the moment but we are hoping to deliver in 6.x. I'll be updating the original description of this issue in the coming days to reflect our most recent requirements.

@Autorola-LHH

This comment has been minimized.

Copy link

commented Feb 19, 2018

Also hoping to use functionality like this. Being able to go directly from a dashboard with visualizations to the underlying searches would make troubleshooting so much easier.

@MattCarothers

This comment has been minimized.

Copy link

commented Aug 17, 2018

Today we can make a field a URL and use the field's value in the URL Template. I'd also like to be able to use meta data such as the current time range. Additionally, it would be helpful if other fields from the same document could be used.

For example, let's say I'm looking at intrusion detection events like this for the 24 hours starting 2018-08-16:

{
    "source_ip" : 1.2.3.4,
    "dest_ip" : 5.6.7.8,
    "signature_id" : 9101112
}

Now I want to pivot to my forensic packet capture tool, which has a URL scheme like this:

https://toolname.mycompany.com/search?query=src:1.2.3.4+AND+dst:5.6.7.8&start=2018-08-16&end=2018-08-17

In order to craft a URL template for that, I need access to the value of both the source_ip and the dest_ip fields from the document as well as the start and end times of the date range.

@gaantunes

This comment has been minimized.

Copy link

commented Aug 22, 2018

+1

1 similar comment
@renicon

This comment has been minimized.

Copy link

commented Aug 28, 2018

+1

@phil-hachey

This comment has been minimized.

Copy link

commented Sep 6, 2018

+1

@joaosp

This comment has been minimized.

Copy link

commented Sep 14, 2018

+1

@roubles

This comment has been minimized.

Copy link

commented Oct 4, 2018

In this thread https://discuss.elastic.co/t/access-first-location-of-all-in-visual-builder-markdown/150500 we discuss a possible minor enhancement to Visual Builder that would enable end users to effectively create all types of drill down displays.

@ko5win

This comment has been minimized.

Copy link

commented Oct 9, 2018

+1 Whats the timeline on this feature?

@devinbfergy

This comment has been minimized.

Copy link

commented Dec 6, 2018

+1

@hdave

This comment has been minimized.

Copy link

commented Apr 1, 2019

Ok, this issue is now more than 2 years old. Can anyone suggest a decent work-around?

Edit: Found This: https://thenewstack.io/inserting-links-kibana-dashboards/

@robcowart

This comment has been minimized.

Copy link

commented Apr 1, 2019

@hdave, if you find that link useful take a look a little further up the thread: #12560 (comment)

@bradquarry

This comment has been minimized.

Copy link

commented Apr 15, 2019

@alexfrancoeur @trevan I just wanted to circle back and provide some field feedback. The most common request I get regarding drill-down from prospects and customers is the ability to click on a datapoint within a chart and use that datapoint to link out to another related dashboard. Then, have that datapoint automatically populated as a filter for the next dashboard. Finally, easily go backwards and retrace steps to the origin of your click path if you do this a few times. I wasn't sure if this is exactly how this would work, but wanted to provide additional feedback in case it's helpful.

@alexfrancoeur

This comment has been minimized.

Copy link
Contributor Author

commented Apr 15, 2019

@bradquarry thanks for sharing and this is exactly one of the workflows we plan to support with Kibana Actions (#32371). There is still a lot of work to do, but I'd subscribe to that meta issue to track progress on the implementation of actions.

Finally, easily go backwards and retrace steps to the origin of your click path if you do this a few times. I wasn't sure if this is exactly how this would work, but wanted to provide additional feedback in case it's helpful.

@bradquarry would the using back button work or are you expecting that breadcrumbs would be populated?

cc: @stacey-gammon @AlonaNadler

@bradquarry

This comment has been minimized.

Copy link

commented Apr 15, 2019

@alexfrancoeur Got it, thanks. Breadcrumbs would provide a smoother experience. If they stand out well and people use them they might even decrease ES search workload in aggregate. If you take 4 drill in steps and can go right back to the beginning vs hitting the back button 3 times you wont have to potentially perform unneeded dashboard refreshes. I think this is good as cluster size is mostly driven by data volumes, and we like highly efficient fast queries.

@aarju

This comment has been minimized.

Copy link

commented Jun 11, 2019

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.