add multifieldhistogram panel

[bfiorini]

This is a new implementation of #374 but in a different panel and rebased on master.

It does more or less the same thing as #374

• you can plot different values taken from different fields, and from different queries
• you can give a new alias and color to this value(s)

My use case: I have documents which contain the percentage of cpu utilization of some machines {idle:30, user:10, iowait: 4, ...} and I want to plot this on the same histogram. I wanna have a different color/alias for each of these fields, cause I'm not interested about the query it comes from.

There is room for improvement in the UI as well as in the JS. Let me know what you think.

Cheers,
Benjamin

[elvarb]

For time series data take a look at using graphite.

Easiest is to use influxdb and then grafana as a front end. Takes a look at the demo at grafana.org

[rashidkpc]

Playing with this a bit today. I like where its going and I can definitely see merging it.

A few notes:

1. Can you remove the bar chart options? Stacked bars probably don't make much sense here anyway.
2. The stack of fields in the editor, can you extend that so it takes up the full width of the dialog?
3. The queries tab becomes confusing here, maybe rename it "Markers"?
4. Neither the legend nor the tooltip gives an indication of what you're looking at

[bfiorini]

1,2,3: fair enough 👍
About the 4), I improved the aliases, to better understand what you're looking at.

[rabson85]

Is this it. Will one of the most anticipated mergegs get merged!? 👍

IMO it would not make sense to limit the settings that can be added here. I can see a number of use cases where a stacked bar chart makes sense (not this use case), but for instance for stacking any given value in a multivalued message.

Hoping this gets merged soon!

[lookastar]

I agree that bar chart would still be quite useful in this version of histogram

[bfiorini]

I've got a bug once it's built: jquery.flot.stack is included twice -- once in the context of the normal histogram, and once in the context of the multi field histogram.

[joaoascenso]

Waiting for this feature for months, using this now for its usefulness for plotting metrics.

Hoping for fixing the issue and merging soon.

[greina]

+1

[randspy]

Hi.

I have started to use this cool feature. And I have two enhancements to think about.

1)When I add a new chart value all queries are selected. This is cumbersome when there are many of them. Maybe a button select/deselect all.
2)Currently for a chart value legend we have format "Mode(Queries.Value Field)" and it does not look nice. It would be great if we could choose an arbitrary name.

BR

[bfiorini]

Hi randspy,

1. agreed, I wanted to do it when I have some time
2. you can, if you click on the little color "dot" at the very left of the row. Same thing as you can give alias to queries. Actually, the default format just sucks and maybe it makes more sense for the alias to be mandatory...

Cheers

[randspy]

Thanks for really fast reply.

[KlavsKlavsen]

Here's hoping this gets merged :)

[rashidkpc]

I'm just getting back from holiday and will make some time to review this

[matwolb]

I'm glad to see this improvement!! Congrats

[dobrzyn2]

do not remove bars. There are fields which relate to each other and to be able to stack these values to get a visual sum or not stacked, to get a visual difference, would be an leading feature of the multifieldhistogram

[rashidkpc]

Looking better. I did note that the scale option doesn't seem to work? That seems like an important one.

@dobrzyn2 stacking is still there, just not in bar charts.

[randspy]

I have one potential problem for fixing. It does not consider a value zero in the plot. It is a problem when values are connected by a line. It gives an illusion that there was no change in the plot.

[wricardo]

+1

[marcok]

Great!
For us, it would be awesome to be able to choose an different timefield for each line.
We have documents (payments) where different things happened in the past. A timestamp for delivered, timestamp for settled, ...

[lucamilanesio]

The multi-field feature is awesome, thanks for sharing it 👍

One question: I've noticed that all queries are selected by default when adding a new char value: is that done on purpose? We have quite a few queries available and deselecting them one by one is quite annoying 😢

[lucamilanesio]

Another problem with the multi-field 🙀 : it seems that once I add one multi-field panel the others (non multi-field) get screwed up (see below):

The panel on the top should not have extra blank space between the vertical bars: it gets added only when I add the multi-field panel and I update the page.

Really weird ... seems like a bug to me 🐝
Any clue ? Does anybody else have the same problem ?

Think it is critical to sort this out before mering the change.

[bfiorini]

@lucamilanesio this is the result of this: #1296 (comment)

[lucamilanesio]

@tvvmb thanks for sharing this: it actually works fine when using it from /src so possibly the build creates this double-inclusion. Any clue when it will be fixed ?

What about the default selection of all queries? That is quite annoying as well 😢

Anyway we'll start using it as is today, it is good enough for us 👍

[bfiorini]

@lucamilanesio yes the problem is here only with the built stuff, I guess it's related to require.js, but I lack the knowledge for this, if anybody has any tip/idea, I'd be grateful!

concerning the selection of the queries, see the answer for #1296 (comment)

[lucamilanesio]

@tvvmb cool, do you think you will have time for the default selection?

[lucamilanesio]

@tvvmb uploaded a single one-line to fix the problem and now works fine for us 👍
The default selection is a minor annoying issue anyway, does not block this PR being merged.

Hoping to see this amazing multi-fields extension merged very soon and congrats for your new panel, looks amazing.

[bfiorini]

@rashidkpc this commit fixes the scale and derivative problem
Also now no queries are selected by default.

@lucamilanesio nice, was easy after all :)

[sips4711]

I really like this new feature.
The biggest problem I ran into was to download from GitHub. I'm not that familiar with github. Especially switching to the branch was not that easy. I suggest to provide some download/installation instructions on the GitHub site. Maybe more people will use it because it closes the gap to graphite...

Von meinem iPhone gesendet

Am 31.07.2014 um 13:55 schrieb Benjamin Fiorini notifications@github.com:

@rashidkpc this commit fixes the scale and derivative problem
Also now no queries are selected by default.

—
Reply to this email directly or view it on GitHub.

[lucamilanesio]

@sips4711 you can get a stable v3.1.0 plus the multi-field panel source code by doing:
git clone https://github.com/elasticsearch/kibana
cd kibana
git checkout v3.1.0
git pull --no-edit origin pull/1296/head
git pull --no-edit origin pull/1403/head

and then build the ZIP distribution by doing:
npm install -g grunt-cli
npm install
grunt build compress:zip_release

and in the tmp/kibana-3.1.0.zip is your home-built Kibana 3.1.0 with the multi-field histogram patch applied 😸

Enjoy the ride 🚗

[rashidkpc]

@tvvmb Excellent, will check it out if not today then Monday!

[lucamilanesio]

@rashidkpc we are already using it now with quite a lot of data and it works fine.

[jmkgreen]

I just replaced our kibana-3.0.1 with a build from the instructions above and things look very broken:

I have shift and ctrl refreshed the page. Should I have expected this? In 3.0.1 it (and our other dashboards) look fine.

[lucamilanesio]

@jmkgreen apologies, forgot to include a "git checkout v3.1.0": we applied this patch on top of a stable v3.1.0 and works fine for us. We were migrating from a v3.0.0-milestone5

[jmkgreen]

So the build no longer breaks, which is good. But I fail to see any new
panel types and histogram appears unchanged. Looking at screenshots I
should be expecting multifieldhistogram? Looking within app/panels/ I do
have a multifieldhistogram folder with files inside...

I wiped the downloaded working copy of kibana and re-did the instructions
from earlier, inserting the v3.1.0 checkout right after the clone.

On 1 August 2014 16:00, Luca Milanesio notifications@github.com wrote:

@jmkgreen https://github.com/jmkgreen apologies, forgot to include a
"git checkout v3.1.0": we applied this patch on top of a stable v3.1.0 and
works fine for us.

—
Reply to this email directly or view it on GitHub
#1296 (comment).

[jrgns]

+1

[listuser]

Thanks for adding this functionality. I was able to download and build without issue. Works as advertised, except when I create a stacked cumulative graph using 2 values, where one of the values is 0 (zero).

"swap_used value": 0
"swap_used free": 2097144

I am able to create the graph using both values and no errors, but the SWAP graph is blank.

But, if I create the same graph and I do not include "swap_used value" in the parameters, then the graph is not blank.

Am I doing something incorrectly? Apologies if this is already a known issue. BTW, here is the event view in JSON.

Thanks for your time.

[listuser]

If I reverse the stack order, "swap_used free" which is > 0, next "swap_used value" which is 0, then a graph is produced.

So, it looks like if the 1st value in a stack is 0 (zero), then the graph ends up blank.

[listuser]

More info. If I remove the "lines" option (checkbox) or the "stack" option (checkbox), then the graph becomes populated.

Here are the parms again.

Remove lines option and the graph shows for "swap_used free"

Remove stack option and the graph shows for "swap_used free"

Here the graph is blank but I expected it to show stacked cumulative values with a line for "swap_used free" even if "swap_used value" is 0 (zero).

Anyone else had time to try reproducing this problem on their system? I am not sure what else I can provide in way of troubleshooting this problem as I am not a programmer / developer. Thanks for your time.

[Jeppesen-io]

listuser,

I'm having similar issues except I can't get the panel to chart anything. I'll play around with it and see what's going on. My environment is slightly different with 4 indexes per day behind a tribe node. Maybe that's causing issues?

Also, I'm using the latest pull from master.

[listuser]

I used points and zero fill to satisfy, as described here.

https://groups.google.com/forum/#!topic/logstash-users/rmmOSA-pswk

Swap graph looks good

Thanks again for adding the multifield histogram option.

Never heard of a Tribe node. Learned something new, yet again.

[Jeppesen-io]

+1 the usefulness of a multifieldhistogram panel.

Still trying to get it to work though. I tried listuser's settings, but nothing renders. Only difference is I'm using count, not mean. Here's the settings I'm using:

[listuser]

Maybe this goes without saying and not too insult, but...

Are you able to chart a single field using the standard histogram panel with Kibana configured to query using the Tribe node?

Have you tried reconfiguring Kibana to query against a single node instead of the Tribe node along with using the multifieldhistogram panel?

[Jeppesen-io]

No insult taken, its a valid question.

I'm able to chart using all the other panels with out issue. In fact, I took existing dashboards and just added multi field histogram.

I'll try bypassing the tribe node, but I'll have to get back to you on that.

[spalger]

Woops! Sorry about that! We recently replaced the master branch with Kibana 4. This action force closed all of the old pull requests against master. We will be reviewing these on a case-by-case basis and creating new tickets as necessary.

The good news is that many long requested features can be found in Kibana 4, and we're being entirely open about our roadmap. Check out the roadmap tickets (which we're still filling in) here.

If you're looking for the old Kibana 3 code you can find it here.

[rashidkpc]

I truly regret not getting the opportunity to merge this. Now that Kibana 4 beta1 is out we've implemented a distributed review process for pull requests that should ensure that important ones like this get the attention they deserve. The implementation of this feature will will come along with support for multiple metric aggregations in Kibana 4. The roadmap ticket is here: #1597