Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extract log event context #275

Closed
mrdavidlaing opened this issue Jul 25, 2013 · 268 comments

Comments

Projects
None yet
@mrdavidlaing
Copy link
Contributor

commented Jul 25, 2013

When troubleshooting the cause of an ERROR log event, its really helpful to be able to see the log event context - i.e, which, say, 10 log events occured before and after the specific ERROR log event.

Loggly has a feature which does this quite nicely, as illustrated below
capture

I have 2 questions concerning this:

  1. Is it feasible to add a similar feature to Kibana?
  2. Are there any existing plans to add said feature? (I'm happy to contribute development resources to assist or lead this if its possible)

Lastly, thanks!

@ramv

This comment has been minimized.

Copy link

commented Dec 2, 2013

I would love to have this feature too.

@aeneaswiener

This comment has been minimized.

Copy link

commented Feb 19, 2014

+1

1 similar comment
@fragosti

This comment has been minimized.

Copy link

commented Feb 21, 2014

+1

@ajczapiewski

This comment has been minimized.

Copy link

commented Feb 21, 2014

+10

@stuart-warren

This comment has been minimized.

Copy link

commented Feb 21, 2014

When you expand a log and get the 'table' view by default, perhaps we could get another 'context' view.
This 'context' view might require another request, but essentially you could add which fields to filter on for the same value as the selected log (eg host, tags), a time period before and after (eg -1m, +10s) and which field to sort on.

Might be wise to open a separate window with a smaller (predefined?) dashboard with the given filters?

@kiranos

This comment has been minimized.

Copy link

commented Feb 24, 2014

+1

@OkkeKlein

This comment has been minimized.

Copy link

commented Feb 26, 2014

+1

1 similar comment
@haagenhasle

This comment has been minimized.

Copy link

commented Mar 18, 2014

+1

@justyns

This comment has been minimized.

Copy link

commented Apr 7, 2014

I'd also like this feature. Is there already a similar workflow/workaround that people already use for this sort of thing?

Currently I search for the error in kibana, then change the filter to include only the file/server it came from and then just zoom in on the time around the error.

@tcostermans

This comment has been minimized.

Copy link

commented Apr 10, 2014

+1

@stbka

This comment has been minimized.

Copy link

commented Apr 28, 2014

+1

@marcelpanse

This comment has been minimized.

Copy link

commented May 12, 2014

+1

Why is this still not in Kibana?
It makes Kibana kind of useless to me, since I typically need to find something in the logs and then find the error preceding the log row, which is currently impossible.

@steveims

This comment has been minimized.

Copy link

commented May 20, 2014

+1

2 similar comments
@markgilmore1322

This comment has been minimized.

Copy link

commented May 20, 2014

+1

@murphe

This comment has been minimized.

Copy link

commented May 22, 2014

+1

@spbever

This comment has been minimized.

Copy link

commented Jun 9, 2014

I will be working on this task in a forked repository for where I work. When I have it completed I will submit a pull request with how I have it implemented.

@spbever

This comment has been minimized.

Copy link

commented Jun 16, 2014

So I have not been able to come up with a "creative" enough solution that I think would be generic enough to pulled into the main Kibana source, but I will still post what my solution to this was and anyone will be free to copy that for their own use.

EDIT:
My solution can be found: https://github.com/spstapleton/kibana/tree/Kibana-275-Log-Context
I have added a field to logstash.json (contextMatchFields) that is an array of keys that the context filter will attempt to find related log entries by.

@stbka

This comment has been minimized.

Copy link

commented Jul 22, 2014

spstapleton this is really great! Thanks!

I`ve done some tests with this feature and found some things not working properly.

  • All events in the context view have the same timestamp
  • If I choose a field with special characters inside like '-' or '/' or ':' the search seems not to work, because no surrounding events can be found
  • Sometimes, it seems like if "Time Variance" and "Max Context Lookup Results" do not fit to each other no results are found. For example: max lookups is 100 and t variance is 300 -> no results. max lookups is 1000 and t variance is 300 -> results are found
@spbever

This comment has been minimized.

Copy link

commented Jul 22, 2014

I can take a look into these in the next week or so and and let you know what I come up with. I think the special characters is an issue with not having the value uri encoded.

@spbever

This comment has been minimized.

Copy link

commented Jul 28, 2014

@tma-ka
I made a fix for the timestamp. The current timestamp filter only worked for the original events returned. I had to make a second filter that just filtered on text instead of an event.

Special Characters: Elastic Search by default will split text entries on certain special characters. For instance "te-st" would get two indexes by default, one for "te" and one for "st". Thus, when I have it do a context search for the term "te-st" it will not return any matching results. If you are predicting that the field you will be basing your context off of will have cases like that, I would suggest marking that field as 'not_analyzed' inside of elasticsearch.

For the last point. It could be the case where you are not doing the sorting based of off the default timestamp. If that is the case the time variance is not actually used and it relies only on the max lookups.

@stbka

This comment has been minimized.

Copy link

commented Jul 29, 2014

I fixed the special char problem by indexing a separate field as an identifier. Inside all special chars are removed so you have got only [A-Za-z0-9].
Unfortunately your timestamp fix is not working for me. I do still have one timestamp in each event.

Anyway, I think the idea and the implementation from user point of view is nice.
Rather the question is the technical implementation good enough for a permanent implementation in kibana.

I would like to use it but cannot while it is in a branch. At the latest when I am updating my kibana from trunk all is lost.

rashidkpc added a commit that referenced this issue Oct 6, 2014

@harshilsharma63

This comment has been minimized.

Copy link

commented Dec 7, 2016

+1 lack of this feature has made Kibana practically useless.

@rifqifatih

This comment has been minimized.

Copy link

commented Dec 14, 2016

+1

1 similar comment
@azitabh

This comment has been minimized.

Copy link

commented Dec 21, 2016

+1

@weltenwort

This comment has been minimized.

Copy link
Contributor

commented Dec 21, 2016

As some might already have noticed by the auto-generated backlink above, I am currently working on a feature related to this enhancement request in PR #9198. Its goals have a large overlap with the requirements described in this issue and the comments.

I would like to invite anyone interested to take a look at the screenshots or even to try out the development version and provide feedback and suggestions. Please keep in mind that this initial PR is intended to only cover basic functionality with more use-case specific features to follow in later PRs.

@allenmchan

This comment has been minimized.

Copy link

commented Jan 3, 2017

I suspect people are starting to trickle back into work after the holidays.
Please provide feedback / suggestions for @weltenwort.

@larryboymi

This comment has been minimized.

Copy link

commented Jan 9, 2017

+1

10 similar comments
@lucas1993ch

This comment has been minimized.

Copy link

commented Jan 10, 2017

+1

@madhavajay

This comment has been minimized.

Copy link

commented Jan 12, 2017

+1

@acbox

This comment has been minimized.

Copy link

commented Jan 13, 2017

+1

@danhy87

This comment has been minimized.

Copy link

commented Jan 16, 2017

+1

@argaldo

This comment has been minimized.

Copy link

commented Jan 25, 2017

+1

@Jipos

This comment has been minimized.

Copy link

commented Jan 27, 2017

+1

@ZillaG

This comment has been minimized.

Copy link

commented Feb 6, 2017

+1

@yami12376

This comment has been minimized.

Copy link

commented Feb 8, 2017

+1

@roubles

This comment has been minimized.

Copy link

commented Feb 20, 2017

+1

@mbriggs

This comment has been minimized.

Copy link

commented Feb 22, 2017

+1

@epixa

This comment has been minimized.

Copy link
Member

commented Feb 22, 2017

Just in case folks aren't watching it, the event context PR got merged this morning: #9198

This is a first step toward addressing the many different use cases that drive this ticket, but feedback is still welcome!

@9h87f497h4387b9

This comment has been minimized.

Copy link

commented Feb 22, 2017

It has been very exciting watching the PR progression. Thanks for the update!

@lisakaymera

This comment has been minimized.

Copy link

commented Mar 6, 2017

Just curious - has anyone used logtrail to address this issue at all? It seems to address the issue of log context. I didn't get to download to play around with it though because my kibana version is slightly different. Would love to hear people's experiences with it.

@wangzhaojiang

This comment has been minimized.

Copy link

commented Mar 16, 2017

+9

@stolsvik

This comment has been minimized.

Copy link

commented Mar 28, 2017

+1

I would be happy to have to define the contextual search parameters: For us, that would either "application name" and the timestamp. Or, a more refined, "application name"+"host name" and timestamp. Or "traceId" and timestamp. They are of use in different scenarios. Then it would just be a matter of somehow select a line, and hit one of the context targets in "Get lines around in context 'App', 'App+Host' or 'TraceId'".

@mohaseeb

This comment has been minimized.

Copy link

commented Apr 13, 2017

+1

1 similar comment
@artisanhe

This comment has been minimized.

Copy link

commented Apr 14, 2017

+1

@epixa epixa removed the P1 label Apr 25, 2017

@Mirae001

This comment has been minimized.

Copy link

commented Apr 26, 2017

+1

@weltenwort

This comment has been minimized.

Copy link
Contributor

commented May 4, 2017

This is to let you know that the first stage of this feature has just been released as part of Kibana 5.4, together with the whole Elastic Stack.

I want to invite you to give it a try, maybe read the documentation, and let us know what you think:

And last but not least, we are hard at work adding filtering capabilities to the context view. You can follow the progress in #11466. As always, any and all feedback on that is appreciated and can only help to make Kibana better.

@weltenwort weltenwort closed this May 4, 2017

@tbragin tbragin added the v5.4.0 label May 4, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.