Address Kibana usage/usability topics

[sopel]

@mrdavidlaing has filed quite some usage/usability topics already as follows (they should be extracted to separate issues in case the discussion yields substantial work):

1. Graph doesn’t work (in IE10) and takes up space, would be good for it to be minimised by default for an investigation query page.
2. Copy pasting the order id I’m after doesn’t return any results. Tried putting a * at the end too.
3. Accidentally copy pasted a user id with white space, doesn’t trim. Would be nice if the default search trimmed this off. Common mistake.
4. Can’t find an easy way of getting the context of the log. Usually the thing that helps identify the issue is actually on the lines above below. Would be nice if could pull back say the 10 lines above and below. Can we do this?
5. The order of the logs when the timestamp is identical is not the same as the log output. Eg the ip audit logs search for:
   264553827 AND @fields.service:"InstructionProcessorGateway-Audit"

❗ Topics 4 & 5 are the meaty ones.

[sopel]

ℹ️ This likely depends on or would at least benefit from #64.

[sopel]

This discussion should also address #69.

[dpb587]

Graph doesn’t work (in IE10) and takes up space, would be good for it to be minimised by default for an investigation query page.

This was (very) recently fixed upstream (not yet checked out in this repository; see #64)...

Previous error (Error: Invalid dimensions for plot, width = 967, height = 0)...

[dpb587]

@mrdavidlaing, further information please...

Copy pasting the order id I’m after doesn’t return any results. Tried putting a * at the end too.

Either I'm copying/pasting the wrong identifiers, or it's working for me. Can you provide the query you're using?

Accidentally copy pasted a user id with white space, doesn’t trim. Would be nice if the default search trimmed this off. Common mistake.

Similarly, I think this should be working; can you provide the query?

[dpb587]

The order of the logs when the timestamp is identical is not the same as the log output. Eg the ip audit logs search for: 264553827 AND @fields.service:"InstructionProcessorGateway-Audit"

I'm fairly certain this is unavoidable without modifying the log messages in some way (i.e. more precise timestamps, or injected ordering by the log writer). The log messages are independently sent into the system and may be parsed and indexed out of order. The timestamp field becomes the only persistent ordering element and when they are the same, there is no guarantee of ordering.

[mrdavidlaing]

Best seems to be append a sequence number in the source log file http://stackoverflow.com/questions/3853644/log4net-how-to-add-simple-indexer-row-number-to-each-log-line

[mrdavidlaing]

Hmm - or add a "sequence" filter in LogStash - https://logstash.jira.com/browse/LOGSTASH-192

[dpb587]

The sequence filter method from LOGSTASH-192 doesn't seem scalable. It assumes a single process (possibly even a single thread?) on a single node. The concept might work with a few changes, but I'd rather see this happening on the log message generator side, if somewhere. My main concern is that everywhere after the log message generator is subject to out-of-order processing depending on how things may scale.

[le-bott]

Yeah, you are right. The only thing that can truly know the sequence is
the thing generating the logs. Once they make it into the Logstash queue,
its too late.

On Tuesday, 23 July 2013, Danny Berger wrote:

The sequence filter method from LOGSTASH-192 doesn't seem scalable. It
assumes a single process (possibly even a single thread?) on a single node.
The concept might work with a few changes, but I'd rather see this
happening on the log message generator side, if somewhere. My main concern
is that everywhere after the log message generator is subject to
out-of-order processing depending on how things may scale.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/68#issuecomment-21435587
.

David Laing
Open source @ City Index - github.com/cityindex
http://davidlaing.com
Twitter: @davidlaing

[mrdavidlaing]

Discussion has been split into the above issues. Closing.