Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SIEM] Formats the bytes columns in timeline #43147

Merged
merged 8 commits into from
Aug 16, 2019

Conversation

stephmilovic
Copy link
Contributor

@stephmilovic stephmilovic commented Aug 12, 2019

Summary

Formats bytes in Timeline and TimelineDetails using the format value from getAllFieldsByName with the new PreferenceFormattedBytes component. https://github.com/elastic/siem-team/issues/341

Before:
Screen Shot 2019-08-12 at 2 01 41 PM

After:
Screen Shot 2019-08-12 at 2 00 38 PM

Checklist

Use strikethroughs to remove checklist items you don't feel are applicable to this PR.

For maintainers

@stephmilovic stephmilovic added Team:SIEM release_note:skip Skip the PR/issue when compiling release notes v7.4.0 labels Aug 12, 2019
@elasticmachine
Copy link
Contributor

Pinging @elastic/siem

@stephmilovic
Copy link
Contributor Author

So far this PR only formats timeline columns with fields that end in .bytes. It does not include bytes fields like bytes_in, bytes_out, zeek.files.seen_bytes, suricata.eve.flow, mysql.slowlog.bytes_sent, etc... there are a lot of misc fields like this that I have not been able to populate data for, so I am hesitant to apply this fix to any field with bytes in the name without seeing what the data looks like... that's why I went with .bytes as the last key. If we can confirm that any field with bytes in the name should be formatted as bytes, I can update this PR to do that. Please let me know @cwurm @tsg

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@cwurm
Copy link
Contributor

cwurm commented Aug 13, 2019

We ship with a hardcoded ECS schema in the SIEM app, can we use that? It contains formatting information, e.g. for source.bytes:

{
name: 'bytes',
level: 'core',
type: 'long',
format: 'bytes',
example: 184,
description: 'Bytes sent from the source to the destination.',
},

That way, we could keep the number of places where we hardcode things to a minimum.

The rest of Kibana uses index patterns for formatting information, and I wish we could do the same, but we don't rely on index patterns in the SIEM app (maybe we should, but that would be a bigger change).

@XavierM
Copy link
Contributor

XavierM commented Aug 13, 2019

I think that with the fields browser, we should be able to do exactly what @cwurm is talking about since it is a mix of the ECS schema and index pattern.

@stephmilovic
Copy link
Contributor Author

@cwurm @XavierM perfect, I'll work on piping that format value through and determining bytes display based off of that

@elasticmachine
Copy link
Contributor

💔 Build Failed

@elasticmachine
Copy link
Contributor

💔 Build Failed

@andrew-goldstein
Copy link
Contributor

Thanks for adding this feature @stephmilovic , and for fixing the flag issue!

For consistency with the special formatting applied to dates and IP addresses, would you be willing to also apply the new bytes formatting to the fields shown in an expanded timeline event, per the screenshot below?

expanded-event

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@stephmilovic stephmilovic marked this pull request as ready for review August 16, 2019 14:28
@elasticmachine
Copy link
Contributor

💚 Build Succeeded

Copy link
Contributor

@andrew-goldstein andrew-goldstein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙏 @stephmilovic for the additional formatting in the expanded events

ran it locally and it works great 💥

LGTM

@stephmilovic stephmilovic merged commit 38ace24 into elastic:master Aug 16, 2019
@stephmilovic stephmilovic deleted the bytes-format-timeline branch August 16, 2019 17:58
stephmilovic added a commit to stephmilovic/kibana that referenced this pull request Aug 16, 2019
jloleysens added a commit to jloleysens/kibana that referenced this pull request Aug 19, 2019
…_update_json_spec

* 'master' of github.com:elastic/kibana: (35 commits)
  fix: 🐛 pass whole action context to isCompatible() method (elastic#43457)
  Deleted old kbn-top-nav directive (elastic#43168)
  [ML] Fixing cloning of single metric distinct count job (elastic#43435)
  Update @elastic/charts version 8.1.6 > 9.1.1 (elastic#43516)
  [Inspector Views] [Request View] - Migrate inspector_views to new platform (elastic#43191)
  [ML] Adding loading indicators to all wizard charts (elastic#43382)
  disable flaky test (elastic#43492)
  feature(code/frontend): cancel file blob and directory commits request if outdated (elastic#43348)
  fix(code/frontend): button group url should have previous query string (elastic#43428)
  [SIEM] Fixes index substring incorrectly matching configured indices and failing to install ML job (elastic#43409)
  [SIEM] Adds performance enhancements such by removing wasted renderers and adding incremental DOM rendering (elastic#43157)
  disable flaky test (elastic#37859)
  Added sass lint to Canvas (elastic#43410)
  [Maps] add indicator when layer is filtered by search bar (elastic#43283)
  Properly validate current user password during password change. (elastic#43447)
  Spaces - allow for hex color codes that include uppercase characters (elastic#43470)
  [Reporting] Add a bit more logging and a few more logging level promotions (elastic#43415)
  Partially convert index pattern server to typescript (elastic#43291)
  [Infra UI] Use sum for aggregating AWS metrics. (elastic#43293)
  [SIEM] Format bytes columns in timeline (elastic#43147)
  ...
@FrankHassanabad FrankHassanabad added release_note:fix and removed release_note:skip Skip the PR/issue when compiling release notes labels Sep 16, 2019
@FrankHassanabad FrankHassanabad changed the title [SIEM] Format bytes columns in timeline [SIEM] Formats the bytes columns in timeline Sep 16, 2019
@elasticmachine
Copy link
Contributor

💔 Build Failed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants