Skip to content

Add hash.* field set #426

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
webmat merged 4 commits into from
May 23, 2019
Merged

Add hash.* field set #426

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
f66ca6d
Add `hash.*` field set.
Apr 10, 2019
cb8fb60
Document how to customize with more hash fields.
May 20, 2019
203f9d1
Trim list down to most common hashes.
May 20, 2019
6de98fe
Amend description.
May 22, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.next.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
* Generator for the asciidoc rendering of field definitions. #347
* Generator for the Beats fields.ecs.yml file. #379
* Added field formats to all `.bytes` fields and `event.duration`. #385
* Added `hash.*` field set. #426
* Added `event.code`, `event.sequence` and `event.provider`. #439
* Added `file.name` and `file.directory`. #441
* Added `file.created`, and `file.accessed`. #445
Expand Down
2 changes: 1 addition & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ gocodegen:

# Generate the Use Cases
.PHONY: legacy_use_cases
legacy_use_cases:
legacy_use_cases: ve
$(PYTHON) scripts/use-cases.py --stdout=true >> /dev/null

# Check Makefile format.
Expand Down
38 changes: 38 additions & 0 deletions code/go/ecs/hash.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

116 changes: 116 additions & 0 deletions docs/field-details.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1170,6 +1170,29 @@ example: `1001`

// ===============================================================

|=====

==== Field Reuse




[[ecs-file-nestings]]
===== Field sets that can be nested under File

[options="header"]
|=====
| Nested fields | Description

// ===============================================================


| <<ecs-hash,file.hash.*>>
| Hashes, usually file hashes.

// ===============================================================


|=====

[[ecs-geo]]
Expand Down Expand Up @@ -1336,6 +1359,76 @@ Note also that the `group` fields may be used directly at the top level.



[[ecs-hash]]
=== Group Fields

The hash fields represent different hash algorithms and their values.

Field names for common hashes (e.g. MD5, SHA1) are predefined. Add fields for other hashes by lowercasing the hash algorithm name and using underscore separators as appropriate (snake case, e.g. sha3_512).

==== Group Field Details

[options="header"]
|=====
| Field | Description | Level

// ===============================================================

| hash.md5
| MD5 hash.

type: keyword



| extended

// ===============================================================

| hash.sha1
| SHA1 hash.

type: keyword



| extended

// ===============================================================

| hash.sha256
| SHA256 hash.

type: keyword



| extended

// ===============================================================

| hash.sha512
| SHA512 hash.

type: keyword



| extended

// ===============================================================

|=====

==== Field Reuse

The `hash` fields are expected to be nested at: `file.hash`, `process.hash`.

Note also that the `hash` fields are not expected to be used directly at the top level.




[[ecs-host]]
=== Host Fields

Expand Down Expand Up @@ -2200,6 +2293,29 @@ example: `/home/alice`

// ===============================================================

|=====

==== Field Reuse




[[ecs-process-nestings]]
===== Field sets that can be nested under Process

[options="header"]
|=====
| Nested fields | Description

// ===============================================================


| <<ecs-hash,process.hash.*>>
| Hashes, usually file hashes.

// ===============================================================


|=====

[[ecs-related]]
Expand Down
2 changes: 2 additions & 0 deletions docs/fields.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,8 @@ all fields are defined.

| <<ecs-group,Group>> | User's group relevant to the event.

| <<ecs-hash,Group>> | Hashes, usually file hashes.

| <<ecs-host,Host>> | Fields describing the relevant computing instance.

| <<ecs-http,HTTP>> | Fields describing an HTTP request.
Expand Down
70 changes: 70 additions & 0 deletions generated/beats/fields.ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -817,6 +817,26 @@
ignore_above: 1024
description: Primary group name of the file.
example: alice
- name: hash.md5
level: extended
type: keyword
ignore_above: 1024
description: MD5 hash.
- name: hash.sha1
level: extended
type: keyword
ignore_above: 1024
description: SHA1 hash.
- name: hash.sha256
level: extended
type: keyword
ignore_above: 1024
description: SHA256 hash.
- name: hash.sha512
level: extended
type: keyword
ignore_above: 1024
description: SHA512 hash.
- name: inode
level: extended
type: keyword
Expand Down Expand Up @@ -955,6 +975,36 @@
type: keyword
ignore_above: 1024
description: Name of the group.
- name: hash
title: Group
group: 2
description: 'The hash fields represent different hash algorithms and their values.

Field names for common hashes (e.g. MD5, SHA1) are predefined. Add fields for
other hashes by lowercasing the hash algorithm name and using underscore separators
as appropriate (snake case, e.g. sha3_512).'
type: group
fields:
- name: md5
level: extended
type: keyword
ignore_above: 1024
description: MD5 hash.
- name: sha1
level: extended
type: keyword
ignore_above: 1024
description: SHA1 hash.
- name: sha256
level: extended
type: keyword
ignore_above: 1024
description: SHA256 hash.
- name: sha512
level: extended
type: keyword
ignore_above: 1024
description: SHA512 hash.
- name: host
title: Host
group: 2
Expand Down Expand Up @@ -1586,6 +1636,26 @@
ignore_above: 1024
description: Absolute path to the process executable.
example: /usr/bin/ssh
- name: hash.md5
level: extended
type: keyword
ignore_above: 1024
description: MD5 hash.
- name: hash.sha1
level: extended
type: keyword
ignore_above: 1024
description: SHA1 hash.
- name: hash.sha256
level: extended
type: keyword
ignore_above: 1024
description: SHA256 hash.
- name: hash.sha512
level: extended
type: keyword
ignore_above: 1024
description: SHA512 hash.
- name: name
level: extended
type: keyword
Expand Down
12 changes: 12 additions & 0 deletions generated/csv/fields.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,10 @@ file.directory,keyword,extended,/home/alice,1.1.0-dev
file.extension,keyword,extended,png,1.1.0-dev
file.gid,keyword,extended,1001,1.1.0-dev
file.group,keyword,extended,alice,1.1.0-dev
file.hash.md5,keyword,extended,,1.1.0-dev
file.hash.sha1,keyword,extended,,1.1.0-dev
file.hash.sha256,keyword,extended,,1.1.0-dev
file.hash.sha512,keyword,extended,,1.1.0-dev
file.inode,keyword,extended,256383,1.1.0-dev
file.mode,keyword,extended,0640,1.1.0-dev
file.mtime,date,extended,,1.1.0-dev
Expand All @@ -118,6 +122,10 @@ geo.region_iso_code,keyword,core,CA-QC,1.1.0-dev
geo.region_name,keyword,core,Quebec,1.1.0-dev
group.id,keyword,extended,,1.1.0-dev
group.name,keyword,extended,,1.1.0-dev
hash.md5,keyword,extended,,1.1.0-dev
hash.sha1,keyword,extended,,1.1.0-dev
hash.sha256,keyword,extended,,1.1.0-dev
hash.sha512,keyword,extended,,1.1.0-dev
host.architecture,keyword,core,x86_64,1.1.0-dev
host.geo.city_name,keyword,core,Montreal,1.1.0-dev
host.geo.continent_name,keyword,core,North America,1.1.0-dev
Expand Down Expand Up @@ -200,6 +208,10 @@ os.platform,keyword,extended,darwin,1.1.0-dev
os.version,keyword,extended,10.14.1,1.1.0-dev
process.args,keyword,extended,"['ssh', '-l', 'user', '10.0.0.16']",1.1.0-dev
process.executable,keyword,extended,/usr/bin/ssh,1.1.0-dev
process.hash.md5,keyword,extended,,1.1.0-dev
process.hash.sha1,keyword,extended,,1.1.0-dev
process.hash.sha256,keyword,extended,,1.1.0-dev
process.hash.sha512,keyword,extended,,1.1.0-dev
process.name,keyword,extended,ssh,1.1.0-dev
process.pgid,long,extended,,1.1.0-dev
process.pid,long,core,,1.1.0-dev
Expand Down
Loading