Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor schema reader to use intermediate schema structure #722

Merged
merged 4 commits into from
Jan 21, 2020
Merged

Refactor schema reader to use intermediate schema structure #722

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
139b363
refactor schema reader to use intermediate schema structure
marshallmain Jan 13, 2020
87a15ef
remove logic for '.' notation in reusable field names
marshallmain Jan 21, 2020
ba0792e
update changelog.next.md
marshallmain Jan 21, 2020
ec8ec6d
Add tests for intermediate schema data structures
marshallmain Jan 21, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
40 changes: 20 additions & 20 deletions generated/csv/fields.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Example,Description
1.5.0-dev,true,client,client.address,keyword,extended,,Client network address.
1.5.0-dev,true,client,client.as.number,long,extended,15169,Unique number allocated to the autonomous system. The autonomous system number (ASN) uniquely identifies each network on the Internet.
1.5.0-dev,true,client,client.as.organization.name,keyword,extended,Google LLC,Organization name.
1.5.0-dev,true,client,as.organization.name.text,text,extended,Google LLC,Organization name.
1.5.0-dev,true,client,client.as.organization.name.text,text,extended,Google LLC,Organization name.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hadn't noticed that before.

PRs that come with bugfixes are my favourites ;-)

1.5.0-dev,true,client,client.bytes,long,core,184,Bytes sent from the client to the server.
1.5.0-dev,true,client,client.domain,keyword,core,,Client domain.
1.5.0-dev,true,client,client.geo.city_name,keyword,core,Montreal,City name.
Expand All @@ -36,14 +36,14 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Example,Description
1.5.0-dev,true,client,client.user.domain,keyword,extended,,Name of the directory the user is a member of.
1.5.0-dev,true,client,client.user.email,keyword,extended,,User email address.
1.5.0-dev,true,client,client.user.full_name,keyword,extended,Albert Einstein,"User's full name, if available."
1.5.0-dev,true,client,user.full_name.text,text,extended,Albert Einstein,"User's full name, if available."
1.5.0-dev,true,client,client.user.full_name.text,text,extended,Albert Einstein,"User's full name, if available."
1.5.0-dev,true,client,client.user.group.domain,keyword,extended,,Name of the directory the group is a member of.
1.5.0-dev,true,client,client.user.group.id,keyword,extended,,Unique identifier for the group on the system/platform.
1.5.0-dev,true,client,client.user.group.name,keyword,extended,,Name of the group.
1.5.0-dev,true,client,client.user.hash,keyword,extended,,Unique user hash to correlate information for a user in anonymized form.
1.5.0-dev,true,client,client.user.id,keyword,core,,One or multiple unique identifiers of the user.
1.5.0-dev,true,client,client.user.name,keyword,core,albert,Short name or login of the user.
1.5.0-dev,true,client,user.name.text,text,core,albert,Short name or login of the user.
1.5.0-dev,true,client,client.user.name.text,text,core,albert,Short name or login of the user.
1.5.0-dev,true,cloud,cloud.account.id,keyword,extended,666777888999,The cloud account or organization id.
1.5.0-dev,true,cloud,cloud.availability_zone,keyword,extended,us-east-1c,Availability zone in which this host is running.
1.5.0-dev,true,cloud,cloud.instance.id,keyword,extended,i-1234567890abcdef0,Instance ID of the host machine.
Expand All @@ -60,7 +60,7 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Example,Description
1.5.0-dev,true,destination,destination.address,keyword,extended,,Destination network address.
1.5.0-dev,true,destination,destination.as.number,long,extended,15169,Unique number allocated to the autonomous system. The autonomous system number (ASN) uniquely identifies each network on the Internet.
1.5.0-dev,true,destination,destination.as.organization.name,keyword,extended,Google LLC,Organization name.
1.5.0-dev,true,destination,as.organization.name.text,text,extended,Google LLC,Organization name.
1.5.0-dev,true,destination,destination.as.organization.name.text,text,extended,Google LLC,Organization name.
1.5.0-dev,true,destination,destination.bytes,long,core,184,Bytes sent from the destination to the source.
1.5.0-dev,true,destination,destination.domain,keyword,core,,Destination domain.
1.5.0-dev,true,destination,destination.geo.city_name,keyword,core,Montreal,City name.
Expand All @@ -82,14 +82,14 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Example,Description
1.5.0-dev,true,destination,destination.user.domain,keyword,extended,,Name of the directory the user is a member of.
1.5.0-dev,true,destination,destination.user.email,keyword,extended,,User email address.
1.5.0-dev,true,destination,destination.user.full_name,keyword,extended,Albert Einstein,"User's full name, if available."
1.5.0-dev,true,destination,user.full_name.text,text,extended,Albert Einstein,"User's full name, if available."
1.5.0-dev,true,destination,destination.user.full_name.text,text,extended,Albert Einstein,"User's full name, if available."
1.5.0-dev,true,destination,destination.user.group.domain,keyword,extended,,Name of the directory the group is a member of.
1.5.0-dev,true,destination,destination.user.group.id,keyword,extended,,Unique identifier for the group on the system/platform.
1.5.0-dev,true,destination,destination.user.group.name,keyword,extended,,Name of the group.
1.5.0-dev,true,destination,destination.user.hash,keyword,extended,,Unique user hash to correlate information for a user in anonymized form.
1.5.0-dev,true,destination,destination.user.id,keyword,core,,One or multiple unique identifiers of the user.
1.5.0-dev,true,destination,destination.user.name,keyword,core,albert,Short name or login of the user.
1.5.0-dev,true,destination,user.name.text,text,core,albert,Short name or login of the user.
1.5.0-dev,true,destination,destination.user.name.text,text,core,albert,Short name or login of the user.
1.5.0-dev,true,dns,dns.answers,object,extended,,Array of DNS answers.
1.5.0-dev,true,dns,dns.answers.class,keyword,extended,IN,The class of DNS data contained in this resource record.
1.5.0-dev,true,dns,dns.answers.data,keyword,extended,10.10.10.10,The data describing the resource.
Expand Down Expand Up @@ -195,25 +195,25 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Example,Description
1.5.0-dev,true,host,host.name,keyword,core,,Name of the host.
1.5.0-dev,true,host,host.os.family,keyword,extended,debian,"OS family (such as redhat, debian, freebsd, windows)."
1.5.0-dev,true,host,host.os.full,keyword,extended,Mac OS Mojave,"Operating system name, including the version or code name."
1.5.0-dev,true,host,os.full.text,text,extended,Mac OS Mojave,"Operating system name, including the version or code name."
1.5.0-dev,true,host,host.os.full.text,text,extended,Mac OS Mojave,"Operating system name, including the version or code name."
1.5.0-dev,true,host,host.os.kernel,keyword,extended,4.4.0-112-generic,Operating system kernel version as a raw string.
1.5.0-dev,true,host,host.os.name,keyword,extended,Mac OS X,"Operating system name, without the version."
1.5.0-dev,true,host,os.name.text,text,extended,Mac OS X,"Operating system name, without the version."
1.5.0-dev,true,host,host.os.name.text,text,extended,Mac OS X,"Operating system name, without the version."
1.5.0-dev,true,host,host.os.platform,keyword,extended,darwin,"Operating system platform (such centos, ubuntu, windows)."
1.5.0-dev,true,host,host.os.version,keyword,extended,10.14.1,Operating system version as a raw string.
1.5.0-dev,true,host,host.type,keyword,core,,Type of host.
1.5.0-dev,true,host,host.uptime,long,extended,1325,Seconds the host has been up.
1.5.0-dev,true,host,host.user.domain,keyword,extended,,Name of the directory the user is a member of.
1.5.0-dev,true,host,host.user.email,keyword,extended,,User email address.
1.5.0-dev,true,host,host.user.full_name,keyword,extended,Albert Einstein,"User's full name, if available."
1.5.0-dev,true,host,user.full_name.text,text,extended,Albert Einstein,"User's full name, if available."
1.5.0-dev,true,host,host.user.full_name.text,text,extended,Albert Einstein,"User's full name, if available."
1.5.0-dev,true,host,host.user.group.domain,keyword,extended,,Name of the directory the group is a member of.
1.5.0-dev,true,host,host.user.group.id,keyword,extended,,Unique identifier for the group on the system/platform.
1.5.0-dev,true,host,host.user.group.name,keyword,extended,,Name of the group.
1.5.0-dev,true,host,host.user.hash,keyword,extended,,Unique user hash to correlate information for a user in anonymized form.
1.5.0-dev,true,host,host.user.id,keyword,core,,One or multiple unique identifiers of the user.
1.5.0-dev,true,host,host.user.name,keyword,core,albert,Short name or login of the user.
1.5.0-dev,true,host,user.name.text,text,core,albert,Short name or login of the user.
1.5.0-dev,true,host,host.user.name.text,text,core,albert,Short name or login of the user.
1.5.0-dev,true,http,http.request.body.bytes,long,extended,887,Size in bytes of the request body.
1.5.0-dev,true,http,http.request.body.content,keyword,extended,Hello world,The full HTTP request body.
1.5.0-dev,true,http,http.request.body.content.text,text,extended,Hello world,The full HTTP request body.
Expand Down Expand Up @@ -263,10 +263,10 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Example,Description
1.5.0-dev,true,observer,observer.name,keyword,extended,1_proxySG,Custom name of the observer.
1.5.0-dev,true,observer,observer.os.family,keyword,extended,debian,"OS family (such as redhat, debian, freebsd, windows)."
1.5.0-dev,true,observer,observer.os.full,keyword,extended,Mac OS Mojave,"Operating system name, including the version or code name."
1.5.0-dev,true,observer,os.full.text,text,extended,Mac OS Mojave,"Operating system name, including the version or code name."
1.5.0-dev,true,observer,observer.os.full.text,text,extended,Mac OS Mojave,"Operating system name, including the version or code name."
1.5.0-dev,true,observer,observer.os.kernel,keyword,extended,4.4.0-112-generic,Operating system kernel version as a raw string.
1.5.0-dev,true,observer,observer.os.name,keyword,extended,Mac OS X,"Operating system name, without the version."
1.5.0-dev,true,observer,os.name.text,text,extended,Mac OS X,"Operating system name, without the version."
1.5.0-dev,true,observer,observer.os.name.text,text,extended,Mac OS X,"Operating system name, without the version."
1.5.0-dev,true,observer,observer.os.platform,keyword,extended,darwin,"Operating system platform (such centos, ubuntu, windows)."
1.5.0-dev,true,observer,observer.os.version,keyword,extended,10.14.1,Operating system version as a raw string.
1.5.0-dev,true,observer,observer.product,keyword,extended,s200,The product name of the observer.
Expand Down Expand Up @@ -363,7 +363,7 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Example,Description
1.5.0-dev,true,server,server.address,keyword,extended,,Server network address.
1.5.0-dev,true,server,server.as.number,long,extended,15169,Unique number allocated to the autonomous system. The autonomous system number (ASN) uniquely identifies each network on the Internet.
1.5.0-dev,true,server,server.as.organization.name,keyword,extended,Google LLC,Organization name.
1.5.0-dev,true,server,as.organization.name.text,text,extended,Google LLC,Organization name.
1.5.0-dev,true,server,server.as.organization.name.text,text,extended,Google LLC,Organization name.
1.5.0-dev,true,server,server.bytes,long,core,184,Bytes sent from the server to the client.
1.5.0-dev,true,server,server.domain,keyword,core,,Server domain.
1.5.0-dev,true,server,server.geo.city_name,keyword,core,Montreal,City name.
Expand All @@ -385,14 +385,14 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Example,Description
1.5.0-dev,true,server,server.user.domain,keyword,extended,,Name of the directory the user is a member of.
1.5.0-dev,true,server,server.user.email,keyword,extended,,User email address.
1.5.0-dev,true,server,server.user.full_name,keyword,extended,Albert Einstein,"User's full name, if available."
1.5.0-dev,true,server,user.full_name.text,text,extended,Albert Einstein,"User's full name, if available."
1.5.0-dev,true,server,server.user.full_name.text,text,extended,Albert Einstein,"User's full name, if available."
1.5.0-dev,true,server,server.user.group.domain,keyword,extended,,Name of the directory the group is a member of.
1.5.0-dev,true,server,server.user.group.id,keyword,extended,,Unique identifier for the group on the system/platform.
1.5.0-dev,true,server,server.user.group.name,keyword,extended,,Name of the group.
1.5.0-dev,true,server,server.user.hash,keyword,extended,,Unique user hash to correlate information for a user in anonymized form.
1.5.0-dev,true,server,server.user.id,keyword,core,,One or multiple unique identifiers of the user.
1.5.0-dev,true,server,server.user.name,keyword,core,albert,Short name or login of the user.
1.5.0-dev,true,server,user.name.text,text,core,albert,Short name or login of the user.
1.5.0-dev,true,server,server.user.name.text,text,core,albert,Short name or login of the user.
1.5.0-dev,true,service,service.ephemeral_id,keyword,extended,8a4f500f,Ephemeral identifier of this service.
1.5.0-dev,true,service,service.id,keyword,core,d37e5ebfe0ae6c4972dbe9f0174a1637bb8247f6,Unique identifier of the running service.
1.5.0-dev,true,service,service.name,keyword,core,elasticsearch-metrics,Name of the service.
Expand All @@ -403,7 +403,7 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Example,Description
1.5.0-dev,true,source,source.address,keyword,extended,,Source network address.
1.5.0-dev,true,source,source.as.number,long,extended,15169,Unique number allocated to the autonomous system. The autonomous system number (ASN) uniquely identifies each network on the Internet.
1.5.0-dev,true,source,source.as.organization.name,keyword,extended,Google LLC,Organization name.
1.5.0-dev,true,source,as.organization.name.text,text,extended,Google LLC,Organization name.
1.5.0-dev,true,source,source.as.organization.name.text,text,extended,Google LLC,Organization name.
1.5.0-dev,true,source,source.bytes,long,core,184,Bytes sent from the source to the destination.
1.5.0-dev,true,source,source.domain,keyword,core,,Source domain.
1.5.0-dev,true,source,source.geo.city_name,keyword,core,Montreal,City name.
Expand All @@ -425,14 +425,14 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Example,Description
1.5.0-dev,true,source,source.user.domain,keyword,extended,,Name of the directory the user is a member of.
1.5.0-dev,true,source,source.user.email,keyword,extended,,User email address.
1.5.0-dev,true,source,source.user.full_name,keyword,extended,Albert Einstein,"User's full name, if available."
1.5.0-dev,true,source,user.full_name.text,text,extended,Albert Einstein,"User's full name, if available."
1.5.0-dev,true,source,source.user.full_name.text,text,extended,Albert Einstein,"User's full name, if available."
1.5.0-dev,true,source,source.user.group.domain,keyword,extended,,Name of the directory the group is a member of.
1.5.0-dev,true,source,source.user.group.id,keyword,extended,,Unique identifier for the group on the system/platform.
1.5.0-dev,true,source,source.user.group.name,keyword,extended,,Name of the group.
1.5.0-dev,true,source,source.user.hash,keyword,extended,,Unique user hash to correlate information for a user in anonymized form.
1.5.0-dev,true,source,source.user.id,keyword,core,,One or multiple unique identifiers of the user.
1.5.0-dev,true,source,source.user.name,keyword,core,albert,Short name or login of the user.
1.5.0-dev,true,source,user.name.text,text,core,albert,Short name or login of the user.
1.5.0-dev,true,source,source.user.name.text,text,core,albert,Short name or login of the user.
1.5.0-dev,true,threat,threat.framework,keyword,extended,MITRE ATT&CK,Threat classification framework.
1.5.0-dev,true,threat,threat.tactic.id,keyword,extended,TA0040,Threat tactic id.
1.5.0-dev,true,threat,threat.tactic.name,keyword,extended,impact,Threat tactic.
Expand Down Expand Up @@ -504,10 +504,10 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Example,Description
1.5.0-dev,true,user_agent,user_agent.original.text,text,extended,"Mozilla/5.0 (iPhone; CPU iPhone OS 12_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Mobile/15E148 Safari/604.1",Unparsed user_agent string.
1.5.0-dev,true,user_agent,user_agent.os.family,keyword,extended,debian,"OS family (such as redhat, debian, freebsd, windows)."
1.5.0-dev,true,user_agent,user_agent.os.full,keyword,extended,Mac OS Mojave,"Operating system name, including the version or code name."
1.5.0-dev,true,user_agent,os.full.text,text,extended,Mac OS Mojave,"Operating system name, including the version or code name."
1.5.0-dev,true,user_agent,user_agent.os.full.text,text,extended,Mac OS Mojave,"Operating system name, including the version or code name."
1.5.0-dev,true,user_agent,user_agent.os.kernel,keyword,extended,4.4.0-112-generic,Operating system kernel version as a raw string.
1.5.0-dev,true,user_agent,user_agent.os.name,keyword,extended,Mac OS X,"Operating system name, without the version."
1.5.0-dev,true,user_agent,os.name.text,text,extended,Mac OS X,"Operating system name, without the version."
1.5.0-dev,true,user_agent,user_agent.os.name.text,text,extended,Mac OS X,"Operating system name, without the version."
1.5.0-dev,true,user_agent,user_agent.os.platform,keyword,extended,darwin,"Operating system platform (such centos, ubuntu, windows)."
1.5.0-dev,true,user_agent,user_agent.os.version,keyword,extended,10.14.1,Operating system version as a raw string.
1.5.0-dev,true,user_agent,user_agent.version,keyword,extended,12.0,Version of the user agent.
Expand Down