-
Notifications
You must be signed in to change notification settings - Fork 0
/
fields.yml
2190 lines (2012 loc) · 62.2 KB
/
fields.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
- key: ecs
title: ECS
description: >
ECS fields.
fields:
- name: agent
title: Agent
group: 2
description: >
The agent fields contain the data about the agent/client/shipper that
created the event.
footnote: >
Examples: In the case of Beats for logs, the agent.name is filebeat. For APM, it is the
agent running in the app/service. The agent information does not change if
data is sent through queuing systems like Kafka, Redis, or processing systems
such as Logstash or APM Server.
type: group
fields:
- name: version
type: keyword
description: >
Version of the agent.
example: 6.0.0-rc2
- name: type
type: keyword
description: >
Name of the agent.
example: filebeat
- name: hostname
type: keyword
description: >
Hostname of the agent.
- name: id
type: keyword
description: >
Unique identifier of this agent (if one exists).
Example: For Beats this would be beat.id.
example: 8a4f500d
- name: ephemeral_id
type: keyword
description: >
Ephemeral identifier of this agent (if one exists).
This id normally changes across restarts, but `agent.id` does not.
example: 8a4f500f
- name: base
title: Base
group: 1
description: >
The base set contains all fields which are on the top level. These fields are common across all types of events.
type: group
fields:
- name: "@timestamp"
type: date
required: true
phase: 3
example: "2016-05-23T08:05:34.853Z"
description: >
Date/time when the event originated.
For log events this is the date/time when the event was generated, and
not when it was read.
Required field for all events.
- name: tags
type: keyword
example: "[\"production\", \"env2\"]"
description: >
List of keywords used to tag each event.
- name: labels
type: object
example: {key1: value1, key2: value2}
description: >
Key/value pairs.
Can be used to add meta information to events. Should not contain nested
objects. All values are stored as keyword.
Example: `docker` and `k8s` labels.
- name: message
type: text
phase: 1
example: "Hello World"
description: >
For log events the message field contains the log message.
In other use cases the message field can be used to concatenate
different values which are then freely searchable. If multiple
messages exist, they can be combined into one message.
- name: cloud
title: Cloud
group: 2
description: >
Fields related to the cloud or infrastructure the events
are coming from.
footnote: >
Examples: If Metricbeat is running on an EC2 host and fetches data from its
host, the cloud info contains the data about this machine. If Metricbeat
runs on a remote machine outside the cloud and fetches data from a service
running in the cloud, the field contains cloud data from the machine the
service is running on.
type: group
fields:
- name: provider
example: ec2
type: keyword
description: >
Name of the cloud provider. Example values are ec2, gce, or
digitalocean.
- name: availability_zone
example: us-east-1c
type: keyword
description: >
Availability zone in which this host is running.
- name: region
type: keyword
example: us-east-1
description: >
Region in which this host is running.
- name: instance.id
type: keyword
example: i-1234567890abcdef0
description: >
Instance ID of the host machine.
- name: instance.name
type: keyword
description: >
Instance name of the host machine.
- name: machine.type
type: keyword
example: t2.medium
description: >
Machine type of the host machine.
- name: account.id
type: keyword
example: 666777888999
description: >
The cloud account or organization id used to identify different
entities in a multi-tenant environment.
Examples: AWS account id, Google Cloud ORG Id, or other unique
identifier.
- name: container
title: Container
group: 2
description: >
Container fields are used for meta information about the specific container
that is the source of information. These fields help correlate data based
containers from any runtime.
type: group
fields:
- name: runtime
type: keyword
description: >
Runtime managing this container.
example: docker
- name: id
type: keyword
description: >
Unique container id.
- name: image.name
type: keyword
description: >
Name of the image the container was built on.
- name: image.tag
type: keyword
description: >
Container image tag.
- name: name
type: keyword
description: >
Container name.
- name: labels
type: object
object_type: keyword
description: >
Image labels.
- name: destination
title: Destination
group: 2
description: >
Destination fields describe details about the destination of a
packet/event.
type: group
fields:
- name: ip
type: ip
description: >
IP address of the destination.
Can be one or multiple IPv4 or IPv6 addresses.
- name: port
type: long
description: >
Port of the destination.
- name: mac
type: keyword
description: >
MAC address of the destination.
- name: domain
type: keyword
phase: 1
description: >
Destination domain.
- name: geo
type: group
description:
Geolocation for destination.
fields:
- name: continent_name
type: keyword
description: >
Name of the continent.
- name: country_iso_code
type: keyword
description: >
Country ISO code.
- name: location
type: geo_point
description: >
Longitude and latitude.
- name: region_name
type: keyword
description: >
Region name.
- name: city_name
type: keyword
description: >
City name.
- name: region_iso_code
type: keyword
description: >
Region ISO code.
- name: device
title: Device
group: 2
description: >
Device fields are used to provide additional information about the device
that is the source of the information. This could be a firewall, network device, etc.
type: group
fields:
- name: mac
type: keyword
description: >
MAC address of the device
- name: ip
type: ip
description: >
IP address of the device.
- name: hostname
type: keyword
description: >
Hostname of the device.
- name: vendor
type: text
description: >
Device vendor information.
- name: version
type: keyword
description: >
Device version.
- name: serial_number
type: keyword
description: >
Device serial number.
- name: timezone.offset.sec
type: long
description: >
Timezone offset of the host in seconds.
Number of seconds relative to UTC. If the offset is -01:30 the
value will be -5400.
example: -5400
- name: type
type: keyword
description: >
The type of the device the data is coming from.
There is no predefined list of device types. Some examples are
`endpoint`, `firewall`, `ids`, `ips`, `proxy`.
example: firewall
- name: error
title: Error
group: 2
description: >
These fields can represent errors of any kind. Use them for errors that
happen while fetching events or in cases where the event itself contains an error.
type: group
fields:
- name: id
type: keyword
description: >
Unique identifier for the error.
- name: message
type: text
description: >
Error message.
- name: code
type: keyword
description: >
Error code describing the error.
- name: event
title: Event
group: 2
description: >
The event fields are used for context information about the data itself.
type: group
fields:
- name: id
type: keyword
description: >
Unique ID to describe the event.
example: 8a4f500d
phase: 1
- name: category
type: keyword
description: >
Event category.
This can be a user defined category.
example: metrics
- name: type
type: keyword
description: >
A type given to this kind of event which can be used for grouping.
This is normally defined by the user.
example: nginx-stats-metrics
- name: action
type: keyword
description: >
The action captured by the event. The type of action will vary from
system to system but is likely to include actions by security services,
such as blocking or quarantining; as well as more generic actions such
as login events, file i/o or proxy forwarding events.
The value is normally defined by the user.
example: reject
- name: module
type: keyword
description: >
Name of the module this data is coming from.
This information is coming from the modules used in Beats or Logstash.
example: mysql
- name: dataset
type: keyword
description: >
Name of the dataset.
The concept of a `dataset` (fileset / metricset) is used in Beats as a
subset of modules. It contains the information which is currently
stored in metricset.name and metricset.module or fileset.name.
example: stats
- name: severity
type: long
phase: 1
example: "7"
description: >
Severity describes the severity of the event. What the different
severity values mean can very different between use cases. It's up to
the implementer to make sure severities are consistent across events.
- name: original
type: keyword
phase: 1
# Unfortunately this example is not shown correctly yet as | do not work
# in tables well
# Is | is the representation of | it works except for cases where
# used ticks.
example: "Sep 19 08:26:10 host CEF:0|Security|
threatmanager|1.0|100|
worm successfully stopped|10|src=10.0.0.1
dst=2.1.2.2spt=1232"
description: >
Raw text message of entire event. Used to demonstrate log integrity.
This field is not indexed and doc_values are disabled. It cannot be
searched, but it can be retrieved from `_source`.
index: false
doc_values: false
- name: hash
type: keyword
phase: 1
example: "123456789012345678901234567890ABCD"
description: >
Hash (perhaps logstash fingerprint) of raw field to be able to
demonstrate log integrity.
- name: version
type: keyword
required: true
description: >
The version field contains the version an event for ECS adheres to.
This field should be provided as part of each event to make it possible
to detect to which ECS version an event belongs.
event.version is a required field and must exist in all events. It
describes which ECS version the event adheres to.
The current version is 0.1.0.
example: 0.1.0
- name: duration
type: long
description: >
Duration of the event in nanoseconds.
- name: created
type: date
description: >
event.created contains the date when the event was created.
This timestamp is distinct from @timestamp in that @timestamp contains
the processed timestamp. For logs these two timestamps can be different
as the timestamp in the log line and when the event is read for example
by Filebeat are not identical. `@timestamp` must contain the timestamp
extracted from the log line, event.created when the log line is read.
The same could apply to package capturing where @timestamp contains the
timestamp extracted from the network package and event.created when the
event was created.
In case the two timestamps are identical, @timestamp should be used.
- name: risk_score
type: float
description: >
Risk score or priority of the event (e.g. security solutions).
Use your system's original value here.
- name: risk_score_norm
type: float
description: >
Normalized risk score or priority of the event, on a scale of 0 to 100.
This is mainly useful if you use more than one system that assigns
risk scores, and you want to see a normalized value across all systems.
- name: file
group: 2
title: File
description: >
File fields provide details about each file.
type: group
fields:
- name: path
type: text
description: Path to the file.
multi_fields:
- name: raw
type: keyword
description: >
Path to the file. This is a non-analyzed field that is useful
for aggregations.
- name: target_path
type: text
description: Target path for symlinks.
multi_fields:
- name: raw
type: keyword
description: >
Path to the file. This is a non-analyzed field that is useful
for aggregations.
- name: extension
type: keyword
description: >
File extension.
This should allow easy filtering by file extensions.
example: png
- name: type
type: keyword
description: File type (file, dir, or symlink).
- name: device
type: keyword
description: Device that is the source of the file.
- name: inode
type: keyword
description: Inode representing the file in the filesystem.
- name: uid
type: keyword
description: >
The user ID (UID) or security identifier (SID) of the file owner.
- name: owner
type: keyword
description: File owner's username.
- name: gid
type: keyword
description: Primary group ID (GID) of the file.
- name: group
type: keyword
description: Primary group name of the file.
- name: mode
type: keyword
example: 0640
description: Mode of the file in octal representation.
- name: size
type: long
description: File size in bytes (field is only added when `type` is
`file`).
- name: mtime
type: date
description: Last time file content was modified.
- name: ctime
type: date
description: Last time file metadata changed.
- name: geo
title: Geo
group: 2
description: >
Geo fields can carry data about a specific location related to an event
or geo information for an IP field.
type: group
fields:
- name: continent_name
type: keyword
description: >
Name of the continent.
- name: country_iso_code
type: keyword
description: >
Country ISO code.
- name: location
type: geo_point
description: >
Longitude and latitude.
- name: region_name
type: keyword
description: >
Region name.
- name: city_name
type: keyword
description: >
City name.
- name: host
title: Host
group: 2
description: >
Host fields provide information related to a host. A host can be a
physical machine, a virtual machine, or a Docker container.
Normally the host information is related to the machine on which the event
was generated/collected, but they can be used differently if needed.
type: group
fields:
- name: timezone.offset.sec
type: long
description: >
Timezone offset of the host in seconds.
Number of seconds relative to UTC. If the offset is -01:30 the
value will be -5400.
phase: 1
example: -5400
- name: name
type: keyword
description: >
host.name is the hostname of the host.
It can contain what `hostname` returns on Unix systems, the fully
qualified domain name, or a name specified by the user. The sender
decides which value to use.
phase: 1
- name: id
type: keyword
phase: 1
description: >
Unique host id.
As hostname is not always unique, use values that are meaningful
in your environment.
Example: The current usage of `beat.name`.
- name: ip
type: ip
description: >
Host ip address.
- name: mac
type: keyword
description: >
Host mac address.
- name: type
type: keyword
description: >
Type of host.
For Cloud providers this can be the machine type like `t2.medium`.
If vm, this could be the container, for example, or other information
meaningful in your environment.
phase: 1
# Operating System information
- name: os.platform
type: keyword
description: >
Operating system platform (centos, ubuntu, windows, etc.)
example: darwin
- name: os.name
type: keyword
example: "Mac OS X"
description: >
Operating system name.
- name: os.family
type: keyword
example: "debian"
description: >
OS family (redhat, debian, freebsd, windows, etc.)
- name: os.version
type: keyword
example: "10.12.6"
description: >
Operating system version.
- name: architecture
type: keyword
example: "x86_64"
description: >
Operating system architecture.
- name: http
title: HTTP
group: 2
description: >
Fields related to HTTP requests and responses.
type: group
fields:
- name: request.method
type: keyword
description: >
Http request method.
example: GET, POST, PUT
- name: response.status_code
type: long
description: >
Http response status code.
example: 404
- name: response.body
type: text
description: >
The full http response body.
example: Hello world
- name: version
type: keyword
description: >
Http version.
example: 1.1
- name: log
title: Log
description: >
Fields which are specific to log events.
type: group
fields:
- name: level
type: keyword
description: >
Log level of the log event.
Some examples are `WARN`, `ERR`, `INFO`.
example: ERR
- name: original
type: keyword
phase: 1
example: "Sep 19 08:26:10 localhost My log"
index: false
doc_values: false
description: >
This is the original log message and contains the full log message
before splitting it up in multiple parts.
In contrast to the `message` field which can contain an extracted part
of the log message, this field contains the original, full log message.
It can have already some modifications applied like encoding or new
lines removed to clean up the log message.
This field is not indexed and doc_values are disabled so it can't be
queried but the value can be retrieved from `_source`.
- name: network
title: Network
group: 2
description: >
Fields related to network data.
type: group
fields:
- name: name
type: text
description: >
Name given by operators to sections of their network.
example: Guest Wifi
multi_fields:
- name: raw
type: keyword
description: >
Name given by operators to sections of their network.
- name: protocol
type: keyword
description: >
Network protocol name.
example: http
- name: direction
type: keyword
description: >
Direction of the network traffic.
Recommended values are:
* inbound
* outbound
* unknown
example: inbound
- name: forwarded_ip
type: ip
description: >
Host IP address when the source IP address is the proxy.
example: 192.1.1.2
# Metrics
- name: inbound.bytes
type: long
description: >
Network inbound bytes.
example: 184
- name: inbound.packets
type: long
description: >
Network inbound packets.
example: 12
- name: outbound.bytes
type: long
description: >
Network outbound bytes.
example: 184
- name: outbound.packets
type: long
description: >
Network outbound packets.
example: 12
- name: total.bytes
type: long
description: >
Network total bytes. The sum of inbound.bytes + outbound.bytes.
example: 368
- name: total.packets
type: long
description: >
Network outbound packets. The sum of inbound.packets + outbound.packets
example: 24
- name: organization
title: Organization
group: 2
description: >
The organization fields enrich data with information about the company or entity
the data is associated with. These fields help you arrange or filter data stored in an index by one or multiple
organizations.
type: group
fields:
- name: name
type: text
description: >
Organization name.
- name: id
type: keyword
description: >
Unique identifier for the organization.
- name: os
title: Operating System
group: 2
description: >
The OS fields contain information about the operating system.
These fields are often used inside other prefixes, such as `host.os.*` or `user_agent.os.*`.
type: group
fields:
- name: platform
type: keyword
description: >
Operating system platform (such centos, ubuntu, windows).
example: darwin
- name: name
type: keyword
example: "Mac OS X"
description: >
Operating system name.
- name: family
type: keyword
example: "debian"
description: >
OS family (such as redhat, debian, freebsd, windows).
- name: version
type: keyword
example: "10.12.6-rc2"
description: >
Operating system version as a raw string.
- name: kernel
type: keyword
example: "4.4.0-112-generic"
description: >
Operating system kernel version as a raw string.
- name: process
title: Process
group: 2
description: >
These fields contain information about a process.
These fields can help you correlate metrics information with a process id/name
from a log message. The `process.pid` often stays in the metric itself and is
copied to the global field for correlation.
type: group
fields:
- name: args
type: keyword
description: >
Process arguments.
May be filtered to protect sensitive information.
example: ["-l", "user", "10.0.0.16"]
- name: name
type: keyword
description: >
Process name.
Sometimes called program name or similar.
example: ssh
- name: pid
type: long
description: >
Process id.
exmple: ssh
- name: ppid
type: long
description: >
Process parent id.
- name: title
type: keyword
description: >
Process title.
The proctitle, often the same as process name.
- name: service
title: Service
group: 2
description: >
The service fields describe the service for or from which the data was
collected. These fields help you find and correlate logs for a specific
service and version.
type: group
fields:
- name: id
type: keyword
description: >
Unique identifier of the running service.
This id should uniquely identify this service. This makes it possible
to correlate logs and metrics for one specific service.
Example: If you are experiencing issues with one redis instance, you
can filter on that id to see metrics and logs for that single instance.
example: d37e5ebfe0ae6c4972dbe9f0174a1637bb8247f6
phase: 1
- name: name
type: keyword
phase: 1
example: "elasticsearch"
description: >
Name of the service data is collected from.
The name can be used to group and correlate logs and metrics from one
service.
Example: If logs or metrics are collected from Redis, `service.name` would be
`redis`.
- name: type
type: keyword
phase: 1
description: >
Service type.
- name: state
type: keyword
phase: 1
description: >
Current state of the service.
- name: version
type: keyword
phase: 1
example: "3.2.4"
description: >
Version of the service the data was collected from.
This allows to look at a data set only for a specific version of a
service.
- name: ephemeral_id
type: keyword
description: >
Ephemeral identifier of this service (if one exists).
This id normally changes across restarts, but `service.id` does not.
example: 8a4f500f
- name: source
title: Source
description: >
Source fields describe details about the source of the event.
type: group
fields:
- name: ip
level: core
type: ip
description: >
IP address of the source.
Can be one or multiple IPv4 or IPv6 addresses.
- name: port
level: core
type: long
description: >
Port of the source.
- name: mac
level: core
type: keyword
description: >
MAC address of the source.
- name: domain