Merge branch 'master' into os-cfam

CHANGELOG.md

@@ -19,14 +19,16 @@ All notable changes to this project will be documented in this file based on the
 * Added `threat.technique.subtechnique` to capture MITRE ATT&CK® subtechniques. #951
 * Added `configuration` as an allowed `event.category`. #963
 * Added a new directory with experimental artifacts, which includes all changes
-  from RFCs that have reached stage 2. #993, #1053
+  from RFCs that have reached stage 2. #993, #1053, #1115, #1117, #1118
 
 #### Improvements
 
 * Expanded field set definitions for `source.*` and `destination.*`. #967
 * Provided better guidance for mapping network events. #969
 * Added the field `.subdomain` under `client`, `destination`, `server`, `source`
   and `url`, to match its presence at `dns.question.subdomain`. #981
+* Clarified ambiguity in guidance on how to use x509 fields for connections with
+  only one certificate. #1114
 
 ### Tooling and Artifact Changes
 

docs/field-details.asciidoc

@@ -6974,7 +6974,11 @@ example: `Critical`
 [[ecs-x509]]
 === x509 Certificate Fields
 
-This implements the common core fields for x509 certificates. This information is likely logged with TLS sessions, digital signatures found in executable binaries, S/MIME information in email bodies, or analysis of files on disk. When only a single certificate is logged in an event, it should be nested under `file`. When hashes of the DER-encoded certificate are available, the `hash` data set should be populated as well (e.g. `file.hash.sha256`). For events that contain certificate information for both sides of the connection, the x509 object could be nested under the respective side of the connection information (e.g. `tls.server.x509`).
+This implements the common core fields for x509 certificates. This information is likely logged with TLS sessions, digital signatures found in executable binaries, S/MIME information in email bodies, or analysis of files on disk.
+
+When the certificate relates to a file, use the fields at `file.x509`. When hashes of the DER-encoded certificate are available, the `hash` data set should be populated as well (e.g. `file.hash.sha256`).
+
+Events that contain certificate information about network connections, should use the x509 fields under the relevant TLS fields: `tls.server.x509` and/or `tls.client.x509`.
 
 [discrete]
 ==== x509 Certificate Field Details

experimental/generated/beats/fields.ecs.yml

@@ -1,5 +1,5 @@
 # WARNING! Do not edit this file directly, it was generated by the ECS project,
-# based on ECS version 2.0.0-dev.
+# based on ECS version 2.0.0-dev+exp.
 # Please visit https://github.com/elastic/ecs to suggest changes to ECS fields.
 
 - key: ecs
@@ -967,6 +967,19 @@
       (`dns.type:answer`).'
     type: group
     fields:
+    - name: answers
+      level: extended
+      type: object
+      description: 'An array containing an object for each answer section returned
+        by the server.
+
+        The main keys that should be present in these objects are defined by ECS.
+        Records that have more information may contain more keys than what ECS defines.
+
+        Not all DNS data sources give all details about DNS answers. At minimum, answer
+        objects must contain the `data` key. If more information is available, map
+        as much of it to ECS as possible, and add any additional fields to the answer
+        objects as custom fields.'
     - name: answers.class
       level: extended
       type: keyword
@@ -1160,7 +1173,6 @@
         norms: false
         default_field: false
       description: The stack trace of this error in plain text.
-      index: true
     - name: type
       level: extended
       type: wildcard
@@ -5943,15 +5955,18 @@
   - name: x509
     title: x509 Certificate
     group: 2
-    description: This implements the common core fields for x509 certificates. This
+    description: 'This implements the common core fields for x509 certificates. This
       information is likely logged with TLS sessions, digital signatures found in
       executable binaries, S/MIME information in email bodies, or analysis of files
-      on disk. When only a single certificate is logged in an event, it should be
-      nested under `file`. When hashes of the DER-encoded certificate are available,
-      the `hash` data set should be populated as well (e.g. `file.hash.sha256`). For
-      events that contain certificate information for both sides of the connection,
-      the x509 object could be nested under the respective side of the connection
-      information (e.g. `tls.server.x509`).
+      on disk.
+
+      When the certificate relates to a file, use the fields at `file.x509`. When
+      hashes of the DER-encoded certificate are available, the `hash` data set should
+      be populated as well (e.g. `file.hash.sha256`).
+
+      Events that contain certificate information about network connections, should
+      use the x509 fields under the relevant TLS fields: `tls.server.x509` and/or
+      `tls.client.x509`.'
     type: group
     fields:
     - name: alternative_names