Introduce url.full (#207)

CHANGELOG.md

@@ -19,6 +19,7 @@ All notable changes to this project will be documented in this file based on the
 * Create new `related` field set with `related.ip`. #206
 * Add `user.group` field. #204
 * Create new `group` field set with `group.id` and `group.name`. #203
+* Add `url.full` field. #207
 
 ### Improvements
 

README.md

@@ -405,7 +405,10 @@ URL fields provide a complete URL, with scheme, host, and path.
 
 | Field  | Description  | Level  | Type  | Example  |
 |---|---|---|---|---|
-| <a name="url.original"></a>url.original | Full original url. The field is stored as keyword. | extended | keyword | `https://www.elastic.co:443/search?q=elasticsearch#top` |
+| <a name="url.original"></a>url.original | Unmodified original url as seen in the event source.<br/>Note that in network monitoring, the observed URL may be a full URL, whereas in access logs, the URL is often just represented as a path.<br/>This field is meant to represent the URL as it was observed, complete or not. | extended | keyword | `https://www.elastic.co:443/search?q=elasticsearch#top
+or
+/search?q=elasticsearch` |
+| <a name="url.full"></a>url.full | If full URLs are important to your use case, they should be stored in `url.full`, whether this field is reconstructed or present in the event source. | extended | keyword | `https://www.elastic.co:443/search?q=elasticsearch#top` |
 | <a name="url.scheme"></a>url.scheme | Scheme of the request, such as "https".<br/>Note: The `:` is not part of the scheme. | extended | keyword | `https` |
 | <a name="url.domain"></a>url.domain | Domain of the request, such as "www.elastic.co".<br/>In some cases a URL may refer to an IP and/or port directly, without a domain name. In this case, the IP address would go to the `domain` field. | extended | keyword | `www.elastic.co` |
 | <a name="url.port"></a>url.port | Port of the request, such as 443. | extended | integer | `443` |

fields.yml

@@ -1180,7 +1180,27 @@
           level: extended
           type: keyword
           description: >
-            Full original url. The field is stored as keyword.
+            Unmodified original url as seen in the event source.
+    
+            Note that in network monitoring, the observed URL may be a full URL,
+            whereas in access logs, the URL is often just represented as a path.
+
+            This field is meant to represent the URL as it was observed, complete
+            or not.
+          example: >
+            https://www.elastic.co:443/search?q=elasticsearch#top
+    
+            or
+
+            /search?q=elasticsearch
+
+        - name: full
+          level: extended
+          type: keyword
+          description: >
+            If full URLs are important to your use case, they should be stored in
+            `url.full`, whether this field is reconstructed or present in the
+            event source.
           example: https://www.elastic.co:443/search?q=elasticsearch#top
 
         - name: scheme

schema.csv

@@ -121,7 +121,10 @@ source.mac,keyword,core,
 source.port,long,core,
 url.domain,keyword,extended,www.elastic.co
 url.fragment,keyword,extended,
-url.original,keyword,extended,https://www.elastic.co:443/search?q=elasticsearch#top
+url.full,keyword,extended,https://www.elastic.co:443/search?q=elasticsearch#top
+url.original,keyword,extended,"https://www.elastic.co:443/search?q=elasticsearch#top
+or
+/search?q=elasticsearch"
 url.password,keyword,extended,
 url.path,keyword,extended,
 url.port,integer,extended,443

schemas/url.yml

@@ -10,7 +10,27 @@
       level: extended
       type: keyword
       description: >
-        Full original url. The field is stored as keyword.
+        Unmodified original url as seen in the event source.
+
+        Note that in network monitoring, the observed URL may be a full URL,
+        whereas in access logs, the URL is often just represented as a path.
+
+        This field is meant to represent the URL as it was observed, complete
+        or not.
+      example: >
+        https://www.elastic.co:443/search?q=elasticsearch#top
+
+        or
+
+        /search?q=elasticsearch
+
+    - name: full
+      level: extended
+      type: keyword
+      description: >
+        If full URLs are important to your use case, they should be stored in
+        `url.full`, whether this field is reconstructed or present in the
+        event source.
       example: https://www.elastic.co:443/search?q=elasticsearch#top
 
     - name: scheme

template.json

@@ -595,6 +595,10 @@
               "ignore_above": 1024,
               "type": "keyword"
             },
+            "full": {
+              "ignore_above": 1024,
+              "type": "keyword"
+            },
             "original": {
               "ignore_above": 1024,
               "type": "keyword"