Skip to content

Commit

Permalink
Introduce url.full (#207)
Browse files Browse the repository at this point in the history
  • Loading branch information
webmat authored Dec 4, 2018
1 parent 6a7234d commit 3d1e7a7
Show file tree
Hide file tree
Showing 6 changed files with 55 additions and 4 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ All notable changes to this project will be documented in this file based on the
* Create new `related` field set with `related.ip`. #206
* Add `user.group` field. #204
* Create new `group` field set with `group.id` and `group.name`. #203
* Add `url.full` field. #207

### Improvements

Expand Down
5 changes: 4 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -405,7 +405,10 @@ URL fields provide a complete URL, with scheme, host, and path.

| Field | Description | Level | Type | Example |
|---|---|---|---|---|
| <a name="url.original"></a>url.original | Full original url. The field is stored as keyword. | extended | keyword | `https://www.elastic.co:443/search?q=elasticsearch#top` |
| <a name="url.original"></a>url.original | Unmodified original url as seen in the event source.<br/>Note that in network monitoring, the observed URL may be a full URL, whereas in access logs, the URL is often just represented as a path.<br/>This field is meant to represent the URL as it was observed, complete or not. | extended | keyword | `https://www.elastic.co:443/search?q=elasticsearch#top
or
/search?q=elasticsearch` |
| <a name="url.full"></a>url.full | If full URLs are important to your use case, they should be stored in `url.full`, whether this field is reconstructed or present in the event source. | extended | keyword | `https://www.elastic.co:443/search?q=elasticsearch#top` |
| <a name="url.scheme"></a>url.scheme | Scheme of the request, such as "https".<br/>Note: The `:` is not part of the scheme. | extended | keyword | `https` |
| <a name="url.domain"></a>url.domain | Domain of the request, such as "www.elastic.co".<br/>In some cases a URL may refer to an IP and/or port directly, without a domain name. In this case, the IP address would go to the `domain` field. | extended | keyword | `www.elastic.co` |
| <a name="url.port"></a>url.port | Port of the request, such as 443. | extended | integer | `443` |
Expand Down
22 changes: 21 additions & 1 deletion fields.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1180,7 +1180,27 @@
level: extended
type: keyword
description: >
Full original url. The field is stored as keyword.
Unmodified original url as seen in the event source.
Note that in network monitoring, the observed URL may be a full URL,
whereas in access logs, the URL is often just represented as a path.

This field is meant to represent the URL as it was observed, complete
or not.
example: >
https://www.elastic.co:443/search?q=elasticsearch#top
or

/search?q=elasticsearch

- name: full
level: extended
type: keyword
description: >
If full URLs are important to your use case, they should be stored in
`url.full`, whether this field is reconstructed or present in the
event source.
example: https://www.elastic.co:443/search?q=elasticsearch#top

- name: scheme
Expand Down
5 changes: 4 additions & 1 deletion schema.csv
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,10 @@ source.mac,keyword,core,
source.port,long,core,
url.domain,keyword,extended,www.elastic.co
url.fragment,keyword,extended,
url.original,keyword,extended,https://www.elastic.co:443/search?q=elasticsearch#top
url.full,keyword,extended,https://www.elastic.co:443/search?q=elasticsearch#top
url.original,keyword,extended,"https://www.elastic.co:443/search?q=elasticsearch#top
or
/search?q=elasticsearch"
url.password,keyword,extended,
url.path,keyword,extended,
url.port,integer,extended,443
Expand Down
22 changes: 21 additions & 1 deletion schemas/url.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,27 @@
level: extended
type: keyword
description: >
Full original url. The field is stored as keyword.
Unmodified original url as seen in the event source.
Note that in network monitoring, the observed URL may be a full URL,
whereas in access logs, the URL is often just represented as a path.
This field is meant to represent the URL as it was observed, complete
or not.
example: >
https://www.elastic.co:443/search?q=elasticsearch#top
or
/search?q=elasticsearch
- name: full
level: extended
type: keyword
description: >
If full URLs are important to your use case, they should be stored in
`url.full`, whether this field is reconstructed or present in the
event source.
example: https://www.elastic.co:443/search?q=elasticsearch#top

- name: scheme
Expand Down
4 changes: 4 additions & 0 deletions template.json
Original file line number Diff line number Diff line change
Expand Up @@ -595,6 +595,10 @@
"ignore_above": 1024,
"type": "keyword"
},
"full": {
"ignore_above": 1024,
"type": "keyword"
},
"original": {
"ignore_above": 1024,
"type": "keyword"
Expand Down

0 comments on commit 3d1e7a7

Please sign in to comment.