ip_range aggregation with mask of 0.0.0.0/0 gets treated as 0.0.0.0/32

[cameronkerrnz]

An ip_range aggregation with mask of 0.0.0.0/0 is not being treated correctly, the prefix-length of 0 is being taken as 'unset' and is being set as /32

I am trying to get aggregated information on expanding scopes (some subnet in our network, our entire network, the internet...), so I want to use 0.0.0.0/0 ... a range of 0.0.0.0 to 255.255.255.255 does work (although is nit-pickingly incorrect as the range is exclusive and so the final address tested is 255.255.255.254, which might be surprising if you wanted information about broadcast packets.... but that is perhaps a separate bug report.

# Replace the template that logstash provides, so
# we add an 'ip' multifield type to the 'clientip'
# field.

PUT /_template/logstash
{
  "template": "logstash-*",
  "settings": {
    "index.refresh_interval": "5s"
  },
  "mappings": {
    "_default_": {
      "dynamic_templates": [
        {
          "message_field": {
            "mapping": {
              "index": "analyzed",
              "omit_norms": true,
              "type": "string"
            },
            "match_mapping_type": "string",
            "match": "message"
          }
        },
        {
          "ip_field": {
            "mapping": {
              "index": "not_analyzed",
              "omit_norms": true,
              "type": "string",
              "fields": {
                "num": {
                  "index": "not_analyzed",
                  "type": "string"
                }
              }
            },
            "match_mapping_type": "string",
            "match": "ip_*"
          }
        },
        {
          "string_fields": {
            "mapping": {
              "index": "analyzed",
              "omit_norms": true,
              "type": "string",
              "fields": {
                "raw": {
                  "index": "not_analyzed",
                  "ignore_above": 256,
                  "type": "string"
                }
              }
            },
            "match_mapping_type": "string",
            "match": "*"
          }
        }
      ],
      "properties": {
        "clientip": {
          "type": "string",
          "fields": {
            "num": {
              "index": "not_analyzed",
              "type": "ip"
            }
          }
        },
        "geoip": {
          "dynamic": true,
          "properties": {
            "location": {
              "type": "geo_point"
            }
          },
          "type": "object"
        },
        "@version": {
          "index": "not_analyzed",
          "type": "string"
        }
      },
      "_all": {
        "enabled": true,
        "omit_norms": true
      }
    }
  },
  "aliases": {}
}

# First, delete the current data, as we need to adjust
# the mapping

DELETE /logstash-test

# Now go ahead and index your logstash data...

PUT /logstash-test/logs/1
{
  "clientip": "12.34.0.1",
  "ip_source": "12.34.0.1",
  "message": "The very model of a modern major general",
  "somestring": "shoestring"
}

PUT /logstash-test/logs/2
{
  "clientip": "23.45.2.3",
  "ip_source": "23.45.2.3",
  "message": "Everyone needs a nemesis",
  "somestring": "stringthing"
}

# Check what we got back
#
# I was expecting to see:
#  - 'somestring' attribute should have a .raw subfield
#  - 'message' should not
#  - 'clientip' should have a .num subfield
#  - 'ip_source' should have a .num subfield
#
# ... but I don't see the subfields (not sure why)

GET /logstash-test/logs/1
GET /logstash-test/logs/2

# Check your mapping after you start indexing

GET logstash-test/_mapping/logs

# Do a search for it (remember it won't show up in
# a search until the index has passed its refresh
# interval)
#
# Remember to specify the subfield you need to work on

# I couldn't find an example of Kibana using a 'count'
# search-type, but it seems to use '"size": 0' instead.

### BUG!!! an ip_range with mask of 0.0.0.0/0 is not
# being treated correctly, the prefix-length of 0 is
# being taken as 'unset' and is being set as /32

GET logstash-test/_search
{
  "size": 0, 
  "aggs": {
    "ip_ranges": {
      "ip_range": {
        "field": "clientip.num",
        "ranges": [
          {
            "mask": "12.0.0.0/8"
          },
          {
            "mask": "0.0.0.0/1"
          },
          {
            "from": "0.0.0.0",
            "to": "255.255.255.255"
          },
          {
            "mask": "0.0.0.0/0"
          }
        ]
      }
    }
  }
}

# There is no IP address value that can legitimately be
# used as a "null" value for a CIDR prefix... 
# well, not in the IP address part; For IPv4, you could
# use /33 or such as an "null" value I suppose, if 
# you didn't want the overhead of adding an extra bit
# of information... so long as that implementation
# detail wasn't exposed to users.

[cameronkerrnz]

This was seen in Elasticsearch 1.6.0

[ruflin]

@cameronkerrnz Here is a potential fix: ruflin@3be9892

So far all tests are green, but I'm not sure if this could also have some unwanted side effects. Please have a look at it.

[clintongormley]

thanks @ruflin - you want to open a PR?

[ruflin]

Done: #12430

[jpountz]

Closed via #12430