Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metricbeats Kubernetes event module unable to parse empty date field #5093

Closed
mikelorant opened this issue Sep 5, 2017 · 2 comments

Comments

Projects
None yet
2 participants
@mikelorant
Copy link

commented Sep 5, 2017

Issue
Getting the following error when trying to parse an event.

2017-09-04T02:11:28Z WARN Can not index event (status=400): {"type":"mapper_parsing_exception","reason":"failed to parse [kubernetes.event.metadata.timestamp.deleted]","caused_by":{"type":"illegal_argument_exception","reason":"Invalid format: \"\""}}

Enabling debug mode showed the following event message.

2017-09-04T02:27:52Z DBG  Publish event: {
  "@timestamp": "2017-09-04T02:27:52.271Z",
  "@metadata": {
    "beat": "\u003cnot set\u003e",
    "type": "doc"
  },
  "kubernetes": {
    "event": {
      "message": "Created pod: ywqvd-kube-state-metrics-483864468-x39ft",
      "reason": "SuccessfulCreate",
      "type": "Normal",
      "count": 1,
      "involved_object": {
        "kind": "ReplicaSet",
        "uid": "86ada97c-9107-11e7-81a9-024fe15315e7",
        "api_version": "extensions",
        "resource_version": "19155721",
        "name": "ywqvd-kube-state-metrics-483864468"
      },
      "metadata": {
        "generate_name": "",
        "uid": "a6da2702-9118-11e7-81a9-024fe15315e7",
        "resource_version": "367834",
        "timestamp": {
          "created": "2017-09-04T02:27:52Z",
          "deleted": ""
        },
        "name": "ywqvd-kube-state-metrics-483864468.14e107918ea3c588",
        "namespace": "infrastructure-metrics-shared-v1",
        "self_link": "/api/v1/namespaces/infrastructure-metrics-shared-v1/events/ywqvd-kube-state-metrics-483864468.14e107918ea3c588"
      },
      "timestamp": {
        "first_occurrence": "2017-09-04T02:27:52.000Z",
        "last_occurrence": "2017-09-04T02:27:52.000Z"
      }
    }
  },
  "metricset": {
    "module": "kubernetes",
    "name": "event"
  },
  "beat": {
    "name": "ywqvd-metricbeat-4231345040-qks11",
    "hostname": "ywqvd-metricbeat-4231345040-qks11",
    "version": "6.0.0-beta2"
  },
  "meta": {
    "cloud": {
      "instance_id": "i-0f919437f290ba006",
      "machine_type": "r4.2xlarge",
      "region": "ap-southeast-2",
      "availability_zone": "ap-southeast-2a",
      "provider": "ec2"
    }
  }
}

The deleted timestamp field is empty.

Looking at the source code it is clear that the field is marked as date and does not expect to be empty.
https://github.com/elastic/beats/blob/master/metricbeat/module/kubernetes/event/_meta/fields.yml#L48

@vjsamuel identified that the root cause is likely the field has been set to string.
https://github.com/elastic/beats/blob/master/metricbeat/module/kubernetes/event/types.go#L8

@exekias

This comment has been minimized.

Copy link
Member

commented Sep 5, 2017

Thank you for the report! What version of Metricbeat are you using?

@mikelorant

This comment has been minimized.

Copy link
Author

commented Sep 6, 2017

We are using metricbeat 6.0.0 beta 2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.