[ForgeRock] Event indexing failuring in system test #6178
Description
While adding more through checks to the test output an error was discovered. And event cannot be index and hence is dropped. Using assert.hit_count in the system test config could have detected this before now.
failed to parse field [forgerock.revision] of type [long] in document with id 'mPyJCogBe7d392w8_tit'. Preview of field's value: '6e415003-2c2e-46a1-9546-8eaa4e5f68d8-2545'
Full error:
test case failed: one or more errors found while examining elastic-agent.log: [0] found error "Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Date(2023, time.May, 11, 11, 19, 54, 517041685, time.Local), Meta:{\"input_id\":\"httpjson-forgerock-0b8ee9c8-1441-4be7-a33e-e4f42e02ed50\",\"raw_index\":\"logs-forgerock.idm_activity-ep\",\"stream_id\":\"httpjson-forgerock.idm_activity-0b8ee9c8-1441-4be7-a33e-e4f42e02ed50\"}, Fields:{\"agent\":{\"ephemeral_id\":\"a9b5648c-48f7-4a04-bf0e-1784ce803b02\",\"id\":\"59955c7e-efdf-47a8-8e7e-662b8f5d8a25\",\"name\":\"docker-fleet-agent\",\"type\":\"filebeat\",\"version\":\"8.7.1\"},\"cloud\":{\"account\":{\"id\":\"elastic-ci-prod\"},\"availability_zone\":\"us-central1-f\",\"instance\":{\"id\":\"327638486640327464\",\"name\":\"fleet-ci-immutable-ubuntu-2004-1683801536921763817\"},\"machine\":{\"type\":\"n1-highmem-8\"},\"project\":{\"id\":\"elastic-ci-prod\"},\"provider\":\"gcp\",\"service\":{\"name\":\"GCE\"}},\"data_stream\":{\"dataset\":\"forgerock.idm_activity\",\"namespace\":\"ep\",\"type\":\"logs\"},\"ecs\":{\"version\":\"8.0.0\"},\"elastic_agent\":{\"id\":\"59955c7e-efdf-47a8-8e7e-662b8f5d8a25\",\"snapshot\":false,\"version\":\"8.7.1\"},\"event\":{\"created\":\"2023-05-11T11:19:54.517Z\",\"dataset\":\"forgerock.idm_activity\"},\"input\":{\"type\":\"httpjson\"},\"message\":\"{\\\"payload\\\":{\\\"_id\\\":\\\"a9a32d9e-7029-45e6-b581-eafb5d502273-268906\\\",\\\"changedFields\\\":[],\\\"eventName\\\":\\\"relationship_created\\\",\\\"level\\\":\\\"INFO\\\",\\\"message\\\":\\\"Relationship originating from managed/alpha_organization/e6df3df4-c798-4187-ba06-db8e6ae3db88 via the relationship field parent and referencing managed/alpha_organization/c4de605d-9d1b-439e-9ea8-9aba47e01008 was created.\\\",\\\"objectId\\\":\\\"managed/alpha_organization/e6df3df4-c798-4187-ba06-db8e6ae3db88/parent/bb20cd10-e6ad-48fd-8ef1-e8d4c3f7859f\\\",\\\"operation\\\":\\\"CREATE\\\",\\\"passwordChanged\\\":false,\\\"revision\\\":\\\"6e415003-2c2e-46a1-9546-8eaa4e5f68d8-2545\\\",\\\"runAs\\\":\\\"9120c7db-d7e6-4b51-b805-07bbee7a4bb9\\\",\\\"source\\\":\\\"audit\\\",\\\"status\\\":\\\"SUCCESS\\\",\\\"timestamp\\\":\\\"2022-11-01T18:02:39.882Z\\\",\\\"topic\\\":\\\"activity\\\",\\\"transactionId\\\":\\\"1667325742545-ee41d6454a6b4a815b69-24798/0\\\",\\\"userId\\\":\\\"9120c7db-d7e6-4b51-b805-07bbee7a4bb9\\\"},\\\"source\\\":\\\"idm-activity\\\",\\\"timestamp\\\":\\\"2022-11-01T18:02:39.885141134Z\\\",\\\"type\\\":\\\"application/json\\\"}\",\"tags\":[\"forwarded\",\"forgerock-audit\",\"forgerock-idm-activity\"]}, Private:(*cursor.updateOp)(0xc0006e1240), TimeSeries:false}, Flags:0x1, Cache:publisher.EventCache{m:mapstr.M(nil)}} (status=400): {\"type\":\"mapper_parsing_exception\",\"reason\":\"failed to parse field [forgerock.revision] of type [long] in document with id 'mPyJCogBe7d392w8_tit'. Preview of field's value: '6e415003-2c2e-46a1-9546-8eaa4e5f68d8-2545'\",\"caused_by\":{\"type\":\"illegal_argument_exception\",\"reason\":\"For input string: \\\"6e415003-2c2e-46a1-9546-8eaa4e5f68d8-2545\\\"\"}}, dropping event!"
Relates: elastic/elastic-package#1188
CI run (ephemeral): https://fleet-ci.elastic.co/blue/organizations/jenkins/Ingest-manager%2Fintegrations/detail/PR-6133/6/tests/