feat: use latest apm-data to set success event.outcome when otel span.Status is Unset

[kyungeunni]

Motivation/summary

The latest apm-data has changed how we map span.Status: Unset when processing otel inputs. Currently, we set event.outcome: unknown when such input is received, but we decided to map it to event.outcome: success.

Checklist

• Update CHANGELOG.asciidoc
  - [ ] Update package changelog.yml (only if changes to apmpackage have been made)
• Documentation has been updated

For functional changes, consider:

• Is it observable through the addition of either logging or metrics?
• Is its use being published in telemetry to enable product improvement?
• Have system tests been added to avoid regression?

How to test these changes

Ingest Otel data, make sure the span.Status is Unset. Then check the transaction doc containing event.outcome: success

Related issues

[mergify]

This pull request does not have a backport label. Could you fix it @kyungeunni? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-7.17 is the label to automatically backport to the 7.17 branch.
• backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit.

NOTE: backport-skip has been added to this pull request.

[mergify]

This pull request is now in conflicts. Could you fix it @kyungeunni? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b bump-apm-data upstream/bump-apm-data
git merge upstream/main
git push upstream bump-apm-data

[kruskall]

@kyungeunni Can you fix the conflicts ?

(I think you might also need to run go mod tidy and make fmt update)

[kruskall]

I think the system tests approved docs need to be updated (make system-test check-approvals)

[marclop]

Sent data using sendotlp because it doesn't call span.SetStatus()

apm-server/systemtest/cmd/sendotlp/main.go

Lines 187 to 202 in c706600

	func generateSpans(ctx context.Context, tracer trace.Tracer) error {
	ctx, parent := tracer.Start(ctx, "parent")
	defer parent.End()
	_, child1 := tracer.Start(ctx, "child1")
	time.Sleep(10 * time.Millisecond)
	child1.AddEvent("an arbitrary event")
	child1.End()
	_, child2 := tracer.Start(ctx, "child2")
	time.Sleep(10 * time.Millisecond)
	child2.RecordError(errors.New("an exception occurred"))
	child2.End()
	return nil
	}

$ go run main.go -secret-token=$ELASTIC_APM_SECRET_TOKEN -endpoint=$APM_SERVER_ADDRESS                                                                                                                               <region:us-east-1>
2024-03-20T15:52:57+01:00	INFO	zapgrpc/zapgrpc.go:178	[core] [Channel #1] Channel created
2024-03-20T15:52:57+01:00	INFO	zapgrpc/zapgrpc.go:178	[core] [Channel #1] original dial target is: "89076c71eb054387ae69b2f50f425642.apm.us-west2.gcp.elastic-cloud.com:443"
2024-03-20T15:52:57+01:00	INFO	zapgrpc/zapgrpc.go:178	[core] [Channel #1] dial target "89076c71eb054387ae69b2f50f425642.apm.us-west2.gcp.elastic-cloud.com:443" parse failed: parse "89076c71eb054387ae69b2f50f425642.apm.us-west2.gcp.elastic-cloud.com:443": first path segment in URL cannot contain colon
2024-03-20T15:52:57+01:00	INFO	zapgrpc/zapgrpc.go:178	[core] [Channel #1] fallback to scheme "passthrough"
2024-03-20T15:52:57+01:00	INFO	zapgrpc/zapgrpc.go:178	[core] [Channel #1] parsed dial target is: {URL:{Scheme:passthrough Opaque: User: Host: Path:/89076c71eb054387ae69b2f50f425642.apm.us-west2.gcp.elastic-cloud.com:443 RawPath: OmitHost:false ForceQuery:false RawQuery: Fragment: RawFragment:}}
2024-03-20T15:52:57+01:00	INFO	zapgrpc/zapgrpc.go:178	[core] [Channel #1] Channel authority set to "89076c71eb054387ae69b2f50f425642.apm.us-west2.gcp.elastic-cloud.com:443"
2024-03-20T15:52:57+01:00	INFO	zapgrpc/zapgrpc.go:178	[core] [Channel #1] Resolver state updated: {
  "Addresses": [
    {
      "Addr": "89076c71eb054387ae69b2f50f425642.apm.us-west2.gcp.elastic-cloud.com:443",
      "ServerName": "",
      "Attributes": null,
      "BalancerAttributes": null,
      "Metadata": null
    }
  ],
  "Endpoints": [
    {
      "Addresses": [
        {
          "Addr": "89076c71eb054387ae69b2f50f425642.apm.us-west2.gcp.elastic-cloud.com:443",
          "ServerName": "",
          "Attributes": null,
          "BalancerAttributes": null,
          "Metadata": null
        }
      ],
      "Attributes": null
    }
  ],
  "ServiceConfig": null,
  "Attributes": null
} (resolver returned new addresses)
2024-03-20T15:52:57+01:00	INFO	zapgrpc/zapgrpc.go:178	[core] [Channel #1] Channel switches to new LB policy "pick_first"
2024-03-20T15:52:57+01:00	INFO	zapgrpc/zapgrpc.go:178	[core] [pick-first-lb 0x140003d4ba0] Received new config {
  "shuffleAddressList": false
}, resolver state {
  "Addresses": [
    {
      "Addr": "89076c71eb054387ae69b2f50f425642.apm.us-west2.gcp.elastic-cloud.com:443",
      "ServerName": "",
      "Attributes": null,
      "BalancerAttributes": null,
      "Metadata": null
    }
  ],
  "Endpoints": [
    {
      "Addresses": [
        {
          "Addr": "89076c71eb054387ae69b2f50f425642.apm.us-west2.gcp.elastic-cloud.com:443",
          "ServerName": "",
          "Attributes": null,
          "BalancerAttributes": null,
          "Metadata": null
        }
      ],
      "Attributes": null
    }
  ],
  "ServiceConfig": null,
  "Attributes": null
}
2024-03-20T15:52:57+01:00	INFO	zapgrpc/zapgrpc.go:178	[core] [Channel #1 SubChannel #2] Subchannel created
2024-03-20T15:52:57+01:00	INFO	zapgrpc/zapgrpc.go:178	[core] [Channel #1] Channel Connectivity change to CONNECTING
2024-03-20T15:52:57+01:00	INFO	zapgrpc/zapgrpc.go:178	[core] [Channel #1 SubChannel #2] Subchannel Connectivity change to CONNECTING
2024-03-20T15:52:57+01:00	INFO	zapgrpc/zapgrpc.go:178	[core] [Channel #1 SubChannel #2] Subchannel picks a new address "89076c71eb054387ae69b2f50f425642.apm.us-west2.gcp.elastic-cloud.com:443" to connect
2024-03-20T15:52:57+01:00	INFO	sendotlp/main.go:171	sending OTLP data to https://89076c71eb054387ae69b2f50f425642.apm.us-west2.gcp.elastic-cloud.com:443 (grpc)
2024-03-20T15:52:57+01:00	INFO	zapgrpc/zapgrpc.go:178	[core] [pick-first-lb 0x140003d4ba0] Received SubConn state update: 0x140003d4d50, {ConnectivityState:CONNECTING ConnectionError:<nil>}
2024-03-20T15:52:57+01:00	INFO	zapgrpc/zapgrpc.go:178	[core] [Channel #1 SubChannel #2] Subchannel Connectivity change to READY
2024-03-20T15:52:57+01:00	INFO	zapgrpc/zapgrpc.go:178	[core] [pick-first-lb 0x140003d4ba0] Received SubConn state update: 0x140003d4d50, {ConnectivityState:READY ConnectionError:<nil>}
2024-03-20T15:52:57+01:00	INFO	zapgrpc/zapgrpc.go:178	[core] [Channel #1] Channel Connectivity change to READY

Navigated to the APM UI and verified that transactions weren't marked as failed: