[connector/datadog] - Use waitgroup instead of closing a channel and waiting on it

[VihasMakwana]

Description

The run() goroutine closes the channel in defer.
There are two sceanrios:

1. Best case scenario: no error encountered while calling ConsumeMetrics.
2. Worst case scenario: we encounter an error while calling ConsumeMetrics, causing the goroutine to exit early and as a result, close the c.exit chan.

We close the channel, again, in Shutdown and this results in a "send on closed channel" panic in worst case scenario. In best case scenario, no panic is encountered.

This PR adds waitgroup to avoid the panic.

Link to tracking issue

Fixes #40845

Testing

Added a test case

[jade-guiton-dd]

I like the fix, but I'm wondering if it makes sense to return from the goroutine when the next consumer returns an error (code link) in the first place, instead of continueing the loop like the case of a failed conversion 🤔 Doesn't this mean that any error in the metrics pipeline will cause the connector to stop working? @songy23

[VihasMakwana]

Doesn't this mean that any error in the metrics pipeline will cause the connector to stop working?

Yes @jade-guiton-dd. You're right.
If there's any error in the pipeline, the connector will stop working.
We can just continue like we do here

opentelemetry-collector-contrib/connector/datadogconnector/connector_native.go

Lines 154 to 158 in ba9f4b4

	mx, err = c.translator.StatsToMetrics(stats)
	if err != nil {
	c.logger.Error("Failed to convert stats to metrics", zap.Error(err))
	continue
	}

and just log the error.

Let me know if it's good @songy23

[songy23]

Thanks @VihasMakwana LGTM

[jade-guiton-dd]

@songy23 The reason I pinged you was to ask whether it would make sense to replace the whole PR with just return → continue in the loop, since you wrote this code