ecs_task_observer: initial structure (#6121)

Adding a feature - These changes introduce a new receiver creator-compatible watch observer for ECS tasks.  It differs from the existing ECS observer in that it supports only sidecar deployments and doesn't have a hard dependency on the Prometheus receiver, instead being able to sync with a receiver creator instance by listing [container endpoints](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/957eb18037f000f5740c4044fdf5ffeee4311ebc/extension/observer/endpoints.go#L166).  This approach was landed on after reviewing the existing and ECS observer config and implementation and from following the guidance of the initial observer's author: #5316 (comment).  It does not resolve [the outstanding item](#1395 (comment)) for #1395, as I think observer endpoint creation should be added to the existing observer as well but should be done in unrelated changes.

**Link to tracking Issue:**
#5316

**Testing:**
~No tests as this time since the changes just include the initial types and readme.~ Basic factory test to help ensure receiver creator type compatibility.*

**Documentation:**
Adding a readme with high level plan and config documentation generated by project configschema tool.

extension/observer/ecstaskobserver/Makefile

@@ -0,0 +1 @@
+include ../../../Makefile.Common

extension/observer/ecstaskobserver/README.md

@@ -0,0 +1,101 @@
+# ECS Task Observer
+
+The `ecs_task_observer` is a [Receiver Creator](../../../receiver/receivercreator/README.md)-compatible "watch observer" that will detect and report
+container endpoints for the running ECS task of which your Collector instance is a member. It is designed for and only supports "sidecar" deployments
+to detect co-located containers. For cluster wide use cases you should use the [ECS Observer](../ecsobserver/README.md) with a corresponding Prometheus receiver.
+
+The Observer works by querying the available [task metadata endpoint](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-metadata-endpoint.html)
+and making all detected running containers available as endpoints for Receiver Creator usage. Because container metadata don't include any port mapping information,
+you must include service-specific port `dockerLabels` in your task definition container entries. A docker label of `ECS_TASK_OBSERVER_PORT` with a valid port
+value will be attempted to be parsed for each reported container by default.
+
+**An instance of the Collector must be running in the ECS task from which you want to detect containers.**
+
+## Example Config
+
+```yaml
+extensions:
+  ecs_task_observer:
+    # the task metadata endpoint. If not set, detected by first of ECS_CONTAINER_METADATA_URI_V4 and ECS_CONTAINER_METADATA_URI
+    # environment variables by default.
+    endpoint: http://my.task.metadata.endpoint
+    # the dockerLabels to use to try to extract target application ports. If not set "ECS_TASK_OBSERVER_PORT" will be used by default.
+    port_labels: [A_DOCKER_LABEL_CONTAINING_DESIRED_PORT, ANOTHER_DOCKER_LABEL_CONTAINING_DESIRED_PORT]
+    refresh_interval: 10s
+
+receivers:
+  receiver_creator:
+    receivers:
+      redis:
+        rule: type == "container" && name matches "redis"
+        config:
+          password: `container.labels["SECRET"]`
+    watch_observers: [ecs_task_observer]
+```
+
+The above config defines a custom task metadata endpoint and provides two port labels that will be used to set the resulting container endpoint's `port`.
+A corresponding redis container definition could look like the following:
+
+```json
+{
+  "containerDefinitions": [
+    {
+      "portMappings": [
+        {
+          "containerPort": 6379,
+          "hostPort": 6379
+        }
+      ],
+      "image": "redis",
+      "dockerLabels": {
+        "A_DOCKER_LABEL_CONTAINING_DESIRED_PORT": "6379",
+        "SECRET": "my-redis-auth"
+      },
+      "name": "redis"
+    }
+  ]
+}
+```
+
+
+### Config
+
+As a rest client-utilizing extension, most of the ECS Task Observer's configuration is inherited from the Collector core
+[HTTP Client Configuration Settings](https://github.com/open-telemetry/opentelemetry-collector/blob/main/config/confighttp/README.md#client-configuration).
+
+All fields are optional.
+
+| Name | Type | Default | Docs |
+| ---- | ---- | ------- | ---- |
+| endpoint |string| <no value> | The task metadata endpoint, detected from first of `ECS_CONTAINER_METADATA_URI_V4` and `ECS_CONTAINER_METADATA_URI` environment variables by default |
+| tls |[configtls-TLSClientSetting](#configtls-TLSClientSetting)| <no value> | TLSSetting struct exposes TLS client configuration.  |
+| read_buffer_size |int| <no value> | ReadBufferSize for HTTP client. See http.Transport.ReadBufferSize.  |
+| write_buffer_size |int| <no value> | WriteBufferSize for HTTP client. See http.Transport.WriteBufferSize.  |
+| timeout |[time-Duration](#time-Duration)| <no value> | Timeout parameter configures `http.Client.Timeout`.  |
+| headers |map[string]string| <no value> | Additional headers attached to each HTTP request sent by the client. Existing header values are overwritten if collision happens.  |
+| customroundtripper |func(http.RoundTripper) (http.RoundTripper, error)| <no value> | Custom Round Tripper to allow for individual components to intercept HTTP requests  |
+| auth |[configauth-Authentication](#configauth-Authentication)| <no value> | Auth configuration for outgoing HTTP calls.  |
+| refresh_interval |[time-Duration](#time-Duration)| 30s | RefreshInterval determines how frequency at which the observer needs to poll for collecting new information about task containers.  |
+| port_labels |[]string| `[ECS_TASK_OBSERVER_PORT]` | PortLabels is a list of container Docker labels from which to obtain the observed Endpoint port. The first label with valid port found will be used.  If no PortLabels provided, default of ECS_TASK_OBSERVER_PORT will be used.  |
+
+### configtls-TLSClientSetting
+
+| Name | Type | Default | Docs |
+| ---- | ---- | ------- | ---- |
+| ca_file |string| <no value> | Path to the CA cert. For a client this verifies the server certificate. For a server this verifies client certificates. If empty uses system root CA. (optional)  |
+| cert_file |string| <no value> | Path to the TLS cert to use for TLS required connections. (optional)  |
+| key_file |string| <no value> | Path to the TLS key to use for TLS required connections. (optional)  |
+| min_version |string| <no value> | MinVersion sets the minimum TLS version that is acceptable. If not set, TLS 1.0 is used. (optional)  |
+| max_version |string| <no value> | MaxVersion sets the maximum TLS version that is acceptable. If not set, TLS 1.3 is used. (optional)  |
+| insecure |bool| <no value> | In gRPC when set to true, this is used to disable the client transport security. See https://godoc.org/google.golang.org/grpc#WithInsecure. In HTTP, this disables verifying the server's certificate chain and host name (InsecureSkipVerify in the tls Config). Please refer to https://godoc.org/crypto/tls#Config for more information. (optional, default false)  |
+| insecure_skip_verify |bool| <no value> | InsecureSkipVerify will enable TLS but not verify the certificate.  |
+| server_name_override |string| <no value> | ServerName requested by client for virtual hosting. This sets the ServerName in the TLSConfig. Please refer to https://godoc.org/crypto/tls#Config for more information. (optional)  |
+
+### configauth-Authentication
+
+| Name | Type | Default | Docs |
+| ---- | ---- | ------- | ---- |
+| authenticator |[config-ComponentID](#config-ComponentID)| <no value> | AuthenticatorID specifies the name of the extension to use in order to authenticate the incoming data point.  |
+
+### time-Duration
+An optionally signed sequence of decimal numbers, each with a unit suffix, such as `300ms`, `-1.5h`, or `2h45m`. Valid time units are `ns`, `us`, `ms`, `s`, `m`, `h`.

extension/observer/ecstaskobserver/config.go

@@ -0,0 +1,53 @@
+// Copyright  The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package ecstaskobserver
+
+import (
+	"time"
+
+	"go.opentelemetry.io/collector/config"
+	"go.opentelemetry.io/collector/config/confighttp"
+)
+
+const (
+	defaultRefreshInterval = 30 * time.Second
+	defaultPortLabel       = "ECS_TASK_OBSERVER_PORT"
+)
+
+type Config struct {
+	config.ExtensionSettings      `mapstructure:",squash"`
+	confighttp.HTTPClientSettings `mapstructure:",squash"`
+
+	// RefreshInterval determines how frequency at which the observer
+	// needs to poll for collecting new information about task containers.
+	RefreshInterval time.Duration `mapstructure:"refresh_interval" yaml:"refresh_interval"`
+
+	// PortLabels is a list of container Docker labels from which to obtain the observed Endpoint port.
+	// The first label with valid port found will be used.  If no PortLabels provided, default of
+	// ECS_TASK_OBSERVER_PORT will be used.
+	PortLabels []string `mapstructure:"port_labels" yaml:"port_labels"`
+}
+
+func (c Config) Validate() error {
+	return nil
+}
+
+func defaultConfig() Config {
+	return Config{
+		ExtensionSettings: config.NewExtensionSettings(config.NewComponentID(typeStr)),
+		RefreshInterval:   defaultRefreshInterval,
+		PortLabels:        []string{defaultPortLabel},
+	}
+}

extension/observer/ecstaskobserver/extension.go

@@ -0,0 +1,35 @@
+// Copyright  The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package ecstaskobserver
+
+import (
+	"go.opentelemetry.io/collector/component"
+
+	"github.com/open-telemetry/opentelemetry-collector-contrib/extension/observer"
+)
+
+var _ component.Extension = (*ecsTaskObserver)(nil)
+var _ observer.EndpointsLister = (*ecsTaskObserver)(nil)
+
+type ecsTaskObserver struct {
+	component.Extension
+	config           *Config
+	endpointsWatcher *observer.EndpointsWatcher
+	telemetry        component.TelemetrySettings
+}
+
+// ListEndpoints is invoked by an observer.EndpointsWatcher helper to report task container endpoints.
+// It's required to implement observer.EndpointsLister
+func (e *ecsTaskObserver) ListEndpoints() []observer.Endpoint { return nil }

extension/observer/ecstaskobserver/factory.go

@@ -0,0 +1,57 @@
+// Copyright  The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package ecstaskobserver
+
+import (
+	"context"
+
+	"go.opentelemetry.io/collector/component"
+	"go.opentelemetry.io/collector/component/componenthelper"
+	"go.opentelemetry.io/collector/config"
+	"go.opentelemetry.io/collector/extension/extensionhelper"
+)
+
+const (
+	typeStr config.Type = "ecs_task_observer"
+)
+
+// NewFactory creates a factory for ECSTaskObserver extension.
+func NewFactory() component.ExtensionFactory {
+	return extensionhelper.NewFactory(
+		typeStr,
+		createDefaultConfig,
+		createExtension)
+}
+
+func createDefaultConfig() config.Extension {
+	cfg := defaultConfig()
+	return &cfg
+}
+
+func createExtension(
+	_ context.Context,
+	params component.ExtensionCreateSettings,
+	cfg config.Extension,
+) (component.Extension, error) {
+	obsCfg := cfg.(*Config)
+	e := &ecsTaskObserver{
+		Extension:        componenthelper.New(),
+		config:           obsCfg,
+		endpointsWatcher: nil,
+		telemetry:        params.TelemetrySettings,
+	}
+
+	return e, nil
+}

extension/observer/ecstaskobserver/factory_test.go

@@ -0,0 +1,35 @@
+// Copyright  The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package ecstaskobserver
+
+import (
+	"context"
+	"testing"
+
+	"github.com/stretchr/testify/require"
+	"go.opentelemetry.io/collector/component/componenttest"
+
+	"github.com/open-telemetry/opentelemetry-collector-contrib/extension/observer"
+)
+
+func TestFactoryCreatedExtensionIsEndpointsLister(t *testing.T) {
+	etoFactory := NewFactory()
+	eto, err := etoFactory.CreateExtension(
+		context.Background(), componenttest.NewNopExtensionCreateSettings(), etoFactory.CreateDefaultConfig(),
+	)
+	require.NoError(t, err)
+	require.NotNil(t, eto)
+	require.Implements(t, (*observer.EndpointsLister)(nil), eto)
+}

extension/observer/ecstaskobserver/go.mod

@@ -0,0 +1,46 @@
+module github.com/open-telemetry/opentelemetry-collector-contrib/extension/observer/ecstaskobserver
+
+go 1.17
+
+require (
+	github.com/open-telemetry/opentelemetry-collector-contrib/extension/observer v0.0.0-00010101000000-000000000000
+	github.com/stretchr/testify v1.7.0
+	go.opentelemetry.io/collector v0.38.1-0.20211103215828-cffbecb2ac9e
+)
+
+require (
+	github.com/davecgh/go-spew v1.1.1 // indirect
+	github.com/felixge/httpsnoop v1.0.2 // indirect
+	github.com/fsnotify/fsnotify v1.4.9 // indirect
+	github.com/gogo/protobuf v1.3.2 // indirect
+	github.com/golang/protobuf v1.5.2 // indirect
+	github.com/knadh/koanf v1.3.0 // indirect
+	github.com/mitchellh/copystructure v1.2.0 // indirect
+	github.com/mitchellh/mapstructure v1.4.2 // indirect
+	github.com/mitchellh/reflectwalk v1.0.2 // indirect
+	github.com/pelletier/go-toml v1.9.3 // indirect
+	github.com/pkg/errors v0.9.1 // indirect
+	github.com/pmezard/go-difflib v1.0.0 // indirect
+	github.com/rogpeppe/go-internal v1.6.1 // indirect
+	github.com/rs/cors v1.8.0 // indirect
+	github.com/spf13/cast v1.4.1 // indirect
+	go.opentelemetry.io/collector/model v0.38.0 // indirect
+	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.26.0 // indirect
+	go.opentelemetry.io/otel v1.1.0 // indirect
+	go.opentelemetry.io/otel/internal/metric v0.24.0 // indirect
+	go.opentelemetry.io/otel/metric v0.24.0 // indirect
+	go.opentelemetry.io/otel/trace v1.1.0 // indirect
+	go.uber.org/atomic v1.9.0 // indirect
+	go.uber.org/multierr v1.7.0 // indirect
+	go.uber.org/zap v1.19.1 // indirect
+	golang.org/x/net v0.0.0-20210614182718-04defd469f4e // indirect
+	golang.org/x/sys v0.0.0-20210816074244-15123e1e1f71 // indirect
+	golang.org/x/text v0.3.6 // indirect
+	google.golang.org/genproto v0.0.0-20210604141403-392c879c8b08 // indirect
+	google.golang.org/grpc v1.41.0 // indirect
+	google.golang.org/protobuf v1.27.1 // indirect
+	gopkg.in/yaml.v2 v2.4.0 // indirect
+	gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect
+)
+
+replace github.com/open-telemetry/opentelemetry-collector-contrib/extension/observer => ../