add tracing support

Signed-off-by: yy <yang.yang@daocloud.io> add some unit test Signed-off-by: yy <yang.yang@daocloud.io> git rebase Signed-off-by: yy <yang.yang@daocloud.io> expose configuration for envoy's RateLimitedAsResourceExhausted (projectcontour#4971) The Rate Limit filter in Envoy translates a 429 HTTP response code to UNAVAILABLE as specified in the gRPC mapping document, but Google recommends translating it to RESOURCE_EXHAUSTED (see https://github.com/grpc/grpc/blob/master/doc/http-grpc-status-mapping.md) This commit introduces a new setting to allow contour to forward the same parameter introduced in envoyproxy/envoy#4879 The default value is disabled to retain the original behaviour of returning UNAVAILABLE, as changing it would be a breaking change. Closes projectcontour#4901. Signed-off-by: Víctor Roldán Betancort <vroldanbet@authzed.com> Signed-off-by: yy <yang.yang@daocloud.io> rebase Signed-off-by: yy <yang.yang@daocloud.io> update tracing config validate Signed-off-by: yy <yang.yang@daocloud.io> make generate Signed-off-by: yy <yang.yang@daocloud.io> add chengelog Signed-off-by: yy <yang.yang@daocloud.io> update make general Signed-off-by: yy <yang.yang@daocloud.io> goimport Signed-off-by: yy <yang.yang@daocloud.io> update tracing Signed-off-by: yy <yang.yang@daocloud.io> fix golint Signed-off-by: yy <yang.yang@daocloud.io> update test Signed-off-by: yy <yang.yang@daocloud.io> delete unused code Signed-off-by: yy <yang.yang@daocloud.io> delete error file Signed-off-by: yy <yang.yang@daocloud.io> update changelog Signed-off-by: yy <yang.yang@daocloud.io> fix some mistake Signed-off-by: yy <yang.yang@daocloud.io> feat: Add HTTP support for External Auth (projectcontour#4994) Support globally configuring an external auth server which is enabled by default for all vhosts, both HTTP and HTTPS. Closes projectcontour#4954. Signed-off-by: claytonig <claytonivorgonsalves@gmail.com> Signed-off-by: yy <yang.yang@daocloud.io> refactor DAG and DAG consumers to support >2 Listeners (projectcontour#5128) Updates projectcontour#4960. Signed-off-by: Steve Kriss <krisss@vmware.com> Signed-off-by: yy <yang.yang@daocloud.io> resolve conflict Signed-off-by: yy <yang.yang@daocloud.io> fix Signed-off-by: yy <yang.yang@daocloud.io>
projectsesame · Mar 27, 2023 · 0ef6373 · 0ef6373
1 parent cc3b6f7
commit 0ef6373
Show file tree

Hide file tree

Showing 20 changed files with 1,662 additions and 4 deletions.
diff --git a/apis/projectcontour/v1alpha1/contourconfig.go b/apis/projectcontour/v1alpha1/contourconfig.go
@@ -81,6 +81,9 @@ type ContourConfigurationSpec struct {
 	// Contour's default is { address: "0.0.0.0", port: 8000 }.
 	// +optional
 	Metrics *MetricsConfig `json:"metrics,omitempty"`
+
+	// Tracing defines properties for exporting trace data to OpenTelemetry.
+	Tracing *TracingConfig `json:"tracing,omitempty"`
 }
 
 // XDSServerType is the type of xDS server implementation.
@@ -659,6 +662,53 @@ type RateLimitServiceConfig struct {
 	EnableResourceExhaustedCode *bool `json:"enableResourceExhaustedCode,omitempty"`
 }
 
+// TracingConfig defines properties for exporting trace data to OpenTelemetry.
+type TracingConfig struct {
+	// IncludePodDetail defines a flag.
+	// If it is true, contour will add the pod name and namespace to the span of the trace.
+	// the default is true.
+	// +optional
+	IncludePodDetail *bool `json:"includePodDetail,omitempty"`
+
+	// ServiceName defines the name for the service.
+	// contour's default is contour
+	ServiceName *string `json:"serviceName,omitempty"`
+
+	// OverallSampling defines the sampling rate of trace data.
+	// contour's default is 100
+	// +optional
+	OverallSampling *string `json:"overallSampling,omitempty"`
+
+	// OverallSampling defines maximum length of the request path
+	// to extract and include in the HttpUrl tag.
+	// contour's default is 256.
+	// +optional
+	MaxPathTagLength *uint32 `json:"maxPathTagLength,omitempty"`
+
+	// CustomTags defines a list of custom tags with unique tag name.
+	// +optional
+	CustomTags []*CustomTag `json:"customTags,omitempty"`
+
+	// ExtensionService identifies the extension service defining the otle-collector.
+	ExtensionService NamespacedName `json:"extensionService,omitempty"`
+}
+
+// CustomTag defines custom tags with unique tag name
+// to create tags for the active span.
+type CustomTag struct {
+	// TagName is the unique name of the custom tag.
+	TagName string `json:"tagName,omitempty"`
+
+	// Literal is a static custom tag value.
+	// +optional
+	Literal string `json:"literal,omitempty"`
+
+	// RequestHeaderName indicates which request header
+	// the label value is obtained from.
+	// +optional
+	RequestHeaderName string `json:"requestHeaderName,omitempty"`
+}
+
 // PolicyConfig holds default policy used if not explicitly set by the user
 type PolicyConfig struct {
 	// RequestHeadersPolicy defines the request headers set/removed on all routes

diff --git a/apis/projectcontour/v1alpha1/zz_generated.deepcopy.go b/apis/projectcontour/v1alpha1/zz_generated.deepcopy.go
diff --git a/changelogs/unreleased/5043-yangyy93-minor.md b/changelogs/unreleased/5043-yangyy93-minor.md
@@ -0,0 +1,14 @@
+## Add Tracing Support
+
+Contour now supports exporting tracing data to [OpenTelemetry][1]
+
+The Contour configuration file and ContourConfiguration CRD will be extended with a new optional `tracing` section. This configuration block, if present, will enable tracing and will define the trace properties needed to generate and export trace data.
+
+### Contour supports the following configurations
+- Custom service name, the default is `contour`.
+- Custom sampling rate, the default is `100`.
+- Custom the maximum length of the request path, the default is `256`.
+- Customize span tags from literal and request headers.
+- Customize whether to include the pod's hostname and namespace.
+
+[1]: https://opentelemetry.io/
diff --git a/cmd/contour/serve.go b/cmd/contour/serve.go
@@ -24,6 +24,8 @@ import (
 	"time"
 
 	"github.com/alecthomas/kingpin/v2"
+	"k8s.io/utils/pointer"
+
 	envoy_server_v3 "github.com/envoyproxy/go-control-plane/pkg/server/v3"
 	contour_api_v1 "github.com/projectcontour/contour/apis/projectcontour/v1"
 	contour_api_v1alpha1 "github.com/projectcontour/contour/apis/projectcontour/v1alpha1"
@@ -354,6 +356,10 @@ func (s *Server) doServe() error {
 		ConnectionBalancer:           contourConfiguration.Envoy.Listener.ConnectionBalancer,
 	}
 
+	if listenerConfig.TracingConfig, err = s.setupTracingService(contourConfiguration.Tracing); err != nil {
+		return err
+	}
+
 	if listenerConfig.RateLimitConfig, err = s.setupRateLimitService(contourConfiguration); err != nil {
 		return err
 	}
@@ -594,6 +600,75 @@ func (s *Server) doServe() error {
 	return s.mgr.Start(signals.SetupSignalHandler())
 }
 
+func (s *Server) setupTracingService(tracingConfig *contour_api_v1alpha1.TracingConfig) (*dag.TracingConfig, error) {
+	if tracingConfig == nil {
+		return nil, nil
+	}
+
+	// ensure the specified ExtensionService exists
+	extensionSvc := &contour_api_v1alpha1.ExtensionService{}
+	key := client.ObjectKey{
+		Namespace: tracingConfig.ExtensionService.Namespace,
+		Name:      tracingConfig.ExtensionService.Name,
+	}
+	// Using GetAPIReader() here because the manager's caches won't be started yet,
+	// so reads from the manager's client (which uses the caches for reads) will fail.
+	if err := s.mgr.GetAPIReader().Get(context.Background(), key, extensionSvc); err != nil {
+		return nil, fmt.Errorf("error getting tracing extension service %s: %v", key, err)
+	}
+	// get the response timeout from the ExtensionService
+	var responseTimeout timeout.Setting
+	var err error
+
+	if tp := extensionSvc.Spec.TimeoutPolicy; tp != nil {
+		responseTimeout, err = timeout.Parse(tp.Response)
+		if err != nil {
+			return nil, fmt.Errorf("error parsing tracing extension service %s response timeout: %v", key, err)
+		}
+	}
+
+	var sni string
+	if extensionSvc.Spec.UpstreamValidation != nil {
+		sni = extensionSvc.Spec.UpstreamValidation.SubjectName
+	}
+
+	var customTags []*dag.CustomTag
+
+	if pointer.BoolDeref(tracingConfig.IncludePodDetail, true) {
+		customTags = append(customTags, &dag.CustomTag{
+			TagName:         "podName",
+			EnvironmentName: "HOSTNAME",
+		}, &dag.CustomTag{
+			TagName:         "podNamespaceName",
+			EnvironmentName: "CONTOUR_NAMESPACE",
+		})
+	}
+
+	for _, customTag := range tracingConfig.CustomTags {
+		customTags = append(customTags, &dag.CustomTag{
+			TagName:           customTag.TagName,
+			Literal:           customTag.Literal,
+			RequestHeaderName: customTag.RequestHeaderName,
+		})
+	}
+
+	overallSampling, err := strconv.ParseFloat(*tracingConfig.OverallSampling, 64)
+	if err != nil || overallSampling == 0 {
+		overallSampling = 100.0
+	}
+
+	return &dag.TracingConfig{
+		ServiceName:      pointer.StringDeref(tracingConfig.ServiceName, "contour"),
+		ExtensionService: key,
+		SNI:              sni,
+		Timeout:          responseTimeout,
+		OverallSampling:  overallSampling,
+		MaxPathTagLength: pointer.Uint32Deref(tracingConfig.MaxPathTagLength, 256),
+		CustomTags:       customTags,
+	}, nil
+
+}
+
 func (s *Server) setupRateLimitService(contourConfiguration contour_api_v1alpha1.ContourConfigurationSpec) (*xdscache_v3.RateLimitConfig, error) {
 	if contourConfiguration.RateLimitService == nil {
 		return nil, nil

diff --git a/cmd/contour/servecontext.go b/cmd/contour/servecontext.go
@@ -20,6 +20,7 @@ import (
 	"errors"
 	"fmt"
 	"os"
+	"strconv"
 	"strings"
 	"time"
 
@@ -30,6 +31,7 @@ import (
 	"github.com/projectcontour/contour/internal/ref"
 	xdscache_v3 "github.com/projectcontour/contour/internal/xdscache/v3"
 	"github.com/projectcontour/contour/pkg/config"
+	"k8s.io/utils/pointer"
 
 	"github.com/sirupsen/logrus"
 	"google.golang.org/grpc"
@@ -366,6 +368,30 @@ func (ctx *serveContext) convertToContourConfigurationSpec() contour_api_v1alpha
 		dnsLookupFamily = contour_api_v1alpha1.AllClusterDNSFamily
 	}
 
+	var tracingConfig *contour_api_v1alpha1.TracingConfig
+	if ctx.Config.Tracing.ExtensionService != "" {
+		namespacedName := k8s.NamespacedNameFrom(ctx.Config.Tracing.ExtensionService)
+		var customTags []*contour_api_v1alpha1.CustomTag
+		for _, customTag := range ctx.Config.Tracing.CustomTags {
+			customTags = append(customTags, &contour_api_v1alpha1.CustomTag{
+				TagName:           customTag.TagName,
+				Literal:           customTag.Literal,
+				RequestHeaderName: customTag.RequestHeaderName,
+			})
+		}
+		tracingConfig = &contour_api_v1alpha1.TracingConfig{
+			IncludePodDetail: ctx.Config.Tracing.IncludePodDetail,
+			ServiceName:      pointer.String(ctx.Config.Tracing.ServiceName),
+			OverallSampling:  pointer.String(strconv.FormatFloat(ctx.Config.Tracing.OverallSampling, 'f', 1, 64)),
+			MaxPathTagLength: pointer.Uint32(ctx.Config.Tracing.MaxPathTagLength),
+			CustomTags:       customTags,
+			ExtensionService: contour_api_v1alpha1.NamespacedName{
+				Name:      namespacedName.Name,
+				Namespace: namespacedName.Namespace,
+			},
+		}
+	}
+
 	var rateLimitService *contour_api_v1alpha1.RateLimitServiceConfig
 	if ctx.Config.RateLimitService.ExtensionService != "" {
 
@@ -538,6 +564,7 @@ func (ctx *serveContext) convertToContourConfigurationSpec() contour_api_v1alpha
 		RateLimitService:            rateLimitService,
 		Policy:                      policy,
 		Metrics:                     &contourMetrics,
+		Tracing:                   tracingConfig,
 	}
 
 	xdsServerType := contour_api_v1alpha1.ContourServerType

diff --git a/cmd/contour/servecontext_test.go b/cmd/contour/servecontext_test.go
@@ -24,17 +24,17 @@ import (
 	"testing"
 	"time"
 
-	"github.com/projectcontour/contour/pkg/config"
-	"github.com/tsaarni/certyaml"
-
 	contour_api_v1 "github.com/projectcontour/contour/apis/projectcontour/v1"
 	contour_api_v1alpha1 "github.com/projectcontour/contour/apis/projectcontour/v1alpha1"
 	"github.com/projectcontour/contour/internal/contourconfig"
 	envoy_v3 "github.com/projectcontour/contour/internal/envoy/v3"
 	"github.com/projectcontour/contour/internal/fixture"
 	"github.com/projectcontour/contour/internal/ref"
+	"github.com/projectcontour/contour/pkg/config"
 	"github.com/stretchr/testify/assert"
+	"github.com/tsaarni/certyaml"
 	"google.golang.org/grpc"
+	"k8s.io/utils/pointer"
 )
 
 func TestServeContextProxyRootNamespaces(t *testing.T) {
@@ -741,6 +741,50 @@ func TestConvertServeContext(t *testing.T) {
 				return cfg
 			},
 		},
+		"tracing config": {
+			getServeContext: func(ctx *serveContext) *serveContext {
+				ctx.Config.Tracing = config.Tracing{
+					IncludePodDetail: nil,
+					ServiceName:      "contour",
+					OverallSampling:  100,
+					MaxPathTagLength: 256,
+					CustomTags: []config.CustomTag{
+						{
+							TagName: "literal",
+							Literal: "this is literal",
+						},
+						{
+							TagName:           "header",
+							RequestHeaderName: ":method",
+						},
+					},
+					ExtensionService: "otel/otel-collector",
+				}
+				return ctx
+			},
+			getContourConfiguration: func(cfg contour_api_v1alpha1.ContourConfigurationSpec) contour_api_v1alpha1.ContourConfigurationSpec {
+				cfg.Tracing = &contour_api_v1alpha1.TracingConfig{
+					ServiceName:      pointer.String("contour"),
+					OverallSampling:  pointer.String("100.0"),
+					MaxPathTagLength: pointer.Uint32(256),
+					CustomTags: []*contour_api_v1alpha1.CustomTag{
+						{
+							TagName: "literal",
+							Literal: "this is literal",
+						},
+						{
+							TagName:           "header",
+							RequestHeaderName: ":method",
+						},
+					},
+					ExtensionService: contour_api_v1alpha1.NamespacedName{
+						Name:      "otel-collector",
+						Namespace: "otel",
+					},
+				}
+				return cfg
+			},
+		},
 	}
 
 	for name, tc := range cases {