Skip to content
This repository has been archived by the owner on Oct 3, 2023. It is now read-only.

Add resource to protocol #137

Merged
merged 1 commit into from
Nov 2, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/opencensus/proto/agent/metrics/v1/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ proto_library(
deps = [
"//opencensus/proto/agent/common/v1:common_proto",
"//opencensus/proto/metrics/v1:metrics_proto",
"//opencensus/proto/resource/v1:resource_proto",
],
)

Expand Down Expand Up @@ -51,6 +52,7 @@ go_proto_library(
proto_deps = [
"//opencensus/proto/agent/common/v1:common_proto_go",
"//opencensus/proto/metrics/v1:metrics_proto_go",
"//opencensus/proto/resource/v1:resource_proto_go",
],
pb_options = [
# omit the go_package declared in proto files to make bazel works as expect
Expand Down
8 changes: 8 additions & 0 deletions src/opencensus/proto/agent/metrics/v1/metrics_service.proto
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ package opencensus.proto.agent.metrics.v1;

import "opencensus/proto/agent/common/v1/common.proto";
import "opencensus/proto/metrics/v1/metrics.proto";
import "opencensus/proto/resource/v1/resource.proto";

option java_multiple_files = true;
option java_package = "io.opencensus.proto.agent.metrics.v1";
Expand All @@ -42,6 +43,13 @@ message ExportMetricsServiceRequest {

// A list of metrics that belong to the last received Node.
repeated opencensus.proto.metrics.v1.Metric metrics = 2;

// The resource for the metrics in this message that do not have an explicit
// resource set.
// If unset, the most recently set resource in the RPC stream applies. It is
// valid to never be set within a stream, e.g. when no resource info is known
// at all or when all sent metrics have an explicit resource set.
opencensus.proto.resource.v1.Resource resource = 3;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One intention for these protocol is to be used for example when collect data from multiple nodes on the same RPC stream:
{App1, App2} -> oc-agent1 -> oc-agent2

Let's say oc-agent1 receives:

  • From App1 {Node1, Metrics1, Resource1}
  • From App2 {Node2, Metrics2, null (no resource)}

Then oc-agent1 sends to oc-agent2:

  • {Node1, Metrics1, Resource1}
  • {Node2, Metrics2, null (no resource)}

Based on the current description Metrics2 will be Resource1 which is not correct.

Couple of suggestions:

  1. If no-resource then we should send an empty Resource. or type = unknown.
  2. Resources are cached per Node. So I would say "If unset, the most recently set resources in the RPC stream for the same Node ..."

Copy link
Contributor Author

@fabxc fabxc Oct 17, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreeing with 1) – empty resource seems fine. I can't see any case where we'd want to distinguish an unset and an empty resource.

Suggestion 2. concerns me a bit, e.g. in particular this case:

  • 1) app1 -> {node1, metrics1, resource1}
  • 2) app2 -> {node2, metrics2, resource2}
  • 3..999) without app1 appearing
  • 1000) app1 -> {node1, metrics3, empty/unset}

It seems awkward to jump back to using resource1 in step 1000. For long streams with lots of senders, this could become rather hard to reason about during debugging. It also requires the cache of a single connection to grow linearly with the number of apps multiplexed over it.
That's probably not a problem in most use cases, but certainly an easy way to DDoS the agent.

Would it make sense to slightly alter this to: "A change in Node always resets the most recent Resource"?

Effectively, changing the node requires the resource to be changed too (or be considered empty after). This way, the node and resource cache of a connection have a constant size of 1 as well.

}

message ExportMetricsServiceResponse {
Expand Down
2 changes: 2 additions & 0 deletions src/opencensus/proto/agent/trace/v1/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ proto_library(
srcs = ["trace_service.proto"],
deps = [
"//opencensus/proto/agent/common/v1:common_proto",
"//opencensus/proto/resource/v1:resource_proto",
"//opencensus/proto/trace/v1:trace_proto",
"//opencensus/proto/trace/v1:trace_config_proto",
],
Expand Down Expand Up @@ -51,6 +52,7 @@ go_proto_library(
],
proto_deps = [
"//opencensus/proto/agent/common/v1:common_proto_go",
"//opencensus/proto/resource/v1:resource_proto_go",
"//opencensus/proto/trace/v1:trace_proto_go",
"//opencensus/proto/trace/v1:trace_config_proto_go",
],
Expand Down
7 changes: 7 additions & 0 deletions src/opencensus/proto/agent/trace/v1/trace_service.proto
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ syntax = "proto3";
package opencensus.proto.agent.trace.v1;

import "opencensus/proto/agent/common/v1/common.proto";
import "opencensus/proto/resource/v1/resource.proto";
import "opencensus/proto/trace/v1/trace.proto";
import "opencensus/proto/trace/v1/trace_config.proto";

Expand Down Expand Up @@ -72,6 +73,12 @@ message ExportTraceServiceRequest {

// A list of Spans that belong to the last received Node.
repeated opencensus.proto.trace.v1.Span spans = 2;

// The resource for the spans in this message that do not have an explicit
// resource set.
// If unset, the most recently set resource in the RPC stream applies. It is
// valid to never be set within a stream, e.g. when no resource info is known.
opencensus.proto.resource.v1.Resource resource = 3;
}

message ExportTraceServiceResponse {
Expand Down
4 changes: 4 additions & 0 deletions src/opencensus/proto/metrics/v1/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ proto_library(
deps = [
"@com_google_protobuf//:timestamp_proto",
"@com_google_protobuf//:wrappers_proto",
"//opencensus/proto/resource/v1:resource_proto",
],
)

Expand All @@ -48,6 +49,9 @@ go_proto_library(
# omit the go_package declared in proto files to make bazel works as expect
"paths=source_relative",
],
proto_deps = [
"//opencensus/proto/resource/v1:resource_proto_go",
],
deps = [
"@com_github_golang_protobuf//ptypes/timestamp:go_default_library",
"@com_github_golang_protobuf//ptypes/wrappers:go_default_library",
Expand Down
5 changes: 5 additions & 0 deletions src/opencensus/proto/metrics/v1/metrics.proto
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ package opencensus.proto.metrics.v1;

import "google/protobuf/timestamp.proto";
import "google/protobuf/wrappers.proto";
import "opencensus/proto/resource/v1/resource.proto";

option go_package = "github.com/census-instrumentation/opencensus-proto/gen-go/metrics/v1";

Expand All @@ -48,6 +49,10 @@ message Metric {
// One or more timeseries for a single metric, where each timeseries has
// one or more points.
repeated TimeSeries timeseries = 3;

// The resource for the metric. If unset, it may be set to a default value
// provided for a sequence of messages in an RPC stream.
opencensus.proto.resource.v1.Resource resource = 4;
}

// Defines a metric type and its schema.
Expand Down
41 changes: 41 additions & 0 deletions src/opencensus/proto/resource/v1/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Copyright 2018, OpenCensus Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

package(default_visibility = ["//visibility:public"])

load("@org_pubref_rules_protobuf//go:rules.bzl", "go_proto_library")

proto_library(
name = "resource_proto",
srcs = ["resource.proto"],
)

cc_proto_library(
name = "resource_proto_cc",
deps = [":resource_proto"],
)

java_proto_library(
name = "resource_proto_java",
deps = [":resource_proto"],
)

go_proto_library(
name = "resource_proto_go",
protos = ["resource.proto"],
pb_options = [
# omit the go_package declared in proto files to make bazel works as expect
"paths=source_relative",
],
)
33 changes: 33 additions & 0 deletions src/opencensus/proto/resource/v1/resource.proto
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
// Copyright 2018, OpenCensus Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

syntax = "proto3";

package opencensus.proto.resource.v1;

option go_package = "github.com/census-instrumentation/opencensus-proto/gen-go/resource/v1";

option java_multiple_files = true;
option java_package = "io.opencensus.proto.resource.v1";
option java_outer_classname = "ResourceProto";

// Resource information.
message Resource {

// Type identifier for the resource.
string type = 1;

// Set of labels that describe the resource.
map<string,string> labels = 2;
}