monitoring_v3.query.Query.align raises a TypeError on execution

[plankthom]

Environment details

• OS type and version: Macos 11.1
• Python version: Python 3.8.5 (anaconda)
• pip version: pip 21.0
• google-cloud-monitoring version: 2.0.0

Steps to reproduce

While I can execute the query

from google.cloud.monitoring_v3.query import Query
query = Query(
     client=client,
     project=project_name,
     metric_type= 'storage.googleapis.com/storage/request_count'
).select_resources(
     bucket_name_prefix=bucket_name_prefix
).select_interval(
     end_time, start_time=end_time - report_interval
)
print(next(iter(query)))

adding an alignment aggregation fails:

from google.cloud.monitoring_v3 import Aggregation
query = query.align(
    Aggregation.Aligner.ALIGN_MAX, seconds=3600
)
print(next(iter(query)))

and raises

        params = self._build_query_params(headers_only, page_size)
>       for ts in self._client.list_time_series(**params):
E       TypeError: list_time_series() got an unexpected keyword argument 'aggregation'

.../site-packages/google/cloud/monitoring_v3/query.py:446: TypeError

Indeed, while

python-monitoring/google/cloud/monitoring_v3/query.py

Line 477 in 8a7e2f7

params["aggregation"] = types.Aggregation(

prepares an aggregation parameter,

python-monitoring/google/cloud/monitoring_v3/services/metric_service/client.py

Line 874 in 8a7e2f7

def list_time_series(

accepts no such parameter.

Workaround

Directly use MetricServiceClient.list_time_series, with the aggregation specification within the request parameter:

request = dict(
      name=project_name,
      filter=f'metric.type = "storage.googleapis.com/storage/request_count" AND resource.label.bucket_name = starts_with("{bucket_name_prefix}")',
      interval=TimeInterval(dict(
          end_time=_to_seconds_nanos(end_time),
          start_time=_to_seconds_nanos(end_time - report_interval)
      )),
      view=ListTimeSeriesRequest.TimeSeriesView.FULL,
      aggregation=Aggregation(dict(
          per_series_aligner=Aggregation.Aligner.ALIGN_SUM,
          alignment_period=dict(seconds=3600)
      ))
  )
print(next(iter(client.list_time_series(request=request))))

[busunkim96]

Hi @plankthom,

As you've noticed, not all the request fields can be passed to the method. The list of parameters can be thought of as something like an overload. The specific set of parameters is determined by an annotation in the source protos called google.api.method_signature. See https://google.aip.dev/client-libraries/4232#method-signatures for details.

This is why name, filter, interval, and view are method parameters but aggregation is not.
https://github.com/googleapis/googleapis/blob/eabe7c0fde64b1451df6ea171b2009238b0df07c/google/monitoring/v3/metric_service.proto#L120-L125

  rpc ListTimeSeries(ListTimeSeriesRequest) returns (ListTimeSeriesResponse) {
    option (google.api.http) = {
      get: "/v3/{name=projects/*}/timeSeries"
    };
    option (google.api.method_signature) = "name,filter,interval,view";
  }

python-monitoring/google/cloud/monitoring_v3/services/metric_service/client.py

Lines 874 to 885 in 8a7e2f7

	def list_time_series(
	self,
	request: metric_service.ListTimeSeriesRequest = None,
	*,
	name: str = None,
	filter: str = None,
	interval: common.TimeInterval = None,
	view: metric_service.ListTimeSeriesRequest.TimeSeriesView = None,
	retry: retries.Retry = gapic_v1.method.DEFAULT,
	timeout: float = None,
	metadata: Sequence[Tuple[str, str]] = (),
	) -> pagers.ListTimeSeriesPager:

[plankthom]

@busunkim96

As you've noticed, not all the request fields can be passed to the method

Indeed i've noticed. The actual bug that I am reporting is that the SDK itself doesn't respect this limitation in

python-monitoring/google/cloud/monitoring_v3/query.py

Line 477 in 8a7e2f7

params["aggregation"] = types.Aggregation(

which breaks the feature documented in https://googleapis.dev/python/monitoring/latest/query.html as align method.

[busunkim96]

Ah sorry about that, I wasn't reading carefully enough. 🤦‍♀️

It looks like iter should construct a request object with the params and then pass the request object to list_time_series.

python-monitoring/google/cloud/monitoring_v3/query.py

Lines 442 to 447 in 8a7e2f7

	if self._end_time is None:
	raise ValueError("Query time interval not specified.")
	params = self._build_query_params(headers_only, page_size)
	for ts in self._client.list_time_series(**params):
	yield ts

Thank you for the report!