Add up space billing get with GCS support

[branden]

Description of your changes

This adds an up subcommand space billing get for getting a billing report from an object storage API, with initial support for GCS. Support for AWS and Azure will be added in followup PRs.

branden@crateria up % ./_output/bin/darwin_arm64/up space billing get -h
Usage: up space billing get --provider=PROVIDER --bucket=STRING --account=STRING --billing-month=TIME --billing-custom=BILLING-CUSTOM

Get a billing report for submission to Upbound.

The storage location for the billing data used to create the report is supplied
using the optional --provider, --bucket, and --endpoint flags. If these flags
are missing, their values will be retrieved from the Spaces cluster from your
kubeconfig. Set --endpoint="" to use the storage provider's default endpoint
without checking your Spaces cluster for a custom endpoint.

Credentials and other storage provider configuration are supplied according to
the instructions for each provider below.

# AWS S3

Supply configuration by setting these environment variables:
AWS_REGION, AWS_ACCESS_KEY_ID, and AWS_SECRET_ACCESS_KEY.
For more options, see the documentation at
https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html.

# GCP Cloud Storage

Supply credentials by setting the environment variable
GOOGLE_APPLICATION_CREDENTIALS with the location of a
credential JSON file. For more options, see the documentation at
https://cloud.google.com/docs/authentication/application-default-credentials.

# Azure Blob Storage

Supply configuration by setting these environment variables:
AZURE_TENANT_ID, AZURE_CLIENT_ID, and AZURE_CLIENT_SECRET.
For more options, see the documentation at
https://learn.microsoft.com/en-us/azure/developer/go/azure-sdk-authentication.

Flags:
  -h, --help                Show context-sensitive help.
      --format="default"    Format for get/list commands. Can be: json, yaml,
                            default
  -v, --version             Print version and exit.
  -q, --quiet               Suppress all output.
      --pretty              Pretty print output.

  -o, --out="upbound_billing_report.tgz"
                            Name of the output file ($UP_BILLING_OUT).

Storage
  --provider=PROVIDER    Storage provider. Must be one of: aws, gcp, azure
                         ($UP_BILLING_PROVIDER).
  --bucket=STRING        Storage bucket ($UP_BILLING_BUCKET).
  --endpoint=STRING      Custom storage endpoint ($UP_BILLING_ENDPOINT).
  --account=STRING       Name of the Upbound account whose billing report is
                         being collected ($UP_BILLING_ACCOUNT).

Billing period
  --billing-month=TIME    Get a report for a billing period of one calendar
                          month. Format: 2006-01 ($UP_BILLING_MONTH).
  --billing-custom=BILLING-CUSTOM
                          Get a report for a custom billing period. Date
                          range is inclusive. Format: 2006-01-02/2006-01-02
                          ($UP_BILLING_CUSTOM).
  --force-incomplete      Get a report for an incomplete billing period
                          ($UP_BILLING_FORCE_INCOMPLETE).
branden@crateria up %

I have:

• Read and followed Upbound's contribution process.
• Run make reviewable to ensure this PR is ready for review.
• Added backport release-x.y labels to auto-backport this PR, as appropriate.

How has this code been tested

branden@crateria up % make build
<snip>
branden@crateria up % gcloud auth application-default login
<snip>
branden@crateria up % ./_output/bin/darwin_arm64/up space billing get --provider gcp --bucket mcp-usage-bucket-dev-00 --account upbound --billing-custom 2023-07-01/2023-07-01
Getting billing report for Upbound account upbound from 2023-07-01T00:00:00Z to 2023-07-02T00:00:00Z.

Reading usage data from storage...
Provider: gcp
Bucket: mcp-usage-bucket-dev-00

Billing report saved to /Users/branden/Projects/up/upbound_billing_report.tgz
branden@crateria up % tar -xzf upbound_billing_report.tgz
branden@crateria up % cat report/meta.json
{
  "account": "upbound",
  "time_range": {
    "start": "2023-07-01T00:00:00Z",
    "end": "2023-07-02T00:00:00Z"
  },
  "collected_at": "2023-07-25T19:46:04.840484-07:00"
}%
branden@crateria up % jq '. | length' < report/usage.json
1032
branden@crateria up % jq '.[0]' < report/usage.json
{
  "name": "max_resource_count_per_gvk_per_mcp",
  "tags": {
    "customresource_group": "ec2.aws.upbound.io",
    "customresource_version": "v1beta1",
    "customresource_kind": "RouteTable",
    "upbound_account": "upbound",
    "mcp_id": "b514c961-9c9c-4868-8e6d-7487341d019b"
  },
  "timestamp": "2023-07-01T00:00:00Z",
  "timestamp_end": "2023-07-01T01:00:00Z",
  "value": 2
}
branden@crateria up %

[darkmuggle]

Couple of suggestions, but nothing major. I do think that using an interface for the cloud providers will make life marginally easier, but not in the critical path for delivery.

[darkmuggle]

WDTH about use a CloudProvider interface instead? Since you're going to have to implement AWS and Azure as well the interface pattern could make things easier.

[branden]

Totally agreed, I have a related TODO here: https://github.com/upbound/up/pull/345/files#diff-827c313d3f294c17efea02ef7099b048dd5362f8de24a14cc9829fab195dd87cR46-R47

I'm not sure what that interface should look like yet, so I was planning on feeling it out when I add support for a second storage provider. I foresee this clientutil/gcs package eventually being used by the GCS implementation of that interface.

[AlainRoy]

Billing report saved to /Users/branden/Projects/up/upbound_billing_report.tgz

It may be nice to put the date in the tarball. For someone running this multiple times (e.g. three monthly reports for the quarter), it would be nice not to have to rename the file. This is a small suggestion -- it could happen in a future revision.

[AlainRoy]

I haven't seen help text done like this in the CLI -- this is great!

[AlainRoy]

Branden --

I read through the code. Overall, it's super clear and well tested. Thank you!

That said, I'm not entirely clear on what the output is. From the sample output above, it looks like we're getting a "max usage" for the time period specified by the user. Is that right?

I thought we were going for total usage. So if someone has an EKS cluster for 30 days, they'd have 720 loop-hours of usage. Is that what you're producing, or are you producing "one EKS cluster"?

[branden]

@AlainRoy The output is a series of events covering the billing period, with each event recording the maximum observed count of a GVK on an MCP within a given hour. This was what product requested, I'll follow up offline.

[AlainRoy]

The output is a series of events covering the billing period, with each event recording the maximum observed count of a GVK on an MCP within a given hour. This was what product requested, I'll follow up offline.

So in my example, we'd have 720 events, each saying "1 EKS cluster"?

[AlainRoy]

For the record, Branden and I talked and we're happy. We have an idea to add a summary of the time period in a separate file, but we can do that in the future -- no need to do it now.