Cost Reasoning

Module: Cost Reasoning

Module flag: cost

Enabled by default: no — explicit billing-export configuration is required

Overview

Cost Reasoning answers more than “what did this project cost?” The read-only gcp_cost_explain tool compares two equal, complete calendar-day windows and explains why regular net usage cost changed. One Aura process can route different configured environments to different detailed billing-export datasets.

One response includes:

• current and baseline gross cost, credits, net cost, delta, and percentage change;
• daily historical cost;
• top spenders and top increases by service, SKU, and resource;
• ranked cost drivers that reconcile to the project-level delta;
• newly billed resources and resources confirmed as newly created;
• active idle-resource recommendations and estimated monthly savings;
• traffic-like usage increases, with Cloud Monitoring corroboration where supported;
• data freshness, resource attribution, bytes processed, evidence confidence, and coverage gaps.

The module is opt-in because it runs chargeable BigQuery queries over your Cloud Billing export. It never modifies billing configuration or GCP resources.

Required Billing Export

Enable the Cloud Billing Detailed usage cost export. The standard export is not sufficient because it lacks the resource-level detail needed for new-resource, idle-resource, and traffic reasoning.

The expected table name starts with:

gcp_billing_export_resource_v1_

See Google Cloud's Detailed usage cost export schema and export setup guide.

Enable This Module

Minimal same-project configuration:

GCP_PROJECT_ID=workload-project \
COST_REASONING_ENABLED=true \
BILLING_EXPORT_DATASET=cloud_billing \
./aura-tracker-gcp --modules cost

For a central FinOps project that owns both the query jobs and export dataset:

GCP_PROJECT_ID=workload-project \
COST_REASONING_ENABLED=true \
COST_QUERY_PROJECT_ID=finops-project \
BILLING_EXPORT_PROJECT_ID=finops-project \
BILLING_EXPORT_DATASET=cloud_billing \
COST_REASONING_TIMEZONE=UTC \
COST_QUERY_MAX_BYTES=5368709120 \
./aura-tracker-gcp --modules cost

BILLING_EXPORT_TABLE is optional. When omitted, the first analysis discovers the dataset's sole table matching gcp_billing_export_resource_v1_*. If several detailed export tables exist, configure the exact table.

Multiple environments and billing sources

Environment selection works the same way for cost reasoning as it does for logs, Monitoring, Recommender, and inventory:

• when a request omits project_id, Aura uses the configured default environment;
• an alias or configured project ID selects that environment explicitly;
• the selected workload project's mapped billing source is used for table discovery and queries;
• the query always filters billing rows with the selected workload project ID; and
• results and errors use the configured alias instead of exposing that workload project ID; and
• distinct BigQuery query/export project IDs are replaced with [COST_PROJECT_ID] before MCP output.

Map environments to separate billing exports in ~/.aura-tracker.yaml:

environments:
  - project_id: my-app-dev
    alias: dev
    default: true
  - project_id: my-app-preprod
    alias: preprod

cost_reasoning:
  enabled: true
  timezone: Asia/Makassar
  history_days: 90
  max_bytes_billed: 5368709120
  sources:
    - environments: [dev]
      query_project_id: finops-dev
      export_project_id: billing-dev
      dataset: cloud_billing
    - environments: [preprod]
      query_project_id: finops-preprod
      export_project_id: billing-preprod
      dataset: cloud_billing

When one central dataset contains billing rows for several projects, map them to one shared source:

cost_reasoning:
  enabled: true
  timezone: UTC
  sources:
    - environments: [dev, preprod]
      query_project_id: central-finops
      export_project_id: central-finops
      dataset: all_billing_data

Container deployments can provide the same source objects as strict JSON:

COST_REASONING_SOURCES_JSON='[{"environments":["dev"],"query_project_id":"finops-dev","export_project_id":"billing-dev","dataset":"cloud_billing"},{"environments":["preprod"],"query_project_id":"finops-preprod","export_project_id":"billing-preprod","dataset":"cloud_billing"}]'

When the cost module is active, every configured environment must be assigned exactly once. Startup then rejects unknown selectors, duplicate or missing mappings, empty source environment lists, invalid identifiers, and missing query projects or datasets. Strict configuration loading rejects unknown YAML/JSON fields regardless of the selected modules. export_project_id defaults to its source's query_project_id; table remains optional and is discovered only inside that source.

The legacy scalar YAML fields and COST_QUERY_PROJECT_ID, BILLING_EXPORT_PROJECT_ID, BILLING_EXPORT_DATASET, and BILLING_EXPORT_TABLE remain supported. A legacy source is shared by every configured environment, preserving central-export installations. Do not combine legacy source settings with YAML sources or COST_REASONING_SOURCES_JSON; Aura rejects ambiguous startup configuration rather than merging it. Global enabled, timezone, history, and byte-limit environment variables can still override YAML.

Aura never falls back to the default environment's billing source when another environment has no mapping. Startup coverage validation normally prevents this; the runtime guard returns a normal cost reasoning is not configured for environment ... tool error, and the MCP privacy layer masks an aliased workload project before returning it.

Cost reasoning is registered only when both conditions are true:

1. COST_REASONING_ENABLED=true (or YAML cost_reasoning.enabled: true); and
2. the cost module is included by --modules, or the flag is omitted.

When cost reasoning is enabled in configuration but cost is excluded by --modules, Aura skips cost-source compilation and cost-specific BigQuery client creation. General YAML/JSON parsing and environment-variable syntax remain strict.

Automatic IAM Setup

Run the idempotent setup script as a team administrator:

PROJECT_ID=workload-project \
COST_REASONING_ENABLED=true \
COST_QUERY_PROJECT_ID=finops-project \
BILLING_EXPORT_PROJECT_ID=finops-project \
BILLING_EXPORT_DATASET=cloud_billing \
RECOMMENDER_ENABLED=true \
bash scripts/setup-iam.sh

The script:

• enables BigQuery in the query project;
• enables Cloud Asset Inventory in the workload project;
• grants roles/bigquery.jobUser on the query project;
• grants roles/bigquery.dataViewer on the export project;
• grants roles/cloudasset.viewer and roles/serviceusage.serviceUsageConsumer on the workload project;
• enables Recommender and grants roles/recommender.viewer when RECOMMENDER_ENABLED=true.

The script uses a project-wide BigQuery Data Viewer grant for idempotent setup. In production, prefer granting that role only on the billing-export dataset and remove the broader project binding.

The current script interface configures one workload/query/export combination for the service account it creates in PROJECT_ID. For a multi-environment installation, use that one Aura service account and apply its additional workload, query-project, and dataset grants manually. Running the script with a different PROJECT_ID would create or select a different service account; Aura does not switch runtime credentials per environment.

The setup script enables BigQuery and Cloud Asset Inventory. If Cloud Monitoring is not already enabled, enable monitoring.googleapis.com in the workload project. The optional RECOMMENDER_ENABLED=true setup flag enables Recommender.

Required APIs and IAM

Scope	API / role	Used for
Every unique query project	bigquery.googleapis.com	Dry runs and bounded query jobs
Every unique query project	roles/bigquery.jobUser	Create query jobs
Every configured export dataset	roles/bigquery.dataViewer	Discover and read the detailed export
Every workload project	cloudasset.googleapis.com	Confirm creation time for newly billed resources
Every workload project	monitoring.googleapis.com	Corroborate supported traffic changes
Every workload project	recommender.googleapis.com	Optional idle-resource findings
Every workload project	roles/cloudasset.viewer	Search resource inventory
Every workload project	roles/serviceusage.serviceUsageConsumer	Consume enabled APIs under the runtime identity
Every workload project	roles/monitoring.viewer	Corroborate supported traffic changes
Every workload project	roles/recommender.viewer	Optional idle-resource findings

If RECOMMENDER_ENABLED=false, core billing analysis still works. The response marks idle-resource coverage as skipped.

Tool

gcp_cost_explain

Parameter	Required	Default	Description
project_id	no	default environment	Environment alias or configured project ID for the billed workload
period	no	last_7_complete_days	last_7_complete_days, last_30_complete_days, month_to_date, or custom
comparison	no	previous_period	Version 1 supports the immediately preceding equal-length period
start_date	custom only	—	Inclusive YYYY-MM-DD start date
end_date	custom only	—	Inclusive YYYY-MM-DD end date; must be before today
timezone	no	configured timezone or UTC	IANA timezone used for complete-day boundaries
detail_level	no	standard	summary, standard, or detailed
max_results	no	10	Maximum ranked findings per dimension/section, 1–25
include_idle	no	true	Query active idle-resource recommendations
include_traffic	no	true	Detect traffic-like billing increases and corroborate supported resources

Custom periods are limited to 366 complete days. The current local day is always excluded because billing data for it is incomplete.

Response Guide

Field	Meaning
status	complete, partial, no_material_change, or no_data
scope	Current, baseline, history, timezone, comparison, and cost basis
totals	Gross cost, credits, regular net usage cost, delta, and percentage change
history	Daily cost series; missing dates are normalized to zero and must be interpreted with freshness warnings
drivers	Ranked, evidence-backed explanations whose deltas reconcile to the total
top_spenders	Highest current spend by service, SKU, and resource
top_increases	Largest positive deltas by service, SKU, and resource
new_resources	newly_billed or confirmed_new resources
idle_resources	Active GCP idle recommendations, current-period cost match, and projected savings
traffic_anomalies	Traffic-like SKU usage increases and optional Monitoring comparison
coverage	Collector status, freshness, export history, attribution, bytes processed, and cache hit
warnings	Missing access, stale data, truncation, mixed currencies, or low attribution

percent_change_defined is false when the baseline is zero. The summary reports the absolute increase without inventing a percentage.

How Reasoning Works

Historical comparison

The engine uses complete local calendar days and compares them with the immediately preceding equal-length window. This avoids comparing seven complete days with a partially billed current day and handles daylight-saving boundaries using calendar arithmetic.

Ranked drivers

The engine starts with resource-level deltas, retains service-level unattributed residuals, and adds a project residual when needed. Lower-ranked findings are rolled into other measured changes so the returned driver deltas still reconcile to the project delta after output limits are applied.

Driver categories include:

• confirmed_new and newly_billed;
• traffic_spike;
• sku_mix_shift;
• usage_growth;
• price_or_discount_change;
• commitment_or_credit_change;
• unattributed.

Usage and effective-rate effects are decomposed deterministically from the dominant SKU's current and baseline usage and net cost.

New resources

A resource first billed during the current period is classified as newly_billed. If Cloud Asset Inventory reports a matching createTime in the same window, it becomes confirmed_new with high confidence.

“Newly billed” means first observed in the configured export-history window. It does not prove that the resource itself was created then.

Idle resources

The collector reads active idle recommendations for Cloud SQL instances, Compute Engine VMs, disks, IP addresses, custom images and reservations, plus idle GKE cluster diagnoses. Per-recommender failures produce partial coverage instead of discarding successful findings.

Successful recommendation sets are cached for 12 hours and checked before the shared per-recommender quota gate. Only complete sets enter that cache. Quota exhaustion returns partial coverage with the quota window and RFC3339 retry time while billing, asset, and Monitoring reasoning continues. The outer 15-minute cost-response cache is shortened to the retry time so a degraded response is refreshed as soon as the blocked recommender can resume.

Unexpected traffic

Traffic candidates require a material cost increase and at least a 1.5× increase in a traffic-like billing usage unit such as requests, network bytes, packets, or operations. Billing evidence starts at medium confidence. Supported Cloud Run resources are promoted to high confidence only when run.googleapis.com/request_count shows the same increase across the exact comparison windows.

Query and Cost Safeguards

• Exactly two fixed, parameterized BigQuery statements run for an uncached analysis.
• Source selection is a direct lookup by the resolved workload project; it never searches or falls back to another source.
• Table discovery is limited to the selected source's export project and dataset.
• Every statement is dry-run first.
• A statement is rejected when its estimate exceeds COST_QUERY_MAX_BYTES.
• Execution uses the same BigQuery maximumBytesBilled ceiling.
• Partition pruning supports ingestion-time and field-partitioned tables.
• BigQuery aggregates and ranks rows before results reach Aura Tracker.
• First-seen resource rows and response sections are bounded.
• Complete responses are cached in-process for 15 minutes. A quota-degraded response is cached only until its earlier Recommender retry time.
• The full reasoning run has a 45-second budget.
• The default ceiling is 5 GiB per statement.

Use the coverage.bytes_processed value to understand the most recent uncached analysis. See Managing GCP Costs for operational cost controls.

Coverage and Limitations

• Detailed-export resource identifiers vary by service. GKE resource detail requires GKE cost allocation.
• Billing export delivery can be delayed; data_through and freshness_hours make this visible.
• Tax and adjustment rows are excluded. The cost basis is regular net usage cost: cost + credits.
• Currency conversion is not attempted. Multiple currencies make the result partial and are labeled MULTIPLE.
• Cloud Asset Inventory does not expose searchable creation time for every asset type.
• Traffic classification is heuristic; high confidence currently requires comparative Cloud Run Monitoring evidence.
• Idle findings depend on GCP Recommender availability and its own observation windows.
• The tool explains historical changes; it does not forecast a future invoice.

Prompt Examples

“Why did costs increase in the last seven complete days? Show the top offenders and evidence.”

“Compare the last 30 complete days with the previous 30 and separate usage growth from rate or credit changes.”

“Which resources appeared for the first time this month, and which are confirmed as newly created?”

“Find idle resources and unexpected traffic behind this month's increase.”

Related Modules

Module flag	Why you'd combine it
monitoring	Inspect the raw operational metrics behind a traffic finding
aura	Compare historical cost explanations with current efficiency scores
cloudrun	Inspect configuration and revisions for a costly Cloud Run resource
cloudsql	Inspect instances returned by idle recommendations
tagging	Audit allocation labels on high-cost or unattributed resources
iam	Verify the workload-project permissions visible to Aura Tracker