Integration BigQuery Extension

The TestTrackingDiagrams.Extensions.BigQuery package adds Google BigQuery REST API operation tracking to your test diagrams. Instead of showing raw HTTP requests like POST /bigquery/v2/projects/my-project/queries, your sequence diagrams show classified operations like Query: /query or Insert: /mydataset/table/mytable.

---

How It Works

The Google BigQuery .NET SDK (Google.Cloud.BigQuery.V2) translates all operations into REST HTTP requests. BigQueryTrackingMessageHandler is a DelegatingHandler that intercepts these HTTP requests, classifies each one into a BigQuery operation (Query, Insert, Read, List, Create, Delete, Update, Cancel) using the HTTP method and URL path, then logs it to RequestResponseLogger with a human-readable label.

Because it logs to the same RequestResponseLogger as the standard TestTrackingMessageHandler, BigQuery operations appear alongside your HTTP API calls in the same sequence diagram — showing the complete flow from test → API → BigQuery.

---

Install

dotnet add package TestTrackingDiagrams.Extensions.BigQuery

---

Verbosity Levels

The extension supports three verbosity levels that control how much detail appears in the diagrams:

Level	Method shown	URI shown	Headers	Request body	Response body
Raw	HTTP method (POST, GET, etc.)	Full BigQuery REST API URI	All except excluded set	Full JSON	Full JSON
Detailed	Classified operation (Query)	Clean path with dataset/resource context (/mydataset/table/mytable)	Filtered (noisy Google API headers excluded)	Full JSON	Full JSON
Summarised	Classified operation (Query)	Resource type and name only (/table/mytable)	None	None	None

The default is Detailed.

Diagram Label Examples

Operation	Raw	Detailed	Summarised
Run a query	POST: /bigquery/v2/projects/proj/queries	Query: /query	Query: /query
Get query results	GET: /bigquery/v2/projects/proj/queries/job123	Query: /query/job123	Query: /query/job123
Streaming insert	POST: /bigquery/v2/projects/proj/datasets/ds/tables/tbl/insertAll	Insert: /ds/table/tbl	Insert: /table/tbl
Read a table	GET: /bigquery/v2/projects/proj/datasets/ds/tables/tbl	Read: /ds/table/tbl	Read: /table/tbl
List tables	GET: /bigquery/v2/projects/proj/datasets/ds/tables	List: /ds/table	List: /table
Create a dataset	POST: /bigquery/v2/projects/proj/datasets	Create: /dataset	Create: /dataset
Delete a table	DELETE: /bigquery/v2/projects/proj/datasets/ds/tables/tbl	Delete: /ds/table/tbl	Delete: /table/tbl
Create a job (load)	POST: /bigquery/v2/projects/proj/jobs	Create: /job	Create: /job
Cancel a job	POST: /bigquery/v2/projects/proj/jobs/job456/cancel	Cancel: /job/job456	Cancel: /job/job456
SDK metadata	GET: /discovery/v1/apis	Other: ...	(skipped)

---

Classified Operations

The classifier recognises these BigQuery operations from the SDK's HTTP traffic:

Operation	HTTP Pattern
Query	POST /projects/{proj}/queries or GET /projects/{proj}/queries/{jobId}
Insert	POST /datasets/{ds}/tables/{tbl}/insertAll
Read	GET on a specific resource (table, dataset, job, model, routine)
List	GET on a collection (/tables, /datasets, /jobs, /models, /routines, /tables/{tbl}/data)
Create	POST on a collection (/datasets, /tables, /jobs)
Delete	DELETE on a specific resource, or POST /jobs/{id}/delete
Update	PUT or PATCH on a specific resource
Cancel	POST /jobs/{id}/cancel
Other	Unrecognised paths (e.g. discovery API, service account metadata)

In Summarised mode, Other operations are silently skipped.

Supported Resource Types

The classifier extracts and tracks these resource types:

Resource Type	URL Path Segment
query	/queries
dataset	/datasets/{id}
table	/datasets/{ds}/tables/{id}
tabledata	/datasets/{ds}/tables/{id}/data
job	/jobs/{id}
model	/datasets/{ds}/models/{id}
routine	/datasets/{ds}/routines/{id}

The upload endpoint variant (/upload/bigquery/v2/projects/...) is also recognised for data loading jobs.

---

Setup

Option A: With BigQueryClientBuilder (recommended)

Use the WithTestTracking() extension method on BigQueryClientBuilder:

var trackingOptions = new BigQueryTrackingMessageHandlerOptions
{
    ServiceName = "BigQuery",
    CallingServiceName = "My API",
    Verbosity = BigQueryTrackingVerbosity.Detailed,
    CurrentTestInfoFetcher = () =>
    {
        var test = TestContext.Current.Test;
        return test is not null
            ? (test.TestDisplayName, test.UniqueID)
            : ("Unknown", "unknown");
    }
};

var client = new BigQueryClientBuilder
{
    ProjectId = "my-project",
    // Credential = ... (for real BigQuery)
}
.WithTestTracking(trackingOptions)
.Build();

WithTestTracking sets the builder's HttpClientFactory to a TrackingBigQueryHttpClientFactory that wraps the default HTTP handler with the tracking handler. The pipeline becomes:

BigQueryClient → ConfigurableMessageHandler (auth, retry) → BigQueryTrackingMessageHandler → HttpClientHandler

Option B: In ASP.NET Core Integration Tests

For WebApplicationFactory-based integration tests where BigQuery clients are registered in DI:

builder.ConfigureTestServices(services =>
{
    // Remove existing BigQueryClient registration
    var descriptor = services.SingleOrDefault(d => d.ServiceType == typeof(BigQueryClient));
    if (descriptor != null) services.Remove(descriptor);

    // Register tracked client
    services.AddSingleton(sp =>
    {
        var trackingOptions = new BigQueryTrackingMessageHandlerOptions
        {
            ServiceName = "BigQuery",
            CallingServiceName = "My API",
            Verbosity = BigQueryTrackingVerbosity.Detailed,
            CurrentTestInfoFetcher = () =>
            {
                var test = TestContext.Current.Test;
                return test is not null
                    ? (test.TestDisplayName, test.UniqueID)
                    : ("Unknown", "unknown");
            }
        };

        return new BigQueryClientBuilder
        {
            ProjectId = "test-project",
        }
        .WithTestTracking(trackingOptions)
        .Build();
    });
});

Option C: Manual Handler Wiring

If you need full control over the HTTP pipeline, use BigQueryTrackingMessageHandler directly:

var trackingOptions = new BigQueryTrackingMessageHandlerOptions
{
    ServiceName = "BigQuery",
    CallingServiceName = "My API",
    Verbosity = BigQueryTrackingVerbosity.Detailed,
    CurrentTestInfoFetcher = () => ("My Test", testId)
};

var handler = new BigQueryTrackingMessageHandler(trackingOptions, new HttpClientHandler());
// Use handler with your own HTTP pipeline setup

---

Configuration Reference

BigQueryTrackingMessageHandlerOptions

Property	Type	Default	Description
ServiceName	string	"BigQuery"	The participant name shown in the diagram for the BigQuery service
CallingServiceName	string	"Caller"	The participant name shown for the service making BigQuery calls
Verbosity	BigQueryTrackingVerbosity	Detailed	Controls how much detail appears in the diagram (Raw, Detailed, Summarised)
CurrentTestInfoFetcher	Func<(string Name, string Id)>?	null	Returns the current test's name and ID. Required — if null, requests are forwarded but not logged
CurrentStepTypeFetcher	Func<string?>?	null	Optional — returns the current BDD step type (Given/When/Then)
ExcludedHeaders	HashSet<string>	See below	Headers to exclude from diagrams in Raw/Detailed mode

Default Excluded Headers

The following Google API headers are excluded by default (they add noise without diagnostic value):

• Authorization
• User-Agent
• x-goog-api-client
• x-goog-request-params
• google-cloud-resource-prefix
• Cache-Control

Override ExcludedHeaders to customise:

new BigQueryTrackingMessageHandlerOptions
{
    ExcludedHeaders = ["Authorization"] // Only exclude Authorization
}

---

Bridging Test Info From HTTP Tracking

If you already use TestTrackingMessageHandlerOptions for HTTP tracking and want to share the same test info fetchers with BigQuery tracking:

var httpOptions = new TestTrackingMessageHandlerOptions
{
    CurrentTestInfoFetcher = () => (testName, testId),
    CurrentStepTypeFetcher = () => currentStepType,
    CallingServiceName = "My API"
};

var bqOptions = new BigQueryTrackingMessageHandlerOptions
{
    ServiceName = "BigQuery",
    CallingServiceName = httpOptions.CallingServiceName,
    CurrentTestInfoFetcher = httpOptions.CurrentTestInfoFetcher,
    CurrentStepTypeFetcher = httpOptions.CurrentStepTypeFetcher,
    Verbosity = BigQueryTrackingVerbosity.Detailed
};

---

Framework Compatibility

The extension works with all TTD framework adapters:

Framework	Test Info Fetcher Pattern
xUnit3	TestContext.Current.Test → (TestDisplayName, UniqueID)
xUnit2	ITestOutputHelper + ambient storage
NUnit4	NUnit.Framework.TestContext.CurrentContext
MSTest	MSTest.TestContext
TUnit	TUnit.Core.TestContext.Current
LightBDD	LightBddTestTrackingMessageHandlerOptions bridge
ReqNRoll	ReqNRollTestTrackingMessageHandlerOptions bridge
BDDfy	BDDfyTestTrackingMessageHandlerOptions bridge

---

Invocation Validation

BigQueryTrackingMessageHandler implements ITrackingComponent and auto-registers with TrackingComponentRegistry on construction. Call TrackingComponentRegistry.ValidateAllComponentsWereInvoked() in test teardown to catch cases where the handler was registered but never processed any BigQuery requests.

See Diagnostics and Debugging for full details on the TrackingComponentRegistry API.