Inventory Collector

Inventory Collector

🇪🇸 Español

The collector is an independent Node.js Azure Functions v4 project in functions/. It collects Azure resources, builds the full JSON document consumed by Goliat and publishes it to cloud-inventory/azure/latest.json. It needs no PostgreSQL and does not import Next.js code.

How a generation works

The daily timer (02:00 UTC by default) and the authenticated HTTP endpoint store an operation and enqueue a message. The queue worker then:

1. Acquires a shared, renewable lease so only one generation publishes at a time.
2. Reads the configured subscriptions and queries Azure Resource Graph with pagination.
3. Completes data from ARM: AKS pools, VM state, networks and subnets, NSG rules, private endpoint connections, Automation runbooks and webhooks, Container Registry webhooks, and PostgreSQL/MySQL Flexible Server details.
4. Builds 69 source reports, totals, tag groupings and tag columns.
5. Validates contract, scope, counts, coverage, IDs and size, then publishes with an ETag condition.

A failed query aborts the generation and keeps the previous inventory: Goliat never publishes partial snapshots.

Each worker run is limited to 14 minutes. The queue retries up to five times before marking the operation as failed. Operation states are kept for 30 days.

What is not collected

The collector reads resource configuration only. It does not download blob contents, database data, Key Vault secrets, account keys or connection strings, and only the columns allowed by the contract are published.

Deploy only the collector

.\functions\deploy.ps1 -ValidateOnly
.\functions\deploy.ps1 -SubscriptionId $subscriptionId -GrantReader -RunAfterDeploy

Useful options:

# Several subscriptions of the same tenant and explicit tag keys
.\functions\deploy.ps1 -SubscriptionId $subscriptionId `
  -InventorySubscriptionIds @($subscriptionId, '<other-subscription>') `
  -InventoryTagKeys @('Environment', 'Owner') -GrantReader

# Disable the timer and run on demand
.\functions\deploy.ps1 -SubscriptionId $subscriptionId -DisableInventoryTimer

# More memory for large inventories
.\functions\deploy.ps1 -SubscriptionId $subscriptionId -InstanceMemoryMB 2048

To deploy the web and PostgreSQL as well, use .\goliat.ps1 deploy (see Deploying to Azure).

Authenticated HTTP API

Method and path	Purpose	Permission
POST /api/inventory/generate	Request a generation	Inventory.Generate
GET /api/inventory/operations/{operationId}	Operation status, ETag and count	Inventory.Generate
GET /api/inventory/operations/current	Latest operation or idle	Inventory.Generate
POST /api/inventory/publish	Validate and replace the whole document	Inventory.Publish

The handlers validate the RS256 signature, issuer, tenant, audience, expiry, client, token type, scope and role before reading the body. Without an API configured, HTTP returns 503 while the timer and worker keep working. The full deployment configures the API automatically and grants only Inventory.Generate to the web identity.

Operation states: queued, running, retrying, succeeded, failed.

Local development

Copy-Item .\functions\local.settings.example.json .\functions\local.settings.json
# Edit tenant, subscriptions and Storage endpoint; use AZURE_AUTH_MODE=default.
npm --prefix functions ci
npm --prefix functions run collect -- --output .\azure-inventory.local.json
npm --prefix functions run validate -- .\azure-inventory.local.json

collect uses your local Azure credentials and writes a local file; it does not publish. Tests need no Azure account:

npm --prefix functions run check
npm --prefix functions test
node --test tests/function-contract.test.mjs

Importing without the Refresh button

npm run collect:azure

Imports the current azure/latest.json directly into PostgreSQL, without requesting a new generation.

Full reference: functions/README.md and docs/azure-function.md.