Skip to content

Development

Jump to bottom
github-actions[bot] edited this page Feb 13, 2026 · 6 revisions

Development

Prerequisites

  • Go 1.25+
  • Docker and Docker Compose (for integration tests and local infrastructure)
  • golangci-lint (for linting)
  • pre-commit (optional, for git hooks)

Setup

git clone <repo-url>
cd storm-data-api
make docker-up   # Start Postgres + Kafka
make run         # Start server (runs migrations automatically)

Install pre-commit hooks (optional):

pre-commit install

Code Generation

GraphQL types and execution engine are generated by gqlgen:

make generate

This regenerates internal/graph/generated.go and internal/graph/models_gen.go from the schema in internal/graph/schema.graphqls. Resolver implementations in schema.resolvers.go are preserved.

Testing

Unit Tests

make test-unit

Runs unit tests with the race detector enabled (-race -count=1). Unit tests validate model deserialization and enum behavior. They do not require Docker or any external services.

Test What it verifies
TestLoadMockData Mock JSON deserializes into 271 StormReport structs
TestMockDataTypes 79 hail, 149 tornado, 43 wind reports
TestMockDataFields All required fields are populated (ID, coordinates, state, event_time, source_office)
TestMockDataOptionalFields Optional fields (severity, distance) are present on some but not all records
TestMockDataHailReport San Saba TX 1.25" hail report matches expected field values
TestSortFieldIsValid SortField enum validates known values and rejects invalid/lowercase strings
TestSortFieldString SortField.String() returns correct uppercase representation
TestSortOrderIsValid SortOrder enum validates ASC/DESC and rejects invalid values
TestSortOrderString SortOrder.String() returns correct uppercase representation

Coverage

make test-cover

Generates coverage.out and opens an HTML coverage report in the browser.

Integration Tests

Integration tests use testcontainers-go to spin up real PostgreSQL and Kafka containers. Docker must be running.

make test-integration
Test What it verifies
TestStoreInsertAndQuery Insert all 271 mock reports, then test: get by ID, list all, filter by type, filter by state, geo radius search, get non-existent returns nil
TestStoreAggregations CountByType (3 groups, max magnitude), CountByState (with county sub-groups, sum validation), CountByHour (bucket totals), LastUpdated, CountByType with type filter
TestStoreFilters Severity filter, multiple severities, counties, minMagnitude, combined filters (type + state + severity), empty result, multiple types
TestStoreSortingAndPagination Sort by magnitude DESC/ASC, sort by state, limit, offset with page comparison, offset beyond total
TestGraphQLEndpoint Full GraphQL query: list all (271 total), filter by type (79 hail)
TestGraphQLAggregations Full GraphQL response: totalCount, byType with counts, byState with county sub-groups, byHour bucket totals, lastUpdated, dataLagMinutes
TestKafkaConsumerIntegration Produce 271 mock messages to Kafka, consume them, insert to Postgres, verify all 271 are in the database

Container Images

Service Image
PostgreSQL postgres:16
Kafka confluentinc/confluent-local:7.6.0

The Kafka integration tests use the Confluent image (not apache/kafka) because the testcontainers Kafka module requires Confluent's startup scripts.

Test Data

Sample storm report JSON files live in data/mock/. These are used by unit tests to verify model deserialization against realistic data for all three event types (hail, tornado, wind).

Linting

make lint

Uses golangci-lint with the configuration in .golangci.yml. Enabled linters include:

  • bodyclose, errorlint, gosec, noctx, sqlclosecheck -- security and correctness
  • gocritic, revive -- style
  • misspell, unparam, errname, testifylint -- hygiene
  • prealloc, unconvert -- performance
  • exhaustive -- completeness
  • gocyclo -- complexity (threshold: 15)

Formatting

make fmt

Runs gofmt and goimports across the project.

Pre-commit Hooks

The .pre-commit-config.yaml configures hooks that run on every commit:

  • Trailing whitespace removal
  • End-of-file newline
  • YAML and JSON validation
  • Merge conflict markers
  • Large file detection
  • Private key detection (gitleaks)
  • gofmt and goimports
  • go vet and go build
  • golangci-lint
  • GraphQL schema linting
  • yamllint
  • markdownlint

CI Pipeline

The .github/workflows/ci.yml workflow runs on pushes and pull requests to main:

Job What It Does
test-unit make test-unit (unit tests with race detector)
lint make lint (golangci-lint with the project config)
build make build (compile check)
sonarcloud Unit tests with coverage + SonarCloud scan

A separate release.yml workflow (triggered by CI success on main) handles versioning, GitHub releases, and Docker image publishing.

Project Conventions

  • Schema-first GraphQL: The schema in schema.graphqls is the source of truth. Run make generate after schema changes.
  • Domain logic is pure: The model package has no infrastructure imports.
  • Concrete store dependency: Resolvers depend on *store.Store directly. The store is the single source of all data access logic.
  • Adapter constructor injection: All adapters (Kafka, HTTP, database) accept *slog.Logger via their constructors for consistent, testable logging.
  • Embedded migrations: SQL migrations in internal/database/migrations/ are embedded via //go:embed and run automatically on startup.

Related

Home | couchcryptid | System | Collector | ETL | Shared

Storm Data API

Related Wikis

Clone this wiki locally