Skip to content

feat(snapshot): add ProbeSource interface with FileProbeSource and CachedProbeSource#32

Open
davidbirdsong wants to merge 1 commit into
mainfrom
davidbirdsong/net-471-probe-snapshot-caching-for-atlasctl-and-pulumi-atlas
Open

feat(snapshot): add ProbeSource interface with FileProbeSource and CachedProbeSource#32
davidbirdsong wants to merge 1 commit into
mainfrom
davidbirdsong/net-471-probe-snapshot-caching-for-atlasctl-and-pulumi-atlas

Conversation

@davidbirdsong

Copy link
Copy Markdown
Collaborator

Add ProbeSource interface with FileProbeSource and CachedProbeSource

Problem

The Pulumi provider needs to run probe selection but has no access to a pre-fetched snapshot file and cannot call the RIPE Atlas API on every resource operation. The CLI has the same latent issue: snapshot.Load errors out if the file is missing rather than fetching transparently.

What changed

New: ProbeSource interface (pkg/snapshot/source.go)

type ProbeSource interface {
    Probes(ctx context.Context) ([]Probe, error)
}

Two implementations:

FileProbeSource reads from a fixed path with no freshness logic. Returns ErrNotFound if the file is absent. Used in CLI tests and by any caller that owns its own refresh lifecycle (e.g. after atlasctl refresh).

CachedProbeSource is a read-through cache backed by a file on disk.

  • Fast path: acquires a shared flock lock, checks mtime against TTL, reads and returns if fresh. No API call.
  • Slow path: releases the shared lock, acquires an exclusive lock, double-checks freshness (another writer may have landed while waiting), then fetches from the RIPE Atlas API and writes atomically to disk.
  • ProbeRefreshDisabled bool prevents any API call. Returns ErrRefreshDisabled when the cache is absent or stale. Intended for CI environments that supply the snapshot as a separate step.
  • TTL and Path are configurable. Defaults: /tmp/atlasctl-probes.json, 2 hours.
  • Depends on github.com/gofrs/flock for cross-process and cross-goroutine lock coordination.

Updated: FakeProbeSource (pkg/snapshot/fake.go)

FakeProbeSource implements ProbeSource for tests, matching the existing FakeClient pattern.

Updated: CLI wire-up

Deps.NewProbeSource factory added to deps.go. Production default is CachedProbeSource. Tests use FileProbeSource so snapshot files written by saveSnap are read directly without TTL or API-call logic.

  • plan and apply: resolve API key first, construct CachedProbeSource with a snapshot client, reuse the key for the measurement client.
  • select: treats the API key as optional (resolveAPIKeyOptional). A fresh cache requires no credentials; a stale or missing cache auto-refreshes if a key is available, otherwise errors clearly.
  • selectAll signature changed from snapshot.Snapshot to []snapshot.Probe since only the probe list was ever used.

Tests

pkg/snapshot/source_test.go covers:

Test What it checks
FileProbeSource_ReturnsProbes reads probes from a saved snapshot
FileProbeSource_MissingFile returns ErrNotFound
FakeProbeSource_ReturnsProbes / _Error fake behaves correctly
CachedProbeSource_FreshCache_NoFetch client is never called when cache is within TTL
CachedProbeSource_MissingCache_FetchesAndWrites absent cache triggers fetch and writes to disk
CachedProbeSource_StaleCache_FetchesAndWrites stale cache (1ns TTL) triggers fetch
CachedProbeSource_RefreshDisabled_MissingCache returns ErrRefreshDisabled
CachedProbeSource_RefreshDisabled_StaleCache returns ErrRefreshDisabled
CachedProbeSource_RefreshDisabled_FreshCache fresh cache is served even when refresh is disabled
CachedProbeSource_NilClient_StaleCache nil client with stale cache returns a clear error
CachedProbeSource_ConcurrentCallers 8 goroutines racing on a missing cache: FetchProbes called exactly once

cmd/atlasctl/cli_test.go adds compile-time interface assertions for all three implementations.

Dependency

github.com/gofrs/flock v0.13.0 added to go.mod.

…chedProbeSource

Defines a ProbeSource abstraction over probe data so the Pulumi provider can obtain the full probe list without assuming a pre-fetched file exists.

FileProbeSource reads from a fixed path with no freshness logic. Used by CLI tests and any caller that manages its own refresh lifecycle.

CachedProbeSource is a read-through cache backed by a file on disk. It serves probes from cache when fresh (shared flock), and fetches from the RIPE Atlas API when stale or absent (exclusive flock with double-check to coalesce concurrent writers). TTL, cache path, and a ProbeRefreshDisabled flag for CI are all configurable. Depends on github.com/gofrs/flock.

CLI commands (select, plan, apply) are wired to use CachedProbeSource via Deps.NewProbeSource. The select command treats the API key as optional so a fresh cache requires no credentials. selectAll is updated to accept []snapshot.Probe directly instead of snapshot.Snapshot.
@davidbirdsong

Copy link
Copy Markdown
Collaborator Author

This change is part of the following stack:

Change managed by git-spice.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant