feat: implement batch insertion for drift analysis projects and add background job supervision

[fernandonogueira]

No description provided.

[Copilot]

Pull request overview

This PR improves drift analysis ingestion performance by switching drift analysis project inserts to a batch COPY FROM path, and makes long-running background jobs more robust by supervising them with a panic-safe supervisor loop. It also adds Docker-backed integration tests covering drift ingestion and org-scoping behavior used by the perms middleware.

Changes:

• Batch insert drift analysis projects via sqlc :copyfrom and use the new repository method from the drift ingest handler.
• Add a background-job supervisor with a panic counter metric; wire background loops to accept context.Context for graceful shutdown.
• Introduce Docker-based integration tests (migrations applied from migrations/*.sql) for drift ingest + perms org scoping.

Reviewed changes

Copilot reviewed 13 out of 17 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
test/integration/perms_test.go	Adds integration test ensuring org scoping returns only organizations a user belongs to.
test/integration/main_test.go	Adds dockertest-based Postgres TestMain, migration runner, and truncation helpers.
test/integration/drift_ingest_test.go	Adds integration tests for drift ingest happy path, invalid token, and idempotency race.
pkg/usecase/sync/user_resources/github/sync.go	Updates user sync loop to accept a cancellable context and stop promptly on shutdown.
pkg/usecase/sync/org/github/sync.go	Updates org sync loop to accept a cancellable context and stop on shutdown.
pkg/usecase/drift_stream/api.go	Pre-validates project types and switches per-project inserts to a batch insert call.
pkg/usecase/auth/github/oauth.go	Fixes transaction error handling by returning errors (allowing rollback) instead of writing HTTP responses inside the tx on failures.
pkg/repository/queries/drift_analysis.sql.go	Adds sqlc-generated batch params struct for drift analysis project copyfrom.
pkg/repository/queries/drift_analysis.sql	Adds CreateDriftAnalysisProjectsBatch :copyfrom query for batch inserting projects.
pkg/repository/queries/db.go	Extends sqlc DBTX interface with CopyFrom to support :copyfrom queries.
pkg/repository/queries/copyfrom.go	Adds sqlc-generated CopyFromSource iterator and CreateDriftAnalysisProjectsBatch method.
pkg/repository/drift_analysis_repository.go	Exposes batch insert method via DriftAnalysisRepository implementation.
pkg/observability/supervisor.go	Introduces supervisor loop to restart background jobs and emit a panic counter.
pkg/observability/metrics.go	Adds bg_job_panics_total metric to track supervised job panics by name.
main.go	Runs token refresher and sync loops under supervisor with shared cancellable context.
go.mod	Adds dockertest dependency (and new indirect deps) for integration tests.
go.sum	Updates checksums for new/updated dependencies.

Files not reviewed (3)

• pkg/repository/queries/copyfrom.go: Language not supported
• pkg/repository/queries/db.go: Language not supported
• pkg/repository/queries/drift_analysis.sql.go: Language not supported

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.