Skip to content

Vault-support #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
boblangley merged 4 commits into from
Feb 8, 2026
Merged

Vault-support #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
67ce24f
Add HashiCorp Vault as pluggable secrets backend
boblangley Feb 8, 2026
13b8f90
Reorganize documentation into docs/ folder
boblangley Feb 8, 2026
0812a2f
feat: Add devcontainer
boblangley Feb 8, 2026
cb6ee24
Fix review findings from Copilot analysis
boblangley Feb 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 76 additions & 0 deletions .devcontainer/devcontainer.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
{
"name": "GitHub App Auth Container Devcontainer",
"image": "mcr.microsoft.com/devcontainers/base:noble",
"features": {
"ghcr.io/devcontainers/features/go:1": {},
"ghcr.io/devcontainers/features/docker-outside-of-docker:1": {}
},
"mounts": [
{
"source": "/var/run/docker.sock",
"target": "/var/run/docker.sock",
"type": "bind"
},
{
"source": "claude",
"target": "/home/vscode/.claude",
"type": "volume"
}
],
"customizations": {
"vscode": {
"extensions": [
// Docker and containers
"ms-azuretools.vscode-docker",

// Data and formats
"redhat.vscode-yaml",

// Markdown and documentation
"yzhang.markdown-all-in-one",
"bierner.markdown-mermaid",

// Protocol Buffers (gRPC)
"zxh404.vscode-proto3",

// Go development
"golang.go",

// General productivity
"Gruntfuggly.todo-tree",
"eamodio.gitlens",
"github.vscode-github-actions",
"GitHub.vscode-pull-request-github",

// AI assistants
"anthropic.claude-code",
"google.geminicodeassist",
"google.gemini-cli-vscode-ide-companion"
],
"settings": {
"go.useLanguageServer": true,
"go.lintTool": "golangci-lint",
"go.lintOnSave": "package",
"[go]": {
"editor.formatOnSave": true,
"editor.codeActionsOnSave": {
"source.organizeImports": "explicit"
}
}
}
}
},

"remoteEnv": {
"PATH": "${containerEnv:PATH}:/usr/local/go/bin:/go/bin:/home/vscode/.bun/bin"
},

"forwardPorts": [],
"portsAttributes": {
},

"postCreateCommand": {
"golangci-lint": "go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@latest",
"claude-code": "curl -fsSL https://claude.ai/install.sh | bash"
}
}
16 changes: 15 additions & 1 deletion .env.example
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ TEST_GITHUB_PRIVATE_KEY_B64=LS0tLS1CRUdJTi...base64...
# =============================================================================
# Test Agent Identity
# =============================================================================
# Agent name used for authentication (must match the Bitwarden item name)
# Agent name used for authentication (must match the item name in your secrets backend)
TEST_AGENT_NAME=test-agent

# Agent email for commit attribution
Expand All @@ -41,6 +41,20 @@ VAULTWARDEN_ADMIN_TOKEN=your-random-admin-token-here
TEST_VAULT_EMAIL=test@localhost
TEST_VAULT_PASSWORD=test-password-for-local-vault

# =============================================================================
# Secrets Backend Selection (for production/runtime)
# =============================================================================
# Choose secrets backend: "bitwarden" (default) or "vault"
# SECRETS_BACKEND=bitwarden

# =============================================================================
# HashiCorp Vault Settings (when SECRETS_BACKEND=vault)
# =============================================================================
# VAULT_ADDR=http://localhost:8200
# VAULT_TOKEN=your-vault-token
# VAULT_MOUNT_PATH=secret
# VAULT_BASE_PATH=agents

# =============================================================================
# Optional: Enable real GitHub API testing
# =============================================================================
Expand Down
92 changes: 90 additions & 2 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ on:
enable_github_api_tests:
description: 'Run tests against real GitHub API'
required: false
default: 'false'
default: false
type: boolean

env:
Expand Down Expand Up @@ -132,6 +132,94 @@ jobs:
run: |
docker-compose -f docker-compose.test.yml down -v --remove-orphans

# E2E tests with HashiCorp Vault backend
e2e-test-vault:
runs-on: ubuntu-latest
timeout-minutes: 10

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Create .env file
run: |
cat > .env << EOF
TEST_AGENT_NAME=${{ env.TEST_AGENT_NAME }}
TEST_AGENT_EMAIL=${{ env.TEST_AGENT_EMAIL }}
TEST_GITHUB_APP_ID=${{ secrets.TEST_GITHUB_APP_ID }}
TEST_GITHUB_INSTALLATION_ID=${{ secrets.TEST_GITHUB_INSTALLATION_ID }}
TEST_GITHUB_PRIVATE_KEY_B64=${{ secrets.TEST_GITHUB_PRIVATE_KEY_B64 }}
TEST_GITHUB_API_ENABLED=false
EOF

- name: Build containers
run: |
docker-compose -f docker-compose.test-vault.yml build

- name: Start Vault server
run: |
docker-compose -f docker-compose.test-vault.yml up -d vault-server

echo "Waiting for Vault..."
timeout 30 bash -c 'until curl -sf http://localhost:8200/v1/sys/health 2>/dev/null; do sleep 1; done' || {
docker-compose -f docker-compose.test-vault.yml logs vault-server
exit 1
}
echo "Vault is ready"

- name: Seed Vault with test credentials
run: |
docker-compose -f docker-compose.test-vault.yml up vault-seeder

EXIT_CODE=$(docker inspect test-vault-seeder-hcv --format='{{.State.ExitCode}}')
if [ "$EXIT_CODE" != "0" ]; then
echo "Vault seeder failed"
docker-compose -f docker-compose.test-vault.yml logs vault-seeder
exit 1
fi

- name: Start auth service
run: |
docker-compose -f docker-compose.test-vault.yml up -d github-auth-service

echo "Waiting for auth service..."
timeout 60 bash -c 'until curl -sf http://localhost:8080/health 2>/dev/null; do sleep 2; done' || {
docker-compose -f docker-compose.test-vault.yml logs github-auth-service
exit 1
}
echo "Auth service is ready"

- name: Run e2e tests
run: |
docker-compose -f docker-compose.test-vault.yml up test-runner

EXIT_CODE=$(docker inspect test-runner-vault --format='{{.State.ExitCode}}')
docker-compose -f docker-compose.test-vault.yml logs test-runner
exit $EXIT_CODE

- name: Collect logs on failure
if: failure()
run: |
echo "=== Vault server logs ==="
docker-compose -f docker-compose.test-vault.yml logs vault-server
echo ""
echo "=== Vault seeder logs ==="
docker-compose -f docker-compose.test-vault.yml logs vault-seeder
echo ""
echo "=== Auth service logs ==="
docker-compose -f docker-compose.test-vault.yml logs github-auth-service
echo ""
echo "=== Test runner logs ==="
docker-compose -f docker-compose.test-vault.yml logs test-runner

- name: Cleanup
if: always()
run: |
docker-compose -f docker-compose.test-vault.yml down -v --remove-orphans

# Build-only job for PRs without secrets
build:
runs-on: ubuntu-latest
Expand All @@ -142,7 +230,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: '1.22'
go-version: '1.23'

- name: Build
run: |
Expand Down
74 changes: 52 additions & 22 deletions AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,22 +9,23 @@ This is a Go REST API service that provides GitHub App authentication and SSH co
## Architecture

```
Agent Container Auth Service Bitwarden
│ │ │
│ X-Agent-Name + X-Agent-Token │ │
├─────────────────────────────────►│ │
│ │ (startup only) │
│ │◄──────────────────────────┤
│ Installation Token │ │
│◄─────────────────────────────────┤ │
│ │ │
Agent Container Auth Service Secrets Backend
│ │ (Bitwarden or Vault)
│ X-Agent-Name + X-Agent-Token │ │
├─────────────────────────────────►│ │
│ │ (startup only) │
│ │◄──────────────────────┤
│ Installation Token │ │
│◄─────────────────────────────────┤ │
│ │ │
```

**Key design decisions:**
- **Token-based auth**: Agents authenticate with pre-signed tokens, not IP-based identification
- **Fetch-at-startup**: All secrets loaded from Bitwarden at startup, session closed immediately
- **Pluggable backends**: SecretsBackend interface supports Bitwarden and HashiCorp Vault
- **Fetch-at-startup**: All secrets loaded from backend at startup, session closed immediately
- **SSH signing**: Uses the GitHub App's RSA private key for commit signing (not GPG)
- **Collection-based config**: Agents defined by Bitwarden collection contents, not a config file
- **Scope-based config**: Agents defined by collection (Bitwarden) or path (Vault) contents

## Directory Structure

Expand All @@ -38,10 +39,11 @@ Agent Container Auth Service Bitwarden
│ ├── api/ # HTTP handlers, middleware, router
│ ├── config/ # Environment variable loading
│ ├── github/ # GitHub API client, JWT generation, token caching
│ ├── secrets/ # Bitwarden backend, in-memory store
│ ├── secrets/ # Secrets backends (Bitwarden, Vault), in-memory store
│ └── ssh/ # SSH signature generation
├── scripts/
│ └── seed-vault.sh # Test setup: seeds Vaultwarden with test data
│ ├── seed-vault.sh # Test setup: seeds Vaultwarden with test data
│ └── seed-vault-hcv.sh # Test setup: seeds HashiCorp Vault with test data
├── tests/
│ └── e2e_test.go # End-to-end tests
└── examples/
Expand All @@ -52,8 +54,10 @@ Agent Container Auth Service Bitwarden

| File | Purpose |
|------|---------|
| `internal/secrets/backend.go` | SecretsBackend interface, shared types |
| `internal/secrets/store.go` | In-memory secrets store, token verification |
| `internal/secrets/bitwarden.go` | Bitwarden CLI wrapper |
| `internal/secrets/bitwarden.go` | Bitwarden CLI backend |
| `internal/secrets/vault.go` | HashiCorp Vault KV v2 backend |
| `internal/api/middleware.go` | Token authentication middleware |
| `internal/api/handlers.go` | REST API endpoint handlers |
| `internal/github/service.go` | GitHub JWT generation and token fetching |
Expand Down Expand Up @@ -82,18 +86,28 @@ All `/api/v1/*` endpoints require `X-Agent-Name` and `X-Agent-Token` headers.

## Testing

E2E tests use Vaultwarden (self-hosted Bitwarden) in Docker Compose:
E2E tests run against both backends:

```bash
docker-compose -f docker-compose.test.yml up --build --abort-on-container-exit
# Bitwarden backend (Vaultwarden)
make test-e2e

# HashiCorp Vault backend
make test-e2e-vault
```

The test stack:
The Bitwarden test stack (`docker-compose.test.yml`):
1. `vaultwarden` - Test Bitwarden server
2. `vault-seeder` - Populates test credentials
3. `github-auth-service` - Service under test
4. `test-runner` - Executes Go tests

The Vault test stack (`docker-compose.test-vault.yml`):
1. `vault-server` - HashiCorp Vault dev server
2. `vault-seeder` - Populates test credentials
3. `github-auth-service` - Service under test (SECRETS_BACKEND=vault)
4. `test-runner` - Executes Go tests (same tests, backend-agnostic)

## Common Tasks

### Adding a new API endpoint
Expand All @@ -103,12 +117,20 @@ The test stack:
3. Register route in `internal/api/router.go`
4. Add test in `tests/e2e_test.go`

### Modifying Bitwarden item structure
### Modifying agent item structure

1. Update `parseAgentItem()` in `internal/secrets/store.go`
2. Update `scripts/seed-vault.sh` for test seeding
2. Update `scripts/seed-vault.sh` (Bitwarden) and `scripts/seed-vault-hcv.sh` (Vault) for test seeding
3. Update README.md documentation

### Adding a new secrets backend

1. Implement `SecretsBackend` interface in `internal/secrets/<backend>.go`
2. Add config fields in `internal/config/config.go`
3. Add case to the backend selection switch in `cmd/server/main.go`
4. Add e2e test infrastructure (docker-compose, seeder script)
5. Update documentation

### Adding a new helper binary

1. Create `cmd/<binary-name>/main.go`
Expand All @@ -119,15 +141,21 @@ The test stack:

- `github.com/go-chi/chi/v5` - HTTP router
- `github.com/golang-jwt/jwt/v5` - JWT generation for GitHub App auth
- `github.com/hashicorp/vault/api` - HashiCorp Vault client
- `golang.org/x/crypto/ssh` - SSH key handling and signatures

## Environment Variables

The service reads all config from environment variables (no config files):

- `BW_SESSION` / `BW_SESSION_FILE` - Bitwarden session token
- `BW_COLLECTION_ID` / `BW_COLLECTION_ID_FILE` - Collection containing agent items
- `BW_SERVER_URL` - Optional Vaultwarden URL
- `SECRETS_BACKEND` - `bitwarden` (default) or `vault`
- `BW_SESSION` / `BW_SESSION_FILE` - Bitwarden session token (bitwarden backend)
- `BW_COLLECTION_ID` / `BW_COLLECTION_ID_FILE` - Collection containing agent items (bitwarden backend)
- `BW_SERVER_URL` - Optional Vaultwarden URL (bitwarden backend)
- `VAULT_ADDR` - Vault server address (vault backend)
- `VAULT_TOKEN` / `VAULT_TOKEN_FILE` - Vault auth token (vault backend)
- `VAULT_MOUNT_PATH` - KV v2 mount path, default `secret` (vault backend)
- `VAULT_BASE_PATH` - Base path for agents, default `agents` (vault backend)
- `GITHUB_API_URL` - Optional GitHub Enterprise URL
- `LOG_LEVEL` - INFO or DEBUG
- `PORT` - Server port (default 8080)
Expand All @@ -138,4 +166,6 @@ The service reads all config from environment variables (no config files):
- There is no Docker socket access - authentication is token-based
- GPG is NOT used - SSH signing with the GitHub App's RSA key
- The Bitwarden CLI is called via `exec.Command`, not a Go library
- The Vault backend uses the `hashicorp/vault/api` Go client (no CLI needed)
- Secrets are loaded once at startup and held in memory
- Backend selection is via `SECRETS_BACKEND` env var, defaulting to `bitwarden`
Loading