Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/core-concepts/channels.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,8 @@ When channels are configured in `forge.yaml`, the build pipeline automatically:

1. **Includes channel config files** — `slack-config.yaml`, `telegram-config.yaml`, etc. are copied into the Docker build context alongside `forge.yaml`
2. **Adds `--with` to the entrypoint** — The container entrypoint becomes `["forge", "run", "--host", "0.0.0.0", "--with", "slack,telegram"]`
3. **Handles auth loopback** — When [external auth](runtime.md#external-authentication) is configured, channel adapters authenticate to the A2A server using an internal token, bypassing the external auth provider
3. **Surfaces channel env vars in the manifests** — Every `_env`-suffixed setting in each `<channel>-config.yaml` (e.g. `bot_token_env: SLACK_BOT_TOKEN`) is unioned into the Kubernetes `secrets.yaml` and `deployment.yaml` (via `secretKeyRef`) and into the docker-compose adapter services. Both outputs derive from the same source — see [Kubernetes — Env Var Injection](../deployment/kubernetes.md#env-var-injection)
4. **Handles auth loopback** — When [external auth](runtime.md#external-authentication) is configured, channel adapters authenticate to the A2A server using an internal token, bypassing the external auth provider

Pass channel secrets via environment variables:

Expand Down
23 changes: 23 additions & 0 deletions docs/deployment/kubernetes.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,33 @@ Every `forge build` generates container-ready artifacts:
| `Dockerfile` | Container image with minimal attack surface |
| `deployment.yaml` | Kubernetes Deployment manifest |
| `service.yaml` | Kubernetes Service manifest |
| `secrets.yaml` | Kubernetes Secret with one empty entry per required env var |
| `network-policy.yaml` | NetworkPolicy restricting pod egress to allowed domains |
| `egress_allowlist.json` | Machine-readable domain allowlist |
| `checksums.json` | SHA-256 checksums + Ed25519 signature |

## Env Var Injection

`deployment.yaml` wires each required env var to a `secretKeyRef` against the agent's `<agent_id>-secrets` Secret. The required set is the union of:

- **Skill env vars** — `metadata.forge.requires.env.required` from every `SKILL.md`.
- **Channel env vars** — every `_env`-suffixed setting in each `<channel>-config.yaml` referenced by `channels:` in `forge.yaml`. For example, `bot_token_env: SLACK_BOT_TOKEN` in `slack-config.yaml` adds `SLACK_BOT_TOKEN` to the required set.

The same canonical source feeds `docker-compose.yaml` when `forge package --with-channels` is used, so the two output paths produce a consistent set.

Adding a new channel env var requires zero edits to the build pipeline — append a new `_env`-suffixed setting to the channel YAML and the next `forge build` picks it up. To wire the secret values into the cluster, populate `secrets.yaml` (or replace it with a sealed-secret / ExternalSecret) before applying.

```yaml
# slack-config.yaml — operator adds a per-project override
adapter: slack
settings:
app_token_env: SLACK_APP_TOKEN
bot_token_env: SLACK_BOT_TOKEN
custom_env: MY_PROJECT_SLACK_OVERRIDE # ← appears in secrets.yaml + deployment.yaml
```

A channel listed in `forge.yaml` whose `<channel>-config.yaml` is missing produces a build warning, not an error — the manifest is generated without that channel's env vars.

## Air-Gap Deployments

Forge can run entirely offline with local models:
Expand Down
77 changes: 77 additions & 0 deletions forge-cli/build/channels_stage.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
package build

import (
"context"
"fmt"
"sort"

clichannels "github.com/initializ/forge/forge-cli/channels"
"github.com/initializ/forge/forge-core/agentspec"
"github.com/initializ/forge/forge-core/pipeline"
)

// ChannelsStage unions env var names declared by the project's configured
// communication channels into Spec.Requirements.EnvRequired so the generated
// Kubernetes secrets and deployment manifests include them alongside skill
// env vars.
//
// The canonical source is the per-channel YAML (workDir/<channel>-config.yaml)
// — every setting key ending in "_env" declares an env-var name. Adding a new
// channel adapter that ships its own config template will pick up here with
// no edits to this file.
type ChannelsStage struct{}

func (s *ChannelsStage) Name() string { return "channel-env-vars" }

func (s *ChannelsStage) Execute(ctx context.Context, bc *pipeline.BuildContext) error {
if bc.Config == nil || len(bc.Config.Channels) == 0 {
return nil
}

channelEnv, missing, err := clichannels.EnvVarsFromConfig(bc.Opts.WorkDir, bc.Config.Channels)
if err != nil {
return fmt.Errorf("reading channel env vars: %w", err)
}
for _, name := range missing {
bc.AddWarning(fmt.Sprintf("channel %q is configured but %s-config.yaml is missing; its env vars will not be included in the generated manifests", name, name))
}
if len(channelEnv) == 0 {
return nil
}

if bc.Spec == nil {
return nil
}
if bc.Spec.Requirements == nil {
bc.Spec.Requirements = &agentspec.AgentRequirements{}
}

// Union with existing skill-required env vars, dedup, sort.
seen := make(map[string]bool, len(bc.Spec.Requirements.EnvRequired)+len(channelEnv))
merged := make([]string, 0, len(bc.Spec.Requirements.EnvRequired)+len(channelEnv))
for _, v := range bc.Spec.Requirements.EnvRequired {
if seen[v] {
continue
}
seen[v] = true
merged = append(merged, v)
}
// Skill-declared optional env vars stay optional even if a channel marks
// them required — but in practice channel and skill env-var namespaces
// don't overlap, so this is just defense-in-depth.
optional := make(map[string]bool, len(bc.Spec.Requirements.EnvOptional))
for _, v := range bc.Spec.Requirements.EnvOptional {
optional[v] = true
}
for _, v := range channelEnv {
if seen[v] || optional[v] {
continue
}
seen[v] = true
merged = append(merged, v)
}
sort.Strings(merged)
bc.Spec.Requirements.EnvRequired = merged

return nil
}
176 changes: 176 additions & 0 deletions forge-cli/build/channels_stage_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
package build

import (
"context"
"os"
"path/filepath"
"slices"
"strings"
"testing"

"github.com/initializ/forge/forge-core/agentspec"
"github.com/initializ/forge/forge-core/pipeline"
"github.com/initializ/forge/forge-core/types"
)

func writeChannelYAML(t *testing.T, dir, name, content string) {
t.Helper()
path := filepath.Join(dir, name+"-config.yaml")
if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
t.Fatalf("writing %s: %v", path, err)
}
}

func TestChannelsStage_UnionsWithSkillEnvRequired(t *testing.T) {
dir := t.TempDir()
writeChannelYAML(t, dir, "slack", `
adapter: slack
settings:
app_token_env: SLACK_APP_TOKEN
bot_token_env: SLACK_BOT_TOKEN
`)
writeChannelYAML(t, dir, "telegram", `
adapter: telegram
settings:
bot_token_env: TELEGRAM_BOT_TOKEN
`)

bc := pipeline.NewBuildContext(pipeline.PipelineOptions{WorkDir: dir})
bc.Config = &types.ForgeConfig{Channels: []string{"slack", "telegram"}}
bc.Spec = &agentspec.AgentSpec{
Requirements: &agentspec.AgentRequirements{
EnvRequired: []string{"SKILL_API_KEY"},
},
}

if err := (&ChannelsStage{}).Execute(context.Background(), bc); err != nil {
t.Fatalf("ChannelsStage.Execute: %v", err)
}

want := []string{"SKILL_API_KEY", "SLACK_APP_TOKEN", "SLACK_BOT_TOKEN", "TELEGRAM_BOT_TOKEN"}
if !slices.Equal(bc.Spec.Requirements.EnvRequired, want) {
t.Errorf("EnvRequired = %v, want %v", bc.Spec.Requirements.EnvRequired, want)
}
}

func TestChannelsStage_PopulatesRequirementsWhenNil(t *testing.T) {
// Project with channels but no skills — Spec.Requirements starts nil
// because RequirementsStage early-returns. ChannelsStage must still
// surface channel env vars to the manifests.
dir := t.TempDir()
writeChannelYAML(t, dir, "slack", `
adapter: slack
settings:
bot_token_env: SLACK_BOT_TOKEN
`)

bc := pipeline.NewBuildContext(pipeline.PipelineOptions{WorkDir: dir})
bc.Config = &types.ForgeConfig{Channels: []string{"slack"}}
bc.Spec = &agentspec.AgentSpec{} // Requirements nil

if err := (&ChannelsStage{}).Execute(context.Background(), bc); err != nil {
t.Fatalf("unexpected error: %v", err)
}
if bc.Spec.Requirements == nil {
t.Fatal("Requirements should be created when channels declare env vars")
}
if !slices.Equal(bc.Spec.Requirements.EnvRequired, []string{"SLACK_BOT_TOKEN"}) {
t.Errorf("EnvRequired = %v, want [SLACK_BOT_TOKEN]", bc.Spec.Requirements.EnvRequired)
}
}

func TestChannelsStage_NoChannels(t *testing.T) {
bc := pipeline.NewBuildContext(pipeline.PipelineOptions{WorkDir: t.TempDir()})
bc.Config = &types.ForgeConfig{}
bc.Spec = &agentspec.AgentSpec{}

if err := (&ChannelsStage{}).Execute(context.Background(), bc); err != nil {
t.Fatalf("unexpected error: %v", err)
}
if bc.Spec.Requirements != nil {
t.Error("expected Requirements to stay nil for project with no channels")
}
}

// TestChannelsStage_FlowsThroughToK8sManifests is the end-to-end regression
// for issue #50: channel env vars must appear in the generated K8s deployment
// and secret manifests, not only in docker-compose.
func TestChannelsStage_FlowsThroughToK8sManifests(t *testing.T) {
workDir := t.TempDir()
outDir := t.TempDir()
writeChannelYAML(t, workDir, "slack", `
adapter: slack
settings:
app_token_env: SLACK_APP_TOKEN
bot_token_env: SLACK_BOT_TOKEN
`)
writeChannelYAML(t, workDir, "telegram", `
adapter: telegram
settings:
bot_token_env: TELEGRAM_BOT_TOKEN
`)

bc := pipeline.NewBuildContext(pipeline.PipelineOptions{
WorkDir: workDir,
OutputDir: outDir,
})
bc.Config = &types.ForgeConfig{Channels: []string{"slack", "telegram"}}
bc.Spec = &agentspec.AgentSpec{
AgentID: "test-agent",
Version: "0.1.0",
Runtime: &agentspec.RuntimeConfig{
Image: "python:3.12-slim",
Entrypoint: []string{"python", "agent.py"},
Port: 8080,
},
}

if err := (&ChannelsStage{}).Execute(context.Background(), bc); err != nil {
t.Fatalf("ChannelsStage.Execute: %v", err)
}
if err := (&K8sStage{}).Execute(context.Background(), bc); err != nil {
t.Fatalf("K8sStage.Execute: %v", err)
}

dep := readFile(t, filepath.Join(outDir, "k8s", "deployment.yaml"))
sec := readFile(t, filepath.Join(outDir, "k8s", "secrets.yaml"))

for _, want := range []string{"SLACK_APP_TOKEN", "SLACK_BOT_TOKEN", "TELEGRAM_BOT_TOKEN"} {
if !strings.Contains(dep, want) {
t.Errorf("deployment.yaml missing channel env var %q", want)
}
if !strings.Contains(sec, want) {
t.Errorf("secrets.yaml missing channel env var %q", want)
}
}
if !strings.Contains(dep, "secretKeyRef:") {
t.Error("deployment.yaml should reference channel env vars via secretKeyRef")
}
}

func readFile(t *testing.T, path string) string {
t.Helper()
b, err := os.ReadFile(path)
if err != nil {
t.Fatalf("reading %s: %v", path, err)
}
return string(b)
}

func TestChannelsStage_MissingConfigWarns(t *testing.T) {
// channels: [slack] declared, but no slack-config.yaml on disk.
// The stage must surface a warning and not fail the build.
bc := pipeline.NewBuildContext(pipeline.PipelineOptions{WorkDir: t.TempDir()})
bc.Config = &types.ForgeConfig{Channels: []string{"slack"}}
bc.Spec = &agentspec.AgentSpec{}

if err := (&ChannelsStage{}).Execute(context.Background(), bc); err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(bc.Warnings) != 1 || !strings.Contains(bc.Warnings[0], "slack") {
t.Errorf("expected one warning mentioning slack, got %v", bc.Warnings)
}
if bc.Spec.Requirements != nil {
t.Error("Requirements should stay nil when no channel env vars discovered")
}
}
55 changes: 55 additions & 0 deletions forge-cli/channels/env.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
package channels

import (
"fmt"
"os"
"path/filepath"
"sort"
"strings"
)

// EnvVarsFromConfig returns the sorted, deduped union of env-var names that
// the configured channel adapters require. The canonical source is the
// project's per-channel YAML — every setting key ending in "_env" declares
// an env var name (e.g. "bot_token_env: SLACK_BOT_TOKEN"), matching the
// runtime contract used by channels.ResolveEnvVars.
//
// channelNames are the values from forge.yaml's `channels:` list. For each
// name, the file workDir/<name>-config.yaml is consulted. A missing file is
// reported via missing[] and produces no env vars; parse errors are returned.
//
// This is the single canonical source — build stages, container packaging,
// and any other tooling that needs to know "which env vars do my channels
// require" should call this. Adding a new channel adapter requires no edits
// here: it ships its own *-config.yaml template and the helper picks it up.
func EnvVarsFromConfig(workDir string, channelNames []string) (envVars []string, missing []string, err error) {
seen := make(map[string]bool)
for _, name := range channelNames {
name = strings.TrimSpace(name)
if name == "" {
continue
}
path := filepath.Join(workDir, name+"-config.yaml")
if _, statErr := os.Stat(path); os.IsNotExist(statErr) {
missing = append(missing, name)
continue
}
cfg, loadErr := LoadChannelConfig(path)
if loadErr != nil {
return nil, missing, fmt.Errorf("channel %q: %w", name, loadErr)
}
for k, v := range cfg.Settings {
base, ok := strings.CutSuffix(k, "_env")
if !ok || base == "" {
continue
}
if v == "" || seen[v] {
continue
}
seen[v] = true
envVars = append(envVars, v)
}
}
sort.Strings(envVars)
return envVars, missing, nil
}
Loading
Loading