Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
106 changes: 106 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
# Raid - Distributed Development Orchestration

## Project Overview

Raid is a Go-based CLI tool that orchestrates development tasks, environments, and dependencies across distributed repositories. It uses YAML/JSON profile configurations to define multi-repo environments and automates setup/execution workflows.

## Architecture

### Core Components
- **CLI Layer**: `src/cmd/` - Cobra-based command structure with subcommands for profiles, installation, and environments
- **Business Logic**: `src/raid/` - High-level API layer that delegates to internal libraries
- **Internal Implementation**: `src/internal/lib/` - Core functionality for profiles, repositories, environments, and task execution
- **System Utilities**: `src/internal/sys/` and `src/utils/` - System-level operations and shared utilities

### Key Design Patterns

#### Configuration Management
- Uses **Viper** for configuration with global state in `src/internal/lib/config.go`
- **Context singleton** pattern: `lib.Context` struct caches active profile and environment
- **Lazy loading**: `Load()` uses cached context, `ForceLoad()` rebuilds from scratch
- Configuration path customizable via `--config/-c` flag

#### Profile System
- Profiles define collections of repositories and environments
- **Multi-document YAML** support using `---` separators for multiple profiles per file
- **JSON Schema validation** against `schemas/raid-profile.schema.json`
- Profile state managed in Viper config under `"profiles"` key

#### Repository Management
- Concurrent cloning with optional thread limits (`--threads/-t` flag)
- Uses Go routines with semaphore pattern for concurrency control
- Repository validation and error aggregation across parallel operations

#### Environment Execution
- Environments contain tasks (Shell commands or Script files) and environment variables
- Task execution supports concurrent execution flag per task
- Environment variables set globally during environment execution

### Key Files & Patterns

#### Entry Points
- `main.go` - Simple delegator to `cmd.Execute()`
- `src/cmd/raid.go` - Root Cobra command with initialization lifecycle

#### Command Structure
```
src/cmd/
├── raid.go # Root command, global flags, initialization
├── profile/ # Profile management (add, list, use, remove)
├── install/ # Repository installation
└── env/ # Environment execution
```

#### Core Business Logic Flow
1. **Initialize**: `raid.Initialize()` → `lib.InitConfig()` → `lib.Load()`
2. **Profile Management**: Viper-backed persistence with JSON schema validation
3. **Repository Installation**: Concurrent git cloning with error aggregation
4. **Environment Execution**: Task orchestration with variable setting

### Development Workflows

#### Building & Testing
```bash
go build -o raid # Build binary
go test ./... # Run tests
go test -coverprofile=coverage.out ./... # Generate coverage
```

#### JSON Schema Integration
- Schemas in `schemas/` directory define validation rules
- Use `github.com/santhosh-tekuri/jsonschema/v6` for validation
- YAML language server integration with `# yaml-language-server: $schema=...` comments

#### Configuration Files
- **Profile configs**: YAML/JSON files following `schemas/raid-profile.schema.json`
- **Multi-profile files**: Use YAML `---` document separators
- **Examples**: See `docs/examples/` for reference configurations

### Common Patterns

#### Error Handling
- Use `fmt.Errorf()` for wrapped errors with context
- Aggregate errors from concurrent operations into slices
- CLI commands print errors to stderr via `cmd.PrintErrln()`

#### Concurrent Operations
```go
// Semaphore pattern for limiting concurrency
semaphore := make(chan struct{}, maxThreads)
var wg sync.WaitGroup
errorChan := make(chan error, len(items))

// In goroutine:
semaphore <- struct{}{}
defer func() { <-semaphore }()
```

#### Viper Configuration
- Global config management via `viper.GetString()`, `viper.Set()`
- Nested keys accessed with dot notation: `viper.GetStringMapString("profiles")`
- Automatic config file discovery and loading

### Testing & Quality
- Uses standard Go testing with coverage reporting
- GitHub Actions CI/CD pipeline defined (`.github/workflows/build.yml`)
- Codecov integration for coverage tracking
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
*.dll
*.so
*.dylib
raid
/raid

# Test binary, built with `go test -c`
*.test
Expand Down
52 changes: 37 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,46 +1,63 @@
# Raid - Distributed Development Orchestration
[![Build and Test](https://github.com/8bitAlex/raid/actions/workflows/build.yml/badge.svg)](https://github.com/8bitAlex/raid/actions/workflows/build.yml)
[![codecov](https://codecov.io/github/8bitAlex/raid/graph/badge.svg?token=Z75V7I2TLW)](https://codecov.io/github/8bitAlex/raid)

# Raid - Distributed Development Orchestration
![Windows](https://img.shields.io/badge/Windows-Yes-blue?logo=windows)
![macOS](https://img.shields.io/badge/macOS-Yes-lightgrey?logo=apple)
![Linux](https://img.shields.io/badge/Linux-Yes-yellow?logo=linux)


`Raid` is a configurable command-line application that orchestrates common development tasks, environments, and dependencies across distributed code repositories.

If you have ever pulled a repo (or repos) that require days of configuration just to get a passing build,
or have onboarded to a new team that has no documentation, or have a folder of scripts to automate your tasks but haven't
shared them yet, then you are probably a software engineer in need of this.
shared them yet, then you are probably in need of this.

`Raid` handles the pain of error-prone knowledge-dependent tasks and management of your development environment. You no longer need
to worry about wasted time onboarding new contributors. Tribal knowledge can be codified into the repo itself. And you will
never miss running that one test ever again.

📖 For a deeper look at the goals and design of raid, see the [design proposal blog post](https://alexsalerno.dev/blog/raid-design-proposal?utm_source=chatgpt.com).

[Getting Started](#getting-started) • [Best Practices](#⚠-best-practices) • [Documentation](#usage--documentation)

## Key Features

- **Portable YAML Configurations**: Define your development environments, tasks, and dependencies using simple, version-controlled YAML files.
- **Portable YAML Configurations**: Define your development environments, tasks, and dependencies using simple, version-controlled YAML files. Your configurations live alongside your code, making them easy to share and maintain.
- **Multiple Profiles**: Easily switch between different project setups or team configurations with isolated profiles.
- **Automated Task Execution**: Orchestrate shell commands, scripts, and custom tasks across multiple repositories with a single command.
- **Environment Management**: Define, share, and execute complex development environments to ensure consistency for all contributors.

| Platform | Supported |
|----------|:---------:|
| Linux | ✅ |
| Mac | ✅ |
| Windows | ✅ |

## Development

`Raid` is currently in the **prototype stage**. Core functionality is still being explored and iterated on, so expect frequent changes and incomplete features.

Feedback, issues, and contributions are welcome as the project takes shape.

---

[Getting Started](#getting-started) • [Best Practices](#best-practices) • [Documentation](#usage--documentation)

---

## Getting Started

### Installation

#### MacOS

```bash
brew install raid # coming soon
```

#### Linux

```bash
# Installation instructions will be added here
# coming soon
```

#### Windows

```bash
# coming soon
```

### Configuration
Expand All @@ -57,10 +74,15 @@ raid install # Clone repos and setup environment
raid env dev # Execute development environment (if configured)
```

## ⚠ Best Practices
## Best Practices

### Store sensitive profiles securely

If your raid profile contains sensitive configuration or secrets, keep it in a secure, private location outside of your public codebase.

### Never commit secrets

- **Store profiles securely:** If your raid profile contains sensitive configuration or secrets, keep it in a secure, private location outside of your public codebase.
- **Never commit secrets:** Always keep secrets and credentials in private raid profiles. Do not store them in public repositories.
Always keep secrets and credentials in private raid profiles. Do not store them in public repositories.

## Usage & Documentation

Expand Down
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ require (
github.com/spf13/cobra v1.9.1
github.com/spf13/viper v1.20.1
gopkg.in/yaml.v3 v3.0.1
github.com/joho/godotenv v1.5.1
)

require (
Expand Down
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0=
github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
Expand Down
37 changes: 30 additions & 7 deletions src/cmd/env/env.go
Original file line number Diff line number Diff line change
@@ -1,23 +1,46 @@
package env

import (
"github.com/8bitalex/raid/src/raid"
"github.com/8bitalex/raid/src/raid/env"
"github.com/spf13/cobra"
)

var (
concurrency int
)

func init() {
Command.Flags().IntVarP(&concurrency, "threads", "t", 0, "Maximum number of concurrent task executions (0 = unlimited)")
Command.AddCommand(ListEnvCmd)
}

var Command = &cobra.Command{
Use: "env [environment-name]",
Short: "Execute an environment",
Long: "Execute an environment by name. The environment will be searched for in the active profile and all repository configurations. Tasks are executed concurrently and environment variables are set globally.",
Args: cobra.ExactArgs(1),
Args: cobra.RangeArgs(0, 1),
Run: func(cmd *cobra.Command, args []string) {

if len(args) == 0 {
env := env.Get()
if env == "" {
cmd.PrintErrln("No active environment set.")
} else {
cmd.Println("Active environment:", env)
}
} else if len(args) == 1 {
name := args[0]
if !env.Contains(name) {
cmd.PrintErrln("Environment not found:", name)
} else {
cmd.Println("Setting up environment:", name)
if err := env.Set(name); err != nil {
cmd.PrintErrln("Failed to switch environment:", err)
}
raid.ForceLoad()
if err := env.Execute(env.Get()); err != nil {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Environment Variable Conflict and Execution Order Issues

The env command's new logic for setting an environment introduces a local env variable that shadows the imported package, causing compilation errors for env.Contains, env.Set, and env.Execute. Additionally, env.Set() runs before raid.ForceLoad(), which it depends on for context. The command also continues execution without handling errors from raid.ForceLoad() or a failed env.Set().

Fix in Cursor Fix in Web

cmd.PrintErrln("Failed to execute environment:", err)
} else {
cmd.Println("Environment executed successfully.")
}
}
} else {
cmd.PrintErrln("Invalid number of arguments.")
}
},
}
25 changes: 25 additions & 0 deletions src/cmd/env/list.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
package env

import (
"fmt"

"github.com/8bitalex/raid/src/raid/env"
"github.com/spf13/cobra"
)

var ListEnvCmd = &cobra.Command{
Use: "list",
Short: "List environments",
Run: func(cmd *cobra.Command, args []string) {
envs := env.ListAll()
if len(envs) == 0 {
fmt.Println("No environments found.")
return
}
fmt.Println("Available environments:")
for _, env := range envs {
fmt.Printf("\t%s\n", env)
}
fmt.Print()
},
}
2 changes: 1 addition & 1 deletion src/cmd/profile/list.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ var ListProfileCmd = &cobra.Command{
Use: "list",
Short: "List profiles",
Run: func(cmd *cobra.Command, args []string) {
profiles := pro.GetAll()
profiles := pro.ListAll()
activeProfile := pro.Get()

if len(profiles) == 0 {
Expand Down
23 changes: 17 additions & 6 deletions src/cmd/profile/profile.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package profile

import (
"fmt"
"os"

pro "github.com/8bitalex/raid/src/raid/profile"
"github.com/spf13/cobra"
Expand All @@ -10,21 +11,31 @@ import (
func init() {
Command.AddCommand(AddProfileCmd)
Command.AddCommand(ListProfileCmd)
Command.AddCommand(UseProfileCmd)
Command.AddCommand(RemoveProfileCmd)
}

var Command = &cobra.Command{
Use: "profile",
Aliases: []string{"p"},
Short: "Manage raid profiles",
Args: cobra.NoArgs,
Args: cobra.RangeArgs(0, 1),
Run: func(cmd *cobra.Command, args []string) {
profile := pro.Get()
if !profile.IsZero() {
fmt.Println(profile.Name)
if len(args) == 0 {
profile := pro.Get()
if !profile.IsZero() {
fmt.Println(profile.Name)
} else {
fmt.Println("No active profile found. Use 'raid profile use <profile>' to set one.")
}
} else if len(args) == 1 {
name := args[0]
if err := pro.Set(name); err != nil {
fmt.Printf("Profile '%s' not found. Use 'raid profile list' to see available profiles.\n", name)
os.Exit(1)
}
fmt.Printf("Profile '%s' is now active.\n", name)
} else {
fmt.Println("No active profile found. Use 'raid profile use <profile>' to set one.")
cmd.PrintErrln("Invalid number of arguments.")
}
},
}
26 changes: 0 additions & 26 deletions src/cmd/profile/use.go

This file was deleted.

Loading