Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,21 +27,21 @@ Fix root cause. Unsure: read more code; if stuck, ask w/ short options. Unrecogn
| 002-parser | Bash syntax parser design |
| 003-vfs | Virtual filesystem abstraction |
| 004-testing | Testing strategy and patterns |
| 005-builtins | Builtin command implementations |
| 005-builtins | Builtin command design (trait, ShellRef, ExecutionPlan) |
| 005-security-testing | Fail-point injection for security testing |
| 006-threat-model | Security threats and mitigations |
| 007-parallel-execution | Threading model, Arc usage |
| 008-documentation | Rustdoc guides, embedded markdown |
| 008-posix-compliance | POSIX design rationale, security exclusions |
| 008-release-process | Version tagging, crates.io + PyPI publishing |
| 009-implementation-status | Feature status, test coverage, limitations |
| 008-release-process | Version tagging, crates.io + PyPI + npm publishing |
| 009-implementation-status | Feature status, test coverage, limitations, POSIX compliance |
| 009-tool-contract | Public LLM Tool trait contract |
| 010-git-support | Sandboxed git operations on VFS |
| 011-python-builtin | Embedded Python via Monty, security, resource limits |
| 012-eval | LLM evaluation harness, dataset format, scoring |
| 012-maintenance | Pre-release maintenance requirements |
| 013-python-package | Python package, PyPI wheels, platform matrix |
| 014-scripted-tool-orchestration | Compose ToolDef+callback pairs into OrchestratorTool via bash scripts |
| 015-ssh-support | Sandboxed SSH/SCP/SFTP operations |
| 016-zapcode-runtime | Embedded TypeScript via ZapCode, VFS bridging, resource limits |
| 017-request-signing | Transparent Ed25519 request signing (bot-auth) per RFC 9421 |
| 018-interactive-shell | Interactive REPL mode with rustyline line editing |
Expand Down
2 changes: 1 addition & 1 deletion crates/bashkit/docs/compatibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
## POSIX Shell Compliance

Bashkit provides substantial compliance with IEEE Std 1003.1-2024 (POSIX.1-2024)
Shell Command Language. See [specs/008-posix-compliance.md](../specs/008-posix-compliance.md)
Shell Command Language. See [specs/009-implementation-status.md](../specs/009-implementation-status.md)
for detailed compliance status.

| POSIX Category | Status |
Expand Down
2 changes: 1 addition & 1 deletion docs/security.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ the sandbox. Key exclusions:
- **`trap`** — conflicts with the stateless execution model
- **Real process spawning** — all subprocess commands stay within the virtual interpreter (`TM-ESC-015`)

These decisions are documented in [`specs/008-posix-compliance.md`](../specs/008-posix-compliance.md).
These decisions are documented in [`specs/009-implementation-status.md`](../specs/009-implementation-status.md).

## Security testing

Expand Down
119 changes: 16 additions & 103 deletions specs/001-architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,103 +9,22 @@ Implemented

Bashkit uses a Cargo workspace with multiple crates:

```
bashkit/
├── crates/
│ ├── bashkit/ # Core library
│ └── bashkit-cli/ # CLI binary (future)
├── specs/ # Design specifications
├── tests/ # Integration tests (future)
└── Cargo.toml # Workspace root
```

### Core Library Structure (`crates/bashkit/`)

```
src/
├── lib.rs # Public API: Bash struct
├── error.rs # Error types
├── limits.rs # Execution limits
├── tool.rs # LLM tool contract (Tool, ToolBuilder)
├── parser/ # Lexer + Parser + AST
│ ├── mod.rs # Parser implementation
│ ├── lexer.rs # Tokenization
│ ├── tokens.rs # Token types
│ └── ast.rs # AST node types
├── interpreter/ # Execution engine
│ ├── mod.rs # Interpreter implementation
│ ├── glob.rs # Glob pattern matching and expansion
│ ├── state.rs # ExecResult and state types
│ └── jobs.rs # Job table for background execution
├── fs/ # Virtual filesystem
│ ├── mod.rs # Module exports
│ ├── traits.rs # FileSystem trait
│ └── memory.rs # InMemoryFs implementation
├── network/ # Network access (optional)
│ ├── mod.rs # Module exports
│ ├── allowlist.rs # URL allowlist
│ └── client.rs # HTTP client
└── builtins/ # Built-in commands
├── mod.rs # Builtin trait + Context
├── echo.rs # echo, printf
├── flow.rs # true, false, exit, break, continue, return
├── navigation.rs # cd, pwd
├── fileops.rs # mkdir, rm, cp, mv, touch, chmod
├── headtail.rs # head, tail
├── sortuniq.rs # sort, uniq
├── cuttr.rs # cut, tr
├── wc.rs # wc
├── date.rs # date
├── sleep.rs # sleep
├── wait.rs # wait
├── curl.rs # curl, wget
└── ... # grep, sed, awk, jq, etc.
```
| Crate | Purpose |
|-------|---------|
| `crates/bashkit/` | Core library (parser, interpreter, VFS, builtins, tool contract) |
| `crates/bashkit-cli/` | CLI binary |
| `crates/bashkit-python/` | Python bindings (PyO3) |
| `crates/bashkit-js/` | JavaScript bindings (NAPI-RS) |
| `crates/bashkit-eval/` | LLM evaluation harness |

The core library modules: `parser/`, `interpreter/`, `fs/`, `builtins/`,
`network/`, `git/`, `ssh/`, `scripted_tool/`. See the source for current
structure — it evolves as features are added.

### Public API

```rust
// Main entry point
pub struct Bash {
fs: Arc<dyn FileSystem>,
interpreter: Interpreter,
}

impl Bash {
pub fn new() -> Self;
pub fn builder() -> BashBuilder;
pub async fn exec(&mut self, script: &str) -> Result<ExecResult>;
}

pub struct ExecResult {
pub stdout: String,
pub stderr: String,
pub exit_code: i32,
}

// LLM Tool Contract
pub trait Tool: Send + Sync {
fn name(&self) -> &str;
fn short_description(&self) -> &str;
fn description(&self) -> String; // Dynamic, includes custom builtins
fn help(&self) -> String; // Full docs for LLMs
fn system_prompt(&self) -> String; // Token-efficient for sysprompt
fn input_schema(&self) -> serde_json::Value;
fn output_schema(&self) -> serde_json::Value;
fn version(&self) -> &str;
async fn execute(&mut self, req: ToolRequest) -> ToolResponse;
async fn execute_with_status(...) -> ToolResponse;
}

pub struct BashTool { /* virtual bash implementing Tool */ }
pub struct BashToolBuilder { /* builder pattern */ }
pub struct ToolRequest { commands: String } // Like bash -c
pub struct ToolResponse { stdout, stderr, exit_code, error }

impl BashTool {
pub fn builder() -> BashToolBuilder;
}
```
Main entry point is `Bash` (library) and `BashTool` (LLM tool contract).
See `crates/bashkit/src/lib.rs` for the full public API surface.

### Design Principles

Expand All @@ -119,24 +38,18 @@ impl BashTool {
### Single crate vs workspace
Rejected single crate because:
- CLI binary would bloat the library
- Python package needs separate crate
- Python/JS packages need separate crates
- Cleaner separation of concerns

### Sync vs async filesystem
Rejected sync because:
- just-bash is fully async
- Future network operations need async
- Bashkit is fully async
- Network operations need async
- tokio is already a dependency

## Verification

```bash
# Build succeeds
cargo build

# Tests pass including e2e
cargo test

# Basic usage works
cargo test --lib -- tests::test_echo_hello
```
139 changes: 13 additions & 126 deletions specs/002-parser.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,93 +13,8 @@ Bashkit uses a recursive descent parser with a context-aware lexer.
Input → Lexer → Tokens → Parser → AST
```

### Token Types

```rust
pub enum Token {
// Literals
Word(String), // Unquoted or quoted words
Number(i64), // Integer literals

// Operators
Pipe, // |
And, // &&
Or, // ||
Semicolon, // ;
Newline, // \n
Background, // &

// Redirections
RedirectOut, // >
RedirectAppend, // >>
RedirectIn, // <
HereDoc, // <<
HereString, // <<<
ProcessSubIn, // <(
ProcessSubOut, // >(

// File descriptor redirections
RedirectBoth, // &>
DupOutput, // >&
RedirectFd(i32), // 2>
RedirectFdAppend(i32), // 2>>
DupFd(i32, i32), // 2>&1

// Grouping
LeftParen, // (
RightParen, // )
LeftBrace, // {
RightBrace, // }
DoubleLeftBracket, // [[
DoubleRightBracket, // ]]

// Keywords (detected after lexing)
// if, then, else, elif, fi
// for, while, until, do, done
// case, esac, in
// function
}
```

### AST Structure

```rust
pub struct Script {
pub commands: Vec<Command>,
}

pub enum Command {
Simple(SimpleCommand), // ls -la
Pipeline(Pipeline), // cmd1 | cmd2 | cmd3
List(CommandList), // cmd1 && cmd2 || cmd3
Compound(CompoundCommand), // if, for, while, case, { }, ( )
Function(FunctionDef), // function foo() { }
}

pub struct SimpleCommand {
pub name: Word,
pub args: Vec<Word>,
pub redirects: Vec<Redirect>,
pub assignments: Vec<Assignment>, // VAR=value cmd
}

pub struct Word {
pub parts: Vec<WordPart>,
}

pub enum WordPart {
Literal(String),
Variable(String), // $VAR
CommandSub(Script), // $(cmd) or `cmd`
ArithmeticSub(String), // $((expr))
DoubleQuoted(Vec<WordPart>), // "text $var"
Length(String), // ${#var}
ArrayAccess { name, index },// ${arr[i]} or ${arr[@]}
ArrayLength(String), // ${#arr[@]}
ArrayIndices(String), // ${!arr[@]}
ParameterExpansion { ... }, // ${var:-default}, ${var#pattern}, etc.
}
```
Token types, AST structures, and parser grammar are defined in
`crates/bashkit/src/parser/`. They evolve as features are added.

### Parser Rules (Simplified)

Expand All @@ -113,12 +28,9 @@ redirect → ('>' | '>>' | '<' | '<<' | '<<<') word
| NUMBER ('>' | '<') word
```

Note: The `&` operator marks the preceding command for background execution.
Currently, background commands run synchronously but are parsed correctly.

### Context-Aware Lexing

The lexer must handle bash's context-sensitivity:
The lexer handles bash's context-sensitivity:
- `$var` in double quotes: expand variable
- `$var` in single quotes: literal text
- Word splitting after expansion
Expand All @@ -128,51 +40,26 @@ The lexer must handle bash's context-sensitivity:

### Arithmetic Expressions

Arithmetic expansion `$((expr))` supports:
- Basic operators: `+`, `-`, `*`, `/`, `%`
- Comparison: `==`, `!=`, `<`, `>`, `<=`, `>=`
- Logical: `&&`, `||` (with short-circuit evaluation)
- Bitwise: `&`, `|`, `^`, `~`, `<<`, `>>`
- Ternary: `cond ? true : false`
- Variable references with or without `$` prefix
`$((expr))` supports: `+`, `-`, `*`, `/`, `%`, comparisons, logical `&&`/`||`
(short-circuit), bitwise operators, ternary `?:`, variable references.

### Error Recovery

Parser produces errors with:
- Line and column numbers
- Expected vs. found token
- Context (what was being parsed)
Parser produces errors with line/column numbers, expected vs. found token,
and context (what was being parsed).

## Alternatives Considered

### PEG parser (pest, pom)
Rejected because:
- Bash grammar is context-sensitive
- PEG can't handle here-docs well
- Manual parser gives better error messages
Rejected: Bash grammar is context-sensitive, PEG can't handle here-docs well,
manual parser gives better error messages.

### Tree-sitter
Rejected because:
- Designed for incremental parsing (overkill)
- Would add large dependency
- Harder to customize for our needs
Rejected: Designed for incremental parsing (overkill), large dependency,
harder to customize.

## Verification

```rust
#[test]
fn test_parse_pipeline() {
let parser = Parser::new("echo hello | cat");
let script = parser.parse().unwrap();
assert!(matches!(script.commands[0], Command::Pipeline(_)));
}

#[test]
fn test_parse_redirect() {
let parser = Parser::new("echo hello > /tmp/out");
let script = parser.parse().unwrap();
if let Command::Simple(cmd) = &script.commands[0] {
assert_eq!(cmd.redirects.len(), 1);
}
}
```bash
cargo test --lib -- parser
```
Loading
Loading