Skip to content

test(python): add TM-* threat model references to security test names #1262

@chaliy

Description

@chaliy

Context

Node security tests (crates/bashkit-js/__test__/security.spec.ts, 99 tests) use explicit threat model IDs from specs/006-threat-model.md as test name prefixes for traceability:

// Node examples:
"WB: command limit is enforced (TM-DOS-002)"
"WB: fork bomb pattern blocked (TM-DOS-021)"
"WB: exec cannot escape sandbox (TM-ESC-001)"
"WB: /etc/passwd not accessible (TM-INF-001)"

Python security tests use descriptive names but lack these references:

# Python current:
def test_fork_bomb_prevented(): ...
def test_max_loop_iterations_enforced(): ...
def test_vfs_cannot_read_host_etc_passwd(): ...

This makes it hard to verify which threat model scenarios are covered and which are missing.

This is part of the Python ↔ Node binding parity effort (Phase 3 — Security Test Alignment).

What to implement

Rename Python security tests in test_security.py (after merge from #1259) and test_python_security.py to include TM-* IDs where applicable.

Naming convention

# After:
def test_tm_dos_021_fork_bomb_prevented(): ...
def test_tm_dos_016_loop_iteration_limit_enforced(): ...
def test_tm_inf_001_vfs_cannot_read_host_etc_passwd(): ...
def test_tm_esc_001_exec_cannot_escape_sandbox(): ...

Full mapping to apply

TM ID Description Apply to tests matching
TM-DOS-002 Command limit enforcement max_commands tests
TM-DOS-005 Large file write limited large file tests
TM-DOS-006 VFS file count limit file count tests
TM-DOS-012 Deep directory nesting deep nesting tests
TM-DOS-013 Long filename/path rejected long filename tests
TM-DOS-016 Loop iteration limit max_loop_iterations tests
TM-DOS-017 While true capping infinite while tests
TM-DOS-018 Nested loop multiplication nested loop tests
TM-DOS-020 Recursive function depth recursive function tests
TM-DOS-021 Fork bomb blocking fork bomb tests
TM-DOS-029 Arithmetic overflow/div-by-zero arithmetic edge tests
TM-DOS-059 Memory limit memory/string doubling tests
TM-ESC-001 Exec escape exec/sandbox escape tests
TM-ESC-002 Process substitution process substitution tests
TM-ESC-003 /proc access /proc tests
TM-ESC-005 Signal trap signal trap tests
TM-INF-001 /etc/passwd exposure /etc/passwd tests
TM-INF-002 Env var leak env var leak tests
TM-INJ-005 Path traversal directory traversal tests
TM-ISO-001 Variable isolation instance isolation tests (vars)
TM-ISO-002 Filesystem isolation instance isolation tests (fs)
TM-ISO-003 Function isolation instance isolation tests (funcs)
TM-INT-001 Host path leak in errors error message tests
TM-INT-002 Memory address/stack trace leak error message tests
TM-NET-001 /dev/tcp escape network escape tests
TM-UNI-002 Zero-width chars unicode tests
TM-UNI-003 Homoglyph unicode tests
TM-UNI-004 RTL override unicode tests

Tests that don't map to a specific TM-* ID (e.g., callback safety, JSON depth) keep their current names.

Acceptance criteria

  • All applicable tests in test_security.py renamed with _tm_XXX_NNN_ prefix
  • All applicable tests in test_python_security.py renamed with _tm_XXX_NNN_ prefix
  • Test functionality unchanged — only names change
  • All tests pass after rename
  • A comment at top of test_security.py explains the naming convention and links to specs/006-threat-model.md
  • Every TM-* ID in the table above appears in at least one test name (if the scenario is tested)
  • Running grep -c "tm_" tests/test_security.py shows 20+ matches

Depends on

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions