Skip to content

tests(runner): honor directory= URL param to scope TestBox bundles #2280

@bpamiri

Description

@bpamiri

Problem

vendor/wheels/tests/runner.cfm:103 hardcodes directory=\"wheels.tests.specs\" when instantiating the TestBox runner:

```cfscript
testBox = new wheels.wheelstest.system.TestBox(
directory="wheels.tests.specs",
options={ coverage = { enabled = false } }
);
```

But CLAUDE.md and docs document a per-package test-run pattern that relies on a `directory=` URL param:

Testing Packages:
```bash

Run a specific package's tests (package must be in vendor/)

curl "http://localhost:60007/wheels/core/tests?db=sqlite&format=json&directory=vendor.wheels-sentry.tests\"
```

The param is parsed into `url.directory` but never consumed by the runner — the full core suite runs regardless of what you pass. The only `url.directory` references in `vendor/wheels/tests/` are in `html.cfm` (display-only) and `BaseReporter.cfc` (normalisation). No code path hands it to TestBox.

Impact

  • First-party packages (`wheels-sentry`, `wheels-hotwire`, `wheels-basecoat`, `wheels-legacy-adapter`, incoming `wheels-i18n`) ship `tests/` directories, but there's no way to run just those specs via the HTTP endpoint.
  • Contributors end up running the entire 3324-spec core suite (~22 s local, much longer in Docker) to verify a single package change.
  • Registry submissions (`wheels-dev/wheels-packages`) can't include "package tests pass" as a validation step — there's no plumbing to run them.

Repro

```bash
bash tools/test-local.sh vendor.wheels-i18n.tests # still runs 3324 specs
curl "http://localhost:8080/wheels/core/tests?db=sqlite&format=json&directory=vendor.wheels-i18n.tests\"

→ totalPass: 3324, totalSuites: 781 — full core suite, not the 17 i18n specs

```

Proposed fix

In `vendor/wheels/tests/runner.cfm`, read `url.directory` (with a conservative allowlist to keep this safe) and pass it to the TestBox constructor:

```cfscript
local.testDirectory = "wheels.tests.specs";
if (StructKeyExists(url, "directory") && Len(Trim(url.directory))) {
// Only accept paths under the app's vendor/ or the core wheels.tests.*
// (prevents arbitrary CFC directory scans)
if (ReFindNoCase("^(wheels\.tests|vendor\.[a-z0-9][a-z0-9\-]*\.tests)", url.directory)) {
local.testDirectory = url.directory;
}
}
testBox = new wheels.wheelstest.system.TestBox(
directory=local.testDirectory,
options={ coverage = { enabled = false } }
);
```

Plus:

  • Update `tools/test-local.sh` to pass arbitrary `directory=` through to the URL (currently the case statement drops unmapped filters).
  • Add a docs page (or extend `.ai/wheels/testing/`) showing the per-package pattern.

Alternatives

  • Ship a dedicated `/wheels/package/tests` endpoint that takes `name=wheels-i18n` and derives the directory internally. More explicit but adds surface area.
  • Add a `wheels test run --package=` CLI subcommand that shells out to the same endpoint.

Discovered while converting `wheels-dev/wheels-i18n` to the 4.0 package format (wheels-dev/wheels#2268, wheels-dev/wheels-i18n#4).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions