Skip to content

docs: test the broken contract at the lowest reachable layer - #306

Merged
leo-aa88 merged 1 commit into
mainfrom
docs/mandatory-pr-tests
Aug 29, 2026
Merged

docs: test the broken contract at the lowest reachable layer#306
leo-aa88 merged 1 commit into
mainfrom
docs/mandatory-pr-tests

Conversation

@leo-aa88

Copy link
Copy Markdown
Member

Description

Follow-up to #305. The first version of the tests policy said that if you cannot show the bug in a test_cases/*.brainrot program, you have not proven the fix. That is too narrow.

arena_reset() is the counterexample: it has no Brainrot-level caller, so a .brainrot fixture cannot exercise the broken contract. The review on #304 already said the proper regression is a host-side C test that calls the arena API directly under ASan/UBSan, wired into make test the same way as tests/abi/struct_layout_abi_check.c.

The rule is now:

  • Add the regression at the lowest appropriate layer that can exercise the broken contract.
  • Prefer test_cases/*.brainrot for language-visible behavior.
  • Internal runtime/library behavior that Brainrot source cannot reach must be a host-side C test integrated into make test.
  • If the existing harness cannot test the behavior, extend the harness or add an appropriate lower-level test — do not skip the test.

The PR checklist is updated to match. This PR is docs-only (CONTRIBUTING.md and .github/PULL_REQUEST_TEMPLATE.md).

Related Issue

N/A — policy correction after #305. Related discussion: #304 / #286 (arena_reset()).

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Performance improvement
  • Refactor

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have documented my changes in the code or documentation
  • I have added tests that prove my change works, at the lowest appropriate layer (test_cases/*.brainrot for language-visible behavior; a host-side C test wired into make test for internal APIs Brainrot source cannot reach). Required for every bug fix, feature, refactor, performance change, and breaking change — not optional, not "if applicable". See CONTRIBUTING.md ("TESTS ARE MANDATORY").
  • Those tests cover the happy path, the original bug (for fixes), error cases, edge cases, and adversarial cases. If the existing harness could not reach the behavior, I extended it or added a lower-level test.
  • If this PR cannot affect program behavior (docs/comments/license only), I explained that in the Description instead of skipping the items above in silence
  • I have run make format-check locally (or make format to fix)
  • I have run the unit tests locally
  • I have run the valgrind memory tests locally
  • All new and existing tests pass

Docs-only: no interpreter or harness change, so there is nothing for test_cases/ or a host-side C test to prove.

.brainrot fixtures are for language-visible behavior. Internal APIs with
no Brainrot-level caller (arena_reset() is the example) need a host-side
C test under ASan/UBSan, wired into make test. If the harness cannot
reach the behavior, extend it rather than skipping the test.
@leo-aa88
leo-aa88 merged commit 463958a into main Aug 29, 2026
2 checks passed
@leo-aa88
leo-aa88 deleted the docs/mandatory-pr-tests branch August 29, 2026 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant