Allow contributors to run app-hosted tests locally#436
Merged
Conversation
Add a Debug-only entitlements file with get-task-allow so macOS relaxes Hardened Runtime restrictions during test injection. Remove CODE_SIGNING_ALLOWED=NO from CotabbyTests so the bundle is ad-hoc signed and can be loaded into the debug-signed host app.
FuJacob
added a commit
that referenced
this pull request
May 31, 2026
Locks the anti-regression invariant behind the chat-template fix: the user turn must be exactly the prefix with no "Text before caret:" / "Final instruction:" label scaffolding (the strings small instruct models echoed into ghost text), the prefix must not leak into the system turn, and profile/rules/clipboard/screen context must land in the system turn (and be omitted when absent). Six pure-function cases. Full app-hosted suite: 320 tests, 0 failures (run with CODE_SIGNING_ALLOWED=NO per the local-test setup added in #436).
FuJacob
added a commit
that referenced
this pull request
May 31, 2026
Locks the anti-regression invariant behind the chat-template fix: the user turn must be exactly the prefix with no "Text before caret:" / "Final instruction:" label scaffolding (the strings small instruct models echoed into ghost text), the prefix must not leak into the system turn, and profile/rules/clipboard/screen context must land in the system turn (and be omitted when absent). Six pure-function cases. Full app-hosted suite: 320 tests, 0 failures (run with CODE_SIGNING_ALLOWED=NO per the local-test setup added in #436).
FuJacob
added a commit
that referenced
this pull request
May 31, 2026
Locks the anti-regression invariant behind the chat-template fix: the user turn must be exactly the prefix with no "Text before caret:" / "Final instruction:" label scaffolding (the strings small instruct models echoed into ghost text), the prefix must not leak into the system turn, and profile/rules/clipboard/screen context must land in the system turn (and be omitted when absent). Six pure-function cases. Full app-hosted suite: 320 tests, 0 failures (run with CODE_SIGNING_ALLOWED=NO per the local-test setup added in #436).
FuJacob
added a commit
that referenced
this pull request
May 31, 2026
Locks the anti-regression invariant behind the chat-template fix: the user turn must be exactly the prefix with no "Text before caret:" / "Final instruction:" label scaffolding (the strings small instruct models echoed into ghost text), the prefix must not leak into the system turn, and profile/rules/clipboard/screen context must land in the system turn (and be omitted when absent). Six pure-function cases. Full app-hosted suite: 320 tests, 0 failures (run with CODE_SIGNING_ALLOWED=NO per the local-test setup added in #436).
This was referenced May 31, 2026
FuJacob
added a commit
that referenced
this pull request
May 31, 2026
Locks the anti-regression invariant behind the chat-template fix: the user turn must be exactly the prefix with no "Text before caret:" / "Final instruction:" label scaffolding (the strings small instruct models echoed into ghost text), the prefix must not leak into the system turn, and profile/rules/clipboard/screen context must land in the system turn (and be omitted when absent). Six pure-function cases. Full app-hosted suite: 320 tests, 0 failures (run with CODE_SIGNING_ALLOWED=NO per the local-test setup added in #436).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
App-hosted unit tests (
CotabbyTests) were unrunnable for any contributor who does not hold theG946M8K23BApple Developer certificate. The root cause is a combination of two settings on the Cotabby target:ENABLE_HARDENED_RUNTIME: YES— macOS enforces that bundles dynamically loaded into a Hardened Runtime process must be signed, and by default requires a matching Team ID.CODE_SIGNING_ALLOWED: NOon the test bundle — prevented even ad-hoc signing, so macOS rejected the injection.Fix: add
Cotabby-Debug.entitlementswithcom.apple.security.get-task-allow = trueand wire it into the app target's Debug config. This entitlement tells macOS to relax Hardened Runtime injection restrictions for debug/test builds. Also removeCODE_SIGNING_ALLOWED: NOfromCotabbyTestsso the bundle receives ad-hoc signing and can be loaded by the relaxed host.Release/Archive builds are unaffected: the entitlements file is only applied in Debug config, and CI continues to sign with the full team certificate.
Validation
End-to-end
testaction requires the team certificate locally;build-for-testingconfirms the test bundle compiles and links correctly against the host app.Linked issues
Refs the note in CLAUDE.md: "If app-hosted tests fail because of local signing or Team ID mismatch, report the exact failure and still run
build-for-testing."Risk / rollout notes
Cotabby-Debug.entitlementsis only applied to Debug builds via theconfigs.Debug.CODE_SIGN_ENTITLEMENTSkey inproject.yml. Release/Archive builds have no entitlements file change.get-task-allowis a standard Apple-blessed entitlement for test hosts; Xcode adds it automatically when managing entitlements for app-hosted tests. We are making it explicit so XcodeGen propagates it correctly.CODE_SIGNING_ALLOWED: NOfromCotabbyTestsallows ad-hoc signing (CODE_SIGN_IDENTITY: "-") to proceed, which is benign for a test bundle that is never distributed.Greptile Summary
This PR fixes app-hosted test execution for contributors without the
G946M8K23Bteam certificate by adding aCotabby-Debug.entitlementsfile withcom.apple.security.get-task-allowand wiring it into the Debug build config, while also removingCODE_SIGNING_ALLOWED: NOfrom the test bundle so ad-hoc signing can proceed.Cotabby-Debug.entitlements: Minimal entitlements plist containing onlycom.apple.security.get-task-allow = true, scoped exclusively to the Debug configuration viaproject.ymlandproject.pbxproj.project.yml/project.pbxproj:CODE_SIGN_ENTITLEMENTSis set underconfigs.Debugfor the Cotabby app target only; the Release config remains unchanged.CODE_SIGNING_ALLOWED: NOis removed fromCotabbyTestsbase settings, allowing ad-hoc signing (CODE_SIGN_IDENTITY: \"-\") to proceed for the test bundle in both Debug and Release.Confidence Score: 5/5
Safe to merge; changes are correctly scoped to Debug builds and do not touch Release or Archive signing.
The entitlements file contains only the single get-task-allow key Apple explicitly designates for debug/test hosts, and it is wired exclusively to the Debug build configuration in both project.yml and the generated project.pbxproj. The Release XCBuildConfiguration block for the Cotabby target has no CODE_SIGN_ENTITLEMENTS entry, so distribution builds are untouched. Removing CODE_SIGNING_ALLOWED: NO from the test bundle is safe because CODE_SIGNING_REQUIRED: NO remains and the bundle is never distributed. The project.yml and project.pbxproj are in sync.
No files require special attention.
Important Files Changed
Flowchart
%%{init: {'theme': 'neutral'}}%% flowchart TD A[xcodebuild test / build-for-testing] --> B{Build Config?} B -- Debug --> C[Cotabby app signed with Cotabby-Debug.entitlements] C --> D[get-task-allow = true Relaxes Hardened Runtime injection restrictions] D --> E[CotabbyTests bundle ad-hoc signed via CODE_SIGN_IDENTITY = '-'] E --> F[macOS loads test bundle into app process] B -- Release / Archive --> G[Cotabby app signed with team certificate only] G --> H[Full Hardened Runtime enforcement]Reviews (1): Last reviewed commit: "Allow contributors to run app-hosted tes..." | Re-trigger Greptile