Skip to content

docs(sdk): clarify machineUserAttributes is required without userProfile#1168

Draft
dqn wants to merge 1 commit into
mainfrom
feat/sdk-staticwebsite-url-docs
Draft

docs(sdk): clarify machineUserAttributes is required without userProfile#1168
dqn wants to merge 1 commit into
mainfrom
feat/sdk-staticwebsite-url-docs

Conversation

@dqn
Copy link
Copy Markdown
Contributor

@dqn dqn commented May 13, 2026

Docs-only fix surfaced by the llm-challenge benchmark. Two adjacent docs (auth.md and staticwebsite.md) implied that machineUserAttributes is optional. The SDK types make it required whenever machineUsers is used without a userProfile, so AI-authored configs that try machineUsers: { runner: {} } fail typecheck.

Why

When the m18-config-static-website-url benchmark problem asks an agent to wire defineStaticWebSite(...).url into defineAuth({...})'s OAuth2 redirect URIs, the agent reads docs/services/staticwebsite.md and docs/services/auth.md. Both showed:

  • staticwebsite.md "OAuth2 Redirect URIs" example used defineAuth without machineUserAttributes.
  • auth.md line 36 commented // Optional when you don't define userProfile: above machineUserAttributes.

The SDK types tell the opposite story: defineAuth with machineUsers and no userProfile requires machineUserAttributes to type every machineUsers[*].attributes entry. Agents copying the docs example silently hit TS2741: Property 'machineUserAttributes' is missing in type ....

Evidence

A/B experiment via pnpm challenge:experiment on m18-config-static-website-url (full-package profile, N=5 iterations):

Metric Baseline (main) Candidate (this branch) Delta
iteration passRate 2/5 (40%) 4/5 (80%) +40 pp
cost median $0.142 $0.157 +$0.015
cost stdev $0.329 $0.112 -66%
turns median delta 0 0 -

The stdev collapse (0.329 → 0.112) shows the docs fix not only raises the pass rate but also reduces the variance: with the corrected example, agents are converging on the same approach instead of thrashing.

What changed

  • packages/sdk/docs/services/auth.md: the machineUserAttributes comment now reads "Required when you don't define userProfile (mutually exclusive with userProfile)" instead of "Optional ...".
  • packages/sdk/docs/services/staticwebsite.md: the "OAuth2 Redirect URIs" example now includes the machineUserAttributes + machineUsers shape inline so the snippet compiles standalone. A short prose note explains the userProfile vs machineUserAttributes exclusivity and that the same website.url placeholder is reused across cors and redirectURIs.

No source code or type changes. The types already encoded the contract; the examples now reinforce what the types say.

Notes

Companion to PR #1167 (hook value nullability), which fixed the equivalent issue for db.type().hooks(). The llm-challenge benchmark surfaces these docs friction signals via the types-only vs full-package profile diff; this PR is the second SDK docs improvement validated by that harness.

Benchmark rebuild itself lives in PR #1148.

The static website OAuth2 example and the auth.md inline comment both
implied `machineUserAttributes` is optional. The SDK types make it
required whenever you use `machineUsers` without a `userProfile`, so
AI-authored configs that try `machineUsers: { runner: {} }` fail
typecheck. Show the machine-user-only shape inline in the static
website "OAuth2 Redirect URIs" example and fix the misleading auth.md
"Optional" comment.
@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented May 13, 2026

⚠️ No Changeset found

Latest commit: 85a6d30

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 13, 2026

⚡ pkg.pr.new

@tailor-platform/sdk

pnpm add https://pkg.pr.new/@tailor-platform/sdk@85a6d30
pnpm dlx https://pkg.pr.new/@tailor-platform/sdk@85a6d30 --help

@tailor-platform/create-sdk

pnpm add https://pkg.pr.new/@tailor-platform/create-sdk@85a6d30
pnpm dlx https://pkg.pr.new/@tailor-platform/create-sdk@85a6d30 my-app

commit: 85a6d30

@github-actions
Copy link
Copy Markdown

Code Metrics Report (packages/sdk)

main (ebfd66c) #1168 (9e82c9c) +/-
Coverage 61.8% 61.8% 0.0%
Code to Test Ratio 1:0.4 1:0.4 0.0
Details
  |                    | main (ebfd66c) | #1168 (9e82c9c) | +/-  |
  |--------------------|----------------|-----------------|------|
  | Coverage           |          61.8% |           61.8% | 0.0% |
  |   Files            |            362 |             362 |    0 |
  |   Lines            |          12647 |           12647 |    0 |
  |   Covered          |           7827 |            7827 |    0 |
  | Code to Test Ratio |          1:0.4 |           1:0.4 |  0.0 |
  |   Code             |          82684 |           82684 |    0 |
  |   Test             |          34422 |           34422 |    0 |

SDK Configure Bundle Size

main (ebfd66c) #1168 (9e82c9c) +/-
configure-index-size 17.04KB 17.04KB 0KB
dependency-chunks-size 34.49KB 34.49KB 0KB
total-bundle-size 51.53KB 51.53KB 0KB

Runtime Performance

main (ebfd66c) #1168 (9e82c9c) +/-
Generate Median 2,534ms 2,620ms 86ms
Generate Max 2,556ms 2,641ms 85ms
Apply Build Median 2,564ms 2,665ms 101ms
Apply Build Max 2,586ms 2,713ms 127ms

Type Performance (instantiations)

main (ebfd66c) #1168 (9e82c9c) +/-
tailordb-basic 35,130 35,130 0
tailordb-optional 3,841 3,841 0
tailordb-relation 7,428 7,428 0
tailordb-validate 2,566 2,566 0
tailordb-hooks 5,767 5,767 0
tailordb-object 12,136 12,136 0
tailordb-enum 2,462 2,462 0
resolver-basic 9,424 9,424 0
resolver-nested 26,111 26,111 0
resolver-array 18,187 18,187 0
executor-schedule 4,234 4,234 0
executor-webhook 873 873 0
executor-record 8,166 8,166 0
executor-resolver 4,369 4,369 0
executor-operation-function 869 869 0
executor-operation-gql 869 869 0
executor-operation-webhook 888 888 0
executor-operation-workflow 1,714 1,714 0

Reported by octocov

dqn added a commit that referenced this pull request May 14, 2026
…(Phase 5a)

First real AI solve cycle (Phase 4) falsified the LLM-as-judge ambition:
judge fired on 1/25 problems, output was a false signal, both true SDK
improvements (PRs #1167, #1168) came from analyze --diff + iter variance
instead. Phase 5a retires the unused machinery; profile-diff + iter
variance become the primary signal in subsequent phases.

# What's removed

- core/judge.ts (486 lines) + core/judge.test.ts (446 lines)
- runJudgePostProcessing / computeJudgeDiff / ImprovementCandidate in cli.ts
- ProblemResult.judge, Analytics.affordanceDistribution / improvementCandidatesPath
- DiffReport.affordanceDelta + showAffordanceDelta in analyze.ts
- "Top affordances" line in formatReportTable
- LLM-as-judge / Affordance Taxonomy 12-label sections in README + SKILL.md
- judge / improvement-candidates references in skill docs

# Renamed

- meta.json: hypothesizedAffordance -> designNote (37 problems)
- ProblemMeta type: same

# Verification

- 291 tests pass (-31 from judge.test.ts retirement, as expected)
- challenge:verify-solution: 37/37 PASS
- typecheck / lint / format clean
- core/ production: 7,370 -> 6,306 lines (-1,064)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant