Skip to content

Proposal: make derived store APIs type-safe and ergonomic - #3194

Draft
GabbeV wants to merge 1 commit into
solidjs:nextfrom
GabbeV:fix/store-projection-types
Draft

Proposal: make derived store APIs type-safe and ergonomic#3194
GabbeV wants to merge 1 commit into
solidjs:nextfrom
GabbeV:fix/store-projection-types

Conversation

@GabbeV

@GabbeV GabbeV commented Sep 1, 2026

Copy link
Copy Markdown

Status

Important

Both the implementation and this PR description were written with substantial AI assistance. They should be treated as an exploratory API draft that requires careful human review, not as ready-to-merge work.

The independently extracted fixes in #3223 and #3224 have now merged. This branch is rebased directly onto the latest next; those fixes are intentionally not described below as part of this proposal.

Summary

This explores a stricter and more ergonomic contract for the store primitives around two related problems:

  1. Store<T> = Readonly<T> means passing an existing store into another store primitive can infer the new T as the already-readonly store type. This makes setter drafts unexpectedly readonly and makes mapped helpers, intersections, and repeated store wrapping difficult to model without increasingly complex type machinery.
  2. Derived stores currently accept seed: Partial<T> while exposing both the callback draft and resulting store as complete T. A partial seed does not satisfy that contract unless user code happens to initialize every required property before it can be observed.

Proposed API

1. Keep Store<T> as a documentation alias for T

export type Store<T> = T;

This removes the synthetic top-level Readonly<T> transformation. Reusing stores therefore preserves normal inference without a private source brand:

const [first] = createStore<{ id: number }[]>([]);
const [second, setSecond] = createStore(first);

setSecond(draft => {
  draft.push({ id: 1 });
});

It also lets omit, merge, built-in mapped types, intersections, and third-party helpers operate on the structural value without carrying stale store identity metadata.

This is intentionally less strict: Store<T> no longer adds readonly properties that were not present in T. User-declared readonly fields remain readonly because the original T is preserved.

2. A provided seed must be a complete T

The seeded form receives a real T draft and may mutate it, return a replacement, or return void:

createProjection<State>(
  draft => {
    draft.count++;
  },
  { count: 0, label: "ready" }
);

The seed is T, not Partial<T>, because it is immediately exposed to the callback as T. The same rule applies to createStore(fn, seed), createProjection(fn, seed), and createOptimisticStore(fn, seed).

Arrays and tuples use this seeded form. An empty array is normally a natural complete seed for an array projection:

createProjection(() => [{ id: 1 }], []);

seedLoadingValue: true is also limited to the seeded form because only a complete seed can honestly be exposed as commit zero.

The old mutation-based empty-object initialization remains available as an explicit escape hatch:

createProjection<State>(
  draft => {
    draft.count = source();
    draft.label = "ready";
  },
  {} as any
);

The cast is required because TypeScript and the runtime cannot prove that every required field is established before observation. Once the caller casts, they take responsibility for that invariant.

3. An omitted or nullish seed selects a return-only initializer

For non-array object roots, a seed can be omitted:

createProjection(() => ({ count: 0, label: "ready" }));
createProjection(async () => await loadState());

createProjection<State>(async function* () {
  yield await loadInitialState();
  yield await loadUpdatedState();
});

The seedless callback receives no draft and must return, resolve, or yield a complete T every time:

() => T | Promise<T> | AsyncIterable<T>

This makes the first phase honest without passing a value typed as T before any T exists. Runtime validation rejects void, null, primitives, and arrays if the type system is bypassed. Array projections require a real seed.

A nullish placeholder is only needed to reach the options argument:

createProjection(derive, undefined, { shallow: true });

The writable seedless forms keep their public setter unavailable until the first complete value has initialized the store. Hydration may carry a private replay backing, but that implementation detail does not turn the public seedless form into a seeded callback or make its setter available early.

This PR currently implements the return-only option suggested in review. The broader question of whether seedless projections should instead expose an explicit second mutation phase, or use another shape entirely, is discussed in this comment.

4. Apply the same overload model to each store primitive

  • createStore(value, options?) and createOptimisticStore(value, options?) create plain stores.
  • createStore(fn, seed?), createProjection(fn, seed?), and createOptimisticStore(fn, seed?) share the complete-seed versus return-only-seedless distinction.
  • Seeded callbacks receive draft: T; seedless callbacks receive no arguments.
  • Callable root values are rejected because the first argument would be ambiguous with a derive callback.
  • Projection forms remain refreshable, and writable forms continue returning their paired setter.

Runtime implementation

The same seeded/seedless distinction is carried through the client, server, hydration, and optimistic implementations:

  • seeded derives receive their draft and may return void
  • seedless derives receive no draft and must produce complete non-array objects
  • seedless writable stores reject setter calls until their first value commits
  • hydration preserves the public seedless phase even when replay requires a private backing object
  • async initializer validation and setter failures are routed through the existing reactive error path

Feedback requested

  1. Is dropping the synthetic Readonly<T> transformation from Store<T> an acceptable tradeoff for straightforward inference and composition?
  2. Is “complete T seed or return-only seedless initializer” the right overload boundary?
  3. Is requiring seeds for arrays and tuples reasonable?
  4. Should seedless projections remain return-only, or is a separate post-initialization mutation phase worth designing?
  5. Is keeping partial mutation-based initialization behind an explicit cast the right escape hatch?

How did you test this change?

From packages/signals:

  • focused store, projection, optimistic, loading-boundary, patch-channel, and transition-isolation tests
  • strict compilation of tests/store/store.type-tests.ts
  • pnpm types

From packages/solid:

  • client hydration and server async test suites
  • pnpm test-types
  • pnpm types

@changeset-bot

changeset-bot Bot commented Sep 1, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 8f6f99d

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 11 packages
Name Type
@solidjs/signals Patch
solid-js Patch
test-integration Patch
@solidjs/web Patch
@solidjs/element Patch
@solidjs/h Patch
@solidjs/html Patch
@solidjs/universal Patch
@solidjs/babel-plugin Patch
@solidjs/compiler Patch
@solidjs/diagnostics Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@GabbeV

GabbeV commented Sep 1, 2026

Copy link
Copy Markdown
Author

@ryansolid This PR is not specifically intended to implement anything from @brenelz proposal. I think the ergonomics of such recursive mapping might need more investigation. There is probably some overlap in what they solve but they are mostly about different things. The goal of this PR is to improve the ergonomics and type safety for the primitives them selves and doesn't concern it self with later consumption of the store in the app like brenelz issue.

I think the runtime guards especially might need your consideration as that goes beyond a simple type change. The AI also claimed it found and fixed SSR bugs along the way that allowed access to the seed value. I have little clue if these fixes are correct.

@ryansolid ryansolid left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the clarification. I initially conflated this with #3092 because both use a marker around Store<T>; that was the wrong frame, so I removed that comment and reviewed this proposal independently as a derived-store seed/initialization contract.

There are valuable pieces here, but I don't think the current combined design is safe to continue as one implementation. My recommendation is to split it before further work.

1. Type/API blockers

The source marker does not mean “exactly this Store.” Mapped transformations preserve it, so it contaminates transformed values:

const [source] = createStore<State>(initial);
createStore(omit(source, "mutable")); // no overload matches

const extended = merge(source, { extra: true });
const [, setExtended] = createStore(extended);
setExtended(draft => { draft.extra = false }); // extra is lost; T collapses to State

The marker is useful for exact source recovery, but omit, merge, built-in mapped types, and intersections need to strip or recompute it rather than carrying a stale source identity.

The unseeded callback type is dishonest. It remains (draft: T) => void | T | ..., while the runtime intentionally makes every first-run draft operation throw. These guaranteed-invalid programs compile:

createProjection<State>(() => {}); // can never initialize
createProjection<State>(draft => ({ count: draft.count })); // first read always throws

A value that cannot be observed as T should not be passed as T. Runtime validation cannot repair that static contract.

Seeded overloads permit callable roots because NoFn<T> is missing from the seeded forms. The draft proxy is not callable, so accepted code fails at runtime. This also contradicts the PR description.

Requiring a complete seed rather than Partial<T> is otherwise directionally correct: if the callback receives draft: T, the seed must establish a real T.

2. Client runtime blockers

The gated draft membrane can be bypassed through exported $TARGET. Internal setter machinery needs that symbol before initialization, but user code can read it too, return the internal target, pass object validation, and expose internals. The proxy also cannot hide typeof, Array.isArray, or identity. “A stable T draft that rejects every observable operation” is not implementable with this shape.

Initialization is tracked outside the scheduler. The closure-local initialized flag opens immediately after write/wrapCommit returns, even when truth is only staged in _pendingValue/store pending backing and still transaction-held or masked. That can enable the public setter/draft before the first value is observable. Core commit state must be the authority; a second state machine will drift.

The optimistic first-landing path conflicts with current next. The PR predates #3146. Its runAuthoritative(write) initialization branch bypasses the now-required flight-owned transaction path (declareFlight / enterFlightTransition), risking premature reveal and breaking fold/until() composition. This cannot be resolved mechanically during rebase.

The gates also run for every already-seeded projection, adding closure/branch overhead and changing reflection behavior where no initialization protection is needed.

3. Server/hydration blockers

There are real baseline SSR bugs here worth extracting: pending server proxies expose seeds through reflection/symbol operations, and rejected projections can become readable as their seed because error handling marks them ready. Those are valid hardening targets.

The proposed membrane is incomplete, though. It omits fundamental proxy traps (getPrototypeOf, setPrototypeOf, isExtensible, preventExtensions). Object.preventExtensions() can succeed while pending and later make Reflect.ownKeys() permanently throw a proxy-invariant TypeError.

Server first-value adoption also diverges from the client: Object.keys/Object.assign leave omitted symbol keys behind and erase class prototypes/methods. Either unseeded values must be restricted to plain records, or adoption must preserve the supported object contract.

Changing client-only hydration from “serve the complete seed” to “hide it and suspend unless seedLoadingValue” is a new public semantic, not an SSR bug fix. It needs an explicit ruling and separate release treatment.

There is also an existing synchronous hydration replay path where processing failures can escape while its promise remains pending. Useful to fix, but separate from this API proposal.

Recommended decomposition

  1. Source recovery + complete seeded forms (types first). Fix marker contamination, exclude callable roots consistently, and add merge/omit/mapped-type/intersection probes. Keep complete T seeds; don't restore Partial<T> under a (draft: T) contract.
  2. Redesign the unseeded API without a fake draft. The honest first phase is return-only: () => T | Promise<T> | AsyncIterable<T>, with no initial void. If post-initialization draft mutation is required, expose that as an explicit second phase/updater contract rather than pretending a gated object is T.
  3. Extract the confirmed SSR hardening separately. Complete all fundamental proxy traps/invariants, preserve the supported object shape, and add reflection/symbol/class/error-stream tests.
  4. Only then design hydration/optimistic integration on current next. It must use scheduler commit state and the flight-owned transaction model, not parallel initialization flags.

So: the complete-seed and source-recovery goals are worth pursuing, and parts of the SSR investigation found genuine bugs. The seedless runtime/state-machine portion should be replaced rather than patched incrementally. Given the stale base and current semantic conflicts, I would not rebase this branch wholesale; split the proposal into independently reviewable pieces first.

@GabbeV

GabbeV commented Sep 2, 2026

Copy link
Copy Markdown
Author

I find my self a bit confused about your response but this is how I parse it:

  • The brand currently causes issues with omit, merge. I'll see what I can do about that.
  • Missing NoFn<T> will be added.
  • You agree with my finding that seed: Partial<T> is unsound as it currently is in the code base. Maybe a separate PR reverting seed to T could land first but would in my opinion be an unfortunate end state.

My goal with this draft PR is to do a API design exploration for how to have a overload where you don't need a full seed but that is also type safe given I found the seed: Partial<T> solution fails that.

One finding in this exploration is that arrays mostly don't have a problem with providing seed as an empty array is almost always fine and easy to provide. Tuples being the exception.

Avoiding arrays by saying they always need a seed opens up additional design space where we can let omitted or nullish seed mean uninitialized object in a way that is known both at compile time and runtime easily.

Given these constraints I see a couple of different options that keep the API type safe:

Option 1: Optional draft (draft: null | T) => void | T | Promise<void | T> | AsyncIterator<void | T>

This works well for functions projections. You get a draft when it is initialized and then the only option you have is to return a T to initialize it. However for async iterators stays alive so would need to get the initialized draft later somehow. Given it is a generator const draft = yield { /* initial object */ }; could be an option but is also a bit exotic.

Options 2: Throwing draft (draft: T) => void | T | Promise<void | T> | AsyncIterator<void | T>

This is the one I chose where trying to access the draft in any way before it you have yielded/returned a real T initializing the store throws. I expect this to be the kind of runtime error you immediately hit first time you run your program explaining what you did wrong. While not as strong as a type check it seems like in the spirit of other "you are using the api wrong" assertions where it is unlikely you don't discover the issue before it is in production or something like this. Linting could potentially raise the issue earlier if desired.

Option 3: seedless doesn't allow mutation () => T | Promise<T> | AsyncIterator<T>

Unfortunate limitation but doesn't need any runtime errors or exotic yielded drafts etc. The limitation can be worked around by providing a seed with a type cast if necessary.

Option 4: always require a seed: T

Avoiding this is the whole motivation of the exploration in the first place.

Other options?

Your comment mentions phases but doesn't really specify what is meant. Are you suggesting that you are open to larger changes here with multiple callbacks or things like that? My attempt here has been to stay close to the API as it already is.

Where we go from here

This draft is primarily intended to work out the public API. Given your much deeper context on the internals, I would appreciate your help in the final implementation of whatever we land on here. I'll work on any API level fixes and push that up but much rather leave the internals to you given I don't feel that I know what I'm doing there.

@GabbeV

GabbeV commented Sep 2, 2026

Copy link
Copy Markdown
Author

This might be controversial but my honest thoughts after having tried to solve the merge/omit issue is that Store making things readonly isn't worth the downstream issues it causes in types. This PR tried to solve it in the simple case with the brand allowing me to extract the original T however that caused issues with merge/omit and still didn't solve other issues like how a store remains readonly when stored as part of another store. I choose to make Store a transparent noop for now and removed the brand as it no longer solves anything this PR is trying to solve.

@GabbeV
GabbeV force-pushed the fix/store-projection-types branch 3 times, most recently from 43da82a to 74a4138 Compare September 2, 2026 12:28
@GabbeV

GabbeV commented Sep 2, 2026

Copy link
Copy Markdown
Author

I’ve split the seeded client-store hydration behavior out into #3223 and removed it from this branch so it can be reviewed independently.

@GabbeV

GabbeV commented Sep 2, 2026

Copy link
Copy Markdown
Author

I’ve split the server projection rejection-state fix into #3224. That PR keeps the existing proxy target and reflection/symbol behavior and only ensures rejected pending projections continue to expose their original error.

@GabbeV
GabbeV force-pushed the fix/store-projection-types branch from 0fb336b to 8f6f99d Compare September 2, 2026 17:37
@codspeed-hq

codspeed-hq Bot commented Sep 2, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 54.89%

❌ 1 regressed benchmark
✅ 135 untouched benchmarks
⏩ 132 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
merge 75.6 µs 167.7 µs -54.89%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing GabbeV:fix/store-projection-types (8f6f99d) with next (e59a8a9)

Open in CodSpeed

Footnotes

  1. 132 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@GabbeV

GabbeV commented Sep 2, 2026

Copy link
Copy Markdown
Author

One issue I missed in the current implementation is that seedless hydration replay needs a mutation phase that the proposed API currently has no way to represent.

The first replay entry is a complete snapshot and can be adopted directly as the store's backing. Later entries are serialized patches, though, so replay needs a draft to apply them to. It effectively needs two phases:

// first value
() => T

// later replay entries
(draft: T) => void

The public seedless overload is return-only, while the projection runner currently uses one seeded distinction both to decide whether the callback receives a draft and whether void is valid. Hydration needs to provide a draft internally for patch replay, while still requiring the first result to be a complete T. Passing the private {} backing as a seed makes those states difficult to distinguish, so the current implementation cannot model seedless hydration replay with an honest initialization contract.

How this should be fixed depends on which direction we choose for the seedless API: it could remain return-only with a separate internal replay phase, or we could design an explicit post-initialization mutation phase. I think we should defer implementing this until that API direction is decided rather than add another provisional initialization state machine here. I'm leaving this comment so the current limitation is explicit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants