Fields on referenced documents not showing

[mckelveygreg]

I'm trying to access fields on an array of referenced documents, but only the baked in reference properties show, not any of my fields.

Is there a proper way to consume a referenced document?

Here is a snippet and codesandbox link:

Code snippet

```ts import { s } from "sanity-typed-schema-builder";

const testReference = s.document({
name: "testReference",
title: "Test Reference",
fields: [
{
name: "title",
title: "Title",
type: s.string(),
},
{
name: "description",
title: "Description",
type: s.text(),
}
]
});

export const testDoc = s.document({
name: "testDoc",
fields: [
{
name: "testArray",
title: "test Array",
type: s.array({
of: [
s.reference({
to: [testReference]
})
]
})
}
]
});

export type TestType = s.infer;

const test = {} as TestType;

// no Title found :(
test.testArray[0].title;

<details>

[saiichihashimoto]

Resolving references is up to you, so s.infer returns the document as you would receive it from sanity, ie with references remaining references. However, if you wanted the type where all references are resolved, try s.resolved instead. Just know that sanity isn't necessarily going to do that unless your groq does that.

[mckelveygreg]

Hmm, so s.resolved will resolve a slug to a string instead of keeping it a Slug, so that kind of messes with our queries.

I'd like to be able to pass in type arguments for the referenced fields, but I'm not seeing a place to do that?

[saiichihashimoto]

Got it, you'll want to mess with the zod property. All of the inference has an s.input<...> or s.infer<...> (which reflects exactly the type sanity returns), s.output<...> or s.parsed<...> (which reflects the type that is returned after it's schema.parse), and the s.resolved<...> (which is mostly to reflect types when generating mocks).

Ultimately, the types we're returning are the type of the document as it is in the confi (parsed or unparsed), but not to reflect queries (which might join, project, filter, or basically anything it feels like). Until we solve generating types from a query (which is a huge problem and not on the immediate roadmap), we can't reflect anything else, so you'll have to join the types somehow, ie QueriedType = Omit<A, 'ref'> & { ref: B }.

[miklschmidt]

One way of doing that is a query builder (which admittedly has drawbacks in edge cases, depending on how complete it is). An example of this is: https://github.com/danielroe/sanity-typed-queries

I'm currently struggling quite a lot with getting the queries to output the expected structure, it gets messy really quickly when dealing with references, "slug" being an example requiring a custom selector instead of just field->. Another is optional references which return null, those have to be wrapped in defined(field) => {field}. Combine this and you get

defined(field) => {field->{..., 'slug': slug.current}}

instead of what could have been

field->

[miklschmidt]

I think this test case is flawed:

sanity-typed-schema-builder/src/reference/index.spec.ts

Line 85 in af68d4d

it("resolves into a document mock", () => {

resolve() doesn't actually expect resolved references, it expects _type and _ref, shouldn't it expect the resolved document?

sanity-typed-schema-builder/src/reference/index.spec.ts

Line 100 in af68d4d

const docMock = docType.resolve(docType.mock(faker));

results in

      {
        foo: false,
        _createdAt: 2021-12-11T11:22:56.021Z,
        _id: '16742cb7-3920-4592-9396-fea7596eb10f',
        _rev: '_]_04B,f>YB?V$I[|z*]4\\0',
        _type: 'foo',
        _updatedAt: 2022-06-04T22:57:13.598Z
      }

ie. not an actual document. To clarify, type.resolve() results in the resolved type but unresolved data, and it throws an error if you pass it actually resolved data.

[saiichihashimoto]

.resolve expects a mocked document, not a real document. It's not going to join docs, query referenced docs, etc. It's purely so mocked documents can reference another mocked doc.

[miklschmidt]

.resolve expects a mocked document, not a real document. It's not going to join docs, query referenced docs, etc. It's purely so mocked documents can reference another mocked doc.

I don't expect it to join docs or query data, i expect it to accept resolved data because it returns a resolved type. It doesn't. Currently there's no way to work with resolved data with zod parsing. Is that intended?

[miklschmidt]

Update: As i read this back i realized i wrote a bunch of nonsense. I think my issue boils down to the hack i made to work around circular type references. I have a "pageReference" with the same name as my "page" to allow referencing a page from somewhere deep within a page. (Edit: it doesn't, .resolve only resolves mocks by default, you have to transform the result yourself, see my next post)

ZodError: [
  {
    "code": "invalid_type",
    "expected": "string",
    "received": "undefined",
    "path": [
      "pageBuilder",
      0,
      "buttons",
      0,
      "href",
      "internal",
      "_ref"
    ],
    "message": "Required"
  },
  {
    "code": "invalid_literal",
    "expected": "reference",
    "path": [
      "pageBuilder",
      0,
      "buttons",
      0,
      "href",
      "internal",
      "_type"
    ],
    "message": "Invalid literal value, expected \"reference\""
  }
]

internal is the resolved document in my data (it's a resolved page reference), and it complains it's missing _ref and _type 'reference'. How does one handle that? The returned type indicates that the pageBuilder[0].buttons[0].href.internal is resolved, but the zod tests for an unresolved reference.

Is "resolve" supposed to replace the reference with data from the referenced type? Ie. i need to call resolve on the referenced type with the actual document that is referenced before .resolve works on the type with the reference?

If i'm not mistaken, that would require me to load all pages and resolve those with my "pageReference" type (still has name: "page") before i can call .resolve on my page? Or am i still confused? :D

Is it possible to get a zod that validates a completely resolved document, ie. where the input data already matches s.resolved<...>? That's really what i'm looking for i guess. zodResolved is not it.

[miklschmidt]

Basically if zodResolved was of type z.ZodType<s.resolved<[union of passed references]>> and returned a zod combining the zodResolved of all passed references with or, it would behave as i'd expect it to. I can't think of a practical usecase for type.resolve outside of testing the library itself? Unless you load all data into memory, but that seems like a bad idea.

[miklschmidt]

Okay.. Sorry for all my rambling here, i'm learning as i type.

For anyone stumbling on this issue, i found a way to deal with it.

For slugs, do this:

{
  type: s.slug({
    zodResolved: zod => zod.transform(slug => slug),
    zod: zod => zod.transform(slug => slug),
  })
}

That will make sure the types match (slug now has _type and current properties).

To resolve references, i have this workaround, it's not ideal (keeping all pages in memory, even if they aren't needed, and old pages won't be removed currently), but until we get a way to parse pre-resolved data from a query, it's the only way i found to make references work:

// schemas/page.ts

export const pageRefs: { [id: string]: s.output<typeof pageReference> } = {};

export const loadAllPageReferences = async (client: SanityClient) => {
	const query = `*[_type == "page"]`;
	const pages = await client.fetch<Array<s.output<typeof pageReference>>>(query);
	pages.forEach(p => {
	        // validate the page
		pageReference.resolve(p);
                // save it for resolving references later.
		pageRefs[p._id] = p;
	});
};

// some schema where you reference a page
{
	type: s.reference({
	to: [pageReference],
		zodResolved: zod => zod.transform(({ _ref }) => pageRefs[_ref]),
	}),
}

Now if you remember to call loadAllPageReferences(client) before calling page.resolve(data), it will resolve the references for you. Do not resolve the reference in your query, it won't parse.

There's probably a better way to construct the query for loading the page references so that it only loads the pages that are referenced somewhere.

[saiichihashimoto]

tbh I'm losing track of what the issues you're having are and how to help. The thing I'm more concerned about is that I think that the documentation isn't clear what it means with s.resolve or how you're supposed to handle docs/types after you make a query that does expand references.

[miklschmidt]

I don't blame you, i apologize for the rambling, i've been working 24/7 trying to meet a deadline and i was partly frustrated, partly confused. I think you're right it's mostly a question of documentation. Primarily how to handle queries with expanded references as you said. Please use fairly complicated doc structures too, more than one level deep - the sanity-typed-queries builder doesn't seem to allow you to resolve deep references yet, so that would be another reason to use this library.

[saiichihashimoto]

Yeah so just to pull it all together:

• s.infer/s.input/s.value will give the document type, no projecting, expanding of references, etc.
• s.output/s.parsed is the type after running schema.parse(doc) on it. Mostly convenience, ie turning dates into date types and flattening the slug object structure. Still, nothing about references.
• schema.mock(faker) will generate something of the s.infer type
• schema.resolve(mock) will return something of type s.resolved, which is only truly relevant for mocks. Only in a mocked document do I feel comfortable having this library actually replace the references with docs that fit, since we will want those to reference other docs.

In your application (mostly in your queries), you'll expand references, make queries that do weird projections, all things that I can't really build for without truly taking on the "let's type your queries" problem as well, which isn't the current goal (although it's definitely something I've wanted to solve). schema.resolve and s.resolve might be the wrong names: it might be closer to schema.resolveMockReferences or something. My issue is that I wanted a type for:

1. parsed types
2. resolved types
3. parsed & resolved types

Currently, the types are for 1 and 3, although s.resolved wrongly implies 2. I'd like to fix that, but also consider a better overall solution.

If we solved this problem for the queries, ie make a typed query builder, we can make all of this irrelevant, since we'll determine the types you need by implication. I have no idea how parsing actually survives that (the query builder would also have to pass through a query.parse, which gets hairy). But it's doable (just a lot of work). I think step one is the breaking change to rename the current resolve methods/types to be explicit about it being about mocks, and then actually providing meaningful resolved types.

[saiichihashimoto]

I made an issue to follow my train of thought off of these learnings. I think there's some immediate helpful tasks that are easy (renaming some of this) and a much larger hidden task (typing queries) that's under this, which I don't have the bandwidth to take on.

[miklschmidt]

and a much larger hidden task (typing queries) that's under this, which I don't have the bandwidth to take on.

Yeah, that one is a lot of work! I wish some of the modern CMS's had this out of the box (cough sanity cough). There are a few that advertises typescript, but if your data isn't typed you lose most of the benefit and it's barely more than an inline documentation tool. Sanity v3 doesn't seem to do this either.

[saiichihashimoto]

From what I understand, sanity started (and grew! a lot!) before typescript became the giant de-facto thing it currently is. To some degree, they need to play catchup. This library (and sanity-codegen) are here to help!

[saiichihashimoto]

I brushed over the names fast and it just dawned on me that @mckelveygreg and @miklschmidt have not been the same person this whole time.

[miklschmidt]

From what I understand, sanity started (and grew! a lot!) before typescript became the giant de-facto thing it currently is. To some degree, they need to play catchup. This library (and sanity-codegen) are here to help!

Apropos of sanity-codegen, i see they are working on typing groq query results: https://sanity-codegen-dev.vercel.app/
Maybe that could be of some use in the future!