-
-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refine validations on object definitions don't get triggered until all fields in the object exist. #479
Comments
What I mentioned related to the form is just an example. My point is still that I created a little code sandbox for you to try. Here, this is what I'm doing in the codesandbox.
For this, I expect these validation results to be produced: 1) "name" is required and 2) "age" must be greater than or equal to 0. However, what I see is only "name" is required. |
@ivan-kleshnin I went through this again, but I still think what you mentioned contradicts with my findings. It'd be great if you could let me know how zod solves this issue. |
Let me reiterate the issue in more detail. It may be by design that One solution is to create a nested object schema and use z.object({
name: z.string,
dates: z.object({
startDate: z.date(),
endDate: z.date()
})
.refine((data) => checkValidDates(data.startDate, data.endDate), 'Start date must be earlier than End date.'),
}) Sure, this works, but what if there's another field that depends on start and end dates? In my scenario, the start and end date are the dates of an event. I also have a due date field which specifies when an event registration is due. So, I need to make sure it's earlier than the start date. Now, I need to wait until start and end dates are both filled in before using a refine validation. But, what I want is to trigger the validation as soon as start and due date are filled in. zod cannot currently handle this if I'm correct. Also, doing this breaks the structure, so it doesn't play nice with Blitz.js. My suggestion is to have something like yup.when which gives you access to values of other fields. To enable this, the z.object({
name: z.string,
startDate: z
.date()
.refine((value, getField) => checkValidDates(value, getField("endDate")), 'Start date must be earlier than End date.'),
endDate: z
.date()
.refine((value, getField) => checkValidDates(getField("startDate"), value), 'Start date must be earlier than End date.'),
dueDate: z
.date()
.refine((value, getField) => checkValidDates(value, getField("startDate")), 'Due date must be earlier than Start date.'),
}) This is my suggestion. By moving the refine into the field schema, then we can makes sure it gets triggered as soon as that field is modified. There's also no need to create a nested object schema. If there's an existing solution, then that's great. Please let met know. I hope I made my point clear enough. I'm not sure if this is technically possible with how zod is written, but this feels like such a basic scenario, so it'd be great support this. |
What I mentioned related forms are still just examples. It is not really what I want to discuss. (Well, if you are curious why missing keys in those "forms" do not have Currently, zod doesn't have a good way to do validations against more than one field and that's what I'm trying to point out here. I also think this is a not a corner case. It's not uncommon to have fields that depend on some other fields (e.g. password and password confirmation, start/end dates, etc.), whether it's a form or any other json data, and we usually want to surface the error as soon as we detect them. The suggestion to use I skimmed through the source code a bit today, but isn't it possible to hold a reference to the original data inside the |
My two cents: You could be explicit bout what you are trying to do and make two schemas: one for the form and one for the model. Something like this? TypeScript Playground import { z } from "zod";
const formSchema = z.object({
name: z.string(),
startDate: z.date(),
endDate: z.date(),
}).partial();
type Form = z.infer<typeof formSchema>;
const modelSchema = formSchema.required().refine((input) => {
return input.startDate < input.endDate;
}, "Start date must be earlier than end date");
type Model = z.infer<typeof modelSchema>;
const blankModel: Model = {
name: "",
startDate: new Date(),
endDate: new Date(),
};
const currentForm: Form = {
name: "A name",
};
const result = modelSchema.safeParse({ ...blankModel, ...currentFrom }); |
A version that embraces the partial nature of the form: TypeScript Playground import { z } from "zod";
const formSchema = z.object({
name: z.string(),
startDate: z.date(),
endDate: z.date(),
}).partial().refine((partialInput) => {
if (!partialInput.startDate || !partialInput.endDate) {
return true;
}
return partialInput.startDate < partialInput.endDate;
});
type Form = z.infer<typeof formSchema>;
const currentForm: Form = {
name: "A name",
};
const result = formSchema.safeParse(currentForm); |
Hi, thanks for you two for taking time for this issue. Your example gets the job done, but there's still one issue in the real world scenarios. If there are separate schemas, then I'll need to add the same I don't think there's a better way to handle this? const CreateTournamentBaseSchema = z
.object({
name: z.string(),
organizer: z.string().optional(),
startDate: z.date(),
endDate: z.date(),
dueDateTime: z.date(),
})
.strict()
const CreateTournamentBaseSchemaPartial = CreateTournamentBaseSchema.partial()
// Accept a partial object type as an argument, so the refine methonds doesn't need to assume
// the fields in the object are always defined and it'll cover both cases.
const addRefines = (schema: typeof CreateTournamentBaseSchemaPartial) => {
return schema.refine((data) => {
if (!partialInput.startDate || !partialInput.endDate) {
return true;
}
return partialInput.startDate < partialInput.endDate;
});
};
export const CreateTournamentFormSchema = addRefines(CreateTournamentBaseSchemaPartial)
export const CreateTournamentModelSchema = addRefines(
// I'll need to force cast it to the partial type to make the compiler happy.
CreateTournamentBaseSchema as unknown as typeof CreateTournamentBaseSchemaPartial
) |
Since those two schemas are not compatible with each other, I think you'll either need to be ok with the cast here, or use the fact that the partial schema will match the non-partial schema and define the refinement over the partial data. Something like this: TypeScript Playground import { z } from "zod";
const CreateTournamentBaseSchema = z
.object({
name: z.string(),
organizer: z.string().optional(),
startDate: z.date(),
endDate: z.date(),
dueDateTime: z.date(),
})
.strict()
const CreateTournamentBaseSchemaPartial = CreateTournamentBaseSchema.partial()
// Accept a partial object type as an argument, so the refine methonds doesn't need to assume
// the fields in the object are always defined and it'll cover both cases.
const checkDates = (data: z.infer<typeof CreateTournamentBaseSchemaPartial>) => {
if (!data.startDate || !data.endDate) {
return true;
}
return data.startDate < data.endDate;
}
export const CreateTournamentFormSchema = CreateTournamentBaseSchemaPartial.refine(checkDates)
export const CreateTournamentModelSchema = CreateTournamentBaseSchema.refine(checkDates) |
Yeah, ok. That works out nicely. I did not think of using Though, I still think this should be somehow built-in to zod. I'm happy with this workaround now, but I believe supporting it would make zod even easier to work with. Well, I'll let the community decide... Anyways, thanks a lot for the help. |
Ultimately, I see the value of Somewhat related to this, there is a PR in the works that adds a concept of "pre-processing" (#468) that allows transforming the input before the schema is checked that might help a bit with this, but not much more elegant (imo) than the solution I outlined above I think.
Glad to help! |
I'm currently in the process of testing Zod to migrate from Yup. I love it so far, but the issue is also an irritant for us |
This seems to be similar to #690, and based on @colinhacks response he is working on a change to the parser to support this in 3.10. You can test it by trying the beta release via |
I don't think this is similar to #690, a use case for triggering |
I think if you have multiple subsets of the final data structure that need to be validated together, you might want to consider making these separate schemas and then combining them together into a single schema (if you need that single schema for something else). Conversely you could use |
I think this is the approach I'm looking for since I need the combined schema because I'm using Zod as a resolver on react-hook-forms. |
Didn't do the trick, or at least I didn't find how to make it work. Let's imagine that I have the following two schemas const step1 = z.object({ name: z.string().nonempty('Name is required') });
const step2 = z.object({
company: z.string().nonempty('Company is required'),
position: z.string(),
});
const step3 = z.object({ team: z.string().nonempty('Team is required') });
const allSteps = step1.merge(step2);
const allStepsWithRefine = step1
.merge(
step2.refine(
({ company, position }) => {
if (company) {
return '' !== position;
}
return true;
},
{
message: "String can't be more than 255 characters",
},
),
)
.merge(step3); I'm looking for a way to make
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
This is a non-starter for us to switch from Yup to Zod. The use case that we have might be addressed by #1394 |
Really sad about this behaviour, tbh. Idk if I'm being too shallow in my thoughts, but to me it seems you could just execute the refinements even if the schema validation fails, then this problem would be solved. But ok, let's try to make workarounds for this |
Just come across this 2 years after it was opened, and also find that this is an annoyance for us. We're using If we had someway to reference other data, that would be perfect:
But this is not made available, and so it seems we are doomed to post-validation refine, or splitting our Zod up into pieces |
SolutionUse
Hope this helps someone |
I'm currently facing the same problem as everyone has mentioned already, refine/superRefine function is not getting called or triggered, I was going through the docs to see why it is happening but nth was mentioned there, if thats the expected behaviour then please update the docs, otherwise I do hope this interesting behaviour is actually fixed. |
hi, i did this just to try and it looks fine. import { z } from "zod";
import { validateVal } from "packages/utils/validateVal";
const val = z.string().refine(
(val) => {
return validateVal(val);
},
{ message: "Invalid val" }
);
export const MySchema = z.object({
id: z.string(),
name: z.string(),
val: val,
});
export type MyType = z.infer<typeof MySchema>; i have no idea if its perfect but its working for now. |
are you working this with react-hook-form? because the issue happens when combining these two library. |
Any updates on clean ways to work around this? this thing is driving me crazy, how hard can front-end validation be... Its unthinkable why this would be a problem, since it's the nature of a form to have undefined fields if the're untouched. |
I wish I could've stumbled upon this issue earlier using zod, should've gone the yup way :/ |
you can make all fields optional and at the end of your schema you can use .refine/.superRefine with something like this: superRefine((input, ctx) => {
const requiredInputs = {
reqInput1: input.reqInput1,
reqInput2: input.reqInput2,
};
for (const [key, value] of Object.entries(requiredInputs)) {
if (!value) {
ctx.addIssue({
code: "custom",
message: "Required",
path: [key],
});
}
}
}) |
I solved this problem with just adding this code at the end of the schema definition.
|
First of all, thank you @Yin117 , this worked as expected! :D This is the solution I used, in my case I'm using it to validate password and password confirm. I'm currently using it in my nextjs application. here is what my solution looks like. Note that I'm using it inside a page.tsx component, this is declared just above the
|
Update For those who are still looking as to why this is, the .refine() won't run if it fails to validate the objects before it. The solution of splitting the schema into two solves this because it isolates the validation to only those specific objects, effectively making them pseudo concurrent. Just like a traffic lane, if one lane is in traffic (assuming you have two lanes on the same direction), just switch lanes, in our case, we create one. |
Indeed splitting the schema does solve the problem, but for complex schemas it isn't a scalable solution. I did it in my project but I sincerely want a solution like yup has. Ok, you can say "use yup" but zod has other features and benefits that I like. I had to split my forms into 5, 6 schemas... And if you need to get a value from another form for some reason, it just makes things difficult... It's a solution, I'm using it, but I really don't like it... |
@mosnamarco Ah THANK YOU, I was loosing my mind over this 😬❤️ |
Is there a way to solve late validations on items within an array to run earlier? I don't see how a schema split could help here. Example: https://stackblitz.com/edit/zod-repro-6a9zqs?file=index.ts |
Your issue is different: in your example you've defined two object schemas: const itemASchema = z.object({
name: z.string(),
});
const itemBSchema = z.object({
number: z.number().nullable(),
}); But const itemBSchema = itemASchema.extend({
number: z.number().nullable(),
}); Which does give you the correct error that you seem to be missing. |
Yeah, I sympathize with all of the form users of Zod here. TypeScript (and therefore Zod) is not very ergonomic with forms, and none of the form libraries seem to make the situation much better here. Form handling is actually why JavaScript is such a weird language with type casts, and the way HTML forms in general work by default is just such a loose and dynamic thing. The fundamental issue here is a mismatch with how Zod defines how "pipeline"-like things work. In order to give you a helpful (and truthful) type to those functions, the candidate object has to pass the validation you've defined, otherwise the best we can do is give you a value of const schema = z
.object({ foo: z.string() })
.refine((value) => {
/* what should `value`'s type here be for an input of `null`? */
});
const result = schema.parse(null); If you think import * as y from "yup";
const schema = y.object({
foo: y.string(),
bar: y.number().when("foo", ([foo], schema) => {
return schema.min(foo / 82); // NaN, nice. Oh, but if the string is `"1e1"` you get `0.12195121951219512` which is cool
}),
});
// What does this 👇 even mean?
const num = y.number().when("boop", ([boop], schema) => {
return schema.max(boop * 82); // lolwut?
}); So, yeah, using Zod with forms comes with a lot of work and overhead and friction. IMO that needs to be solved from the side of the form integration libraries to give HTML forms a more sane API. Zod itself is unopinionated about what the source of the data is, but is very opinionated (like TypeScript) about the truth of it's assertions. Sorry there isn't a better story here yet, but I expect that to come from the other side of the fence here. FWIW, URLSearchParams are also a pretty unwieldy creature, but it's just not quite as common as HTML forms. |
@scotttrinh I don't think this is a working solution. The updated example using I don't think is a form-related problem. When I validate API requests in a server-to-server communication I'd like to have the same behaviour: I'd like to see as many validation errors as possible. Would it be possible to introduce some kind of "select" API instead of splitting schemas (which doesn't seem to be always possible). Could an API like this theoretically work (using #479 (comment) as an example)?: const ourSchema = z
.object({
someProperty: z.string().min(1),
startDate: z.date({ required_error: 'startDateRequired' }),
endDate: z.date({ required_error: 'endDateRequired' }),
})
.refine((data) => data.endDate > data.startDate, {
message: 'datesReversed',
path: ['endDate'],
// if selectPaths is defined the only those paths have to pass the validation
// in order to execute the refine callback. "someProperty" will be ignored.
selectPaths: [['startDate'], ['endDate']],
}); |
We need |
I also encountered a problem when switching from yup to zod, when I couldn’t normally “turn on/off” fields depending on the value of one field. |
I came up with this helper that treats the data as if its a zod any but then returns the type that was passed into it. Not sure if theres any differences between z.any and z.*, but all I need from this is refine before using the product as a child validation unit or for final validation. My use-case for this is validating forms with react hook forms, where the validator gets run on field change and the data partially fills the schema. export function zodAlwaysRefine<T extends z.ZodTypeAny>(zodType: T) {
return z.any().superRefine(async (value, ctx) => {
const res = await zodType.safeParseAsync(value);
if (res.success === false)
for (const issue of res.error.issues) {
ctx.addIssue(issue);
}
}) as unknown as T;
}
// example:
zodAlwaysRefine(
z.array(
z.object({
key: z.string(),
name: z.string(),
}),
),
).superRefine((value, ctx) => {
// this should get run even if any `name`s are missing, and the zod type is untouched for further chaining
// even though certain properties could be undefined (so its technically not typesafe, but hopefully logic in the refine is small)
//
// ex. *find duplicates and add errors*
// ...
}); However, it would be nice to have some sort of flag that you can pass into any zod unit that validates regardless of the intrinsic validation result. |
I came up with something. I'm not sure our favorite developer will like it, but it works like a charm. const form = useForm({
resolver: (values, context, options) => zodResolver(yourSchema(values))(values, context, options)
}) then, in schema: const yourSchema = (values: FormValuesTypes) => then you just need to destruct object based on `values` data like `...(values.field === 1 ? {schema one} : {schema two})`. Thus, by destructuring the schema object, you can simply turn off the necessary fields depending on the data from |
can you like give some detailed example bro, im trying to use inside useForm also plus we need zod solution... |
This was amazing @Mjtlittle , thanks! |
this is working. thanks @Mjtlittle |
this just saved me hours of manual reverse-engineering of automatically-generated zod objects. thank you so much @Mjtlittle |
This works for me nicely, thanks |
Thanks for this helper @Mjtlittle! @colinhacks this solution would be amazing as a flag option as requested here |
Over a year since my last input on this and we still don't have a good solution, but I'm back with a new approach, I'll preface this by saying I equally hate this approach and I'm not yet sure how it'll impact React Hook Form. My goal was to find a solution that gives me the "everything gets validated" that I wanted to reach without splitting out the schema. The concept is:
Like I said, I don't like it, but it gives a workable result that is still type-strong, and is decently readable. type ZodPath = (string | number)[] | undefined;
function customRequired(ctx: z.RefinementCtx, value: unknown, path: ZodPath, message: string) {
if (!value && value !== 0 && value !== false) {
ctx.addIssue({
code: z.ZodIssueCode.custom,
message,
path,
})
}
}
export const theZodSchema = z.object({
iAmRequired: z.string().optional(),
startDate : z.date().optional(),
variableABoolean : z.coerce.boolean().optional(),
variableBString: z.string().optional(),
})
.superRefine((data, ctx) => {
customRequired(
ctx,
data.iAmRequired,
['iAmRequired'],
'This is required',
);
if (!data.startDate || data.startDate < new Date()) {
ctx.addIssue({
code: z.ZodIssueCode.custom,
message: 'Start date must be in the future',
path: ['startDate'],
})
}
if (data.variableABoolean && !data.variableBString) {
ctx.addIssue({
code: z.ZodIssueCode.custom,
message: 'If Variable A is true, then Variable B must be specficied',
path: ['variableB'],
})
}
})
; Resulting type: export const theZodSchema:
z.ZodEffects<
z.ZodObject<{
iAmRequired: z.ZodOptional<z.ZodString>,
startDate: z.ZodOptional<z.ZodDate>,
variableABoolean: z.ZodOptional<z.ZodBoolean>,
variableBString: z.ZodOptional<z.ZodString>
},
"strip",
z.ZodTypeAny,
{},
{}
>, // ZodObject
{},
{}
> // ZodEffects This is an imperfect solution for a problem without one, so take it as you like; feedback and extension welcome. |
Building on top of @Mjtlittle 's solution, I added some code to make sure that any transformations that the original schema declares are also applied when the validation succeeds: import { z } from 'zod';
/**
* Helper function to make `refine` and `superRefine` always get called
* even if schema validation fails.
*/
export default function zodAlwaysRefine<ZodSchema extends z.ZodTypeAny>(
schema: ZodSchema,
) {
type Value = any;
type TransformedValue = z.infer<ZodSchema>;
// using this transformCache because I couldn't find a way to only run the transform without running parse/parseAsync
// Also it's probably a good idea to not re-run validations/transformations if we can avoid it
const transformCache: WeakMap<Value, TransformedValue> = new WeakMap();
return z
.any()
.superRefine(async (value, ctx) => {
const res = await schema.safeParseAsync(value);
if (res.success) {
transformCache.set(value, res.data);
} else {
for (const issue of res.error.issues) {
ctx.addIssue(issue);
}
}
})
.transform((value) => {
if (transformCache.has(value)) {
const cached = transformCache.get(value);
transformCache.delete(value);
return cached;
}
// we can use parseAsync because zod only runs `transform` when the value is valid
return schema.parseAsync(value);
}) as unknown as ZodSchema;
} I ran some tests and the object reference used for the The If someone else comes up with another more elegant solution please post it! But in the meantime this at least gets the job done. |
Isn't it a bug that causes the issue that is discussed here? const obj = z
.object({
foo: z.string().min(1),
})
.superRefine((val, ctx) => {
ctx.addIssue({
message: 'Error from superRefine',
});
}); Why does the
but
Shouldn't the custom error from |
Applies to zod v3. I have not tried the same scenario on older versions.
I have two dates,
startDate
andendDate
and I want to verify that the former is always earlier than the latter. I tried doing something like this:The problem is that the
.refine()
function defined on theobject
type doesn't get triggered until the all fields in the object are defined. So, imagine you have a form and user entered bothstartDate
andendDate
first, but in a wrong order. At this point, the validations for these dates do not trigger because user has not enteredname
. Then, user goes ahead entersname
and user now sees the validations for the dates kicking in. This is very awkward experience.I can see that in this issue #61, the example adds a
.partial()
to get around this problem, but this really isn't a solution because all of these fields are required in my case. It'd be nice if the.refine
function defined on each individual field had access to the current form values, but it has access to its own value only. Any ideas, or did I miss something?The text was updated successfully, but these errors were encountered: