Zod validation support, switch to Fetch API, and refactoring #147

ahejlsberg · 2023-12-13T01:07:32Z

This PR introduces the following changes:

Support for validation using Zod, a popular TypeScript-first schema validation library.
Switch to using the JavaScript Fetch API for LLM access and remove dependency on axios package.
Refactoring of certain functions and module names.

Zod support

The new Zod support is useful in applications that already incorporate JSON validation and/or dynamically change schema based on interaction history, databases, or other information sources. Also, Zod is significantly lighter weight than incorporating the TypeScript compiler for validation purposes.

Even though validation of JSON responses can now be performed by the Zod validator, schemas are still communicated to the LLM as TypeScript type definitions. The new typechat/zod validator automatically converts Zod schema to TypeScript source code suitable for inclusion in LLM prompts (and the conversion is available in a helper function for other purposes). In our experience, using TypeScript to describe schema to LLMs produces better outcomes and consumes fewer tokens than using JSON Schema.

Two new examples, examples/coffeeShop-zod and examples/sentiment-zod, demonstrate the new Zod support. These are Zod equivalents of the corresponding examples that use the TypeScript compiler for validation.

An example of a simple Zod schema for determining the sentiment of a some user input.:

import { createLanguageModel, createJsonTranslator, processRequests } from "typechat";
import { createZodJsonValidator } from "typechat/zod";
import { z } from "zod";

const SentimentResponse = z.object({
    sentiment: z.enum(["negative", "neutral", "positive"]).describe("The sentiment of the text")
});

const SentimentSchema = {
    SentimentResponse
};

const model = createLanguageModel(process.env);
const validator = createZodJsonValidator(SentimentSchema, "SentimentResponse");
const translator = createJsonTranslator(model, validator);

// Process requests interactively or from the input file specified on the command line
processRequests("😀> ", undefined, async (request) => {
    const response = await translator.translate(request);
    if (!response.success) {
        console.log(response.message);
        return;
    }
    console.log(`The sentiment is ${response.data.sentiment}`);
});

In addition to the createZodJsonValidator function, the new typechat/zod module exports a convenient getZodSchemaAsTypeScript function that can be used to convert Zod schemas into TypeScript source code.

Switch to Fetch API

The implementation has switched to using the JavaScript Fetch API instead of depending on the axios package. This helps reduce bundle sizes with no effect on client applications.

Refactoring

The implementation has been refactored as follows:

Validators are now in the typechat/ts and typechat/zod modules (instead of the main typechat module).
The createJsonTranslator function now takes a validator parameter instead of actually creating the validator.
The TypeChatJsonValidator interface has been changed to have methods instead of properties for accessing the schema text and target type name, and the validate method has been changed to take an object instead of a string parameter.

These are breaking changes, but quite easy to adopt.

jakebailey · 2023-12-13T01:59:31Z

package.json

-    "axios": "^1.4.0",
-    "typescript": "^5.1.3"
+    "typescript": "^5.3.3",
+    "zod": "^3.22.4"


Did the optional peer dependency trick not work out? (Happy to help get that working!)

steveluc · 2023-12-13T17:46:18Z

👍

DanielRosenwasser · 2023-12-13T21:33:03Z

examples/music/package.json

-    "typechat": "^0.0.10"
+    "typechat": "^1.0.0"


This seems off, right?

Yeah, we probably want a 0.1.0 change at most

DanielRosenwasser · 2023-12-13T21:33:24Z

examples/restaurant/package.json

@@ -12,11 +12,11 @@
  "license": "MIT",
  "dependencies": {
    "dotenv": "^16.3.1",
-    "typechat": "^0.0.10"
+    "typechat": "^1.0.0"


Seems like all of these got rev'd - I'm not sure why.

DanielRosenwasser · 2023-12-13T21:34:22Z

package.json

@@ -1,7 +1,7 @@
 {
  "name": "typechat",
  "author": "Microsoft",
-  "version": "0.0.10",
+  "version": "1.0.0",


Woah, I don't think we want to bump to 1.0.0 - I see now why the other packages got updated!

DanielRosenwasser · 2023-12-13T21:36:03Z

examples/calendar/src/main.ts

-translator.validator.stripNulls = true;
+const validator = createTypeScriptJsonValidator<CalendarActions>(schema, "CalendarActions");
+const translator = createJsonTranslator(model, validator);
+//translator.stripNulls = true;


Suggested change

//translator.stripNulls = true;

Should we drop this?

src/zod/validate.ts

DanielRosenwasser · 2023-12-13T21:42:47Z

src/zod/validate.ts

+    function validate(jsonObject: object) {
+        const result = targetType.safeParse(jsonObject);
+        if (!result.success) {
+            return error(result.error.issues.map(({ path, message }) => `${path.map(key => `[${JSON.stringify(key)}]`).join("")}: ${message}`).join("\""));


In Python I just join the paths with a . - which is not exactly correct of course, but it might be more readable in the usual case.

DanielRosenwasser · 2023-12-13T22:56:43Z

src/zod/validate.ts

+            case z.ZodFirstPartyTypeKind.ZodNumber:
+                return append("number");


One of the things I'm wondering about is how you perform deduplication. For example, if a user wrote:

z.union([z.number().positive(), z.number().negative()])

Currently it doesn't. Maybe that's fine for now.

DanielRosenwasser · 2023-12-13T23:16:42Z

src/typechat.ts

+ */
+export interface TypeChatJsonValidator<T extends object> {
+    /**
+     * Return a string containing TypeScript source code for the validation schema.


One of the things I did in Python was I completely decoupled the translator and the validator. So here what you could do is create a ZodTranslator and a ZodValidator. The ZodTranslator would know everything it needs for generating TypeScript and crafting the appropriate prompt for a model. The ZodValidator would know everything it needs to validating input JSON.

While it does mean that you need a different kind of validator, I think that separation of concerns would make a lot more sense here. It would also decouple TypeChat validators from specifically talking about TypeScript as a schema language.

On the flip side, you could take the stance that the Python implementation implicitly couples the default translator to talking about TypeScript as a schema language - which itself might be worth discussing.

What was your take on this idea @ahejlsberg?

I prefer to have the validator have all knowledge of schema (including how to construct the schema string). This works much better in a dynamic schema scenario.

DanielRosenwasser · 2023-12-15T22:38:14Z

package.json

+      "types": "./dist/index.d.ts",
+      "default": "./dist/index.js"
+    },
+    "./ts": {
+      "types": "./dist/ts/index.d.ts",
+      "default": "./dist/ts/index.js"
+    },
+    "./zod": {
+      "types": "./dist/zod/index.d.ts",
+      "default": "./dist/zod/index.js"


This looks right to me, but figure we can check in with @andrewbranch and @jakebailey.

Looks good, but can be simplified to

".": "./dist/index.js", "./ts": "./dist/ts/index.js", "./zod: "./dist/zod/index.js"

TS knows how to find sibling types.

Also, note that anyone accidentally using --moduleResolution node10 (still a default for some settings) will not be able to access these subpaths. TS will tell them to import from "typechat/dist/zod" even though that won’t work in anything modern at runtime. It’s up to you whether you want to use that as an opportunity to get them to update their settings or provide a mitigation like one of these: https://github.com/andrewbranch/example-subpath-exports-ts-compat

DanielRosenwasser · 2024-01-18T21:52:48Z

If we're good with the current state of things, can you just pull from main, run an npm install, and update the package-lock.json?

hillary-mutisya · 2024-01-19T00:12:26Z

src/typechat.ts

+ */
+export interface TypeChatJsonValidator<T extends object> {
+    /**
+     * Return a string containing TypeScript source code for the validation schema.


I prefer to have the validator have all knowledge of schema (including how to construct the schema string). This works much better in a dynamic schema scenario.

# Conflicts: # package-lock.json

ahejlsberg added 4 commits December 12, 2023 15:14

Add Zod validation support, switch from axios to fetch API

ef74a9b

Minor changes

9ebcc27

Add sentiment-zod example

6b5ad94

Update package-lock.json

34f3fd7

ahejlsberg mentioned this pull request Dec 13, 2023

Bump axios from 1.4.0 to 1.6.0 #138

Closed

ahejlsberg requested a review from DanielRosenwasser December 13, 2023 01:22

jakebailey reviewed Dec 13, 2023

View reviewed changes

danmarshall mentioned this pull request Dec 13, 2023

TypeChat design question: Aren't we mixing schema and data? #123

Open

DanielRosenwasser mentioned this pull request Dec 13, 2023

Provide a Python implementation of TypeChat #148

Merged

DanielRosenwasser requested changes Dec 13, 2023

View reviewed changes

DanielRosenwasser reviewed Dec 15, 2023

View reviewed changes

ahejlsberg added 3 commits December 15, 2023 17:50

Update version to 0.1.0

103a352

Specify target type as a string in createZodJsonValidator

a210824

Simplify package.json

a5fc03b

DanielRosenwasser mentioned this pull request Dec 22, 2023

should we read interface as a text file? #142

Open

hillary-mutisya approved these changes Jan 19, 2024

View reviewed changes

ahejlsberg added 2 commits January 22, 2024 11:28

Merge branch 'main' into zodAndFetch

0d218de

# Conflicts: # package-lock.json

Update package-lock.json

53c874d

DanielRosenwasser approved these changes Jan 23, 2024

View reviewed changes

ahejlsberg merged commit 25a33a6 into main Jan 23, 2024
6 checks passed

jakebailey mentioned this pull request Jan 23, 2024

Make typescript and zod optional peer dependencies #174

Merged

DanielRosenwasser deleted the zodAndFetch branch March 29, 2024 21:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Zod validation support, switch to Fetch API, and refactoring #147

Zod validation support, switch to Fetch API, and refactoring #147

ahejlsberg commented Dec 13, 2023 •

edited

Loading

jakebailey Dec 13, 2023

steveluc commented Dec 13, 2023

DanielRosenwasser Dec 13, 2023

hillary-mutisya Jan 18, 2024

DanielRosenwasser Dec 13, 2023

DanielRosenwasser Dec 13, 2023

DanielRosenwasser Dec 13, 2023

DanielRosenwasser Jan 18, 2024

DanielRosenwasser Dec 13, 2023 •

edited

Loading

DanielRosenwasser Dec 13, 2023

DanielRosenwasser Jan 18, 2024

DanielRosenwasser Dec 13, 2023

DanielRosenwasser Jan 18, 2024

hillary-mutisya Jan 19, 2024

DanielRosenwasser Dec 15, 2023

andrewbranch Dec 15, 2023

DanielRosenwasser commented Jan 18, 2024

hillary-mutisya Jan 19, 2024

		case z.ZodFirstPartyTypeKind.ZodNumber:
		return append("number");

Zod validation support, switch to Fetch API, and refactoring #147

Zod validation support, switch to Fetch API, and refactoring #147

Conversation

ahejlsberg commented Dec 13, 2023 • edited Loading

Zod support

Switch to Fetch API

Refactoring

Choose a reason for hiding this comment

steveluc commented Dec 13, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DanielRosenwasser Dec 13, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DanielRosenwasser commented Jan 18, 2024

Choose a reason for hiding this comment

ahejlsberg commented Dec 13, 2023 •

edited

Loading

DanielRosenwasser Dec 13, 2023 •

edited

Loading