Add E2E tests for SDK #35
Conversation
mattapperson
force-pushed
the
fix/move-e2e-tests
branch
from
7edd54d to
7eb198a
Compare
- Create composite action for SDK validation steps (build, typecheck, tests) - Update speakeasy workflow to use composite action - Add new PR validation workflow that runs on non-OpenAPI changes - Create reusable validation-checks workflow for shared logic
|
|
||
| - Tests make real API calls to OpenRouter, so you need a valid API key | ||
| - Tests may consume API credits | ||
| - Some tests use the `openai/gpt-3.5-turbo` model by default |
There was a problem hiding this comment.
Nit: should we replace gpt-3.5-turbo openai/gpt-4.1-nano. gpt-3.5-turbo will be deprecated in a year. Also it's more expensive than nano.
There was a problem hiding this comment.
actually, would meta-llama/llama-3.2-1b-instruct be better/cheaper/faster to run?
| - Fetching the list of available models | ||
| - Validating model properties | ||
| - Filtering models by category | ||
| - Getting the total count of models |
There was a problem hiding this comment.
should we add one test where we intentionally garble the openrouter key or ask for an invalid model so that we can check that the error parsing works correctly?
| @@ -0,0 +1,59 @@ | |||
| import { beforeAll, describe, expect, it } from "vitest"; | |||
| import { OpenRouter } from "../../src/sdk/sdk.js"; | |||
|
|
|||
| } | ||
| } | ||
|
|
||
| expect(foundFinishReason).toBe(true); |
There was a problem hiding this comment.
Let's also add a usage check to this test.
| }); | ||
|
|
||
| it("should return models with expected properties", async () => { | ||
| const response = await client.models.list(); |
There was a problem hiding this comment.
we will eventually need to paginate this API, and may be able to do it without versioning the api. should we have the SDK prep for it?
Summary
Test plan