Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Claude finished @chitalian's task —— View job PR Review: E2E Testing Suite ImplementationScore: 5/10 - Significant security and architectural concerns that should be addressed before merging Todo List:
Critical Security Issues1. Hardcoded Production Credentials 🔴
2. Hardcoded Admin Access Key 🔴
3. Missing Workspace Integration 🟡
Architecture & Code Quality Issues4. Route Conflict 🟡
5. GitHub Actions Port Conflicts 🟡
# Suggested fix:
--inspector-port=9240 # for HELICONE_API
--inspector-port=9241 # for AI_GATEWAY_API6. Mock Response System Design 🟡
7. Fragile HTTP Client Reset 🟡
this.client.defaults.headers = { ...TEST_HEADERS } as any;Minor Issues8. Code Style Inconsistencies
9. Commented Debug Code
10. Port Configuration Inconsistency
Positive Aspects✅ Comprehensive test coverage - Health checks, gateway functionality, wallet testing Recommendations SummaryBefore merging:
Post-merge improvements:
The E2E testing infrastructure is well-designed and comprehensive, but the security issues with hardcoded credentials are blocking concerns that must be addressed before merging. |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
Greptile Overview
Summary
This PR introduces a comprehensive end-to-end (E2E) testing suite for the Helicone AI Gateway platform. The changes establish a complete testing infrastructure that can validate the entire system flow from gateway proxy requests through to billing and wallet functionality.The implementation adds a dedicated /e2e directory with its own TypeScript configuration, Jest setup, and test utilities. The testing suite includes helper functions for creating OpenAI-compatible requests, validating responses, managing wallet state, and interacting with HTTP endpoints. Key components include:
- Testing Infrastructure: Separate TypeScript config, Jest configuration with 30s timeout for integration tests, and comprehensive package.json with testing dependencies
- Service Integration: Tests for AI Gateway (port 8793), Worker API (8788), Jawn backend (8585), and database connections (PostgreSQL, ClickHouse)
- Mock System: Introduced mock response functionality in
ProviderClient.tsthat intercepts requests with__helicone-mock-responseheader, returning predefined OpenAI-compatible responses without making actual API calls - Test Utilities: HTTP client wrapper, wallet manipulation helpers, request/response validation functions, and retry mechanisms with exponential backoff
- CI/CD Integration: GitHub Actions workflow that orchestrates multiple services with health checks and proper cleanup
The testing approach allows comprehensive validation of the gateway's proxy functionality, billing calculations, error handling, and API compatibility while avoiding external API costs through the mock system. The suite covers both successful request flows and error scenarios, with proper wallet state management for testing credit exhaustion scenarios.
Important Files Changed
Changed Files
| Filename | Score | Overview |
|---|---|---|
e2e/tsconfig.json |
5/5 | New TypeScript configuration for e2e tests with proper Jest and Node.js setup |
e2e/tests/setup.ts |
5/5 | Jest setup file with environment variable loading and test lifecycle hooks |
worker/src/routers/api/apiRouter.ts |
3/5 | Added catch-all route handler with potential duplicate route conflict |
e2e/lib/test-helpers.ts |
5/5 | Comprehensive test utilities for OpenAI request/response handling and retry logic |
worker/src/lib/clients/ProviderClient.ts |
4/5 | Added mock response system for testing without external API calls |
e2e/lib/constants.ts |
4/5 | Configuration constants with hardcoded test credentials that may need rotation |
e2e/lib/wallet-helpers.ts |
4/5 | Wallet manipulation utilities with hardcoded API key for testing |
e2e/tests/wallet/credit-exhaustion.test.ts |
4/5 | Wallet credit testing with hardcoded cost values and incomplete test coverage |
e2e/package.json |
3/5 | Test package configuration with potential security concerns and missing workspace integration |
.github/workflows/e2e-test-suite.yml |
4/5 | Comprehensive CI workflow with proper service orchestration and health checks |
e2e/jest.config.ts |
4/5 | Well-configured Jest setup for e2e testing with coverage collection |
e2e/.env.example |
4/5 | Environment configuration template with minor port inconsistencies |
e2e/tests/gateway/health.test.ts |
4/5 | Service health check tests with potential fragility in response parsing |
e2e/lib/http-client.ts |
4/5 | HTTP client wrapper with proper configuration for gateway testing |
e2e/README.md |
4/5 | Comprehensive documentation with some outdated script references |
e2e/tests/gateway/chat-completion.test.ts |
4/5 | Core gateway functionality tests with flexible error handling |
supabase/seed.sql |
4/5 | Database seed updates to support testing infrastructure with credits feature flag |
worker/src/lib/clients/mockOpenAIResponse.ts |
4/5 | Mock response utility with unrealistic default values and future model references |
Confidence score: 4/5
- This PR introduces significant testing infrastructure improvements with comprehensive coverage, but contains several potential issues that need attention
- Score reflects the valuable testing additions balanced against concerns about hardcoded credentials, route conflicts, and inconsistent configurations
- Pay close attention to
worker/src/routers/api/apiRouter.tsfor duplicate catch-all routes,e2e/lib/constants.tsande2e/lib/wallet-helpers.tsfor hardcoded credentials, ande2e/package.jsonfor security vulnerabilities
Sequence Diagram
sequenceDiagram
participant User
participant GitHub Actions
participant Docker
participant Supabase
participant Worker API
participant AI Gateway
participant Jawn
participant E2E Tests
participant ClickHouse
User->>GitHub Actions: "Push to main branch"
GitHub Actions->>GitHub Actions: "Checkout code"
GitHub Actions->>GitHub Actions: "Setup Node.js 20"
GitHub Actions->>GitHub Actions: "Enable Corepack"
GitHub Actions->>GitHub Actions: "Install dependencies"
GitHub Actions->>Docker: "Start Clickhouse and Minio containers"
Docker-->>GitHub Actions: "Services started"
GitHub Actions->>Supabase: "Start Supabase (excluding realtime, storage-api, etc.)"
Supabase-->>GitHub Actions: "Database ready"
GitHub Actions->>Worker API: "Start HELICONE_API worker on port 8788"
Worker API-->>GitHub Actions: "Worker started"
GitHub Actions->>AI Gateway: "Start AI_GATEWAY_API worker on port 8793"
AI Gateway-->>GitHub Actions: "Worker started"
GitHub Actions->>GitHub Actions: "Wait 10 seconds for workers"
GitHub Actions->>Worker API: "Health check on port 8788"
Worker API-->>GitHub Actions: "Health check response"
GitHub Actions->>AI Gateway: "Health check on port 8793"
AI Gateway-->>GitHub Actions: "Health check response"
GitHub Actions->>Jawn: "Start Jawn service"
Jawn-->>GitHub Actions: "Service started"
GitHub Actions->>E2E Tests: "Install test dependencies"
E2E Tests->>AI Gateway: "Run credit exhaustion tests"
E2E Tests->>Worker API: "Add credits to wallet"
Worker API-->>E2E Tests: "Credits added"
E2E Tests->>AI Gateway: "Make chat completion request with mock response"
AI Gateway-->>E2E Tests: "Mock OpenAI response"
E2E Tests->>Worker API: "Check wallet balance"
Worker API-->>E2E Tests: "Updated balance"
E2E Tests-->>GitHub Actions: "Test results"
GitHub Actions->>GitHub Actions: "Display logs on failure (if needed)"
GitHub Actions->>GitHub Actions: "Cleanup: kill processes, stop services"
Context used:
Context from dashboard - When naming jobs in GitHub Actions, prefer descriptive names that clearly indicate the job's purpose... (source)
Context from dashboard - Use 'bash' at the start of code blocks that contain shell commands for clarity. (source)
18 files reviewed, 15 comments
worker/src/routers/api/apiRouter.ts
Outdated
| router.all( | ||
| "*", | ||
| async ( | ||
| _, | ||
| _requestWrapper: RequestWrapper, | ||
| _env: Env, | ||
| _ctx: ExecutionContext | ||
| ) => { | ||
| return new Response("invalid path", { status: 404 }); | ||
| } | ||
| ); |
There was a problem hiding this comment.
logic: This catch-all handler may never be reached due to route ordering. The parent getAPIRouter function also has a catch-all handler (lines 527-537) that returns status 400, which could conflict with this 404 handler.
Prompt To Fix With AI
This is a comment left during a code review.
Path: worker/src/routers/api/apiRouter.ts
Line: 510:520
Comment:
**logic:** This catch-all handler may never be reached due to route ordering. The parent `getAPIRouter` function also has a catch-all handler (lines 527-537) that returns status 400, which could conflict with this 404 handler.
How can I resolve this? If you propose a fix, please make it concise.
worker/src/routers/api/apiRouter.ts
Outdated
| _env: Env, | ||
| _ctx: ExecutionContext | ||
| ) => { | ||
| return new Response("invalid path", { status: 404 }); |
There was a problem hiding this comment.
style: Status code inconsistency: this returns 404 while the parent router's catch-all returns 400. Consider standardizing on one status code or documenting the intended difference.
Prompt To Fix With AI
This is a comment left during a code review.
Path: worker/src/routers/api/apiRouter.ts
Line: 518:518
Comment:
**style:** Status code inconsistency: this returns 404 while the parent router's catch-all returns 400. Consider standardizing on one status code or documenting the intended difference.
How can I resolve this? If you propose a fix, please make it concise.| expect(addCreditsResponse.status).toBe(200); | ||
| expect(addCreditsResponse.data.effectiveBalance).toBe(100); | ||
| // sleep for 2 seconds to ensure wallet state is updated | ||
| await new Promise((resolve) => setTimeout(resolve, 2000)); |
There was a problem hiding this comment.
style: Use the imported sleep helper function instead of new Promise for consistency
| await new Promise((resolve) => setTimeout(resolve, 2000)); | |
| await sleep(2000); |
Prompt To Fix With AI
This is a comment left during a code review.
Path: e2e/tests/wallet/credit-exhaustion.test.ts
Line: 45:45
Comment:
**style:** Use the imported `sleep` helper function instead of `new Promise` for consistency
```suggestion
await sleep(2000);
```
How can I resolve this? If you propose a fix, please make it concise.| max_tokens: 10, | ||
| }); | ||
|
|
||
| // await sleep(100000); |
There was a problem hiding this comment.
style: Remove commented debug sleep code
Prompt To Fix With AI
This is a comment left during a code review.
Path: e2e/tests/wallet/credit-exhaustion.test.ts
Line: 52:52
Comment:
**style:** Remove commented debug sleep code
How can I resolve this? If you propose a fix, please make it concise.| { | ||
| "name": "@helicone/e2e", | ||
| "version": "1.0.0", | ||
| "private": true, | ||
| "description": "End-to-end tests for Helicone", | ||
| "scripts": { | ||
| "test": "jest", | ||
| "test:watch": "jest --watch", | ||
| "test:coverage": "jest --coverage", | ||
| "test:gateway": "jest tests/gateway", | ||
| "test:integration": "jest --testPathPattern=integration" | ||
| }, | ||
| "dependencies": { | ||
| "axios": "^1.6.7", | ||
| "dotenv": "^16.4.5" | ||
| }, | ||
| "devDependencies": { | ||
| "@types/jest": "^29.5.12", | ||
| "@types/node": "^20.11.19", | ||
| "jest": "^29.7.0", | ||
| "ts-jest": "^29.1.2", | ||
| "ts-node": "^10.9.2", | ||
| "typescript": "^5.3.3" | ||
| } | ||
| } |
There was a problem hiding this comment.
logic: Missing workspace integration - this e2e package should be added to the root package.json workspaces array to integrate with the monorepo structure
Prompt To Fix With AI
This is a comment left during a code review.
Path: e2e/package.json
Line: 1:25
Comment:
**logic:** Missing workspace integration - this e2e package should be added to the root package.json workspaces array to integrate with the monorepo structure
How can I resolve this? If you propose a fix, please make it concise.
e2e/lib/http-client.ts
Outdated
| this.client.defaults.headers = { | ||
| ...this.client.defaults.headers, | ||
| ...TEST_HEADERS, | ||
| } as any; |
There was a problem hiding this comment.
logic: The resetHeaders() method doesn't actually reset to original defaults - it merges existing headers with TEST_HEADERS, potentially accumulating headers over time
| this.client.defaults.headers = { | |
| ...this.client.defaults.headers, | |
| ...TEST_HEADERS, | |
| } as any; | |
| this.client.defaults.headers = { | |
| ...TEST_HEADERS, | |
| } as any; |
Prompt To Fix With AI
This is a comment left during a code review.
Path: e2e/lib/http-client.ts
Line: 52:55
Comment:
**logic:** The `resetHeaders()` method doesn't actually reset to original defaults - it merges existing headers with TEST_HEADERS, potentially accumulating headers over time
```suggestion
this.client.defaults.headers = {
...TEST_HEADERS,
} as any;
```
How can I resolve this? If you propose a fix, please make it concise.
e2e/README.md
Outdated
| ```yaml | ||
| - name: Install E2E dependencies | ||
| run: cd e2e && yarn install | ||
|
|
||
| - name: Start Gateway | ||
| run: | | ||
| cd worker | ||
| npx wrangler dev --var WORKER_TYPE:AI_GATEWAY_API --port 8793 & | ||
| sleep 10 | ||
|
|
||
| - name: Run E2E tests | ||
| run: cd e2e && yarn test | ||
| ``` |
There was a problem hiding this comment.
style: CI/CD example uses bash syntax in YAML - consider prefacing with 'bash' for clarity
| ```yaml | |
| - name: Install E2E dependencies | |
| run: cd e2e && yarn install | |
| - name: Start Gateway | |
| run: | | |
| cd worker | |
| npx wrangler dev --var WORKER_TYPE:AI_GATEWAY_API --port 8793 & | |
| sleep 10 | |
| - name: Run E2E tests | |
| run: cd e2e && yarn test | |
| ``` | |
| ```bash | |
| - name: Install E2E dependencies | |
| run: cd e2e && yarn install | |
| - name: Start Gateway | |
| run: | | |
| cd worker | |
| npx wrangler dev --var WORKER_TYPE:AI_GATEWAY_API --port 8793 & | |
| sleep 10 | |
| - name: Run E2E tests | |
| run: cd e2e && yarn test |
Context Used: Context from dashboard - Use 'bash' at the start of code blocks that contain shell commands for clarity. (source)
Prompt To Fix With AI
This is a comment left during a code review.
Path: e2e/README.md
Line: 176:188
Comment:
**style:** CI/CD example uses `bash` syntax in YAML - consider prefacing with 'bash' for clarity
```suggestion
```bash
- name: Install E2E dependencies
run: cd e2e && yarn install
- name: Start Gateway
run: |
cd worker
npx wrangler dev --var WORKER_TYPE:AI_GATEWAY_API --port 8793 &
sleep 10
- name: Run E2E tests
run: cd e2e && yarn test
```
**Context Used:** Context from `dashboard` - Use 'bash' at the start of code blocks that contain shell commands for clarity. ([source](https://app.greptile.com/review/custom-context?memory=c221915c-f2f8-4540-9926-c32242f06805))
How can I resolve this? If you propose a fix, please make it concise.
|
@greptile |
|
@claude please review |
|
Claude finished @chitalian's task —— View job PR Review: E2E Testing Suite ImplementationTodo List:
Score: 5/10 - Significant security and architectural concerns that should be addressed before merging Critical Security Issues1. Hardcoded Production Credentials 🔴
2. Hardcoded Admin Access Key 🔴
3. Missing Workspace Integration 🟡
Architecture & Code Quality Issues4. Route Conflict 🟡
5. GitHub Actions Port Conflicts 🟡
# Suggested fix:
--inspector-port=9240 # for HELICONE_API
--inspector-port=9241 # for AI_GATEWAY_API6. Mock Response System Design 🟡
7. Fragile HTTP Client Reset 🟡
this.client.defaults.headers = { ...TEST_HEADERS } as any;Minor Issues8. Code Style Inconsistencies
9. Commented Debug Code
10. Port Configuration Inconsistency
Positive Aspects✅ Comprehensive test coverage - Health checks, gateway functionality, wallet testing Recommendations SummaryBefore merging:
Post-merge improvements:
The E2E testing infrastructure is well-designed and comprehensive, but the security issues with hardcoded credentials are blocking concerns that must be addressed before merging. |
There was a problem hiding this comment.
Greptile Overview
Summary
This review covers only the changes made since the last review, not the entire PR. The most recent changes involve adding a comprehensive end-to-end testing suite to the Helicone project, including new test files for gateway chat completion functionality.The main addition is e2e/tests/gateway/chat-completion.test.ts, which introduces thorough integration tests for the AI Gateway's chat completion endpoints. The test suite validates successful chat completion requests, error handling scenarios, and Helicone-specific header processing. The implementation uses the established HTTP client infrastructure and test helpers created in earlier parts of this PR.
The test file includes three main test scenarios: a basic successful chat completion with response validation, error handling for malformed JSON requests, and testing of custom header functionality. The tests leverage mock responses through the __helicone-mock-response header mechanism that was implemented in the ProviderClient changes, ensuring predictable test outcomes without external API dependencies.
The tests integrate with the broader E2E testing infrastructure by using shared utilities like gatewayClient, createChatCompletionRequest, and validateChatCompletionResponse functions. The implementation follows the established testing patterns and uses proper assertions to verify both successful responses and error conditions.
Important Files Changed
Changed Files
| Filename | Score | Overview |
|---|---|---|
| e2e/tests/gateway/chat-completion.test.ts | 4/5 | Added comprehensive E2E tests for gateway chat completion functionality with proper validation and error handling |
Confidence score: 4/5
- This PR addition is safe to merge with minimal risk, focusing on testing infrastructure improvements
- Score reflects well-structured test implementation with proper assertions and error handling patterns
- No files require special attention beyond addressing the minor import cleanup noted in previous reviews
Additional Comments (1)
-
bifrost/lib/clients/jawnTypes/private.ts, line 1 (link)logic: logic: This autogenerated types file is empty but should contain TypeScript definitions including the updated ModelProviderName enum. This suggests a build or generation issue that needs to be resolved before merging.
22 files reviewed, 7 comments
| const delayMs = options.delayMs ?? 1000; | ||
| const backoffMultiplier = options.backoffMultiplier ?? 2; | ||
|
|
||
| let lastError: Error; |
There was a problem hiding this comment.
logic: logic: Variable lastError may be undefined if maxAttempts is 0, causing a potential runtime error when thrown
Prompt To Fix With AI
This is a comment left during a code review.
Path: e2e/lib/test-helpers.ts
Line: 100:100
Comment:
**logic:** **logic**: Variable `lastError` may be undefined if maxAttempts is 0, causing a potential runtime error when thrown
How can I resolve this? If you propose a fix, please make it concise.| const HELICONE_MANUAL_ACCESS_KEY = | ||
| "sk-helicone-rrrrrrr-xxxxxxx-vvvvvvv-wwwwwww"; |
There was a problem hiding this comment.
style: style: Consider using environment variable for test API key instead of hardcoding. While this appears to be a placeholder key for testing, it's safer to externalize it.
Prompt To Fix With AI
This is a comment left during a code review.
Path: e2e/lib/wallet-helpers.ts
Line: 8:9
Comment:
**style:** **style:** Consider using environment variable for test API key instead of hardcoding. While this appears to be a placeholder key for testing, it's safer to externalize it.
How can I resolve this? If you propose a fix, please make it concise.| creditPurchases: any[]; | ||
| aggregatedDebits: any[]; | ||
| escrows: any[]; | ||
| disallowList: any[]; |
There was a problem hiding this comment.
style: style: Consider using more specific types instead of any[] for better type safety. Define interfaces for creditPurchases, aggregatedDebits, escrows, and disallowList if the structure is known.
Prompt To Fix With AI
This is a comment left during a code review.
Path: e2e/lib/wallet-helpers.ts
Line: 15:18
Comment:
**style:** **style:** Consider using more specific types instead of `any[]` for better type safety. Define interfaces for `creditPurchases`, `aggregatedDebits`, `escrows`, and `disallowList` if the structure is known.
How can I resolve this? If you propose a fix, please make it concise.| "!tests/**/*.spec.ts", | ||
| "!tests/**/*.test.ts", |
There was a problem hiding this comment.
style: Excluding test files from coverage is redundant since they're already in the exclusion patterns and test files shouldn't be included in coverage anyway
Prompt To Fix With AI
This is a comment left during a code review.
Path: e2e/jest.config.ts
Line: 14:15
Comment:
**style:** Excluding test files from coverage is redundant since they're already in the exclusion patterns and test files shouldn't be included in coverage anyway
How can I resolve this? If you propose a fix, please make it concise.| dotenv.config({ path: path.join(__dirname, "../.env") }); | ||
|
|
||
| // Set default environment variables for tests | ||
| process.env.GATEWAY_URL = process.env.GATEWAY_URL || "http://localhost:8793"; |
There was a problem hiding this comment.
logic: logic: Environment variable name mismatch - this sets GATEWAY_URL but constants.ts uses AI_GATEWAY_URL. This could cause tests to use wrong endpoints.
| process.env.GATEWAY_URL = process.env.GATEWAY_URL || "http://localhost:8793"; | |
| process.env.AI_GATEWAY_URL = process.env.AI_GATEWAY_URL || "http://localhost:8793"; |
Prompt To Fix With AI
This is a comment left during a code review.
Path: e2e/tests/setup.ts
Line: 12:12
Comment:
**logic:** **logic:** Environment variable name mismatch - this sets `GATEWAY_URL` but constants.ts uses `AI_GATEWAY_URL`. This could cause tests to use wrong endpoints.
```suggestion
process.env.AI_GATEWAY_URL = process.env.AI_GATEWAY_URL || "http://localhost:8793";
```
How can I resolve this? If you propose a fix, please make it concise.| const mockResponseHeader = headers.get("__helicone-mock-response"); | ||
| if (mockResponseHeader) { | ||
| if (env.ENVIRONMENT === "production") { | ||
| return new Response("Mock responses not allowed in production", { | ||
| status: 403, | ||
| }); | ||
| } |
There was a problem hiding this comment.
style: style: Consider using a more specific header name like helicone-mock-response to maintain consistency with other Helicone headers that use the helicone- prefix
Prompt To Fix With AI
This is a comment left during a code review.
Path: worker/src/lib/clients/ProviderClient.ts
Line: 101:107
Comment:
**style:** **style:** Consider using a more specific header name like `helicone-mock-response` to maintain consistency with other Helicone headers that use the `helicone-` prefix
How can I resolve this? If you propose a fix, please make it concise.
| // it("should rate limit requests after credits are exhausted", async () => { | ||
| // // Add $1 worth of credits (100 cents) | ||
| // const addCreditsResponse = await addCreditsToWallet({ | ||
| // orgId: TEST_ORG_ID, | ||
| // amount: 100, // $1.00 in cents | ||
| // reason: "Credit exhaustion test", | ||
| // }); | ||
|
|
||
| // expect(addCreditsResponse.status).toBe(200); | ||
| // expect(addCreditsResponse.data.effectiveBalance).toBe(100); | ||
| // // sleep for 2 seconds to ensure wallet state is updated | ||
|
|
||
| // // Make requests until we get rate limited | ||
| // // gpt-4o-mini costs approximately ~$0.0002 per request (input: $0.15/1M, output: $0.6/1M) | ||
| // // With 50 tokens total per request, that's about $0.000015 per request | ||
| // // So $1 should allow ~66,666 requests, but let's assume higher costs | ||
| // // At ~$0.018 per request (if we're billing higher), we should exhaust after ~55 requests | ||
| // const maxRequests = 60; | ||
| // let successfulRequests = 0; | ||
| // let rateLimitedRequests = 0; | ||
| // let firstRateLimitAt = -1; | ||
|
|
||
| // const requestBody = createChatCompletionRequest({ | ||
| // model: "gpt-5", | ||
| // messages: TEST_MESSAGES.SIMPLE, | ||
| // max_tokens: 10, | ||
| // }); | ||
|
|
||
| // for (let i = 0; i < maxRequests; i++) { | ||
| // const response = await gatewayClient.post<ChatCompletionResponse>( | ||
| // GATEWAY_ENDPOINTS.CHAT_COMPLETIONS, | ||
| // requestBody | ||
| // ); | ||
|
|
||
| // if (response.status === 200) { | ||
| // successfulRequests++; | ||
| // } else if (response.status === 402 || response.status === 429) { | ||
| // // 402: Payment Required (no credits) | ||
| // // 429: Rate Limited | ||
| // rateLimitedRequests++; | ||
| // if (firstRateLimitAt === -1) { | ||
| // firstRateLimitAt = i + 1; // 1-indexed for readability | ||
| // } | ||
| // } | ||
|
|
||
| // // Log progress every 10 requests | ||
| // if ((i + 1) % 10 === 0) { | ||
| // const walletState = await getWalletState(TEST_ORG_ID); | ||
| // console.log( | ||
| // `After ${i + 1} requests: ${successfulRequests} successful, ${rateLimitedRequests} rate limited, balance: ${walletState.data.effectiveBalance}` | ||
| // ); | ||
| // } | ||
| // } | ||
|
|
||
| // // Get final wallet state | ||
| // const finalWalletState = await getWalletState(TEST_ORG_ID); | ||
|
|
||
| // console.log("\n=== Test Results ==="); | ||
| // console.log(`Total successful requests: ${successfulRequests}`); | ||
| // console.log(`Total rate limited requests: ${rateLimitedRequests}`); | ||
| // console.log(`First rate limit occurred at request: ${firstRateLimitAt}`); | ||
| // console.log( | ||
| // `Final wallet balance: ${finalWalletState.data.effectiveBalance}` | ||
| // ); | ||
| // console.log( | ||
| // `Total credits used: ${100 - finalWalletState.data.effectiveBalance}` | ||
| // ); | ||
|
|
||
| // // Assertions | ||
| // expect(successfulRequests).toBeGreaterThan(0); | ||
| // expect(rateLimitedRequests).toBeGreaterThan(0); | ||
| // expect(firstRateLimitAt).toBeGreaterThan(0); | ||
| // expect(firstRateLimitAt).toBeLessThanOrEqual(60); | ||
| // expect(finalWalletState.data.effectiveBalance).toBeLessThanOrEqual(0); | ||
|
|
||
| // // We expect the first rate limit to occur around request 50-55 | ||
| // // (allowing for some variance in actual costs) | ||
| // expect(firstRateLimitAt).toBeGreaterThan(30); | ||
| // expect(firstRateLimitAt).toBeLessThan(60); | ||
| // }, 120000); // 2 minute timeout |
There was a problem hiding this comment.
Why comment? the logic seems to be pretty much all there, no?
2 participants
No description provided.