Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions typescript/effect-ai/src/gemini-thinking/KNOWN-ISSUE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Known Issue: Schema Validation Failure

## Problem

The Effect AI example for Gemini 3 reasoning currently fails with a schema validation error:

```
Expected "unknown" | "openai-responses-v1" | "anthropic-claude-v1"
Actual: "google-gemini-v1"
```

## Root Cause

The `@effect/ai-openrouter` package uses strict schema validation for API responses. The `reasoning_details[].format` field in the response schema only includes:
- `"unknown"`
- `"openai-responses-v1"`
- `"anthropic-claude-v1"`

However, Gemini 3 returns `"google-gemini-v1"` as the format, which is not yet in the allowed enum.

## Evidence

Error occurs at:
```
packages/router/adapters/google-gemini/schemas.ts
ReasoningDetailText (Encoded side)
└─ ["format"]
└─ Expected "unknown" | "openai-responses-v1" | "anthropic-claude-v1"
Actual: "google-gemini-v1"
```

## Solution

The schema in `@effect/ai-openrouter` needs to be updated to include `"google-gemini-v1"` as a valid format option for reasoning details.

## Workaround

Until the schema is updated, use:
1. ✅ **Fetch example** - Works correctly (no schema validation)
2. ✅ **AI SDK v5 example** - Works correctly (looser schema validation)
3. ❌ **Effect AI example** - Fails due to strict schema enforcement

## Impact

This is a schema definition issue, not a functionality issue. The API is working correctly and returning valid data. The Effect AI provider just needs its schema updated to accept the new format.

## Related Files

- `refs/ai-sdk-provider/packages/ai-openrouter/src/OpenRouterClient.ts` - Schema definitions
- Needs update to include `"google-gemini-v1"` in reasoning detail format enum
126 changes: 126 additions & 0 deletions typescript/effect-ai/src/gemini-thinking/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
# Google Gemini 3 Reasoning/Thinking Examples

This directory contains examples of using Google Gemini 3's reasoning/thinking feature via OpenRouter.

## What is Gemini Reasoning?

Gemini 3 models can engage in internal reasoning before generating responses. This "thinking mode" allows the model to:
- Work through complex problems step-by-step
- Show its reasoning process
- Improve answer quality on difficult tasks

## How It Works

1. **Request**: Set `reasoning.enabled: true` (or `reasoning.max_tokens`, or `reasoning.effort`)
2. **Processing**: The model uses "thinking tokens" for internal reasoning
3. **Response**: You receive both the reasoning process and the final answer

## Examples

### `basic-reasoning.ts`

Demonstrates basic usage of Gemini reasoning with a multi-step problem.

**Run:**
```bash
bun run src/gemini-thinking/basic-reasoning.ts
```

**Key Features:**
- Enables reasoning mode
- Shows thinking token usage
- Displays reasoning process
- Returns final answer

## API Parameters

### Request Format

```typescript
{
model: 'google/gemini-3-pro-preview',
reasoning: {
enabled: true, // Enable thinking mode
max_tokens: 4096, // Token budget for thinking
exclude: false // true = hide thoughts, false = show thoughts
},
messages: [...]
}
```

### Alternative: Effort Levels

```typescript
{
model: 'google/gemini-3-pro-preview',
reasoning: {
effort: 'medium' // 'low', 'medium', 'high'
},
messages: [...]
}
```

## Response Format

```typescript
{
choices: [{
message: {
content: "The final answer",
reasoning: "The model's thinking process...",
reasoning_details: [
{
type: "reasoning.text",
text: "Internal reasoning...",
format: "gemini"
},
{
type: "reasoning.encrypted",
data: "encrypted_signature",
format: "gemini"
}
]
}
}],
usage: {
prompt_tokens: 123,
completion_tokens: 456,
completion_tokens_details: {
reasoning_tokens: 234 // Tokens used for thinking
}
}
}
```

## Key Points

### Model Support
- ✅ `google/gemini-3-pro-preview` - Reasoning MANDATORY (always enabled)
- ✅ `google/gemini-2.5-pro` - Reasoning MANDATORY (always enabled)
- ✅ `google/gemini-2.5-flash` - Reasoning OPTIONAL

### Token Budgets
- **Gemini 3 Pro**: Max 200,000 thinking tokens, 1M context window
- **Gemini 2.5 Pro**: Max 32,768 thinking tokens
- **Gemini 2.5 Flash**: Max 24,576 thinking tokens

### Important Notes
- **Preserve reasoning_details**: Must include `reasoning_details` from previous messages in follow-up requests
- **Cost**: Thinking tokens are billed separately (usually at a lower rate)
- **Latency**: More thinking tokens = longer response time
- **Quality**: Higher thinking budgets improve answer quality on complex tasks

## OpenRouter Transformation

OpenRouter automatically transforms Google's native API to OpenAI-compatible format:

| Google Native | OpenRouter (OpenAI-compatible) |
|--------------|-------------------------------|
| `usageMetadata.thoughtsTokenCount` | `usage.completion_tokens_details.reasoning_tokens` |
| `parts[].thought: true` | `message.reasoning` |
| `thoughtSignature` | `reasoning_details[].data` |

## Resources

- [OpenRouter Docs - Reasoning Tokens](https://openrouter.ai/docs/use-cases/reasoning-tokens)
- [Google Gemini API Docs](https://ai.google.dev/docs)
123 changes: 123 additions & 0 deletions typescript/effect-ai/src/gemini-thinking/basic-reasoning.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
/**
* Example: Google Gemini 3 Reasoning/Thinking Details (Effect AI)
*
* This example demonstrates requesting reasoning details from Gemini 3 models
* via OpenRouter using Effect AI.
*
* Pattern: Single request with reasoning enabled using Effect patterns
* - Effect.gen for effect composition
* - Layer-based dependency injection
* - Type-safe error handling
*
* NOTE: This example currently fails due to a schema limitation in @effect/ai-openrouter.
* The reasoning_details format "google-gemini-v1" is not yet included in the schema.
* Expected formats: "unknown" | "openai-responses-v1" | "anthropic-claude-v1"
* Actual format returned: "google-gemini-v1"
*
* This will be fixed once the schema is updated to include "google-gemini-v1".
* The fetch and AI SDK v5 examples work correctly as they don't enforce strict schemas.
*/

import * as OpenRouterClient from '@effect/ai-openrouter/OpenRouterClient';
import * as OpenRouterLanguageModel from '@effect/ai-openrouter/OpenRouterLanguageModel';
import * as LanguageModel from '@effect/ai/LanguageModel';
import * as Prompt from '@effect/ai/Prompt';
import { FetchHttpClient } from '@effect/platform';
import * as BunContext from '@effect/platform-bun/BunContext';
import { Console, Effect, Layer, Redacted } from 'effect';

const program = Effect.gen(function* () {
yield* Console.log(
'╔════════════════════════════════════════════════════════════════════════════╗',
);
yield* Console.log(
'║ Google Gemini 3 - Reasoning/Thinking Details (Effect AI) ║',
);
yield* Console.log(
'╚════════════════════════════════════════════════════════════════════════════╝',
);
yield* Console.log('');
yield* Console.log('Testing Gemini reasoning feature with a multi-step problem');
yield* Console.log('');

const prompt = Prompt.make([
{
role: 'user' as const,
content:
'Solve this problem step by step: If a train leaves station A at 2pm traveling 60mph, and another train leaves station B (120 miles away) at 2:30pm traveling 80mph toward station A, when and where do they meet?',
},
]);

yield* Console.log('Request with Reasoning Enabled');
const response = yield* LanguageModel.generateText({
prompt,
});

// Access usage metrics
const reasoningTokens = response.usage.reasoningTokens ?? 0;
const promptTokens = response.usage.inputTokens ?? 0;
const completionTokens = response.usage.outputTokens ?? 0;

yield* Console.log(
` prompt=${promptTokens}, completion=${completionTokens}, reasoning=${reasoningTokens}`,
);

// Analysis
yield* Console.log('\n' + '='.repeat(80));
yield* Console.log('ANALYSIS');
yield* Console.log('='.repeat(80));

yield* Console.log(`Reasoning tokens: ${reasoningTokens}`);

if (reasoningTokens > 0) {
yield* Console.log(`✓ Reasoning enabled: ${reasoningTokens} tokens used for thinking`);
} else {
yield* Console.log('✗ No reasoning tokens detected');
}

yield* Console.log('\n--- Final Answer ---');
yield* Console.log(response.text);

const success = reasoningTokens > 0;

if (success) {
yield* Console.log(
'\n════════════════════════════════════════════════════════════════════════════',
);
yield* Console.log('✓ SUCCESS - Gemini reasoning is working correctly');
yield* Console.log(
'════════════════════════════════════════════════════════════════════════════',
);
} else {
yield* Console.log(
'\n════════════════════════════════════════════════════════════════════════════',
);
yield* Console.log('✗ FAILURE - Gemini reasoning is not working as expected');
yield* Console.log(
'════════════════════════════════════════════════════════════════════════════',
);
}
});

const OpenRouterClientLayer = OpenRouterClient.layer({
apiKey: Redacted.make(process.env.OPENROUTER_API_KEY!),
}).pipe(Layer.provide(FetchHttpClient.layer));

const OpenRouterModelLayer = OpenRouterLanguageModel.layer({
model: 'google/gemini-3-pro-preview',
config: {
reasoning: {
enabled: true,
max_tokens: 2000,
exclude: false,
},
},
}).pipe(Layer.provide(OpenRouterClientLayer));

await program.pipe(
Effect.provide(OpenRouterModelLayer),
Effect.provide(BunContext.layer),
Effect.runPromise,
);

console.log('\n✓ Program completed successfully');