Skip to content

add thinking support and examples #223

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 30, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -59,6 +59,7 @@ ollama.chat(request)
- `images` `<Uint8Array[] | string[]>`: (Optional) Images to be included in the message, either as Uint8Array or base64 encoded strings.
- `format` `<string>`: (Optional) Set the expected format of the response (`json`).
- `stream` `<boolean>`: (Optional) When true an `AsyncGenerator` is returned.
- `think` `<boolean>`: (Optional) When true, the model will think about the response before responding. Requires thinking support from the model.
- `keep_alive` `<string | number>`: (Optional) How long to keep the model loaded. A number (seconds) or a string with a duration unit suffix ("300ms", "1.5h", "2h45m", etc.)
- `tools` `<Tool[]>`: (Optional) A list of tool calls the model may make.
- `options` `<Options>`: (Optional) Options to configure the runtime.
@@ -81,6 +82,7 @@ ollama.generate(request)
- `images` `<Uint8Array[] | string[]>`: (Optional) Images to be included, either as Uint8Array or base64 encoded strings.
- `format` `<string>`: (Optional) Set the expected format of the response (`json`).
- `stream` `<boolean>`: (Optional) When true an `AsyncGenerator` is returned.
- `think` `<boolean>`: (Optional) When true, the model will think about the response before responding. Requires thinking support from the model.
- `keep_alive` `<string | number>`: (Optional) How long to keep the model loaded. A number (seconds) or a string with a duration unit suffix ("300ms", "1.5h", "2h45m", etc.)
- `options` `<Options>`: (Optional) Options to configure the runtime.
- Returns: `<GenerateResponse>`
2 changes: 1 addition & 1 deletion examples/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
## Examples

> [!IMPORTANT]
> Note: Ensure that `npm build` has been run before running the examples.
> Note: Ensure that `npm run build` has been run before running the examples.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙏


To run the examples run:

20 changes: 20 additions & 0 deletions examples/thinking/thinking-enabled.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
import ollama from 'ollama'

async function main() {
const response = await ollama.chat({
model: 'deepseek-r1',
messages: [
{
role: 'user',
content: 'What is 10 + 23',
},
],
stream: false,
think: true,
})

console.log('Thinking:\n========\n\n' + response.message.thinking)
console.log('\nResponse:\n========\n\n' + response.message.content + '\n\n')
}

main()
36 changes: 36 additions & 0 deletions examples/thinking/thinking-streaming.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
import ollama from 'ollama'

async function main() {
const response = await ollama.chat({
model: 'deepseek-r1',
messages: [
{
role: 'user',
content: 'What is 10 + 23',
},
],
stream: true,
think: true,
})

let startedThinking = false
let finishedThinking = false

for await (const chunk of response) {
if (chunk.message.thinking && !startedThinking) {
startedThinking = true
process.stdout.write('Thinking:\n========\n\n')
} else if (chunk.message.content && startedThinking && !finishedThinking) {
finishedThinking = true
process.stdout.write('\n\nResponse:\n========\n\n')
}

if (chunk.message.thinking) {
process.stdout.write(chunk.message.thinking)
} else if (chunk.message.content) {
process.stdout.write(chunk.message.content)
}
}
}

main()
4 changes: 4 additions & 0 deletions src/interfaces.ts
Original file line number Diff line number Diff line change
@@ -56,13 +56,15 @@
format?: string | object
images?: Uint8Array[] | string[]
keep_alive?: string | number // a number (seconds) or a string with a duration unit suffix ("300ms", "1.5h", "2h45m", etc)
think?: boolean

options?: Partial<Options>
}

export interface Message {
role: string
content: string
thinking?: string
images?: Uint8Array[] | string[]
tool_calls?: ToolCall[]
}
@@ -71,7 +73,7 @@
function: {
name: string;
arguments: {
[key: string]: any;

Check warning on line 76 in src/interfaces.ts

GitHub Actions / test

Unexpected any. Specify a different type
};
};
}
@@ -84,15 +86,15 @@
type?: string;
parameters?: {
type?: string;
$defs?: any;

Check warning on line 89 in src/interfaces.ts

GitHub Actions / test

Unexpected any. Specify a different type
items?: any;

Check warning on line 90 in src/interfaces.ts

GitHub Actions / test

Unexpected any. Specify a different type
required?: string[];
properties?: {
[key: string]: {
type?: string | string[];
items?: any;

Check warning on line 95 in src/interfaces.ts

GitHub Actions / test

Unexpected any. Specify a different type
description?: string;
enum?: any[];

Check warning on line 97 in src/interfaces.ts

GitHub Actions / test

Unexpected any. Specify a different type
};
};
};
@@ -106,6 +108,7 @@
format?: string | object
keep_alive?: string | number // a number (seconds) or a string with a duration unit suffix ("300ms", "1.5h", "2h45m", etc)
tools?: Tool[]
think?: boolean

options?: Partial<Options>
}
@@ -174,6 +177,7 @@
model: string
created_at: Date
response: string
thinking?: string
done: boolean
done_reason: string
context: number[]
@@ -247,9 +251,9 @@
details: ModelDetails
messages: Message[]
modified_at: Date
model_info: Map<string, any>,

Check warning on line 254 in src/interfaces.ts

GitHub Actions / test

Unexpected any. Specify a different type
capabilities: string[],
projector_info?: Map<string, any>

Check warning on line 256 in src/interfaces.ts

GitHub Actions / test

Unexpected any. Specify a different type
}

export interface ListResponse {
Loading
Oops, something went wrong.