Skip to content

Add configurable context window size#165

Merged
jehna merged 1 commit intomainfrom
add/configurable-context-window-size
Oct 18, 2024
Merged

Add configurable context window size#165
jehna merged 1 commit intomainfrom
add/configurable-context-window-size

Conversation

@jehna
Copy link
Copy Markdown
Owner

@jehna jehna commented Oct 18, 2024

It seems that context window of 200 was way too small, I saw quite many _________id style renamess in the issues. Using a bigger context window should fix this issue, and mmaking the window configurable should help to debug any future issues.

It seems that context window of 200 was way too small, I saw quite many
`_________id` style renamess in the issues. Using a bigger context
window should fix this issue, and mmaking the window configurable
should help to debug any future issues.

This value is a balance between giing enough context for the LLM to
infer the meaning correctly, and overwhelming the LLM with too much
noise.
@jehna jehna added bug Something isn't working enhancement New feature or request test-openai Run openai tests towards this PR and removed bug Something isn't working labels Oct 18, 2024
@jehna jehna force-pushed the add/configurable-context-window-size branch from 81de649 to 22b6e4f Compare October 18, 2024 20:23
@jehna jehna merged commit ad6839e into main Oct 18, 2024
@jehna jehna deleted the add/configurable-context-window-size branch October 18, 2024 20:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request test-openai Run openai tests towards this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant