Extract openai API calls and retry at lowest level #3696

collijk · 2023-05-02T15:39:09Z

Background

Fourth PR in the chain to organize the LLM interaction. See:

In a previous PR, I extracted a retry decorator from the embedding calls. This PR uses the same retry decorator on an extracted call to create chat completions from the OpenAI API.

Changes

APIManager now only manages API budgets. Will be renamed and updated to use LLM structs in the next PR.
autogpt.llm.llm_utils.retry_openai_api moved to autogpt.llm.providers.openai.retry_api
Low level API methods for creating embeddings and chat completions have been extracted to autogpt.llm.providers.openai and have been appropriately decorated to consistently handle API failure cases.
autogpt.llm.llm_utils.create_chat_completion refactored to use low-level wrapped api calls and update the budget manually.
Move retry handler tests from tests.unit.test_llm_utils to tests.unit.test_openai

Documentation

N/A, this is primarily code motion

Test Plan

N/A this is primarily code motion and the code in question has unit tests.

PR Quality Checklist

My pull request is atomic and focuses on a single change.
I have thoroughly tested my changes with multiple different prompts.
I have considered potential risks and mitigations for my changes.
I have documented my changes clearly and comprehensively.
I have not snuck in any "extra" small tweaks changes

vercel · 2023-05-02T15:39:13Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
docs	⬜️ Ignored (Inspect)	Visit Preview		Jun 12, 2023 0:42am

github-actions · 2023-05-02T15:39:45Z

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

…k/Auto-GPT into refactor/extract-openai-api-calls

github-actions · 2023-05-02T15:53:43Z

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

codecov · 2023-05-02T15:59:22Z

Codecov Report

Patch coverage: 86.31% and project coverage change: +0.38 🎉

Comparison is base (49d1a5a) 70.74% compared to head (5c7b222) 71.13%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3696      +/-   ##
==========================================
+ Coverage   70.74%   71.13%   +0.38%     
==========================================
  Files          72       72              
  Lines        3504     3509       +5     
  Branches      556      556              
==========================================
+ Hits         2479     2496      +17     
+ Misses        856      843      -13     
- Partials      169      170       +1

Impacted Files	Coverage Δ
autogpt/app.py	`48.35% <0.00%> (-1.09%)`	⬇️
autogpt/llm/utils/__init__.py	`49.20% <57.14%> (-12.54%)`	⬇️
autogpt/llm/providers/openai.py	`93.50% <93.15%> (-6.50%)`	⬇️
autogpt/llm/api_manager.py	`100.00% <100.00%> (+3.57%)`	⬆️
autogpt/llm/base.py	`92.77% <100.00%> (+0.17%)`	⬆️
autogpt/memory/vector/utils.py	`83.87% <100.00%> (-1.85%)`	⬇️

... and 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

github-actions · 2023-05-02T16:12:44Z

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

p-i- · 2023-05-05T00:53:37Z

This is a mass message from the AutoGPT core team.
Our apologies for the ongoing delay in processing PRs.
This is because we are re-architecting the AutoGPT core!

For more details (and for infor on joining our Discord), please refer to:
https://github.com/Significant-Gravitas/Auto-GPT/wiki/Architecting

github-actions · 2023-05-13T21:31:01Z

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

github-actions · 2023-05-13T23:13:52Z

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

lc0rp · 2023-05-19T12:51:24Z

James, @collijk - I know you've got your head down. Is this worth trying to put in 0.3.2? Is anyone else available to resolve conflicts?

lc0rp · 2023-05-19T12:53:17Z

Seems to me, this could still be valuable even after the re-arch. Marking for maintainer-review

vercel · 2023-05-25T03:23:54Z

Deployment failed with the following error:

Resource is limited - try again in 3 minutes (more than 100, code: "api-deployments-free-per-day").

github-actions · 2023-05-25T03:24:07Z

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

github-actions · 2023-06-13T22:04:47Z

This PR exceeds the recommended size of 500 lines. Please make sure you are NOT addressing multiple issues with one PR.

github-actions · 2023-06-13T22:17:47Z

This PR exceeds the recommended size of 500 lines. Please make sure you are NOT addressing multiple issues with one PR.

github-actions · 2023-06-14T10:34:10Z

This PR exceeds the recommended size of 500 lines. Please make sure you are NOT addressing multiple issues with one PR.

github-actions · 2023-06-14T14:46:28Z

This PR exceeds the recommended size of 500 lines. Please make sure you are NOT addressing multiple issues with one PR.

Auto-GPT-Bot · 2023-06-14T14:54:47Z

You changed AutoGPT's behaviour. The cassettes have been updated and will be merged to the submodule when this Pull Request gets merged.

Extract openai API calls and retry at lowest level (Significant-Gravitas#3696) * Extract open ai api calls and retry at lowest level * Forgot a test * Gotta fix my local docker config so I can let pre-commit hooks run, ugh * fix: merge artiface * Fix linting * Update memory.vector.utils * feat: make sure resp exists * fix: raise error message if created * feat: rename file * fix: partial test fix * fix: update comments * fix: linting * fix: remove broken test * fix: require a model to exist * fix: BaseError issue * fix: runtime error * Fix mock response in test_make_agent * add 429 as errors to retry --------- Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com> Co-authored-by: Nicholas Tindle <nick@ntindle.com> Co-authored-by: Reinier van der Leer <github@pwuts.nl> Co-authored-by: Nicholas Tindle <nicktindle@outlook.com> Co-authored-by: Luke K (pr-0f3t) <2609441+lc0rp@users.noreply.github.com> Co-authored-by: Merwane Hamadi <merwanehamadi@gmail.com> Add 16k gpt model Refactor module layout of command classes Remove nonessential commands Basic working version of functions support Make required args required Rewrite write_file to work with new arguments Replace gpt-3.5-turbo by gpt-3.5-turbo-0613 gpt-3.5-turbo-16k-0613 everywhere models that are not 0613 are gone Refactor message history and cycling Ask users to update their models Fix issues with token counting Fix issues with history trimming Remove self feedback and rename user_input to triggering_prompt when necessary Fix json parsing issues Document the steps of the interaction loop Fix issues with parsing reply Refactor MessageCycle Up word limit in prompt Change command prompt constraint to reference functions Fix {} arguments error Fix issues with command arguments Fix issues with function arguments attached to assistant responses Evolve prompts to improve AI understand of function calls Fix issues with command result serialization Fix read_file path bug Fix a bunch of tests Abandon function memory role for now Fix history and summarization issues and its tests Change task_complete so the AI doesn't call it accidentally Remove agent manager as it is unused and now broken Fix token encoding model edge cases Fix openai provider tests Update setup test Cleanup imports Trigger CI Fix LogCycle and stringify commands Prompt change Reduce diff size due to moving functions Include missing test

Extract openai API calls and retry at lowest level (#3696) * Extract open ai api calls and retry at lowest level * Forgot a test * Gotta fix my local docker config so I can let pre-commit hooks run, ugh * fix: merge artiface * Fix linting * Update memory.vector.utils * feat: make sure resp exists * fix: raise error message if created * feat: rename file * fix: partial test fix * fix: update comments * fix: linting * fix: remove broken test * fix: require a model to exist * fix: BaseError issue * fix: runtime error * Fix mock response in test_make_agent * add 429 as errors to retry --------- Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com> Co-authored-by: Nicholas Tindle <nick@ntindle.com> Co-authored-by: Reinier van der Leer <github@pwuts.nl> Co-authored-by: Nicholas Tindle <nicktindle@outlook.com> Co-authored-by: Luke K (pr-0f3t) <2609441+lc0rp@users.noreply.github.com> Co-authored-by: Merwane Hamadi <merwanehamadi@gmail.com> Add 16k gpt model Refactor module layout of command classes Remove nonessential commands Basic working version of functions support Make required args required Rewrite write_file to work with new arguments Replace gpt-3.5-turbo by gpt-3.5-turbo-0613 gpt-3.5-turbo-16k-0613 everywhere models that are not 0613 are gone Refactor message history and cycling Ask users to update their models Fix issues with token counting Fix issues with history trimming Remove self feedback and rename user_input to triggering_prompt when necessary Fix json parsing issues Document the steps of the interaction loop Fix issues with parsing reply Refactor MessageCycle Up word limit in prompt Change command prompt constraint to reference functions Fix {} arguments error Fix issues with command arguments Fix issues with function arguments attached to assistant responses Evolve prompts to improve AI understand of function calls Fix issues with command result serialization Fix read_file path bug Fix a bunch of tests Abandon function memory role for now Fix history and summarization issues and its tests Change task_complete so the AI doesn't call it accidentally Remove agent manager as it is unused and now broken Fix token encoding model edge cases Fix openai provider tests Update setup test Cleanup imports Trigger CI Fix LogCycle and stringify commands Prompt change Reduce diff size due to moving functions Include missing test Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

Extract openai API calls and retry at lowest level (Significant-Gravitas#3696) * Extract open ai api calls and retry at lowest level * Forgot a test * Gotta fix my local docker config so I can let pre-commit hooks run, ugh * fix: merge artiface * Fix linting * Update memory.vector.utils * feat: make sure resp exists * fix: raise error message if created * feat: rename file * fix: partial test fix * fix: update comments * fix: linting * fix: remove broken test * fix: require a model to exist * fix: BaseError issue * fix: runtime error * Fix mock response in test_make_agent * add 429 as errors to retry --------- Co-authored-by: k-boikov <64261260+k-boikov@users.noreply.github.com> Co-authored-by: Nicholas Tindle <nick@ntindle.com> Co-authored-by: Reinier van der Leer <github@pwuts.nl> Co-authored-by: Nicholas Tindle <nicktindle@outlook.com> Co-authored-by: Luke K (pr-0f3t) <2609441+lc0rp@users.noreply.github.com> Co-authored-by: Merwane Hamadi <merwanehamadi@gmail.com> Add 16k gpt model Refactor module layout of command classes Remove nonessential commands Basic working version of functions support Make required args required Rewrite write_file to work with new arguments Replace gpt-3.5-turbo by gpt-3.5-turbo-0613 gpt-3.5-turbo-16k-0613 everywhere models that are not 0613 are gone Refactor message history and cycling Ask users to update their models Fix issues with token counting Fix issues with history trimming Remove self feedback and rename user_input to triggering_prompt when necessary Fix json parsing issues Document the steps of the interaction loop Fix issues with parsing reply Refactor MessageCycle Up word limit in prompt Change command prompt constraint to reference functions Fix {} arguments error Fix issues with command arguments Fix issues with function arguments attached to assistant responses Evolve prompts to improve AI understand of function calls Fix issues with command result serialization Fix read_file path bug Fix a bunch of tests Abandon function memory role for now Fix history and summarization issues and its tests Change task_complete so the AI doesn't call it accidentally Remove agent manager as it is unused and now broken Fix token encoding model edge cases Fix openai provider tests Update setup test Cleanup imports Trigger CI Fix LogCycle and stringify commands Prompt change Reduce diff size due to moving functions Include missing test Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com>

Extract open ai api calls and retry at lowest level

a912131

Merge branch 'master' into refactor/extract-openai-api-calls

579de97

github-actions bot added the size/xl label May 2, 2023

collijk added this to the v0.3.1 Release milestone May 2, 2023

collijk added 2 commits May 2, 2023 08:52

Forgot a test

ad7cd3d

Merge branch 'refactor/extract-openai-api-calls' of github.com:collij…

840ba44

…k/Auto-GPT into refactor/extract-openai-api-calls

Gotta fix my local docker config so I can let pre-commit hooks run, ugh

4294ca4

Sudo-psc previously approved these changes May 4, 2023

View reviewed changes

Merge branch 'master' into refactor/extract-openai-api-calls

dab77b1

vercel bot temporarily deployed to Preview May 13, 2023 21:31 Inactive

github-actions bot added the conflicts Automatically applied to PRs with merge conflicts label May 13, 2023

p-i- modified the milestones: v0.3.1 Release, v0.3.2-release May 14, 2023

lc0rp added the ready-for-maintainer Catalyst has decided this PR looks ready for a maintainer to merge label May 19, 2023

ntindle added 2 commits May 24, 2023 22:23

Merge branch 'master' into pr/3696

338b3ca

fix: merge artiface

bf61234

ntindle dismissed Sudo-psc’s stale review via bf61234 May 25, 2023 03:23

github-actions bot removed the conflicts Automatically applied to PRs with merge conflicts label May 25, 2023

Fix mock response in test_make_agent

26a7822

erik-megarad mentioned this pull request Jun 14, 2023

OpenAI Functions Support #4683

Merged

Merge branch 'master' into refactor/extract-openai-api-calls

6255d5f

lc0rp modified the milestones: v0.4.1 Release, v0.4.2 Release Jun 14, 2023

add 429 as errors to retry

5c7b222

Auto-GPT-Bot added the behaviour change label Jun 14, 2023

waynehamadi approved these changes Jun 14, 2023

View reviewed changes

waynehamadi merged commit 6e6e7fc into Significant-Gravitas:master Jun 14, 2023
16 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Extract openai API calls and retry at lowest level #3696

Extract openai API calls and retry at lowest level #3696

collijk commented May 2, 2023 •

edited

vercel bot commented May 2, 2023 •

edited

github-actions bot commented May 2, 2023

github-actions bot commented May 2, 2023

codecov bot commented May 2, 2023 •

edited

github-actions bot commented May 2, 2023

p-i- commented May 5, 2023

github-actions bot commented May 13, 2023

github-actions bot commented May 13, 2023

lc0rp commented May 19, 2023

lc0rp commented May 19, 2023

vercel bot commented May 25, 2023

github-actions bot commented May 25, 2023

github-actions bot commented Jun 13, 2023

github-actions bot commented Jun 13, 2023

github-actions bot commented Jun 14, 2023

github-actions bot commented Jun 14, 2023

Auto-GPT-Bot commented Jun 14, 2023

Extract openai API calls and retry at lowest level #3696

Extract openai API calls and retry at lowest level #3696

Conversation

collijk commented May 2, 2023 • edited

Background

Changes

Documentation

Test Plan

PR Quality Checklist

vercel bot commented May 2, 2023 • edited

github-actions bot commented May 2, 2023

github-actions bot commented May 2, 2023

codecov bot commented May 2, 2023 • edited

Codecov Report

github-actions bot commented May 2, 2023

p-i- commented May 5, 2023

github-actions bot commented May 13, 2023

github-actions bot commented May 13, 2023

lc0rp commented May 19, 2023

lc0rp commented May 19, 2023

vercel bot commented May 25, 2023

github-actions bot commented May 25, 2023

github-actions bot commented Jun 13, 2023

github-actions bot commented Jun 13, 2023

github-actions bot commented Jun 14, 2023

github-actions bot commented Jun 14, 2023

Auto-GPT-Bot commented Jun 14, 2023

collijk commented May 2, 2023 •

edited

vercel bot commented May 2, 2023 •

edited

codecov bot commented May 2, 2023 •

edited