Make Benchmarks write cassettes without using them

[waynehamadi]

Background

Benchmarks retry tests once if they fail. If they use cassettes, the second test is doomed to fail as well.

Changes

• add --record-mode=all to benchmarks.yml so that we record interactions but without replaying them

Documentation

Test Plan

PR Quality Checklist

• My pull request is atomic and focuses on a single change.
• I have thoroughly tested my changes with multiple different prompts.
• I have considered potential risks and mitigations for my changes.
• I have documented my changes clearly and comprehensively.
• I have not snuck in any "extra" small tweaks changes.
• I have run the following commands against my code to ensure it passes our linters:

  black .
  isort .
  mypy
  autoflake --remove-all-unused-imports --recursive --ignore-init-module-imports --ignore-pass-after-docstring autogpt tests --in-place

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
docs	⬜️ Ignored (Inspect)			Jun 12, 2023 0:48am

[netlify]

✅ Deploy Preview for auto-gpt-docs canceled.

Name	Link
🔨 Latest commit	7214b2e
🔍 Latest deploy log	https://app.netlify.com/sites/auto-gpt-docs/deploys/6488b1d0c487420008f61736

[codecov]

Codecov Report

Patch coverage has no change and project coverage change: +0.20 🎉

Comparison is base (07d9b58) 70.43% compared to head (7214b2e) 70.64%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4664      +/-   ##
==========================================
+ Coverage   70.43%   70.64%   +0.20%     
==========================================
  Files          70       70              
  Lines        3437     3437              
  Branches      547      547              
==========================================
+ Hits         2421     2428       +7     
+ Misses        848      837      -11     
- Partials      168      172       +4     

see 2 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[github-actions]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[github-actions]

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

[Auto-GPT-Bot]

You changed AutoGPT's behaviour. The cassettes have been updated and will be merged to the submodule when this Pull Request gets merged.

[dschonholtz]

Yay run more tests from scratch. use cassettes less. get more data