Set up unified pre-commit + CI w/ linting + type checking & FIX EVERYTHING

[Pwuts]

Goals

• Set up proper pre-commit and CI for the project as a whole
• Add type checking to pre-commit and CI
• Fix any existing linting/typing issues that will otherwise cause the checks to fail

Changes

• FIX ALL LINT/TYPE ERRORS IN AUTOGPT, FORGE, AND BENCHMARK

Linting

• Clean up linter configs for autogpt, forge, and benchmark
• Add type checking with Pyright
• Create unified pre-commit config
• Create unified linting and type checking CI workflow

Testing

• Synchronize CI test setups for autogpt, forge, and benchmark
  • Add missing pytest-cov to benchmark dependencies
• Mark GCS tests as slow to speed up pre-commit test runs
• Repair forge test suite
  • Add AgentDB.close() method for test DB teardown in db_test.py
  • Use actual temporary dir instead of forge/test_workspace/
• Move left-behind dependencies for moved forge-code to from autogpt to forge

Notable type changes

• Replace uses of ChatModelProvider by MultiProvider

• Removed unnecessary exports from various init.py

• Simplify FileStorage.open_file signature by removing IOBase from return type union

  • Implement S3BinaryIOWrapper(BinaryIO) type interposer for S3FileStorage

• Expand overloads of GCSFileStorage.open_file for improved typing of read and write modes

  Had to silence type checking for the extra overloads, because (I think) Pyright is reporting a false-positive:
  reportIncompatibleMethodOverride false positive on extra overloads in subclass microsoft/pyright#8007

• Change count_tokens, get_tokenizer, count_message_tokens methods on ModelProviders from class methods to instance methods

• Move CompletionModelFunction.schema method -> helper function format_function_def_for_openai in forge.llm.providers.openai

• Rename ModelProvider -> BaseModelProvider

• Rename ChatModelProvider -> BaseChatModelProvider

• Add type ChatModelProvider which is a union of all subclasses of BaseChatModelProvider

Removed rather than fixed

• Remove deprecated and broken autogpt/agbenchmark_config/benchmarks.py
• Various base classes and properties on base classes in forge.llm.providers.schema and forge.models.providers

Fixes for other issues that came to light

• Clean up forge.agent_protocol.api_router, forge.agent_protocol.database, and forge.agent.agent

• Add fallback behavior to ImageGeneratorComponent

  • Remove test for deprecated failure behavior

• Fix agbenchmark.challenges.builtin challenge exclusion mechanism on Windows

• Fix _tool_calls_compat_extract_calls in forge.llm.providers.openai

• Add support for any (= no type specified) in JSONSchema.typescript_type

[netlify]

✅ Deploy Preview for auto-gpt-docs canceled.

Name	Link
🔨 Latest commit	2b74a5e
🔍 Latest deploy log	https://app.netlify.com/sites/auto-gpt-docs/deploys/66554898d35b9e0008d77ab1

[codecov]

Codecov Report

Attention: Patch coverage is 55.93496% with 271 lines in your changes are missing coverage. Please review.

Project coverage is 39.75%. Comparing base (2c13a27) to head (2b74a5e).

Files	Patch %	Lines
forge/forge/file_storage/s3.py	0.00%	62 Missing ⚠️
forge/forge/agent_protocol/api_router.py	8.33%	32 Missing and 1 partial ⚠️
forge/forge/file_storage/gcs.py	0.00%	30 Missing ⚠️
forge/forge/models/json_schema.py	21.42%	22 Missing ⚠️
forge/forge/llm/providers/groq.py	50.00%	12 Missing ⚠️
forge/forge/llm/providers/openai.py	52.00%	11 Missing and 1 partial ⚠️
forge/forge/agent/agent.py	66.66%	7 Missing and 2 partials ⚠️
benchmark/agbenchmark/conftest.py	0.00%	8 Missing ⚠️
benchmark/agbenchmark/app.py	0.00%	7 Missing ⚠️
forge/forge/file_storage/local.py	73.07%	7 Missing ⚠️
... and 26 more

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7171      +/-   ##
==========================================
+ Coverage   36.05%   39.75%   +3.70%     
==========================================
  Files          19      104      +85     
  Lines        1273     6756    +5483     
  Branches      182     1000     +818     
==========================================
+ Hits          459     2686    +2227     
- Misses        786     3962    +3176     
- Partials       28      108      +80     

Flag	Coverage Δ
Linux	39.75% <55.93%> (+3.70%)	⬆️
Windows	41.86% <56.50%> (+5.94%)	⬆️
agbenchmark	33.80% <48.27%> (?)
autogpt-agent	36.02% <42.30%> (-0.04%)	⬇️
forge	44.21% <57.43%> (?)
macOS	39.75% <55.93%> (+3.70%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.