Improvement in web commands functionalities

[amirdaaee]

Background

this pr is to address feature requests in #7067 and #7066. in summary it adds two functionalities:

• adding support for alternative duckduckgo_search SDK supported backends (as described here).
• support for proxy in selenium for read_web command

Changes 🏗️

• autogpts/autogpt/autogpt/config/config.py: added duckduckgo_backend and selenium_proxy attributes to Config class to control new feature behavior
• autogpts/autogpt/autogpt/commands/web_search.py: added backend argument in calling for text() method of DDGS in web_search function
• autogpts/autogpt/autogpt/commands/web_selenium.py: added --proxy-server argument in web-drive options
• autogpts/autogpt/.env.template: added documents about duckduckgo_backend and selenium_proxy variables

PR Quality Scorecard ✨

• Have you used the PR description template?   +2 pts
• Is your pull request atomic, focusing on a single change?   +5 pts
• Have you linked the GitHub issue(s) that this PR addresses?   +5 pts
• Have you documented your changes clearly and comprehensively?   +5 pts
• Have you changed or added a feature?   -4 pts
  • Have you added/updated corresponding documentation?   +4 pts
  • Have you added/updated corresponding integration tests?   +5 pts
• Have you changed the behavior of AutoGPT?   -5 pts
  • Have you also run agbenchmark to verify that these changes do not regress performance?   +10 pts

[netlify]

✅ Deploy Preview for auto-gpt-docs canceled.

Name	Link
🔨 Latest commit	3fcee82
🔍 Latest deploy log	https://app.netlify.com/sites/auto-gpt-docs/deploys/660d1e31f3c76a0009793b36

[codecov]

Codecov Report

Attention: Patch coverage is 71.42857% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 45.51%. Comparing base (fb8ed0b) to head (3fcee82).
Report is 15 commits behind head on master.

Files	Patch %	Lines
autogpts/autogpt/autogpt/commands/web_selenium.py	33.33%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7068      +/-   ##
==========================================
+ Coverage   45.49%   45.51%   +0.01%     
==========================================
  Files         139      139              
  Lines        6528     6534       +6     
  Branches      917      918       +1     
==========================================
+ Hits         2970     2974       +4     
- Misses       3408     3409       +1     
- Partials      150      151       +1     

Flag	Coverage Δ
Linux	45.40% <71.42%> (+0.01%)	⬆️
Windows	43.65% <71.42%> (+0.02%)	⬆️
autogpt-agent	45.48% <71.42%> (+0.01%)	⬆️
macOS	44.85% <71.42%> (+0.02%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[github-actions]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[Swiftyos]

@CodiumAI-Agent /review

[CodiumAI-Agent]

PR Review

⏱️ Estimated effort to review [1-5]	3, because the PR involves multiple files and introduces new functionalities which require understanding the existing configuration and command structures. The changes are moderate in complexity, involving backend integrations and proxy settings which need careful review to ensure they are implemented correctly and securely.
🧪 Relevant tests	No
🔍 Possible issues	Possible Bug: The duckduckgo_backend variable is fetched from agent.legacy_config but there is no check to ensure that legacy_config or duckduckgo_backend is not None before using it. This could lead to AttributeError if legacy_config is not properly configured.
	Possible Bug: The proxy configuration in web_selenium.py directly appends the proxy server argument without validating the format or existence of the proxy variable. This might lead to incorrect browser behavior if the proxy string is malformed.
🔒 Security concerns	No

Code feedback:

---

relevant file	autogpts/autogpt/autogpt/commands/web_search.py
suggestion	Add a null check for ddc_backend before using it in the DDGS().text method to prevent potential runtime errors if the configuration is missing. [important]
relevant line	query, max_results=num_results, backend=ddc_backend

---

relevant file	autogpts/autogpt/autogpt/commands/web_selenium.py
suggestion	Validate the format of the proxy variable to ensure it includes the necessary protocol (e.g., http:// or https://). This will prevent potential issues with browser proxy configuration. [important]
relevant line	options.add_argument("--proxy-server=%s" % proxy)

---

---

✨ Review tool usage guide:

---

Overview:
The review tool scans the PR code changes, and generates a PR review which includes several types of feedbacks, such as possible PR issues, security threats and relevant test in the PR. More feedbacks can be added by configuring the tool.

The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on any PR.

• When commenting, to edit configurations related to the review tool (pr_reviewer section), use the following template:

/review --pr_reviewer.some_config1=... --pr_reviewer.some_config2=...

• With a configuration file, use the following template:

[pr_reviewer]
some_config1=...
some_config2=...

See the review usage page for a comprehensive guide on using this tool.