Skip to content

Cached Google/DDG-Search to reduce rate limit penalty #2338#2460

Closed
janpdevops wants to merge 11 commits intoagno-agi:mainfrom
janpdevops:feature/issue2338-google-cache
Closed

Cached Google/DDG-Search to reduce rate limit penalty #2338#2460
janpdevops wants to merge 11 commits intoagno-agi:mainfrom
janpdevops:feature/issue2338-google-cache

Conversation

@janpdevops
Copy link

@janpdevops janpdevops commented Mar 19, 2025

Cached Google/DDG-Search to reduce rate limit penalty Issue #2338

See #2338

  • Summary of changes: I added a cached version of the Google Search. Also, an example on how to use it is included in cookbook/agent_concepts/other/google_cache_demo.py
  • Related issues: Cached Google/DDG-Search to reduce rate limit penalty #2338
  • Motivation and context:When developing, you will pretty soon hit a rate limit with DDG and, later with google. Since the requests are repetitive, you can avoid this with a cache.
  • Environment or dependencies: You will need a database as in the pgvector example.
  • Impact on metrics: (If applicable) Describe changes in any metrics or performance benchmarks.

Fixes # (2338)


Type of change

Please check the options that are relevant:

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Model update (Addition or modification of models)
  • Other (please describe):

Checklist

  • Adherence to standards: Code complies with Agno’s style guidelines and best practices.
  • [ ]
  • Formatting and validation: You have run ./scripts/format.sh and ./scripts/validate.sh to ensure code is formatted and linted. No, please add the .bat files again.
  • Self-review completed: A thorough review has been performed by the contributor(s).
  • Documentation: Docstrings and comments have been added or updated for any complex logic.
  • Examples and guides: Relevant cookbook examples have been included or updated (if applicable).
  • Tested in a clean environment: Changes have been tested in a clean environment to confirm expected behavior.
  • Tests (optional): Tests have been added or updated to cover any new or changed functionality.

Additional Notes

I am still new to python, so I am open to suggestions. I am willing to help the project along.

@janpdevops janpdevops requested a review from a team as a code owner March 19, 2025 22:21
@dirkbrnd
Copy link
Contributor

Love the contribution. We think it should be done a little differently, i'd rather have an in-memory cache managed by a decorator, which we can easily add to more functions.
We just haven't gotten around to it.

@janpdevops
Copy link
Author

@dirkbrnd
I think I could create a wrapper. That would be used in the Agent similar like this:
tools=[CachedSearchTool(
wrappedTool=GoogleSearchTools(
fixed_max_results=10,
fixed_language="en"
),
),
db_url="postgresql://ai:ai@localhost:5532",
table_name="search_cache",
)],

You could use Duck Duck Go or whatever you like instead of google.

@janpdevops
Copy link
Author

Hi, I changed the cache to a wrapper of another search tool. It worked with the Duck Duck Go and Google.

@janpdevops
Copy link
Author

Hi @dirkbrnd,
I choose the database instead of a in memory cache - this is useful, when you run the small tutorial examples (that just perform a couple of steps and terminate). It was useful for me, when I had to run the same code again and again to experiment. If you think this is a step, I'd be happy to merge.

@dirkbrnd
Copy link
Contributor

Hi @janpdevops
I have since introduced caching on some of our tools (I can easily add to more). Check out the updates to DDG tools for example.
If that works for you, I suggest we close this PR. I'll do so in the mean time, but please let me know if that is not working.

@dirkbrnd dirkbrnd closed this Mar 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants