-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enable deep searching of specific URLs #92
Comments
Try this: https://github.com/mendableai/firecrawl |
Firecrawl is pretty slow when I tested it |
Ahh nevermind, the crawl api which scrapes multiple pages is slow, just scraping one page is decent |
It's a good try!
|
Firecrawl uses playwright + turndown 😆 I'll give it a shot, already got the researcher able to refer to the previous sources it pulled, just that the types are a bit tricky. I'll work on this one first and I'll see if I can get back to video section next week, it's been busy at work |
jina seems to me to be a tad bit faster btw, if you wanna try it out |
Thank you. I was stuck with the chat history feature. I'll be able to start working on a different task soon. |
Hey @miurla, fixed the types related to enabling follow up conversations and also found a way to enable function calling with groq so now I'm moving forward with adding FireCrawl, planning to push a PR by Sunday! |
@albertdbio Wow, that's great. Is the function calling an implementation that uses groq's sdk to replace the researcher's model? Looking forward to the PR. |
@albertdbio I've merged significant changes, so it's best to develop from main. |
@miurla not using the groq sdk, it turns out the vercel ai sdk needed to be upgraded to a minor version. That minor version set's content as optional in the OpenAI zod schema. Groq doesn't return content when doing function calls so it would throw an invalid response error. Going to test it out with the inquire agent to verify that it's working. |
Got caught up with work over the weekend. Will continue chipping away at it this week. First I'll rebase on to main. |
I'm working on this feature. |
When a specific URL is included in the query, retrieve the data from that URL instead of searching.
The text was updated successfully, but these errors were encountered: