Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add SearchApi tools #4648

Merged
merged 3 commits into from
May 30, 2024
Merged

Add SearchApi tools #4648

merged 3 commits into from
May 30, 2024

Conversation

SebastjanPrachovskij
Copy link
Contributor

Description

We have integrated a new tool - SearchApi. Right now we added Google, Google Jobs, Google News, and YouTube Transcripts engines.

  • This is our initial PR; we want to improve it later based on the users' feedback. Most likely changes will affect how the final results string is being built.
  • Feel free to assign @SebastjanPrachovskij as a main reviewer for any SearchApi-related searches. We will be glad to help and support dify development.

Type of Change

Please delete options that are not relevant.

  • New feature (non-breaking change which adds functionality)

How Has This Been Tested?

  • By creating a workflow in the studio.

Suggested Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings
  • I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods
  • optional I have made corresponding changes to the documentation
  • optional I have added tests that prove my fix is effective or that my feature works
  • optional New and existing unit tests pass locally with my changes

@dosubot dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. 💪 enhancement New feature or request 🔨 feat:tools Tools for agent, function call related stuff. labels May 24, 2024
@SebastjanPrachovskij
Copy link
Contributor Author

Hey Dify team!

We would like to include an example of a workflow where the YouTube Transcripts API is used to retrieve text from a YouTube video, and then an LLM model is used to summarize the text. More details can be found in the LangChainJS Summarization Example

How could we create such an example with LLM?

@crazywoola crazywoola self-requested a review May 25, 2024 05:03
@crazywoola
Copy link
Member

Hello thanks for your contribution.

We are working on restructure codes and UI of the tools. It will be release in a few days. So I will check this pr by that time.

See the attachment below.
image

crazywoola
crazywoola requested changes May 28, 2024
View reviewed changes
Copy link
Member

@crazywoola crazywoola left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello, we have release the new version v0.6.9.
&
Please run ./dev/reformat in root dir as well.

@SebastjanPrachovskij The rest part LGTM. Thanks again for your contribution. :)

api/core/tools/provider/builtin/searchapi/searchapi.py Outdated Show resolved Hide resolved
api/core/tools/provider/builtin/searchapi/searchapi.py Show resolved Hide resolved
@dosubot dosubot bot added size:XXL This PR changes 1000+ lines, ignoring generated files. and removed size:XL This PR changes 500-999 lines, ignoring generated files. labels May 28, 2024
@SebastjanPrachovskij SebastjanPrachovskij changed the base branch from main to feat/move-tenant-id-to-jwt May 28, 2024 12:01
@SebastjanPrachovskij SebastjanPrachovskij changed the base branch from feat/move-tenant-id-to-jwt to main May 28, 2024 12:01
@dosubot dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:XXL This PR changes 1000+ lines, ignoring generated files. labels May 28, 2024
@SebastjanPrachovskij
Copy link
Contributor Author

Hey @crazywoola,

Thanks for the review. I addressed suggested changes, updated the SearchApi description, and returned results based on result_type.

I think it is ready to be reviewed again 😊

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label May 29, 2024
@crazywoola
Copy link
Member

crazywoola commented May 29, 2024 •
edited
Loading 

Error: /workspace/api/core/tools/provider/builtin/searchapi/tools/youtube_transcripts.yaml:22:22: [error] too many spaces after colon (colons)

And there is a lint error as well. @SebastjanPrachovskij

@dosubot dosubot bot added size:XXL This PR changes 1000+ lines, ignoring generated files. and removed size:XL This PR changes 500-999 lines, ignoring generated files. labels May 29, 2024
@crazywoola crazywoola requested a review from Yeuoly May 29, 2024 15:33
Yeuoly
Yeuoly approved these changes May 30, 2024
View reviewed changes
Copy link
Collaborator

@Yeuoly Yeuoly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Yeuoly
Copy link
Collaborator

Yeuoly commented May 30, 2024

Thanks for contributions!

@Yeuoly Yeuoly merged commit b10e67b into langgenius:main May 30, 2024
7 checks passed
vicwang3 pushed a commit to vicwang3/dify that referenced this pull request May 30, 2024
Merge remote-tracking branch 'origin/main' into fix/brrop
55d0b69 
* origin/main: (93 commits)
  chore: node help link (langgenius#4795)
  Add SearchApi tools (langgenius#4648)
  chore: improve node user experience (langgenius#4792)
  fix: in tool and http node of iteration can not show item var correctly (langgenius#4791)
  feat: Add logging warning when MAIL_TYPE is not set (langgenius#4771)
  fix organize agent's history messages without recalculating tokens (langgenius#4324)
  feat: support define tags in tool yaml (langgenius#4763)
  feat: update ernie model (langgenius#4756)
  fix: incorrect workflow max call depth (langgenius#4759)
  feat: support baichuan3 turbo, baichuan3 turbo 128k, and baichuan4 (langgenius#4762)
  fix: confusing chart description (langgenius#4760)
  style: fix annotation panel display misalignment (langgenius#4750)
  Add WORKFLOW_CALL_MAX_DEPTH env var. (langgenius#4713)
  Fix/4742 ollama num gpu option not consistent with allowed values (langgenius#4751)
  style: the 'all' of add tool panel should contain workflow tools (langgenius#4755)
  fix: Corrected schema link in model_runtime's README.md (langgenius#4757)
  fix: optimize sticky header styles z-index in tools - ProviderList component (langgenius#4746)
  fix: workflow run sequence number slow sql (langgenius#4737)
  Show tool i18n name on chat pannel (langgenius#4724)
  fix: Correct context size for banchuan2-53b and banchuan2-turbo (langgenius#4721)
  ...
Sakura4036 pushed a commit to Sakura4036/dify that referenced this pull request Jun 4, 2024
@SebastjanPrachovskij @Sakura4036
Add SearchApi tools (langgenius#4648)
a1b542c
@takatost takatost mentioned this pull request Jun 4, 2024
Merged
@bowenliang123 bowenliang123 mentioned this pull request Jun 16, 2024
Merged
14 tasks
dengpeng pushed a commit to dengpeng/dify that referenced this pull request Jun 16, 2024
@SebastjanPrachovskij @dengpeng
Add SearchApi tools (langgenius#4648)
1719072
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Yeuoly Yeuoly Yeuoly approved these changes

@crazywoola crazywoola crazywoola approved these changes

Assignees
No one assigned
Labels
💪 enhancement New feature or request 🔨 feat:tools Tools for agent, function call related stuff. lgtm This PR has been approved by a maintainer size:XXL This PR changes 1000+ lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@SebastjanPrachovskij @crazywoola @Yeuoly