Add rate limiting to websearch and title summary #433
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nsarrazin
added a commit
that referenced
this pull request
Sep 20, 2023
nsarrazin
added a commit
that referenced
this pull request
Sep 20, 2023
* Bump mongodb from 5.3.0 to 5.8.0 Bumps [mongodb](https://github.com/mongodb/node-mongodb-native) from 5.3.0 to 5.8.0. - [Release notes](https://github.com/mongodb/node-mongodb-native/releases) - [Changelog](https://github.com/mongodb/node-mongodb-native/blob/v5.8.0/HISTORY.md) - [Commits](mongodb/node-mongodb-native@v5.3.0...v5.8.0) --- updated-dependencies: - dependency-name: mongodb dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * Store IP in messageEvents * IP based rate limit * Revert "IP based rate limit" This reverts commit 87c6937. * ip rate limit * move rate limit event to top * Add rate limiting to websearch and title summary (#433) * [Websearch] update (#427) * Fix reuqest body * update webSearchQueryPromptTemplate * update generate google query parser * Add today's date to google search query creator * crawl top stories if exts; remove answer_box & knowledgeGraph * Create paragraph chunks from top articles * flattened paragprah chunks * update status texts * add gradio client * call gradio app for RAG * Web scrape only "p, li, span" els * add MAX_N_CHUNKS * gradio result typing * parse only <p> elements * rm dev change * update typing WebSearch * buld RAG prompt * Rm dev change * change websearch context msg from user to assisntat type * use hosted gradio app * fix lint * prompt engineering * more prompt engineering * MAX_N_PAGES_SCRAPE = 10 * better error msg * more prompt engineering * revert websearch prompt to previous * rm `top_stories` from websearch as the results are not good * Stop using gradio client, use regular fetch * chore * Rm websearchsummary references as it is no longer used * update readme * Apply suggestions from code review Co-authored-by: Julien Chaumond <julien@huggingface.co> * Use tfjs to do embeddings in server node * fix websearch component disapperar after finishing generation * Show sources of closest embeddings used in RAG * fix prompting and also add current date * add comment * comment for search query * sources * hide www * using hostname direclty * Show successful web pages instead of failed ones * rm noisy messages * google query generation using previous messaages as context * handle falcon generation * bring back Browsing webpage msg --------- Co-authored-by: Julien Chaumond <julien@huggingface.co> Co-authored-by: Victor Mustar <victor.mustar@gmail.com> * bump to 0.6.0 (#434) * Update README.md (#435) * Update README.md * add description of websearch on readme * Apply suggestions from code review Co-authored-by: Victor Muštar <victor.mustar@gmail.com> * Update README.md --------- Co-authored-by: Mishig Davaadorj <dmishig@gmail.com> Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu> * Mobile: fix model selection (#448) * adjustments and mobile modal * use dvh unit * margin * fix lint on main * Add latex support with marked-katex-extension (#450) * Add latex support with marked-katex-extension * Add renderer * Fix marked default option problem * Fix linting error * Fix lock error --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Nathan Sarrazin <sarrazin.nathan@gmail.com> Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu> Co-authored-by: Julien Chaumond <julien@huggingface.co> Co-authored-by: Victor Mustar <victor.mustar@gmail.com> Co-authored-by: Mishig Davaadorj <dmishig@gmail.com> Co-authored-by: Blanchon <julien@blanchon.cc>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.