Support Fireworks batching (#8) #12052

ZixinYang · 2023-10-19T21:12:18Z

Description

Add _generate and _agenerate to support Fireworks batching.
Add stop words test cases
Opt out retry mechanism

Issue - Not applicable
Dependencies - None
Tag maintainer - @baskaryan

vercel · 2023-10-19T21:12:23Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
langchain	⬜️ Ignored (Inspect)	Visit Preview		Oct 26, 2023 6:37pm

eyurtsev · 2023-10-19T21:25:21Z

libs/langchain/langchain/chat_models/fireworks.py

@@ -55,7 +67,7 @@ def _convert_delta_to_message_chunk(
    elif role or default_class == ChatMessageChunk:
        return ChatMessageChunk(content=content, role=role)
    else:
-        return default_class(content=content)
+        return default_class(type="", content=content)


type should not be specified -- it's assigned by default depending on the message class

I got a linter error says argument type is missing. Let me update it later to see if it passes the CI here.

The type should not be provided -- an empty string is incorrect here as it'll break serialization of messages if these messages are every serialized directly using pydantic.

Could you include the linter message here? And perhaps use a type: ignore[ speciifc error] to mute it -- sounds like the linter may be incorrect

Have passed CI here without specifying value to type, so should be the linter I used was incorrect.

libs/langchain/langchain/chat_models/fireworks.py

eyurtsev · 2023-10-19T21:33:17Z

libs/langchain/langchain/llms/fireworks.py

@@ -179,6 +228,74 @@ async def _completion_with_retry(**kwargs: Any) -> Any:
    return await _completion_with_retry(**kwargs)


+def completion_with_retry_batching(


we handle retries via -- fireworks should not contain any logic for retries

https://api.python.langchain.com/en/latest/schema.runnable/langchain.schema.runnable.retry.RunnableRetry.html

fireworks.client has own error handling wrapper
e.g. in _create_retry_decorator

fireworks.client.error.RateLimitError, fireworks.client.error.ServiceUnavailableError,

Can we keep it to use our fireworks library and use the custom error handling?

RunnableRetry allows using custom exceptions;

Once Fireworks inherits from BaseLLM, one can do this:

llm = Fireworks(..).with_retry() # Will automatically retry any exception 3 times

retryable_exceptions = (fireworks.client.error.RateLimitError, fireworks.client.error.ServiceUnavailableError) llm = Fireworks(..).with_retry(retry_if_exception_type=retryable_exceptions) # Will automatically retry any exception 3 times

Would either of these options work?

Got it. But then users have to handle retries themselves. We handled retries from the beginning. Is there any way we can still handle retries on our side to avoid breaking users' workflow?

Following the comment in the slack, we keep retry mechanism but allow users to opt out with use_retry=False.

eyurtsev

Sorry didn't meant to accept the previous version...

tjaffri · 2023-10-25T22:54:29Z

libs/langchain/langchain/llms/fireworks.py

+        choices = []
+        for _prompts in sub_prompts:
+            response = await acompletion_with_retry_batching(
+                self, self.use_retry, prompt=_prompts, run_manager=run_manager, **params


I think you need to send the stop words in to acompletion_with_retry_batching?

stop is in params

See other comment, I don't believe this is correct.

libs/langchain/tests/integration_tests/chat_models/test_fireworks.py

libs/langchain/tests/integration_tests/llms/test_fireworks.py

libs/langchain/langchain/llms/fireworks.py

* Support Fireworks batching * Support ChatFireworks batching

libs/langchain/langchain/llms/fireworks.py

libs/langchain/tests/integration_tests/chat_models/test_fireworks.py

libs/langchain/tests/integration_tests/llms/test_fireworks.py

eyurtsev

@baskaryan this looks good to me.

@baskaryan

Description * Add _generate and _agenerate to support Fireworks batching. * Add stop words test cases * Opt out retry mechanism Issue - Not applicable Dependencies - None Tag maintainer - @baskaryan

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [langchain](https://togithub.com/langchain-ai/langchain) | `^0.0.323` -> `^0.0.326` | [![age](https://developer.mend.io/api/mc/badges/age/pypi/langchain/0.0.326?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/langchain/0.0.326?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/langchain/0.0.323/0.0.326?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/langchain/0.0.323/0.0.326?slim=true)](https://docs.renovatebot.com/merge-confidence/) | | [langchain](https://togithub.com/langchain-ai/langchainjs) | [`^0.0.173` -> `^0.0.176`](https://renovatebot.com/diffs/npm/langchain/0.0.173/0.0.176) | [![age](https://developer.mend.io/api/mc/badges/age/npm/langchain/0.0.176?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/npm/langchain/0.0.176?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/npm/langchain/0.0.173/0.0.176?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/npm/langchain/0.0.173/0.0.176?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>langchain-ai/langchain (langchain)</summary> ### [`v0.0.326`](https://togithub.com/langchain-ai/langchain/releases/tag/v0.0.326) [Compare Source](https://togithub.com/langchain-ai/langchain/compare/v0.0.325...v0.0.326) #### What's Changed - MongoDB $vectorSearch doc update by [@prakul](https://togithub.com/prakul) in [langchain-ai/langchain#12404 - rfc: type LLMChain.llm as runnable by [@baskaryan](https://togithub.com/baskaryan) in [langchain-ai/langchain#12385 - Update long_context_reorder.py by [@ennio1991](https://togithub.com/ennio1991) in [langchain-ai/langchain#12422 - optional reusable connection by [@theromis](https://togithub.com/theromis) in [langchain-ai/langchain#12051 - added rrf argument in ApproxRetrievalStrategy class **init**() by [@HwangJohn](https://togithub.com/HwangJohn) in [langchain-ai/langchain#11987 - Fix Typo in clickhouse.ipynb file by [@Uxywannasleep](https://togithub.com/Uxywannasleep) in [langchain-ai/langchain#12429 - cli updates oct27 by [@efriis](https://togithub.com/efriis) in [langchain-ai/langchain#12436 - Fix the missing temperature parameter for Baichuan-AI chat_model by [@henter](https://togithub.com/henter) in [langchain-ai/langchain#12420 - make doc utils public by [@baskaryan](https://togithub.com/baskaryan) in [langchain-ai/langchain#12394 - Trace if run tree set by [@hinthornw](https://togithub.com/hinthornw) in [langchain-ai/langchain#12444 - AWS Bedrock RAG template by [@rlancemartin](https://togithub.com/rlancemartin) in [langchain-ai/langchain#12450 - Update AWS Bedrock README.md by [@rlancemartin](https://togithub.com/rlancemartin) in [langchain-ai/langchain#12451 - cli updates 2 by [@efriis](https://togithub.com/efriis) in [langchain-ai/langchain#12447 - Improve handling of empty queries for timescale vector by [@cevian](https://togithub.com/cevian) in [langchain-ai/langchain#12393 - Add security note to API chain by [@eyurtsev](https://togithub.com/eyurtsev) in [langchain-ai/langchain#12452 - Bump [@babel/traverse](https://togithub.com/babel/traverse) from 7.22.8 to 7.23.2 in /docs by [@dependabot](https://togithub.com/dependabot) in [langchain-ai/langchain#12453 - add reranking to azuresearch by [@lawadam](https://togithub.com/lawadam) in [langchain-ai/langchain#12454 - Add Weaviate RAG template by [@rlancemartin](https://togithub.com/rlancemartin) in [langchain-ai/langchain#12460 - Update SQL templates by [@rlancemartin](https://togithub.com/rlancemartin) in [langchain-ai/langchain#12464 - Update llama.cpp and Ollama templates by [@rlancemartin](https://togithub.com/rlancemartin) in [langchain-ai/langchain#12466 - Updated the Bedrock rag template by [@3coins](https://togithub.com/3coins) in [langchain-ai/langchain#12462 - cli improvements by [@efriis](https://togithub.com/efriis) in [langchain-ai/langchain#12465 - Redis langserve template by [@tylerhutcherson](https://togithub.com/tylerhutcherson) in [langchain-ai/langchain#12443 - bump to 36 by [@hwchase17](https://togithub.com/hwchase17) in [langchain-ai/langchain#12487 - Added a rag template for Kendra by [@3coins](https://togithub.com/3coins) in [langchain-ai/langchain#12470 - Bagatur/self query doc update by [@baskaryan](https://togithub.com/baskaryan) in [langchain-ai/langchain#12461 - Harrison/quick start by [@hwchase17](https://togithub.com/hwchase17) in [langchain-ai/langchain#12491 - Sphinxbio nls/add plate chain template by [@hwchase17](https://togithub.com/hwchase17) in [langchain-ai/langchain#12502 - Update tool.py by [@0xC9](https://togithub.com/0xC9) in [langchain-ai/langchain#12472 - add gha for cli by [@hwchase17](https://togithub.com/hwchase17) in [langchain-ai/langchain#12492 - add cookbook for selectins llms based on context length by [@hwchase17](https://togithub.com/hwchase17) in [langchain-ai/langchain#12486 - various templates improvements by [@hwchase17](https://togithub.com/hwchase17) in [langchain-ai/langchain#12500 - Evaluation Callback Multi Response by [@hinthornw](https://togithub.com/hinthornw) in [langchain-ai/langchain#12505 - Patch forward ref bug by [@hinthornw](https://togithub.com/hinthornw) in [langchain-ai/langchain#12508 - OpenAI runnable constructor by [@baskaryan](https://togithub.com/baskaryan) in [langchain-ai/langchain#12455 - `_dalle_image_url` returns list of urls if n>1 by [@silvhua](https://togithub.com/silvhua) in [langchain-ai/langchain#11800 - docs(prompt_templates): fix typo in prompt template by [@nbbaier](https://togithub.com/nbbaier) in [langchain-ai/langchain#12497 - Mask API key for AI21 LLM by [@gautamanirudh](https://togithub.com/gautamanirudh) in [langchain-ai/langchain#12418 - Harrison/add descriptions by [@hwchase17](https://togithub.com/hwchase17) in [langchain-ai/langchain#12522 - notebook fmt by [@baskaryan](https://togithub.com/baskaryan) in [langchain-ai/langchain#12498 - Bagatur/fix doc ci by [@baskaryan](https://togithub.com/baskaryan) in [langchain-ai/langchain#12529 - update contributing by [@baskaryan](https://togithub.com/baskaryan) in [langchain-ai/langchain#12532 - Add unit tests for document_transformers/beautiful_soup_transformer.py by [@petervandenabeele](https://togithub.com/petervandenabeele) in [langchain-ai/langchain#12520 - More comprehensive readthedocs document loader by [@adrwz](https://togithub.com/adrwz) in [langchain-ai/langchain#12382 - Masking of API Key for GooseAI LLM by [@samadkoita](https://togithub.com/samadkoita) in [langchain-ai/langchain#12496 - feat: Add Google Cloud Translation document transformer by [@holtskinner](https://togithub.com/holtskinner) in [langchain-ai/langchain#12433 - goog translate nb formatting by [@baskaryan](https://togithub.com/baskaryan) in [langchain-ai/langchain#12534 - Update neo4j template readmes by [@tomasonjo](https://togithub.com/tomasonjo) in [langchain-ai/langchain#12540 - Allow astream_log to be used inside atrace_as_chain_group by [@nfcampos](https://togithub.com/nfcampos) in [langchain-ai/langchain#12558 - Image Caption accepts bytes for images by [@eyurtsev](https://togithub.com/eyurtsev) in [langchain-ai/langchain#12561 - bump 326 by [@baskaryan](https://togithub.com/baskaryan) in [langchain-ai/langchain#12569 #### New Contributors - [@ennio1991](https://togithub.com/ennio1991) made their first contribution in [langchain-ai/langchain#12422 - [@theromis](https://togithub.com/theromis) made their first contribution in [langchain-ai/langchain#12051 - [@HwangJohn](https://togithub.com/HwangJohn) made their first contribution in [langchain-ai/langchain#11987 - [@Uxywannasleep](https://togithub.com/Uxywannasleep) made their first contribution in [langchain-ai/langchain#12429 - [@henter](https://togithub.com/henter) made their first contribution in [langchain-ai/langchain#12420 - [@dependabot](https://togithub.com/dependabot) made their first contribution in [langchain-ai/langchain#12453 - [@lawadam](https://togithub.com/lawadam) made their first contribution in [langchain-ai/langchain#12454 - [@0xC9](https://togithub.com/0xC9) made their first contribution in [langchain-ai/langchain#12472 - [@silvhua](https://togithub.com/silvhua) made their first contribution in [langchain-ai/langchain#11800 - [@nbbaier](https://togithub.com/nbbaier) made their first contribution in [langchain-ai/langchain#12497 - [@gautamanirudh](https://togithub.com/gautamanirudh) made their first contribution in [langchain-ai/langchain#12418 - [@petervandenabeele](https://togithub.com/petervandenabeele) made their first contribution in [langchain-ai/langchain#12520 - [@adrwz](https://togithub.com/adrwz) made their first contribution in [langchain-ai/langchain#12382 - [@samadkoita](https://togithub.com/samadkoita) made their first contribution in [langchain-ai/langchain#12496 **Full Changelog**: langchain-ai/langchain@v0.0.325...v0.0.326 ### [`v0.0.325`](https://togithub.com/langchain-ai/langchain/releases/tag/v0.0.325) [Compare Source](https://togithub.com/langchain-ai/langchain/compare/v0.0.324...v0.0.325) #### What's Changed - Add template for Pinecone + Multi-Query by [@rlancemartin](https://togithub.com/rlancemartin) in [langchain-ai/langchain#12353 - Docs: Add lcel to combine_docs chains by [@baskaryan](https://togithub.com/baskaryan) in [langchain-ai/langchain#12310 - Update multi query template README, ntbk by [@rlancemartin](https://togithub.com/rlancemartin) in [langchain-ai/langchain#12356 - langserve doc by [@baskaryan](https://togithub.com/baskaryan) in [langchain-ai/langchain#12357 - E2B tool - Improve description wuth uploaded files info by [@jakubno](https://togithub.com/jakubno) in [langchain-ai/langchain#12355 - Fix redis vectorfield schema defaults by [@tylerhutcherson](https://togithub.com/tylerhutcherson) in [langchain-ai/langchain#12223 - update chains how to by [@baskaryan](https://togithub.com/baskaryan) in [langchain-ai/langchain#12362 - Support Fireworks batching ([#8](https://togithub.com/langchain-ai/langchain/issues/8)) by [@ZixinYang](https://togithub.com/ZixinYang) in [langchain-ai/langchain#12052 - fix some stuff by [@hwchase17](https://togithub.com/hwchase17) in [langchain-ai/langchain#12292 - Harrison/improve cli by [@hwchase17](https://togithub.com/hwchase17) in [langchain-ai/langchain#12368 - Adds linter in templates by [@jacoblee93](https://togithub.com/jacoblee93) in [langchain-ai/langchain#12321 - Fix langsmith walkthrough doc dataset by [@lucasc896](https://togithub.com/lucasc896) in [langchain-ai/langchain#12027 - Mention other function calling/grammar support by [@hinthornw](https://togithub.com/hinthornw) in [langchain-ai/langchain#12369 - rm init by [@baskaryan](https://togithub.com/baskaryan) in [langchain-ai/langchain#12374 - Fix SupbaseVectoreStore write operation timeout by [@j1philli](https://togithub.com/j1philli) in [langchain-ai/langchain#12318 - fireworks scheduled integration tests by [@baskaryan](https://togithub.com/baskaryan) in [langchain-ai/langchain#12373 - Cohere re-rank template by [@rlancemartin](https://togithub.com/rlancemartin) in [langchain-ai/langchain#12378 - cli fix by [@efriis](https://togithub.com/efriis) in [langchain-ai/langchain#12380 - better lint triggering by [@baskaryan](https://togithub.com/baskaryan) in [langchain-ai/langchain#12376 - Minor updates to ReRank template by [@rlancemartin](https://togithub.com/rlancemartin) in [langchain-ai/langchain#12388 - Update broken redis tests by [@tylerhutcherson](https://togithub.com/tylerhutcherson) in [langchain-ai/langchain#12371 - CLI CI 2 by [@efriis](https://togithub.com/efriis) in [langchain-ai/langchain#12387 - add template for hyde by [@hwchase17](https://togithub.com/hwchase17) in [langchain-ai/langchain#12390 - Wfh/json edit dist by [@hinthornw](https://togithub.com/hinthornw) in [langchain-ai/langchain#12361 - Fixed some grammatical and Exception types issues by [@ShorthillsAI](https://togithub.com/ShorthillsAI) in [langchain-ai/langchain#12015 - Format Templates by [@efriis](https://togithub.com/efriis) in [langchain-ai/langchain#12396 - Add HTML Title and Page Language into metadata for AsyncHtmlLoader by [@kkk55596](https://togithub.com/kkk55596) in [langchain-ai/langchain#11326 - johnsnowlabs embeddings support by [@C-K-Loan](https://togithub.com/C-K-Loan) in [langchain-ai/langchain#11271 - Templates CI by [@efriis](https://togithub.com/efriis) in [langchain-ai/langchain#12313 - Wfh/json schema evaluation by [@hinthornw](https://togithub.com/hinthornw) in [langchain-ai/langchain#12389 - Add space by [@hinthornw](https://togithub.com/hinthornw) in [langchain-ai/langchain#12395 - Str Evaluator Mapper by [@hinthornw](https://togithub.com/hinthornw) in [langchain-ai/langchain#12401 - Clean-up template READMEs by [@rlancemartin](https://togithub.com/rlancemartin) in [langchain-ai/langchain#12403 - Mask API key for Aleph Alpha LLM by [@slangenbach](https://togithub.com/slangenbach) in [langchain-ai/langchain#12377 - fix(openai-callback): completion count logic by [@mspronesti](https://togithub.com/mspronesti) in [langchain-ai/langchain#12383 - Fix templates typos by [@dqbd](https://togithub.com/dqbd) in [langchain-ai/langchain#12428 - feat: Add Google Speech to Text API Document Loader by [@holtskinner](https://togithub.com/holtskinner) in [langchain-ai/langchain#12298 - LLaMA2 with JSON schema support template by [@rlancemartin](https://togithub.com/rlancemartin) in [langchain-ai/langchain#12435 - clean up deprecated agents by [@hwchase17](https://togithub.com/hwchase17) in [langchain-ai/langchain#12231 - Add invoke example to LLaMA2 function template notebook by [@rlancemartin](https://togithub.com/rlancemartin) in [langchain-ai/langchain#12437 - Deprecate PythonRepl tools and Pandas/Xorbits/Spark DataFrame/Python/CSV agents by [@eyurtsev](https://togithub.com/eyurtsev) in [langchain-ai/langchain#12427 - Bagatur/bump 325 by [@baskaryan](https://togithub.com/baskaryan) in [langchain-ai/langchain#12440 #### New Contributors - [@jakubno](https://togithub.com/jakubno) made their first contribution in [langchain-ai/langchain#12355 - [@lucasc896](https://togithub.com/lucasc896) made their first contribution in [langchain-ai/langchain#12027 - [@kkk55596](https://togithub.com/kkk55596) made their first contribution in [langchain-ai/langchain#11326 - [@C-K-Loan](https://togithub.com/C-K-Loan) made their first contribution in [langchain-ai/langchain#11271 - [@slangenbach](https://togithub.com/slangenbach) made their first contribution in [langchain-ai/langchain#12377 #### CVEs CVE-2023-39659 resolved in [langchain-ai/langchain#12427 **Full Changelog**: langchain-ai/langchain@v0.0.324...v0.0.325 ### [`v0.0.324`](https://togithub.com/langchain-ai/langchain/releases/tag/v0.0.324) [Compare Source](https://togithub.com/langchain-ai/langchain/compare/v0.0.323...v0.0.324) #### What's Changed - Pop max concurrency when recursing by [@nfcampos](https://togithub.com/nfcampos) in [langchain-ai/langchain#12281 - Remove CLI by [@nfcampos](https://togithub.com/nfcampos) in [langchain-ai/langchain#12283 - CohereEmbeddings: Add max_retries and request_timeout by [@ephe-meral](https://togithub.com/ephe-meral) in [langchain-ai/langchain#12275 - response parser for ArceeRetriever by [@EricLiclair](https://togithub.com/EricLiclair) in [langchain-ai/langchain#12270 - CLI by [@efriis](https://togithub.com/efriis) in [langchain-ai/langchain#12284 - chat loader doc titles by [@baskaryan](https://togithub.com/baskaryan) in [langchain-ai/langchain#12289 - Update vectorstore.mdx\[Make an improvement] by [@SilvaXiang](https://togithub.com/SilvaXiang) in [langchain-ai/langchain#12252 - rm .html from local doc links by [@baskaryan](https://togithub.com/baskaryan) in [langchain-ai/langchain#12293 - dev guide by [@baskaryan](https://togithub.com/baskaryan) in [langchain-ai/langchain#12291 - Update README.md by [@RS-labhub](https://togithub.com/RS-labhub) in [langchain-ai/langchain#12286 - Strips leading/trailing whitespace before parsing xml by [@donatoaz](https://togithub.com/donatoaz) in [langchain-ai/langchain#12297 - fix sparql queries for relations in schema description by [@felixocker](https://togithub.com/felixocker) in [langchain-ai/langchain#9136 - use snippet search optionally by [@sam-h-bean](https://togithub.com/sam-h-bean) in [langchain-ai/langchain#12236 - Add baidu cloud vector search in vectorstore and fix some unit test in vectorstores by [@wemysschen](https://togithub.com/wemysschen) in [langchain-ai/langchain#11605 - Rm langchain server by [@hinthornw](https://togithub.com/hinthornw) in [langchain-ai/langchain#12305 - Relax type annotation for custom input/output types by [@eyurtsev](https://togithub.com/eyurtsev) in [langchain-ai/langchain#12300 - Better wrap traceable by [@hinthornw](https://togithub.com/hinthornw) in [langchain-ai/langchain#12303 - CLI Git Improvements by [@efriis](https://togithub.com/efriis) in [langchain-ai/langchain#12311 - Templates by [@efriis](https://togithub.com/efriis) in [langchain-ai/langchain#12294 - removed CardLists for LLMs and ChatModels by [@leo-gan](https://togithub.com/leo-gan) in [langchain-ai/langchain#12307 - Allow index name customization via env var in rag-conversation by [@jacoblee93](https://togithub.com/jacoblee93) in [langchain-ai/langchain#12315 - add docs for templates by [@hwchase17](https://togithub.com/hwchase17) in [langchain-ai/langchain#12346 - e2b tool - fix initialization and improve tool description by [@mlejva](https://togithub.com/mlejva) in [langchain-ai/langchain#12345 - Fix Typo in CONTRIBUTING.md by [@kishanrajput23](https://togithub.com/kishanrajput23) in [langchain-ai/langchain#12320 - Fix typos in quickstart.mdx by [@LaurentAjdnik](https://togithub.com/LaurentAjdnik) in [langchain-ai/langchain#12333 - fix self query constructor by [@baskaryan](https://togithub.com/baskaryan) in [langchain-ai/langchain#12349 - add allowed_operators property in QdrantTranslator by [@xieqihui](https://togithub.com/xieqihui) in [langchain-ai/langchain#12328 - fix typo by [@wongzc](https://togithub.com/wongzc) in [langchain-ai/langchain#12338 - Fix the exception from langchain.utilities import ArceeWrapper by [@gnakw](https://togithub.com/gnakw) in [langchain-ai/langchain#12342 - Fix a typo in the summarization use case. by [@season179](https://togithub.com/season179) in [langchain-ai/langchain#12316 - Update code_understanding.ipynb by [@kengoodridge](https://togithub.com/kengoodridge) in [langchain-ai/langchain#12309 - Finetuned openai azure models cost calculation by [@nirkopler](https://togithub.com/nirkopler) in [langchain-ai/langchain#12267 - fixed error message and a check for processor name by [@lkuligin](https://togithub.com/lkuligin) in [langchain-ai/langchain#12200 - Takeoff pro support by [@hoyungcher](https://togithub.com/hoyungcher) in [langchain-ai/langchain#12070 - Comprehend Moderation 0.2 by [@nikhilkjha](https://togithub.com/nikhilkjha) in [langchain-ai/langchain#11730 - Fix json key output parser in partial (streaming) mode by [@nfcampos](https://togithub.com/nfcampos) in [langchain-ai/langchain#12332 - bump 324 and 35 by [@baskaryan](https://togithub.com/baskaryan) in [langchain-ai/langchain#12352 #### New Contributors - [@SilvaXiang](https://togithub.com/SilvaXiang) made their first contribution in [langchain-ai/langchain#12252 - [@RS-labhub](https://togithub.com/RS-labhub) made their first contribution in [langchain-ai/langchain#12286 - [@donatoaz](https://togithub.com/donatoaz) made their first contribution in [langchain-ai/langchain#12297 - [@kishanrajput23](https://togithub.com/kishanrajput23) made their first contribution in [langchain-ai/langchain#12320 - [@LaurentAjdnik](https://togithub.com/LaurentAjdnik) made their first contribution in [langchain-ai/langchain#12333 - [@wongzc](https://togithub.com/wongzc) made their first contribution in [langchain-ai/langchain#12338 - [@gnakw](https://togithub.com/gnakw) made their first contribution in [langchain-ai/langchain#12342 - [@season179](https://togithub.com/season179) made their first contribution in [langchain-ai/langchain#12316 - [@kengoodridge](https://togithub.com/kengoodridge) made their first contribution in [langchain-ai/langchain#12309 **Full Changelog**: langchain-ai/langchain@v0.0.323...v0.0.324 </details> <details> <summary>langchain-ai/langchainjs (langchain)</summary> ### [`v0.0.176`](https://togithub.com/langchain-ai/langchainjs/releases/tag/0.0.176) [Compare Source](https://togithub.com/langchain-ai/langchainjs/compare/0.0.175...0.0.176) #### What's Changed - Release 0.0.175 by [@jacoblee93](https://togithub.com/jacoblee93) in [langchain-ai/langchainjs#3082 - Update ollama.mdx - `numThreads` to `numThread` by [@paulwongx](https://togithub.com/paulwongx) in [langchain-ai/langchainjs#3083 - Update RAG fusion notebook by [@jacoblee93](https://togithub.com/jacoblee93) in [langchain-ai/langchainjs#3095 - JSON functions output parser docs by [@bracesproul](https://togithub.com/bracesproul) in [langchain-ai/langchainjs#3098 - Reduce max size for docs build by [@jacoblee93](https://togithub.com/jacoblee93) in [langchain-ai/langchainjs#3100 - Implement streaming for GoogleVertexAI text and chat models by [@afirstenberg](https://togithub.com/afirstenberg) in [langchain-ai/langchainjs#3088 - Increase docs build RAM by [@jacoblee93](https://togithub.com/jacoblee93) in [langchain-ai/langchainjs#3101 - Jtcorrin/plan execute adjustments by [@JTCorrin](https://togithub.com/JTCorrin) in [langchain-ai/langchainjs#3072 #### New Contributors - [@paulwongx](https://togithub.com/paulwongx) made their first contribution in [langchain-ai/langchainjs#3083 - [@JTCorrin](https://togithub.com/JTCorrin) made their first contribution in [langchain-ai/langchainjs#3072 **Full Changelog**: langchain-ai/langchainjs@0.0.175...0.0.176 ### [`v0.0.175`](https://togithub.com/langchain-ai/langchainjs/releases/tag/0.0.175) [Compare Source](https://togithub.com/langchain-ai/langchainjs/compare/0.0.174...0.0.175) #### What's Changed - Release 0.0.174 by [@jacoblee93](https://togithub.com/jacoblee93) in [langchain-ai/langchainjs#3064 - Adds basic critique revise cookbook by [@jacoblee93](https://togithub.com/jacoblee93) in [langchain-ai/langchainjs#3067 - fix: calculate Milvus textlength in bytes by [@kurtmoser](https://togithub.com/kurtmoser) in [langchain-ai/langchainjs#3079 - Add Convex integration by [@xixixao](https://togithub.com/xixixao) in [langchain-ai/langchainjs#2977 - Make "IN-filters" possible for Vercel Postgres Metadata by [@rbouschery](https://togithub.com/rbouschery) in [langchain-ai/langchainjs#3040 - Fix Convex typo by [@jacoblee93](https://togithub.com/jacoblee93) in [langchain-ai/langchainjs#3081 #### New Contributors - [@kurtmoser](https://togithub.com/kurtmoser) made their first contribution in [langchain-ai/langchainjs#3079 - [@xixixao](https://togithub.com/xixixao) made their first contribution in [langchain-ai/langchainjs#2977 - [@rbouschery](https://togithub.com/rbouschery) made their first contribution in [langchain-ai/langchainjs#3040 **Full Changelog**: langchain-ai/langchainjs@0.0.174...0.0.175 ### [`v0.0.174`](https://togithub.com/langchain-ai/langchainjs/releases/tag/0.0.174) [Compare Source](https://togithub.com/langchain-ai/langchainjs/compare/0.0.173...0.0.174) #### What's Changed - Release 0.0.173 by [@jacoblee93](https://togithub.com/jacoblee93) in [langchain-ai/langchainjs#3054 - fix: LANGCHAIN_VERBOSE=false does not turn off verbose logging. by [@xianminx](https://togithub.com/xianminx) in [langchain-ai/langchainjs#3057 - Format by [@jacoblee93](https://togithub.com/jacoblee93) in [langchain-ai/langchainjs#3058 - fix: improper markdown links by [@bracesproul](https://togithub.com/bracesproul) in [langchain-ai/langchainjs#3061 - Adds MMR search for Pinecone by [@jacoblee93](https://togithub.com/jacoblee93) in [langchain-ai/langchainjs#3059 - Allow vectorstore retrievers in SemanticSimilarityExampleSelector.ts by [@jacoblee93](https://togithub.com/jacoblee93) in [langchain-ai/langchainjs#3056 - Add getOutputValue to langchain/memory exports by [@mattmcgiv](https://togithub.com/mattmcgiv) in [langchain-ai/langchainjs#3060 #### New Contributors - [@xianminx](https://togithub.com/xianminx) made their first contribution in [langchain-ai/langchainjs#3057 - [@mattmcgiv](https://togithub.com/mattmcgiv) made their first contribution in [langchain-ai/langchainjs#3060 **Full Changelog**: langchain-ai/langchainjs@0.0.173...0.0.174 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://togithub.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/autoblocksai/autoblocks-examples).  Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

@baskaryan

Description * Add _generate and _agenerate to support Fireworks batching. * Add stop words test cases * Opt out retry mechanism Issue - Not applicable Dependencies - None Tag maintainer - @baskaryan

@baskaryan

Description * Add _generate and _agenerate to support Fireworks batching. * Add stop words test cases * Opt out retry mechanism Issue - Not applicable Dependencies - None Tag maintainer - @baskaryan

dosubot bot added Ɑ: models Related to LLMs or chat model modules 🤖:improvement Medium size change to existing code to handle new use-cases labels Oct 19, 2023

eyurtsev self-requested a review October 19, 2023 21:27

eyurtsev self-assigned this Oct 19, 2023

eyurtsev approved these changes Oct 19, 2023

View reviewed changes

eyurtsev requested changes Oct 19, 2023

View reviewed changes

ZixinYang force-pushed the master branch from cf2686e to 6b96e83 Compare October 24, 2023 16:52

ZixinYang requested review from eyurtsev and hwchase17 October 24, 2023 18:21

tjaffri suggested changes Oct 25, 2023

View reviewed changes

tjaffri reviewed Oct 25, 2023

View reviewed changes

libs/langchain/langchain/llms/fireworks.py Show resolved Hide resolved

ZixinYang force-pushed the master branch from 440d857 to fe69c14 Compare October 25, 2023 23:26

ZixinYang commented Oct 25, 2023

View reviewed changes

libs/langchain/langchain/llms/fireworks.py Outdated Show resolved Hide resolved

ZixinYang force-pushed the master branch 2 times, most recently from a0f6adb to df2cb06 Compare October 26, 2023 04:34

ZixinYang added 6 commits October 26, 2023 17:55

Support Fireworks batching (#8)

c3ec6f3

* Support Fireworks batching * Support ChatFireworks batching

Support Fireworks batching

71f628b

Fix lint issue

88f3be1

Add serialisation props to Fireworks and ChatFireworks

72bdc5e

Add type annotation

eb90d57

Update retry mechanism with new error type

b80e462

ZixinYang force-pushed the master branch 2 times, most recently from d923d9c to b80e462 Compare October 26, 2023 18:04

Add invoke/batch test cases and pass stop to batch retry

c2f6ce4

ZixinYang requested a review from tjaffri October 26, 2023 18:20

tjaffri suggested changes Oct 26, 2023

View reviewed changes

libs/langchain/langchain/llms/fireworks.py Outdated Show resolved Hide resolved

libs/langchain/tests/integration_tests/chat_models/test_fireworks.py Outdated Show resolved Hide resolved

libs/langchain/tests/integration_tests/llms/test_fireworks.py Outdated Show resolved Hide resolved

Remove redundant test cases

2c1926a

ZixinYang requested a review from tjaffri October 26, 2023 18:38

eyurtsev approved these changes Oct 26, 2023

View reviewed changes

eyurtsev merged commit 6ce276e into langchain-ai:master Oct 26, 2023
39 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support Fireworks batching (#8) #12052

Support Fireworks batching (#8) #12052

ZixinYang commented Oct 19, 2023 •

edited

vercel bot commented Oct 19, 2023 •

edited

eyurtsev Oct 19, 2023

ZixinYang Oct 19, 2023

eyurtsev Oct 24, 2023

ZixinYang Oct 25, 2023

eyurtsev Oct 19, 2023

ZixinYang Oct 23, 2023

eyurtsev Oct 24, 2023

ZixinYang Oct 24, 2023

ZixinYang Oct 25, 2023 •

edited

eyurtsev left a comment

tjaffri Oct 25, 2023

ZixinYang Oct 25, 2023

tjaffri Oct 25, 2023 •

edited

eyurtsev left a comment

		@@ -179,6 +228,74 @@ async def _completion_with_retry(**kwargs: Any) -> Any:
		return await _completion_with_retry(**kwargs)


		def completion_with_retry_batching(

Support Fireworks batching (#8) #12052

Support Fireworks batching (#8) #12052

Conversation

ZixinYang commented Oct 19, 2023 • edited

vercel bot commented Oct 19, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ZixinYang Oct 25, 2023 • edited

Choose a reason for hiding this comment

eyurtsev left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tjaffri Oct 25, 2023 • edited

Choose a reason for hiding this comment

eyurtsev left a comment

Choose a reason for hiding this comment

ZixinYang commented Oct 19, 2023 •

edited

vercel bot commented Oct 19, 2023 •

edited

ZixinYang Oct 25, 2023 •

edited

tjaffri Oct 25, 2023 •

edited