Expose kwargs in `LLMChainExtractor.from_llm` #3748

ravwojdyla · 2023-04-28T23:34:07Z

skcoirz

Hi ravwojdyla, thank you for the proposal! Since there isn’t much context in the summary, do you mind sharing a little bit why do you need this parameter and under which scenario this would benefit other users too? Would like to learn more from you! Thank you!

skcoirz · 2023-04-28T23:52:49Z

langchain/retrievers/document_compressors/chain_extract.py

@@ -69,9 +69,10 @@ def from_llm(
        llm: BaseLanguageModel,
        prompt: Optional[PromptTemplate] = None,
        get_input: Optional[Callable[[str, Document], str]] = None,
+        **llm_chain_kwargs: dict[str, Any]


From my understanding, this is for llm_chain creation, but the purpose of this function is to load from llm directly with no additional customization of llm_chain.

If my understanding is correct, I would suggest the following

construct a LLMChain outside and then create LLMChainExtractor.

if you need further convenience beyond step 1, I would suggest writing a static utility function with your preferred input and output in this file or another file in this folder.

I could be wrong. Will wait for maintainer to further comment here. :)

👋 @skcoirz see context in #3747, this is the same as in BaseQAWithSourcesChain https://github.com/hwchase17/langchain/blob/72c5c15f7fdc1918880e3cfd0949199e5a0b5bda/langchain/chains/qa_with_sources/base.py#L40-L47

Yeah, got you! Thank you for sharing the example! This is just my personal idea, please let me know if this doesn’t make sense to you. :)

In the example you shared, the kwargs is passed to the class itself. So we can consider this function as an entry to initiate this class itself. Code below:
https://github.com/hwchase17/langchain/blob/72c5c15f7fdc1918880e3cfd0949199e5a0b5bda/langchain/chains/qa_with_sources/base.py#L39-L64

But in the PR here, the parameter kwargs is used as input for LLMChain which is a intermediate layer between the function and the class. From my understanding, a more suitable solution is to build another entry function which build this extractor from LLMChain directly, which, however, is overlapping with the default creation method of this class. That’s why I think it’s easier to create a LLMChain outside and then build this class directly.

Please let me know if this makes sense to you. Thank you!

Sure, that sounds good, but then I/user would need to initialize the LLMChain much the same way it's being done in the from_llm (which is a couple of extra lines of code, just to pass verbose=True). I really appreciate the convenience of the from_llm method (where I can reuse existing LLM object), so why not (in my opinion) improve it slightly and let the user pass kwargs to the internal LLMChain.

Ah that makes sense. If verbose is the only param which we need, maybe build a another wrapper function on top of from_LLM with name postfix _with_verbose ? In this way, we can get the verbose and we don't need to worry about other parameters in kwargs. How do you think?

@skcoirz verbose is just an example, a specialized _with_verbose seems a bit smelly.

Sure, it's just my 2 cents. :)

Sounds good, appreciate that :)

ravwojdyla · 2023-04-29T02:35:15Z

Fixed up linting error ^

langchain/retrievers/document_compressors/chain_extract.py

hwchase17 · 2023-04-29T03:41:20Z

langchain/retrievers/document_compressors/chain_extract.py

@@ -69,9 +69,10 @@ def from_llm(
        llm: BaseLanguageModel,
        prompt: Optional[PromptTemplate] = None,
        get_input: Optional[Callable[[str, Document], str]] = None,
+        llm_chain_kwargs: Union[Dict[str, Any], None] = None,


this can just be Optional[dict] = None

Thanks, sorry we are on py3.10, forgot all about Optional ... :)

hwchase17

awesome - thanks!

#3773) a simple follow up of #3748 - added test case - improve annotation when function return type is class itself.

Re: langchain-ai#3747

langchain-ai#3773) a simple follow up of langchain-ai#3748 - added test case - improve annotation when function return type is class itself.

ravwojdyla changed the title ~~Expose kwargs in LLMChainExtractor.from_llm~~ Expose kwargs in LLMChainExtractor.from_llm Apr 28, 2023

skcoirz reviewed Apr 28, 2023

View reviewed changes

ravwojdyla force-pushed the llm_chain_extractor_from_llm_kwargs branch from 2ed62a0 to 32e40a2 Compare April 29, 2023 02:34

hwchase17 reviewed Apr 29, 2023

View reviewed changes

langchain/retrievers/document_compressors/chain_extract.py Outdated Show resolved Hide resolved

ravwojdyla force-pushed the llm_chain_extractor_from_llm_kwargs branch 2 times, most recently from 4dfa4c7 to 24571f9 Compare April 29, 2023 03:22

hwchase17 reviewed Apr 29, 2023

View reviewed changes

ravwojdyla force-pushed the llm_chain_extractor_from_llm_kwargs branch from 24571f9 to 71cf30f Compare April 29, 2023 03:43

Expose kwargs in LLMChainExtractor.from_llm

0d45b14

ravwojdyla force-pushed the llm_chain_extractor_from_llm_kwargs branch from 71cf30f to 0d45b14 Compare April 29, 2023 03:46

hwchase17 approved these changes Apr 29, 2023

View reviewed changes

hwchase17 merged commit 57e0285 into langchain-ai:master Apr 29, 2023
9 checks passed

skcoirz mentioned this pull request Apr 29, 2023

[simple] added test case and improve self class return type annotation #3773

Merged

hwchase17 pushed a commit that referenced this pull request Apr 29, 2023

[simple] added test case and improve self class return type annotation (

ce4fea9

#3773) a simple follow up of #3748 - added test case - improve annotation when function return type is class itself.

samching pushed a commit to samching/langchain that referenced this pull request May 1, 2023

Expose kwargs in LLMChainExtractor.from_llm (langchain-ai#3748)

16c791f

Re: langchain-ai#3747

ugm2 mentioned this pull request Dec 4, 2023

Adds llm_chain_kwargs to BaseRetrievalQA.from_llm #14224

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Expose kwargs in `LLMChainExtractor.from_llm` #3748

Expose kwargs in `LLMChainExtractor.from_llm` #3748

ravwojdyla commented Apr 28, 2023

skcoirz left a comment

skcoirz Apr 28, 2023

ravwojdyla Apr 29, 2023

skcoirz Apr 29, 2023

ravwojdyla Apr 29, 2023

skcoirz Apr 29, 2023

ravwojdyla Apr 29, 2023

skcoirz Apr 29, 2023

ravwojdyla Apr 29, 2023

ravwojdyla commented Apr 29, 2023

hwchase17 Apr 29, 2023

ravwojdyla Apr 29, 2023

hwchase17 left a comment

Expose kwargs in LLMChainExtractor.from_llm #3748

Expose kwargs in LLMChainExtractor.from_llm #3748

Conversation

ravwojdyla commented Apr 28, 2023

skcoirz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ravwojdyla commented Apr 29, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hwchase17 left a comment

Choose a reason for hiding this comment

Expose kwargs in `LLMChainExtractor.from_llm` #3748

Expose kwargs in `LLMChainExtractor.from_llm` #3748