langchain[patch]: Add Possibility to use Contextual chunk headers in Parent Document Retriever #4651
Conversation
dosubot
bot
added
the
size:M
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
dosubot
bot
added
the
auto:improvement
karol-f
force-pushed
the
chunk-header-ParentDocumentRetriever
branch
2 times, most recently
from
d001d19
to
4601cb9
Compare
|
Nice one, thank you! |
|
Looks like you might need to run |
jacoblee93
changed the title
You are right, sorry. Fixed. |
| @@ -159,7 +162,11 @@ export class ParentDocumentRetriever extends MultiVectorRetriever { | |||
| for (let i = 0; i < parentDocs.length; i += 1) { | |||
| const parentDoc = parentDocs[i]; | |||
| const parentDocId = parentDocIds[i]; | |||
| const subDocs = await this.childSplitter.splitDocuments([parentDoc]); | |||
| const chunkHeaderOptions = chunkHeader && parentDoc?.metadata?.chunkHeader ? { | |||
| chunkHeader: parentDoc.metadata.chunkHeader, | |||
There was a problem hiding this comment.
Depending on this to be in the metadata field of the document is a bit weird - could we make this a property on the class instead? Like this.childSplitterOptions?
There was a problem hiding this comment.
Ignore the above, I see the problem. Let me think about it a little longer - would be nice to not have a dependency on an untyped metadata key like this.
There was a problem hiding this comment.
I think we just have to the try the same way of passing chunkHeader as this is currently implemented in LangChainJs. It will be better solution, I will see if this can be done the same way. Thanks for spotting.
|
See comment, also please run |
karol-f
force-pushed
the
chunk-header-ParentDocumentRetriever
branch
from
e93e1ce
to
447dd93
Compare
karol-f
force-pushed
the
chunk-header-ParentDocumentRetriever
branch
2 times, most recently
from
1ed471a
to
97a739d
Compare
karol-f
force-pushed
the
chunk-header-ParentDocumentRetriever
branch
from
97a739d
to
c28f039
Compare
|
Thank you again for the suggestions. I've pushed the improved version that looks cleaner and doesn't use |
dosubot
bot
added
size:L
jacoblee93
added
lgtm
|
Thank you! |
Add possibility to use Contextual chunk headers in Parent Document Retriever.
This is particularly important if you have several fine-grained child chunks that need to be correctly retrieved from the vector store.