Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Documentation error under Text Splitter #3414

Closed
durgendra opened this issue Apr 24, 2023 · 2 comments
Closed

Documentation error under Text Splitter #3414

durgendra opened this issue Apr 24, 2023 · 2 comments

Comments

@durgendra
Copy link

Current documentation text under Text Splitter throws error :
texts = text_splitter.create_documents([state_of_the_union])
Screen Shot 2023-04-23 at 9 04 28 PM

The error is on both these pages
https://python.langchain.com/en/latest/modules/indexes/text_splitters/getting_started.html
https://python.langchain.com/en/latest/modules/indexes/text_splitters/examples/character_text_splitter.html

I think the above line should be revised to
texts = text_splitter.split_documents([state_of_the_union])

@lengocgiang
Copy link

You can try to removing the separators=["\n\n", "\n"] parameter in the RecursiveCharacterTextSplitter function.
After removing it, the code would looks like this

   text_splitter = RecursiveCharacterTextSplitter(chunk_size=chunk_size, chunk_overlap=chunk_overlap)

In the document document_transformers, default the characters it tries to split on are ["\n\n", "\n", " ", ""]
https://python.langchain.com/docs/modules/data_connection/document_transformers/

@dosubot
Copy link

dosubot bot commented Sep 21, 2023

Hi, @durgendra! I'm Dosu, and I'm helping the LangChain team manage their backlog. I wanted to let you know that we are marking this issue as stale.

From what I understand, you pointed out an error in the documentation for the Text Splitter in the LangChain repository. Lengocgiang suggested a possible solution by removing a parameter in the code. However, I see that the issue has been resolved and the error in the documentation has been fixed.

Before we close this issue, I wanted to check with you if it is still relevant to the latest version of the LangChain repository. If it is, please let the LangChain team know by commenting on this issue. Otherwise, feel free to close the issue yourself or it will be automatically closed in 7 days.

Thank you for your contribution to the LangChain project! Let me know if you have any further questions or concerns.

@dosubot dosubot bot added the stale Issue has not had recent activity or appears to be solved. Stale issues will be automatically closed label Sep 21, 2023
@dosubot dosubot bot closed this as not planned Won't fix, can't repro, duplicate, stale Sep 28, 2023
@dosubot dosubot bot removed the stale Issue has not had recent activity or appears to be solved. Stale issues will be automatically closed label Sep 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants