CharacterTextSplitter with keep_separator=True sets the separator to the beginning of each chunk instead of an end

[VPetukhov]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

from langchain.text_splitter import CharacterTextSplitter

splitter = CharacterTextSplitter(
    chunk_size=10,
    chunk_overlap=0,
    separator=". ",
    keep_separator=True,
)

for d in splitter.create_documents(["Text 1. Text 2. Text 3. Text 4."]):
    print(d.page_content)
    print("---")

Error Message and Stack Trace (if applicable)

Text 1
---
. Text 2
---
. Text 3
---
. Text 4.
---

Description

I'm trying to split text by sentence, while keeping end-of-sentence punctuation. Instead of putting the punctuation back at the end of the corresponding chunk, the library adds it to the front of the following chunk.

This problem is quite critical if the output is used for text-to-speech input.

System Info

langchain==0.1.16
langchain-community==0.0.34
langchain-core==0.1.46
langchain-text-splitters==0.0.1

MacOS, Python 3.10.13

[harsh204016]

Hi @VPetukhov , can I work on this issue ?