How to do paragraph/section text splitting?

[craighagerman]

The "Long Text Splitting" demo page (see link below) shows examples of paragraph and section text splitting but there is no associated Colab notebook. I don't see any mention of paragraph or section text splitting annotators anywhere in the code docs. Does any one have a code sample for how this is done? Of can anyone explain the approach used for paragraph and section splitting with JSL?

• JSL Finance Demos: https://nlp.johnsnowlabs.com/financial_document_splitting
• Text splitting Demo link: https://demo.johnsnowlabs.com/finance/TEXT_SPLITTING/

[maziyarpanahi]

Hi,

That seems to be a licensed demo, so I can't speak to it. But now that we have Llama.cpp native support, maybe we can use a small gguf model internally, detect section, paragraphs, etc. and extract chunks.

@DevinTDHa what do you think?

[DevinTDHa]

Hi Everyone,

I'm not sure what the licensed text splitting model is based on. I am assuming it is a token or sentence entity recognition model.

@maziyarpanahi Are you referring to using a LLM in Llama.cpp as a zero shot classifier? Or perhaps use it to create embeddings for classification? I think both would be possible, but I am not sure about the accuracy.

[maziyarpanahi]

Thanks @DevinTDHa
I was mostly thinking on having a new annotator called ParagraphSplitter and inside we can use a tiny LLM to classify each sections, paragraphs, etc. I know we can use question-answering models for that, the spit out begin and end for each questions that can be "what's the title", "what's the section", etc. but we need to train one

[maziyarpanahi]

Like I can do this with Llama or Qwen, i know it's not perfect, but we don't need to train a specific model for it. Unless this QuestionAnswering model already exists openly:

[
  {
    "label": "HEADER",
    "begin": 0,
    "end": 3,
    "proof": "Execution Version INTELLECTUAL"
  },
  {
    "label": "SECTION",
    "begin": 4,
    "end": 63,
    "proof": "This INTELLECTUAL PROPERTY AGREEMENT"
  },
  {
    "label": "SECTION",
    "begin": 64,
    "end": 134,
    "proof": "WHEREAS, Seller and Buyer have entered"
  }
]

[craighagerman]

Interesting idea @maziyarpanahi - I'll play around with this idea a bit.
I thought perhaps there was some obvious annotator that was being used in those demos. (Why have demos and be completely opaque about them? Oh well.)

I found this description of text splitting in the 01.Page_Splitting.ipynb notebook:

1.4. Paragraph / Section / Items splitting
📌 TextSplitter
To split by sentences or paragraphs or ...

In most cases:

- paragraphs can be extracted using just \n\n.
- sections and items may be delimited by headers and subheaders. We can train DL models to detect them or, if use numeration (if they are numerated sections) or other patterns to delimit their boundaries

So now I assume in those page/section splitting demos they are using a simplistic rule-based paragraph splitter together with a heading NER classifier for headers/subheaders. All of the (few) examples have sections that are delimited by \n\n. That isn't the case for my use case.

[DevinTDHa]

Thanks @DevinTDHa I was mostly thinking on having a new annotator called ParagraphSplitter and inside we can use a tiny LLM to classify each sections, paragraphs, etc. I know we can use question-answering models for that, the spit out begin and end for each questions that can be "what's the title", "what's the section", etc. but we need to train one

@maziyarpanahi I think this should be possible. Our AutoGGUFModel annotator also supports grammar constrained generation, which would be a good fit.