Enhance textractor to better support RAG use cases

[davidmezzetti]

Currently, the textractor pipeline is rather basic. It simply calls Apache Tika (if available) and returns the raw text. If Tika isn't available, the textractor converts HTML to raw text.

Tika is a mature and stable project with a large number of file formats supported. It also supports extracting content to XHTML. The following improvements should be made to better support downstream retrieval augmented generation (RAG) use cases.

• Support section parsing. This will add a new flag called sections. When enabled, it will split the text by section or page breaks. This will better organize content into related sections.
• Improve paragraph parsing. Add better paragraph detection.
• Preserve whitespace formatting. Currently the textractor strips a lot of formatting that would be useful to a RAG pipeline.
• Export all formats to XHTML. Add an XHTML parser that can cleanly convert content to raw text.