Skip to content

v0.5.4-dev

v0.5.4-dev: feat: add inline::unstructured file processor (#6159)
@sahana-sreeram sahana-sreeram tagged this 26 Jun 17:47 
# What does this PR do?
Adds inline::unstructured as an opt-in file processor provider for
processing 65+ document formats using the local Unstructured library.
Follows the same structure as inline::docling provider.

Provides the same multi-format document processing capabilities with
structure-aware chunking options as remote::unstructured-api, but runs
locally without API costs. Follows the same structure as inline::docling
provider.

## Test Plan
Tested with real PDF processing via /v1alpha/file-processors/process
endpoint:

``` bash
  # Start server
  uv run ogx stack run \
    --providers "file_processors=inline::unstructured,files=inline::localfs,vector_io=inline::faiss,inference=inline::sentence-transformers" \
    --port 8321

  # Process PDF
  curl -X POST http://localhost:8321/v1alpha/file-processors/process \
      -F "file=@example_pdf" \
      -F 'chunking_strategy={"type":"auto"}

Results:
  {
      "metadata": {
          "processor": "unstructured",
          "processing_time_ms": 44121,
          "extraction_method": "unstructured-local",
          "file_size_bytes": 25766,
          "total_elements": 174,
          "strategy": "auto"
      }
  }
```
Output:
- Processed 25.7 KB PDF in 44 seconds
- Extracted 174 elements
- Created 2 chunks with auto chunking strategy
- Metadata preserved (filename, page numbers, element types)
- All unit tests passed

---------

Signed-off-by: Sahana Sreeram <sahanasreeram01@gmail.com>
Assets 2
Loading