Feat: Native hierarchies for docx element types

[newelh]

Summary

Improves hierarchy from docx files by leveraging natural hierarchies built into docx documents. Hierarchy can now be detected from an indentation level for list bullets/numbers and by style name (e.g. Heading 1, List Bullet 2, List Number).

Hierarchy detection is improved by determining category depth via the following:

1. Check if the paragraph item has an indentation level (ilvl) xpath - these are typically on list bullet/numbers. Return the indentation level if it exists
2. Check the name of the paragraph style if it contains any category depth information (e.g. Heading 1 vs Heading 2 or List Bullet vs List Bullet 2). Return the category depth if found, else default to depth of 0.
3. Check the paragraph ilvl via the paragraph's style name. Outside of the paragraph's metadata, docx stores default ilvls for various style names, which requires a complex lookup. This check is yet to be implemented, as the above methods cover most usecases but the implementation is stubbed out.

Testing

This PR adds an additional docx file to example-docs: category-level.docx. This document illustrates heading and bullet hierarchies in a docx file.

Unit tests

1. test_unstructured/partition/docx/test_docx::test_parse_category_depth_by_style-- Verifies that a paragraph correctly returns the expected category depth.
2. test_unstructured/partition/docx/test_docx::test_parse_category_depth_by_style_name-- Verifies that the category depth for the majority of style names work correctly.
3. test_unstructured/partition/docx/test_docx::test_parse_category_depth_by_style_ilvl-- Verifies the stubbed out implementation returns 0 for category depth.

Example

Sample document:

Before

After

[scanny]

LGTM Newel :)

A couple of remarks to consider.

[scanny]

You might extracting paragraphs = partitioner._document.paragraphs to a line at the top here just to reduce the word-count below. The repeated dereferencings don't buy you anything as far as I can see. They're also expensive, but of course that only matters at scale; still, as a matter of style you wouldn't want repeated identical dereferencings (say more than two; each dot is a dict lookup) in your production code so consistent style would be to apply the same to test code.

You could also consider a more compact structure like:

    test_cases = [
        (0, "Some text before any heading"),
        (0, "A Heading 1"),
        # etc. ...
    ]

    partitioner = _DocxPartitioner("example-docs/category-level.docx", None, None, False, None)
    paragraphs = partitioner._document.paragraphs

    for idx, (depth, text) in enumerate(test_cases):
        paragraph = paragraphs[idx]
        actual_depth = partitioner._parse_category_depth_by_style(paragraph)
        assert (
            actual_depth == depth
        ), f"expected paragraph[{idx}] to have depth=={depth}, got {actual_depth}"
        assert text in paragraph.text, f"paragraph[{idx}].text does not contain {text}"

Then you could probably see the whole test without scrolling. This also has the benefit being able to produce nice test-failure messages without repeating yourself a lot.

Your call though, just an idea :)

[newelh]

Much more elegant. Thank you!