fix: deep-copy document metadata in Markdown, CSV and hierarchical splitters

[bharadwaj-pendyala]

Related Issues

• fixes Splitters share nested metadata between chunks and with the input document #12248

Proposed Changes:

MarkdownHeaderSplitter and CSVDocumentSplitter copied the input document's meta shallowly, so every chunk they emitted held the same nested objects as its siblings and as the caller's document. HierarchicalDocumentSplitter had the same problem on its root node.

doc = Document(content="# Setup\nInstall it.\n# Usage\nRun it.", meta={"tags": ["docs"]})
chunks = MarkdownHeaderSplitter().run(documents=[doc])["documents"]
chunks[0].meta["tags"].append("install")
print(chunks[1].meta["tags"], doc.meta["tags"])

On main (51f78e5) that prints ['docs', 'install'] ['docs', 'install']. Both the sibling chunk and the caller's document picked up an edit made to the first chunk.

The fix is deepcopy, matching DocumentSplitter (document_splitter.py:325), RecursiveDocumentSplitter (recursive_splitter.py:437), EmbeddingBasedDocumentSplitter (embedding_based_document_splitter.py:501), DocumentCleaner and CSVDocumentCleaner. Those five were already correct; these three were the outliers.

For HierarchicalDocumentSplitter the copy goes at the root call site rather than inside _add_meta_data. _add_meta_data also runs on every child, and children come out of DocumentSplitter with metadata that has already been copied, so putting it there would copy each child's metadata a second time for nothing.

Scope

Only the three splitters. The extractors at llm_document_content_extractor.py:354 and llm_metadata_extractor.py:364 also build {**document.meta}, but they emit one document per input rather than many chunks, so I left them alone.

I did not change Document to copy meta on construction. That would put the cost on every document in every pipeline and change identity semantics repo-wide, which is a bigger call than this bug needs. The convention here is that the component producing new documents owns the copy, same as #12107 did for normalize_metadata.

How did you test it?

Three regression tests, one per component. All three fail on main and pass with the fix:

$ hatch run test:unit -k nested_metadata_is_not_shared
# with haystack/ reverted: 3 failed, 6262 deselected
# with the fix:            3 passed, 6262 deselected

Full gates on Python 3.14, macOS:

• hatch run test:unit test/components/preprocessors/ : 354 passed, 19 deselected
• hatch run test:unit : 5930 passed, 8 skipped, 2 failed. Both failures are in test/utils/test_type_serialization.py (test_output_type_serialization_typing_generic_with_nonetype, test_output_type_serialization_callable_with_parameter_list) and reproduce on a clean main checkout, so they are not from this change.
• hatch run test:types : Success, no issues found in 391 source files
• hatch run fmt-check : All checks passed

Notes for the reviewer

Cost, since deepcopy is not free and CSVDocumentSplitter now does it once per sub-table. Measured on this branch, best of five runs:

case	main	this branch
40 sub-tables, meta {"file_path", "source_id", "tags": [2 items]}	0.061s	0.063s
40 sub-tables, meta with a 500-item list and a 200-key nested dict	0.083s	0.120s
200 markdown chunks, realistic meta	n/a	0.014s

Realistic metadata is noise. The second row is a deliberately unpleasant payload and still only costs 37ms across 40 chunks. Copying per chunk is what makes siblings independent, so hoisting a single copy out of the loop would fix the leak into the caller's document but leave the chunks sharing state with each other.

Document.meta is documented as "Must be JSON-serializable" (document.py:42), so deepcopy is safe for anything the contract allows, and split Document.ids are unchanged because ids come from json.dumps(self.meta, sort_keys=True, ...).

This PR was fully generated with an AI assistant. I have reviewed the changes and run the tests and checks listed above.

Checklist

• I have read the contributors guidelines and the code of conduct.
• I have updated the related issue with new insights and changes.
• I have added unit tests and updated the docstrings.
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
• I have documented my code.
• I have added a release note file, following the contributors guidelines.
• I have run pre-commit hooks and fixed any issue.

[vercel]

@bharadwaj-pendyala is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

[anakin87]

See #12248 (comment)

[CLAassistant]

All committers have signed the CLA.

[HaystackBot]

Hi @bharadwaj-pendyala, thanks a lot for your contribution! 🙏

We noticed that the Contributor License Agreement (CLA) check (license/cla) hasn't passed yet, so we've temporarily moved this PR to draft and paused the review assignment.

To get your PR reviewed, please sign the CLA via the link in the license/cla check below (or in the CLA bot comment). As soon as the check turns green, this PR will automatically be marked ready for review again and a reviewer will be re-assigned.

[HaystackBot]

Thanks for signing the CLA, @bharadwaj-pendyala! 🎉 This PR is now ready for review again and the reviewer has been re-assigned.

[sjrl]

We use restructured text markdown so for inline comments please use double back ticks

Suggested change

	`MarkdownHeaderSplitter` and `CSVDocumentSplitter` now deep-copy the metadata of the document they split,
	``MarkdownHeaderSplitter`` and ``CSVDocumentSplitter`` now deep-copy the metadata of the document they split,

[github-actions]

Coverage report

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
haystack/components/preprocessors
csv_document_splitter.py
hierarchical_document_splitter.py
markdown_header_splitter.py
Project Total

This report was generated by python-coverage-comment-action

[bharadwaj-pendyala]

recheck