Preserve structured topic references during finalization

[DemonGiggle]

Summary

• send structured source reference objects in digest and finalize prompts
• restore topic references from fallback topics when finalized provider payloads omit them
• add regression coverage for structured prompt payloads and reference restoration

Testing

• PYTHONPATH=src .venv/bin/pytest -q

[gemini-code-assist]

Code Review

This pull request refactors the handling of source references during topic finalization. It introduces a fallback mechanism in the parsing logic to restore missing references from original topics using their slugs if the LLM output is incomplete. Additionally, it standardizes the reference payload format within prompts to include specific fields like source ID and path. Review feedback suggests improving the robustness of the JSON parsing by ensuring that null values returned by the LLM are converted to empty strings rather than the literal string "None", which could otherwise break fallback lookups and data integrity.

[gemini-code-assist]

Using str(raw_ref.get(key, "")) can lead to the string "None" if the LLM returns a null value for a field in the JSON payload. It is safer to use the or operator to ensure a None value is converted to an empty string before calling str().

Suggested change

	source_id = str(raw_ref.get("source_id", "")).strip()
	source_path = str(raw_ref.get("source_path", "")).strip()
	locator = str(raw_ref.get("locator", "")).strip()
	source_id = str(raw_ref.get("source_id") or "").strip()
	source_path = str(raw_ref.get("source_path") or "").strip()
	locator = str(raw_ref.get("locator") or "").strip()

[gemini-code-assist]

Similar to the reference parsing, using str(raw_topic.get("slug", "")) will result in the string "None" if the LLM returns a null slug. This can break the fallback reference lookup on line 49. Using or "" ensures None is handled correctly.

Suggested change

	slug = str(raw_topic.get("slug", "")).strip()
	slug = str(raw_topic.get("slug") or "").strip()

[gemini-code-assist]

The title field should also be protected against null values from the LLM to avoid literal "None" strings in the finalized output.

Suggested change

	title=str(raw_topic.get("title", "")).strip(),
	title=str(raw_topic.get("title") or "").strip(),