chore: add validation for chunk sizes and chunking methods

[Mateusz-Switala]

Assisted-by: Claude Code

Description

Extends prepare_search_space_with_ogx to accept optional chunking_method and chunk_size
overrides directly via the payload, and adds chunk_sizes as a new parameter to
prepare_search_space_report. Also lowers the minimum allowed chunk size to 128 tokens and
deduplicates any repeated values in the chunking inputs.

Motivation

The existing prepare_search_space_with_ogx always used platform-default chunking methods
(recursive, hybrid) and chunk sizes (512, 1024, 2048). Users running experiments on
shorter documents or with a single chunking strategy had no way to constrain these dimensions
without post-processing the search space manually. This change exposes that control at the
preparation step, in the single function callers already use.

Changes

• ai4rag/utils/constants.py — lowered ChunkingConstraints.MIN_CHUNK_SIZE from 512 to 128
• ai4rag/search_space/prepare/input_payload_types.py — added optional chunking_methods and
  chunk_sizes fields to AI4RAGConstraints; both are pydantic-validated (non-empty list,
  non-empty strings / positive integers respectively)
• ai4rag/search_space/prepare/prepare_search_space.py
  • Extracted three private helpers to eliminate duplication: _resolve_models_from_payload,
    _apply_language_detection, _build_model_params
  • Extended prepare_search_space_with_ogx to accept and validate chunking_methods and
    chunk_sizes in the payload; when provided they override the platform defaults for those
    dimensions; duplicate values are deduplicated with a warning
  • Updated docstring to clarify that only model validation goes through OGX; chunking is
    validated locally against ChunkingConstraints
• ai4rag/search_space/prepare/__init__.py — removed prepare_search_space_custom
  (functionality merged into prepare_search_space_with_ogx)
• ai4rag/components/optimization/search_space_preparation.py
  • Added chunk_sizes: list[int] | None = None parameter to prepare_search_space_report
  • Updated _validate_chunking_methods to validate values against ChunkingConstraints.METHODS,
    providing an early, clear error
  • prepare_search_space_report now passes chunking_methods and chunk_sizes into the payload
    and calls prepare_search_space_with_ogx

Testing

• Added tests for chunking overrides, individual and combined, via prepare_search_space_with_ogx
• Added tests for duplicate deduplication in chunking_methods and chunk_sizes
• Added TestValidateChunkSizes (11 tests) covering: None, valid list, single element,
  non-list, empty list, zero, negative, float, below-min, above-max, and exact boundary values
• Added TestPrepareSearchSpaceReportChunkSizesValidation (3 tests) covering: non-list, empty
  list, and non-positive integer
• Simplified TestUnsupportedChunkingMethods — validation fires before any I/O so complex
  mocking is no longer needed

Checklist

• Tests added/updated
• Documentation updated
• Code follows style gu

[filip-komarzyniec]

🆗 /lgtm

[filip-komarzyniec]

some comments to think through, everything else totally fine 👌🏻