Add time filtering to SDK + extra model fields

[klpoland]

No description provided.

[semanticdiff-com]

Review changes with  

Changed Files

File	Status
gateway/sds_gateway/api_methods/tasks.py	59% smaller
gateway/sds_gateway/users/views/downloads.py	57% smaller
sdk/src/spectrumx/ops/pagination.py	31% smaller
gateway/sds_gateway/api_methods/tests/test_celery_tasks.py	29% smaller
gateway/sds_gateway/api_methods/helpers/temporal_filtering.py	28% smaller
sdk/tests/ops/test_paginator.py	25% smaller
sdk/src/spectrumx/api/sds_files.py	16% smaller
sdk/src/spectrumx/client.py	12% smaller
gateway/sds_gateway/api_methods/views/dataset_endpoints.py	10% smaller
gateway/sds_gateway/api_methods/serializers/capture_serializers.py	10% smaller
sdk/src/spectrumx/gateway.py	5% smaller
sdk/src/spectrumx/models/captures.py	5% smaller
gateway/sds_gateway/api_methods/views/file_endpoints.py	4% smaller
gateway/pyproject.toml	Unsupported file format
gateway/sds_gateway/api_methods/serializers/dataset_serializers.py	0% smaller
gateway/sds_gateway/api_methods/serializers/file_serializers.py	0% smaller
gateway/sds_gateway/api_methods/tests/test_composite_capture_serialization.py	0% smaller
gateway/sds_gateway/api_methods/tests/test_dataset_endpoints.py	0% smaller
gateway/sds_gateway/api_methods/tests/test_dataset_manifest_filters.py	0% smaller
gateway/sds_gateway/api_methods/tests/test_file_endpoints.py	0% smaller
gateway/sds_gateway/api_methods/tests/test_file_get_serializer_captures.py	0% smaller
gateway/sds_gateway/api_methods/utils/dataset_manifest_filters.py	0% smaller
gateway/sds_gateway/api_methods/utils/swagger_example_schema.py	0% smaller
sdk/docs/mkdocs/changelog.md	Unsupported file format
sdk/pyproject.toml	Unsupported file format
sdk/src/spectrumx/api/datasets.py	0% smaller
sdk/src/spectrumx/models/capture_enums.py	0% smaller
sdk/src/spectrumx/models/datasets.py	0% smaller
sdk/tests/integration/test_file_ops.py	0% smaller
sdk/tests/models/test_files.py	0% smaller
sdk/tests/ops/test_datasets.py	0% smaller
sdk/tests/ops/test_files.py	0% smaller
sdk/tests/test_client.py	0% smaller

[Copilot]

Pull request overview

This PR adds end-to-end temporal filtering for DigitalRF (rf@*.h5) file listings/downloads, and expands dataset/capture models and APIs to expose richer relationship/ownership metadata (including a new dataset detail endpoint).

Changes:

• SDK: Add start_time / end_time support to list_files and download, plus pagination propagation and warning forwarding.
• Gateway: Add temporal query params to file listing with a warnings array included in paginated responses; add dataset detail retrieve returning captures + artifact files.
• Models/serializers/tests: Introduce shared capture enums, expand dataset/capture fields, and add unit/integration coverage for temporal filtering and composite capture serialization.

Reviewed changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
sdk/tests/ops/test_paginator.py	Adds unit tests for preserving temporal kwargs across pages and logging API warnings.
sdk/tests/ops/test_files.py	Adds unit tests for temporal param validation and datetime-to-ISO query formatting.
sdk/tests/integration/test_file_ops.py	Adds integration tests for temporal listing/download behavior and warning logging.
sdk/src/spectrumx/ops/pagination.py	Forwards first-page warnings from API via log_user_warning.
sdk/src/spectrumx/models/datasets.py	Adds captures/files to Dataset and introduces nested dataset-side models.
sdk/src/spectrumx/models/captures.py	Moves enums to shared module; adds additional indexed time display fields and sharing flags.
sdk/src/spectrumx/models/capture_enums.py	New shared CaptureType / CaptureOrigin enums module.
sdk/src/spectrumx/gateway.py	Adds start_time/end_time query params to list_files; adds get_dataset.
sdk/src/spectrumx/client.py	Adds temporal params to download/list_files; adds dataset convenience methods.
sdk/src/spectrumx/api/sds_files.py	Validates paired temporal params and formats datetimes as UTC ISO for gateway.
sdk/src/spectrumx/api/datasets.py	Adds dataset detail fetch + helpers to list captures/artifact files from that payload.
sdk/pyproject.toml	Updates Ruff per-file ignores for monorepo path layouts.
sdk/docs/mkdocs/changelog.md	Documents new temporal filtering and model expansions.
gateway/sds_gateway/api_methods/views/file_endpoints.py	Adds temporal query params, RF filtering, and always returns warnings in paginated responses.
gateway/sds_gateway/api_methods/views/dataset_endpoints.py	Adds dataset retrieve endpoint returning dataset metadata with captures + artifact files.
gateway/sds_gateway/api_methods/utils/swagger_example_schema.py	Adds warnings key to example paginated file list response.
gateway/sds_gateway/api_methods/tests/test_file_endpoints.py	Adds tests for warnings key presence and temporal filtering behavior.
gateway/sds_gateway/api_methods/tests/test_dataset_endpoints.py	Adds test for dataset detail retrieval containing captures/files.
gateway/sds_gateway/api_methods/tests/test_composite_capture_serialization.py	Adds serializer-level tests for multi-channel composite capture output.
gateway/sds_gateway/api_methods/tests/test_celery_tasks.py	Updates temporal filtering task docstring references.
gateway/sds_gateway/api_methods/serializers/file_serializers.py	Adds a nested artifact file summary serializer for dataset payloads.
gateway/sds_gateway/api_methods/serializers/dataset_serializers.py	Extends dataset serializer to embed captures/artifact files and break serializer cycles.
gateway/sds_gateway/api_methods/serializers/capture_serializers.py	Adds new derived time fields and enriches composite channel rows with OpenSearch bounds.
gateway/sds_gateway/api_methods/helpers/temporal_filtering.py	Refactors temporal filtering to share filter_files_by_temporal_bounds.
gateway/pyproject.toml	Adds Ruff ignore for composite serialization test magic numbers.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[lucaspar]

let me know if you need help with the conflicts after #279

[klpoland]

See changelog for full list of changes, but basically added some methods to make dataset captures and artifact files more easily visible (list those directly) and then added some new parameters for getting/downloading dataset files:

capture_uuids - for users entering the captures directly that they wish to download from a dataset
top_level_dirs - this could be capture top_level_dir to download OR just directories to select and download files within (could be a directory with multiple connected captures or artifact files)
artifacts_only - boolean for users to set to only download artifact files, this will nullify any entered capture_uuids but they can still provide top_level_dirs to filter artifact files to download further if they wish (say it's a large dataset with folders of artifacts included and they only want to download a specific directory of artifact files or smth)

lmk if this makes sense or if you have any logic suggestions/modifications here.