Skip to content

Add unit tests for pipeline.py + fix broken export.py import#259

Merged
tkswanson merged 4 commits into
devfrom
copilot/dev-create-comprehensive-unittest
Apr 7, 2026
Merged

Add unit tests for pipeline.py + fix broken export.py import#259
tkswanson merged 4 commits into
devfrom
copilot/dev-create-comprehensive-unittest

Conversation

Copy link
Copy Markdown

Copilot AI commented Apr 6, 2026

  • Fix pre-existing import error in src/animl/export.py
  • Create tests/test_pipeline.py (29 tests)
  • Create tests/test_split.py (46 tests) covering get_animals, get_empty, train_val_test
Original prompt

Task

Create a comprehensive unittest file at tests/test_pipeline.py on the dev branch for src/animl/pipeline.py.

Source file to test

src/animl/pipeline.py contains two public functions:

  • from_paths(image_dir, detector_file, classifier_file, classlist_file, ...)
  • from_config(config)

Both functions are orchestration pipelines that call many submodules. All submodule calls must be mocked — do not load real models or real images.

Key dependencies to mock

All of the following must be patched in tests (patch target is the animl.pipeline namespace):

  • animl.pipeline.file_management.WorkingDirectory
  • animl.pipeline.file_management.build_file_manifest
  • animl.pipeline.file_management.check_file
  • animl.pipeline.file_management.load_data
  • animl.pipeline.file_management.save_data
  • animl.pipeline.video_processing.extract_frames
  • animl.pipeline.detection.load_detector
  • animl.pipeline.detection.detect
  • animl.pipeline.detection.parse_detections
  • animl.pipeline.split.get_animals
  • animl.pipeline.split.get_empty
  • animl.pipeline.classification.load_classifier
  • animl.pipeline.classification.classify
  • animl.pipeline.classification.single_classification
  • animl.pipeline.classification.sequence_classification
  • animl.pipeline.export.export_folders
  • animl.pipeline.visualization.plot_all_bounding_boxes

WorkingDirectory mock setup

The WorkingDirectory mock must have these attributes set as MagicMock() or Path-like objects:

  • filemanifest
  • imageframes
  • detections
  • mdraw
  • predictions
  • results
  • linkdir
  • visdir

Test data setup (setUpClass)

Create shared test fixtures:

cls.image_dir = tempfile.mkdtemp()
cls.detector_file = str(Path(cls.image_dir) / 'detector.pt')
cls.classifier_file = str(Path(cls.image_dir) / 'classifier.pt')
cls.classlist_file = str(Path(cls.image_dir) / 'classes.csv')
Path(cls.detector_file).touch()
Path(cls.classifier_file).touch()

cls.mock_manifest = pd.DataFrame({
    'filepath': ['a.jpg', 'b.jpg'],
    'frame': [0, 0],
    'station': ['cam1', 'cam1'],
    'datetime': ['2023-01-01 10:00:00', '2023-01-01 10:01:00'],
    'extension': ['.jpg', '.jpg'],
})
cls.mock_animals = cls.mock_manifest.copy()
cls.mock_animals['category'] = 1
cls.mock_animals['conf'] = 0.9
cls.mock_animals['bbox_x'] = 0.1
cls.mock_animals['bbox_y'] = 0.1
cls.mock_animals['bbox_w'] = 0.2
cls.mock_animals['bbox_h'] = 0.3

cls.mock_empty = pd.DataFrame(columns=cls.mock_manifest.columns)
cls.mock_predictions = np.array([[0.9, 0.05, 0.05], [0.1, 0.8, 0.1]])
cls.mock_results = cls.mock_animals.copy()
cls.mock_results['prediction'] = 'cat'
cls.mock_results['confidence'] = 0.9
cls.mock_class_list = pd.DataFrame({'class': ['cat', 'dog', 'bird']})

from_paths tests — class TestFromPaths

Write tests using unittest.mock.patch as context managers or decorators. For each test, mock all dependencies listed above. The check_file mock should return False by default (simulate no cached files).

Tests to write:

  1. test_returns_dataframe — returns a pd.DataFrame
  2. test_calls_build_file_manifestbuild_file_manifest is called once
  3. test_calls_extract_framesvideo_processing.extract_frames is called once
  4. test_calls_load_detector_for_pt_fileload_detector called with model_type="mdv5" for .pt file
  5. test_calls_load_detector_for_onnx_fileload_detector called with "onnx" for .onnx detector file. Create a separate detector_onnx path for this.
  6. test_calls_detectdetection.detect is called once
  7. test_calls_parse_detectionsdetection.parse_detections is called once
  8. test_calls_load_classifierclassification.load_classifier is called once
  9. test_calls_classifyclassification.classify is called once
  10. test_single_classification_called_by_defaultsingle_classification called when sequence=False
  11. test_sequence_classification_called_when_flag_setsequence_classification called when sequence=True
  12. test_detect_only_returns_early — when detect_only=True, returns detections df without calling classifier
  13. test_sort_calls_export_folders — when sort=True, export.export_folders is called
  14. test_sort_false_does_not_call_export — when sort=False, export.export_folders not called
  15. test_visualize_calls_plot_bounding_boxes — when visualize=True, visualization.plot_all_bounding_boxes called
  16. test_visualize_false_does_not_plot — when visualize=False, plot function not called
  17. test_save_data_calledfile_management.save_data is called once with the final manifest
  18. test_uses_cached_detections_when_check_file_true — when check_file returns True for detections, load_data is called and detect is NOT called
  19. test_missing_image_dir_raises — patch WorkingDirectory to raise FileNotFoundError, assert it propagates

from_config tests — class TestFromConfig

Create a helper `_write_config(tmp...

This pull request was created from Copilot chat.

Copilot AI changed the title [WIP] Add comprehensive unittest file for pipeline functions Add unit tests for pipeline.py + fix broken export.py import Apr 6, 2026
Copilot AI requested a review from tkswanson April 6, 2026 23:03
@tkswanson tkswanson marked this pull request as ready for review April 7, 2026 18:55
@tkswanson tkswanson merged commit d91de56 into dev Apr 7, 2026
Copilot stopped work on behalf of tkswanson due to an error April 7, 2026 19:07
@tkswanson tkswanson deleted the copilot/dev-create-comprehensive-unittest branch May 11, 2026 19:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants