feat: conditional framework extras

[marcellodebernardi]

This PR aligns framework dependency handling with the internal implementation by making CatBoost/LightGBM optional, dynamically detecting installed frameworks for defaults, validating user-selected model types, and documenting framework/task extras. It also adds Docker build args for optional extras, ensures the Keras backend is set when retrain runs outside the main entry point, and logs a search success summary at the end of the workflow. The aim is to keep the open-source packaging flexible while preserving a batteries-included default image.

Testing

poetry run pytest tests/unit/

[Copilot]

Pull request overview

This PR makes CatBoost and LightGBM optional framework dependencies while preserving a batteries-included default Docker image. The core changes implement dynamic framework detection, validate user selections against installed packages, add proper ImportError handling, and introduce new extras for flexible installation options. The PR also adds Docker build arguments for controlling which optional frameworks are installed, ensures Keras backend configuration when retrain runs standalone, and logs a search success summary at workflow completion.

Changes:

• Made CatBoost and LightGBM optional dependencies with new package extras (catboost, lightgbm, tabular, vision)
• Implemented detect_installed_frameworks() to dynamically determine available ML frameworks at runtime
• Added ImportError handling with clear error messages in submission.py and retrain.py for optional frameworks
• Added Docker build args (INSTALL_PYTORCH, POETRY_EXTRAS) to control which frameworks are installed in images
• Added search summary logging to display successful/failed solution attempts at workflow completion
• Updated tests to handle optional framework scenarios gracefully

Reviewed changes

Copilot reviewed 11 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
pyproject.toml	Version bump to 1.3.0; moved catboost/lightgbm to optional dependencies; added new extras (catboost, lightgbm, tabular, vision)
poetry.lock	Updated lock file to reflect optional status of catboost, lightgbm, and their transitive dependencies; updated vision extra markers
plexe/config.py	Added detect_installed_frameworks() function to dynamically detect installed frameworks; made DEFAULT_MODEL_TYPES computed from detection
plexe/helpers.py	Added validation that user-selected frameworks are installed before use; raises ValueError with clear message if not
plexe/retrain.py	Added KERAS_BACKEND environment setup for standalone invocation; wrapped keras import in try/except; added ImportError handling for catboost/lightgbm
plexe/tools/submission.py	Added ImportError handling with clear error messages for catboost and lightgbm model validation
plexe/workflow.py	Added search summary logging showing successful/total solution attempts at workflow completion
tests/unit/test_imports.py	Added logic to detect missing optional frameworks and skip importing their modules during tests
tests/unit/test_helpers.py	Updated test to dynamically check installed frameworks and skip if no image-capable frameworks available
Dockerfile	Added build args for PyTorch installation and poetry extras selection; default includes aws and catboost
Makefile	Removed extra blank line (cosmetic)
README.md	Updated installation documentation to explain core vs optional dependencies and new extras options

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[greptile-apps]

Greptile Summary

This PR successfully implements optional framework dependencies while maintaining a batteries-included default experience. The changes make CatBoost and LightGBM optional dependencies, dynamically detect installed frameworks, and validate user selections before attempting to use them.

Key improvements:

• Framework dependencies are now properly tiered: core (XGBoost, Keras), optional (CatBoost, LightGBM, PyTorch)
• Dynamic framework detection via detect_installed_frameworks() ensures the system adapts to what's installed
• User-facing validation in helpers.py prevents cryptic import errors by checking availability upfront
• Defensive error handling in retrain.py and submission.py provides clear messages if frameworks are missing
• Docker build is now configurable via INSTALL_PYTORCH and POETRY_EXTRAS args for custom deployments
• Tests properly skip framework-specific modules when dependencies aren't installed
• Keras backend is explicitly set to TensorFlow before import in retrain.py for standalone invocations

Additional enhancements:

• Added search success summary logging at workflow completion
• Documented new extras structure in README (tabular, vision, framework-specific extras)

The implementation is thorough, well-tested, and maintains backward compatibility through sensible defaults.

Confidence Score: 5/5

• This PR is safe to merge with minimal risk
• The changes are well-architected with proper validation, error handling, and test coverage. The implementation follows a defensive programming approach with validation at the entry point (helpers.py) and additional safeguards in usage locations. All framework imports are properly guarded, tests handle missing dependencies gracefully, and the documentation accurately reflects the new structure. The PR maintains backward compatibility through sensible defaults.
• No files require special attention

Important Files Changed

Filename	Overview
pyproject.toml	Moved CatBoost and LightGBM to optional dependencies, added task-based extras (tabular, vision), bumped version to 1.3.0
plexe/config.py	Added detect_installed_frameworks() to dynamically detect available ML frameworks, treats XGBoost and Keras as always available
plexe/helpers.py	Added validation to ensure user-selected frameworks are actually installed, provides clear error messages for missing dependencies
plexe/retrain.py	Set Keras backend before import to support standalone retrain invocations, wrapped CatBoost/LightGBM imports in try-except with clear error messages
plexe/tools/submission.py	Wrapped CatBoost/LightGBM imports in try-except blocks to handle missing optional dependencies gracefully
Dockerfile	Made PyTorch installation conditional via INSTALL_PYTORCH build arg, made poetry extras configurable via POETRY_EXTRAS (default: aws catboost)

_{Last reviewed commit: 4e73dc3}

[greptile-apps]

_{12 files reviewed, 2 comments}

_{Edit Code Review Agent Settings | Greptile}

[marcellodebernardi]

@greptileai please review again with latest changes

[marcellodebernardi]

@greptileai please review again with latest changes