* Add CODEOWNERS
* Add core Docker image for Katabatic
* adding in the evaluation-pipeline created by yloopez. Also added fixes in for CODI and benchmark runner.py. Incorporated the katabatic-mentorship repo into the katabatic datascience repo. First draft, more cleaning required to add more models in and clean up code/documentation.
* chore: move pytest and test plugins into dev group
Move test tooling from optional extras into the [tool.poetry.group.dev.dependencies]
so CI's `poetry install --with dev` installs pytest and related packages.
* ci(workflow): upgrade Poetry install in CI to avoid poetry.lock incompatibility (fix failing PR#37 jobs)
* Fix: feature name conflict during evaluation causing integration tests to fail. Fixed by converting df to np.array.
* ruff: removed unused import
* ruff: remove used imports. Remove quotations from type hints. Add errors in exception args. Rearrange imports. Convert lambda expressions to functions. Convert unused variables to underscore. None eval change to reference equality. f-string to string.
* supply default llm to GReaT init. Remove model instantiation from integration test.
* regenerate poetry.lock. ci: pinned poetry ==2.4.1, bumped actions version, added poetry lock check, dependency caching, remove "master" from triggers.
* upgrade: poetry 1.6.1 -> 2.4.1. Fix: reorganise and add optional to dependencies.
* DEBUG: check whl name
* fix: replace with updated ci pipeline.
* Update format to conform with PEP 508
* update actions versions
* Enhance scaffold and scripts; add tests for datasets and integration
- Updated `scaffold.py` to improve usage message formatting.
- Modified `pin_notebook_kernel.py` for better argument handling.
- Introduced shared pytest fixtures in `conftest.py` for testing.
- Added tests for dataset registration and compatibility in `test_datasets.py`.
- Implemented integration tests for GANBLR and GReaT models in `test_integration_ganblr.py` and `test_integration_great.py`.
- Created legacy pipeline tests in `test_train_test_split_pipeline.py` to ensure proper functionality and artifact handling.
* update poetry.lock
* Large refactor of dataset handling and model evaluation:
* Ruff Linting
* Modified model instantiation in TrainTestSplitPipeline to handle model type correctly.
* Refactored split_dataset and included compute_train_test_split from dev branch.
* Skipped FidelityEvaluation test due to pipeline incompatability. Will mention to Eval team lead.
* Remove unnecessary import in wheel smoke test
* delete data files and add to .gitignore (files should be local-only)
* poetry version updated
* * + CI check to Docker.
* + env specification arg.
* + non-root user to avoid security issue of using root directly.
* Remove docker-build to speed up ci checks. Checks done in another feature.
* Delete .python-version
* * Implement model matrix and include ganblr, great.
* Optimisation: use cpu version of pytorch.
* add docker-build ci check
* * Set coverage gate at 60%. Current check hits 62%
* omit experimental models from coverage run.
* Decreased coverage to 45%. Reporting 48% on deployment.
* * Moved accelerate library out of core dependencies as it's only used by great.
* Add comments about ruff lint options
* update actions to latest supported version
* add 'changes' job to run only changed per-model/core CI checks
* Upgrade project dev packages
* Fix action execution: Generate model matrix before if conditional
* Security: Add bandit and pip-audit.
* ignore local project package check
* Security: Update pip-audit command to use requirements file for auditing
* Use Open Source Vulnerabilities database backend to avoid rate-limiting.
* Security: update tornado 6.5.1 -> 6.5.7
* Security:add coverage reporting
* Add Ruff pre-commit configuration
* fix logic in inger_categorical_columns check
* Replace error logic with pytest
* Implement ctgan test integration and add to model marker
* Bumped up tooling versions
* Added pre-commit to CI
* ruff formatting
* ruff formatting. Removed deprecated utils.py
* ruff fixes
* ruff fixes
* Removed ctgan from omit code coverage. Add Alex as author.
* Refactored model matrix harness to minimise required updates to model test implementation.
* fixed spacing
* Add pre-commit to dev dependencies.
Add quality checks in MakeFile.
Remove .md files from ruff formatting.
* fix whitespace
* Updating the GANBLR codebase and associated benchmark runner files.
* Updating the code to run through pre-commit checks.
* add model state persistence
* Fix ARTIFACT_STATE_FILES naming and correctly handle setting of test harness epoch kwargs.
* implement parameterized model promotion contract test (S1-7)
* Add public method for retrieving model config from ModelRegistry. Ruff fixes.
* Add additional checks in test model registry and use public method in ModelRegistry.
* Remove other model checks and moved to test_model_registry.py
* Add model contract job
* Add generic install-model make command, remove model-specific ones. Add contract job.
* ignore test_model_registry.py in coverage. Evaluated in `model-contract` job
* Add core output to changes job and update merge-coverage conditions
* chore: Moved CODEOWNERS into corrects directory-level
* Optimize Docker model dependencies
* chore: update codeowners for refined PR allocation.
chore: add semantic commit message requirement.
* chore: streamline Dockerfile by removing unused ARG and optimising installation steps
* chore(ci): bump the actions group with 3 updates (#73)
Bumps the actions group with 3 updates: [actions/checkout](https://github.com/actions/checkout), [actions/setup-python](https://github.com/actions/setup-python) and [actions/download-artifact](https://github.com/actions/download-artifact).
Updates `actions/checkout` from 5 to 7
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v5...v7)
Updates `actions/setup-python` from 6 to 7
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v6...v7)
Updates `actions/download-artifact` from 7 to 8
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v7...v8)
---
updated-dependencies:
- dependency-name: actions/checkout
dependency-version: '7'
dependency-type: direct:production
update-type: version-update:semver-major
dependency-group: actions
- dependency-name: actions/setup-python
dependency-version: '7'
dependency-type: direct:production
update-type: version-update:semver-major
dependency-group: actions
- dependency-name: actions/download-artifact
dependency-version: '8'
dependency-type: direct:production
update-type: version-update:semver-major
dependency-group: actions
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): bump the dev-tooling group with 4 updates (#74)
Bumps the dev-tooling group with 4 updates: [mypy](https://github.com/python/mypy), [ruff](https://github.com/astral-sh/ruff), [ipykernel](https://github.com/ipython/ipykernel) and [pre-commit](https://github.com/pre-commit/pre-commit).
Updates `mypy` from 2.3.0 to 2.3.1
- [Changelog](https://github.com/python/mypy/blob/master/CHANGELOG.md)
- [Commits](https://github.com/python/mypy/compare/v2.3.0...v2.3.1)
Updates `ruff` from 0.16.1 to 0.16.4
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.16.1...0.16.4)
Updates `ipykernel` from 6.31.0 to 7.3.0
- [Release notes](https://github.com/ipython/ipykernel/releases)
- [Changelog](https://github.com/ipython/ipykernel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ipython/ipykernel/compare/v6.31.0...v7.3.0)
Updates `pre-commit` from 4.6.1 to 4.6.2
- [Release notes](https://github.com/pre-commit/pre-commit/releases)
- [Changelog](https://github.com/pre-commit/pre-commit/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pre-commit/pre-commit/compare/v4.6.1...v4.6.2)
---
updated-dependencies:
- dependency-name: mypy
dependency-version: 2.3.1
dependency-type: direct:development
update-type: version-update:semver-patch
dependency-group: dev-tooling
- dependency-name: ruff
dependency-version: 0.16.4
dependency-type: direct:development
update-type: version-update:semver-patch
dependency-group: dev-tooling
- dependency-name: ipykernel
dependency-version: 7.3.0
dependency-type: direct:development
update-type: version-update:semver-major
dependency-group: dev-tooling
- dependency-name: pre-commit
dependency-version: 4.6.2
dependency-type: direct:development
update-type: version-update:semver-patch
dependency-group: dev-tooling
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Promote CTGAN: implement artifact persistence and register as supported (#67)
* Promote Model CTGAN: implement artifact persistence and promote to supported
* fix: include ArtifactStore and ModelRef imports; ruff fixes
* fix(sec): Added nosec
---------
Co-authored-by: Talos-1 <adityaprayoga@hotmail.com>
* chore: update CODEOWNERS to include Laxmi as a contact for code ownership
* chore: update CODEOWNERS and CI configurations for frontend ownership
* chore: update CODEOWNERS and CI configurations for frontend ownership (#85)
* chore: Added Talos-1 (Alex) to all sub-team permissions for support
* chore(fix): Add PR frontend path targetting to frontend ci
* feat: add frontend directory for development (#86)
Co-authored-by: Talos-1 <adityaprayoga@hotmail.com>
* Dependency vulnerability clean-up (#66)
* Bump keras from 3.10.0 to 3.15.0
Bumps [keras](https://github.com/keras-team/keras) from 3.10.0 to 3.15.0.
- [Release notes](https://github.com/keras-team/keras/releases)
- [Commits](https://github.com/keras-team/keras/compare/v3.10.0...v3.15.0)
---
updated-dependencies:
- dependency-name: keras
dependency-version: 3.15.0
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: remove stale per-model poetry files
* chore(deps): Removed outdated tf-keras dependecy
* chore(deps): bumped torch 2.7.1 -> 2.13.0, tensorflow 2.19.* -> 2.21.*, transformers 4.53.2 -> 4.57.0
* chore(deps): regenerate lock after merging development
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore: add read permissions to ci & dependabot from `main` (#77)
* chore: add read permissions to ci
* chore(deps): add dependabot configuration for pip and GitHub Actions
* Increase open pull requests limit to 5
* chore(deps): bump the ml-stack group with 6 updates (#78)
* chore(deps): bump the ml-stack group with 6 updates
Bumps the ml-stack group with 6 updates:
| Package | From | To |
| --- | --- | --- |
| [torch](https://github.com/pytorch/pytorch) | `2.9.0` | `2.13.0` |
| [tensorflow](https://github.com/tensorflow/tensorflow) | `2.19.0` | `2.19.1` |
| [transformers](https://github.com/huggingface/transformers) | `4.53.2` | `4.57.6` |
| [datasets](https://github.com/huggingface/datasets) | `4.0.0` | `4.8.5` |
| [accelerate](https://github.com/huggingface/accelerate) | `1.10.1` | `1.14.0` |
| [xgboost](https://github.com/dmlc/xgboost) | `3.0.2` | `3.2.0` |
Updates `torch` from 2.9.0 to 2.13.0
- [Release notes](https://github.com/pytorch/pytorch/releases)
- [Changelog](https://github.com/pytorch/pytorch/blob/main/RELEASE.md)
- [Commits](https://github.com/pytorch/pytorch/compare/v2.9.0...v2.13.0)
Updates `tensorflow` from 2.19.0 to 2.19.1
- [Release notes](https://github.com/tensorflow/tensorflow/releases)
- [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md)
- [Commits](https://github.com/tensorflow/tensorflow/compare/v2.19.0...v2.19.1)
Updates `transformers` from 4.53.2 to 4.57.6
- [Release notes](https://github.com/huggingface/transformers/releases)
- [Commits](https://github.com/huggingface/transformers/compare/v4.53.2...v4.57.6)
Updates `datasets` from 4.0.0 to 4.8.5
- [Release notes](https://github.com/huggingface/datasets/releases)
- [Commits](https://github.com/huggingface/datasets/compare/4.0.0...4.8.5)
Updates `accelerate` from 1.10.1 to 1.14.0
- [Release notes](https://github.com/huggingface/accelerate/releases)
- [Commits](https://github.com/huggingface/accelerate/compare/v1.10.1...v1.14.0)
Updates `xgboost` from 3.0.2 to 3.2.0
- [Release notes](https://github.com/dmlc/xgboost/releases)
- [Changelog](https://github.com/dmlc/xgboost/blob/master/NEWS.md)
- [Commits](https://github.com/dmlc/xgboost/compare/v3.0.2...v3.2.0)
---
updated-dependencies:
- dependency-name: torch
dependency-version: 2.13.0
dependency-type: direct:development
update-type: version-update:semver-minor
dependency-group: ml-stack
- dependency-name: tensorflow
dependency-version: 2.19.1
dependency-type: direct:development
update-type: version-update:semver-patch
dependency-group: ml-stack
- dependency-name: transformers
dependency-version: 4.57.6
dependency-type: direct:development
update-type: version-update:semver-minor
dependency-group: ml-stack
- dependency-name: datasets
dependency-version: 4.8.5
dependency-type: direct:development
update-type: version-update:semver-minor
dependency-group: ml-stack
- dependency-name: accelerate
dependency-version: 1.14.0
dependency-type: direct:development
update-type: version-update:semver-minor
dependency-group: ml-stack
- dependency-name: xgboost
dependency-version: 3.2.0
dependency-type: direct:development
update-type: version-update:semver-minor
dependency-group: ml-stack
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore(fix): poetry lock
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Talos-1 <adityaprayoga@hotmail.com>
* chore(test): add importorskips
* chore: add tests to makefile, bumpy pyproject version to 0.2.0, fix dependencies, add 0.2.0 changelog.
* chore: Moved GReaT out of model matrix. Not currently supported due to unimplemented artifact state.
* chore: automate project version retrieval
* chore(test): add version consistency check in wheel-smoke job
* chore: bump version to 0.2.0rc1
* chore(docs): update changelog with 0.2.0rc1
* chore: update tags in publish workflows for PyPI and TestPyPI
* Version 0.2.0rc1 release preparation (#90)
* chore: create dependabot.yml to target development branch. (#72)
* chore: ruff linting
* chore(test): add importorskips
* chore: add tests to makefile, bumpy pyproject version to 0.2.0, fix dependencies, add 0.2.0 changelog.
* chore: Moved GReaT out of model matrix. Not currently supported due to unimplemented artifact state.
* chore: automate project version retrieval
* chore(test): add version consistency check in wheel-smoke job
* chore: bump version to 0.2.0rc1
* chore(docs): update changelog with 0.2.0rc1
* chore: update tags in publish workflows for PyPI and TestPyPI
* chore(release): 0.2.0
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Hannah Rajput <hannahrajput1803@gmail.com>
Co-authored-by: Nathan Damaj <nathan.damaj@hey.com>
Co-authored-by: Zachariah George Christie <zachariah.christie05@gmail.com>
Co-authored-by: janudi0531-debug <janudi0531@gmail.com>
Co-authored-by: mbkskarthik-beep <mbkskarthik@gmail.com>
Co-authored-by: laxmi_narsimha_reddy_nalla <126189357+laxminarsimha01@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dipsan Bhattarai <89036536+Dipsan49@users.noreply.github.com>
