Skip to content

Commit

Permalink
Update drift detection script and workflow
Browse files Browse the repository at this point in the history
  • Loading branch information
albertovalerio committed Dec 25, 2023
1 parent 24ae277 commit 46c92dd
Show file tree
Hide file tree
Showing 2 changed files with 23 additions and 11 deletions.
16 changes: 10 additions & 6 deletions .github/workflows/drift-detection.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,15 +27,15 @@ jobs:
with:
python-version: '3.11'

- name: Install Python Dependencies
- name: Install Python dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements-dev.txt
- name: Set up DVC
uses: iterative/setup-dvc@v1

- name: Pull Data with DVC
- name: Pull data with DVC
env:
GDRIVE_CREDENTIALS_DATA: ${{ secrets.GDRIVE_CREDENTIALS_DATA }}
run: |
Expand All @@ -47,11 +47,11 @@ jobs:
sudo apt-get install -y unzip
unzip data/processed/datasets_processed.zip -d data/processed/datasets_processed
- name: Data Drift Detection
- name: Data drift detection
run: |
python src/features/drift_detection.py
- name: Upload Log Artifact
- name: Upload log artifact
uses: actions/upload-artifact@v3
with:
name: drift-detection
Expand All @@ -60,7 +60,11 @@ jobs:
- name: Commit report
run: |
git status
git config --global user.name "github-actions[bot]"
git config --global user.email "github-actions[bot]@users.noreply.github.com"
git config --global user.name "Gitty[bot]"
git config --global user.email "Gitty[bot]@users.noreply.github.com"
git commit -am "Automated drift detection logs"
git push
- name: Data Drift Detection - Debug mode
run: |
python src/features/drift_detection.py debug=1
18 changes: 13 additions & 5 deletions src/features/drift_detection.py
Original file line number Diff line number Diff line change
Expand Up @@ -118,9 +118,17 @@ def detect():


if __name__ == "__main__":
if detect():
print('Something is seriously wrong.'.upper(), file=sys.stderr)
sys.exit(1)
args = sys.argv[1:]
if len(args) == 0:
detect()
print('Test completed. Use "DEBUG=1" to get test output.'.upper())
else:
print('Everything\'s gonna be alright.'.upper())
sys.exit(0)
keys = [i.split('=')[0].upper() for i in args]
values = [i.split('=')[1] for i in args]
if 'DEBUG' in keys and values[keys.index('DEBUG')] == '1':
if detect():
print('Something is seriously wrong.'.upper(), file=sys.stderr)
sys.exit(1)
else:
print('Everything\'s gonna be alright.'.upper())
sys.exit(0)

0 comments on commit 46c92dd

Please sign in to comment.