Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run precommit in its own job, cache the data #3094

Merged
merged 8 commits into from
Dec 16, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 40 additions & 6 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -64,12 +64,6 @@ jobs:
pip freeze > test-reports/pip_versions.txt
pytest --cov=ml-agents --cov=ml-agents-envs --cov=gym-unity --cov-report html --junitxml=test-reports/junit.xml -p no:warnings

- run:
name: Check Code Style using pre-commit
command: |
. venv/bin/activate
pre-commit run --show-diff-on-failure --all-files

- run:
name: Verify there are no hidden/missing metafiles.
# Renaming files or deleting files can leave metafiles behind that makes Unity very unhappy.
Expand All @@ -88,6 +82,45 @@ jobs:
path: htmlcov
destination: htmlcov

pre-commit:
docker:
- image: circleci/python:3.7.3
working_directory: ~/repo/

steps:
- checkout
- run:
name: Combine precommit config and python versions for caching
command: |
cat .pre-commit-config.yaml > pre-commit-deps.txt
python -VV >> pre-commit-deps.txt

- restore_cache:
keys:
- v1-precommit-deps-{{ checksum "pre-commit-deps.txt" }}

- run:
name: Install Dependencies
command: |
python3 -m venv venv
. venv/bin/activate
pip install --upgrade pip
pip install --upgrade setuptools
pip install pre-commit
# Install the hooks now so that they'll be cached
pre-commit install-hooks

- save_cache:
paths:
- ~/.cache/pre-commit
- ./venv
key: v1-precommit-deps-{{ checksum "pre-commit-deps.txt" }}

- run:
name: Check Code Style using pre-commit
command: |
. venv/bin/activate
pre-commit run --show-diff-on-failure --all-files

markdown_link_check:
parameters:
Expand Down Expand Up @@ -241,6 +274,7 @@ workflows:
pip_constraints: test_constraints_max_tf2_version.txt
- markdown_link_check
- protobuf_generation_check
- pre-commit
- deploy:
name: deploy ml-agents-envs
directory: ml-agents-envs
Expand Down
5 changes: 3 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -55,14 +55,15 @@ repos:
types: [markdown]
- id: check-merge-conflict
args: [--assume-in-merge]
- id: check-yaml
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can back this out, but I it saved me at least one bad config.yml push.

# Won't handle the templating in yamato
exclude: \.yamato/*
surfnerd marked this conversation as resolved.
Show resolved Hide resolved

- repo: https://github.com/pre-commit/pygrep-hooks
rev: v1.4.2
hooks:
- id: python-check-mock-methods



- repo: https://github.com/pre-commit/mirrors-pylint
rev: v2.4.4
hooks:
Expand Down
4 changes: 2 additions & 2 deletions config/offline_bc_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ default:
num_layers: 2
sequence_length: 32
memory_size: 256
demo_path: ./UnitySDK/Assets/Demonstrations/<Your_Demo_File>.demo
# Change the path here to point to your demo files
demo_path: ./UnitySDK/Assets/Demonstrations/Your_Demo_File.demo

FoodCollector:
trainer: offline_bc
Expand All @@ -34,7 +35,6 @@ Hallway:
batches_per_epoch: 5
num_layers: 2
hidden_units: 128
sequence_length: 16
use_recurrent: true
memory_size: 256
sequence_length: 32
Expand Down
2 changes: 0 additions & 2 deletions config/sac_trainer_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,6 @@ Reacher:
summary_freq: 3000

Hallway:
use_recurrent: true
sequence_length: 32
num_layers: 2
hidden_units: 128
Expand All @@ -218,7 +217,6 @@ Hallway:
use_recurrent: true

VisualHallway:
use_recurrent: true
sequence_length: 32
num_layers: 1
hidden_units: 128
Expand Down
1 change: 0 additions & 1 deletion test_requirements.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
# Test-only dependencies should go here, not in setup.py
pytest>=3.2.2,<4.0.0
pre-commit
pytest-cov==2.6.1