Skip to content

Commit

Permalink
Support uploads and downloads
Browse files Browse the repository at this point in the history
  • Loading branch information
ZainRizvi committed May 3, 2023
1 parent 954a173 commit 1ed17ec
Show file tree
Hide file tree
Showing 3 changed files with 54 additions and 33 deletions.
3 changes: 2 additions & 1 deletion .github/actions/pytest-cache-upload/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ runs:
JOB: ${{ github.job }}
SHARD: ${{ inputs.shard }}
run: |
python3 .github/scripts/pytest_upload_cache.py \
python3 .github/scripts/pytest_cache.py \
--upload \
--cache_dir $CACHE_DIR \
--pr_identifier $PR_IDENTIFIER \
--workflow $WORKFLOW \
Expand Down
52 changes: 52 additions & 0 deletions .github/scripts/pytest_cache.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
import argparse
import sys
from pytest_caching_utils import *

def main():

Check failure on line 5 in .github/scripts/pytest_cache.py

View workflow job for this annotation

GitHub Actions / lintrunner / linux-job

MYPYSTRICT [no-untyped-def]

Function is missing a return type annotation
parser = argparse.ArgumentParser(description="Upload this job's the pytest cache to S3")

mode = parser.add_mutually_exclusive_group(required=True)
mode.add_argument('--upload', action='store_true', help='Upload the pytest cache to S3')
mode.add_argument('--download', action='store_true', help='Download the pytest cache from S3, merging it with any local cache')

parser.add_argument('--cache_dir', required=True, help='Path to the folder pytest uses for its cache')
parser.add_argument('--pr_identifier', required=True, help='A unique PR identifier')
parser.add_argument('--workflow', required=True, help='The workflow name')
parser.add_argument('--job', required=True, help='The job name')
parser.add_argument('--shard', required='--upload' in sys.argv, help='The shard id') # Only required for upload

parser.add_argument('--temp_dir', required=False, help='Directory to store temp files')
parser.add_argument('--bucket', required=False, help='The S3 bucket to upload the cache to')

args = parser.parse_args()

if args.upload:
print(f"Uploading cache with args {args}")

# TODO: First check if it's even worth uploading a new cache:
# Does the cache even mark any failed tests?

upload_pytest_cache(

Check failure on line 29 in .github/scripts/pytest_cache.py

View workflow job for this annotation

GitHub Actions / lintrunner / linux-job

MYPYSTRICT [no-untyped-call]

Call to untyped function "upload_pytest_cache" in typed context
pr_identifier=args.pr_identifier,
workflow=args.workflow,
job=args.job,
shard=args.shard,
pytest_cache_dir=args.pytest_cache_dir,
bucket=args.bucket,
temp_dir=args.temp_dir,
)

if args.download:
print(f"Downloading cache with args {args}")
download_pytest_cache(

Check failure on line 41 in .github/scripts/pytest_cache.py

View workflow job for this annotation

GitHub Actions / lintrunner / linux-job

MYPYSTRICT [no-untyped-call]

Call to untyped function "download_pytest_cache" in typed context

Check failure on line 41 in .github/scripts/pytest_cache.py

View workflow job for this annotation

GitHub Actions / lintrunner / linux-job

MYPYSTRICT [call-arg]

Unexpected keyword argument "job" for "download_pytest_cache"

Check failure on line 41 in .github/scripts/pytest_cache.py

View workflow job for this annotation

GitHub Actions / lintrunner / linux-job

MYPYSTRICT [call-arg]

Unexpected keyword argument "pytest_cache_dir" for "download_pytest_cache"; did you mean "pytest_cache_dir_new"?
pr_identifier=args.pr_identifier,
workflow=args.workflow,
job=args.job,
pytest_cache_dir=args.pytest_cache_dir,
bucket=args.bucket,
temp_dir=args.temp_dir,
)


if __name__ == '__main__':
main()

Check failure on line 52 in .github/scripts/pytest_cache.py

View workflow job for this annotation

GitHub Actions / lintrunner / linux-job

MYPYSTRICT [no-untyped-call]

Call to untyped function "main" in typed context
32 changes: 0 additions & 32 deletions .github/scripts/pytest_upload_cache.py

This file was deleted.

0 comments on commit 1ed17ec

Please sign in to comment.