Skip to content

ensure xml report path are relative to CI prefix #60380

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

walterddr
Copy link
Contributor

@walterddr walterddr commented Jun 21, 2021

Root cause of this is for some reason: inspect.getfile returns absolute path instead of relative path to os.getcwd in newer python version. we sanitize this by removing the CI_PREFIX if applies

See:
https://app.circleci.com/pipelines/github/pytorch/pytorch/339568/workflows/43cac71c-759e-471f-83c2-d59c152dcd8a/jobs/14278585 vs. https://app.circleci.com/pipelines/github/pytorch/pytorch/339568/workflows/43cac71c-759e-471f-83c2-d59c152dcd8a/jobs/14278285

Test Plan:
CI

Plot twist:

windows tests are actually launched via

pushd test
python run_test.py

while linux/macos tests are

python test/run_test.py

This might cause problem when using os.getcwd() we will see from PR CI results.

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Jun 21, 2021

💊 CI failures summary and remediations

As of commit 83595fb (more details on the Dr. CI page and at hud.pytorch.org/pr/60380):


  • 3/4 failures possibly* introduced in this PR
    • 1/3 non-scanned failure(s)
  • 1/4 broken upstream at merge base 65f33ec on Jun 21 from 1:29pm to 2:54pm

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

See CircleCI build pytorch_macos_10_13_py3_test (1/1)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

Jun 21 21:55:09 ERROR [0.004s]: test_poisson_sample (__main__.TestDistributions)
Jun 21 21:55:09   File "distributions/test_distributions.py", line 805, in _check_sampler_discrete
Jun 21 21:55:09     chisq, p = scipy.stats.chisquare(counts[msk], pmf[msk] * num_samples)
Jun 21 21:55:09   File "/Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/scipy/stats/stats.py", line 6853, in chisquare
Jun 21 21:55:09     lambda_="pearson")
Jun 21 21:55:09   File "/Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/scipy/stats/stats.py", line 6694, in power_divergence
Jun 21 21:55:09     raise ValueError(msg)
Jun 21 21:55:09 ValueError: For each axis slice, the sum of the observed frequencies must agree with the sum of the expected frequencies to a relative tolerance of 1e-08, but the percent differences are:
Jun 21 21:55:09 0.008265582255680495
Jun 21 21:55:09 
Jun 21 21:55:09 ======================================================================
Jun 21 21:55:09 ERROR [0.004s]: test_poisson_sample (__main__.TestDistributions)
Jun 21 21:55:09 ----------------------------------------------------------------------
Jun 21 21:55:09 Traceback (most recent call last):
Jun 21 21:55:09   File "distributions/test_distributions.py", line 1333, in test_poisson_sample
Jun 21 21:55:09     failure_rate=1e-3)
Jun 21 21:55:09   File "distributions/test_distributions.py", line 805, in _check_sampler_discrete
Jun 21 21:55:09     chisq, p = scipy.stats.chisquare(counts[msk], pmf[msk] * num_samples)
Jun 21 21:55:09   File "/Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/scipy/stats/stats.py", line 6853, in chisquare
Jun 21 21:55:09     lambda_="pearson")
Jun 21 21:55:09   File "/Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/scipy/stats/stats.py", line 6694, in power_divergence
Jun 21 21:55:09     raise ValueError(msg)

1 failure not recognized by patterns:

Job Step Action
CircleCI pytorch_linux_bionic_py3_8_gcc9_coverage_test2 Run tests 🔁 rerun

🚧 1 fixed upstream failure:

These were probably caused by upstream breakages that were already fixed.

Please rebase on the viable/strict branch (expand for instructions)

If your commit is older than viable/strict, run these commands:

git fetch https://github.com/pytorch/pytorch viable/strict
git rebase FETCH_HEAD

This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

@@ -255,7 +255,8 @@ def chunk_list(lst, nchunks):

# sanitize filename e.g., distributed/pipeline/sync/skip/test_api.py -> distributed.pipeline.sync.skip.test_api
def sanitize_test_filename(filename):
strip_py = re.sub(r'.py$', '', filename)
strip_py = re.sub(r'.*/pytorch/test/', '', filename)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
strip_py = re.sub(r'.*/pytorch/test/', '', filename)
# Replace initial dot with underscore
strip_py = re.sub(r'^\.', '_', filename)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this wouldn't work because the sanitized test file path needs to be one of those listed in

TESTS = [

@walterddr walterddr force-pushed the be_ensure_path_is_relative branch from 1f62b2e to 1e44b44 Compare June 21, 2021 21:13
@walterddr walterddr requested a review from janeyx99 June 21, 2021 21:14
@facebook-github-bot
Copy link
Contributor

@walterddr has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@walterddr walterddr changed the title ensure xml report path are relative to */pytorch/test ensure xml report path are relative to CI prefix Jun 21, 2021
@facebook-github-bot
Copy link
Contributor

@walterddr merged this pull request in 5921b54.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants