Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[internal] flaky CI tests regarding registry #4842

Closed
aryx opened this issue Mar 19, 2022 · 4 comments
Closed

[internal] flaky CI tests regarding registry #4842

aryx opened this issue Mar 19, 2022 · 4 comments
Assignees
Labels
devops Building issues, distribution issues, CI issues priority:medium

Comments

@aryx
Copy link
Collaborator

aryx commented Mar 19, 2022

A lot of recent PRs got one failure in CI because of the registry. See below.
I propose ot remove those tests or find a way to mark them so that if they fail they do not
compromise the CI results.

For instance:

current directory: /tmp/pytest-of-runner/pytest-0/popen-gw0/test_auto_config0
semgrep command: /home/runner/work/semgrep/semgrep/semgrep/.tox/py/bin/python -m semgrep --disable-version-check --config auto --json targets/basic
--- stdout from semgrep process ---
{"errors": [{"code": 2, "level": "error", "message": "Failed to download config from https://semgrep.dev/c/auto: ('Connection aborted.', RemoteDisconnected('Remote end closed\nconnection without response'))", "type": "SemgrepError"}, {"code": 7, "level": "error", "message": "invalid configuration file found (1 configs were invalid)", "type": "SemgrepError"}], "paths": {"_comment": "<add --verbose for a list of skipped paths>", "scanned": []}, "results": []}

--- end semgrep stdout ---
--- stderr from semgrep process ---
Auto config uses Semgrep rules to scan your codebase and the Semgrep Registry to generate recommended rules based on
your languages and frameworks.
Fetching rules from https://semgrep.dev/registry.

--- end semgrep stderr ---
=============================== warnings summary ===============================
semgrep/ignores.py:133
semgrep/ignores.py:133
  /home/runner/work/semgrep/semgrep/semgrep/semgrep/ignores.py:133: DeprecationWarning: invalid escape sequence \:
    """

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED tests/e2e/test_check.py::test_auto_config - subprocess.CalledProcessEr...
============ 1 failed, 984 passed, 2 warnings in 476.37s (0:07:56) =============
ERROR: InvocationError for command /opt/hostedtoolcache/Python/3.7.12/x64/bin/pipenv run pytest --ignore=tests/qa/test_public_repos.py -n auto -vv --tb=short (exited with code 1)

@aryx aryx added the devops Building issues, distribution issues, CI issues label Mar 19, 2022
@aryx
Copy link
Collaborator Author

aryx commented Mar 19, 2022

@mmcqd got this one recently, @mjambon too. It's slowing us down. Each time you need to wait another 20min, you lose focus.

@r2c-demo
Copy link
Collaborator

@aryx
Copy link
Collaborator Author

aryx commented Mar 19, 2022

@brendongo
Copy link
Member

We havent seen this since https://github.com/returntocorp/semgrep/pull/4893/files where we extended the timeout for downloading a config

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
devops Building issues, distribution issues, CI issues priority:medium
Development

No branches or pull requests

3 participants