CoverageReportIntegrationTest::test_coverage_report is failing

[oliverchang]

From e.g. https://github.com/google/oss-fuzz/runs/5309713893?check_suite_focus=true

=================================== FAILURES ===================================
______________ CoverageReportIntegrationTest.test_coverage_report ______________
[gw0] linux -- Python 3.8.12 /opt/hostedtoolcache/Python/3.8.12/x64/bin/python

self = <run_fuzzers_test.CoverageReportIntegrationTest testMethod=test_coverage_report>
_ = <MagicMock name='_upload_artifact_with_upload_js' id='140482468299728'>

    @mock.patch('filestore.github_actions._upload_artifact_with_upload_js')
    def test_coverage_report(self, _):
      """Tests generation of coverage reports end-to-end, from building to
      generation."""
    
      with test_helpers.docker_temp_dir() as temp_dir:
        shared = os.path.join(temp_dir, 'shared')
        os.mkdir(shared)
        copy_command = ('cp -r /opt/code_coverage /shared && '
                        'cp $(which llvm-profdata) /shared && '
                        'cp $(which llvm-cov) /shared')
        assert helper.docker_run([
            '-v', f'{shared}:/shared', 'gcr.io/oss-fuzz-base/base-runner', 'bash',
            '-c', copy_command
        ])
    
        os.environ['CODE_COVERAGE_SRC'] = os.path.join(shared, 'code_coverage')
        os.environ['PATH'] += os.pathsep + shared
        # Do coverage build.
        build_config = test_helpers.create_build_config(
            oss_fuzz_project_name=EXAMPLE_PROJECT,
            project_repo_name='oss-fuzz',
            workspace=temp_dir,
            git_sha='0b95fe1039ed7c38fea1f97078316bfc1030c[523](https://github.com/google/oss-fuzz/runs/5309713893?check_suite_focus=true#step:9:523)',
            base_commit='da07464[524](https://github.com/google/oss-fuzz/runs/5309713893?check_suite_focus=true#step:9:524)33dc18bae699e355a9821285d863c8',
            sanitizer=self.SANITIZER,
            cfl_platform='github',
            # Needed for test not to fail because of permissions issues.
            bad_build_check=False)
        self.assertTrue(build_fuzzers.build_fuzzers(build_config))
    
        # TODO(metzman): Get rid of this here and make 'compile' do this.
        chmod_command = ('chmod -R +r /out && '
                         'find /out -type d -exec chmod +x {} +')
    
        assert helper.docker_run([
            '-v', f'{os.path.join(temp_dir, "build-out")}:/out',
            'gcr.io/oss-fuzz-base/base-builder', 'bash', '-c', chmod_command
        ])
    
        # Generate report.
        run_config = test_helpers.create_run_config(fuzz_seconds=FUZZ_SECONDS,
                                                    workspace=temp_dir,
                                                    sanitizer=self.SANITIZER,
                                                    mode='coverage',
                                                    cfl_platform='github')
        result = run_fuzzers.run_fuzzers(run_config)
        self.assertEqual(result, run_fuzzers.RunFuzzersResult.NO_BUG_FOUND)
        expected_summary_path = os.path.join(
            TEST_DATA_PATH, 'example_coverage_report_summary.json')
        with open(expected_summary_path) as file_handle:
          expected_summary = json.loads(file_handle.read())
          actual_summary_path = os.path.join(temp_dir, 'cifuzz-coverage',
                                             'report', 'linux', 'summary.json')
>       with open(actual_summary_path) as file_handle:
E       FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpfqnfmle8/cifuzz-coverage/report/linux/summary.json'

Related error:

------------------------------ Captured log call -------------------------------
ERROR    root:github_api.py:75 Request to https://api.github.com/repos/None/None/actions/artifacts?per_page=100&page=1 failed. Code: 401. Response: {'message': 'Bad credentials', 'documentation_url': 'https://docs.github.com/rest'}
ERROR    root:clusterfuzz_deployment.py:143 Failed to download corpus for target: do_stuff_fuzzer. Error: Github API request failed.
=========================== short test summary info ============================
FAILED infra/cifuzz/run_fuzzers_test.py::CoverageReportIntegrationTest::test_coverage_report```

[oliverchang]

Looks like the cause of the second error is we were passing our GITHUB_TOKEN to the GitHub artifacts API. This was the wrong token.

For public repos, read operations didn't need this. For private repos, this probably was broken.

The behaviour of the artifacts API must have changed, in that it now tries to validate the token passed even if it's not required, so now we see this failure.

[oliverchang]

Setting the correct ACTIONS_RUNTIME_TOKEN (which is undocumented), and setting the project owner/repo to "google/oss-fuzz" instead gets us past the initial error, but we still see:

        expected_summary_path = os.path.join(
            TEST_DATA_PATH, 'example_coverage_report_summary.json')
        with open(expected_summary_path) as file_handle:
          expected_summary = json.loads(file_handle.read())
          actual_summary_path = os.path.join(temp_dir, 'cifuzz-coverage',
                                             'report', 'linux', 'summary.json')
>       with open(actual_summary_path) as file_handle:
E       FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp2mypsq2s/cifuzz-coverage/report/linux/summary.json'

[oliverchang]

@jonathanmetzman I'm unable to reproduce the failure in #7324 (comment). Any ideas why? Is this some docker-in-docker config breakage?

[jonathanmetzman]

I suspect the reason no one can reproduce locally is because the test behaves different when run on github actions (because of some config that doesn't exist locally but does on GH).

[oliverchang]

This is not fixed. 9553ab1 was for a related but different bug.