Skip to content

Commit

Permalink
[iOS] Fix dependencies for Clusterfuzz test scripts
Browse files Browse the repository at this point in the history
The Python scripts used for fuzzing iOS builds on Clusterfuzz depend
on various Python libraries (directly and transitively), and some of
these are not installed by default on the Clusterfuzz bots, causing
iOS fuzzing jobs to fail.

The normal way to encode python dependencies in Chromium is by using
.vpython3 files and then invoking the Python script using vpython3,
which then takes care of setting up a virtual environment which
contains the dependencies.

This CL updates fuzzing scripts to use vpython3 and adds a file
listing the required dependencies.

This CL also adds a new run_clusterfuzz_test.sh script that includes
the device type and OS version used for fuzzing, so that changes to
these values are version-controlled rather than changed ad-hoc in
the Clusterfuzz configuration UI.

Bug: 1380928
Change-Id: I6989e97b219e0b7476bec86ba75aa4c588097bd8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4003809
Reviewed-by: Mike Dougherty <michaeldo@chromium.org>
Commit-Queue: Ali Juma <ajuma@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1101036}
  • Loading branch information
alijuma authored and Chromium LUCI CQ committed Feb 3, 2023
1 parent 84d4822 commit f38b8f1
Show file tree
Hide file tree
Showing 5 changed files with 87 additions and 16 deletions.
1 change: 1 addition & 0 deletions ios/chrome/test/wpt/BUILD.gn
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ copy("copy_clusterfuzz_scripts") {
sources = [
"//ios/build/bots/scripts",
"//ios/chrome/test/wpt/tools",
"//ios/testing/plugin",

# Variations test utilities used by variations_runner script.
"//testing/scripts/variations_seed_access_helper.py",
Expand Down
58 changes: 58 additions & 0 deletions ios/chrome/test/wpt/tools/.vpython3
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# This is a vpython "spec" file that documents dependencies needed
# to run the python scripts in this directory.
#
# Read more about `vpython` and how to modify this file here:
# https://chromium.googlesource.com/infra/infra/+/master/doc/users/vpython.md

# This specifies the platforms to be checked when verifying the validity of the
# dependencies listed in this file. This verification happens during presubmit.
verify_pep425_tag: [
{python: "cp38", abi: "cp38", platform: "macosx_11_0_intel"},
{python: "cp38", abi: "cp38", platform: "macosx_11_0_arm64"}
]

wheel: <
name: "infra/python/wheels/psutil/${vpython_platform}"
version: "version:5.8.0.chromium.3"
>

# requests and its transitive dependencies
wheel: <
name: "infra/python/wheels/requests-py2_py3"
version: "version:2.26.0"
>

wheel: <
name: "infra/python/wheels/charset_normalizer-py3"
version: "version:2.0.4"
>

wheel: <
name: "infra/python/wheels/idna-py2_py3"
version: "version:2.10"
>

wheel: <
name: "infra/python/wheels/urllib3-py2_py3"
version: "version:1.26.6"
>

wheel: <
name: "infra/python/wheels/certifi-py2_py3"
version: "version:2021.5.30"
>

wheel: <
name: "infra/python/wheels/grpcio/${vpython_platform}"
version: "version:1.44.0"
>

wheel <
name: "infra/python/wheels/six-py2_py3"
version: "version:1.15.0"
>

wheel: <
name: "infra/python/wheels/protobuf-py3"
version: "version:3.20.0"
>
23 changes: 23 additions & 0 deletions ios/chrome/test/wpt/tools/run_clusterfuzz_test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
#!/bin/bash
# Copyright 2023 The Chromium Authors
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.

# This script is used by Clusterfuzz bots. For local fuzzing, including
# trying to reproduce bugs found in Clusterfuzz, use run_fuzz_test.py
# directly instead of using this script.

function main() {
local test_path=$1
local device_type="iPhone 14 Pro"
local ios_version="16.1"
local script_dir=$(dirname "$0")
local vpython_path="/opt/vpython"
local python_path="/opt/python3/bin"
PATH="${vpython_path}:${python_path}:${PATH}"

${script_dir}/run_fuzz_test.py --os="${ios_version}" \
--device="${device_type}" ${test_path}
}

main "$@"
2 changes: 1 addition & 1 deletion ios/chrome/test/wpt/tools/run_cwt_chromedriver.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/usr/bin/env python3
#!/usr/bin/env vpython3
# Copyright 2019 The Chromium Authors
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.
Expand Down
19 changes: 4 additions & 15 deletions ios/chrome/test/wpt/tools/run_fuzz_test.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/usr/bin/env python3
#!/usr/bin/env vpython3
# Copyright 2021 The Chromium Authors
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.
Expand Down Expand Up @@ -46,24 +46,15 @@ def StartServer(*, port, build_dir, device, os_version):
else:
time.sleep(1)

def IsCurrentVersion(*, version, revision, build_dir):
def IsCurrentVersion(*, version, build_dir):
plist_path = os.path.join(build_dir, 'ios_cwt_chromedriver_tests.app',
'Info.plist')
version_command = 'defaults read ' + plist_path + ' CFBundleVersion'
completed_process = subprocess.run(version_command, shell=True,
capture_output=True)
current_version = completed_process.stdout.decode('utf-8').strip()

if version != current_version:
return False

revision_command = 'defaults read ' + plist_path + ' SCMRevision'
completed_process = subprocess.run(revision_command, shell=True,
capture_output=True)
full_revision = completed_process.stdout.decode('utf-8').strip()
current_revision = re.search('.*{#(.+?)}', full_revision).group(1)

return current_revision == revision
return version == current_version

def KillServer():
# Gather all running run_cwt_chromedriver.py and xcodebuild instances. There
Expand Down Expand Up @@ -98,9 +89,7 @@ def EnsureServerStarted(*, port, build_dir, device, os_version):
response = requests.get(server_url + '/session/chrome_versionInfo')
assert response.status_code == 200
chrome_version = response.json()['value']['browserVersion']
chrome_revision = response.json()['value']['chrome_revisionNumber']
if IsCurrentVersion(version=chrome_version, revision=chrome_revision,
build_dir=build_dir):
if IsCurrentVersion(version=chrome_version, build_dir=build_dir):
return True
else:
KillServer()
Expand Down

0 comments on commit f38b8f1

Please sign in to comment.