STP downloads are failing #34752

foolip · 2022-07-08T07:18:36Z

STP downloads have begun failing. https://github.com/web-platform-tests/wpt/runs/7245617686 is a recent example with this log:

./wpt install --channel preview --download-only -d . --rename STP safari browser
INFO:install:Now installing safari browser...
INFO:install:Finding STP download URLs
WARNING:install:Did not find any STP URLs
Traceback (most recent call last):
  File "/Users/runner/work/1/s/./wpt", line 10, in <module>
    wpt.main()
  File "/Users/runner/work/1/s/tools/wpt/wpt.py", line 220, in main
    rv = script(*args, **kwargs)
  File "/Users/runner/work/1/s/tools/wpt/install.py", line 89, in run
    install(browser, kwargs["component"], destination, channel, logger=logger,
  File "/Users/runner/work/1/s/tools/wpt/install.py", line 109, in install
    path = getattr(browser_cls(logger), method)(dest=destination, channel=channel, **kwargs)
  File "/Users/runner/work/1/s/tools/wpt/browser.py", line 1606, in download
    image_path = self._download_image(stp_downloads, tmpdir, system_version)
  File "/Users/runner/work/1/s/tools/wpt/browser.py", line 1524, in _download_image
    raise ValueError(f"no download for {system_version}")
ValueError: no download for 12.4

The error comes from here:

wpt/tools/wpt/browser.py

Lines 1516 to 1521 in 36fc6e2

    
           chosen_url = None 
        
           for version_spec, url in downloads: 
        
               if system_version in version_spec: 
        
                   self.logger.debug(f"Will download Safari for {version_spec}") 
        
                   chosen_url = url 
        
                   break

Right now https://developer.apple.com/safari/resources/ says "Requires macOS 12.3 or later" so the problem is that we're matching the version exactly and don't consider this STP for "12.3 or later" a match for macOS 12.4.

The text was updated successfully, but these errors were encountered:

gsnedders · 2022-07-08T17:04:10Z

Right now https://developer.apple.com/safari/resources/ says "Requires macOS 12.3 or later" so the problem is that we're matching the version exactly and don't consider this STP for "12.3 or later" a match for macOS 12.4.

That's not a recent change. I suspect the actual change here that's broken things is the "Requires macOS 13 beta 3 or later." (notably with the number of the beta), though clearly something else has gone wrong for this to break the macOS 12 case too.

gsnedders · 2022-07-08T17:18:53Z

For future testing usage:

                        <div class="column large-6 medium-12 small-12">
                            <div class="callout">
                                <figure class="app-icon large-icon safari-preview-icon" aria-hidden="true" data-hires-status="pending"></figure>
                                <h4>Safari Technology Preview</h4>
                                <p class="margin-bottom-small">Get a sneak peek at upcoming web technologies in macOS and iOS with <a href="/safari/technology-preview/" class="nowrap">Safari Technology Preview</a> and experiment with these technologies in your websites and extensions.</p>
                                <ul class="links small">
                                	<li class="dmg" data-hires-status="pending"><a class="inline" href="https://secure-appldnld.apple.com/STP/012-38225-20220706-237860CD-5766-4F53-AAC7-1CE26023A959/SafariTechnologyPreview.dmg">Safari Technology Preview<br>for macOS&nbsp;Ventura</a><br><span class="smaller lighter nowrap nowrap-small">Requires macOS&nbsp;13 beta&nbsp;3 or later.</span></li>
                                    <li class="dmg margin-top-small" data-hires-status="pending"><a class="inline" href="https://secure-appldnld.apple.com/STP/012-32918-20220629-B3452905-0138-4CA9-A4E6-334B63585653/SafariTechnologyPreview.dmg">Safari Technology Preview<br>for macOS&nbsp;Monterey</a><br><span class="smaller lighter">Requires macOS&nbsp;12.3 or later.</span></li>
                                    <li class="document margin-top-small" data-hires-status="pending"><a href="/safari/technology-preview/release-notes/">Release Notes</a></li>
                                </ul>
                                <div class="row gutter text-left">
                                    <div class="column">
                                        <p class="sosumi no-margin-bottom margin-top-small">Release</p>
                                        <p class="smaller lighter no-margin">148</p>
                                    </div>
                                    <div class="column">
                                        <p class="sosumi no-margin-bottom margin-top-small">Posted</p>
                                        <p class="smaller lighter no-margin">June 29, 2022</p>
                                    </div>
                                </div>
                            </div>
                        </div>

I'm out all of next week, and don't really have enough internet connection right now, but this should just be changing the requirement regex in tools.wpt.browser.Safari._find_downloads.

jcscottiii · 2022-07-11T18:16:32Z

I investigated the issue. The regex capturing Safari Technology Preview was failing to match both Mac OS 12 and Mac OS 13 because the html was as follows: Safari Technology Preview<br>for macOS Ventura. When sending that to text_content, the string became Safari Technology Previewfor macOS Ventura
The regex was previously
"^\s*Safari\s+Technology\s+Preview\s+(?:[0-9]+\s+)?for\s+macOS"

I changed the \s after Preview from \s+ to \s*

Now it is:
"^\sSafari\s+Technology\s+Preview\s(?:[0-9]+\s+)?for\s+macOS"

TalbotG · 2022-07-12T12:18:48Z

Are you sure that the correction does indeed fix the problem?

I just rerun the Azure Pipelines checks on a commit in PR34705 and I got the same error, same description.

Details:
View raw log

Starting: Install Safari Technology Preview

Task : Command line
Description : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
Version : 2.201.1
Author : Microsoft Corporation
Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line

Generating script.
========================== Starting Command Output ===========================
/bin/bash --noprofile --norc /Users/runner/work/_temp/0e405ccf-534b-4f44-b455-45873934aaff.sh

./wpt install --channel preview --download-only -d . --rename STP safari browser
INFO:install:Now installing safari browser...
INFO:install:Finding STP download URLs
WARNING:install:Did not find any STP URLs
Traceback (most recent call last):
File "/Users/runner/work/1/s/./wpt", line 10, in
wpt.main()
File "/Users/runner/work/1/s/tools/wpt/wpt.py", line 220, in main
rv = script(*args, **kwargs)
File "/Users/runner/work/1/s/tools/wpt/install.py", line 89, in run
install(browser, kwargs["component"], destination, channel, logger=logger,
raise ValueError(f"no download for {system_version}")
ValueError: no download for 12.4
##[error]Bash exited with code '1'.
Finishing: Install Safari Technology Preview

2022-07-12T12:08:57.6652490Z ##[section]Starting: Install Safari Technology Preview
2022-07-12T12:08:57.6661810Z ==============================================================================
2022-07-12T12:08:57.6662400Z Task : Command line
2022-07-12T12:08:57.6662840Z Description : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
2022-07-12T12:08:57.6663340Z Version : 2.201.1
2022-07-12T12:08:57.6663650Z Author : Microsoft Corporation
2022-07-12T12:08:57.6664340Z Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
2022-07-12T12:08:57.6664870Z ==============================================================================
2022-07-12T12:08:57.9544080Z Generating script.
2022-07-12T12:08:57.9852560Z ========================== Starting Command Output ===========================
2022-07-12T12:08:58.0610710Z [command]/bin/bash --noprofile --norc /Users/runner/work/_temp/0e405ccf-534b-4f44-b455-45873934aaff.sh
2022-07-12T12:08:58.0612670Z + ./wpt install --channel preview --download-only -d . --rename STP safari browser
2022-07-12T12:09:05.3926870Z INFO:install:Now installing safari browser...
2022-07-12T12:09:05.3928890Z INFO:install:Finding STP download URLs
2022-07-12T12:09:05.9320440Z WARNING:install:Did not find any STP URLs
2022-07-12T12:09:05.9556010Z Traceback (most recent call last):
2022-07-12T12:09:05.9559840Z File "/Users/runner/work/1/s/./wpt", line 10, in
2022-07-12T12:09:05.9561210Z wpt.main()
2022-07-12T12:09:05.9562460Z File "/Users/runner/work/1/s/tools/wpt/wpt.py", line 220, in main
2022-07-12T12:09:05.9564570Z rv = script(*args, **kwargs)
2022-07-12T12:09:05.9565810Z File "/Users/runner/work/1/s/tools/wpt/install.py", line 89, in run
2022-07-12T12:09:05.9568100Z install(browser, kwargs["component"], destination, channel, logger=logger,
2022-07-12T12:09:05.9569520Z File "/Users/runner/work/1/s/tools/wpt/install.py", line 109, in install
2022-07-12T12:09:05.9570900Z path = getattr(browser_cls(logger), method)(dest=destination, channel=channel, **kwargs)
2022-07-12T12:09:05.9572360Z File "/Users/runner/work/1/s/tools/wpt/browser.py", line 1606, in download
2022-07-12T12:09:05.9573800Z image_path = self._download_image(stp_downloads, tmpdir, system_version)
2022-07-12T12:09:05.9575250Z File "/Users/runner/work/1/s/tools/wpt/browser.py", line 1524, in _download_image
2022-07-12T12:09:05.9576630Z raise ValueError(f"no download for {system_version}")
2022-07-12T12:09:05.9577900Z ValueError: no download for 12.4
2022-07-12T12:09:06.0064750Z ##[error]Bash exited with code '1'.
2022-07-12T12:09:06.0082590Z ##[section]Finishing: Install Safari Technology Preview

jcscottiii · 2022-07-12T12:36:18Z

@TalbotG Thanks for that stack trace.
It looks like the build is not using the latest commit with my changes in browser.py

The ValueError Exception occurs at line 1524.

At line 1524 of my commit, this is the code:

wpt/tools/wpt/browser.py

Lines 1523 to 1526 in f96f182

    
           for version_spec, url in downloads: 
        
               if system_version in version_spec: 
        
                   self.logger.debug(f"Will download Safari for {version_spec}") 
        
                   chosen_url = url

At line 1524 of the previous commit, you will find the ValueError exception

wpt/tools/wpt/browser.py

Lines 1523 to 1525 in 3fc0804

    
           if chosen_url is None: 
        
               raise ValueError(f"no download for {system_version}")

jcscottiii · 2022-07-12T13:03:58Z

@TalbotG You will need to rebase your changes. I created a PR with your changes rebase on top of the latest code and it works: #34806

foolip · 2022-07-12T14:25:42Z

Runs of STP are also back in https://wpt.fyi/runs, so let's call this fixed! Thanks @jcscottiii!

foolip added infra priority:urgent labels Jul 8, 2022

foolip assigned gsnedders Jul 8, 2022

foolip mentioned this issue Jul 8, 2022

Implement our own STP download logic #34553

Merged

stephenmcgruer mentioned this issue Jul 8, 2022

[SPC] Update tests for input parameter validation #34739

Merged

whimboo mentioned this issue Jul 11, 2022

Azure Pipelines build error: Bash exited with code '1' #34760

Closed

past mentioned this issue Jul 11, 2022

Fix flaky test memory leak #34723

Merged

jcscottiii mentioned this issue Jul 11, 2022

Fix the stp downloads: Mac OS Monterey and Mac OS Ventura #34771

Merged

foolip closed this as completed in #34771 Jul 11, 2022

TalbotG reopened this Jul 12, 2022

jcscottiii mentioned this issue Jul 12, 2022

[DNM] Testing TalbotG's changes for build #34806

Closed

foolip closed this as completed Jul 12, 2022

foolip mentioned this issue Jul 12, 2022

RFC 114: Change results aggregation on wpt.fyi and visualization on Interop-20** views web-platform-tests/rfcs#114

Merged

jcscottiii mentioned this issue Jul 18, 2022

Safari Technology Preview downloads are failing after page update #34875

Closed

TalbotG mentioned this issue Jul 21, 2022

Azure pipeline error, Install Safari Technology Preview "no download for 10.16" #34926

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

STP downloads are failing #34752

STP downloads are failing #34752

foolip commented Jul 8, 2022

gsnedders commented Jul 8, 2022

gsnedders commented Jul 8, 2022

jcscottiii commented Jul 11, 2022

TalbotG commented Jul 12, 2022

jcscottiii commented Jul 12, 2022

jcscottiii commented Jul 12, 2022

foolip commented Jul 12, 2022

STP downloads are failing #34752

STP downloads are failing #34752

Comments

foolip commented Jul 8, 2022

gsnedders commented Jul 8, 2022

gsnedders commented Jul 8, 2022

jcscottiii commented Jul 11, 2022

TalbotG commented Jul 12, 2022

Starting: Install Safari Technology Preview

Task : Command line Description : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows Version : 2.201.1 Author : Microsoft Corporation Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line

jcscottiii commented Jul 12, 2022

jcscottiii commented Jul 12, 2022

foolip commented Jul 12, 2022

Task : Command line
Description : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
Version : 2.201.1
Author : Microsoft Corporation
Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line