Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Safari Technology Preview to 109 #24393

Conversation

@autofoolip
Copy link
Contributor

autofoolip commented Jun 30, 2020

Version as reported by safaridriver --version:
Included with Safari Technology Preview (Release 109, 15610.1.17.2)

Source: https://developer.apple.com/safari/download/
Build: https://dev.azure.com/foolip/safari-technology-preview-updater/_build/results?buildId=1940&view=logs

Version as reported by safaridriver --version:
Included with Safari Technology Preview (Release 109, 15610.1.17.2)

Source: https://developer.apple.com/safari/download/
Build: https://dev.azure.com/foolip/safari-technology-preview-updater/_build/results?buildId=1940&view=logs
@foolip
Copy link
Member

foolip commented Jul 1, 2020

@burg @stephenmcgruer I expect this won't be possible to land before #21628, making the problems with Catalina somewhat urgent, if it means we can't have any new results for Safari STP until it's resolved.

@stephenmcgruer
Copy link
Contributor

stephenmcgruer commented Jul 1, 2020

Thanks Philip, I had not realized that STP 109 was Catalina-and-up only. That certainly raises the priority; as a note the tracking issue for the problem is actually #21751, not #21628 (the latter is the pull request where we have been debugging).

cc @othermaciej as an FYI that if we are unable to resolve the Catalina issue here, we will no longer be able to get WPT results on new Safari Tech Preview versions.

@mustjab - is there someone from the Azure Pipelines team we could talk to about this issue? As per #21751 (comment), no-one has been able to reproduce this issue on anything outside of the Azure Pipeline images, and notably @burg found when looking at the sysdiagnose dump that "no error is returned on the file write operation" - but the plist file still isn't written. This all makes it seem like there's something special about the Azure Pipelines setup that we're failing to account for in our attempts to reproduce.

In the meantime, I'm going to try both the Catalina upgrade + STP 109 at the same time, just in case something new and wonderful has happened and the problem has gone away...

@foolip
Copy link
Member

foolip commented Jul 1, 2020

If there's a suspicion that this is something about Azure Pipelines, then it would be worth trying to run a few tests on other CI systems, like GitHub Actions, which doesn't appear to use the same underlying infra for macOS. We might not have enough capacity there to actually use it for our runs, but it might help form a better hypothesis of the problem.

@mustjab
Copy link
Contributor

mustjab commented Jul 1, 2020

@stephenmcgruer Sent mail to Azure Pipelines MacOS team, will let you know when I hear back.

@foolip
Copy link
Member

foolip commented Jul 1, 2020

I just tried my own suggestion in foolip/safari-ci-starter-kit@ecaf066 and it seems to me the problem is not specific to Azure Pipelines. Using macOS Catalina in all of Azure Pipelines, CircleCI, Cirrus CI, GitHub Actions, Travis CI, all of them fail in the same way: "Could not create a session: You must enable the 'Allow Remote Automation' option in Safari's Develop menu to control Safari via WebDriver"

Is there something that everyone has to do with macOS in a CI environment that isn't the case on a developer machine? What's going on? :D

@burg
Copy link
Contributor

burg commented Jul 4, 2020

@foolip
Copy link
Member

foolip commented Jul 4, 2020

Is there a command to test for that difference?

@stephenmcgruer
Copy link
Contributor

stephenmcgruer commented Jul 6, 2020

Is there a command to test for that difference?

To literally check for SSH we could look for SSH_TTY, SSH_CONNECTION, etc.

For other approaches, I'm not sure. Seems like we would be better off asking the Azure Pipelines team how the sessions work?

@foolip
Copy link
Member

foolip commented Jul 16, 2020

Obsoleted by #24631 (STP 110)

@foolip foolip closed this Jul 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

7 participants
You can’t perform that action at this time.