Mac test ignoring os field on prod task, ran on newer version of macOS

[jmagman]

Mac web_tool_tests ran on a macOS 12 machine, even though .ci.yaml specifies Mac-10.15.

flutter/.ci.yaml

Lines 3077 to 3078 in 37a1aaf

	os: Mac-10.15 # Override OS and Xcode for https://github.com/flutter/flutter/issues/98278
	xcode: 12c33

https://ci.chromium.org/p/flutter/builders/prod/Mac%20web_tool_tests/5034

https://chromium-swarm.appspot.com/task?id=5946338c72923b10&o=true&w=true
Task dimensions shows:

          {
            "key": "os",
            "value": "Mac-12"
          },

Note it failed due to #98278 which seems to only reproduce on macOS 12, which is how I noticed this.

[jmagman]

@keyonghan

[jmagman]

os isn't in the Input Properties list, but xcode is:

[jmagman]

More:
https://ci.chromium.org/p/flutter/builders/prod/Mac%20web_tool_tests/5026
https://ci.chromium.org/p/flutter/builders/prod/Mac%20web_tool_tests/5028
https://ci.chromium.org/p/flutter/builders/prod/Mac%20web_tool_tests/5030
https://ci.chromium.org/p/flutter/builders/prod/Mac%20web_tool_tests/5031

[keyonghan]

I am afraid this is expected. For post-submit builders, their configs will be auto-rolled after it's merged. Let me double check.

[jmagman]

How long does it usually take to auto-roll, that config change happened 21 hours ago #98297

[keyonghan]

The culprit is here: https://flutter.googlesource.com/infra/+/refs/heads/main/config/lib/ci_yaml/ci_yaml.star#315
It only reads the platform os value, ignoring target level os property.

[keyonghan]

https://flutter-review.googlesource.com/c/infra/+/27401 to fix.

[keyonghan]

New os version is picked up: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20web_tool_tests/5037/overview, but it seems still using new xcode version..

[keyonghan]

@CaseyHillers IIRC, the properties will be merged among platform and target. Is the xcode version defined in target level expected to be auto rolled?

[CaseyHillers]

It looks like def _properties(ci_yaml, target, default_properties = {}, repo = None, branch = None): in the starlark needs to be updated

It should pull from the merged _properties instead of the platform

[keyonghan]

https://flutter-review.googlesource.com/c/infra/+/27420 to override with target level property.

[keyonghan]

Changes populated, and test is validated to run on expected test beds.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.