Entra ID or AAD Functional test failures for MS.AAD.7.4v1 and MS.AAD.7.5v1

[nanda-katikaneni]

🐛 Summary

This issue was found through nightly product functional tests. Non-complaint test cases for MS.AAD.7.4v1 and MS.AAD.7.5v1 are failing on G5 and GCCHIGH tenants when run through GitHub action "Nightly functional product tests". However, the same tests are passing when run manually using a service principal. This issue and be used to debug and then fix the nightly product tests for aad.g5.testplan.yaml.

We have two service principals in the test tenants: GitHubActions and FunctionalTesting. GitHub actions use the first service principal app, while the dev team use the second app service for internal on-demand testing. Manually ran tests using both these apps with my certificate thumbprint - tests ran fine on g5 and gcchigh. But tests from GitHub actions are consistently failing non-complaint test cases for 7.4 and 7.5 policies. These policy test cases are run using run cached mode and are structured differently from the complaint test cases. These two policy test cases should be investigated further to make the nightly product test automation have clean runs.

To reproduce

Steps to reproduce the behavior:

1. Run nightly product test GH action for the AAD G5 test plan.
2. Observe results and failures for 7.4 and 7.5 policies.

Expected behavior

The nightly product functional testing using AAD G5 test plan should have a clean run with all tests being executed and passed.

Add any screenshots of the problem here.

[tkol2022]

The root cause for this was the functional test case hard coding the privileged_roles Assignments element [0] in the JSON array. That crashed in certain tenants.