Arlington automation

[abhidnya13]

E2E Test Scenarios added:

• Acquire token by username password
• Acquire token by client_secret
• Acquire token by on behalf of flow
• Acquire token by device code flow

Besides, acquire token silent is tested in each of the flows individually (except acquire token by client secret)

[abhidnya13]

cc: @henrik-me

[rayluo]

Really THANK YOU for your effort here! Seeing all those actual test cases passing in CI, gives us a high confidence on the recent Arlington testing work. For that reason, I think you can go ahead to document our findings in the test, into that Arlington workitem, and close that part of work there.

For this PR here, we do have room for improvement, from a code maintenance perspective. I leave some comments below (and more to come). Let's polish it a little bit more in next couple days. Again, this part is NOT a blocker for the Arlington test effort.

[rayluo]

Historically, there was no explicit test case to cover confidential client in world-wide (WW) cloud. (We implicitly tested it by creating a confidential client as test infrastructure, though.)
That's why I LOVE seeing you add an explicit test case for confidential client this time! Let's see if we can follow the suggestion above, to try to backport this test case for WW cloud, in a reusable style.

[abhidnya13]

Thats good to have. I think that should go in a separate PR though once this is merged. Lets keep the scope of this PR to Arlington test cases ? Let me know if that works for you. If yes, I can go and create an issue tracking this.

[henrik-me]

If not doing this now, then please create a github issue to refactor test cases for multiple cloud enablement. You can point to this comment. Expect that I will be asking for the same automation to run in all the clouds we support and that code should be well structured to handle that (similar discussion happening in other repo's at this time)

[henrik-me]

I would have expected to see something like this:

test_allenvironments(config)
{
foreach (envriroment in config.TestEnvironments)
test_envrionment(envrironment, config)
}

test_environment(environment, config)
{
test_devicecode(environment, ...)
test_xyz(environment, ...)
}

test_devicecode(environment, ....) {}
test_xyz(environment) {}

Are there many differences between the environments making that not easy to do? Am I reading the code in the wrong way?

[henrik-me]

There seems to be several references to client id's etc. in the tests. Any reason the lab API is not used to get those as well as the secrets?

[abhidnya13]

Hey @henrik-me,

1. The code structured right now is implicitly doing a similar thing as you have mentioned. By default it is running through each environment and if we want to test a specific environment, we can run the test cases under that class. We cannot have a single config for an environment as we have multiple different users as a part of each environment right now, for example- usertype: federated, onprem, cloud version(for a federated user): adfsv2, v3, v4 etc. So, the queries to pull up config will have these values along with the environment. The base methods that these calls for different types of flows within each environment are defined in a parent class and then depending on the kind of user in that environment we provide a config to these base methods.
2. Thats a good point. For the new tests I've added for the Arlington , I have pulled the client_ids from the Lab API. There are a few test cases which were already present that don't do this and it will be good to get those from LabAPI as well and we should be able to do it. Created an issue to track this here

[rayluo]

A batch of editorial changes, many of them are provided with the Github Suggestion feature, for potentially easier adoption.

[rayluo]

This is probably the 2 last comments to this PR. We will be merging this soon. :-)

[rayluo]

Suggested change

	scope=["https://msidlabb2c.onmicrosoft.com/msidlabb2capi/read"]
	scope=["https://msidlabb2c.onmicrosoft.com/msidlabb2capi/read"],

Reasoning: PEP 8 on Trailing commas

---

PS: It is not necessarily that I paid attention to this kind of details. It is the github PR review UI highlighting the different lines in red-and-green in order to grab reviewer's attention. So, a universal hint for a PR author is to conduct a self-review on github PR UI to see whether each of those red-and-green snippets is intended and/or necessary.

[rayluo]

Thanks for all the effort! The entire end-to-end test suite looks more organized now. 👍

Let's squash and merge it! 🚢