Cannot re-record AKS tests

[tjprescott]

Whilst re-recording tests for network API version 2019-02-01, several AKS tests failed to re-record. Since these are also failing in the daily live test run, they have been marked @live_only to permit the network changes to be merged.

CONVERTED TO LIVE ONLY

• test_aks_create_default_service
• test_aks_create_scale_with_custom_nodepool_name
• test_aks_create_default_service_without_skip_role_assignment
• test_aks_create_default_service_with_skip_role_assignment
• test_aks_create_default_service_without_SP_and_with_role_assignment
• test_aks_create_default_service_with_monitoring_addon
• test_aks_create_service_no_wait

[mboersma]

@zqingqing1 @GaneshaThirumurthi @samkreter could someone on the AKS team please take a look?

[tjprescott]

Updated to add test_aks_create_scale_with_custom_nodepool_name and test_aks_create_service_no_wait

[Juliehzl]

Remember here and here.. Also need to re-record because there are many conflicts in test_aks_create_default_service_with_skip_role_assignment.yaml and test_aks_create_default_service_without_skip_role_assignment.yaml

[Juliehzl]

@jluk Can you have a look at this?

[jluk]

Asked around, it seems like these tests fail due to AAD replication slowness - something that has been impacting prod customers as well. We have a separate ask on AAD to improve this but the timeline isn't in the near-term 3 months.

@Juliehzl is it possible to disable these tests as long as we run the existing tests and record new test cases on each change? Could you explain the purpose of these re-recordings, not sure I understand how it detects issues in AKS CLI paths?