Skip to content
This repository was archived by the owner on Nov 24, 2025. It is now read-only.

TO API Tests in CiaB: Do not test API v1#6669

Merged
ocket8888 merged 1 commit intoapache:masterfrom
zrhoffman:ciab-to-tests-no-api-v1
Mar 17, 2022
Merged

TO API Tests in CiaB: Do not test API v1#6669
ocket8888 merged 1 commit intoapache:masterfrom
zrhoffman:ciab-to-tests-no-api-v1

Conversation

@zrhoffman
Copy link
Member

@zrhoffman zrhoffman commented Mar 17, 2022

This PR makes the traffic_ops_integration_tests CDN in a Box service no longer try to run v1 of the Traffic Ops API tests.


Which Traffic Control components are affected by this PR?

  • CDN in a Box

What is the best way to verify this PR?

cd infrastructure/cdn-in-a-box
docker-compose up -d
docker-compose -f docker-compose.yml -f docker-compose.traffic-ops-test.yml up --build integration

and verify that it does not try to run v1 of the API tests

PR submission checklist

@zrhoffman zrhoffman added tests related to tests and/or testing infrastructure cdn-in-a-box related to the Docker-based CDN-in-a-Box system labels Mar 17, 2022
@ocket8888 ocket8888 added the tech debt rework due to choosing easy/limited solution label Mar 17, 2022
Copy link
Contributor

@ericholguin ericholguin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Ran the TO Integration tests and v1 did not try to run.

@ocket8888
Copy link
Contributor

I ran the tests locally and got a v4 failure. I don't think it's related to your changes, though.

JUNIT output
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
	<testsuite tests="315" failures="1" time="363.980" name="golang.test.toapi.v4">
		<properties>
			<property name="go.version" value="go1.18"></property>
		</properties>
		<testcase classname="golang.test.toapi.v4" name="TestAbout" time="0.020"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestAbout/GET" time="0.020"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestAbout/GET/OK_when_VALID_request" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestAbout/GET/UNAUTHORIZED_when_NOT_LOGGED_IN" time="0.020"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestAcmeAutoRenew" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestAcmeAutoRenew/POST" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestAcmeAutoRenew/POST/OK_when_VALID_request" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestASN" time="1.130"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestASN/GET_AFTER_CHANGES" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestASN/GET_AFTER_CHANGES/OK_when_CHANGES_made" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestASN/GET" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestASN/GET/NOT_MODIFIED_when_NO_CHANGES_made" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestASN/GET/OK_when_VALID_request" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestASN/GET/OK_when_VALID_ASN_PARAMETER" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestASN/PUT" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestASN/PUT/OK_when_VALID_request" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups" time="6.380"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/GET_AFTER_CHANGES" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/GET_AFTER_CHANGES/OK_when_CHANGES_made" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/GET" time="0.150"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/GET/OK_when_VALID_request" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/GET/OK_when_VALID_TOPOLOGY_parameter" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/GET/EMPTY_RESPONSE_when_INVALID_TYPE_parameter" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/GET/SECOND_RESULT_when_LIMIT=1_PAGE=2" time="0.020"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/GET/SECOND_RESULT_when_LIMIT=1_OFFSET=1" time="0.020"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/GET/NOT_MODIFIED_when_NO_CHANGES_made" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/GET/OK_when_VALID_SHORTNAME_parameter" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/GET/OK_when_VALID_TYPE_parameter" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/GET/EMPTY_RESPONSE_when_INVALID_ID_parameter" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/GET/FIRST_RESULT_when_LIMIT=1" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/GET/OK_when_VALID_NAME_parameter_AND_Lat/Long_are_0" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/GET/NOT_MODIFIED_when_VALID_SHORTNAME_parameter_when_NO_CHANGES_made" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/GET/OK_when_VALID_NAME_parameter" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/GET/BAD_REQUEST_when_INVALID_PAGE_parameter" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/GET/NOT_MODIFIED_when_VALID_NAME_parameter_when_NO_CHANGES_made" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/GET/BAD_REQUEST_when_INVALID_LIMIT_parameter" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/GET/BAD_REQUEST_when_INVALID_OFFSET_parameter" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/GET/UNAUTHORIZED_when_NOT_LOGGED_IN" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/POST" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/POST/UNAUTHORIZED_when_NOT_LOGGED_IN" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/PUT" time="0.080"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/PUT/OK_when_VALID_request" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/PUT/OK_when_updating_CG_with_null_Lat/Long" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/PUT/BAD_REQUEST_when_updating_TYPE_of_CG_in_TOPOLOGY" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/PUT/PRECONDITION_FAILED_when_updating_with_IMS_&amp;_IUS_Headers" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/PUT/PRECONDITION_FAILED_when_updating_with_IFMATCH_ETAG_Header" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/PUT/UNAUTHORIZED_when_NOT_LOGGED_IN" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/DELETE" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/DELETE/NOT_FOUND_when_INVALID_ID_parameter" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroups/DELETE/UNAUTHORIZED_when_NOT_LOGGED_IN" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCacheGroupsDeliveryServices" time="8.200"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDomains" time="2.300"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNLocks" time="8.520"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNLocks/GET" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNLocks/GET/OK_when_VALID_request" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNLocks/GET/OK_when_VALID_CDN_parameter" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNLocks/POST" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNLocks/POST/CREATED_when_VALID_request" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNLocks/DELETE" time="0.280"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNLocks/DELETE/OK_when_VALID_request" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNLocks/DELETE/FORBIDDEN_when_NON-ADMIN_USER_DOESNT_OWN_LOCK" time="0.280"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNLocks/DELETE/OK_when_ADMIN_USER_DOESNT_OWN_LOCK" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNLocks/SNAPSHOT" time="0.320"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNLocks/SNAPSHOT/OK_when_USER_OWNS_LOCK" time="0.030"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNLocks/SNAPSHOT/FORBIDDEN_when_ADMIN_USER_DOESNT_OWN_LOCK" time="0.290"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNLocks/SERVERS_QUEUE_UPDATES" time="0.380"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNLocks/SERVERS_QUEUE_UPDATES/FORBIDDEN_when_ADMIN_USER_DOESNT_OWN_LOCK" time="0.360"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNLocks/SERVERS_QUEUE_UPDATES/OK_when_USER_OWNS_LOCK" time="0.020"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNLocks/TOPOLOGY_QUEUE_UPDATES" time="0.320"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNLocks/TOPOLOGY_QUEUE_UPDATES/FORBIDDEN_when_ADMIN_USER_DOESNT_OWN_LOCK" time="0.310"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNLocks/TOPOLOGY_QUEUE_UPDATES/OK_when_USER_OWNS_LOCK" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNLocks/CACHE_GROUP_UPDATE" time="0.320"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNLocks/CACHE_GROUP_UPDATE/OK_when_USER_OWNS_LOCK" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNLocks/CACHE_GROUP_UPDATE/FORBIDDEN_when_ADMIN_USER_DOESNT_OWN_LOCK" time="0.300"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNQueueUpdateByProfileAndType" time="6.520"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNFederations" time="6.370"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestFederationFederationResolvers" time="7.140"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestFederationFederationResolvers/Successfully_assign_one_federation_resolver_to_a_federation" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestFederationFederationResolvers/Successfully_assign_multiple_federation_resolver_to_a_federation" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestFederationFederationResolvers/Successfully_replace_all_federation_resolver_for_a_federation" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestFederationFederationResolvers/Fail_to_assign_federation_resolver_to_a_federation_when_federation_does_not_exist" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestFederationFederationResolvers/successfully_get_federation_federation_resolvers_for_a_federation_with_some" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestFederationFederationResolvers/successfully_get_federation_federation_resolvers_for_a_federation_without_any" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNNotifications" time="0.100"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNs" time="1.140"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNs/get_CDNs_filtered_by_Domain_Name" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCDNsDNSSEC" time="10.810"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCookies" time="0.260"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCoordinates" time="0.420"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestCRConfig" time="7.050"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServiceRequestComments" time="0.960"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServiceRequests" time="0.660"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServiceRequestRequired" time="0.930"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServiceRequestRules" time="0.580"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServiceRequestTypeFields" time="0.710"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServiceRequestBad" time="0.900"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServiceRequestWorkflow" time="0.840"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities" time="7.460"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/assign_a_deliveryservice_to_a_required_capability;_deliveryServiceID:_0,_requiredCapability:_foo" time="0.030"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/assign_a_deliveryservice_to_a_required_capability;_deliveryServiceID:_0,_requiredCapability:_bar" time="0.030"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/assign_a_deliveryservice_to_a_required_capability;_deliveryServiceID:_0,_requiredCapability:_bar#01" time="0.030"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/re-assign_a_deliveryservice_to_a_required_capability;_deliveryServiceID:_764,_requiredCapability:_foo" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/assign_a_deliveryservice_to_a_required_capability_with_no_delivery_service_id;_deliveryServiceID:_0,_requiredCapability:_foo" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/assign_a_deliveryservice_to_a_required_capability_with_no_requiredCapability;_deliveryServiceID:_764,_requiredCapability:_0" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/assign_a_deliveryservice_to_a_required_capability_with_an_invalid_required_capability;_deliveryServiceID:_764,_requiredCapability:_bogus" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/assign_a_deliveryservice_to_a_required_capability_with_an_invalid_delivery_service_id;_deliveryServiceID:_-1,_requiredCapability:_foo" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/assign_a_deliveryservice_to_a_required_capability_with_an_invalid_deliveryservice_type" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/get_all_deliveryservices_required_capabilities" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/get_all_deliveryservices_required_capabilities_by_deliveryServiceID:_764" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/get_all_deliveryservices_required_capabilities_by_xmlID:_ds1" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/get_all_deliveryservices_required_capabilities_by_requiredCapability:_foo" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/get_all_deliveryservices_required_capabilities#01" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/get_all_deliveryservices_required_capabilities_by_deliveryServiceID:_764#01" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/get_all_deliveryservices_required_capabilities_by_xmlID:_ds1#01" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/get_all_deliveryservices_required_capabilities_by_requiredCapability:_foo#01" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/get_all_deliveryservices_required_capabilities#02" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/get_all_deliveryservices_required_capabilities_by_deliveryServiceID:_764#02" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/get_all_deliveryservices_required_capabilities_by_xmlID:_ds1#02" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/get_all_deliveryservices_required_capabilities_by_requiredCapability:_foo#02" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/get_all_deliveryservices_required_capabilities#03" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/get_all_deliveryservices_required_capabilities_by_deliveryServiceID:_764#03" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/get_all_deliveryservices_required_capabilities_by_xmlID:_ds1#03" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/get_all_deliveryservices_required_capabilities_by_requiredCapability:_foo#03" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/delete_a_deliveryservices_required_capability_with_an_invalid_delivery_service_id;_deliveryServiceID:_-1,_requiredCapability:_foo" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/delete_a_deliveryservices_required_capability_with_an_invalid_required_capability;_deliveryServiceID:_764,_requiredCapability:_bogus" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/delete_a_deliveryservices_required_capability;_deliveryServiceID:_764,_requiredCapability:_foo" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/delete_a_deliveryservices_required_capability;_deliveryServiceID:_765,_requiredCapability:_bar" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRequiredCapabilities/delete_a_deliveryservices_required_capability;_deliveryServiceID:_769,_requiredCapability:_bar" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestTopologyBasedDeliveryServicesRequiredCapabilities" time="7.740"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestTopologyBasedDeliveryServicesRequiredCapabilities/get_all_deliveryservices_required_capabilities" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestTopologyBasedDeliveryServicesRequiredCapabilities/get_all_deliveryservices_required_capabilities_by_deliveryServiceID:_785" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestTopologyBasedDeliveryServicesRequiredCapabilities/get_all_deliveryservices_required_capabilities_by_xmlID:_ds1" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestTopologyBasedDeliveryServicesRequiredCapabilities/get_all_deliveryservices_required_capabilities_by_requiredCapability:_foo" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestTopologyBasedDeliveryServicesRequiredCapabilities/delete_a_deliveryservices_required_capability_with_an_invalid_delivery_service_id;_deliveryServiceID:_-1,_requiredCapability:_ram" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestTopologyBasedDeliveryServicesRequiredCapabilities/delete_a_deliveryservices_required_capability_with_an_invalid_required_capability;_deliveryServiceID:_793,_requiredCapability:_bogus" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestTopologyBasedDeliveryServicesRequiredCapabilities/delete_a_deliveryservices_required_capability;_deliveryServiceID:_793,_requiredCapability:_ram" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestTopologyBasedDeliveryServicesRequiredCapabilities/delete_a_deliveryservices_required_capability;_deliveryServiceID:_793,_requiredCapability:_disk" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestTopologyBasedDeliveryServicesRequiredCapabilities/delete_a_deliveryservices_required_capability;_deliveryServiceID:_794,_requiredCapability:_ram" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices" time="30.890"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Verify_SSL_key_generation_on_DS_creation" time="13.230"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Update_CDN_for_a_Delivery_Service_with_SSL_keys" time="1.470"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Create_URL_Signature_keys_for_a_Delivery_Service" time="0.050"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Retrieve_URL_Signature_keys_for_a_Delivery_Service" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Delete_URL_Signature_keys_for_a_Delivery_Service" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Create_URI_Signing_Keys_for_a_Delivery_Service" time="0.030"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Retrieve_URI_Signing_keys_for_a_Delivery_Service" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Delete_URI_Signing_keys_for_a_Delivery_Service" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Delete_old_CDN_SSL_keys" time="2.730"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Create_and_retrieve_SSL_keys_for_a_Delivery_Service" time="2.680"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Create_a_Delivery_Service_with_geo_limit_countries" time="0.030"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Create_a_Delivery_Service_with_the_removed_Long_Description_2_and_3_fields" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Update_a_Delivery_Service,_setting_its_removed_Long_Description_2_and_3_fields" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Getting_unmodified_Delivery_Services_using_the_If-Modified-Since_header" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Getting_Delivery_Services_accessible_to_a_Tenant" time="0.030"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Basic_update_of_some_Delivery_Service_fields" time="0.030"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Assign_an_Origin_not_in_a_Cache_Group_used_by_a_Delivery_Service&#39;s_Topology_to_that_Delivery_Service" time="0.040"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Attempt_to_update_a_Delivery_Service_with_If-Unmodified-Since" time="0.020"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Basic_update_of_some_other_Delivery_Service_fields" time="0.030"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Update_a_Delivery_Service_giving_it_invalid_Raw_Remap_text" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Update_a_Delivery_Service_giving_it_invalid_combinations_of_Range_Slice_Block_Size_and_Range_Request_Handling" time="0.020"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Update_a_Delivery_Service_giving_it_invalid_combinations_of_Range_Slice_Block_Size_and_Range_Request_Handling/Missing_slice_plugin_size" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Update_a_Delivery_Service_giving_it_invalid_combinations_of_Range_Slice_Block_Size_and_Range_Request_Handling/Slice_plugin_size_set_with_incorrect_range_request_setting" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Update_a_Delivery_Service_giving_it_invalid_combinations_of_Range_Slice_Block_Size_and_Range_Request_Handling/Slice_plugin_size_set_to_small" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Update_a_Delivery_Service_giving_it_invalid_combinations_of_Range_Slice_Block_Size_and_Range_Request_Handling/Slice_plugin_size_set_to_large" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Invalid_Topology-to-Delivery_Service_assignments" time="0.160"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Getting_modified_Delivery_Services_using_the_If-Modified-Since_header" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Basic_update_of_Delivery_Service_header_rewrite_fields" time="0.320"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Basic_GET_request" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/GET_requests_using_the_&#39;active&#39;_query_string_parameter" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Basic_GET_request_for_/deliveryservices/{{ID}}/capacity" time="0.120"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Update_fields_added_in_new_minor_versions_of_the_API" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Verify_Tenancy-restricted_Delivery_Service_access" time="0.950"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Attempt_to_create_invalid_Delivery_Services" time="0.030"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Attempt_to_update_a_Delivery_Service_with_If-Match" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/GET_requests_using_pagination-controlling_query_string_parameters" time="0.030"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/GET_requests_using_the_&#39;cdn&#39;_query_string_parameter" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Check_behavior_of_&#39;cdn&#39;_query_string_parameter_when_CDN_doesn&#39;t_exist" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Check_behavior_of_&#39;profile&#39;_query_string_parameter_when_Profile_doesn&#39;t_exist" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Check_behavior_of_&#39;tenant&#39;_query_string_parameter_when_Tenant_doesn&#39;t_exist" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Check_behavior_of_&#39;type&#39;_query_string_parameter_when_Type_doesn&#39;t_exist" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Check_behavior_of_&#39;accessibleTo&#39;_query_string_parameter_when_the_Tenant_doesn&#39;t_exist" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Check_behavior_of_&#39;xmlId&#39;_query_string_parameter_when_xmlId_doesn&#39;t_exist" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/GET_request_using_the_&#39;logsEnabled&#39;_query_string_parameter" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/GET_request_using_the_&#39;profile&#39;_query_string_parameter" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/GET_request_using_the_&#39;tenant&#39;_query_string_parameter" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/GET_request_using_the_&#39;type&#39;_query_string_parameter" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/GET_request_using_the_&#39;xmlId&#39;_query_string_parameter" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/GET_request_using_the_&#39;topology&#39;_query_string_parameter" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/GET_request_using_the_&#39;servicecategory&#39;_query_string_parameter" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Descending_order_sorted_response_to_GET_request" time="0.020"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/Create/_Update/_Delete_delivery_services_with_locks" time="1.410"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServices/TLS_Versions_property" time="0.030"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServiceServers" time="6.530"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServiceServersWithRequiredCapabilities" time="8.090"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServiceServersWithRequiredCapabilities/assign_a_deliveryservice_to_a_required_capability;_deliveryServiceID:_0,_requiredCapability:_foo" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServiceServersWithRequiredCapabilities/assign_a_deliveryservice_to_a_required_capability;_deliveryServiceID:_0,_requiredCapability:_bar" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServiceServersWithRequiredCapabilities/assign_a_deliveryservice_to_a_required_capability;_deliveryServiceID:_0,_requiredCapability:_bar#01" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServiceServersWithRequiredCapabilities/re-assign_a_deliveryservice_to_a_required_capability;_deliveryServiceID:_855,_requiredCapability:_foo" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServiceServersWithRequiredCapabilities/assign_a_deliveryservice_to_a_required_capability_with_no_delivery_service_id;_deliveryServiceID:_0,_requiredCapability:_foo" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServiceServersWithRequiredCapabilities/assign_a_deliveryservice_to_a_required_capability_with_no_requiredCapability;_deliveryServiceID:_855,_requiredCapability:_0" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServiceServersWithRequiredCapabilities/assign_a_deliveryservice_to_a_required_capability_with_an_invalid_required_capability;_deliveryServiceID:_855,_requiredCapability:_bogus" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServiceServersWithRequiredCapabilities/assign_a_deliveryservice_to_a_required_capability_with_an_invalid_delivery_service_id;_deliveryServiceID:_-1,_requiredCapability:_foo" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServiceServersWithRequiredCapabilities/assign_a_deliveryservice_to_a_required_capability_with_an_invalid_deliveryservice_type" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServiceServersWithRequiredCapabilities/missing_requirements_for_server_-&gt;_DS_assignment" time="0.030"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServiceServersWithRequiredCapabilities/successful_server_-&gt;_DS_assignment" time="0.030"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServiceServersWithRequiredCapabilities/delete_a_deliveryservices_required_capability_with_an_invalid_delivery_service_id;_deliveryServiceID:_-1,_requiredCapability:_foo" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServiceServersWithRequiredCapabilities/delete_a_deliveryservices_required_capability_with_an_invalid_required_capability;_deliveryServiceID:_855,_requiredCapability:_bogus" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServiceServersWithRequiredCapabilities/delete_a_deliveryservices_required_capability;_deliveryServiceID:_855,_requiredCapability:_foo" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServiceServersWithRequiredCapabilities/delete_a_deliveryservices_required_capability;_deliveryServiceID:_856,_requiredCapability:_bar" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServiceServersWithRequiredCapabilities/delete_a_deliveryservices_required_capability;_deliveryServiceID:_860,_requiredCapability:_bar" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesEligible" time="7.480"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDeliveryServicesRegexes" time="7.750"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestDivisions" time="0.360"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestFederationResolvers" time="0.410"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestFederationResolvers/Get_Federation_Resolvers_by_ID" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestFederationResolvers/Get_Federation_Resolvers_by_IPAddress" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestFederationResolvers/Get_Federation_Resolvers_by_Type" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestFederationUsers" time="7.660"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestFederations" time="6.490"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestGetOSVersions" time="0.040"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestGetOSVersions/authenticated" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestGetOSVersions/un-authenticated" time="0.020"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestGetOSVersions/parameter-invalid" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestJobs" time="6.780"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestLoginFail" time="1.090"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestLogs" time="1.200"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestMonitoring" time="6.360"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestOrigins" time="9.070"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestParameters" time="0.880">
			<failure message="Failed" type="">    parameters_test.go:489: Invalid ID shouldn&#39;t have any response {[{url_sig_msods1.config 10000 0001-01-01 00:00:00 +0000 UTC location [91 93] false /remap/config/location/parameter}] {[]}} Error &lt;nil&gt;&#xA;    parameters_test.go:340: cannot DELETE Parameters by Invalid ID: &lt;nil&gt; - {[{param was deleted. success}]}&#xA;    parameters_test.go:343: Expected 404 status code, got 200</failure>
		</testcase>
		<testcase classname="golang.test.toapi.v4" name="TestPhysLocations" time="0.750"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestPing" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestProfileParameters" time="2.760"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestProfiles" time="3.440"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestProfiles/Attempt_to_create_invalid_Profiles" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestProfiles/Basic_update_of_properties" time="0.020"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestProfiles/Try_to_update_a_Profile_with_If-Unmodified-Since_set_to_5_seconds_ago_-_presumably_before_its_creation" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestProfiles/Support_for_If-Modified-Since_in_GET" time="0.030"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestProfiles/Basic_GET_request_for_/profiles_and_export_endpoint" time="0.100"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestProfiles/Check_&#39;parameters&#39;_property_returned_in_GET_requsets" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestProfiles/Import_endpoint_basic_operation" time="0.030"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestProfiles/Copy_endpoint_basic_operation" time="0.020"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestProfiles/Copy_endpoint_basic_operation/new_profile_name_contains_spaces" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestProfiles/Copy_endpoint_basic_operation/copy_profile" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestProfiles/Copy_endpoint_basic_operation/existing_profile_does_not_exist" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestProfiles/Copy_endpoint_basic_operation/new_profile_already_exists" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestProfiles/Try_to_update_a_Profile_with_If-Match_set" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestProfiles/Verify_pagination_query_string_parameters_support" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestProfiles/Verify_Profile_endpoints_are_locked_by_CDN_locks" time="1.050"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestReadOnlyCannotModify" time="7.230"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestRegions" time="0.370"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestRoles" time="0.080"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerCapabilities" time="0.100"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerCheckExtensions" time="0.960"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerChecks" time="6.610"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServers" time="12.650"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestAssignments" time="7.290"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerServerCapabilities" time="8.190"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerServerCapabilities/assign_a_deliveryservice_to_a_required_capability;_deliveryServiceID:_0,_requiredCapability:_foo" time="0.020"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerServerCapabilities/assign_a_deliveryservice_to_a_required_capability;_deliveryServiceID:_0,_requiredCapability:_bar" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerServerCapabilities/assign_a_deliveryservice_to_a_required_capability;_deliveryServiceID:_0,_requiredCapability:_bar#01" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerServerCapabilities/re-assign_a_deliveryservice_to_a_required_capability;_deliveryServiceID:_1086,_requiredCapability:_foo" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerServerCapabilities/assign_a_deliveryservice_to_a_required_capability_with_no_delivery_service_id;_deliveryServiceID:_0,_requiredCapability:_foo" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerServerCapabilities/assign_a_deliveryservice_to_a_required_capability_with_no_requiredCapability;_deliveryServiceID:_1086,_requiredCapability:_0" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerServerCapabilities/assign_a_deliveryservice_to_a_required_capability_with_an_invalid_required_capability;_deliveryServiceID:_1086,_requiredCapability:_bogus" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerServerCapabilities/assign_a_deliveryservice_to_a_required_capability_with_an_invalid_delivery_service_id;_deliveryServiceID:_-1,_requiredCapability:_foo" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerServerCapabilities/assign_a_deliveryservice_to_a_required_capability_with_an_invalid_deliveryservice_type" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerServerCapabilities/delete_a_deliveryservices_required_capability_with_an_invalid_delivery_service_id;_deliveryServiceID:_-1,_requiredCapability:_foo" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerServerCapabilities/delete_a_deliveryservices_required_capability_with_an_invalid_required_capability;_deliveryServiceID:_1086,_requiredCapability:_bogus" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerServerCapabilities/delete_a_deliveryservices_required_capability;_deliveryServiceID:_1086,_requiredCapability:_foo" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerServerCapabilities/delete_a_deliveryservices_required_capability;_deliveryServiceID:_1087,_requiredCapability:_bar" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerServerCapabilities/delete_a_deliveryservices_required_capability;_deliveryServiceID:_1091,_requiredCapability:_bar" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerServerCapabilities/delete_a_deliveryservices_required_capability;_deliveryServiceID:_1104,_requiredCapability:_blah" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerServerCapabilitiesForTopologies" time="7.130"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerServerCapabilitiesForTopologies/delete_a_deliveryservices_required_capability_with_an_invalid_delivery_service_id;_deliveryServiceID:_-1,_requiredCapability:_ram" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerServerCapabilitiesForTopologies/delete_a_deliveryservices_required_capability_with_an_invalid_required_capability;_deliveryServiceID:_1115,_requiredCapability:_bogus" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerServerCapabilitiesForTopologies/delete_a_deliveryservices_required_capability;_deliveryServiceID:_1115,_requiredCapability:_ram" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerServerCapabilitiesForTopologies/delete_a_deliveryservices_required_capability;_deliveryServiceID:_1115,_requiredCapability:_disk" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerServerCapabilitiesForTopologies/delete_a_deliveryservices_required_capability;_deliveryServiceID:_1116,_requiredCapability:_ram" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerUpdateStatusLastAssigned" time="6.490"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerUpdateStatus" time="5.830"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerQueueUpdate" time="5.440"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerQueueUpdate/true" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerQueueUpdate/false" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServerQueueUpdate/validations" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestSetServerUpdateStatuses" time="5.750"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestSetTopologiesServerUpdateStatuses" time="5.770"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestServiceCategories" time="0.070"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestStaticDNSEntries" time="7.940"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestStatsSummary" time="0.040"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestStatsSummary/get_all_summary_stats" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestStatsSummary/non-existant_stat_name" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestStatsSummary/non-existant_ds_name" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestStatsSummary/non-existant_cdn_name" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestStatsSummary/get_stats_summary_by_stat_name" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestStatsSummary/get_stats_summary_by_cdn_name" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestStatsSummary/get_stats_summary_by_ds_name" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestStatsSummary/latest_updated_timestamp" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestStatsSummary/non-existant_stat_name#01" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestStatsSummary/latest_updated_timestamp_for_-_daily_maxgbps" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestStatsSummary/latest_updated_timestamp_for_-_daily_bytesserved" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestStatuses" time="0.330"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestSteering" time="7.990"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestSteeringTargets" time="8.030"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestTenants" time="0.150"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestTenantsActive" time="7.470"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestTopologiesQueueUpdate" time="7.260"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestTopologies" time="9.420"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestTopologies/delete_a_deliveryservices_required_capability_with_an_invalid_delivery_service_id;_deliveryServiceID:_-1,_requiredCapability:_ram" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestTopologies/delete_a_deliveryservices_required_capability_with_an_invalid_required_capability;_deliveryServiceID:_1262,_requiredCapability:_bogus" time="0.000"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestTopologies/delete_a_deliveryservices_required_capability;_deliveryServiceID:_1262,_requiredCapability:_ram" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestTopologies/delete_a_deliveryservices_required_capability;_deliveryServiceID:_1262,_requiredCapability:_disk" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestTopologies/delete_a_deliveryservices_required_capability;_deliveryServiceID:_1263,_requiredCapability:_ram" time="0.010"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestTrafficVaultPing" time="6.850"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestTypes" time="1.060"></testcase>
		<testcase classname="golang.test.toapi.v4" name="TestUsers" time="3.040"></testcase>
	</testsuite>
</testsuites>

Curious why this is happening - also, why do we still have this dockerfile if they can not pass for months at a time without anyone noticing? Maybe we should just get rid of it instead? The GHAs already run the API tests.

@ericholguin
Copy link
Contributor

Curious why this is happening - also, why do we still have this dockerfile if they can not pass for months at a time without anyone noticing? Maybe we should just get rid of it instead? The GHAs already run the API tests.

Doesn't automation use these tests?

@ocket8888
Copy link
Contributor

ocket8888 commented Mar 17, 2022

Curious why this is happening - also, why do we still have this dockerfile if they can not pass for months at a time without anyone noticing? Maybe we should just get rid of it instead? The GHAs already run the API tests.

Doesn't automation use these tests?

No, the GHAs run the tests without Docker or CDN-in-a-Box using the workflow at .github/workflows/traffic-ops.yml and the action at .github/actions/to-integration-tests/ (which is why they didn't trigger on this PR and why we didn't notice that they've apparently been trying to test a non-existent API version for I think a whole year now).

@ericholguin
Copy link
Contributor

Curious why this is happening - also, why do we still have this dockerfile if they can not pass for months at a time without anyone noticing? Maybe we should just get rid of it instead? The GHAs already run the API tests.

Doesn't automation use these tests?

No, the GHAs run the tests without Docker or CDN-in-a-Box using the workflow at .github/workflows/traffic-ops.yml and the action at .github/actions/to-integration-tests/

I meant lab automation

@zrhoffman
Copy link
Member Author

I ran the tests locally and got a v4 failure.
for readability:

parameters_test.go:489: Invalid ID shouldn't have any response {[{url_sig_msods1.config 10000 0001-01-01 00:00:00 +0000 UTC location [91 93] false /remap/config/location/parameter}] {[]}} Error <nil>
    parameters_test.go:340: cannot DELETE Parameters by Invalid ID: <nil> - {[{param was deleted. success}]}
    parameters_test.go:343: Expected 404 status code, got 200

Earlier today, I got a different failure from v4 in CiaB:

servers_test.go:1342: got expected error when updating Server Type of a server assigned to DSes - type change update alerts: {Alerts:[{Text:server type can not be updated when it is currently assigned to Delivery Services Level:error}]}, err: error requesting Traffic Ops: path '[https://trafficops.infra.ciab.test:443/api/4.0/servers/3438](https://trafficops.infra.ciab.test/api/4.0/servers/3438)' gave HTTP error 409 Conflict - error-level alerts: server type can not be updated when it is currently assigned to Delivery Services
    servers_test.go:501: Expected 200 status code, got 304

Curious why this is happening - also, why do we still have this dockerfile if they can not pass for months at a time without anyone noticing?

We need to pay better attention to it

Maybe we should just get rid of it instead? The GHAs already run the API tests.

Yes the GHAs run the TO API tests, but this failure is not in GHA, so we lose test coverage by removing it from CiaB.

@zrhoffman
Copy link
Member Author

We could start by having a GitHub Action that runs the API tests in CDN in a Box, but only when stuff in the infrastructure/cdn-in-a-box/traffic_ops_integration_test/ directory is modified. But IMO it's important not to just remove failing tests.

@ocket8888
Copy link
Contributor

Yes the GHAs run the TO API tests, but this failure is not in GHA, so we lose test coverage by removing it from CiaB.

Sure, but that's unintentional. My understanding is the two are supposed to be equivalent. They're both running the same test suite, after all. And is it really "test coverage" the tests are never being run and nobody knows when they fail?

We need to pay better attention to it

We could - or we could figure out what cases it's covering that aren't covered by existing automation and absorb those.

On that note, I have no idea what those are. I'm truly puzzled why it's failing (and inconsistently, too) in this CiaB extension and not in general. Whatever we do with this test suite, we'll need to figure that out.


All else aside, it is true that when I ran the tests they skipped APIv1. So this PR does what it says it does and presumably helps the tests get closer to a passing state. I'll merge it if you want, but at some point we gotta figure out why they fail for us locally and then either fix them or figure out what needs to be done to other tests to make sure these don't cover a gap somewhere else. So you could add that to this PR if you want, but it's good to go as-is as far as I'm concerned.

@ocket8888
Copy link
Contributor

IMO it's important not to just remove failing tests.

Oh, no, I never meant to suggest we just sweep this under the rug. We need to know why these fail, whatever we decide to do with them.

@zrhoffman
Copy link
Member Author

zrhoffman commented Mar 17, 2022

We could - or we could figure out what cases it's covering that aren't covered by existing automation and absorb those.

On that note, I have no idea what those are. I'm truly puzzled why it's failing (and inconsistently, too) in this CiaB extension and not in general. Whatever we do with this test suite, we'll need to figure that out.

Part of it is that t3c and Traffic Monitor are running in CDN in a Box, so they are hitting endpoints while the API tests are running

I'll merge it if you want

want

but at some point we gotta figure out why they fail for us locally and then either fix them or figure out what needs to be done to other tests to make sure these don't cover a gap somewhere else.

agree

So you could add that to this PR if you want

The way I see it, there are a few things we could add:

  • The aforementioned GHA for TO API Tests in CiaB but only runs when infrastructure/cdn-in-a-box/traffic_ops_integration_test/ is modified but can also be run using workflow_dispatch (button)
  • Junit parsing that annotates each line of code that had a failure
  • Consider adding t3c or traffic monitor to the GHA?

I don't know what else I could add for any of those that would be relevant to this PR, though. Additional stuff seems better-suited for a different PR

@ocket8888
Copy link
Contributor

Additional stuff seems better-suited for a different PR

Agreed. It discomforts me a bit to merge failing tests, but they already were and fixing them is out-of-scope for the purpose of this PR.

@ocket8888 ocket8888 merged commit 9c7786b into apache:master Mar 17, 2022
@zrhoffman zrhoffman deleted the ciab-to-tests-no-api-v1 branch March 17, 2022 21:33
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

cdn-in-a-box related to the Docker-based CDN-in-a-Box system tech debt rework due to choosing easy/limited solution tests related to tests and/or testing infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants