Get-AzReservationCatalog fails with 'BadRequest', debug log shows 'SubscriptionNotRegistered'

[jberezanski-mdg]

Description

I am unable to retrieve the available reservation options for virtual machines because the Get-AzReservationCatalog cmdlet fails with status code 'BadRequest'. The debug output shows error code SubscriptionNotRegistered, message "Subscription is still pending registration or not registered.".

The subscription in question is a regular Pay-As-You-Go subscription, created over a year ago and used on a daily basis to run Service Fabric, Kubernetes and other services. I tried two user accounts: 1) the Account Admin of the subscription, 2) another user, who has the Owner role assigned on that subscription. Both users are added in AAD as Members from a different AAD directory than the one the subscription is linked to.

Steps to reproduce

Get-AzReservationCatalog -ReservedResourceType VirtualMachines -Location EastUS

Environment data

> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      5.1.14393.3471
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.14393.3471
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

Module versions

> Get-Module

ModuleType Version    Name
---------- -------    ----
Script     1.7.5      Az.Accounts
Script     0.8.2      Az.Reservations
Script     0.0        chocolateyProfile
Manifest   3.1.0.0    Microsoft.PowerShell.Management
Manifest   3.0.0.0    Microsoft.PowerShell.Security
Manifest   3.1.0.0    Microsoft.PowerShell.Utility
Manifest   3.0.0.0    Microsoft.WSMan.Management
Script     0.7.3.1    posh-git
Script     1.2        PSReadline

Debug output

DEBUG: 18:48:39 - GetCatalog begin processing with ParameterSet '__AllParameterSets'.
DEBUG: 18:48:39 - using account id 'REDACTED-LOGIN@REDACTED-COMPANY.com'...
DEBUG: [Common.Authentication]: Authenticating using Account: 'REDACTED-LOGIN@REDACTED-COMPANY.com', environment: 'AzureCloud', tenant:
'REDACTED-TENANTID'
DEBUG: [Common.Authentication]: Authenticating using configuration values: Domain: 'REDACTED-TENANTID', Endpoint:
'https://login.microsoftonline.com/', ClientId: '1950a258-227b-4e31-a9cf-717495945fc2', ClientRedirect: 'urn:ietf:wg:oauth:2.0:oob', ResourceClientUri:
'https://management.core.windows.net/', ValidateAuthority: 'True'
DEBUG: [Common.Authentication]: Acquiring token using context with Authority 'https://login.microsoftonline.com/REDACTED-TENANTID/',
CorrelationId: '00000000-0000-0000-0000-000000000000', ValidateAuthority: 'True'
DEBUG: [Common.Authentication]: Acquiring token using AdalConfiguration with Domain: 'REDACTED-TENANTID', AdEndpoint:
'https://login.microsoftonline.com/', ClientId: '1950a258-227b-4e31-a9cf-717495945fc2', ClientRedirectUri: urn:ietf:wg:oauth:2.0:oob
DEBUG: [ADAL]: Information: 2020-06-10T16:48:39.6473272Z: f079fe60-608c-4c4f-ba14-8f66ed322117 - LoggerBase.cs: ADAL PCL.Desktop with assembly version
'3.19.2.6005', file version '3.19.50302.0130' and informational version '2a8bec6c4c76d0c1ef819b55bdc3cda2d2605056' is running...
DEBUG: [ADAL]: Information: 2020-06-10T16:48:39.6473272Z: f079fe60-608c-4c4f-ba14-8f66ed322117 - LoggerBase.cs: ADAL PCL.Desktop with assembly version
'3.19.2.6005', file version '3.19.50302.0130' and informational version '2a8bec6c4c76d0c1ef819b55bdc3cda2d2605056' is running...
DEBUG: [ADAL]: Information: 2020-06-10T16:48:39.6473272Z: f079fe60-608c-4c4f-ba14-8f66ed322117 - LoggerBase.cs: === Token Acquisition started:
 CacheType: null
 Authentication Target: User
 , Authority Host: login.microsoftonline.com
DEBUG: [ADAL]: Information: 2020-06-10T16:48:39.6482149Z: f079fe60-608c-4c4f-ba14-8f66ed322117 - LoggerBase.cs: === Token Acquisition started:
 Authority: https://login.microsoftonline.com/REDACTED-TENANTID/
 Resource: https://management.core.windows.net/
 ClientId: 1950a258-227b-4e31-a9cf-717495945fc2
 CacheType: null
 Authentication Target: User

DEBUG: [ADAL]: Verbose: 2020-06-10T16:48:39.6482149Z: f079fe60-608c-4c4f-ba14-8f66ed322117 - LoggerBase.cs: Loading from cache.
DEBUG: [ADAL]: Verbose: 2020-06-10T16:48:39.6482149Z: f079fe60-608c-4c4f-ba14-8f66ed322117 - LoggerBase.cs: Loading from cache.
DEBUG: [ADAL]: Information: 2020-06-10T16:48:39.6512106Z: 00000000-0000-0000-0000-000000000000 - LoggerBase.cs: Deserialized 13 items to token cache.
DEBUG: [ADAL]: Verbose: 2020-06-10T16:48:39.6512106Z: f079fe60-608c-4c4f-ba14-8f66ed322117 - LoggerBase.cs: Looking up cache for a token...
DEBUG: [ADAL]: Verbose: 2020-06-10T16:48:39.6512106Z: f079fe60-608c-4c4f-ba14-8f66ed322117 - LoggerBase.cs: Looking up cache for a token...
DEBUG: [ADAL]: Information: 2020-06-10T16:48:39.6512106Z: f079fe60-608c-4c4f-ba14-8f66ed322117 - LoggerBase.cs: An item matching the requested resource was
found in the cache
DEBUG: [ADAL]: Information: 2020-06-10T16:48:39.6512106Z: f079fe60-608c-4c4f-ba14-8f66ed322117 - LoggerBase.cs: An item matching the requested resource was
found in the cache
DEBUG: [ADAL]: Information: 2020-06-10T16:48:39.6512106Z: f079fe60-608c-4c4f-ba14-8f66ed322117 - LoggerBase.cs: 48,8039298233333 minutes left until token in
cache expires
DEBUG: [ADAL]: Information: 2020-06-10T16:48:39.6512106Z: f079fe60-608c-4c4f-ba14-8f66ed322117 - LoggerBase.cs: 48,8039298233333 minutes left until token in
cache expires
DEBUG: [ADAL]: Information: 2020-06-10T16:48:39.6512106Z: f079fe60-608c-4c4f-ba14-8f66ed322117 - LoggerBase.cs: A matching item (access token or refresh token
or both) was found in the cache
DEBUG: [ADAL]: Information: 2020-06-10T16:48:39.6512106Z: f079fe60-608c-4c4f-ba14-8f66ed322117 - LoggerBase.cs: A matching item (access token or refresh token
or both) was found in the cache
DEBUG: [ADAL]: Information: 2020-06-10T16:48:39.6522104Z: f079fe60-608c-4c4f-ba14-8f66ed322117 - LoggerBase.cs: === Token Acquisition finished successfully. An
 access token was returned: Expiration Time: 2020-06-10 17:37:27 +00:00
DEBUG: [ADAL]: Information: 2020-06-10T16:48:39.6522104Z: f079fe60-608c-4c4f-ba14-8f66ed322117 - LoggerBase.cs: === Token Acquisition finished successfully. An
 access token was returned: Expiration Time: 2020-06-10 17:37:27 +00:00Access Token Hash: pIubdTZcfYNxLA+40cr5w2pazLhT1SSg6fiVzynEUUs=
  User id: REDACTED-USERID
DEBUG: [Common.Authentication]: Renewing Token with Type: 'Bearer', Expiry: '06/10/2020 17:37:27 +00:00', MultipleResource? 'True', Tenant:
'REDACTED-TENANTID', UserId: 'REDACTED-LOGIN@REDACTED-COMPANY.com'
DEBUG: [Common.Authentication]: User info for token DisplayId: 'REDACTED-LOGIN@REDACTED-COMPANY.com', Name:
REDACTED-LOGIN@REDACTED-COMPANY.com Account Admin, IdProvider: 'https://sts.windows.net/5addfe4b-05fa-4f67-9da0-7e4cb9842e1d/', Uid:
'REDACTED-USERID'
DEBUG: [Common.Authentication]: Checking token expiration, token expires '06/10/2020 17:37:27 +00:00' Comparing to '06/10/2020 16:48:39 +00:00' With threshold
'00:05:00', calculated time until token expiry: '00:48:48.2347896'
DEBUG: ============================ HTTP REQUEST ============================

HTTP Method:
GET

Absolute Uri:
https://management.azure.com/subscriptions/7dd06553-2984-4967-a430-33b5f3a0bf67/providers/Microsoft.Capacity/catalogs?api-version=2019-04-01&reservedResourceTy
pe=VirtualMachines&location=EastUS

Headers:
x-ms-client-request-id        : 673c7da3-b643-4b08-9e7c-7fdcfd0a4f78
accept-language               : en-US

Body:


DEBUG: ============================ HTTP RESPONSE ============================

Status Code:
BadRequest

Headers:
Pragma                        : no-cache
x-ms-request-id               : northeurope:21e97072-a4a7-4a6c-892a-5f6b9b5a726b
X-Content-Type-Options        : nosniff
x-ms-ratelimit-remaining-subscription-reads: 11999
x-ms-correlation-request-id   : 3be4bc61-ca74-43c6-abf1-c9a8b886cae3
x-ms-routing-request-id       : WESTEUROPE:20200610T164840Z:3be4bc61-ca74-43c6-abf1-c9a8b886cae3
Strict-Transport-Security     : max-age=31536000; includeSubDomains
Cache-Control                 : no-cache
Date                          : Wed, 10 Jun 2020 16:48:39 GMT
Server                        : Microsoft-IIS/10.0
X-AspNet-Version              : 4.0.30319

Body:
{
  "error": {
    "code": "SubscriptionNotRegistered",
    "message": "Subscription is still pending registration or not registered."
  }
}

Get-AzReservationCatalog : Operation returned an invalid status code 'BadRequest'
At line:1 char:1
+ Get-AzReservationCatalog -ReservedResourceType VirtualMachines -Locat ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : CloseError: (:) [Get-AzReservationCatalog], ErrorException
    + FullyQualifiedErrorId : Microsoft.Azure.Commands.Reservations.Cmdlets.GetCatalog

DEBUG: AzureQoSEvent: CommandName - Get-AzReservationCatalog; IsSuccess - False; Duration - 00:00:00.9747140;; Exception -
Microsoft.Azure.Management.Reservations.Models.ErrorException: Operation returned an invalid status code 'BadRequest'
   at Microsoft.Azure.Management.Reservations.AzureReservationAPIClient.<GetCatalogWithHttpMessagesAsync>d__56.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Microsoft.Azure.Management.Reservations.AzureReservationAPIClientExtensions.<GetCatalogAsync>d__1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at Microsoft.Azure.Management.Reservations.AzureReservationAPIClientExtensions.GetCatalog(IAzureReservationAPIClient operations, String subscriptionId,
String reservedResourceType, String location)
   at Microsoft.Azure.Commands.Reservations.Cmdlets.GetCatalog.ExecuteCmdlet()
   at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord();
DEBUG: Finish sending metric.
DEBUG: 18:48:41 - GetCatalog end processing.

Error output

DEBUG: 18:49:16 - ResolveError begin processing with ParameterSet 'AnyErrorParameterSet'.
DEBUG: 18:49:16 - using account id 'REDACTED-LOGIN@REDACTED-COMPANY.com'...
WARNING: Breaking changes in the cmdlet 'Resolve-AzError' :
WARNING:  - The `Resolve-Error` alias will be removed in a future release.  Please change any scripts that use this alias to use `Resolve-AzError` instead.

WARNING: NOTE : Go to https://aka.ms/azps-changewarnings for steps to suppress this breaking change warning, and other information on breaking changes in Azure
 PowerShell.


   HistoryId: 15


Message        : Operation returned an invalid status code 'BadRequest'
StackTrace     :    at Microsoft.Azure.Management.Reservations.AzureReservationAPIClient.<GetCatalogWithHttpMessagesAsync>d__56.MoveNext()
                 --- End of stack trace from previous location where exception was thrown ---
                    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
                    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
                    at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
                    at Microsoft.Azure.Management.Reservations.AzureReservationAPIClientExtensions.<GetCatalogAsync>d__1.MoveNext()
                 --- End of stack trace from previous location where exception was thrown ---
                    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
                    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
                    at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
                    at Microsoft.Azure.Management.Reservations.AzureReservationAPIClientExtensions.GetCatalog(IAzureReservationAPIClient operations, String sub
                 scriptionId, String reservedResourceType, String location)
                    at Microsoft.Azure.Commands.Reservations.Cmdlets.GetCatalog.ExecuteCmdlet()
                    at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord()
Exception      : Microsoft.Azure.Management.Reservations.Models.ErrorException
InvocationInfo : {Get-AzReservationCatalog}
Line           : Get-AzReservationCatalog -ReservedResourceType VirtualMachines -Location EastUS
Position       : At line:1 char:1
                 + Get-AzReservationCatalog -ReservedResourceType VirtualMachines -Locat ...
                 + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
HistoryId      : 15



   HistoryId: 10


Message        : Operation returned an invalid status code 'BadRequest'
StackTrace     :    at Microsoft.Azure.Management.Reservations.AzureReservationAPIClient.<GetCatalogWithHttpMessagesAsync>d__56.MoveNext()
                 --- End of stack trace from previous location where exception was thrown ---
                    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
                    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
                    at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
                    at Microsoft.Azure.Management.Reservations.AzureReservationAPIClientExtensions.<GetCatalogAsync>d__1.MoveNext()
                 --- End of stack trace from previous location where exception was thrown ---
                    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
                    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
                    at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
                    at Microsoft.Azure.Management.Reservations.AzureReservationAPIClientExtensions.GetCatalog(IAzureReservationAPIClient operations, String sub
                 scriptionId, String reservedResourceType, String location)
                    at Microsoft.Azure.Commands.Reservations.Cmdlets.GetCatalog.ExecuteCmdlet()
                    at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord()
Exception      : Microsoft.Azure.Management.Reservations.Models.ErrorException
InvocationInfo : {Get-AzReservationCatalog}
Line           : Get-AzReservationCatalog -ReservedResourceType VirtualMachines -Location 'East US'
Position       : At line:1 char:1
                 + Get-AzReservationCatalog -ReservedResourceType VirtualMachines -Locat ...
                 + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
HistoryId      : 10



   HistoryId: 8


Message        : Operation returned an invalid status code 'BadRequest'
StackTrace     :    at Microsoft.Azure.Management.Reservations.AzureReservationAPIClient.<GetCatalogWithHttpMessagesAsync>d__56.MoveNext()
                 --- End of stack trace from previous location where exception was thrown ---
                    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
                    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
                    at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
                    at Microsoft.Azure.Management.Reservations.AzureReservationAPIClientExtensions.<GetCatalogAsync>d__1.MoveNext()
                 --- End of stack trace from previous location where exception was thrown ---
                    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
                    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
                    at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
                    at Microsoft.Azure.Management.Reservations.AzureReservationAPIClientExtensions.GetCatalog(IAzureReservationAPIClient operations, String sub
                 scriptionId, String reservedResourceType, String location)
                    at Microsoft.Azure.Commands.Reservations.Cmdlets.GetCatalog.ExecuteCmdlet()
                    at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord()
Exception      : Microsoft.Azure.Management.Reservations.Models.ErrorException
InvocationInfo : {Get-AzReservationCatalog}
Line           : Get-AzReservationCatalog -ReservedResourceType VirtualMachines -Location 'East US'
Position       : At line:1 char:1
                 + Get-AzReservationCatalog -ReservedResourceType VirtualMachines -Locat ...
                 + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
HistoryId      : 8


The Azure PowerShell team is listening, please let us know how we are doing: https://aka.ms/azpssurvey?Q_CHL=ERROR.

DEBUG: AzureQoSEvent: CommandName - Resolve-AzError; IsSuccess - True; Duration - 00:00:00.2146119;
DEBUG: Finish sending metric.
DEBUG: 18:49:17 - ResolveError end processing.

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @corquiri.

[dingmeng-xue]

I include service team to look into this issue. @corquiri , if there is a general problem from customer, we prefer to updating online help and help user to know what is missed. Please let us know your suggestion.

[sdwire]

I'm seeing the same symptoms in a slightly different environment:

> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.0.3
PSEdition                      Core
GitCommitId                    7.0.3
OS                             Microsoft Windows 10.0.19041
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0


> Get-Module

ModuleType Version    PreRelease Name                                ExportedCommands
---------- -------    ---------- ----                                ----------------
Script     1.7.3                 Az.Accounts                         {Add-AzEnvironment, Clear-AzContext, Clear-AzDefault, Connect-AzAccount…}
Script     3.5.0                 Az.Compute                          {Add-AzContainerServiceAgentPoolProfile, Add-AzImageDataDisk, Add-AzVhd, Add-AzVMAdditionalUnattendContent…}
Script     0.8.2                 Az.Reservations                     {Get-AzReservation, Get-AzReservationCatalog, Get-AzReservationHistory, Get-AzReservationOrder…}
Script     1.12.0                Az.Resources                        {Add-AzADGroupMember, Export-AzResourceGroup, Get-AzADAppCredential, Get-AzADApplication…}

[sdwire]

I don't know if it's related or not, but after doing a Clear-AzContext and Connect-AzAccount this morning, this 'SubscriptionNotRegistered' issue went away for me. Hard to think that would fix it, but I don't know what else might have changed for me to start making it work.

[corquiri]

You need to register your subscription with Microsoft.Capacity in order to see the catalog.
Please use the command Register-AzResourceProvider
Doc link : https://docs.microsoft.com/en-us/powershell/module/az.resources/Register-AzResourceProvider?view=azps-4.4.0

[corquiri]

Could you also try registering Microsoft.Compute with the subscription?

[jberezanski-mdg]

Get-AzReservationCatalog now works as expected (returns a list of reservation options) for both my subscriptions, but I have not registered the Microsoft.Capacity provider explicitly. I suppose it was automatically registered when I purchased some reservations via the Azure Portal (after being unable to do it via PowerShell).

So, as far as I am concerned, the issue can be closed. I would suggest, however, to include information about needing to register the provider in the documentation on reservations, for example on the page describing the REST API (https://docs.microsoft.com/en-us/azure/cost-management-billing/reservations/reservation-apis).

[sdwire]

I also found that when I call Register-AzResourceProvider, I can then look at it using Get-AzResourceProvider, and its status will remain Registering for several seconds, even up to a minute, before it finally turns to Registered.

That is expected.

What is not expected, however, is that even after the status turns to Registered, calls to Get-AzReservationCatalog will continue to fail for up to a few minutes with the same error as above. Eventually, though, the calls to Get-AzReservationCatalog will succeed, but there's no way to tell when the registration is truly finished except to watch for Get-AzReservationCatalog to suddenly start succeeding.

[rkapso]

This is a known limitation currently, the need to register Microsoft.Capacity and the lag from the registered call and when it starts to work, we have this item in the backlog to address the same.