Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proxy connectivity errors with Azure Data Studio connection to Azure Portal #24471

Closed
emhansonoh opened this issue Sep 19, 2023 · 11 comments
Closed

Comments

@emhansonoh
Copy link

  • Azure Data Studio Version: 1.45.1 (and 1.47.0-insider)
  • OS Version: Windows 10

Steps to Reproduce:

  1. Utilize a proxy server for connectivity to the internet
  2. Add your Azure account to the Azure section of the tool
  3. Attempt to browse through the inventory of SQL resources

Does this issue occur when all extensions are disabled?: Yes/No - YES

This issue is very specific to use of the tool at a large corporation which utilizes Proxy devices to access the internet. The condition appears both on Windows machines as well as Mac. And, this condition isn't limited to just this piece of the tool but is significantly more impactful.

In a large environment which utilizes proxy appliances for external connectivity, the users find the tool to be unusable for one of the primary features - connecting to their Azure tenant to provide the inventory of the many different assets (SQL Database / SQL Managed Instance). The initial add of the account is done and may take a retry or 2 - receiving 407 errors along the way until it's succesful. The inventory is then displayed in the Azure section of the tool. As you drill down to the resources at almost every layer a 407 will happen requiring several refresh attempts to be made so it works.

If the tool is not used for a day or 2, the credentials need to be refreshed and often will get an error saying there are multiple files in the \AppData\Roaming\azuredatastudio\AzureAccounts folder. Those end up needing to be deleted, the account removed within Azure Data Studio and then start the process over.

Example -

Your credentials could not be authenticated: "Credentials are missing.". You will not be permitted access until your credentials can be verified.
This is typically caused by an incorrect username and/or password, but could also be caused by network problems.

[Error]: Failed to acquireTokenSilent - [{"errorCode":"client_error","errorMessage":"undefined - [undefined]: A client error occured.\nHttp status code: 407\nHttp status message: Proxy Authentication Required\nHeaders: {"proxy-authenticate":"NEGOTIATE, NTLM, BASIC realm=\"Prod\"","cache-control":"no-cache","x-xss-protection":"1","connection":"close","content-type":"text/html; charset=utf-8","content-length":"1120","pragma":"no-cache"} - Correlation ID: undefined - Trace ID: undefined","subError":"","name":"ServerError"}]
[Error]: MSAL: getToken call failed: [object Object] - []
[Error]: Error: client_error occurred when acquiring token.
undefined - [undefined]: A client error occured.
Http status code: 407
Http status message: Proxy Authentication Required
Headers: {"proxy-authenticate":"NEGOTIATE, NTLM, BASIC realm="Prod"","cache-control":"no-cache","x-xss-protection":"1","connection":"close","content-type":"text/html; charset=utf-8","content-length":"1120","pragma":"no-cache"} - Correlation ID: undefined - Trace ID: undefined - []

@cheenamalhotra
Copy link
Member

Hi @emhansonoh

The initial add of the account is done and may take a retry or 2 - receiving 407 errors along the way until it's succesful. The inventory is then displayed in the Azure section of the tool. As you drill down to the resources at almost every layer a 407 will happen requiring several refresh attempts to be made so it works.

We have not heard of the 407 errors from other customers who also run ADS regularly in proxy-enabled environments so it may be related with lifetime of proxy authentication tokens.

Has customer tried providing username/password combination in http.proxy setting in ADS in format described here: https://learn.microsoft.com/en-us/azure-data-studio/azure-connectivity?view=sql-server-ver16#supported-environment-variables-for-proxy

Also please request them to try with the latest insiders build as we've brought new changes from VS Code which may improve user experience.

@emhansonoh
Copy link
Author

emhansonoh commented Oct 26, 2023 via email

@cheenamalhotra
Copy link
Member

Steps to repro - Add azure account --> browser is opened to log in to the Microsoft account --> the validation page never loads / never allows connectivity.

We've identified the issue that contributed to this (#24839), and fix is underway. I'll update again when fix is available in insiders so we can continue testing for 407 errors.

@emhansonoh
Copy link
Author

That sounds good. Thank you. I did look at the link you had provided. I can try this solution to see if the behavior is any better / different. But - we will not be providing our username / password in clear text in a config file at volume. If this was a couple of people temporarily getting around an issue - that would be a possibility. But, in a large environment this will not be a workable solution.

@cheenamalhotra
Copy link
Member

This primarily depends on VS Code's missing proxy support for extensions: ref. microsoft/vscode#12588
Until then the workaround is to provide credentials in the URL which I agree is not an ideal scenario.

@cheenamalhotra
Copy link
Member

The latest insiders build is out that contains the fix to this issue: #24839
And user:pass combinarion should help resolve 407 errors in the meantime. I will continue to follow up with VS Code teams on any alternative solutions till microsoft/vscode#12588 is addressed.

@emhansonoh
Copy link
Author

So, partially working, partially not. I have the insider build - 1.48 / 4 days old. I'm able to connect / browse through our subscriptions. I still get periodic proxy errors (BlueCoat timeouts) but refresh and it's good. It does seem to be more response than it had been, but - still there periodically.

Likely unrelated - I am not able to connect to our managed instances still. That appeared with that other bug which was fixed and allowed the Azure piece to work. Whenever we try to connect to any of our SQL resources using Microsoft Entra ID - Universal with MFA support - we get "User credentials received in invalid format"

I'll keep trying things to see if something sticks out but looks like that will still be problematic. And - it does start off with "Proxy" so that is a bit concerning. More to come though if I find something.

Microsoft.Data.SqlClient.SqlException (0x80131904): User credentials received in invalid format.
---> System.Exception: User credentials received in invalid format.
at Microsoft.SqlTools.Authentication.Utility.HttpClientProxyFactory..ctor(String proxyUrl, Boolean proxyStrictSSL) in //src/Microsoft.SqlTools.Authentication/Utility/ProxyHttpClientFactory.cs:line 39
at Microsoft.SqlTools.Authentication.Authenticator.CreatePublicClientAppInstance(String authority, String audience) in /
/src/Microsoft.SqlTools.Authentication/Authenticator.cs:line 163
at Microsoft.SqlTools.Authentication.Authenticator.GetPublicClientAppInstance(String authority, String audience) in //src/Microsoft.SqlTools.Authentication/Authenticator.cs:line 155
at Microsoft.SqlTools.Authentication.Authenticator.GetTokenAsync(AuthenticationParams params, CancellationToken cancellationToken) in /
/src/Microsoft.SqlTools.Authentication/Authenticator.cs:line 52
at Microsoft.SqlTools.Authentication.Sql.AuthenticationProvider.AcquireTokenAsync(SqlAuthenticationParameters parameters) in //src/Microsoft.SqlTools.Authentication/Sql/AuthenticationProvider.cs:line 90
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.<>c__DisplayClass147_1.<b__1>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.GetFedAuthToken(SqlFedAuthInfo fedAuthInfo)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.GetFedAuthToken(SqlFedAuthInfo fedAuthInfo)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.OnFedAuthInfo(SqlFedAuthInfo fedAuthInfo)
at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
at Microsoft.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.CompleteLogin(Boolean enlistOK)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean ignoreSniOpenTimeout, TimeoutTimer timeout, Boolean withFailover)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString connectionOptions, SqlCredential credential, TimeoutTimer timeout)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(TimeoutTimer timeout, SqlConnectionString connectionOptions, SqlCredential credential, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, SqlCredential credential, Object providerInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString userConnectionOptions, SessionData reconnectSessionData, Boolean applyTransientFaultHandling, String accessToken, DbConnectionPool pool)
at Microsoft.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions)
at Microsoft.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnectionPool pool, DbConnection owningObject, DbConnectionOptions options, DbConnectionPoolKey poolKey, DbConnectionOptions userOptions)
at Microsoft.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection)
at Microsoft.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection)
at Microsoft.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)
at Microsoft.Data.ProviderBase.DbConnectionPool.WaitForPendingOpen()
--- End of stack trace from previous location ---
at Microsoft.Data.SqlClient.SqlRetryLogicProvider.ExecuteAsync(Object sender, Func1 function, CancellationToken cancellationToken) at Microsoft.Data.SqlClient.SqlRetryLogicProvider.ExecuteAsync(Object sender, Func1 function, CancellationToken cancellationToken)
at Microsoft.SqlTools.ServiceLayer.Connection.ReliableConnection.ReliableSqlConnection.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /
/src/Microsoft.SqlTools.ManagedBatchParser/ReliableConnection/ReliableSqlConnection.cs:line 319
--- End of stack trace from previous location ---
at Microsoft.SqlTools.ServiceLayer.Connection.ConnectionService.TryOpenConnection(ConnectionInfo connectionInfo, ConnectParams connectionParams) in /_/src/Microsoft.SqlTools.ServiceLayer/Connection/ConnectionService.cs:line 705
ClientConnectionId:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

@emhansonoh
Copy link
Author

Received the update yesterday on the release product - 1.47.0 / 2023-11-07. This appears to have the new connectivity in it as the driver is now labeled Microsoft Entra ID. The Azure window seems to behave / able to browse through. There do not appear to be as frequent of issues requiring retry as we had previously seen. But, this has introduced a new issue which has resulted in the tool being unusable. Unknown if this is proxy related or if this is a new / different issue. Please advise if a new bug should be logged specific to this issue.

Microsoft.Data.SqlClient.SqlException (0x80131904): The SSL connection could not be established, see inner exception.
---> System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
---> System.Security.Authentication.AuthenticationException: The remote certificate was rejected by the provided RemoteCertificateValidationCallback.
at System.Net.Security.SslStream.CompleteHandshake(SslAuthenticationOptions sslAuthenticationOptions)
at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](Boolean receiveFirst, Byte[] reAuthenticationData, CancellationToken cancellationToken)
at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem)
at System.Threading.Tasks.TaskCompletionSourceWithCancellation1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.HttpConnectionWaiter1.WaitForConnectionAsync(Boolean async, CancellationToken requestCancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
at Microsoft.Identity.Client.Http.HttpManager.ExecuteAsync(Uri endpoint, IDictionary2 headers, HttpContent body, HttpMethod method, ILoggerAdapter logger, CancellationToken cancellationToken) at Microsoft.Identity.Client.Http.HttpManagerWithRetry.SendRequestAsync(Uri endpoint, IDictionary2 headers, HttpContent body, HttpMethod method, ILoggerAdapter logger, Boolean doNotThrow, Boolean retry, CancellationToken cancellationToken)
at Microsoft.Identity.Client.Http.HttpManagerWithRetry.SendGetAsync(Uri endpoint, IDictionary2 headers, ILoggerAdapter logger, Boolean retry, CancellationToken cancellationToken) at Microsoft.Identity.Client.OAuth2.OAuth2Client.ExecuteRequestAsync[T](Uri endPoint, HttpMethod method, RequestContext requestContext, Boolean expectErrorsOn200OK, Boolean addCommonHeaders, Func2 onBeforePostRequestData)
at Microsoft.Identity.Client.OAuth2.OAuth2Client.DiscoverAadInstanceAsync(Uri endpoint, RequestContext requestContext)
at Microsoft.Identity.Client.Instance.Discovery.NetworkMetadataProvider.SendInstanceDiscoveryRequestAsync(Uri authority, RequestContext requestContext)
at Microsoft.Identity.Client.Instance.Discovery.NetworkMetadataProvider.FetchAllDiscoveryMetadataAsync(Uri authority, RequestContext requestContext)
at Microsoft.Identity.Client.Instance.Discovery.NetworkMetadataProvider.GetMetadataAsync(Uri authority, RequestContext requestContext)
at Microsoft.Identity.Client.Instance.Discovery.InstanceDiscoveryManager.FetchNetworkMetadataOrFallbackAsync(RequestContext requestContext, Uri authorityUri)
at Microsoft.Identity.Client.Instance.Discovery.InstanceDiscoveryManager.GetMetadataEntryAsync(AuthorityInfo authorityInfo, RequestContext requestContext, Boolean forceValidation)
at Microsoft.Identity.Client.Instance.AuthorityManager.RunInstanceDiscoveryAndValidationAsync()
at Microsoft.Identity.Client.Internal.SilentRequestHelper.RefreshAccessTokenAsync(MsalRefreshTokenCacheItem msalRefreshTokenItem, RequestBase request, AuthenticationRequestParameters authenticationRequestParameters, CancellationToken cancellationToken)
at Microsoft.Identity.Client.Internal.Requests.Silent.CacheSilentStrategy.RefreshRtOrFailAsync(CancellationToken cancellationToken)
at Microsoft.Identity.Client.Internal.Requests.Silent.CacheSilentStrategy.ExecuteAsync(CancellationToken cancellationToken)
at Microsoft.Identity.Client.Internal.Requests.Silent.SilentRequest.ExecuteAsync(CancellationToken cancellationToken)
at Microsoft.Identity.Client.Internal.Requests.RequestBase.RunAsync(CancellationToken cancellationToken)
at Microsoft.Identity.Client.ApiConfig.Executors.ClientApplicationBaseExecutor.ExecuteAsync(AcquireTokenCommonParameters commonParameters, AcquireTokenSilentParameters silentParameters, CancellationToken cancellationToken)
at Microsoft.SqlTools.Authentication.Authenticator.GetTokenAsync(AuthenticationParams params, CancellationToken cancellationToken) in //src/Microsoft.SqlTools.Authentication/Authenticator.cs:line 108
at Microsoft.SqlTools.Authentication.Sql.AuthenticationProvider.AcquireTokenAsync(SqlAuthenticationParameters parameters) in /
/src/Microsoft.SqlTools.Authentication/Sql/AuthenticationProvider.cs:line 90
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.<>c__DisplayClass147_1.<b__1>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.GetFedAuthToken(SqlFedAuthInfo fedAuthInfo)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.GetFedAuthToken(SqlFedAuthInfo fedAuthInfo)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.OnFedAuthInfo(SqlFedAuthInfo fedAuthInfo)
at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
at Microsoft.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.CompleteLogin(Boolean enlistOK)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean ignoreSniOpenTimeout, TimeoutTimer timeout, Boolean withFailover)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString connectionOptions, SqlCredential credential, TimeoutTimer timeout)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(TimeoutTimer timeout, SqlConnectionString connectionOptions, SqlCredential credential, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, SqlCredential credential, Object providerInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString userConnectionOptions, SessionData reconnectSessionData, Boolean applyTransientFaultHandling, String accessToken, DbConnectionPool pool)
at Microsoft.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions)
at Microsoft.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnectionPool pool, DbConnection owningObject, DbConnectionOptions options, DbConnectionPoolKey poolKey, DbConnectionOptions userOptions)
at Microsoft.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection)
at Microsoft.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection)
at Microsoft.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)
at Microsoft.Data.ProviderBase.DbConnectionPool.WaitForPendingOpen()
--- End of stack trace from previous location ---
at Microsoft.Data.SqlClient.SqlRetryLogicProvider.ExecuteAsync(Object sender, Func1 function, CancellationToken cancellationToken) at Microsoft.Data.SqlClient.SqlRetryLogicProvider.ExecuteAsync(Object sender, Func1 function, CancellationToken cancellationToken)
at Microsoft.SqlTools.ServiceLayer.Connection.ReliableConnection.ReliableSqlConnection.<>c__DisplayClass30_0.<b__0>d.MoveNext() in //src/Microsoft.SqlTools.ManagedBatchParser/ReliableConnection/ReliableSqlConnection.cs:line 319
--- End of stack trace from previous location ---
at Microsoft.SqlTools.ServiceLayer.Connection.ConnectionService.TryOpenConnection(ConnectionInfo connectionInfo, ConnectParams connectionParams) in /
/src/Microsoft.SqlTools.ServiceLayer/Connection/ConnectionService.cs:line 705

@cheenamalhotra
Copy link
Member

Hi @emhansonoh

The fixes we're making for proxy environments will arrive in Insiders build.

You could try disabling the setting MSSQL: Enable Sql Authentication Provider to make this work in the stable (1.47) build for the time being.

@emhansonoh
Copy link
Author

emhansonoh commented Nov 10, 2023

Thanks @cheenamalhotra ! That disablement resolved the connectivity issue in 1.47. Much appreciated!

@kisantia
Copy link
Contributor

closing as connectivity issue confirmed resolved above

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Development

No branches or pull requests

4 participants