Skip to content

Get-DatabricksSCIMUser not able to handle large volume outputs #41

@PeteAds

Description

@PeteAds

I have a workspace with 685 users in it and the call to Get-DatabricksSCIMUser always times out. Below error message is what you get. If I rewrite the code to simply use invoke-RestMethod same problem as expected. But using native curl from the same host does work meaning I suspect it's some powershell internals and not the API itself

From the error message and googled articles, have tried setting the below to no avail.
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls -bor [Net.SecurityProtocolType]::Tls11 -bor [Net.SecurityProtocolType]::Tls12

Fix : The SCIM Preview API supports pagination so I had to use pagination to workaround the issue. I set pagination at 300 successfully but other mileage may vary.

Error:

PS P:> Set-DatabricksEnvironment -ApiRootUrl $apiUrl -AccessToken dapiXXXX

PS P:> $users = Get-DatabricksSCIMUser
Invoke-RestMethod : The underlying connection was closed: The connection was closed unexpectedly.
At C:\Program Files\WindowsPowerShell\Modules\DatabricksPS\1.9.0.0\Public\General.ps1:85 char:13

  • ... $result = Invoke-RestMethod -Uri $apiUrl -Method $Method -Headers $ ...
  •             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    • CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
    • FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions