Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WaitADDomain: fails if DC not ready #530

Closed
ghost opened this issue Nov 19, 2019 · 11 comments · Fixed by #556
Closed

WaitADDomain: fails if DC not ready #530

ghost opened this issue Nov 19, 2019 · 11 comments · Fixed by #556
Assignees
Labels
bug The issue is a bug.

Comments

@ghost
Copy link

ghost commented Nov 19, 2019

WaitADDomain: fails if DC not ready

Verbose logs showing the problem

[EC2AMAZ-0J01R66]: LCM:  [ Start  Resource ]  [[WaitForADDomain]ForestWait]
[EC2AMAZ-0J01R66]: LCM:  [ Start  Test     ]  [[WaitForADDomain]ForestWait]
[EC2AMAZ-0J01R66]:                            [[WaitForADDomain]ForestWait] Determining the current state of the 
Active Directory domain 'tf-tf.local'. (WFADD0013)
[EC2AMAZ-0J01R66]:                            [[WaitForADDomain]ForestWait] Searching for a domain controller in the 
domain 'tf-tf.local'. (WFADD0001)
[EC2AMAZ-0J01R66]:                            [[WaitForADDomain]ForestWait] Impersonating the credentials 
'tf-tf\xxxxxxxxx' when looking for a domain controller. (WFADD0011)
[EC2AMAZ-0J01R66]:                            [[WaitForADDomain]ForestWait] Searching for a domain controller in the 
domain 'tf-tf.local'. (ADCOMMON0052)
[EC2AMAZ-0J01R66]:                            [[WaitForADDomain]ForestWait] The type 
'System.DirectoryServices.ActiveDirectory.DirectoryContext' is already loaded into the PowerShell session. 
(ADCOMMON0043)
[EC2AMAZ-0J01R66]:                            [[WaitForADDomain]ForestWait] Get a new Active Directory context of the 
type 'Domain'. (ADCOMMON0046)
[EC2AMAZ-0J01R66]:                            [[WaitForADDomain]ForestWait] The Active Directory context will target 
'tf-tf.local'. (ADCOMMON0047)
[EC2AMAZ-0J01R66]:                            [[WaitForADDomain]ForestWait] The Active Directory context will be 
accessed using the 'tf-tf\xxxxxxxxx' credentials. (ADCOMMON0048)
[EC2AMAZ-0J01R66]: LCM:  [ End    Test     ]  [[WaitForADDomain]ForestWait]  in 4.8180 seconds.
PowerShell DSC resource MSFT_WaitForADDomain  failed to execute Test-TargetResource functionality with error message: 
Exception calling "FindOne" with "1" argument(s): "Domain controller not found in the domain "tf-tf.local"." 
    + CategoryInfo          : InvalidOperation: (:) [], CimException
    + FullyQualifiedErrorId : ProviderOperationExecutionFailure
    + PSComputerName        : localhost
 

Suggested solution to the issue

Wait until the DC is ready

The DSC configuration that is used to reproduce the issue (as detailed as possible)

I create new server and wait for DC to become ready to join the domain. If the DC is ready, server joins the domain without problem, but if WaitForADDomain runs when DC is being configured, the resource fails. During the next consistency check the server joins the domain as the DC is ready by that time (also proving the configuration is hopefully correct).

        WaitForADDomain ForestWait
        {
            DomainName = $InternalDomain
            WaitTimeout = 900
            Credential = $default_creds
            DependsOn = "[DNSServerAddress]DNS"
            WaitForValidCredentials = $true
        }

The operating system the target node is running

OsName               : Microsoft Windows Server 2019 Datacenter
OsOperatingSystemSKU : DatacenterServerEdition
OsArchitecture       : 64-bit
WindowsVersion       : 1809
WindowsBuildLabEx    : 17763.1.amd64fre.rs5_release.180914-1434
OsLanguage           : en-US
OsMuiLanguages       : {en-US}

Version and build of PowerShell the target node is running

Name                           Value
----                           -----
PSVersion                      5.1.17763.771
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.17763.771
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

Version of the DSC module that was used ('dev' if using current dev branch)

4.2.0.0

@X-Guardian
Copy link
Contributor

Hi @Crusad, looking at the Find-DomainController function, it looks like the FindOne() method call in Find-DomainControllerFindOneWrapper is returning an unexpected exception. To see what the exception is, can you add the following line to Modules/ActiveDirectoryDsc.Common/ActiveDirectoryDsc.Common.psm1 before line 2260 (throw $_) to output the exception name to the logs and then run the Dsc again.

Write-Verbose -Message "Unknown Exception: $($_.Exception.GetType().FullName)" -Verbose

@johlju
Copy link
Member

johlju commented Nov 21, 2019

Yes I agree, it looks like another error makes the resource not wait the 900 seconds (the timeout value) as it should 🤔

@johlju johlju added bug The issue is a bug. help wanted The issue is up for grabs for anyone in the community. needs more information The issue needs more information from the author or the community. and removed bug The issue is a bug. help wanted The issue is up for grabs for anyone in the community. labels Nov 21, 2019
@X-Guardian
Copy link
Contributor

Yes, I see @Crusad is using Server 2019, so I'm suspicious that the exception name may have changed that the function is catching on.

@ghost
Copy link
Author

ghost commented Nov 21, 2019

I will definitely test what @X-Guardian suggested, just testing this is a bit time-consuming for me and I was too busy past two days to get into it. I will get back to you with results. And yep, I am using 2019 (1809).

@X-Guardian
Copy link
Contributor

No problem @Crusad.

@johlju, I was thinking that we should output the exception full name whenever we hit an unknown exception in any of the resources. It would make diagnosing these issues easier.

@xpn
Copy link
Contributor

xpn commented Feb 4, 2020

@X-Guardian So I've come across this issue myself when I'm attempting to create a new lab environment. This involves a server waiting for a domain controller to come online and results in the following entry being added to the "Desired State Configuration - Operational" event log:

Job {A84EF340-4741-11EA-8249-06C12448A1E2} : 
Message Exception calling "FindOne" with "1" argument(s): "Domain controller not found in the domain "lab.local"." 
HResult -2146233087 
StackTrack    at System.Management.Automation.Runspaces.PipelineBase.Invoke(IEnumerable input)
   at System.Management.Automation.PowerShell.Worker.ConstructPipelineAndDoWork(Runspace rs, Boolean performSyncInvoke)
   at System.Management.Automation.PowerShell.Worker.CreateRunspaceIfNeededAndDoWork(Runspace rsToUse, Boolean isSync)
   at System.Management.Automation.PowerShell.CoreInvokeHelper[TInput,TOutput](PSDataCollection`1 input, PSDataCollection`1 output, PSInvocationSettings settings)
   at System.Management.Automation.PowerShell.CoreInvoke[TInput,TOutput](PSDataCollection`1 input, PSDataCollection`1 output, PSInvocationSettings settings)
   at System.Management.Automation.PowerShell.Invoke(IEnumerable input, PSInvocationSettings settings)
   at Microsoft.PowerShell.DesiredStateConfiguration.Internal.ResourceProviderAdapter.ExecuteCommand(PowerShell powerShell, ResourceModuleInfo resInfo, String operationCmd, List`1 acceptedProperties, CimInstance nonResourcePropeties, CimInstance resourceConfiguration, LCMDebugMode debugMode, PSInvocationSettings pSInvocationSettings, UInt32& resultStatusHandle, Collection`1& result, ErrorRecord& errorRecord, PSModuleInfo localRunSpaceModuleInfo)

Reviewing the Test-TargetResource method, I have managed to recreate the issue with the following Powershell snippet:

$typeName = 'System.DirectoryServices.ActiveDirectory.DirectoryContext'

Add-Type -AssemblyName 'System.DirectoryServices'

$argumentList = @(
         "Domain"
         "doesnotexistyet.local"
     )

$objectParameters = @{
         TypeName     = $typeName
         ArgumentList = $argumentList
     }

$findDomain = New-Object @objectParameters

[System.DirectoryServices.ActiveDirectory.DomainController]::FindOne($findDomain)

I've also made the update to the exception handler as requested above and the following information is returned:

Unknown Exception: System.Management.Automation.MethodInvocationException

This is re-thrown from the following line where it is never handled from Test-TargetResource it seems:

Please let me know if you require any further information to help recreate this issue.

@X-Guardian
Copy link
Contributor

Hi @xpn. Thanks for the extra info. With your code snippet I can reproduce the error and see the problem. The exception thrown is of type System.Management.Automation.MethodInvocationException with an inner exception of type System.DirectoryServices.ActiveDirectory.ActiveDirectoryObjectNotFoundException. The current code only checks for a base exception of System.DirectoryServices.ActiveDirectory.ActiveDirectoryObjectNotFoundException.

Should be a simple case of adding a check to the Find-DomainController function for the ActiveDirectoryObjectNotFoundException within the current code that processes the MethodInvocationException.

I'm happy to review a PR if you want to submit one?

@X-Guardian X-Guardian added bug The issue is a bug. in progress The issue is being actively worked on by someone. and removed needs more information The issue needs more information from the author or the community. labels Feb 5, 2020
@X-Guardian
Copy link
Contributor

OK I've done some more testing on this, and I think we still have an issue with a different exception.

Expanding @xpn's code snippet with a try/catch block:

$typeName = 'System.DirectoryServices.ActiveDirectory.DirectoryContext'

Add-Type -AssemblyName 'System.DirectoryServices'

$argumentList = @(
    "Domain"
    "doesnotexistyet.local"
)

$objectParameters = @{
    TypeName     = $typeName
    ArgumentList = $argumentList
}

$findDomain = New-Object @objectParameters

try
{
    [System.DirectoryServices.ActiveDirectory.DomainController]::FindOne($findDomain)
}
catch [System.DirectoryServices.ActiveDirectory.ActiveDirectoryObjectNotFoundException]
{
    Write-Host 'Specific catch: ActiveDirectoryObjectNotFoundException'
    Write-Host 'Exception Name: ' + $_.Exception.GetType().FullName
}
catch [System.Management.Automation.MethodInvocationException]
{
    Write-Host 'Specific catch: MethodInvocationException'
    Write-Host 'Exception Name: ' + $_.Exception.GetType().FullName
    Write-Host 'Inner Exception Name: ' + $_.Exception.InnerException.GetType().FullName
} 
catch
{
    Write-Host 'Generic catch'
    Write-Host 'Exception Name: ' + $_.Exception.GetType().FullName
    Write-Host 'Inner Exception Name: ' + $_.Exception.InnerException.GetType().FullName
}

For me, both on a 2012 R2 server running PowerShell 4 and a 2019 server running PowerShell 5.1, the above code hits the ActiveDirectoryObjectNotFoundException specific catch, even though the original outer exception is of type MethodInvocationException. You can test this for yourself by running the above code with and without the two specific catches.

We must therefore be hitting a different exception.

@xpn, if you are able to recreate this consistently within a DSC run, can you add the following code just before both throw statements in the Find-DomainController function so that we can capture both the outer and inner exception names:

Write-Verbose -Message "Exception Name: $($_.Exception.GetType().FullName)" -Verbose
Write-Verbose -Message "Inner Exception Name: $($_.Exception.InnerException.GetType().FullName)" -Verbose

This will hopefully give us the information we need to fix this.

@X-Guardian X-Guardian added the needs more information The issue needs more information from the author or the community. label Feb 5, 2020
@xpn
Copy link
Contributor

xpn commented Feb 6, 2020

Hmm, yeah I can recreate this with the code example above, similar if you remove the catch for the ActiveDirectoryObjectNotFoundException from your code, you then hit the MethodInvocationExcetpion catch... It's Schrodinger exception :)

I'll spin up the lab and test with your requested code and get back to you.

@jcuzzi
Copy link

jcuzzi commented Mar 3, 2020

@X-Guardian I added verbose logging on both throw statements in the Find-DomainController function. Here is the output when using the resource:

VERBOSE: [DC2]: LCM:  [ Start  Resource ]  [[WaitForADDomain]WaitForPrimaryDC]
VERBOSE: [DC2]: LCM:  [ Start  Test     ]  [[WaitForADDomain]WaitForPrimaryDC]
VERBOSE: [DC2]:                            [[WaitForADDomain]WaitForPrimaryDC] Determining the current state of the Active Directory domain 'testdomain.com'. (WFADD0013)
VERBOSE: [DC2]:                            [[WaitForADDomain]WaitForPrimaryDC] Searching for a domain controller in the domain 'testdomain.com'. (WFADD0001)
VERBOSE: [DC2]:                            [[WaitForADDomain]WaitForPrimaryDC] Impersonating the credentials 'testdomain\Administrator' when looking for a domain controller. (WFADD0011)
VERBOSE: [DC2]:                            [[WaitForADDomain]WaitForPrimaryDC] Searching for a domain controller in the domain 'testdomain.com'. (ADCOMMON0052)
VERBOSE: [DC2]:                            [[WaitForADDomain]WaitForPrimaryDC] The type 'System.DirectoryServices.ActiveDirectory.DirectoryContext' is already loaded into the PowerShell session. (ADCOMMON0043)
VERBOSE: [DC2]:                            [[WaitForADDomain]WaitForPrimaryDC] Get a new Active Directory context of the type 'Domain'. (ADCOMMON0046)
VERBOSE: [DC2]:                            [[WaitForADDomain]WaitForPrimaryDC] The Active Directory context will target 'testdomain.com'. (ADCOMMON0047)
VERBOSE: [DC2]:                            [[WaitForADDomain]WaitForPrimaryDC] The Active Directory context will be accessed using the 'testdomain\Administrator' credentials. (ADCOMMON0048)
VERBOSE: [DC2]:                            [[WaitForADDomain]WaitForPrimaryDC] Exception Name: System.Management.Automation.MethodInvocationException
VERBOSE: [DC2]:                            [[WaitForADDomain]WaitForPrimaryDC] Inner Exception Name: System.DirectoryServices.ActiveDirectory.ActiveDirectoryObjectNotFoundException
VERBOSE: [DC2]: LCM:  [ End    Test     ]  [[WaitForADDomain]WaitForPrimaryDC]  in 12.5290 seconds.
PowerShell DSC resource MSFT_WaitForADDomain  failed to execute Test-TargetResource functionality with error message: Exception calling "FindOne" with "1" argument(s): "Domain controller not found in the domain "testdomain.com"."
    + CategoryInfo          : InvalidOperation: (:) [], CimException
    + FullyQualifiedErrorId : ProviderOperationExecutionFailure
    + PSComputerName        : DC2

VERBOSE: [DC2]: LCM:  [ End    Set      ]
The SendConfigurationApply function did not succeed.
    + CategoryInfo          : NotSpecified: (root/Microsoft/...gurationManager:String) [], CimExce
   ption
    + FullyQualifiedErrorId : MI RESULT 1
    + PSComputerName        : DC2

VERBOSE: Operation 'Invoke CimMethod' complete.
VERBOSE: Time taken for configuration job to complete is 29.308 seconds

@X-Guardian
Copy link
Contributor

Cheers @jcuzzi, you are right, having the FindOne method call in a function does affect the catch handling of any exception from it. Here is the above test code modified with the FindOne call in a function:

function Find-DomainControllerFindOneWrapper
{
    [CmdletBinding()]
    [OutputType([System.DirectoryServices.ActiveDirectory.DomainController])]
    param
    (
        [Parameter(Mandatory = $true)]
        [System.DirectoryServices.ActiveDirectory.DirectoryContext]
        $DirectoryContext
    )

    [System.DirectoryServices.ActiveDirectory.DomainController]::FindOne($DirectoryContext)
}

$typeName = 'System.DirectoryServices.ActiveDirectory.DirectoryContext'

Add-Type -AssemblyName 'System.DirectoryServices'

$argumentList = @(
    "Domain"
    "doesnotexistyet.local"
)

$objectParameters = @{
    TypeName     = $typeName
    ArgumentList = $argumentList
}

$findDomain = New-Object @objectParameters

try
{
    Find-DomainControllerFindOneWrapper -DirectoryContext $findDomain
}
catch [System.DirectoryServices.ActiveDirectory.ActiveDirectoryObjectNotFoundException]
{
    Write-Host 'Specific catch: ActiveDirectoryObjectNotFoundException'
    Write-Host 'Exception Name: ' + $_.Exception.GetType().FullName
}
catch [System.Management.Automation.MethodInvocationException]
{
    Write-Host 'Specific catch: MethodInvocationException'
    Write-Host 'Exception Name: ' + $_.Exception.GetType().FullName
    Write-Host 'Inner Exception Name: ' + $_.Exception.InnerException.GetType().FullName
} 
catch
{
    Write-Host 'Generic catch'
    Write-Host 'Exception Name: ' + $_.Exception.GetType().FullName
    Write-Host 'Inner Exception Name: ' + $_.Exception.InnerException.GetType().FullName
}

For me, the exception is now caught by the MethodInvocationException catch instead of the ActiveDirectoryObjectNotFoundException. I'm going to take a look at the two exceptions and see what the difference is.

@johlju johlju removed in progress The issue is being actively worked on by someone. needs more information The issue needs more information from the author or the community. labels Apr 9, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug The issue is a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants