Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: Allow It description to be defined for each test case in TestCases #1361

Closed
deadlydog opened this issue Sep 9, 2019 · 3 comments

Comments

@deadlydog
Copy link

commented Sep 9, 2019

A new feature I would like to see introduced into Pester is the ability to pass in the description of the It block to the TestCases parameter, so that every test case can have a unique, contextual description.

The problem with the TestCases parameter today is that when the hashtable contains many parameters, you lose the context of what is actually being tested and why. I give a more detailed explanation of the problem on my blog, but in short, this is the problem.

I might have 4 tests that look something like this:

Describe 'Get-WorkingDirectory' {
    Context 'When requesting the Application Directory as the working directory' {
        It 'Returns the applications directory when no Custom Working Directory is given' {
            $result = Get-WorkingDirectory -workingDirectoryOption 'ApplicationDirectory' -customWorkingDirectory '' -applicationPath 'C:\AppDirectory\MyApp.exe'
            $result | Should -Be 'C:\AppDirectory'
        }
        It 'Returns the applications directory when a Custom Working Directory is given' {
            $result = Get-WorkingDirectory -workingDirectoryOption 'ApplicationDirectory' -customWorkingDirectory 'C:\SomeDirectory' -applicationPath 'C:\AppDirectory\MyApp.exe'
            $result | Should -Be 'C:\AppDirectory'
        }
    }

    Context 'When requesting a custom working directory' {
        It 'Returns the custom directory' {
            $result = Get-WorkingDirectory -workingDirectoryOption 'CustomDirectory' -customWorkingDirectory 'C:\SomeDirectory' -applicationPath 'C:\AppDirectory\MyApp.exe'
            $result | Should -Be 'C:\SomeDirectory'
        }
        It 'Returns the custom directory even if its blank' {
            $result = Get-WorkingDirectory -workingDirectoryOption 'CustomDirectory' -customWorkingDirectory '' -applicationPath 'C:\AppDirectory\MyApp.exe'
            $result | Should -Be ''
        }
    }
}

And I get rich contextual output:

Describing Get-WorkingDirectory

  Context When requesting the Application Directory as the working directory
    [+] Returns the applications directory when no Custom Working Directory is given 98ms
    [+] Returns the applications directory when a Custom Working Directory is given 15ms

  Context When requesting a custom working directory
    [+] Returns the custom directory 67ms
    [+] Returns the custom directory even if its blank 15ms

Now, this is how you might shorten and simplify that same code using TestCases:

Describe 'Get-WorkingDirectory' {
    It 'Returns the correct working directory' -TestCases @(
        @{ workingDirectoryOption = 'ApplicationDirectory'; customWorkingDirectory = ''; applicationPath = 'C:\AppDirectory\MyApp.exe'; expectedWorkingDirectory = 'C:\AppDirectory' }
        @{ workingDirectoryOption = 'ApplicationDirectory'; customWorkingDirectory = 'C:\SomeDirectory'; applicationPath = 'C:\AppDirectory\MyApp.exe'; expectedWorkingDirectory = 'C:\AppDirectory' }
        @{ workingDirectoryOption = 'CustomDirectory'; customWorkingDirectory = ''; applicationPath = 'C:\AppDirectory\MyApp.exe'; expectedWorkingDirectory = '' }
        @{ workingDirectoryOption = 'CustomDirectory'; customWorkingDirectory = 'C:\SomeDirectory'; applicationPath = 'C:\AppDirectory\MyApp.exe'; expectedWorkingDirectory = 'C:\SomeDirectory' }
    ) {
        param([string] $workingDirectoryOption, [string] $customWorkingDirectory, [string] $applicationPath, [string] $expectedWorkingDirectory)

        $result = Get-WorkingDirectory -workingDirectoryOption $workingDirectoryOption -customWorkingDirectory $customWorkingDirectory -applicationPath $applicationPath
        $result | Should -Be $expectedWorkingDirectory
    }
}

And it's output would look like this:

Describing Get-WorkingDirectory
  [+] Returns the correct working directory 56ms
  [+] Returns the correct working directory 11ms
  [+] Returns the correct working directory 15ms
  [+] Returns the correct working directory 13ms

Do you see the problem? I've lost all context of what is actually being tested and why, both in my code and in the Pester output.

What I would like to see is something like this, where each test case can have its own It description:

Describe 'Get-WorkingDirectory' {
    It 'Default description used if none is provided in the test case' -TestCases @(
        @{ testDescription = 'Returns the applications directory when no Custom Working Directory'; workingDirectoryOption = 'ApplicationDirectory'; customWorkingDirectory = ''; applicationPath = 'C:\AppDirectory\MyApp.exe'; expectedWorkingDirectory = 'C:\AppDirectory' }
        @{ testDescription = 'Returns the applications directory when a Custom Working Directory is given';workingDirectoryOption = 'ApplicationDirectory'; customWorkingDirectory = 'C:\SomeDirectory'; applicationPath = 'C:\AppDirectory\MyApp.exe'; expectedWorkingDirectory = 'C:\AppDirectory' }
        @{ testDescription = 'Returns the custom directory';workingDirectoryOption = 'CustomDirectory'; customWorkingDirectory = ''; applicationPath = 'C:\AppDirectory\MyApp.exe'; expectedWorkingDirectory = '' }
        @{ testDescription = 'Returns the custom directory even if its blank';workingDirectoryOption = 'CustomDirectory'; customWorkingDirectory = 'C:\SomeDirectory'; applicationPath = 'C:\AppDirectory\MyApp.exe'; expectedWorkingDirectory = 'C:\SomeDirectory' }
    ) {
        param([string] $workingDirectoryOption, [string] $customWorkingDirectory, [string] $applicationPath, [string] $expectedWorkingDirectory)

        $result = Get-WorkingDirectory -workingDirectoryOption $workingDirectoryOption -customWorkingDirectory $customWorkingDirectory -applicationPath $applicationPath
        $result | Should -Be $expectedWorkingDirectory
    }
}

and it would produce output like this:

Describing Get-WorkingDirectory
    [+] Returns the applications directory when no Custom Working Directory is given 98ms
    [+] Returns the applications directory when a Custom Working Directory is given 15ms
    [+] Returns the custom directory 67ms
    [+] Returns the custom directory even if its blank 15ms

You may be able to think of a better name for the variable than testDescription, but that's essentially the functionality I'm looking for.

Using TestCases for parameterized tests can greatly reduce the amount of code that needs to be written, but because it comes at the cost of losing all context around the test cases, I find myself only using it for the simplest of scenarios. If the test case parameters are testing anything that's not super obvious, or if I have a lot of parameters (like in the examples above), I skip using TestCases in favour of getting clarity about what it is that I'm actually trying to test.

I know that I can write my own code with some loops and functions to achieve what I'm after, which is actually what I show how to do in the blog post I mentioned earlier, but it would be great if this functionality was native to Pester so that we wouldn't need to continually write the same looping code for all parameterized tests.

@nohwnd

This comment has been minimized.

Copy link
Member

commented Sep 10, 2019

There is <key> syntax that you can use to expand the test case values to the name of the test. It is typically used like this: Given word '<value>' it reverses it to '<expected>' where Value and Expected are keys in the TestCases hashtable. But you can use it to expand any key from the hashtable, even values that your test code does not consume as parameters. So to get the desired result simply do: It -Name '<testDescription>'. Here are few more examples of how I am using it in Assert, and you can surely find some in Pester itself as well, for example in the Planets example.

@deadlydog

This comment has been minimized.

Copy link
Author

commented Sep 10, 2019

Thanks @nohwnd . I actually just noticed that <variable> syntax later today when looking over the It source code. I thought that was pretty cool, but it didn't occur to me that I could use is to replace the entire It description. Your suggestion of using It '<testDescription>' is exactly what I was looking for; I think that will provide the functionality I was looking for just fine. Plus, it doesn't require adding some new special keyword to Pester (i.e. the testDescription variable I proposed), so that's even better! I'm going to go ahead and close this issue. Thanks!

@deadlydog deadlydog closed this Sep 10, 2019

@nohwnd

This comment has been minimized.

Copy link
Member

commented Sep 10, 2019

Awesome, solving issues without doing anything is my favorite way to do it 😁

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.