Testing a Command that Expects Input #5246

Closed
ofbeaton opened this Issue May 11, 2015 · 7 comments

Projects

None yet

3 participants

@ofbeaton

The documentation at: http://symfony.com/doc/current/components/console/helpers/questionhelper.html#testing-a-command-that-expects-input

Gives an example of using setInputStream to test command input.

This only works if you provide at minimum the amount of input that is expected. It does not work on failed tests.

If you do not provide enough input, then the input request will go to the console during the run of 'phpunit', which will look like it is hung, as it will be waiting for input until you provide it.

Externally effecting the test runner like this is not correct code.

Either the instruction are missing a step, or this is not how the testing should be done.

A blog post I found about the topic: http://marekkalnik.tumblr.com/post/32601882836/symfony2-testing-interactive-console-command

Recommends mock'ing the helper. Since we don't use Dialog helper anymore, it would be mocking the QuestionHelper instead, ask() method. The example in the article is bad though, because if you ask additional questions you get the same problem, but you should be able to make a mock that does a throw() on further calls or something...

To be honest I don't know what the correct solution is, that is why I came to the docs. I also in IRC and got one response but they couldn't help me. If my question about this on stackoverflow gets answered I'll update this thread:

http://stackoverflow.com/questions/30174946/testing-symfony2-console-commands-that-expect-input

Thanks!

@ofbeaton

My recommendation is to do the Mock. The key is to mock the QuestionHelper ask() method using whatever method you like, like PHPUnit's getMockBuilder or Mockery or a custom class you build yourself.

Then per the blog mentioned, call $cmd->getHelperSet()->set($mockHelper, 'question');

The key is to make sure that any subsequent calls to ask() throw an exception / fail the tests, which can get complicated when mocking multiple questions in a row. A good example handling at least 2 questions would be good.

This is much better than faking the input stream as the current example suggests.

@ofbeaton

Here's an example implementation from my own code, using PHPUnit's built-in mocks. I'm sure there are better ways to do this, and I'll be happy to refactor after more experienced programmers weight in.

My main concerns were:

  • Handle multiple questions, maybe out of order. Mocking with ->at(0) does not work for this.
  • Preserve the usefulness of $cmdTester->getDisplay() hence the writeln usage
  • Throw an exception if we encounter an input request we didn't handle. The key.
public function testExecute()
    {
        $app = new Application();
        $app->add(new GenerateCommand());

        $cmd = $app->find('generate');

        $ask = function (InputInterface $input, OutputInterface $output, Question $question) {
            static $order = -1;

            $order = $order + 1;
            $text = $question->getQuestion();

            // you can check against $text to see if this is the question you want to handle
            // and you can check against $order (starts at 0) for the order the questions come in

            $output->write($text." => ");

            // handle a question
            if (strpos($text, 'overwrite') !== false) {
                $response = true;

            // handle another question
            } elseif (strpos($text, 'api_key') !== false) {
                $response = 'api-test-key';
            }

            if (isset($response) === false) {
                throw new \RuntimeException('Was asked for input on an unhandled question: '.$text);
            }

            $output->writeln(print_r($response, true));
            return $response;
        };
        $helper = $this->getMock('\Symfony\Component\Console\Helper\QuestionHelper', ['ask']);
        $helper->expects($this->any())
            ->method('ask')
            ->will($this->returnCallback($ask));

        $cmd->getHelperSet()->set($helper, 'question');

        $cmdTester = new CommandTester($cmd);

        $cmdTester->execute([
           'command' => $cmd->getName(),
        ]);

        // $cmdTester->getDisplay() can be used to see the output and input of the interaction
        // you could also assert against other side effects like files your application writes
        // ... $this->assert()
    }

One possible improvement would be to mock more downstream, after the Symfony component has already output the question to the user, and only override the input mechanism. My cursory glance did not reveal how to do this and have access to the question asked however, which is important when dealing with multiple input requests, the default scenario I would think.

I've since abstracted this implementation into a "helper" method on my test class, which cleans it up nicely but you may not want to add that much complexity in your documentation.

@ofbeaton

Lastly, here's the abstracted implementation I used which is useful when doing multiple tests, which is what most people will be doing. I throw the exception in the user mock because there is no value or exception that could not be thrown from the user function. In my own implementation I create my own exception class, but I figured that was too much for an example here.

public function testExecute()
    {
        $app = new Application();
        $app->add(new GenerateCommand());

        $cmd = $app->find('generate');

        $this->mockQuestions($cmd, function($text, $order, Question $question) {
            // you can check against $text to see if this is the question you want to handle
            // and you can check against $order (starts at 0) for the order the questions come in

            // handle a question
            if (strpos($text, 'overwrite') !== false) {
                return true;

            // handle another question
            } elseif (strpos($text, 'api_key') !== false) {
                return 'bnet-api-test-key';
            }

            throw new \RuntimeException('Was asked for input on an unhandled question: '.$text);
        });

        $cmdTester = new CommandTester($cmd);

        $cmdTester->execute([
           'command' => $cmd->getName(),
        ]);

        // $cmdTester->getDisplay() can be used to see the output and input of the interaction
        // you could also assert against other side effects like files your application writes
        // ... $this->assert()
    }

protected function mockQuestions(Command $cmd, callable $questions)
    {
        $ask = function (InputInterface $input, OutputInterface $output, Question $question) use ($questions) {
            static $order = -1;

            $order = $order + 1;
            $text = $question->getQuestion();

            $output->write($text." => ");
            $response = call_user_func($questions, $text, $order, $question);

            $output->writeln(print_r($response, true));
            return $response;
        };

        $helper = $this->getMock('\Symfony\Component\Console\Helper\QuestionHelper', ['ask']);
        $helper->expects($this->any())
            ->method('ask')
            ->will($this->returnCallback($ask));

        $cmd->getHelperSet()->set($helper, 'question');
    }

I posted this solution as a blog post (http://io.ofbeaton.com/2015/05/symfony-console-input-testing/) but I imagine the docs giving a working test example compared to the current non-working one would be good!

@ofbeaton

Another day another comment. I figured others may want to benefit from this code without copy + pasting it in their own project. So I made a package on packagist that provides a QuestionTester trait to do it. This is probably the most elegant solution of them all.

composer.phar require ofbeaton/console-tester
// MyCommandTest.php

use Symfony\Component\Console\Application;
use Symfony\Component\Console\Tester\CommandTester;
use Symfony\Component\Console\Question\Question;
use Ofbeaton\Console\Tester\QuestionTester;
use Ofbeaton\Console\Tester\UnhandledQuestionException;

class MyCommandTest extends \PHPUnit_Framework_TestCase
{
  use QuestionTester;

  // ...
  public function testExecute()
  {
    $app = new Application();

    // replace MyCommand with your command
    $app->add(new MyCommand());
    $cmd = $app->find('mycommand');

    // return the input you want to answer the question with
    $this->mockQuestions($cmd, function($text, $order, Question $question) {
        // you can check against $text to see if this is the question you want to handle
        // and you can check against $order (starts at 0) for the order the questions come in

        // handle a question
        if (strpos($text, 'overwrite') !== false) {
            return true;

        // handle another question
        } elseif (strpos($text, 'api_key') !== false) {
            return 'api-test-key';
        }

        throw new UnhandledQuestionException();
    });

    $cmdTester = new CommandTester($cmd);
    $cmdTester->execute([
       'command' => $cmd->getName(),
    ]);

    // $cmdTester->getDisplay() can be used to see the output and input of the interaction
    // you could also assert against other side effects like files your application writes
    // $this->assertRegExp('/.../', $commandTester->getDisplay());
}
@ofbeaton

Upon further inspection, it is better to mock the doAsk method instead of ask so that the validators and retry mechanism works. I did so in my trait on packagist/github. It was a straight forward change here: https://github.com/ofbeaton/console-tester/blob/master/src/QuestionHelperMock.php

@javiereguiluz
Member

@ofbeaton now that we've simplified the testing of command inputs and that we've made the appropriate changes in the docs (see #6623) do you think that this issue is still relevant? Thanks!

@ofbeaton

I haven't tested the new way, but from looking at the tests it should handle it. I'll close this and if ever I revisit and find it broken I'll re-open.

@ofbeaton ofbeaton closed this Oct 12, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment