The setup:di:compile command should not process folders as files

[astorm]

The setup:di:compile command is a little too greedy in scanning for PHP files. This can lead to the following exception

$ php bin/magento -vvv setup:di:compile
Compilation was started.
Repositories code generation... 1/7 [====>-----------------------]  14% 3 secs 58.8 MiB

  [Zend\Code\Exception\RuntimeException]  
  No tokens were provided                 

Exception trace:
 () at /Users/alanstorm/Sites/magento-2-with-keys/magento2/vendor/zendframework/zend-code/src/Scanner/TokenArrayScanner.php:298
 Zend\Code\Scanner\TokenArrayScanner->scan() at /Users/alanstorm/Sites/magento-2-with-keys/magento2/vendor/zendframework/zend-code/src/Scanner/TokenArrayScanner.php:134
 Zend\Code\Scanner\TokenArrayScanner->getClassNames() at /Users/alanstorm/Sites/magento-2-with-keys/magento2/setup/src/Magento/Setup/Module/Di/Code/Reader/ClassesScanner.php:67
 Magento\Setup\Module\Di\Code\Reader\ClassesScanner->getList() at /Users/alanstorm/Sites/magento-2-with-keys/magento2/setup/src/Magento/Setup/Module/Di/App/Task/Operation/RepositoryGenerator.php:66
 Magento\Setup\Module\Di\App\Task\Operation\RepositoryGenerator->doOperation() at /Users/alanstorm/Sites/magento-2-with-keys/magento2/setup/src/Magento/Setup/Module/Di/App/Task/Manager.php:56
 Magento\Setup\Module\Di\App\Task\Manager->process() at /Users/alanstorm/Sites/magento-2-with-keys/magento2/setup/src/Magento/Setup/Console/Command/DiCompileCommand.php:211
 Magento\Setup\Console\Command\DiCompileCommand->execute() at /Users/alanstorm/Sites/magento-2-with-keys/magento2/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:257
 Symfony\Component\Console\Command\Command->run() at /Users/alanstorm/Sites/magento-2-with-keys/magento2/vendor/symfony/console/Symfony/Component/Console/Application.php:874
 Symfony\Component\Console\Application->doRunCommand() at /Users/alanstorm/Sites/magento-2-with-keys/magento2/vendor/symfony/console/Symfony/Component/Console/Application.php:195
 Symfony\Component\Console\Application->doRun() at /Users/alanstorm/Sites/magento-2-with-keys/magento2/vendor/magento/framework/Console/Cli.php:49
 Magento\Framework\Console\Cli->doRun() at /Users/alanstorm/Sites/magento-2-with-keys/magento2/vendor/symfony/console/Symfony/Component/Console/Application.php:126
 Symfony\Component\Console\Application->run() at /Users/alanstorm/Sites/magento-2-with-keys/magento2/bin/magento:25

This appears to be the result of the setup:di:compile command processing a directory/folder that ends in .php as a file. Steps should be taken to ensure setup:di:compile only processes actual files,
and not directories. This is more than a theoretical concerns -- this directly impacted a popular third party Magento extension: UltimateModuleCreator/Umc_Base#16 (comment)

Steps to reproduce

1. Install Magento from develop branch.
2. Create a folder named Example.php in a module folder
3. Run the setup:di:compile command

Expected result

1. Compilation finishes

Actual result

1. Compilation halts with No tokens were provided error.

[der-workfloh]

Additionally, empty files also provokes the No tokens were provided issue, so there should be a check on ClassesScanner to evaluate if used directory entry is a valid file and if its content is not empty. Added #3836 to fix this.

[mazhalai]

@astorm @websharp we have created MAGETWO-50848 to investigate and fix.

[piotrekkaminski]

The fix should be already live.