Unable to compile PHP 7.2 using Visual Studio 2017

[yitam]

I followed the steps outlined in Basic usage example, trying to compile PHP. I have Visual Studio 2017 (v15.0) installed, and I've used Developer Prompt to run the starter batch (tested with both phpsdk-vc15-x86 and phpsdk-vc15-x64).

When I ran configure I saw PHP Version: 7.3.0-dev, which is not what I want.
Therefore, I downloaded php-7.2.0-beta1-src.zip today and simply tried phpsdk_deps -u.

The SDK was downloaded into deps, but when trying the last step buildconf && configure --enable-cli && nmake, I have the same issue as described in http://news.php.net/php.pecl.dev/11459.

An excerpt of the output of nmake:

Recreating build dirs
	"cl.exe" /D _USRDLL /D PHP7DLLTS_EXPORTS /D PHP_EXPORTS /D LIBZEND_EXPORTS /D TSRM_EXPORTS /D SAPI_EXPORTS /D WINVER=0x0601 /D COMPILE_DL_GD /D COMPILE_DL_OPCACHE -Iext/bcmath/libbcmath/src /DZEND_ENABLE_STATIC_TSRMLS_CACHE=1  /DZEND_ENABLE_STATIC_TSRMLS_CACHE=1 /DHAVE_MSCOREE_H=1  /Iext/date/lib /DZEND_ENABLE_STATIC_TSRMLS_CACHE=1 /DHAVE_TIMELIB_CONFIG_H=1 /wd4244 /DZEND_ENABLE_STATIC_TSRMLS_CACHE=1 /I "ext/hash/sha3/generic32lc" /DHAVE_KECCAKHASH_H=1 /DKeccakP200_excluded /DKeccakP400_excluded /DKeccakP800_excluded /I "C:\php-sdk\bin\phpmaster\vc15\x86\deps\include" /DHAVE_ICONV_H=1 /DZEND_ENABLE_STATIC_TSRMLS_CACHE=1 /D PHP_ICONV_EXPORTS /DZEND_ENABLE_STATIC_TSRMLS_CACHE=1 /DZEND_ENABLE_STATIC_TSRMLS_CACHE=1 -Iext/pcre/pcrelib -DZEND_ENABLE_STATIC_TSRMLS_CACHE=1 /D HAVE_CONFIG_H /I "C:\php-sdk\bin\phpmaster\vc15\x86\deps\include" /DHAVE_EDITLINE_READLINE_H=1 /D HAVE_LIBEDIT /D HAVE_RL_COMPLETION_MATCHES /DZEND_ENABLE_STATIC_TSRMLS_CACHE=1 /DZEND_ENABLE_STATIC_TSRMLS_CACHE=1 /DZEND_ENABLE_STATIC_TSRMLS_CACHE=1 /I "ext/date/lib" /DHAVE_TIMELIB_CONFIG_H=1 /DZEND_ENABLE_STATIC_TSRMLS_CACHE=1  /I "C:\php-sdk\bin\phpmaster\vc15\x86\deps\include" /DHAVE_ZIP_H=1 /I "C:\php-sdk\bin\phpmaster\vc15\x86\deps\lib\libzip\include" /DHAVE_ZIPCONF_H=1 /D _WIN32 /D ZLIB_EXPORTS /DZEND_ENABLE_STATIC_TSRMLS_CACHE=1 /I "C:\php-sdk\bin\phpmaster\vc15\x86\deps\include\libxml2" /DHAVE_LIBXML_PARSER_H=1 /DHAVE_LIBXML_TREE_H=1 /DZEND_ENABLE_STATIC_TSRMLS_CACHE=1 /D LIBXML_STATIC /D LIBXML_STATIC_FOR_DLL /D HAVE_WIN32_THREADS /I "C:\php-sdk\bin\phpmaster\vc15\x86\deps\include\libxml2" /DHAVE_LIBXML_PARSER_H=1 /D LIBXML_STATIC /DZEND_ENABLE_STATIC_TSRMLS_CACHE=1 /I "C:\php-sdk\bin\phpmaster\vc15\x86\deps\include\libxml2" /DHAVE_LIBXML_TREE_H=1 /D LIBXML_STATIC /D PHP_SIMPLEXML_EXPORTS /I "C:\php-sdk\bin\phpmaster\vc15\x86\deps\include\libxml2" /DHAVE_LIBXML_PARSER_H=1 /DHAVE_LIBXML_TREE_H=1 /DZEND_ENABLE_STATIC_TSRMLS_CACHE=1 /D LIBXML_STATIC /I "C:\php-sdk\bin\phpmaster\vc15\x86\deps\include\libxml2" /DHAVE_LIBXML_PARSER_H=1 /D LIBXML_STATIC /I "ext/date/lib" /DHAVE_TIMELIB_CONFIG_H=1 /I "C:\php-sdk\bin\phpmaster\vc15\x86\deps\include\libxml2" /DHAVE_LIBXML_PARSER_H=1 /DHAVE_LIBXML_TREE_H=1 /D LIBXML_STATIC /I "C:\php-sdk\bin\phpmaster\vc15\x86\deps\include\libxml2" /DHAVE_LIBXML_XMLWRITER_H=1 /D LIBXML_STATIC /nologo /I . /I main /I Zend /I TSRM /I ext /D _WINDOWS /D WINDOWS=1 /D ZEND_WIN32=1 /D PHP_WIN32=1 /D WIN32 /D _MBCS /W3 /D _USE_MATH_DEFINES /FD /wd4996 /D_USE_32BIT_TIME_T=1 /guard:cf /Zc:inline /MP /LD /MD /W3 /Ox /D NDebug /D NDEBUG /D ZEND_WIN32_FORCE_INLINE /GF /D ZEND_DEBUG=0 /I "C:\php-sdk\bin\phpmaster\vc15\x86\deps\include" /D FD_SETSIZE=256 /FoC:\php-sdk\bin\phpmaster\vc15\x86\php-7.2.0-beta1-src\Release\Zend\ /FpC:\php-sdk\bin\phpmaster\vc15\x86\php-7.2.0-beta1-src\Release\Zend\ /FRC:\php-sdk\bin\phpmaster\vc15\x86\php-7.2.0-beta1-src\Release\Zend\ /FdC:\php-sdk\bin\phpmaster\vc15\x86\php-7.2.0-beta1-src\Release\Zend\ /D ZEND_ENABLE_STATIC_TSRMLS_CACHE=1 /c Zend\zend.c Zend\zend_API.c Zend\zend_alloc.c Zend\zend_ast.c Zend\zend_builtin_functions.c Zend\zend_closures.c Zend\zend_compile.c Zend\zend_constants.c Zend\zend_default_classes.c Zend\zend_exceptions.c Zend\zend_execute.c Zend\zend_execute_API.c Zend\zend_extensions.c Zend\zend_float.c Zend\zend_gc.c Zend\zend_generators.c Zend\zend_hash.c Zend\zend_highlight.c Zend\zend_inheritance.c Zend\zend_ini.c Zend\zend_ini_parser.c Zend\zend_ini_scanner.c Zend\zend_interfaces.c Zend\zend_iterators.c Zend\zend_language_parser.c Zend\zend_language_scanner.c Zend\zend_list.c Zend\zend_llist.c Zend\zend_multibyte.c Zend\zend_object_handlers.c Zend\zend_objects.c Zend\zend_objects_API.c Zend\zend_opcode.c Zend\zend_operators.c Zend\zend_ptr_stack.c Zend\zend_smart_str.c Zend\zend_sort.c Zend\zend_sprintf.c Zend\zend_stack.c Zend\zend_stream.c Zend\zend_string.c Zend\zend_strtod.c Zend\zend_ts_hash.c Zend\zend_variables.c Zend\zend_virtual_cwd.c Zend\zend_vm_opcodes.c 
zend.c
zend_API.c
zend_alloc.c
zend_ast.c
zend_builtin_functions.c
zend_closures.c
zend_compile.c
zend_constants.c
Zend\zend_compile.c(23): fatal error C1083: Cannot open include file: 'zend_language_parser.h': No such file or directory
zend_default_classes.c
Zend\zend_ast.c(25): fatal error C1083: Cannot open include file: 'zend_language_parser.h': No such file or directory
zend_exceptions.c
zend_execute.c
zend_execute_API.c
zend_extensions.c
...

I also did the followings:
where bison.exe
this returns c:\php-sdk\msys2\usr\bin\bison.exe

but when I ran bison.exe against Zend\zend_language_parser.y I got the following error
c:\php-sdk\phpmaster\vc15\x86\php-src\Zend\zend_language_parser.y: fatal error: invalid @ in skeleton: @

I also have Visual Studio 2015 installed in the same PC, and I never had problems with php 7.0.* or php 7.1.* using Visual Studio native prompts

Forgot to mention that I did try version 2.0.0, 2.0.7 and the latest (2.0.8-dev) in this step:
git checkout php-sdk-2.0.0 or later

[weltling]

Thanks for the report.

To use the versions from git, just checkout the corresponding branch or tag.

The bison issue is likely caused by the erroneous environment. Please check also the pitfalls section in the readme, especially the presence of Cygwin, MingW, etc. Unfortunately there's no clarity about this bug yet, despite it seems to be also have been showing up in the old SDK, as the ML link witnesses. It is currently not reproducible on clean environments and with Appveyor, fe https://ci.appveyor.com/project/php/php-src/build/master.build.2697/job/lgdjglnto6fch2e1#L493.

Thanks.

[weltling]

@yitam i've an idea how you can debug this. Please fetch procmon and set a filter to see anything related to bison.exe. In particular of the interest are all the file related calls, like paths where dependents are loaded from, or the skeleton path. Some registry keys or env vars might affect these behaviors, too.

Thanks.

[yitam]

Thanks @weltling for your prompt reply and suggestions. I'll try this in a clean vm today.

So this sounds like we can't compile both PHP 7.1 and PHP 7.2 in the same environment even though Visual Studio 15 and 17 could coexist? Or have I misunderstood?

[yitam]

Hi @weltling
For some reason I could not get procmon to find bison info, so I resorted to using a clean vm. Good news is that I can successfully compile PHP 7.2 and the extensions. I still prefer to figure out what's wrong in my current environment. But anyway, thanks for your help.

[dktapps]

I have the same problem on my machine with only VS2017 installed. I can compile PHP 7.0 with the legacy SDK + VS2017 without a problem, but with this version I have this problem.

[weltling]

@yitam thanks for the update. Yeah, if bison isn't invoked, that makes the thing even more unclear. I kinda expected a new VM with only dedicated things on it to do the job.

@dktapps unfortunately this issue is not reproduceable neither on build hosts, nor on AppVeyor, etc. It has to be something on the given environment that causes it, without having the environment it's just shooting in the sky. Were you up to debug what bison does when you nmake? The issue is long standing, so any additional information is valuable.

Thanks.

[dktapps]

@weltling I'm not sure how to debug what bison is up to, but I noticed today that if I download a zip of the SDK from releases, that works perfectly, but a git clone has problems.

I believe this is something to do with the line endings. The release archives have LF line endings, but the git repo seems to have CRLF, which appears to be causing problems with the binary tools. Maybe consider adding a .gitattributes?

[dktapps]

Indeed the only difference between my git clone and the release archive I download from github is the line endings. I'm almost 100% sure this is the problem.

[weltling]

@dktapps to debug what bison does, an option could be checking the system calls it does in procmon.

With EOLs, yeah - I was mentioning that else where as well, but unfortunately had no feedback. LF in the only sensible options, especially it for sure can affect bison as it depends on the skeletons. I couldn't think we could control that behavior from within the repository. Would it be not overridden by the global ~/.gitconfig still? I'd happily merge a patch, if you got one that solves it on your side.

Thanks.

[dktapps]

The gitconfig core.autocrlf causes this problem when set to true. The referenced PR adds a gitattributes to override this behaviour.

[weltling]

@dktapps looks like we're now a step forward towards a solution, thanks for the PR! I've also extended the pitfalls section in the readme and added some more files.

@yitam could you please check the current master on the environment reproducing the issues?

Thanks!

[marinesovitch]

@weltling
I confirm it fixes the issue we discussed in previous week. Currently new SDK does work for us as expected 👍

[yitam]

Kudos to you both @dktapps and @weltling :)
I'll definitely give this a try and get back to you... however, just wondering, why did it work in a clean vm before this PR fix?

[dktapps]

@yitam check your gitconfig, automatic line ending conversion won't happen if core.autocrlf is set to false.

[yitam]

Thank you all for the help. I can now build php7.2 in my existing environment (although I must say I've removed VS 2015 since I reported this issue). All the best.

[weltling]

Looks like we're done here :) 2.0.9 is tagged. Grateful thanks everyone involved, especially @dktapps for the investigation and the patch!