ExtUtils::MM_Win32 should not generate "mt" command when CRT is statically linked

[p5pRT]

Migrated from rt.perl.org#61492 (status was 'resolved')

Searchable as RT61492$

[p5pRT]

From esquel@sommarskog.se

Created by esquel@sommarskog.se

ExtUtils​::MM_Win32 includes these lines​:

  # VS2005 (aka VC 8) or higher, but not for 64-bit compiler from Platform SDK
  if ($Config{ivsize} == 4 && $Config{cc} eq 'cl' and $Config{ccversion} =~ /^(\d+)/ and $1 >= 14)
  {
  push(@​m,
  q{
  mt -nologo -manifest $@​.manifest -outputresource​:$@​;2 && del $@​.manifest});
  }

I am told the that "mt" command is there, because VC8 and later generates a manifest
file that is needed to use the MSVCRT80.DLL or corresponding file.

However, if you opt to link statically to the CRT and thus include the CRT routines in
your own DLL, VC80 will not generate any manifest file, and the "mt" command and thus
the entire build will fail.

Note that there are several options to request static linkage​: /MT,/MTd and a couple more.

Perl Info

Flags:
    category=library
    severity=high

Site configuration information for perl 5.10.0:

Configured by SYSTEM at Wed Sep  3 13:16:08 2008.

Summary of my perl5 (revision 5 version 10 subversion 0) configuration:
  Platform:
    osname=MSWin32, osvers=5.00, archname=MSWin32-x86-multi-thread
    uname=''
    config_args='undef'
    hint=recommended, useposix=true, d_sigaction=undef
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=undef, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cl', ccflags ='-nologo -GF -W3 -MD -Zi -DNDEBUG -O1 -DWIN32 -D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT -DUSE_SITECUSTOMIZE -DPRIVLIB_LAST_IN_INC -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -DPERL_MSVCRT_READFIX',
    optimize='-MD -Zi -DNDEBUG -O1',
    cppflags='-DWIN32'
    ccversion='15.0.30729', gccversion='', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='link', ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf  -libpath:"C:\Perl\lib\CORE"  -machine:x86'
    libpth=\lib
    libs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib  comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib  netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib msvcrt.lib
    perllibs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib  comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib  netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib msvcrt.lib
    libc=msvcrt.lib, so=dll, useshrplib=true, libperl=perl510.lib
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf  -libpath:"C:\Perl\lib\CORE"  -machine:x86'

Locally applied patches:
    ACTIVEPERL_LOCAL_PATCHES_ENTRY
    33741 avoids segfaults invoking S_raise_signal() (on Linux)
    33763 Win32 process ids can have more than 16 bits
    32809 Load 'loadable object' with non-default file extension
    32728 64-bit fix for Time::Local


@INC for perl 5.10.0:
    C:/Perl/site/lib
    C:/Perl/lib
    .


Environment for perl 5.10.0:
    HOME (unset)
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE;C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN;C:\Program Files\Microsoft Visual Studio 9.0\Common7\Tools;C:\WINDOWS\Microsoft.NET\Framework\v3.5;C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727;C:\Program Files\Microsoft Visual Studio 9.0\VC\VCPackages;C:\Program Files\Microsoft SDKs\Windows\v6.0A\bin;C:\Perl\site\bin;C:\Perl\bin;C:\Perl\AS-820\site\bin;C:\Perl\AS-820\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;c:\Program Files\Microsoft SQL Server\100\Tools\Binn\;c:\Program Files\Microsoft SQL Server\100\DTS\Binn\
    PERL_BADLANG (unset)
    SHELL (unset)

[p5pRT]

From @steve-m-hay

Erland Sommarskog (via RT) wrote​:

ExtUtils​::MM_Win32 includes these lines​:

  \# VS2005 \(aka VC 8\) or higher\, but not for 64\-bit compiler from
  Platform SDK if \($Config\{ivsize\} == 4 && $Config\{cc\} eq 'cl'
and $Config\{ccversion\} =~ /^\(\\d\+\)/ and $1 >= 14\) \{
    push\(@​m\,
      q\{

mt -nologo -manifest $@​.manifest -outputresource​:$@​;2 && del
$@​.manifest}); }

I am told the that "mt" command is there, because VC8 and later
generates a manifest
file that is needed to use the MSVCRT80.DLL or corresponding file.

However, if you opt to link statically to the CRT and thus include
the CRT routines in
your own DLL, VC80 will not generate any manifest file, and the "mt"
command and thus
the entire build will fail.

Note that there are several options to request static linkage​:
/MT,/MTd and a couple more.

What is it that you're static-linking to the CRT?

I see from your perl -V output​:

[...]

Compiler​:
cc='cl', ccflags ='-nologo -GF -W3 -MD -Zi -DNDEBUG -O1 -DWIN32
-D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT -DUSE_SITECUSTOMIZE
-DPRIVLIB_LAST_IN_INC -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS
-DUSE_PERLIO -DPERL_MSVCRT_READFIX', optimize='-MD -Zi -DNDEBUG
-O1', cppflags='-DWIN32' ccversion='15.0.30729', gccversion='',
[...]

that you've built perl with the -MD option (for dynamic-linking the
MSVCR*.dll) [and, indeed, that's the only option available in perl's
win32/Makefile], so EU​::MM just assumes that when you're building XS
extensions you're using the same build options.

I'm not even sure I understand how you've told EU​::MM to static-link the
CRT. I thought if you're using EU​::MM to build anything then it just
automatically uses the perl build options. Can we see a sample Makefile
that exhibits the problem?

I'm also always slightly uneasy about mix'n'match-ing different CRTs. If
one component (exe/dll) is built using MSVCR*.dll, then I'd always
prefer all the other components to do likewise, rather than having some
others built with LIBC.lib or whatever.

Is it possible for you to build everything with -MD, like EU​::MM is
expecting anyway?

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @jandubois

On Fri, 19 Dec 2008, Steve Hay wrote​:

Erland Sommarskog (via RT) wrote​:

ExtUtils​::MM_Win32 includes these lines​:

  \# VS2005 \(aka VC 8\) or higher\, but not for 64\-bit compiler from
  Platform SDK if \($Config\{ivsize\} == 4 && $Config\{cc\} eq 'cl'
and $Config\{ccversion\} =~ /^\(\\d\+\)/ and $1 >= 14\) \{
    push\(@​m\,
      q\{

mt -nologo -manifest $@​.manifest -outputresource​:$@​;2 && del
$@​.manifest}); }

I am told the that "mt" command is there, because VC8 and later
generates a manifest file that is needed to use the MSVCRT80.DLL or
corresponding file.

However, if you opt to link statically to the CRT and thus
include the CRT routines in your own DLL, VC80 will not generate
any manifest file, and the "mt" command and thus the entire build
will fail.

Note that there are several options to request static linkage​:
/MT,/MTd and a couple more.

I don't think it is feasible to detect static vs. dynamic linking by analyzing
all the compile and link options. It would be much easier to only invoke the
`mt` command if the compile step actually generated a manifest​:

  if exist $@​.manifest mt -nologo -manifest $@​.manifest ...
  if exist $@​.manifest del $@​.manifest

[...]

I'm not even sure I understand how you've told EU​::MM to static-link the
CRT. I thought if you're using EU​::MM to build anything then it just
automatically uses the perl build options. Can we see a sample Makefile
that exhibits the problem?

He is just adding '/MT' to CCFLAGS in WriteMakeFile().

I'm also always slightly uneasy about mix'n'match-ing different CRTs. If
one component (exe/dll) is built using MSVCR*.dll, then I'd always
prefer all the other components to do likewise, rather than having some
others built with LIBC.lib or whatever.

Yes, it is preferable to use just a single CRT, but mixing them generally
works as long as you are careful not to pass FILE* pointers to a different
CRT than the one that created it, and not to free() memory via a CRT that
didn't malloc() it etc. There also used to be problems with C++ structured
exception handling, but those don't matter if you just use plain C.

Is it possible for you to build everything with -MD, like EU​::MM is
expecting anyway?

He is getting linking problems without /MT and is not interested in
figuring out how to solve them.

I think the change I suggested above should be safe enough to apply,
even if you personally don't want to mix and match CRTs yourself.

Cheers,
-Jan

[p5pRT]

From @steve-m-hay

Jan Dubois wrote​:

Erland Sommarskog (via RT) wrote​:

ExtUtils​::MM_Win32 includes these lines​:

  \# VS2005 \(aka VC 8\) or higher\, but not for 64\-bit compiler
  from Platform SDK if \($Config\{ivsize\} == 4 && $Config\{cc\} eq
'cl' and $Config\{ccversion\} =~ /^\(\\d\+\)/ and $1 >= 14\) \{        
      push\(@​m\, q\{

mt -nologo -manifest $@​.manifest -outputresource​:$@​;2 && del
$@​.manifest}); }

I am told the that "mt" command is there, because VC8 and later
generates a manifest file that is needed to use the MSVCRT80.DLL or
corresponding file.

However, if you opt to link statically to the CRT and thus
include the CRT routines in your own DLL, VC80 will not generate
any manifest file, and the "mt" command and thus the entire build
will fail.

Note that there are several options to request static linkage​:
/MT,/MTd and a couple more.

I don't think it is feasible to detect static vs. dynamic linking by
analyzing all the compile and link options. It would be much easier
to only invoke the `mt` command if the compile step actually
generated a manifest​:

if exist $@​\.manifest mt \-nologo \-manifest $@​\.manifest \.\.\.
if exist $@​\.manifest del $@​\.manifest

Thanks, applied as 4adc95e.

Erland, are you able to rsync the latest perl-current and verify whether
this change solves your problem?

[p5pRT]

From @jandubois

On Fri, 19 Dec 2008, Steve Hay wrote​:

Jan Dubois wrote​:

I don't think it is feasible to detect static vs. dynamic linking by
analyzing all the compile and link options. It would be much easier
to only invoke the `mt` command if the compile step actually
generated a manifest​:

if exist $@​\.manifest mt \-nologo \-manifest $@​\.manifest \.\.\.
if exist $@​\.manifest del $@​\.manifest

Thanks, applied as 4adc95e.

On second thoughts I think we should also remove the compiler check and
emit this code now unconditionally. It will be a no-op if you use a
compiler that doesn't generate a manifest.

The advantage of removing the compiler check is that this will make
things work for the latest 64-bit VC versions as well. The current check
was explicitly not adding the manifest code in 64-bit build because the
VC version in the Windows 2003 Platform SDK uses the same version number
as VS 2005, but links against the 64-bit version of MSVCRT.dll and
therefore doesn't generate a manifest. The new code should be safe for
all compiler versions.

Cheers,
-Jan

[p5pRT]

From @samv

On Fri, 2008-12-19 at 17​:43 +0000, Steve Hay wrote​:

Thanks, applied as 4adc95e.

Just to nitpick this change; the body of the commit is this​:

Subject​: RE​: [perl #61492] ExtUtils​::MM_Win32 should not generate "mt"
command when CRT is statically linked
From​: "Jan Dubois" <jand@​activestate.com>
Date​: Fri, 19 Dec 2008 03​:02​:45 -0800
Message-ID​: <00f901c961c9$5321e1e0$f965a5a0$@​com>

Ideally the "From" and "Date" headers should go into the "Author" fields
of the commit. This happens automatically if you save the e-mail with
the patch in it, and apply it using 'git-am'. Or you could change it
after committing (before pushing) using 'git commit --amend
--author="XXX"'.

Also, the general practice is to separate the one-line description of
the change from the first stanza of information about the change. Ie,
the first line of the commit message is implied to be the Subject​:, and
after the first line feed comes the long description of the commit. Eg,
a summary of why the change was required - perhaps an appropriate
comment lifted from the list message. And after that, the other
RFC822-style headers which are appropriate for the commit can be listed.

I realise that including a summary isn't the current practice, but it
does add value and if you look at the "timinator"-era and Chip's
history, I've actually gone and thrown (trimmed) p5p message bodies as
the long description to save the maintainer a visit to the list
archive.

Sam.

[p5pRT]

From @steve-m-hay

Jan Dubois wrote​:

On Fri, 19 Dec 2008, Steve Hay wrote​:

Jan Dubois wrote​:

I don't think it is feasible to detect static vs. dynamic linking by
analyzing all the compile and link options. It would be much easier
to only invoke the `mt` command if the compile step actually
generated a manifest​:

if exist $@​\.manifest mt \-nologo \-manifest $@​\.manifest \.\.\.
if exist $@​\.manifest del $@​\.manifest

Thanks, applied as 4adc95e.

On second thoughts I think we should also remove the compiler check
and emit this code now unconditionally. It will be a no-op if you use
a compiler that doesn't generate a manifest.

Thanks, applied as http​://perl5.git.perl.org/perl.git/commit/200cbd6

[p5pRT]

From @steve-m-hay

Steve Hay wrote​:

Jan Dubois wrote​:

On Fri, 19 Dec 2008, Steve Hay wrote​:

Jan Dubois wrote​:

I don't think it is feasible to detect static vs. dynamic linking
by analyzing all the compile and link options. It would be much
easier to only invoke the `mt` command if the compile step
actually generated a manifest​:

if exist $@​\.manifest mt \-nologo \-manifest $@​\.manifest \.\.\.
if exist $@​\.manifest del $@​\.manifest

Thanks, applied as 4adc95e.

On second thoughts I think we should also remove the compiler check
and emit this code now unconditionally. It will be a no-op if you use
a compiler that doesn't generate a manifest.

Thanks, applied as http​://perl5.git.perl.org/perl.git/commit/200cbd6

..., er, and http​://perl5.git.perl.org/perl.git/commit/3ddd48a ;-)

[p5pRT]

From @steve-m-hay

Fixes were committed to bleadperl, and are now in
ExtUtils-MakeMaker-6.50 as well.

[p5pRT]

@steve-m-hay - Status changed from 'open' to 'resolved'