ipyc.exe broken on 2.7.7

[slide]

From @andygraham on December 13, 2016 9:44

My IronPython IDE uses ipyc.exe to compile standalone exes. It worked fine with 2.7.6.3 but with 2.7.7 it fails with an error like

ERROR: Could not find file 'C:\Users\Andy\Documents\IronPython Projects\Template
s\WpfXamlTemplate.py.dll'.

It has successfully produced WpfXamlTemplate.dll but as you can see it is looking for the wrong dll to finalise the exe.

Copied from original issue: IronLanguages/main#1554

[slide]

Can you give the command line that you are using for ipyc.exe?

[slide]

From @andygraham on December 13, 2016 13:35

I'm using a response file but I've found where part of the problem lies.
IronPythonCompiler Program.cs line 164 if (config.embed)

            if (config.Embed) {
                // put the generated DLL into the resources for the stub exe
                var dllname = Path.GetDirectoryName(config.Output) + "\\" + Path.GetFileNameWithoutExtension(config.Output) + ".dll";
                Console.WriteLine("DLL : " + dllname);
                Console.WriteLine("Res : " + "IPDll." + Path.GetFileNameWithoutExtension(config.Output) + ".dll");
                var mem = new MemoryStream();
                var rw = new ResourceWriter(mem);
                rw.AddResource("IPDll." + Path.GetFileNameWithoutExtension(config.Output) + ".dll", dllname);
                rw.Generate();
                mem.Position = 0;
                mb.DefineManifestResource("IPDll.resources", mem, ResourceAttributes.Public);
                File.Delete(dllname);

config.Ouput seems to be carrying the extension. The above code fixes the error but the generated exe fails so I've broken something. Looking at the generated exe with Reflector shows that the generated dll has not been embedded as a resource but I can't immediately see anything wrong in the fragment above so I'm stuck at the moment

[slide]

From @andygraham on December 13, 2016 13:47

Heres the response file

/target:winexe
/standalone
/out:C:\Users\Andy\Documents\IronPython Projects\Templates\WpfXamlTemplate.py
/main:C:\Users\Andy\Documents\IronPython Projects\Templates\WpfXamlTemplate.py
/win32icon:
C:\Users\Andy\Documents\IronPython Projects\Templates\WpfXamlTemplate.py
C:\IronPython 2.7\Lib\genericpath.py
C:\IronPython 2.7\Lib\sysconfig.py
C:\IronPython 2.7\Lib\_abcoll.py
C:\IronPython 2.7\Lib\warnings.py
C:\IronPython 2.7\Lib\ntpath.py
C:\IronPython 2.7\Lib\UserDict.py
C:\IronPython 2.7\Lib\ntpath.py
C:\IronPython 2.7\Lib\types.py
C:\IronPython 2.7\Lib\linecache.py
C:\IronPython 2.7\Lib\os.py
C:\IronPython 2.7\Lib\_weakrefset.py
C:\IronPython 2.7\Lib\stat.py
C:\IronPython 2.7\Lib\traceback.py
C:\IronPython 2.7\Lib\site.py
C:\IronPython 2.7\Lib\abc.py

[slide]

From @andygraham on December 13, 2016 13:55

Removing /standalone and compiling to a separate stub exe and dll compiles without error but fails to run with a file not found error. I assume it's the same naming problem as above..

[slide]

From @andygraham on December 13, 2016 14:3

Sure enough, looking in the stub exe with Reflector it's got a bad filename. But it looks odd that it has embedded the full path in the stub exe. Surely this means that you can't move the exe and dll to another location?

        return PythonOps.InitializeModuleEx(Assembly.LoadFile(Path.GetFullPath(@"C:\Users\Andy\Documents\IronPython Projects\Templates\WpfXamlTemplate.py.dll")), "__main__", null, false, null);

[slide]

From @slozier on December 13, 2016 14:32

I'm looking thought the code and it looks like the way the out parameter is handled is a bit of a mess. Maybe try with /out:WpfXamlTemplate?

[slide]

Yeah, I'm fixing it up

[slide]

Just as an FYI, this doesn't look correct:

/win32icon:
C:\Users\Andy\Documents\IronPython Projects\Templates\WpfXamlTemplate.py

It should be the path to a .ico file.

[slide]

From @andygraham on December 13, 2016 17:15

In previous IronPython versions it worked fine both with and without specifying an icon filename and still seems to now I have crudely patched ipyc to work again - at least for what I immediately need. It seems ipyc will accept an empty /win32icon: line in an response file though probably not on the command line (I never tried it) so my IDE response file template always includes that line.

[slide]

Does it set a weird icon for the the resulting executable?

[slide]

From @andygraham on December 13, 2016 17:30

Not that I can see. I think there is no icon embedded in the exe and File Explorer etc. just use a default icon. I can't actually see in the exe as Reflector only shows .NETty stuff and not the win32 stuff and I don't have a resource editor to hand to take a peek

[slide]

From @andygraham on December 14, 2016 18:16

There's a further problem with ipyc.exe which may go back earlier than 2.7.7.

The /standalone option doesn't seem to be standalone at all. After I changed my main development machine to 2.7.7 when I tried a program compiled against 2.7.7 on a VM with 2.7.6.3 installed it failed with the event log showing a file not found exception. After lots of dead end attempts to find out what was wrong I updated the VM to 2.7.7 and lo and behold! the 2.7.7 compiled program now worked but the ones compiled to use 2.7.6.3 failed. They also wouldn't run on another VM without IronPython installed at all.

So it looks like despite including the IronPython assemblies in the exe at runtime it is still trying to use the ones in the IronPython installation.

[slide]

What file wasn't found?

[slide]

From @andygraham on December 14, 2016 19:12

It doesn't say, but it looked like an assembly resolution failure

Application: WpfTemplateold.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.ArgumentException
   at System.IO.UnmanagedMemoryStream.Read(Byte[], Int32, Int32)
   at PythonMain.AssemblyResolve(System.Object, System.ResolveEventArgs)
   at System.AppDomain.OnAssemblyResolveEvent(System.Reflection.RuntimeAssembly, System.String)

Exception Info: System.IO.FileLoadException
   at PythonMain.Main()

But I am now totally baffled :(

I validated the above finding in the VM by uninstalling and re-installing the two flavours of IronPython, 2.7.6.3 and 2.7.7 and sure enough the failing compiled exes swapped over as expected. The VM is running 32 bit Windows 7 Pro with the error above in the event log.

So I then tried my Asus VivoPad Note 8 which is running 32bit Windows 8.1 Pro and has IronPython 2.7.6.3 installed and it behaved identically to the VM, running only the exes compiled under 2.7.6.3 with the same error in the event log.

I tried a Windows 10 VM running 32bit Windows 10 Pro that has Ironpython 2.7.6.3 installed and that behaved the same as the Asus and the Windows 7 VM running only the exes compiled under 2.7.6.3 with the same error in the event log.

I tried a brand new 32bit Windows 7 VM without IronPython installed and none of the exes ran.

However I noticed that both both2.7.6.3 and the 2.7.7 exes worked fine on my development machine which is 64bit Windows 7 Home with IronPython 2.7.7 installed.

So I tried on my Surface Pro 4 which is 64bit Windows 10 Pro with IronPython 2.7.6.3 installed and all the exes worked there!

I don't have a bare 64bit machine but I uninstalled IronPython 2.7.6.3 from the Surface Pro 4 and checked that my own written REPL failed, as expected, but the compiled exes all still worked!

So two 64bit machines with different Windows behave the same (properly) and three 32bit machines, each with different Windows behave the same (improperly :))!

[slide]

My mistake on the /win32icon stuff, I was reading on my phone and it looked like you were passing a .py file as the parameter to /win32icon.

I don't have IronPython installed on my development machine and I am able to run the compiled exe just fine. My system is Windows 10 64-bit. I'll see if I can get a 32-bit Windows 7 machine to try.