Override file version for win32

[SuslikV]

The pywin32 module, when used by cx_Freeze in Windows doesn't allow not integer versioning of the files in the first 4 numbers.

This change allows "freeze" while versioning style may be kept.

It doesn't changes logging info of the OpenShot, only file description is affected.

[SuslikV]

More info. Just got an error when tried appveyor to build with pywin32 module installed in Python 3.6.8 (cx_Freeze suggested to install the pywin32 ^_^), so decided to change the source here.

May become useful as first step to resolve the #2762

[ferdnyc]

Interesting!

This version-identifier change makes a lot of sense to me, it seems like a good solution to the problem of pywin32's versioning rules.

However, if the inclusion of executable metadata is predicated on pywin32 being installed (which would seem to be the case, as the version string would've always been incompatible with Win32, yet it never came up before), then we may have a larger issue.

The GitLab builders that build the official Windows packages for OpenShot are, I believe, using the python3 executable from MinGW-64. (Specifically, it would be c:\msys64\mingw64\bin\python3.exe:)

openshot-qt/.gitlab-ci.yml

Lines 81 to 87 in ba259c1

	- Copy-Item "$CI_PROJECT_DIR/build/install-x64/python/*" -Destination "$CI_PROJECT_DIR"
	- $env:Path = "$CI_PROJECT_DIR\build\install-x64\lib;C:\msys64\mingw64\bin;C:\msys64\mingw64\lib;C:\msys64\usr\lib\cmake\UnitTest++;C:\msys64\home\jonathan\depot_tools;C:\msys64\usr;C:\msys64\usr\lib;" + $env:Path;
	- New-Item -path "build/install-x64/share/" -Name "$CI_PROJECT_NAME" -Value "CI_PROJECT_NAME:$CI_PROJECT_NAME`nCI_COMMIT_REF_NAME:$CI_COMMIT_REF_NAME`nCI_COMMIT_SHA:$CI_COMMIT_SHA`nCI_JOB_ID:$CI_JOB_ID" -ItemType file -force
	- $PREV_GIT_LABEL=(git describe --tags --abbrev=0)
	- git log "$PREV_GIT_LABEL..HEAD" --oneline --pretty=format:"%C(auto,yellow)%h%C(auto,magenta)% %C(auto,blue)%>(12,trunc)%ad %C(auto,green)%<(25,trunc)%aN%C(auto,reset)%s%C(auto,red)% gD% D" --date=short > "build/install-x64/share/$CI_PROJECT_NAME.log"
	- python3 freeze.py build
	- python3 installer/build-server.py "$SLACK_TOKEN" "$S3_ACCESS_KEY" "$S3_SECRET_KEY" "$WINDOWS_KEY" "$WINDOWS_PASSWORD" "$GITHUB_USER" "$GITHUB_PASS" "False" "$CI_COMMIT_REF_NAME"

The problem? pywin32 isn't available there, and probably never will be, as it apparently uses M$-proprietary hooks. As one pyinstaller developer put it, "pywin32 is pretty horrible IMHO, using MSVC specific features for no other reason than to be unportable." So it's only ever going to be available in the Windows-native Python, not any of the open-source environments.

There is pywin32-ctypes, which is available for all platforms including MinGW-64 (pip install pywin32-ctypes) — I guess the question is whether cx_Freeze will or can be made to use that instead, to achieve the same thing.

[SuslikV]

But the version string still needs adaptation. Windows expects only 4 numbers as far as I understand. No matter what tool will be used (if ever will it).

[ferdnyc]

Oh, absolutely, 100% agreed. This change makes total sense regardless. It's a step in the right direction, at least.

[ferdnyc]

Looking through cx_Freeze, it picks up pywin32 by importing from a module named win32verstamp, which is definitely not present in pywin32-ctypes. Nor does there appear to be anything resembling a stamp() function. So, it looks like the metadata features may indeed be native-Win/MSVC-only for the moment.

(Referring to cx_Freeze/freezer.py[152:167] from cx_Freeze 5.1.1...)

    def _AddVersionResource(self, exe):
        try:
            from win32verstamp import stamp
        except:
            print("*** WARNING *** unable to create version resource")
            print("install pywin32 extensions first")
            return
        fileName = exe.targetName
        versionInfo = VersionInfo(self.metadata.version,
                comments = self.metadata.long_description,
                description = self.metadata.description,
                company = self.metadata.author,
                product = self.metadata.name,
                copyright = exe.copyright,
                trademarks = exe.trademarks)
        stamp(fileName, versionInfo)

[SuslikV]

Custom Win32GUI from cx_Freeze (http://www.mingw.org/wiki/ms_resource_compiler) each time?

[ferdnyc]

Problem is that the file that becomes launch.exe (or openshot-qt.exe if #2796 merges) is pre-compiled. It lives in the cx_Freeze/bases/ directory of the module and is simply copied into the output directory by freezer.py. If it were being compiled, we could find a way to link a resource file into it. But because it's not the only recourse is to modify the resources of the existing executable.

That's apparently the purpose of the stamp() function in the non-portable pywin32 module. (I've since discovered the sense of that name. One commonly-used name for the VERSIONINFO section of the Windows resource file is VS_VERSION_INFO. I assumed the VS_ part was related to Visual Studio, but turns out it's short for Version Stamp, which is the (official? unofficial?) term for that resource section.) Anyway, so far there don't appear to be any free/cross-platform equivalents to stamp(), in the Python world.

On the Perl side there seem to be a few more options:

• Win32::Resources which I can't get to compile under Linux
• Win32::PEFile which has no support for editing data, only parsing
• Win32::Exe which does compile and install under Linux

I can even use Win32::Exe to modify the data of an executable's existing version resources. For instance, this Perl program:

#!/bin/perl -w

use strict qw(subs vars refs);
use Win32::Exe;
use Data::Dumper;

my $exe = Win32::Exe->new($ARGV[0]);

print("Loaded ".$exe->filename."\n");
print("Subsystem: ".$exe->get_subsystem."\n");
print("Cannot create resource section!\n") unless $exe->can_create_resource_section;

my $inforef = $exe->get_version_info;
#my $manifest = $exe->get_manifest if $exe->has_manifest;
my @iconnames = $exe->get_group_icon_names;
print("Iconnames: ".join(' ',@iconnames)."\n");
print Data::Dumper->Dump([$inforef], ["inforef"]);

$inforef->{'ProductName'} = 'OpenShot Installer';
$inforef->{'ProductVersion'} = '2.4.4.1';
$exe->set_version_info($inforef);
$exe->write;

$inforef = $exe->get_version_info;
print Data::Dumper->Dump([$inforef], ["inforef"]);

When run on the OpenShot 2.4.4 installer generated by InnoSetup (saved as /tmp/openshot-installer.exe) it outputs:

$ perl testres.pl /tmp/openshot-installer.exe
Loaded /tmp/openshot-installer.exe
Subsystem: windows
Cannot create resource section!
Iconnames: MAINICON
$inforef = {
             'LegalCopyright' => 'Copyright (c) 2008-2016 OpenShot Studios, LLC                                                       ',
             'FileVersion' => '2.4.4.0',
             'ProductName' => 'OpenShot Video Editor                                       ',
             'FileDescription' => 'OpenShot Video Editor Setup                                 ',
             'ProductVersion' => '2.4.4.0',
             'CompanyName' => 'OpenShot Studios, LLC                                       ',
             'Comments' => 'This installation was built with Inno Setup.'
           };
$inforef = {
             'FileVersion' => '2.4.4.0',
             'CompanyName' => 'OpenShot Studios, LLC                                       ',
             'Comments' => 'This installation was built with Inno Setup.',
             'FileDescription' => 'OpenShot Video Editor Setup                                 ',
             'ProductName' => 'OpenShot Installer',
             'InternalName' => 'openshot-installer.exe',
             'OriginalFilename' => 'openshot-installer.exe',
             'LegalCopyright' => 'Copyright (c) 2008-2016 OpenShot Studios, LLC                                                       ',
             'LegalTrademarks' => '',
             'ProductVersion' => '2.4.4.1'
           };

Success! (It even adds a few properties, like OriginalFilename, and sets them automatically.)

But Win32::Exe has the same limitation as pywin32: it requires Windows APIs for some operations. That's evidenced by the can_create_resource_section method, and the fact that it returns false under Linux. So because launch.exe has no version resources at all, the same thing doesn't work if I copy it to /tmp/:

$ perl testres.pl /tmp/launch.exe
Loaded /tmp/launch.exe
Subsystem: windows
Cannot create resource section!
Iconnames: #1
$inforef = undef;
$inforef = undef;

So CLOSE... and yet still not quite there, when it comes to our specific needs.

There are, I suppose, two possible solutions to this:

• We convince cx_Freeze to add an empty version resource section to the standard bases/Win32GUI executable, or we replace it with our own that includes same. Then, the resources could be modified during the freezing process — by calling out to perl's Win32::Exe module at least, if not by the Python script itself.

• We just suck it up, abandon cross-platform solutions, and add a separate, scriptable Win32-native resource tool to the Windows build host environment. The freezing process for Win32 OpenShot has to be run on a Windows host anyway — that's already a requirement for things like the InnoSetup compiler that builds the installer, and the kSignCMD.exe tool used to sign the resulting executable. So something like verpatch.exe or rcedit.exe or the like could be used to add a version stamp to the existing pre-built binary launcher.

[ferdnyc]

The freezing process for Win32 OpenShot has to be run on a Windows host anyway — that's already a requirement for things like the InnoSetup compiler that builds the installer, and the kSignCMD.exe tool used to sign the resulting executable.

(To say nothing of the fact that bases/Win32GUI itself is only available in cx_Freeze on Win32, of course. While cx_Freeze is a cross-platform Python tool, "cross-freezing" would be impossible anyway since part of cx_Freeze's function is to collect the native libraries required to run on the current platform.)

[SuslikV]

On .rc change the Win32GUI.exe can be recompiled I think, it is small file. You don't like the method?

[ferdnyc]

On .rc change the Win32GUI.exe can be recompiled I think, it is small file. You don't like the method?

It's more an issue of, cx_freeze doesn't like that method (seemingly).

You're right, recompiling the launcher isn't really a big thing. But cx_freeze avoids doing that, preferring instead to stamp() the precompiled launcher. Now, granted, the reason cx_freeze takes that approach is that it makes for a pure-Python solution (thanks to pywin32), whereas recompiling would require external resources. We're talking about going outside of Python regardless, so to some extent that's a moot point.

Nevertheless, because it's not how cx_freeze approaches the problem, in order to compile a Win32GUI launcher with metadata included, we'd have to:

1. Locate the source code for the launcher somewhere in the project tree (as it's not normally installed with cx_freeze)
2. Make sure the necessary compiler/linker tools are available
3. Generate a .rc file from the project data
4. Compile and link an updated launcher
5. Swap out cx_freeze's launcher with our compiled version

And even if we do all that, then for anyone using a different version of cx_freeze, or if we ever decide to update our own version, the sources have to be synced up to ensure that we aren't potentially using an incompatible version of the launcher.

The path of least resistance may just be to install verpatch, and have installers/build-server.py build and execute a command line like:

verpatch_command = '"C:\path\to\verpatch.exe" "{}" /va /high {} /pv {} /s product {} /s company {}
    /s copyright {} {}'.format(os.path.join(exe_path, "launch.exe"), infoVersion, infoVersion,
    info.PRODUCT_NAME, info.COMPANY_NAME, info.COPYRIGHT);

[ferdnyc]

Heh! In fact, it turns out verpatch can handle version strings like 2.4.4-dev1 just fine, and Does The Right Thing™. This is the result of the following command:

verpatch launch.exe /va /high 2.4.4-dev1 /pv 2.4.4-dev1 /s company "OpenShot Studios, LLC" 
/s copyright "Copyright (c) 2008-2019 OpenShot Studios, LLC" /s product "OpenShot Video Editor"

[ferdnyc]

Hm, we should also make sure to set the File Description field (/s desc something in verpatch), as that's what displays in the Task Manager "Description" column. So either "OpenShot Video Editor" or "Launch OpenShot" would be appropriate.

[SuslikV]

Probably, better to not set anything until application's major issues will be fixed. Because the number of Windows users is really huge.

[jonoomph]

So, if we can successfully set the version (as is) using verpatch, I don't think this PR is needed anymore. But I still appreciate your for submitting this one, because it led us to a good solution I think (#2810)!! Thanks again... going to close it.

[SuslikV]

@jonoomph You are going against Windows tools, this is not the best decision on Windows, IMHO.

[ferdnyc]

You are going against Windows tools, this is not the best decision on Windows, IMHO.

I don't claim to be any kind of Windows expert, but from what I've been able to learn, that's not the case. Windows may use those M.N.O.P version numbers internally, but they don't require that other software follow their internal numbering scheme. Those appear to be legacy versioning, which is why for the most part they're not displayed in favor of the free-format textual representations.

So, the impression I get is that this is fine. And I got that impression from Microsoft.

Their own docs for the Windows Version Information resource include a String table that holds FileVersion and ProductVersion resource members. Here are the examples they give for those resources:

FileVersion

The Value member identifies the version of this file. For example, Value could be "3.00A" or "5.00.RC2".

ProductVersion

The Value member identifies the version of the product with which this file is distributed. For example, Value could be "3.00A" or "5.00.RC2".

Seems like we're fine, by my reading. ¯\_(ツ)_/¯

[nanohtpc]

i´m on windows can´t manipulate or recompile the software!!!... i´m loosing all my projects!!!