Can we get an MSI installer for something past 3.4.4?

[DGentry]

BPO	31148
Nosy	@pfmoore, @tjguk, @bitdancer, @zware, @zooba, @indygreg
Files	Python 3.6.2 (64-bit)_20170808103648.log

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-08-10.13:24:11.338>
created_at = <Date 2017-08-08.16:38:31.085>
labels = ['type-bug', '3.7', 'OS-windows']
title = 'Can we get an MSI installer for something past 3.4.4?'
updated_at = <Date 2021-12-04.14:57:11.167>
user = 'https://bugs.python.org/DGentry'

bugs.python.org fields:

activity = <Date 2021-12-04.14:57:11.167>
actor = 'zach.ware'
assignee = 'none'
closed = True
closed_date = <Date 2017-08-10.13:24:11.338>
closer = 'steve.dower'
components = ['Windows']
creation = <Date 2017-08-08.16:38:31.085>
creator = 'D Gentry'
dependencies = []
files = ['47066']
hgrepos = []
issue_num = 31148
keywords = []
message_count = 17.0
messages = ['299936', '299942', '299946', '299961', '299994', '300059', '300082', '300111', '300113', '300185', '300262', '300322', '300325', '300398', '300432', '309573', '309592']
nosy_count = 8.0
nosy_names = ['paul.moore', 'tim.golden', 'r.david.murray', 'zach.ware', 'steve.dower', 'indygreg', 'D Gentry', 'macoptimizer']
pr_nums = []
priority = 'normal'
resolution = 'works for me'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue31148'
versions = ['Python 3.5', 'Python 3.6', 'Python 3.7']

[DGentry]

I've noticed that the only installers that are able to successfully complete on my windows 7 x64 system are the MSI installers.

Unfortunately, python.org hasn't released an MSI version since 3.4.4 it seems.

Is it possible to get an MSI installer for one of the more recent versions?

[bitdancer]

The short answer is no. We no longer use the MSI installer.

Perhaps the windows experts will be interested in exploring why you can't use the current installers, since they work for most people.

[zooba]

Looking at your log, you clearly have some sort of configuration blocking installers or else corruption somewhere on your system. There really isn't enough information available for me to be able to tell.

Perhaps you have a virus scanner running that is blocking the installer?

[DGentry]

If that's the best thing that you can suggest, it's obvious why my request
is being redirected.

That is unfortunate that you have decided to opt out of using the MSI
installer, as that seems to be the solution as far as I can see.

I've enjoyed working with Python for years and was hoping for a new MSI
installer to be released eventually, but hearing that you have decided to
discontinue porting for any future release for MSI is definitely not what I
was expecting.

I guess I'll wait to see if your Windows guys have anything to add but save
it to say that I have already tried disabling any anti virus software,
checked my user permission as well as the folder permissions, and attempted
installing via administrator as well, all with no difference in the result.
Meanwhile, I have been able to uninstall python 2.7 and re-install it
without issue (at the time I was investigating a possible path conflict as
the issue) and even installed 3.4.4rc1, as that is the most current version
available in MSI format.

Your exe files all fail to execute properly, and ends up throwing the same
non-descript error in a log that offers very little insight into the root
of the problem, yet you tell me the problem is with my system when I have
investigated the alternatives and the only issue I have found is you don't
have a MSI version offered any more.

Are you expecting users to compile the source code into a MSI installer, if
that's what they require?

If that's what it's going to take, I'll do it myself, get what I need done
and move on from python after this project.

On Tue, Aug 8, 2017 at 2:57 PM, Steve Dower <report@bugs.python.org> wrote:

Changes by Steve Dower <steve.dower@python.org>:

----------
type: crash -> behavior

---

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue31148\>

---

[pfmoore]

To be clear, Steve *is* our main Windows developer, and specifically the person who developed the Windows installers we now use. They work perfectly for many people, including myself, so there certainly isn't a general issue with them. I myself routinely install Python with the new installers on Windows 7 x64 systems, so it's definitely something specific to your system.

Is your PC running a "corporate" build that might include security restrictions or other permission limitations that might be affecting the behaviour of the installer?

As a workaround, Python is installable using nuget, from https://www.nuget.org/packages/python. I don't know much myself about nuget, but basically "nuget install python" will create a local copy of Python in the current directory that you can use without needing to be installed.

[DGentry]

Ok, it sounds like as good a solution as I can expect.

No, I don't have a corporate build but I think my registry developed issues
some time in the past and windows locked it down to prevent exe from
installing.

It's been a while but that is the only thing I can come up with that would
be why it won't work.

Your exe isn't the only one that gives me issues but its usually a false
positive error with other installs and everything works as expected.

I have tried every suggestion I have found online and nothing seems to
improve the issue.

If the problem is the registry issue that I suspect, then nothing short of
rebuilding the entire system from initial install would solve the problem.

I have communicated with Microsoft, who's answer was to upgrade to Windows
10.

I have two systems with python installed, both with windows 7 x64, the
older system won't install without an MSI where as the new one would.

The closest thing I can figure is there is some permission handling that is
done differently via MSI vs executable installs and this is causing the
issue, as all the documentation online seems to agree that it is a
permissions related error.

I can't fix my system if the problem that I suspect is the culprit so my
only option is to build the MSI of your installer.

It definitely seems like that will be more successful than continuing to be
directed and redirected to perform the same tasks that have already been
completed to no avail.

From what I have been reading online, its simple enough with Visual Studio
as long as you have access to the source code.

Thanks for enlightening me to the most of your abilities.

It seems pretty clear that I on my own on this one.

On Wed, Aug 9, 2017 at 6:10 AM, Paul Moore <report@bugs.python.org> wrote:

Paul Moore added the comment:

To be clear, Steve *is* our main Windows developer, and specifically the
person who developed the Windows installers we now use. They work perfectly
for many people, including myself, so there certainly isn't a general issue
with them. I myself routinely install Python with the new installers on
Windows 7 x64 systems, so it's definitely something specific to your system.

Is your PC running a "corporate" build that might include security
restrictions or other permission limitations that might be affecting the
behaviour of the installer?

As a workaround, Python is installable using nuget, from
https://www.nuget.org/packages/python. I don't know much myself about
nuget, but basically "nuget install python" will create a local copy of
Python in the current directory that you can use without needing to be
installed.

----------

---

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue31148\>

---

[zooba]

Okay, if that's the way you want to go. Be aware that the old MSI code is gone though, so rebuilding it will not be trivial. Though if you do, there are a few people who would find it more convenient, so you may be able to enlist help maintaining it (we didn't get any offers of help, just requests for free work, so you may not get it either...)

If you're having trouble with a range of installers, there's a process you can run to reset your security policy, which is what I'd suggest doing. I'd have to find the details for you, but it basically resets permissions on everything. Doesn't delete any files, but may help with this problem. Let me know if you'd like me to find the details (requires Windows Pro IIRC, at least pre-Win 10).

[bitdancer]

Steve, when we changed installers was that when we also fixed the security/permissions problems with the install dir? If permissions are the issue the OP's problem may have nothing to do with it not being msi.

[zooba]

when we changed installers was that when we also fixed the security/permissions problems with the install dir

Yes, but the permissions issue here isn't the install directory - it is probably the TEMP directory or some other system restriction. It's basically impossible to tell from the logs, and I don't even know where to start asking for configuration settings to see what may be wrong.

[DGentry]

If you would like to keep track of others who would find it useful, I'd be
happy to contribute the msi to the community once its completed.

Also, any information or good references that you could recommend that
might aid in producing the installer would be appreciated.

Steve mentioned the temp directory as possibly being part of the problem
and that would make sense as its only when installer make use of this that
the issues seem to occur so I will definitely be looking at that as a
possible root cause for the failure.

Steve also mentioned that it's not as easy to produce an installer as it
may seem.I was wondering if you could elaborate some on what you mean.

On a scale of one to ten, with ten being the most difficult, how difficult
of a task would you say it is to create this as it seems you have some
experience doing this in the past and this will be my first attempt, so I
would like to know what I'm getting into before I start.

From the Microsoft article on producing MSI installers,
https://msdn.microsoft.com/en-us/library/aa266427(v=vs.60).aspx#vehowbuildinginstallerpackagefileanchor1,
it seems fairly simple as long as you have Visual Studio.

Is there something I'm not seeing or understanding in the process?

Again, if you know of any references that might help educate me or direct
me through the process, these would be helpful.

I already have a couple strategies I'm thinking of to do this but I would
like to see what someone with some experience has to say regarding the
issue.

-David Gentry

On Thu, Aug 10, 2017 at 2:40 PM, Steve Dower <report@bugs.python.org> wrote:

Steve Dower added the comment:

> when we changed installers was that when we also fixed the
security/permissions problems with the install dir

Yes, but the permissions issue here isn't the install directory - it is
probably the TEMP directory or some other system restriction. It's
basically impossible to tell from the logs, and I don't even know where to
start asking for configuration settings to see what may be wrong.

----------

---

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue31148\>

---

[zooba]

The general requirements of an installer are:

• runs without any preinstalled prerequisites
• does not install anything permanent until the user says so (Visual Studio breaks this rule :( )
• installs all required prerequisites
• install on all supported operating systems
• allow users to [de]select optional components
• correctly uninstall previous compatible versions
• correctly do *not* uninstall previous identical versions
• update user environment settings (registry, environment, etc.)
• create Start Menu shortcuts
• create file associations
• create context menus
• install files into customizable location with correct security settings
• correctly roll back when installation fails, including restoring previously uninstalled components and attempting to execute installed tools that may not be installed correctly
• attempt to execute just-installed components that may not be installed correctly (e.g. ensurepip, compileall, trigger environment variable reload)
• detect and abort when installation is going to fail, and explain to the user how to solve the problem
• provide live progress feedback and information about what operations are occurring
• provide detailed log files to help offer user support post-installation, and help users find those logs
• provide registration information so the operating system can help users modify or remove the installation
• allow users to modify, repair and upgrade the installation without corrupting it (or any others) or forcing them to remove/reinstall
• allow users to choose whether to install just for themselves or for the entire machine, and request correct permissions based on this choice
• allow administrators to script an entire installation and have it run silently and/or with progress (and cancellation) only
• allow users to minimize download size by deselecting optional components
• allow users to obtain all optional components to allow full installation without public internet access
• install without executing arbitrary third-party code that may have been maliciously placed in a user's download directory
• automatically include new files added by developers who have not explicitly modified the installer

These are all the features of our current installer, and nearly all of them work for basically every user. Many of these are literally not achievable with just an MSI, or require significantly more complicated commands to be run by the end user, rather than providing options in a user interface.

If you're still interested in giving it a go, visit http://wixtoolset.org/ and have a look at their tools. These are the gold-standard right now for building MSIs (and Burn bundles, which is what we currently have).

[DGentry]

Thank you for the information. That will definitely be helpful.

Also, thank you for being so detailed with your explanation.

Even though it will be difficult and may take more time than I was
initially intending, I believe its within the realm of possibility.

Therefore, I will be trying my hand at crafting the MSI, though like you
said earlier, it's not nearly as easy as the msdn article made it appear.

The toolset you offered should help out a lot.

Thank you for all of your expert advice.

I have one last question before I begin on my journey.

You mentioned "burn bundles" at the end of the message. Are these something
I would have to build myself or is there a repository where I can download
the files?

On Mon, Aug 14, 2017 at 12:55 PM, Steve Dower <report@bugs.python.org>
wrote:

Steve Dower added the comment:

The general requirements of an installer are:

• runs without any preinstalled prerequisites
• does not install anything permanent until the user says so (Visual
  Studio breaks this rule :( )
• installs all required prerequisites
• install on all supported operating systems
• allow users to [de]select optional components
• correctly uninstall previous compatible versions
• correctly do *not* uninstall previous identical versions
• update user environment settings (registry, environment, etc.)
• create Start Menu shortcuts
• create file associations
• create context menus
• install files into customizable location with correct security settings
• correctly roll back when installation fails, including restoring
  previously uninstalled components and attempting to execute installed tools
  that may not be installed correctly
• attempt to execute just-installed components that may not be installed
  correctly (e.g. ensurepip, compileall, trigger environment variable reload)
• detect and abort when installation is going to fail, and explain to the
  user how to solve the problem
• provide live progress feedback and information about what operations are
  occurring
• provide detailed log files to help offer user support post-installation,
  and help users find those logs
• provide registration information so the operating system can help users
  modify or remove the installation
• allow users to modify, repair and upgrade the installation without
  corrupting it (or any others) or forcing them to remove/reinstall
• allow users to choose whether to install just for themselves or for the
  entire machine, and request correct permissions based on this choice
• allow administrators to script an entire installation and have it run
  silently and/or with progress (and cancellation) only
• allow users to minimize download size by deselecting optional components
• allow users to obtain all optional components to allow full installation
  without public internet access
• install without executing arbitrary third-party code that may have been
  maliciously placed in a user's download directory
• automatically include new files added by developers who have not
  explicitly modified the installer

These are all the features of our current installer, and nearly all of
them work for basically every user. Many of these are literally not
achievable with just an MSI, or require significantly more complicated
commands to be run by the end user, rather than providing options in a user
interface.

If you're still interested in giving it a go, visit http://wixtoolset.org/
and have a look at their tools. These are the gold-standard right now for
building MSIs (and Burn bundles, which is what we currently have).

----------

---

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue31148\>

---

[zooba]

Burn bundles are part of Wix, but it's the part you don't like :)

If you just want a single MSI, you won't want a bundle. But you will have to leave out nearly half of the features I listed because they simply are not possible without it.

[DGentry]

So basically what your saying is that the MSI that I compile will only work
for my specific situation as I'm not building it with a bundle?

Hmm, that kind of sucks.

Disregarding the difficulty of manually converting from source to MSI, if I
was to choose to produce a version that included all the features you
listed so it was able to be considered a full MSI release so others could
benefit as well,
would it be possible to get a bundle, even if was only for some of the
older releases?

I have a strategy I am considering employing to solve the problem where the
more data I can get, the better it will all come together.

Basically, I have a 5x raspberry pi 3 neural network unit that I am
planning on getting here within the next couple of weeks to experiment with
machine learning.
Considering the situation, I believe this may be a good task to apply a
convolution neural net to solve.

The problem is essentially converting the raw source code into the
corresponding MSI, via Wix.
If I use the old release's source and MSI repositories as models, I believe
it should be possible to train the system to do the conversion for future
versions as well.
The more data I have to feed the system, such as the bundles you mentioned,
would definitely help increase the accuracy of the algorithm more quickly,
but I'm not sure if this would be something Python.org would be willing to
share.
Even without the bundles, I think it will still be achievable.
It just may take the system longer to attain sufficient accuracy as it
would have to extrapolate more from the raw data.

Is this an idea that intrigues you even slightly?
Even if its a long shot, I think the benefits would be worth the attempt
and the vast majority of the investment of time and resources would be my
responsibilty.
After the fact, should the algorithm succeed, I would be happy to share all
my data with python.org regarding the experiment so you could begin
providing MSI releases again with minimal effort on your part.

Should you feel like there is something that would inherently and
irrefutably inhibit the process, making this a completely hopeless
endeavor, I will resign myself to producing the one-time use, stripped down
version you were describing previously.

I just hate investing a bunch of time into a project and the code I create
is basically useless after I use it once, but I do respect your opinion
therefore wanted to submit the proposal for your consideration before
diving down the rabbit hole.

<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=icon\>
Virus-free.
www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=link\>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Tue, Aug 15, 2017 at 10:12 PM, Steve Dower <report@bugs.python.org>
wrote:

Steve Dower added the comment:

Burn bundles are part of Wix, but it's the part you don't like :)

If you just want a single MSI, you won't want a bundle. But you will have
to leave out nearly half of the features I listed because they simply are
not possible without it.

----------

---

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue31148\>

---

[zooba]

The idea interests me from a theoretical point of view, but then I did my PhD in that field. I don't believe it's a practical solution, if only because you would need to manually generate the result in order to validate before releasing, and that's the step you're going to avoid.

I don't think there's anything left to pursue here. Good luck with your efforts.

[indygreg]

I was going to update some CI that tests all supported versions of Python and pins the Python 3 versions. That CI was previously using the MSI installers for Python 3.4. To my surprise, the MSIs stopped being generated after the 3.4.4 release! https://www.python.org/ftp/python/3.4.4/ has MSIs (and MacOS .pkg files). But all subsequent releases (e.g. https://www.python.org/ftp/python/3.4.7/) do not.

I'm OK with the decision to abandon MSIs and move to the .exe installer for newer Python releases (like 3.5 and 3.6). However, ceasing to produce the MSIs (and MacOS .pkg files for that matter) midway through 3.4's support cycle feels wrong to me. 3.4.4 was the last release with binary distributions for Windows and MacOS. 3.4.5 and newer releases have fixes to security issues. That means people using the binary distributions of 3.4 on these platforms are stuck on 3.4.4 and are thus vulnerable to known security issues. That's... not great.

I think the MSI (and .pkg) distributions should be restored for 3.4. Furthermore, I would encourage Python to adopt the practice that distribution mechanisms aren't modified during a Python version's support cycle. Someone will inevitably rely on any distribution format that is published. And taking it away in a support cycle will orphan those users on an old, buggy release.

[zooba]

It is standard procedure for us to stop producing binary releases when a version switches to security-only mode. The burden on our volunteers would become overwhelming otherwise.

If you'd like to propose a change to this policy, start by posting on python-list, and then if there is general support someone there will bring the conversation to the development team. Resurrecting old bugs is not an efficient way to request this kind of change.