Move the Windows "nightlies" over to this system?

[tkelman]

Also cc @ihnorton - something seems wrong or in need of manual intervention with the cross-compiled Windows binaries, whereas this new buildbot system looks like it's been building bottles and OSX nightlies pretty reliably. I was experimenting with making Docker images with all the cross-compiling dependencies recently, so I can help translate those into the format used here (mostly Vagrant?). Cross-compilation is doable from Ubuntu 14.04 if you use a PPA I made for the right configuration of MinGW cross-compiler, or from openSUSE, or possibly from Arch but I think the MinGW package there is already on 4.9 which might break binary compatibility with everyone's packages on Windows.

[ihnorton]

I think this might be running on the same machine I am using, but otherwise - sure, sounds good. I actually just reset things a couple hours ago on the build system - the linux julia build was broken so it couldn't do the uploads.

[tkelman]

For Ubuntu 14.04 in Docker, I was doing this:

apt-get update
apt-get install --no-install-recommends software-properties-common git make curl patch python g++ m4 p7zip-full -y
add-apt-repository ppa:ubuntu-wine/ppa -y
add-apt-repository ppa:tkelman/mingw-backport -y
dpkg --add-architecture i386
apt-get update
apt-get install --no-install-recommends wine1.7 g++-mingw-w64 gfortran-mingw-w64 -y

git clone git://github.com/JuliaLang/julia.git /home/user/julia
cd /home/user/julia
git submodule init && git submodule update
for XC_HOST in i686-w64-mingw32 x86_64-w64-mingw32; do \
      export XC_HOST; \
      echo 'LLVM_FLAGS = ac_cv_have_decl_strerror_s=no' > Make.user; \
      for i in openblas llvm fftw gmp mpfr suitesparse-wrapper arpack pcre; do \
        make -j4 -C deps install-$i || exit 1; \
      done; \
      $XC_HOST-ar cr usr/lib/libgtest.a; \
      $XC_HOST-ar cr usr/lib/libgtest_main.a; \
      tar -cJf $XC_HOST.tar.xz usr; \
      make distcleanall; \
      rm Make.user; \
      unset XC_HOST; \
    done

# to build:
echo 'VERBOSE = 1' > Make.user
echo 'override DEP_LIBS = uv openlibm openspecfun dsfmt Rmath mojibake' >> Make.user
export XC_HOST=i686-w64-mingw32 && tar -xJf $XC_HOST.tar.xz && make win-extras && make -j4 dist
# UPLOAD BINARY, TRIGGER TEST RUN, ETC
make distcleanall
rm -rf dist-extras Make.user

Although due to some strange Docker-Wine interaction, building the Julia sysimg would only work from a privileged interactive Docker session. Most VM-based systems seem fine though.

Equivalent setup steps for openSUSE 13.1:

zypper addrepo http://download.opensuse.org/repositories/windows:mingw:win32/openSUSE_13.1/windows:mingw:win32.repo
zypper addrepo http://download.opensuse.org/repositories/windows:mingw:win64/openSUSE_13.1/windows:mingw:win64.repo
zypper addrepo http://download.opensuse.org/repositories/Emulators/openSUSE_13.1/Emulators.repo
zypper --gpg-auto-import-keys refresh
zypper -n install mingw32-cross-gcc-c++ mingw32-cross-gcc-fortran
zypper -n install mingw64-cross-gcc-c++ mingw64-cross-gcc-fortran
zypper -n install mingw32-libstdc++6 mingw32-libgfortran3 mingw32-libssp0
zypper -n install mingw64-libstdc++6 mingw64-libgfortran3 mingw64-libssp0
# opensuse packages the mingw runtime dlls under sys-root/mingw/bin, not /usr/lib64/gcc
cp /usr/i686-w64-mingw32/sys-root/mingw/bin/*.dll /usr/lib*/gcc/i686-w64-mingw32/*/
cp /usr/x86_64-w64-mingw32/sys-root/mingw/bin/*.dll /usr/lib*/gcc/x86_64-w64-mingw32/*/
zypper -n install --no-recommends git make tar wine which curl patch gcc-c++ m4 p7zip.i586

[staticfloat]

I'm all for this, and cross-compilation will be easy. Do we want to try native compilation, or is cross-compilation still the best way to go forward? I have scripts for creating boxes suitable for building Julia in the julia-vagrant repository (currently only Linux and OSX, I've been thinking of trying to use the Modern IE VMs Microsoft provides for windows), and I think the first way to move forward on this is to just generate a buildslave with these packages added and see how it goes with cross-compiling.

[tkelman]

Native compilation will be slower and a bit more difficult to script, but has the advantage of being able to run the tests. As long as failures get flagged somewhere, that would be huge. Does vagrant have tools for scripting setup of Windows VM's? Presumably in Powershell?

[staticfloat]

That's right. You can read about it here. Getting CentOS building RPMs is next on my list, but if you are impatient, feel free to experiment with the windows side of things. :)

[tkelman]

Yeah, googled for and read that right after posting. Looks like the vagrant package in Ubuntu 14.04 is a little too old, I'll have to install it from the deb instead.

[staticfloat]

Yes, vagrant and packer, which is what I use for building vagrant base boxes, both need to be installed from their binary packages, as they move more quickly than the distributions.

[tkelman]

Gotcha. I can figure out packer at some point. Amazingly https://vagrantcloud.com/ferventcoder/boxes/win7pro-x64-nocm-lite actually seems to work, with an up-to-date version of vagrant. Seems to need vagrant reload a bit more often than a Linux guest.

I would lean towards using Chocolatey to install Cygwin and build from there, technically also a cross-compile but can run the tests just fine. I don't see MSYS2 in Chocolatey and the MinGW setup process tends to be a few more manual steps with MSYS2 than it is using cyg-get to install them in Cygwin. I should really get around to documenting the Cygwin cross-compile method in README.windows.md already...

[tkelman]

Slowly figuring out how vagrant provisioning works. I think a powershell script should be good enough here, I can't figure out what we would need packer for if we already have access to a Windows Vagrant box that works. Although I'm not sure how the licenses work... https://github.com/ferventcoder/vagrant-windows-puppet

The single most important line in the Vagrantfile that makes shell provisioning work for Windows guests is config.vm.communicator = "winrm".

[tkelman]

Man, when I said "Native compilation will be slower," I didn't realize it would be continental-drift slow. It's taking about a day to build all the dependencies inside a Windows VM, and still going. I have a lot more sympathy now for Appveyor with all the performance and timing-out issues we had there. Windows on bare metal is already pretty slow, but inside a VM it's just awful.

[ihnorton]

How much control do you have over the VM - can you turn off the services I
mentioned at the end of README.windows? Might help Building on my windows
box became much saner after that.

On Sat, Sep 20, 2014 at 4:00 AM, Tony Kelman notifications@github.com
wrote:

Man, when I said "Native compilation will be slower," I didn't realize it
would be continental-drift slow. It's taking about a day to build all the
dependencies inside a Windows VM, and still going. I have a lot more
sympathy now for Appveyor with all the performance and timing-out issues we
had there. Windows on bare metal is already pretty slow, but inside a VM
it's just awful.

—
Reply to this email directly or view it on GitHub
#3 (comment)
.

[tkelman]

Good question. If I can figure out how to do so via powershell, it's worth trying. This box is also apparently single core, which doesn't help. Not sure whether that's fixable without rolling a new vagrant box from scratch.

[staticfloat]

You can edit the vagrant config to set the number of cores and memory.
On Sep 20, 2014 6:38 AM, "Tony Kelman" notifications@github.com wrote:

Good question. If I can figure out how to do so via powershell, it's worth
trying. This box is also apparently single core, which doesn't help. Not
sure whether that's fixable without rolling a new vagrant box from scratch.

—
Reply to this email directly or view it on GitHub
#3 (comment)
.

[tkelman]

You can edit the vagrant config to set the number of cores and memory.

Oh. Well now I feel kinda dumb for letting it go for 20-odd hours. It even managed to crash vagrant at one point, possibly because the logs were too long? This is also running on an old machine that only has 2 cores. Anyway, it did finally finish building the dependencies, but as soon as it tries to start bootstrapping the sysimg, I get

socket: (10106) The requested service provider could not be loaded or initialized.

I'm not sure what that's about, but my guess is it has something to do with the "Unnecessary services are shut off." I'm trying netsh winsock reset catalog then restarting the VM, we'll see if that helps.