Building MRI on Windows

jonforums edited this page Nov 19, 2011 · 23 revisions

To hack on a fix you must first find a way to easily and repeatedly build MRI on Windows, preferably using a toolchain you're already familiar with. This HOWTO will show you three ways to build MRI on Windows. We expect you'll automate these methods to minimize build distractions. We also hope to provide cross-platform Ruby or Python based build helpers, and would greatly appreciate contributions in this area.

If you're familiar with MinGW GCC toolchains, we will show you two ways of building that do not require that you to first assemble an MSYS + MinGW toolchain. If you're familiar with the Microsoft toolchain, we will show you one way to build using the Command Prompt from your Visual Studio IDE or the freely available Windows SDK for Windows 7.

In all three cases you will be building a profilable and debuggable minimal MRI rather than a fully functioning Ruby environment such as those available from the RubyInstaller project. This means that capabilities like RubyGems and certain extensions like Psych and OpenSSL will not be available. We believe the root causes of the performance problems are contained in the core code rather than 3rd party libraries. By building just the core, the (re)build process is much faster and much less complex.

All three build methods require the following prerequisites:

  • A functioning MRI installation such the latest 1.9.x version from the RubyInstaller project on your PATH
  • A functioning Windows Git installation such as msysGit
  • A local clone of The Code Shop MRI repository. For the following examples we'll assume your local clone resides in C:\ruby-src

Building with the DevKit

  1. Download the latest DevKit from our Downloads page. and extract it into a directory with no spaces. For this example we'll use C:\DevKit

  2. Open a new Command Prompt (cmd.exe), checkout a new local tracking branch, and create a build subdirectory in your local MRI clone. For this example, we'll create a local branch tracking the upstream origin/jonforums/ruby_1_9_3 branch. You'll do something similar when investigating work on other upstream branches.

    cd C:\ruby-src
    mkdir build
    git checkout -b jonforums/ruby_1_9_3 origin/jonforums/ruby_1_9_3
    
  3. Add the DevKit tools to your PATH and run the familiar ./configure && make && make install build sequence. In this example we'll install to C:\projects\ruby193-opt, but you can change the location by specifying a different value for the --prefix option to configure. Ensure that the install path contains no spaces. Your freshly built and installed MRI can now be profiled by AQtime Standard.

    C:\DevKit\devkitvars.bat
    sh -c "autoconf"
    cd build
    sh -c "../configure --prefix=C:/projects/ruby193-opt --enable-shared --disable-install-doc optflags='-O3' debugflags='-gstabs+'"
    make
    make test
    make install
    
  4. A typical profile/benchmark-hack-rebuild-test cycle in which you'll count on make to update the build directory based upon your source mods looks like:

    # profile/benchmark
    # hack
    make
    make test 
    # delete old install
    make install
    
  5. If you need to return to a pristine state and fully cleaning your source tree, run the following. You'll typically not need to perform this step if you're making smallish source code modifications, rebuilding, testing, and profiling/benchmarking.

    cd C:\ruby-src
    del revision.h configure
    rmdir /s autom4te.cache
    sh -c "autoconf"
    cd build
    make distclean
    del *.* && rmdir /s enc && rmdir /s .ext
    

Building with the RubyInstaller recipes

Building with the RubyInstaller recipes is a much more automated process, but still has a couple of caveats. First, the recipes must download all required dependencies, which can take a long time depending upon your network connection. Network problems may require that you manually delete partially downloaded artifacts and try again. If you experience weird problems, verify that no 0-byte files exist in the downloads subdirectory of your local RubyInstaller clone. Also verify that none of the downloads were corrupted during the download. If you have 0-byte or corrupted dependencies, delete them and try again.

Once the recipes have successfully downloaded everything, using the recipes to (re)build is a pleasant experience.

  1. Open a new Command Prompt (cmd.exe) and checkout a new local tracking branch in your local MRI clone. For this example, we'll create a local branch tracking the upstream origin/jonforums/ruby_1_9_3 branch. You'll do something similar when investigating work on other upstream branches.

    cd C:\ruby-src
    git checkout -b jonforums/ruby_1_9_3 origin/jonforums/ruby_1_9_3
    
  2. Create a local clone of the RubyInstaller repository into a directory with no spaces. Foor this example, we'll clone into C:\rubyinstaller.

    cd C:\
    git clone https://github.com/oneclick/rubyinstaller.git rubyinstaller
    cd rubyinstaller
    
  3. Create a build configuration override file to prevent RubyInstaller from building 3rd party dependencies, and ensure MRI is built with the correct symbols needed to profile with AQtime Standard. Override files are normal Ruby files (of any name) that live in the override subdirectory of the your local RubyInstaller repository; C:\rubyinstaller\override in this example. Create a C:\rubyinstaller\override\profiling_build.rb with the following content.

    if ENV['AQTIME'] then
      puts '[INFO] Overriding to enable AQtime profiled Ruby 1.9.x...'
    
      RubyInstaller::Ruby19.dependencies.clear
    
      RubyInstaller::Ruby19.configure_options << "optflags='-O3'"
      RubyInstaller::Ruby19.configure_options << "debugflags='-gstabs+'"
    end
    
  4. Coax RubyInstaller into downloading the required dependencies and build MRI from your local MRI source repository using the TDM GCC 4.5.2 toolchain.

    rake clean
    rake ruby19 local=C:\ruby-src notk=1 nogems=1 dkver=tdm-32-4.5.2 aqtime=1
    rake devkit:sh
    cd sandbox/ruby19_build
    make test
    exit
    
  5. A minimal but functional MRI environment is now installed to the sandbox\ruby19_mingw subdirectory; C:\rubyinstaller\sandbox\ruby19_mingw in this example. It can be copied to another location of your choice. You may also run rake ruby19:package:archive to create a 7z archive and accompanying MD5 checksum file in the pkg subdirectory. Another option is to use the sandbox\ruby19_mingw install directory as-is for your analysis. In that case, a typical profile/benchmark-hack-rebuild-test cycle looks like.

    COMING SOON
    
  6. To fully clean your source tree and build sandbox while keeping the previously downloaded dependencies cached in downloads, run the following. You'll typically not perform this step if you're making smallish source code modifications, rebuilding, testing, and profiling/benchmarking.

    rake clean
    cd C:\ruby-src
    del revision.h configure
    rmdir /s autom4te.cache
    

Building with Windows SDK for Windows 7

Dependencies:

  • The DevKit from our Downloads for the bison and sed dependencies
  • MS Visual Studio 2010, Windows SDK for Windows 7, or a similar build environment

Instructions:

  1. Extract the DevKit into a directory with no spaces. We'll use C:\DevKit for this example.
  2. Start a 'Visual Studio Command Prompt' from the IDE. If you're using the Windows SDK for Windows 7, start a Command Prompt by selecting the following from the Start menu:

    All Programs -> Microsoft Windows SDK v7.1 -> Windows SDK 7.1 Command Prompt
    

    Or you may wish to configure a shortcut on the Desktop with a Target: similar to:

    C:\Windows\System32\cmd.exe /E:ON /V:ON /T:0E /K "C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\SetEnv.Cmd" /x86 /release /win7
    
  3. Clone the ruby repo and build. If running configure.bat causes a fatal error U1034: syntax error : separator missing error, ensure cmd.exe is configured with /V:OFF. While fixed in MRI trunk it has not yet been backported.

    mkdir thecodeshop
    cd thecodeshop
    git clone git://github.com/thecodeshop/ruby.git ruby-src
    cd ruby-src
    mkdir build && cd build
    C:\DevKit\devkitvars.bat
    ..\win32\configure.bat --prefix=C:/thecodeshop/ruby --disable-install-doc --disable-win95
    nmake
    nmake test
    nmake install
    C:\thecodeshop\ruby\bin\ruby -e "puts 'hello world!'"