OSX 10.9 Dependencies

Cecil edited this page Nov 10, 2015 · 3 revisions
Clone this wiki locally

In order to build Shoes for old versions of OSX you need dependent libraries that are properly compiled. This is hard to do if, for example you want to build Shoes 10.9 from 10.10 or 10.11. The homebrew package manager is not up to that task. Trust me. So, we're going to do something similar to Windows - Build them ourselves. We do that in Linux too if we are building Shoes for distribution but in that situation apt-get work fine.

Basic Idea:

  • Get 10.9, Xcode and xcode command line tools installed in a VM in Parallels (or OracleBox - it can be done).
  • install homebrew for a few dependencies.
  • Compile Ruby and it's dependencies and install into /usr/local.
  • Build Shoes and test it in 10.9 VM. Test that it works in 10.x
  • Copy /usr/local from the VM to a directory that OSX 10.x can build 10.9 from. (Down-version build) Sounds simple. It isn't - way too many ways to do it wrong. I've found them.

OSX 10.9 install in OracleBox.

This is a giant pain in the ass! Give yourself 40GB of disk space and a couple of days. Maybe more. Even more time if you want to mount an NFS disk. You might as well create two VM's 'test' and 'develop'. Test doesn't need the NFS stuff or the Xcode. 20GB is fine for a Test VM.

get an 10.9 iso

Because I use the free beer OracleBox version there are hackintosh hurdles. One google word: Niresh

install xcode and command line tools

This takes longer than installing OSX! Gawd awful slow. You'll need an Apple ID to get it. I have one because I'm using a real Mac. I decided to get Xcode 6.1.1 and the relevant command line tools. It's the last Xcode (Dec 1, 2014) that I'm sure was building Shoes properly for 10.9 (darwin13)

get NFS or CIFS (samba) working in the VM and host.

It is very helpful (critical!) to get a decent way of moving files between your virtual machine and the VM host or some other system.

get the base set up

Because NFS is so damn slow from a OSX VM I'm going put all the stuff in ~/deps. All the downloaded tar balls and build scripts. I'm going to install into /user/local/ Once everything is working then I'll tar up the important stuff in `/usr/local/ and copy/expand that in the master build accessible directories (nfs mounted). I want to build everything in the VM but I also want to build (cross build) Shoes/OSX 10.9 from OSX 10.x

build zlib-1.2.8

This is the root of many issues. Gem nokogiri won't build without it. OSX 10.9 provides 1.2.5. Because Shoes provides it's own dylib's and otool's them ahead of the system libs we can do this. PITA and I don't trust brew to do the right thing here. And I'm correct about brew.

download, untar it and cd in.

./configure --prefix /usr/local 
make 
make install

verify that it is a x86_64 file /usr/local/lib/libz.1.dylib and otool -L

build openssl-1.0.2d

This takes damn near forever. Download it, untar, cd inside and execute

./Configure darwin64-x86_64-cc --prefix=/usr/local
make 
make install

gdbm

We don't really need this but it doesn't hurt.

#! /bin/bash
# execute it instead of ./configure
export dest="/usr/local"
export CPPFLAGS="-I${dest}/include"
export LDFLAGS="-L${dest}/lib"
./configure \
  --enable-shared \
  --enable-libgdbm-compat \
  --prefix=${dest}

libffi-3.2.1 ??

#! /bin/bash
# execute it instead of ./configure
export dest="/usr/local"
export CPPFLAGS="-I${dest}/include"
export LDFLAGS="-L${dest}/lib"
./configure \
  --prefix=${dest}

libyaml

#! /bin/bash
# execute it instead of ./configure
export dest=/usr/local
export CPPFLAGS="-I${dest}/include"
export LDFLAGS="-L${dest}/lib"
./configure \
  --enable-shared \
  --prefix="$dest"

Ruby build

OSX 10.9 has most of the things we need for a decent Ruby (2.1.7 in this case). By default, it includes Tcl and Tk gems which we don't want.

Building Ruby is not that difficult. Download 2.1.7 (this example). I put the tar ball in ~/deps cd into there. untar it tar -xvf ruby-2.1.7.tar.gz. Create this osx-ruby.sh file in ~/deps

#! /bin/bash
# copy this to the ruby dir or soft link it.
# execute it instead of ./configure
export dest="/usr/local"
export CPPFLAGS="-I${dest}/include"
export LDFLAGS="-L${dest}/lib"
./configure \
  --enable-shared \
  --enable-load-relative \
  --disable-install-doc \
  --with-openssl-dir=/usr/local/opt/openssl \
  --without-tk --without-tcllib --without-tcltk \
  --prefix="${dest}"

Then do the make and if everything looks good to you, sudo make install

We need to get ruby gems updated to something current. So check to make sure your path has /usr/local/bin ahead of everything and ruby -v returns the correct version. We also want to tell gems to not install documentation. Download rubygems (2.4.8) for me. Untar and cd.

`sudo -s`
echo "gem: --no-document" >>/usr/local/etc/gemrc
ruby setup.rb

Exit sudo. (^D)

A note about sudo: (I give myself rw permissions to /usr/local just like brew does) so it might not be required to sudo make install then again, maybe it is. Of course, you'll do a gem -v to verify. 2.4.7 for me (instead of 2.2.5).

Sometimes I need to do sudo chown -R <myname> /usr/local/* where is your account name. Before you go security crazy remember this a VM and all we use if for is building Shoes and possibly only once. Shortcuts are acceptable

Shoes deps.

gettext-0.19.4 aka libintl.8

In this case you get a gettext-latest.lz so we have to expand it with ?? How do you do that? xv.

You might be tempted to brew install gettext because unzipping a tar.lz is a little difficult on Mavericks. That is a bad idea - brew uninstall gettext. Damn kegs.

Create script osx-gettext.sh:

#! /bin/bash
# execute it instead of ./configure
export dest="/usr/local"
export CPPFLAGS="-I${dest}/include"
export LDFLAGS="-L${dest}/lib"
./configure \
  --prefix=${dest}

You might need to install java/javac first.

This probably won't go as well as I've shown here. It might take forever to configure (yes), build (yes) and install (yes). This creates /usr/local/include/libintl.h and /usr/local/lib/libintl.* which is used by damn near everything.

libpng-1.16.16

Easy

#! /bin/bash
# execute it instead of ./configure
export dest=/usr/local
export CPPFLAGS="-I${dest}/include"
export LDFLAGS="-L${dest}/lib"
./configure \
  --prefix=${dest}

Then

cd deps/libpng-1.6.16
../osx-png.sh
make
make install

NOTE: because I have access rights to write in /usr/local/* I don't need to sudo.

BIG NOTE I'm just going to show the configure script until the pattern changes (it will).

jpeg-8d (not 9)

# execute it instead of ./configure
export dest="/usr/local"
export CPPFLAGS="-I${dest}/include"
export LDFLAGS="-L${dest}/lib"
./configure \
  --prefix=${dest}

iconv-1.14

Do not build this on osx. We want to use the one in the system. You may have to rebuild everything if you did build it.

libxml-2.9.2

This one can be trouble. Nokogiri does NOT need libxml2 (it does need libxlst below). So even though I show how to build it you SHOULD NOT unless you have a gem that really, really needs it. Shoes uses /usr/lib/libexpat on OSX for both 10.9 and 10.10 (same version of expat)

#! /bin/bash
# execute it instead of ./configure
export dest="/usr/local"
export CPPFLAGS="-I${dest}/include"
export LDFLAGS="-L${dest}/lib"
./configure \
  --without-python \
  --prefix=${dest}

Note: --without-python means don't create the python bindings. They would be meaningless for this setup.

Remember: you don't need xml2 now so don't build it now.

pkg-config 0.28

Yes we need this. It's surprising I got this far without it. It can depend on glib and glib wants to use pkg-config. To break that cycle, compile pkg-config this way:

#! /bin/bash
# execute it instead of ./configure
export dest="/usr/local"
export CPPFLAGS="-I${dest}/include"
export LDFLAGS="-L${dest}/lib"
./configure \
  --with-internal-glib \
  --prefix=${dest}

glib-2.42.2

Wake up! This one is important and big and confusing. We may have to come back here to do something different. It is essential that you build your own gettext from source - DO NOT use homebrew!

export dest=/usr/local
export CFLAGS="-I${dest}/include"
export LDFLAGS="-L${dest}/lib"
export PKG_CONFIG_PATH="${dest}/lib/pkgconfig"
#export LIBFFI_CFLAGS="-I${dest}/lib/libffi-3.2.1/include"
#export LIBFFI_LIBS="-L${dest}/lib/"
./configure \
  --with-libiconv=native \
  --prefix="${dest}"

--with-libiconv= has three values, 'no','gnu', 'native' and whatever the default is. 'gnu' doesn't work here. 'native' does. default does. No idea if 'no' would work.

Yes, it does take forever to configure. And to build.

yaml-0.14 (may not be needed?)

#! /bin/bash
# execute it instead of ./configure
export dest=/usr/local
./configure \
  --enable-shared \
  --prefix="$dest"

giflib-4.2.3

#! /bin/bash
# execute it instead of ./configure
export dest="/usr/local"
export CPPFLAGS="-I${dest}/include"
export LDFLAGS="-L${dest}/lib"
./configure \
  --prefix=${dest}

Don't worry about the make error in doc.

freetype-2.5.5

#! /bin/bash
# execute it instead of ./configure
export dest="/usr/local/"
export CPPFLAGS="-I${dest}/include"
export LDFLAGS="-L${dest}/lib"
export LIBPNG_CFLAGS="-I ${dest}/include/libpng16"
export LIBPNG_LIBS="-L${dest}/lib -lpng16"
./configure \
  --enable-shared \
  --prefix=${dest}

fontconfig-2.11.1

#! /bin/bash
# execute it instead of ./configure
export dest="/usr/local"
export CPPFLAGS="-I${dest}/include"
export LDFLAGS="-L${dest}/lib"
export PKG_CONFIG_PATH="${dest}/lib/pkgconfig"
export FREETYPE_CFLAGS="-I${dest}/include/freetype2"
./configure \
  --prefix=${dest}

pixman-0.32.6

#! /bin/bash
# execute it instead of ./configure
export dest="/usr/local"
export CPPFLAGS="-I${dest}/include"
export LDFLAGS="-L${dest}/lib"
export PNG_CFLAGS="-I${dest}/include/libpng16"
export PNG_LIBS="-L${dest}/lib -lpng16"
./configure \
  --prefix=${dest}

cairo-1.40.0

See the article [Tower of Cairo] for some hints about cairo. In this case it appears that the configure does pick the quartz backend. So far, no X11 has been dragged in.

#! /bin/bash
# execute it instead of ./configure
export dest="/usr/local"
export CFLAGS="-I${dest}/include"
export LDFLAGS="-L${dest}/lib"
export png_CFLAGS="-I${dest}/include/libpng16"
export png_LIBS="-L${dest}/lib -lpng16"
./configure \
  --enable-xcb=no \
  --enable-xlib=no \
  --enable-xlib-xrender=no \
  --prefix=${dest}

harfbuzz-0.9.38

#! /bin/bash
# execute it instead of ./configure
export dest="/usr/local"
export CPPFLAGS="-I${dest}/include"
export LDFLAGS="-L${dest}/lib"
export PKG_CONFIG_PATH="${dest}/lib/pkgconfig"
export FREETYPE_CFLAGS="-I${dest}/include/freetype2"
./configure \
  --prefix=${dest}

pango-1.38.0

I'm going to try 1.38.0 because it does away with pango-querymodules which has been a pain in the behind since forever.

# execute it instead of ./configure
export dest="/usr/local"
export CPPFLAGS="-I${dest}/include"
export LDFLAGS="-L${dest}/lib"
./configure \
  --with-cairo \
  --without-xft \
  --with-included-modules=yes \
  --disable-debug --disable-gtk-doc \
  --prefix=${dest}

Gems

Shoes includes some binary gems (nokogiri, byebug). Some of these are difficult to cross compile (cough nokogiri) or down-build. Later versions of Shoes 3.2 uses prebuilt gems. These gems are gem install from the command line, and then extracted out that ruby to a place that the shoes build can use them for including into Shoes builds.

For this example I created a ~/deps/gems/ directory and we'll copy our gems from ruby to there and then we'll uninstall that gem(s).

libxlst-1.1.28

Nokogiri needs xslt to build but we do NOT want it to pickup our stuff in /usr/local/include or /usr/local/lib

#! /bin/bash
# execute it instead of ./configure
export dest="/usr/local"
./configure \
  --without-python \
  --prefix=${dest}

We do all this from where we stored the Shoes source code. 'cd ~/Projects/shoes3` for me. All paths in the example below will need to change for some other system (We're still using the 10.9 VM)

$ cd ~/Projects/shoes3
$ gem install nokogiri
$ which ruby
/usr/local/bin/ruby
$ ruby -v
ruby 2.1.7p400 (2015-08-18 revision 51632) [x86_64-darwin13.0]
$ ruby gemutils/extract1gem.rb /usr/local/lib/ruby/gems/2.1.0/specifications/nokogiri-1.6.6.2.gemspec ~/deps/gems/
copy nokogiri-1.6.6.2 from /usr/local/lib/ruby/gems/2.1.0 to /Users/ccoupe/deps/gems/
binary /usr/local/lib/ruby/gems/2.1.0/extensions/x86_64-darwin-13/2.1.0/nokogiri-1.6.6.2/gem.build_complete
Require paths ["/usr/local/lib/ruby/gems/2.1.0/extensions/x86_64-darwin-13/2.1.0/nokogiri-1.6.6.2", "lib"]
weird lib copy 
$ gem uninstall nokogiri
$ gem uninstall mini_portile

libsqlite3

Before we can install the sqlite3 gem we need to build and install the sqlite3.dylib and headers. So untar that sqlite-autoconf-3080500.tar.gz, cd inside and then run the following configure script, make, make install - just like almost everything above.

#! /bin/bash
# execute it instead of ./configure
export dest=/usr/local
export CPPFLAGS="-I${dest}/include"
export LDFLAGS="-L${dest}/lib"
./configure \
  --prefix="$dest"

That's just the /usr/local/lib/libsqlite3.dylib. Now we need to get the ruby gem installed, copied, and removed. Now do the gem install, extract1gem.rb and gem uninstall

rb-readline

I'm not going to show the commands - the only change from the nokogiri example about by gem name and how many gems were dragged in as dependencies. This is a pure ruby gem (or uses existing dylibs. gem install, extract1gem.rb and gem uninstall.

byebug

This one doesn't require building any new external libraries so you can do the gem install, extract1gem.rb and gem uninstall.

mini_portile

Part of nokogiri

columnize

part of byebug

building the Shoes exts and special gems

It's quite possible those gem install/extract1gem/uninstall brought in a newer version so you need to update your mavericks-custom.yaml to reflect any changed version numbers. For instance my mavericks-custom.yaml is:

Deps: /usr/local
Zlib: /usr/local/lib
Gemloc: /Users/ccoupe/Projects/gems/shoes
Extloc: /Users/ccoupe/Projects/gems/shoes
Exts:
  - ftsearch
  - chipmunk
Gems:
  - hpricot
  - sqlite3
InclGems:
  - mini_portile-0.6.2
  - sqlite3
  - columnize-0.9.0
  - nokogiri-1.6.6.2
  - byebug-6.0.2
  - rb-readline-0.5.3
Debug: false
CFLAGS: -DNEW_RADIO

Before you build Shoes you have one more step. Muy importante! rake shoesgems which will compile entries in the Ext: section and the special shoes gems in the Gems section and copy them to that Gemloc: location. The InclGems section is the gems that you want to copy into a new Shoes (so sqlite3 has to be in there too).

Downshift build

Since you have the dependencies for a 10.9 Shoes in /usr/local they could be copied to another system and used for building Shoes without having to do all the hand crafted compiling. Those can also be used to build Shoes/OSX 10.9 from 10.10 or 10.11. It's like a cross compile only it's for an older OS. It works if you have the proper SDK in Xcode. That is what is going to be done for Shoes 3.3. Brew has completely screwed up my 10.10 system so those brew libs can't be used to compile for 10.9. `rake osx:setup:

Librsvg

Shoes 3.3 may have a new widget for dealing with SVG (files, image display) that exposes an API for libsrvg. Of course before we can do that we need to compile librsvg and that's not the easiest task because of it's dependencies. The versions I chose do compile and link on 10.9 but they are not bleeding edge.

gdk-pixbuf 2.30.8

This builds on OSX and so far, it hasn't dragged in gtk.

#! /bin/bash
# execute it instead of ./configure
export dest="/usr/local"
export CPPFLAGS="-I${dest}/include"
export LDFLAGS="-L${dest}/lib"
./configure \
  --without-libtiff \
  --prefix=${dest}

libcroco 0.6

#! /bin/bash
# execute it instead of ./configure
export dest="/usr/local"
export CPPFLAGS="-I${dest}/include"
export LDFLAGS="-L${dest}/lib"
./configure \
  --disable-gtk-doc-html \
  --disable-Bsymbolic \
  --prefix=${dest}

libxml-2.0

Now we have to build it. - See the notes above.

librsvg 2.40.6

# execute it instead of ./configure
export dest="/usr/local"
export CPPFLAGS="-I${dest}/include"
export LDFLAGS="-L${dest}/lib"
./configure \
  --disable-tools \
  --disable-introspection \
  --disable-Bsymbolic \
  --prefix=${dest}

### how to test it

`gem install rsvg` with the Ruby/Gem that we built above. The go look into the gem../exe/rsvg2/rsvg2.bundle and make sure all the links are to our libraries.
If tiff is a problem later we'll have to build it later.