New build dir for openbsd amd64

Author: ckeen

build.openbsd64x64/HowToBuild

@@ -0,0 +1,256 @@
+How To Build On Linux
+---------------------
+
+
+Contents:
+	- Overview
+	- Checking out sources to build out-of-the-box
+	- Building out of the box
+	- Building the VM Simulator Support Libraries
+	- How to configure and build a VM on Unix
+	- Debugging configuration failures
+	- Testing an external plugin has completely linked
+	- Optimization level and gcc version (please read!)
+	- Installing support libraries
+
+
+Overview
+--------
+The "Cog" VM comes in a bewildering variety of forms.  The first distinction
+is between Squeak/Croquet VMs that run Squeak, Pharo, Cuis, Croquet images
+and their ilk, and between Newspeak VMs that run Newspeak.
+
+Another distinction is between Stack, Cog and Sista VMs.  Stack VMs are those
+with context-to-stack mapping that optimise message sending by keeping method
+activations on a stack instead of in contexts.  These are pure interpreters but
+significantly faster than the standard context-based Interpreter VM.  Cog VMs
+add a JIT to the mix, compiling methods used more than once to maxchine code on
+the fly.  Sista VMs, as yet unrealised and in development, add support for
+adaptive optimization that does speculative inlining at the bytecode-to-bytecode
+level.  These are targeted for release in 2015.
+
+Another distinction is between "v3" VMs and Spur VMs.  "v3" is the original
+object representation for Squeak as described in the back-to-the-future paper.
+Spur, as described on the www.mirandabanda.org blog, is a faster object
+representation which uses generation scavenging, lazy forwarding for fast
+become, and a single object header format common to 32 and 64 bit versions.
+
+Another distinction is between normal single-threaded VMs that schedule "green"
+Smalltalk processes above a single-threaded VM, and "multi-threaded" VMs that
+share the VM between any number of native threads such that only one native
+thread owns the VM at any one time, switching between threads on FFI calls and
+callbacks or on Smalltalk process switches when Smalltalk processes are owned
+by threads.  This multi-threaded support is as yet experimental.
+
+A distinction on linux is between VMs with an itimer hearbeat or a threaded
+heartbeat.  VMs with an itimer hearbeat  use setitimer to deliver a SIGALRM
+signal at regular intervals to interrupt the VM to check for events.  These
+signals can be troublesome, interrupting foreign code that cannot cope with
+such signals.  VMs with a threaded heartbeat use a high-priority thread that
+loops, blocking on nanosleep and then interrupting the VM, performing the same
+function as the itimer heartbeat but without using signals.  These VMs are to
+be preferred but suport for multiple thread priorities in user-level processes
+has only been available on linux in kernels later than 2.6.12.
+
+The final distinction is between production, assert and debug VMs.  Production
+VMs are fully optimized, although they may include debugging symbols, and as
+their name implies are for use in production.  Assert and debug VMs include
+many assert checks that are disabled in the production VMs.  These asserts are
+very helpful in debugging VM problems but significantly impact performance.
+The difference between assert and debug VMs is that assert VMs are compiled
+with moderate optimization, which improves the performance of the asserts,
+whereas debug VMs are compiled with no optimization at all, providing maximum
+debuggability with minimum performance.
+
+This directory tree provides build directories for some of this matrix.  For
+example, squeak.cog.v3 contains build directories for Smalltalk Cog VMs using
+the old object representation, newspeak.stack.spur contains build directories
+for Newspeak Stack VMs using the Spur object representation.  Build as desired.
+
+
+Checking out sources to build out-of-the-box
+--------------------------------------------
+Check-out the repository from github:
+	git clone http://www.github.com/OpenSmalltalk/opensmalltalk-vm oscogvm
+	cd ./oscogvm
+	more README.md
+
+
+Building out of the box
+-----------------------
+Install the tools (gcc, X11-devel, etc, e.g. libpng, libuuid libX11 & libxt
+source). See "Installing support libraries" below.  If the configure step fails
+when "checking for C compiler default output file name", you have yet to install
+all the necessary support packages (e.g. libuuid).
+Then cd to the build directory of your choice, e.g.
+	build.linux32x86/squeak.cog.spur/build
+Then execute
+	./mvm
+answering "y" to perform a clean build or "n" to rebuild without recionfiguring.
+Again, if the configure step fails when "checking for C compiler default output
+file name", you have yet to install all the necessary support (e.g. lubuuid).
+
+The subdirectories confrm to the production/assert/debug x itimer vs threaded
+heartbeat x single vs multi-threaded parts of the matrix described above.  For
+example, build.linux32x86/squeak.cog.v3 includes
+
+	build
+	build.itimerheartbeat
+	build.multithreaded
+
+	build.assert
+	build.assert.itimerheartbeat
+	build.multithreaded.assert
+
+	build.debug
+	build.multithreaded.debug
+	build.debug.itimerheartbeat
+
+subdirectories. It includes two convenience scripts that will make all
+configurations:
+	makeallclean
+	makealldirty
+
+Each build directory contains three files
+	mvm
+	plugins.int
+	plugins.ext
+The mvm script runs ../platforms/unix/config/configure with the relevant
+options, runs make, and then make install to create a VM directory tree in
+../products, ../products/assert or ../products/debug as appropriate.
+plugins.int and plugins.ext determine the set of plugins to be taken from
+the supplied plugins directory (which defaults to ../src/plugins), and which
+are to be linked into the VM (plugins.int) or compiled as external shared
+objects to be dynamically linked at run-time (plugins.ext).
+
+Finally, at the build.linux32x86 level the makeall script will run all the
+makeallclean scripts it can find.
+
+
+Building the VM Simulator Support Libraries
+-------------------------------------------
+If you want to get the Cog VM simulator working you'll need to build one or more
+of the processor simulator plugins, each of which has support libraries that
+must be built:
+	Processor   Plugin           Support Library
+	x86         BochsIA32Plugin  build.linux32x86/bochsx86
+	x86_64/x64  BochsX64Plugin   build.linux32x86/bochsx64
+	ARMv5       GdbARMPlugin     build.linux32x86/gdbarm32
+cd to the relevant directories; run conf.COG and then the build script, e.g.
+  $ cd build.linux32x86/bochsx86
+  $ ./conf.COG
+  $ ./makeem
+
+Then when Squeak VMs are built they will include the plugin(s) for which support
+libraries have been provided.
+
+
+How to configure and build a VM on Unix
+---------------------------------------
+The mvm scripts are themselves wrappers around an adaptation of Ian Piumarta's
+Squeak build system above autoconf to the Cog sources.  One can choose the vm
+source files, plugin source files, and optimization level to compile a VM of
+your choice.  To find the full set of options via
+
+	../platforms/unix/config/configure --help
+
+You can see the use of configure in the various mvm scripts in each build
+directory.  
+
+e.g.
+     ../../platforms/unix/config/configure --without-npsqueak CFLAGS="-g -O2 -msse2 -D_GNU_SOURCE -DNDEBUG -DITIMER_HEARTBEAT=1 -DCOGMTVM=0 -DDEBUGVM=0" LIBS=-lpthread
+     make install prefix=WhereYouWantTheVmToGo
+
+	N.B. If you're on a 64-bit linux read 3e below!!
+	N.B. On Ubuntu *do not* supply "LIBS=-lpthread -luuid", i.e. use
+     ../../platforms/unix/config/configure --without-npsqueak CFLAGS="-g -O2 -msse2 -D_GNU_SOURCE -DNDEBUG -DITIMER_HEARTBEAT=1 -DCOGMTVM=0 -DDEBUGVM=0"
+
+
+N.B. The plugin set is defined by plugins.ext and plugins.int in the build dir.
+
+Be prepared to install libuuid support.  e.g. on CentOS 6.5 use
+sudo yum -y install libuuid-devel
+
+
+Debugging configuration failures
+--------------------------------
+If your development system does not have the required libraries installed then
+the configure step run by the mvm script will fail cryptically.  You will see 
+something like
+	checking sanity of generated src directory... okay
+	checking build system type... i686-pc-linux-gnu
+	checking host system type... i686-pc-linux-gnu
+	checking target system type... i686-pc-linux-gnu
+
+	Configuring Squeak  (.-) for i686-pc-linux-gnu
+
+	checking whether make sets $(MAKE)... yes
+	checking for gcc... gcc -m32
+	checking for C compiler default output file name... configure: error: C compiler cannot create executables
+	See `config.log' for more details.
+But config.log is missing the relevant details.  It only contains the failing gcc
+invocation, not the output from that failing command.  To diagnose, grep to find
+the gcc invocation:
+	$ grep gcc config.log
+	...
+	configure:2269: gcc -m32 -g3 -O0 -fwrapv -DDEBUGVM=1 -DEnforceAccessControl=0 -msse2 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64  -Wl,-z,now conftest.c -lpthread -luuid >&
+and then repeat the compilation supplying your own test file, e.g.
+	$ gcc -m32 -g3 -O0 -fwrapv -DDEBUGVM=1 -DEnforceAccessControl=0 -msse2 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64  -Wl,-z,now helloworld.c -lpthread -luuid
+	helloworld.c: In function 'main':
+	/usr/bin/ld: cannot find -luuid
+	collect2: ld returned 1 exit status
+From which one can infer that one needs to install either the 64-bit or the 32-bit
+version of libuuid, e.g. sudo yum install libuuid-devel
+
+
+Testing an external plugin has completely linked
+------------------------------------------------
+You may find that an external plugin compiles and links but does not load.
+This is usually because it contans undefined symbols.  To find undefined
+symbols, remake the plugin, capturing the link step and then supply
+	-Wl,--warn-unresolved-symbols -Wl,--no-allow-shlib-undefined
+when manually repeating the link command
+
+
+Optimization level and gcc version
+----------------------------------
+There are issues with gcc version > 4.2.1.  Any of the following flags may break the build at -O2:
+-ftree-pre
+-fpartial-inlining
+-fcaller-saves
+
+So turn them off.  e.g.
+     ../../platforms/unix/config/configure --without-npsqueak CFLAGS="-g -O2 -msse2 -fno-caller-saves -fno-partial-inlining -fno-tree-pre -D_GNU_SOURCE -DNDEBUG -DCOGMTVM=0 -DDEBUGVM=0" LIBS="-lpthread -luuid"
+See http://smallissimo.blogspot.fr/2013/02/compiling-squeak-cog-virtual-machine-on.html
+
+There appear to be issues with 3.4.x gcc version on RedHat.  In particular
+compiling the Newspeak VM with either of
+	gcc version 3.4.6 20060404 (Red Hat 3.4.6-3)
+	gcc version 3.4.6 20060404 (Red Hat 3.4.6-10)
+using -O2 results in a VM that segfaults early in startup.  For these compilers
+it is probably wise to use -O1, even though -O3 seems to work.
+
+People have reported that the OSProcessPlugin is broken on gcc versions > 4.8
+on Debian.
+
+
+Installing support libraries
+----------------------------
+Different linux distributions have different mechanisms for loading packages.
+Here are some examples:
+
+CentOS
+	sudo yum install cairo-devel
+	sudo yum install pango-devel
+	sudo yum install libuuid-devel
+	sudo yum install libX11-devel
+	sudo yum install mesa-libGL mesa-libGL-devel 
+N.B. if you get failures due to version conflicts try
+	sudo yum update
+and then retry.
+
+
+Ubuntu
+	sudo apt-get install uuid-dev libcairo2-dev libpango1.0-dev libgl1-mesa-dev libgl1-mesa-glx
+
+More advice and examples for other distros gratefully received.

build.openbsd64x64/editnewspeakinstall.sh

@@ -0,0 +1,36 @@
+#!/bin/bash -e
+# Edit the installed directory tree to rename squeak to nsvm and install source
+INSTALLDIR="$1"
+shift
+cd $INSTALLDIR
+
+if [ "$1" = -source ]; then
+	SourceFile="$2"
+	shift; shift
+else
+	SourceFile=SqueakV50
+fi
+SOURCE=../../sources/$SourceFile.sources
+test -f $SOURCE || SOURCE=../../../sources/$SourceFile.sources
+if [ -f squeak ]; then
+	mv squeak nsvm
+	sed -i.bak 's/squeak/nsvm/g' nsvm
+fi
+if [ -f bin/squeak ]; then
+	mv bin/squeak bin/nsvm
+	sed -i.bak 's/squeak/nsvm/g' bin/nsvm
+fi
+rm -rf man doc
+LIBDIR="`echo lib/squeak/[0-9.-]*`"
+test -f $LIBDIR/squeak && mv $LIBDIR/squeak $LIBDIR/nsvm
+test -d lib/squeak && mv lib/squeak lib/nsvm
+LIBDIR="`echo lib/nsvm/[0-9.-]*`"
+if [ "$1" = -copysource ]; then
+	cp $SOURCE $LIBDIR
+elif [ -h $SOURCE ]; then
+	ln "`readlink $SOURCE`" $LIBDIR || (echo "can't hard link $SOURCE. Using symbolic link." 1>&2; ln -s $SOURCE $LIBDIR)
+elif [ -f $SOURCE ]; then
+	ln $SOURCE $LIBDIR || (echo "can't hard link $SOURCE. Using symbolic link." 1>&2; ln -s $SOURCE $LIBDIR)
+else
+	echo "can't find `basename $SOURCE`" 1>&2
+fi

build.openbsd64x64/editpharoinstall.sh

@@ -0,0 +1,37 @@
+#!/bin/bash -e
+# Edit the installed directory tree to rename squeak to pharo and install source
+INSTALLDIR="$1"
+
+shift
+cd $INSTALLDIR
+
+if [ "$1" = -source ]; then
+	SourceFile="$2"
+	shift; shift
+else
+	SourceFile="PharoV50"
+fi
+SOURCE=../../sources/$SourceFile.sources
+test -f $SOURCE || SOURCE=../../../sources/$SourceFile.sources
+if [ -f squeak ]; then
+	mv squeak pharo
+	sed -i 's/squeak/pharo/g' pharo
+fi
+if [ -f bin/squeak ]; then
+	mv bin/squeak bin/pharo
+	sed -i 's/squeak/pharo/g' bin/pharo
+fi
+rm -rf man doc
+LIBDIR="`echo lib/squeak/[0-9.-]*`"
+test -f $LIBDIR/squeak && mv $LIBDIR/squeak $LIBDIR/pharo
+test -d lib/squeak && mv lib/squeak lib/pharo
+LIBDIR="`echo lib/pharo/[0-9.-]*`"
+if [ "$1" = -copysource ]; then
+	cp $SOURCE $LIBDIR
+elif [ -h $SOURCE ]; then
+	ln "`readlink $SOURCE`" $LIBDIR
+elif [ -f $SOURCE ]; then
+	ln $SOURCE $LIBDIR
+else
+	echo "can't find `basename $SOURCE`" 1>&2
+fi

build.openbsd64x64/makeall

@@ -0,0 +1,9 @@
+#!/bin/bash -e
+trap 'exit 2' HUP INT PIPE TERM
+for d in *.spur *.v3; do
+	if test -d "$d"; then
+		(cd ./$d;./makealldirty "$@")
+	else
+		echo no $d directory found
+	fi
+done

build.openbsd64x64/makeallclean

@@ -0,0 +1,9 @@
+#!/bin/bash -e
+trap 'exit 2' HUP INT PIPE TERM
+for d in *.spur *.v3; do
+	if test -d "$d"; then
+		(cd ./$d;./makeallclean "$@")
+	else
+		echo no $d directory found
+	fi
+done

build.openbsd64x64/makeallmakefiles

@@ -0,0 +1,11 @@
+#!/bin/bash -e
+trap 'exit 2' HUP INT PIPE TERM
+for td in *.v3 *.spur; do
+	for d in $td/build*; do
+		if test -d "$d"; then
+			(cd ./$d;../../../platforms/unix/config/mkmf)
+		else
+			echo no $d directory found
+		fi
+	done
+done

build.openbsd64x64/makeallsqueak

@@ -0,0 +1,9 @@
+#!/bin/bash -e
+trap 'exit 2' HUP INT PIPE TERM
+for d in *.spur *.v3; do
+	if test -d "$d"; then
+		(cd ./$d;./makealldirty "$@")
+	else
+		echo no $d directory found
+	fi
+done

build.openbsd64x64/makeproduct

@@ -0,0 +1,9 @@
+#!/bin/bash -e
+trap 'exit 2' HUP INT PIPE TERM
+for d in newspeak.cog.spur squeak.cog.spur; do
+	if test -d "$d"; then
+		(cd ./$d;./makealldirty "$@")
+	else
+		echo no $d directory found
+	fi
+done

build.openbsd64x64/makeproductclean

@@ -0,0 +1,9 @@
+#!/bin/bash -e
+trap 'exit 2' HUP INT PIPE TERM
+for d in newspeak.cog.spur squeak.cog.spur; do
+	if test -d "$d"; then
+		(cd ./$d;./makeallclean "$@")
+	else
+		echo no $d directory found
+	fi
+done