sabrown256/pactnew
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
# # Source Version: 3.0 # Software Release #: LLNL-CODE-422942 # # include "cpyright.h" # A Field Guide to Installing PACT on UNIX Systems PACT (Portable Application Code Toolkit) is a set of libraries and utilities to facilitate the development of portable software systems. It has a bias toward scientific and engineering applications especially in terms of the data handling services. It also supports shared memory and distributed parallelism. It is intended to build on any POSIX compliant computer with an ANSI standard C compiler. Consult the PACT User's Guide which is included in the document distribution for some additional information. TO INSTALL PACT IMPORTANT: You may use any directory you choose to do the install. One of your own (e.g.; ~myhome/pactdir) or one in /usr/local (e.g.; /usr/local/src). Of course you must have read/write/execute permissions for the install to work. You do not want to have a subdirectory of the directory in which you have put the distribution file named pact. The installation process will mangle it thoroughly if you do! 1) Obtain the distribution file (e.g. pact08_3_17-src) by some means and put it in a place such as /usr/local/src as follows: -> cd /usr/local/src -> cp <somewhere>/pact08_3_17-src pact-src -> chmod a+x pact-src You do not have to rename the distribution file. The example here is showing that only so that we can refer to it as pact-src throughout the remainder of this document. 2) After reading the section "PACT DISTRIBUTION BUILDER DESCRIPTION" below, run the distribution file (the C Shell must be installed on your system). Running the distribution file without arguments will give you a usage summary. Select the arguments you wish and run it again. For example: -> pact-src -h . <usage summary> . . -> pact-src -i /usr/local/pact This does the following things: configures the distribution, compiles all packages, runs all of the PACT tests, and finally installs it in the designated place (unless "-i none" is used). On a fast machine all of the above will take a few minutes and on a slow machine it will take an hour or so. NOTE: By default the directory designated for installation, if any, is assumed to exist already and must contain at least the following subdirectories: include, bin, lib, etc, man, and scheme. It and its subdirectories must be writable by you. If you add the -c option all installation directories will be created by pact-src. As the installation proceeds, most messages from component programs and scripts are routed into a file called Build.Log. When the installation completes the name is changed to <site>.BUILD and the file is moved up one directory level so as to be out of the way in case another attempt or a build on a different system is to be done next. You may wish to check this file for problems or information about the process itself. There should be no error or warning messages in a normal installation. 3) If the installation failed skip over to step (5). Now that PACT is installed. You may get any environment variable settings special to PACT by sourcing either <pact-dir>/etc/env-pact.csh or <pact-dir>/etc/env-pact.sh depending on which SHELL you are using. <pact-dir> refers to the value of the -i flag you specified when building. These should probably be put into your .cshrc or your .profile so that they appear when you log into your system. 4) If everything has completed successfully, you are done and PACT is installed on your system. Otherwise you have some work cut out for you. 5) Here is some guidance for trouble shooting. We start off with a partial list of assumptions (we try to have as few of these as possible). ASSUMPTIONS ABOUT UNIX SYSTEMS A) You have configured your environment for code development. This means that you have your path set to find compilers and debuggers, that your manpath is right to access man pages on development tools such as the compiler. If you don't know your own system DSYS CONFIG certainly can't be expected to either. It uses your environment ($path, etc) to know where to find what it needs to know. B) You have CSH installed on your system (TCSH is even better). If you do not have the C Shell on your system, install it before proceeding. C) When pact-src runs it checks the system for utilities is needs before doing anything else. It will report any missing utilities and quit if everthing it needs is not found. You should install anything is reports before trying again. WHAT TO DO TO GET PACT TO BUILD A) Read the sections below to orient yourself to PACT. When you configure/build PACT is creates a directory tree, dev, which contains the libraries, executable, and so forth. In fact you can do multiple builds from a single set of sources. Each such build has its own subdirectory of dev identified with a unique identifier that is "managed" by the pact/scripts/cfgman utility. For ease of reading below this identifier will be given as '<sid>'. In pact/dev/<sid>/log you will find the log files for configuration, analysis, building, and testing. This is the first place to go look if there are any problems. Check the config log and the analyze.xxx log files. This is not necessarily easy - the information you would need to figure out how get PACT to configure is there, but it is not always easy to spot the salient facts among the routine message. It is very difficult to predict what will fail and how. Therefore, it was deemed better to log everything than to try to generate a clean human friendly report. In general the procedure to be followed is to study the configuration summary that you see in the main build log. This is what you would see if you were to go to the pact/manager directory and do: dsys reconfig You can often see results from the analysis section that differ from what you expected or requested. From there you can look in the relevant pact/dev/<sid>/log/analysis.xxx file. This in turn will lead you to the particular tool or group variables that you need to change. Often this can be accomplished by a change to the configuration command line. You can go to the pact/manager directory and do 'dsys config -h' to see the command line options If that is not the case, you will have to create a config file with the changes or additions you need. You can start with the config file originally specified or the config file generated by PACT itself. The latter can be found in the pact/manager/gen/<arch> directory. B) Once you have settled on a proposed remedy: i) Command line: -> cd pact/manager -> dsys config <options> ii) Config file: Build a <cfg> file for your installation. For example, if your host machine is foo with its various features, you might create a file "foo". The "foo" base will be used by 'dsys config' as described below. Use one of the supplied configurations (in pact/manager/std) as an example. -> cd pact/manager -> dsys config <cfg> Consult the list of configuration parameters and the syntax description below for more information about the configuration files and how to build them. iii) Then do: C) Once the 'dsys config' looks reasonable try building PACT: -> cd pact/manager -> dsys build D) Test the distribution as follows: -> cd pact/manager -> dsys test E) Even if there are problems with some parts of PACT you can still install it so that you can use the functioning parts. To do so type the following: -> cd pact/manager -> dsys install MULTIPLE PLATFORM INSTALLATIONS If you have a network setup in which you have a single network path (independent of node) to the PACT sources, you can build PACT for each machine type on your net by doing an appropriate 'dsys config' on each different platform. This creates separate bin, lib, include, and scheme directories for each platform and keeps machine dependent files (libraries, executables, etc) out of the source directories and in the machine dependent ones. Perform the following steps (assuming for example that the sources are in /usr/local/src/pact on all platforms): 1) Follow the instructions in TO INSTALL PACT and install PACT on one system. 2) For each additional platform: a) log onto the platform (ssh, telnet, rlogin, rsh, etc) -> ssh <next-platform> b) go into the manager directory (pact/manager) -> cd /usr/local/src/pact/manager c) do a dsys config -> dsys config -c -i /usr/local/pact -i defines directory where PACT will be installed during step (2-e). d) source <pact-dir>/etc/env-pact.csh or . <pact-dir>/etc/env-pact.sh e) do a pact update to compile, test, and install PACT -> pact update This should be all that is required for this platform! If any there are any problems with (2-c) or (2-e) see the trouble shooting discussion in the TO INSTALL PACT section. PACT DISTRIBUTION BUILDER DESCRIPTION The PACT distribution is a C shell script prepended to the compressed tar file containing the PACT sources. In this fashion the distribution is self-extracting, self-configuring, self-compiling, self-testing, and self- installing. Recognizing that there is a certain minimal set of information which the installer may wish to specify, the distribution file takes command line arguments. The script will write a log file called Build.Log which should be monitored (using tail -f) for problems in the build. Here is a synopsis of the distribution file option: Usage: pact-src [-c] [-cc <C Compiler>] [-cfg] [-db <db> <path>] [-force] [-extr] [-g] [-h] [-i <dir>] [-src] [-t] [-tg <var> <value>] [-ub] [-uc] [-ui] [-ur] [-ut] [-vers] [-vrb] ... -cfg stop after configuring the distribution -extr setup/extract sources only -force force install if tests fail -h this help package -lst list some possible configurations that are available -src do not setup sources -vers print the PACT Version and exit -c create missing directories for -i option -db supplement configuration from database <path> options for <db> are: cfg configuration database run run signature database site site signature databae -f a directory containing your configuration file -g build an absolutely debuggable version -i base installation directory (default /usr/local) to omit installation use "-i none" -llp use <path> for LD_LIBRARY_PATH when analyzing compilers special value '.' for path computed from compiler location (old default) -p build a profiling version -ub do not do the build step -uc do not do the config step -ui do not do the install step -ur do not do the remove step -ut do not do the test step -v display the Version of the Source Control System -vrb run verbosely The following options apply only if no config file or database is specified: -b size of pointers in bits for platforms with option the default is determined by the platform defaults -cc specify the compiler set to be used (default gnu) Used to build with other compilers or compilers versions. For a multi-platform config a colon delimited list compilers may be specified with the order matching the individual configurations (usually frontend first then backend). Best practice is to use the full path to the compiler you select. If you use just the compiler or vendor name, PACT will try to figure out the compiler to be used. Use your PATH variable to select among multiple installed versions of a single vendor's compilers. A partial list of available compilers/vendors: clang CLANG compilers intel Intel compilers gnu GNU compilers mingw MinGW compilers open64 Open64 compilers path PathScale compilers pgi PGI compilers sun SunStudio compilers The following are multi-platform configurations for cross build situations and are safe: mp-gnu-bgq GNU for IBM BG/Q frontend and backend mp-gnu-cxe GNU for CRAY XE frontend and backend mp-gnu-win-64 GNU frontend and MinGW Win64 backend mp-pgi-cxe PGI for CRAY XE frontend and backend mp-xlc-bgq XLC for IBM BG/Q frontend and backend -epy <python-exe>[,<python-exe>]* full path to python executable(s) for which bindings will be built (useful for building against 2.7 and 3.4 python versions) -feature <feature>[,<feature>]* look in pact/manager/features for available features -mpi build MPI version (equivalent to -feature mpi) -s build OpenMP version (equivalent to -feature openmp) -sl build shared libraries (equivalent to -feature shared) -t build threaded version (equivalent to -feature pthread) -tg specify tool/group variable - used to select tool versions Config files are optional and should be used to exercise more control than is available from the command line options <config> the configuration for the installation sample configs (see manager/std) are: - auto-aix IBM RS6000 systems - auto-darwin Mac OS X systmes - auto-linux LINUX systems - auto-sol Sun Solaris systems (check manager/local for any local configs) Database files are generated by a full config after reading one or more config files and then analyzing the system. The database can be used to configure other PACT installations saving the overhead of processing config files and analyzing the system. When using a database no features or compilers should be specified - they are included in the database already. Database info cannot be overridden except for the base installation directory and overall debug/optimization/profile desired. <db> database from a previous configuration NOTE: the distribution installation process assumes that the umask may not be appropriately set for the purpose of installing PACT. It therefore sets the umask to 002. BACKGROUND AND ORGANIZATION OF PACT PACT includes the following software libraries and applications: SCORE - a low level, environment balancing library PML - a portable math library PDB - a portable binary data base management library PGS - a portable graphics library PANACEA - a portable numerical simulation code development system SCHEME - a portable Scheme interpreter SX - Scheme with extensions (PDBVIEW and ULTRA are SX programs) In the standard distribution each of the components resides in a subdirectory of the main pact directory. In addition there is a subdirectory called manager which contains various configuration files and shell scripts to automate the installation of PACT. In the following discussion, it will be assumed that there is a base directory called pact which is a subdirectory of some other directory such as /usr/local/src, so that when the directory score is referred to as pact/score, the directory /usr/local/src/pact/score is meant. To make it as easy as possible to port/maintain/install PACT on a variety of systems, PACT uses the MAKE utility to control compilation, loading, and other tasks related to management of the system. Due to differences in operating systems and site installations, the PACT system uses a three part Makefile. The first part has site dependent information and this will be the same for all of the components of PACT. This part is called make-def. The second part contains the specifics of the components and therefore each component of PACT has a separate file called pre-Make. In pact/score, pact/pml, and so on there are pre-Make files which contain dependencies particular to those parts of PACT. The third part contains a number of macros and dependencies common to all of PACT but which depend on the operating system. The utility PACT runs builds the Makefile from these pieces, runs MAKE over it, then deletes the Makefile. The reason for the deletion is that the Makefile has system dependencies and in the context of multi-platform builds such a file cannot persist indefinitely lest it interfere with the Makefile from another platform. The files make-def and make-macros are generated automatically using your supplied configuration info and an analysis of the system on which you are building. They live in the dev/<system>/etc directory thereafter. CONFIGURATION DATABASE When PACT configures itself it uses the PACT utility PERDB which stands for PERsistent DataBase. The persistent database is a database that is somewhat similar to the environment which POSIX processes manage. Unlike the environment, the persistent database transcends the usual parent/child restrictions. Any process can read, write, or query database variables. With this capability in hand it is possible for the configuration process to run programs in order to obtain configuration in addition to reading static files filled with specifications. This dynamic character make for a great deal of flexibility when trying to analyze the system on which you want to build PACT. It also means that the analysis scripts have no special protocols for passing information up the parent/child stdin/stdout/stderr chains. Each process invokes perdb directly to access the configuration database. There are a set of perdb using commands made availble for CSH (via aliases), SH, and Perl (via functions). Some of these are: dbset set an environment variable and its database counterpart dbsets set a shell variable and its database counterpart dbmset set multiple database variables in one request dbexp export an environment variable to the database dbget get an environment variable from its database counterpart dbgets get a shell variable from its database counterpart dbmget get multiple database variables in one request dbunset remove a variable from the database and the environment dbunsets remove a variable from the database and the shell variable set dbdef return 0 iff the variable exists in the databae dbini initialize the variable in the database iff it does not exist CONFIGURATION ELEMENT SYNTAX The syntax of a configuration file is fairly simple. Blank lines and lines beginning with a '#' are ignored. The following directives define the configuration specifications: 1) include <filename> this allows you to add specifications from another file 2) run <filename> this allows you to run any executable to add specifications to the configuration database 3) setenv <variable> <value> this adds "setenv <variable> <value>" to the env-pact.csh and env-pact.dk files and "export <variable>=<value>" to the env-pact.sh file 4) <var> <oper> <value> this defines or modifies variable <var> in the database <oper> defines the exact behavior and must be one of: a) '=' Standard assigment operator. This also adds "<variable> = <value>" to the make-def file. b) '+=' Appends <value> to the front of existing <variable> value. c) '=+' Appends <value> to the end of existing <variable> value. d) '-=' Removes <value> from the existing <variable> value. This is a literal string substitution ala 'sed s/<value>//'. e) '=?' Assigns <value> to <variable> only if <value> is not empty. f) '?=' Assigns <value> to <variable> only if <variable> has no existing value and <value> is not empty A special form for <value> is recognized. See the Abstract Options section below. 5) Group <foo> { this defines and groups together related variables. For example, Group Std { CC_Flags = ${CC_Flags} FC_Flags = ${FC_Flags} LD_Flags = ${LD_Flags} } This defines the variables which specify the "standard" compiles and loads. Group Cfg { Use = Std CC_Exe = gcc } This defines the variables which specify the configuration time compiles. 6) Tool <foo> { this defines the variables which specify the behavior of the tool <foo>. For example, Tool AR { Exe = /bin/ar IFlag = rc } In addition to the variable definitions there are the following special sections: ".c.i:", ".c.o:", ".c.a:", ".l.o:", ".l.a:", ".l.c", ".y.o:", ".y.a:", ".y.c", ".f.o:", and ".f.a:". These are to define the make rules for preprocessing a C source file, compiling a C source file into an object file, compiling a C source file "directly" into an archive file (if your system's make is capable of doing so), compiling a LEX source file into an object module, compiling a LEX source file directly into an archive file (make willing), processeing a LEX source file into a C source file, the equivalents for YACC source files (the .y suffix), compiling a FORTRAN source file into an object file, and compiling a FORTRAN source file directly into an archive file (make willing) respectively. These sections are begun by a line containing the section name. ABSTRACT OPTIONS Sometimes it is useful for configuration specification options to be specified in context and not be absolute. The context referred to can be the command line of PCO or the environment. In order for this to be useful the option must be abstracted so that PCO does not have to know about each and every such specification. The syntax for abstract options is: '[' <clause> [ ; <clause>] ']' <clause> := <var-clause> | <val-clause> <var-clause> := <cmd-var> | <env-var> <cmd-var> := _cmd_ '=' <arg> [ '@' ] <env-var> := _env_ '=' <env> <arg> := command line flag (e.g. -p) <env> := environment variable name <val-clause> := { on | off | <aval> } '=' <value> <aval> := abstract value name - command line argument value or environment variable value <value> := context specific value This is best illustrated by example: [ _cmd_ = -p ; on = -pg ] A '-p' on the command line results in a value of '-pg'. Without a '-p' on the command line, the value is ''. [ _cmd_ = -g ; on = -g -Wall ; off = -g ] A '-g' on the command line results in a value of '-g Wall'. Without a '-g' on the command line, the value is '-g'. [ _env_ = Bits ; 32 = -m32 ; 64 = -m64 -fPIC ] If the environment variable 'Bits' is 32 the value is '-m32', while if 'Bits' is 64 the value is '-m64 -fPIC'. Any other value of 'Bits' is an error (including an empty value). [ _cmd_ = -t @ ; * = -mtune=* ] Here the -t option requires a value. A '-t foo' on the command line results in the value '-mtune foo'. An empty value for '-t' is an error. CONFIGURATION ELEMENTS DESCRIBED Abstractly, PACT configures itself by defining and characterizing a set of tools (e.g. C compiler, linker, and so on). Groups of tools are defined to build different parts of PACT in particular ways according to their requirements and roles. If the tools are properly defined and characterized then their use in the various groups will also be well defined and PACT will be able to build and test successfully. Conversely (in the majority of cases), if PACT fails to build or test, it is due to a failure to properly characterize the tools used to build it. This is almost a definition and it is the goal is to make it a strict definition. TOOLS You can look at the contents of pact/manager/std/program-init for a complete list of the default Tool and Group sets used to build PACT. Here are a few to give you the idea of the schema. CC Specification of C compiler. Variables: Exe, Linker, Flags, Version, Debug, Optimize, Inc, MDGInc, DPInc Defaults are: Exe = cc, Linker = "", Flags = "", Debug = "-g", Optimize = "-O", Inc = "", MDGInc = "", DPInc = "". FC Specification of Fortran compiler. Variables: Exe, Linker, Flags, Version, Debug, Optimize, Inc, MDGInc, DPInc Defaults are: Exe = f90, Linker = "", Flags = "", Debug = "-g", Optimize = "-O" CXX Specification of C++ compiler. Variables: Exe, Linker, Flags, Version, Debug, Optimize, Inc, MDGInc, DPInc Defaults are: Exe = g++, Linker = "", Flags = "", Debug = "-g", Optimize = "-O" LD Specification of loader/linker. Variables: Exe, Flags, Version, Lib, MDGLib, DPLib Defaults are: Exe = ld, Flags = "", Lib = "", MDGLib = "", DPLib = "". Ar Specifications to use ar. IFlag is specific to insert and XFlag is specific to extract operation of ar. Variables: Exe, Flags, Version, IFlag, XFlag Defaults are: Exe = ar, Flags = "", IFlag = <computed>, XFlag = <computed>. Lex Specifications to use lex (the lexical scanner generator). Variables: Exe, Flags, Version Defaults are: Exe = lex, Flags = "". Ranlib Specifications to use ranlib (not often needed anymore). Variables: Exe Defaults are: Exe = touch. GROUPS There are several groups defined by various configuration elements depending on requested features. Three in particular are always defined. Std the standard group Cfg the configuration group Cfe the front end group You cannot/should not eliminate or subvert these groups, but it may be helpful, when tracking down problems, to understand the central role that they play. OS In pact/manager/os you fill find config file elements that define operating specific elements used to build PACT. aix IBM AIX freebsd FreeBSD hurd GNU Hurd lnx Linux osx OS X solaris Solaris COMPILERS In pact/manager/compilers you fill find config file elements that define compilers that can be used to build PACT. When you specify a compiler in the command line or in a config file, the actual information about the compiler is found in one of the pact/manager/compilers files. If you want to build PACT with a new compiler, you would have to write a compilers file for it. You would do best, in that case, to clone one of the existing files. A partial list of compilers is: clang description of Clang compilers gnu description of GNU compilers icc description of Intel compilers mingw description of MinGW compilers path description of PathScale compilers pgi description of PGI compilers sol description of old Solaris compiler sun description of Sunstudio compilers xlc description of IBM XL compilers Sometimes the OS and compiler files are closely related and several hybrid files are available: clang-bsd Clang on FreeBSD clang-lnx Clang on Linux clang-osx Clang on OS X gnu-cygwin GNU on Cygwin gnu-freebsd GNU on FreeBSD gnu-lnx GNU on Linux gnu-osx GNU on OS X gnu-sol GNU on Solaris icc-lnx Intel on Linux icc-osx Intel on OS X pgi-lnx PGI on Linux pgi-osx PGI on OSX xlc-aix IBM XL on AIX FEATURES In pact/manager/features you fill find config file elements that define various configuration features. For example, if you want to configure a build with OpenMP you would add: include features/openmp in your config file or add: -feature openmp to the 'dsys config' command line. A partial list of features is: 32-bit build 32 bit binaries and libraries 64-bit build 64 bit binaries and libraries c11 use ANSI C 11 features c89 use ANSI C 89 features c99 use ANSI C 99 features cx11 use ANSI C 11, POSIX and XOPEN features cx89 use ANSI C 99, POSIX and XOPEN features cx99 use ANSI C 89, POSIX and XOPEN features dld link dependent libraries dynamically dls link dependent libraries statically mpi build with and for MPI no-fortran do not build any Fortran bindings no-jpeg do not build with JPEG no-png do not build with PNG no-python do not build Python bindings no-sql do not build with with SQLITE or MySQL no-x11 do not build with X11 openmp build with OpenMP pthread build with Pthreads python build Python bindings if possible shared build shared PACT libraries SPECIAL CIRCUMSTANCES For the most part PACT is configured and built on computers like workstations and servers. There are certain machines which don't fit a simple mold like that. Some supercomputers have complex architectures which optimize high performance parallel execution. In order to try to be portable the PACT configuration system has to model such platforms as well as the simple platforms on which it is built most of the time. To accomodate this, the PACT configuration system has a multi-platform model. This provides a complete framework to build PACT consistently for more than a single target platform. Examples of multi-platform builds would be: 1) Linux/Windows build. All libraries and executables are build for each platform, but utilities such as the do-run utility are used to configure, and test the Windows build under Wine. 2) Front End/Back End supercomputer build. All the libraries are built for each platform. Not all utilities are built for the back end because is makes no sense to run them there. Again do-run is used to run configuration and test executables on the back end. Multi-platforms builds are not be limited to two platforms. For example, building for a heterogenous environment where some nodes have differents kinds of special purpose co-processors. There are four kinds of compilations that are done when configuring and building PACT: 1) small test programs which probe the features of the system; 2) infrastructure utilities used for building PACT; 3) building the libraries and utilities; and 4) programs for the test suite. The multi-platform model requires that the individual configurations for each platform must be fully realized. The model addresses the issue of running certain executables on a platform other than the one on which the build is being done. In the multi-platform model, the category (1), (3), and (4) compilations are done on the primary platform but targeted to the configuration of each platform where they are run via do-run. The category (2) compilations are done on the primary platform and also run there. Category (3) and (4) compilations will in general have platform specific compiler flags. Category (1) and (2) compilations are always serial and minimalist in some sense. Finally, in the config files the compiler and flags for (3) and (4) are specified in the standard group Std. The group Cfg is used to specify category (2) compilations. Depending on the system, the category (1) compiler and flags could be the same as (2) or (3), that is to say either the Cfg or the Std group. The configuration process decides whether the Cfg or Std group is needed to build category (1) programs and creates a new group Cfe which contains the result. Some examples: 1) DO-RUN and SMAKE must be built with group Cfg to run on the front end 2) TEMPLATE must be built with group Cfe to run on the back end 3) SCTTY must be built with group Std to run parallel on the back end 4) the libraries must be built with group Std for the back end 5) ULTRA and SX follow the libraries 6) DMAKE also follows the libraries but must run on the front end and so it will not work if the target cannot also run on the front end Along with this we distinguish to kinds of frontending: configurational (configuration time) and parallel (run time). There are two variables that control this: CROSS_FE for CROSS platform front ending DP_FE for parallel front ending
About
Automatically exported from code.google.com/p/pactnew
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published