Switch branches/tags
Find file History
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
.settings
CVS
c_src
f77_src
inc
lib/CVS
.cdtproject
.cproject
.cvsignore
.gdb_history
.project
Makefile
README
README-KAMELEON-CONVERSION-INSTALL-HELP.txt

README

________________________________________________________________________________

KAMELEON Conversion Software Version 5.2.0                            10.05.2010

Copyright © 1999-2009 United States Government as represented by Administrator
for The National Aeronautics and Space Administration. All Rights Reserved

NASA Goddard Space Flight Center
Community Coordinated Modeling Center
http://ccmc.gsfc.nasa.gov

Government Agency Original Software Designation: GSC-15540-1
Government Agency Original Software Title: Kameleon Software Suite
_______________________________________________________________________________

TECHNICAL CONTACT:

Marlo Maddox
NASA Goddard Space Flight Center
Science Data Processing Branch
Phone: 301.286.5202
email: Marlo.Maddox@nasa.gov

________________________________________________________________________________

LEGAL STUFF

Copyright © 1999-2009 United States Government as represented by Administrator
for The National Aeronautics and Space Administration. All Rights Reserved

For Specific Licensing and Software Usage Agreements  Info, Contact:

Melissa Jackson SGT/NASA                ||  Ms. Dale L. Hithon Clarke
Innovative Partnerships Program Office  /\  Innovative Partnership Program Office
Phone: 301-286-7958                     OR  Phone: (301)286-2691
Fax:  301-286-0301 Fax                  \/  Fax:  (301)286-0301
Email: Melissa.S.Jackson@nasa.gov       ||  Email: Dale.L.Hithon@nasa.gov
http://ipp.gsfc.nasa.gov                ||  http://ipp.gsfc.nasa.gov
________________________________________________________________________________

DESCRIPTION:

The KAMELEON conversion software generates standardized versions of ingested
model output files.  The generated file will contain all of the data from the
original output file as well as additional metadata elements that describe the
entire data-set, individual variables, and the grid(s).  Additional variables
may also be added to the resulting output file.

________________________________________________________________________________

SUPPORTED MODELS:

The KAMELEON conversion software currently supports the following models & data:

Magnetosphere:

1.    BATSRUS:      BATS-R-US, the Block-Adaptive-Tree-Solarwind-Roe-
                    Upwind-Scheme, was developed by the Computational
                    Magnetohydrodynamics (MHD) Group at the University
                    of Michigan, now Center for Space Environment
                    Modeling (CSEM)

1b.                 BATSRUS component of Space Weather Modeling Framework (SWMF)

2.    OpenGGCM:     The Open ( formaerly UCLA/NOAA ) Geospace General
                    Circulation Model (GGCM)  was originally developed as a
                    magnetohydrodynamic ( MHD ) model of Earth's magnetosphere
                    at UCLA in the early 1990's by J.Raeder.

Ionosphere:

3.    ctip:         The coupled thermosphere-ionosphere-plasmasphere (CTIP)
                    model consists of three distinct components:
                    * A global thermosphere model;
                    * A high latitude ionosphere model;
                    * A mid and low-latitude ionosphere/plasmasphere model.
                    Authors/Developers: Dr. Timothy Fuller-Rowell et al NOAA SEC

Heliosphere:

4.    ENLIL:        ENLIL is a time-dependent 3D MHD model of the heliosphere.
                    It solves for plasma mass, momentum and energy density, and
                    magnetic field,using a Flux-Corrected-Transport (FCT)
                    algorithm.
                    Model Authors/Developers: D. Odstrcil - University of
                    Boulder, Colorado

Solar:

5.    MAS           MAS is a MHD Model of Solar Corona. Model domain: 1 - 30
                    solar radii. Model Authors/Developers: J. Linker, Z. Mikic,
                    R. Lionello, P. Riley
                    Science Applications International Corporation (SAIC)
                    San Diego, California

________________________________________________________________________________

SUPPORTED SCIENCE DATA FORMATS

The KAMELEON conversion software will currently convert any supported Model
output to the following Science Data Formats

1.    cdf:         Common Data Format ( distributions tested : V2.7, V3.0, V3.1 )
                   (Common Data Format) developed by National Space Science
                   Data Center ( NSSDC ) http://nssdc.gsfc.nasa.gov

________________________________________________________________________________


REQUIREMENTS:

* CDF Library:      The KAMELEON conversion software uses CDF Library
                    (Common Data Format) developed by National Space Science
                    Data Center ( NSSDC ) http://nssdc.gsfc.nasa.gov

* argtable2.4:      Argtable 2.4 is an ANSI C library for parsing GNU
                    style command line arguments developed by
                    Stewart Heitman http://argtable.sourceforge.net/
                    sheitmann@users.sourceforge.net

* g77 & gcc:        The GNU fortran and c compilers have been used in
                    development and testing.

                    NOTE: g77 vs gfortran & libg2c vs libgfortran

                    The gnu compiler suite version 4.1 changed the fortran
                    compiler from g77 to gfortran.  To compile certain
                    fortran interface components, the shared library libg2c
                    ( used with g77 ) or libgfortran ( used with gfortran ) is
                    needed.

                    Once you install/locate the required components, keep note
                    of their locations as they will need to be specified during
                    the invocation of make on Kameleon.

                    See the BUILDING KAMELEON section below for more details.

* cxform:           The cxform package contains a set of routines to convert
                    spacecraft coordinates from one system to another.

                    Originally written by Ed Santiago (LANL)
                    Extended & maintained by Ryan Boller (NASA/GSFC)
                    http://nssdcftp.gsfc.nasa.gov/selected_software/
                    coordinate_transform/

                    As of 07.2008 kameleon-converter-v4.7.0 uses cxform to
                    automatically calculate and update the dipole titlt for
                    BATSRUS & SWMF.BATSRUS

                    Specific cxform source code ( Version 0.0.7 ) pre-packaged
                    and used by kameleon:

                        c_src/cxform-auto.c
                        c_src/cxform-manual.c
                        inc/cxform.h

________________________________________________________________________________


OPTIONAL LIBRARIES THAT CAN BE BUILT INTO KAMELEON:

The following distributions are required to read certain model output types.
In their absence, the KAMELEON Conversion software will still build but it will
not be able to read any model output that requires the specific libraries if
they are missing.

* NetCDF            The netcdf library is required to read model output
                    that is originally stored in the netcdf format i.e. enlil

                    Developed with netcdf-3.6.0-.p1
                    http://www.unidata.ucar.edu/packages/netcdf/

                    REQUIRED TO READ: ENLIL data

* HDF               The HDF library is required to read model output
                    that is originally stored in the HDF4 format i.e. MAS

                    NOTE:  HDF has several dependencies itself.  The following
                    libraries are all used by HDF:

                    - libjpeg.so
                    - libz.so
                    - libsz.so

                    REQUIRED TO READ: MAS data

                    NOTE: The SD interface used by HDF requires the "libmfhdf"
                    library. However, if HDF is to be built and used along
                    with NetCDF, an existing namespace conflict will arise as
                    both libraries share the same internal function names.  To
                    counteract this problem, the HDF/MFHDF distribution must be
                    compiled/configured with the appropriate option.

                    Setting the appropriate flag will inform the compiler to
                    rename the HDF version of the C-interface ( nc* ) of the
                    netCDF API tp sd_nc*.

                    HDF4.2r1 - used the '-DHAVE_NETCDF' compiler option

                    HDF4.2r3 - uses the '--disable-netcdf' argument to configure

                    Check your local HDF install documentation to determine what
                    is required for your particular version.

* FITS              The FITS library is required to read model output that is
                    originally stored in the FITS format ...

                    REQUIRED TO READ: MSFC_TVM and KPVT Solar Magnetogram Data

________________________________________________________________________________

SUPPORTED OS PLATFORMS:

- Software Tested on:

        Sun Solaris

        MAC OSX

        Fedora

        RedHat Linux

        Cygwin

________________________________________________________________________________

BUILDING KAMELEON

To build the Kameleon converter, cdf and argtable2 must already be installed.

Optionally, HDF, netCDF, and FITS can also be built into kameleon to support the
ingestion of data that is stored in these formats.  If you need to install any
of these libraries, see the "REQUIRED AND OPTIONAL LIBRARY NOTES" section of
this README file.

To build the kameleon converter, invoke make with the following variables and
set the values = the description/location of each specific resource ie:

    make LIB_CDF=/Applications/cdf31-dist/lib/libcdf.a ...

    where ... could be a combination of the following options:

Required?     Description              Variable            Example
--------     -----------------        ------------        ----------------------
   -          C Compiler               CC                  CC=gcc
   -          FORTRAN Compiler         F77                 F77=gfortran
  NO          Special Library Dir      LIB_PATH            LIB_PATH=-L/weird_library_path/lib

  YES         CDF Library              LIB_CDF             LIB_CDF=/usr/lib/libcdf.a
  YES         CDF Include Files        CDF_INC             CDF_INC=/usr/cdf31-dist/include/
  YES         ARGTABLE2 Library        LIB_ARG_TABLE_2     LIB_ARG_TABLE_2=../argtable2-6/src/libargtable2.a
  YES         ARGTABLE2 Include Files  ARG_TABLE_2_INC     ARG_TABLE_2_INC=../argtable2-6/src/

  NO          netCDF Library           LIB_NETCDF          LIB_NETCDF=/usr/local/lib/libnetcdf.a
  NO          netCDF Include Files     NETCDF_INC          NETCDF_INC=/usr/local/include/
  NO          FITS Library             LIB_FITS            LIB_FITS=../cfitsio/libcfitsio.a
  NO          FITS Include Files       FITS_INC            FITS_INC=../cfitsio/

  NO          HDF4 df Library          LIB_DF              LIB_DF=/Applications/HDF4.2.3/lib/libdf.a
  NO          HDF4 mfhdf Library       LIB_MFHDF           LIB_MFHDF=/Applications/HDF4.2.3/lib/libmfhdf.a
  NO          HDF4 Include Files       HDF_INC             HDF_INC=/Applications/HDF4.2.3/include/
  NO          HDF4 SZ Compression Lib  LIB_SZ              LIB_SZ=/usr/local/lib/libsz.a
  NO          HDF4 Z Compression Lib   LIB_Z               LIB_Z=/usr/local/lib/libz.a
  NO          HDF4 JPEG Dependency     LIB_JPEG            LIB_JPEG=/usr/local/lib/libjpeg.a

________________________________________________________________________________

REQUIRED AND OPTIONAL LIBRARY NOTES:

CDF ( REQUIRED ) :

  CDF needs to be installed prior to the installation of the Kameleon
  Conversion software ( Preferably  version 3.0 or higher ). See the
  README.install file located in the cdf distribution for instructions.

ARGTABLE 2 ( REQUIRED ) :

  argtable2.4 needs to be installed.  See install.txt located in the argtable2.4
  distribution.

  The KAMELEON conversion software requires the libcdf.a & libargtable2.a static
  libraries.  Proper installation of both cdf and argtable2 should produce
  these two files.  Take note of their locations as these two libraries will
  be used to build kameleon.  You can either copy the libraries into kameleon's
  local ./lib directory, or specify the library locations during the invocation
  of make on kameleon.

  See BUILDING KAMELEON section for specific Makefile options

  The corresponding header/include files for the cdf and argtable2 libraries
  will also be required.  Specifically cdf.h and artable2.h.  These header files
  can be placed directly into kameleon's local ./inc directory, or their
  locations can be specified during the invocation of make on kameleon.

  See BUILDING KAMELEON section for specific Makefile options

                            and/or

  See INCLUDE FILE section of README for more detail on incorporating external
  include files into kameleon build.

    NOTES ON Updating the cdf and argtable2 ".a" static archives if copying the
    libraries directly into kameleon's local ./lib directory:

    To update the archives table of contents if you have copied the libraries
    into the local lib directory for the Kameleon Converter, you should run
    ranlib on the newly copied/relocated libraries:

      ranlib libargtable2.a
      ranlib libcdf.a


NETCDF ( OPTIONAL ) :

  The libnetcdf.a static library is used by the KAMELEON conversion software to
  read model output files that are originally stored in the netcdf .nc format.

  NOTE: The converter will still build if this library is not available, but you
  will not be able to read and convert model output files that are in the netCDF
  format such as enlil.

  See the INSTALL file located in the src directory of the netcdf distribution
  for details.

  The KAMELEON conversion software uses the libnetcdf.a static library.  Proper
  installation of netcdf should produce this file.

  The netcdf.h header/include file will also be required.  This header file
  can be placed directly into kameleon's local ./inc directory, or its location
  can be specified during the invocation of make on kameleon.

  See INCLUDE FILE section of this README for more detail on incorporating external
  include files into kameleon build.

  NOTES ON Updating the libnetcdf ".a" static archive if copying the
  library directly into kameleon's local ./lib directory:

  To update the archive table of contents if you have copied the library
  into the local lib directory for the Kameleon Converter, you should run
  ranlib on the newly copied/relocated library:

      ranlib libnetcdf.a

HDF ( OPTIONAL ) :

  The HDF library is used by the KAMELEON conversion software to read model
  output files that are originally stored in the hdf .hdf format.

  NOTE: The converter will still build if this library is not available but you
  will not be able to read and convert model output files that are in the HDF
  format such as MAS.

  The HDF distribution yields two main libraries

    libmfhdf.a
    libdf.a

  HDF itself also uses three supporting shared libraries

    libjpeg.so
    libz.so
    libsz.so

  !!!NOTE: The SD interface used by HDF requires the "libmfhdf" library as noted
  above. However, if HDF is to be built along with NetCDF, an existing namespace
  conflict will arise as both libraries share the same internal function names.
  To counteract this problem, the HDF/MFHDF distribution must be compiled with
  the option '-DHAVE_NETCDF'.  This will inform the compiler to rename the HDF
  version of the C-interface ( nc* ) of the netCDF API tp sd_nc*.

  Installation Notes:

  *For some versions of HDF ie 4.2r1 we need to compile hdf/mfhdf with the flag:

                         -DHAVE_NETCDF

  Thsi may require you to add it to the os specific config file located in the
  config directory. For example, if building hdf on gnu linux, we edit the file:

              ./config/linux-gnu ie CFLAGS="-DHAVE_NETCDF"

  Other options include modifying the Makefiles under the mfhdf, hdf, and/or the
  main distribution directories.  Be sure to confirm that -DHAVE_NETCDF option
  bieng used when the distribution is compiling.

  *The lastest version of HDF4 tested with Kameleon required the:

            --disable-netcdf option be spesified duringthe configure stage.


  Check your local HDF install documentation to determine what is required for
  your particular version of HDF.

  Assuming that libjpeg and libz are already installed, you can configure the
  hdf distribution by executing:

  ./configure --with-zlib=/path_2_zlib_directory \
  --with-jpeg=/path_2_jpeg_directory --prefix=/path_2_install_directory \
  --with-szlib=/path_to_SZIP_install_directory \
  --disable-netcdf

  set your LD_LIBRARY_PATH=location_of_libz.so:location_of_libjpeg.so

  on a mac use DYLD_LIBRARY_PATH

  and build the libraries using

  gmake
  gmake check
  gmake install

  The corresponding header/include files for the HDF library will also be
  required. Since the HDF distribution has a buch of include files for itself,
  its best to keep them seperate and reference the group collectively with the
  macro LOCAL_HDF_INC which can be specified during the invocation of make on
  kameleon.

  See INCLUDE FILE section of README for more detail on incorporating external
  include files into kameleon build.

FITS:

  The libcfitsio.a static library is used by the KAMELEON conversion software to
  read model output files that are originally stored in the fits .fts format.

  NOTE: The converter will still build if this library is not available but you
  will not be able to read and convert model/data output files that are in the
  FITS format.

  The FITS distribution yields the library

    libcfitsio.a

  The fitsio.h & longnam.h header/include files will also be required.  These
  header files can be placed directly into kameleon's local ./inc directory, or
  their locations can be specified during the invocation of make on kameleon.

  See INCLUDE FILE section of README for more detail on incorporating external
  include files into kameleon build.

  NOTES ON Updating the libcfitsio ".a" static archive if copying the
  library directly into kameleon's local ./lib directory:

  To update the archive table of contents if you have copied the library
  into the local lib directory for the Kameleon Converter, you should run
  ranlib on the newly copied/relocated library:

      ranlib libcfitsio.a
________________________________________________________________________________

INCLUDE FILES:

  By default, the kameleon makefile is configured to look in kameleon's local
  inc/ directory for all of the required header files that are both specific to
  kameleon as well as the exernal libraries that kameleon uses.

  External include file can either be copied to kameleon's local inc/ directory
  or specified during the invocation of make using specific MACRO's

  **Required Include files**

    cdf.h - can be set using the CDF_INC macro ie.

        make CDF_INC=../location_of_your_local_cdf_inc_directory

    argtable2.h - can be set with the ARG_TABLE_2_INC macro ie

        make ARG_TABLE_2_INC=../location_of_your_local_argtable_inc_directory

  **Optional Include files**

    -If reading FITS files:

    fitsio.h
    longnam.h

    -If reading netCDF files:

    netcdf.h

    -If reading HDF files

  NOTE:  The HDF distribution has a bunch of include files by itself so keep those
  seperate and reference in the MAKEFILE or during invocation of make with the
  macro:

    HDF_INC = ...

________________________________________________________________________________

MAKEFILE:

The Makefile provided with the kameleon distribution should not need to be
modified directly.  All of the user configurable options can be specified during
the invocation of make as specified under the BUILDING KAMELEON section of this
README file.

    you can always do a:

        make help


  With the proper compilers, static libraries, header files, and library flags
  identified.

  to compile type:
    make CDF_INC=../your_cdf_inc_dir/ LIB_CDF=../your_dir/libcdf.a \
    LIB_ARG_TABLE_2=../your_dir/libargtable2.a ARG_TABLE_2_INC=../your_argt_inc_dir/

    or also include the optional libraries as well

  This should produce an executable named kameleon

  to clean the build directory type:
    make clean
    or
    make real_clean

  NOTE: make real_clean removes the libraries and executables generated by the
  Kameleon conversion software.

________________________________________________________________________________

USAGE:

You can always type:

  kameleon -h

to print a synopsis of the available command line options.

%kameleon -h
This program converts model output into a standardized format
Usage: ./kameleon  [--version] [-h|--help] [-v|-V|--verbose] [--nominmax] -f|--format=string -m|--model=string [-a|--aux_file=string] [-d|--database=string] [-i|--indir=string] [-o|--outdir=string] [<file>]...
 --version                 print software version number
 -h, --help                print this help
 -v, -V, --verbose         turn verbose output on
 --nominmax                turn OFF actual min/max calculations for each variable
 -f, --format=string       data format to convert to
 -m, --model=string        model name
 -a, --aux_file=string     auxiliary input file i.e.. a grid file separate from general data files
 -d, --database=string     ccmc DatabaseInfo file location - for CCMC use only
 -i, --indir=string        directory containing input files to be converted
 -o, --outdir=string       output directory where converted files will be placed default is ./
 <file>                    input files
%

example 1: converting a batsrus file

kameleon -v -f cdf -m batsrus -o ./my_output_directory ./my_input_directory/my_batsrus_dot_out_file

example 2: converting an open_ggcm file

kameleon -v -f cdf -m ucla_ggcm -o ./my_output_directory -a ./my_grid_file ./my_input_directory/my_3df_file

NOTE:  The use of the -a auxiliary input flag to specify the location of the grid file


To quickly browse cdf files, a useful command line tool is skeletontable.  Typing:

  ../your_cdf_distribution/bin/skeletontable your_cdf_file.cdf

produces an ascii text file named your_cdf_file.cdf.skt.  You

________________________________________________________________________________

KNOWN BUGS:

FIXED       1.  Using the wild-card meta character * in the file name argument
                may cause unexpected command line parsing errors or cause
                the program to crash

            2.  OpenGGCM/UCLA-GGCM - pre 4.4.1 kameleon converter versions
                assumed variables where in a specific order in the original
                files. If/when this order changes, some variables may not have
                be read in.  Fixed by verifying read return status and rewinding
                file pointer to re-read as necessary.  This rewind functionality
                was originally available but turned off as it makes the read 4x
                slower.  Old functionality either had rewind on or off, now it
                is set dynamically as required.

            3.  ENLIL reading with -d option: pre 4.4.1 would segfault or report
                a bus error when reaching the clouds_start_times entry in the
                CCMC DatabaseInfo file.  Fixed by verifying date string length >
                1 before calling the date_formatter date processing function

TESTING     4.  When using the -indir argument, some files may not be accessible
                do to system reporting "too many open files error". Files that
                could not be accessed will be skipped.

TESTING     5.  When using the -indir argument, superfluous files may not always
                be ignored as they should. ie. an existing .cdf file may be read
                into the program as input

WORKAROUND  6.  NEW SWMF-BATSRUS .out files were missing the NX,NY,NZ special
   |            parameters required for number_of_blocks calculation.  New
   |            functionality searches the DatabaseInfo file as an alternative
   |            and exits program if those parameters cannot be found.
   |
   |            Also missing were the P1,P2,P3 special parameters as well.
   |
   |            The workaround is to add the required info as key-value pairs to
   |            the DatabaseInfo file. The DatabaseInfo file is used at the CCMC
   |            for the Runs-On-Request System, but external user can generate a
   |            simple text file as well.  Either way the following lines need
   |            to exist:
   |
   |                    6    # batsrus_special_parameter_NX
   |                    6    # batsrus_special_parameter_NY
   |                    6    # batsrus_special_parameter_NZ
   |                    2    # batsrus_special_parameter_P1
   |                    1    # batsrus_special_parameter_P2
   |                    1    # batsrus_special_parameter_P3
   |
   |            ...adjusting the numbers in the first column as necessary.
   |
   |            You can tell the converter where the DatabaseInfo file is with
   |            the -d option.
   v
FIXED       6.  AS of v4.7.0, the read batsrus module in now capable of
                calculating/determmining the missing special paramenters without
                having to specify them in an external DatabaseInfo file.