Fetching latest commit…
Cannot retrieve the latest commit at this time.
|Type||Name||Latest commit message||Commit time|
|Failed to load latest commit information.|
________________________________________________________________________________ KAMELEON Conversion Software Version 5.2.0 10.05.2010 Copyright © 1999-2009 United States Government as represented by Administrator for The National Aeronautics and Space Administration. All Rights Reserved NASA Goddard Space Flight Center Community Coordinated Modeling Center http://ccmc.gsfc.nasa.gov Government Agency Original Software Designation: GSC-15540-1 Government Agency Original Software Title: Kameleon Software Suite _______________________________________________________________________________ TECHNICAL CONTACT: Marlo Maddox NASA Goddard Space Flight Center Science Data Processing Branch Phone: 301.286.5202 email: Marlo.Maddox@nasa.gov ________________________________________________________________________________ LEGAL STUFF Copyright © 1999-2009 United States Government as represented by Administrator for The National Aeronautics and Space Administration. All Rights Reserved For Specific Licensing and Software Usage Agreements Info, Contact: Melissa Jackson SGT/NASA || Ms. Dale L. Hithon Clarke Innovative Partnerships Program Office /\ Innovative Partnership Program Office Phone: 301-286-7958 OR Phone: (301)286-2691 Fax: 301-286-0301 Fax \/ Fax: (301)286-0301 Email: Melissa.S.Jackson@nasa.gov || Email: Dale.L.Hithon@nasa.gov http://ipp.gsfc.nasa.gov || http://ipp.gsfc.nasa.gov ________________________________________________________________________________ DESCRIPTION: The KAMELEON conversion software generates standardized versions of ingested model output files. The generated file will contain all of the data from the original output file as well as additional metadata elements that describe the entire data-set, individual variables, and the grid(s). Additional variables may also be added to the resulting output file. ________________________________________________________________________________ SUPPORTED MODELS: The KAMELEON conversion software currently supports the following models & data: Magnetosphere: 1. BATSRUS: BATS-R-US, the Block-Adaptive-Tree-Solarwind-Roe- Upwind-Scheme, was developed by the Computational Magnetohydrodynamics (MHD) Group at the University of Michigan, now Center for Space Environment Modeling (CSEM) 1b. BATSRUS component of Space Weather Modeling Framework (SWMF) 2. OpenGGCM: The Open ( formaerly UCLA/NOAA ) Geospace General Circulation Model (GGCM) was originally developed as a magnetohydrodynamic ( MHD ) model of Earth's magnetosphere at UCLA in the early 1990's by J.Raeder. Ionosphere: 3. ctip: The coupled thermosphere-ionosphere-plasmasphere (CTIP) model consists of three distinct components: * A global thermosphere model; * A high latitude ionosphere model; * A mid and low-latitude ionosphere/plasmasphere model. Authors/Developers: Dr. Timothy Fuller-Rowell et al NOAA SEC Heliosphere: 4. ENLIL: ENLIL is a time-dependent 3D MHD model of the heliosphere. It solves for plasma mass, momentum and energy density, and magnetic field,using a Flux-Corrected-Transport (FCT) algorithm. Model Authors/Developers: D. Odstrcil - University of Boulder, Colorado Solar: 5. MAS MAS is a MHD Model of Solar Corona. Model domain: 1 - 30 solar radii. Model Authors/Developers: J. Linker, Z. Mikic, R. Lionello, P. Riley Science Applications International Corporation (SAIC) San Diego, California ________________________________________________________________________________ SUPPORTED SCIENCE DATA FORMATS The KAMELEON conversion software will currently convert any supported Model output to the following Science Data Formats 1. cdf: Common Data Format ( distributions tested : V2.7, V3.0, V3.1 ) (Common Data Format) developed by National Space Science Data Center ( NSSDC ) http://nssdc.gsfc.nasa.gov ________________________________________________________________________________ REQUIREMENTS: * CDF Library: The KAMELEON conversion software uses CDF Library (Common Data Format) developed by National Space Science Data Center ( NSSDC ) http://nssdc.gsfc.nasa.gov * argtable2.4: Argtable 2.4 is an ANSI C library for parsing GNU style command line arguments developed by Stewart Heitman http://argtable.sourceforge.net/ email@example.com * g77 & gcc: The GNU fortran and c compilers have been used in development and testing. NOTE: g77 vs gfortran & libg2c vs libgfortran The gnu compiler suite version 4.1 changed the fortran compiler from g77 to gfortran. To compile certain fortran interface components, the shared library libg2c ( used with g77 ) or libgfortran ( used with gfortran ) is needed. Once you install/locate the required components, keep note of their locations as they will need to be specified during the invocation of make on Kameleon. See the BUILDING KAMELEON section below for more details. * cxform: The cxform package contains a set of routines to convert spacecraft coordinates from one system to another. Originally written by Ed Santiago (LANL) Extended & maintained by Ryan Boller (NASA/GSFC) http://nssdcftp.gsfc.nasa.gov/selected_software/ coordinate_transform/ As of 07.2008 kameleon-converter-v4.7.0 uses cxform to automatically calculate and update the dipole titlt for BATSRUS & SWMF.BATSRUS Specific cxform source code ( Version 0.0.7 ) pre-packaged and used by kameleon: c_src/cxform-auto.c c_src/cxform-manual.c inc/cxform.h ________________________________________________________________________________ OPTIONAL LIBRARIES THAT CAN BE BUILT INTO KAMELEON: The following distributions are required to read certain model output types. In their absence, the KAMELEON Conversion software will still build but it will not be able to read any model output that requires the specific libraries if they are missing. * NetCDF The netcdf library is required to read model output that is originally stored in the netcdf format i.e. enlil Developed with netcdf-3.6.0-.p1 http://www.unidata.ucar.edu/packages/netcdf/ REQUIRED TO READ: ENLIL data * HDF The HDF library is required to read model output that is originally stored in the HDF4 format i.e. MAS NOTE: HDF has several dependencies itself. The following libraries are all used by HDF: - libjpeg.so - libz.so - libsz.so REQUIRED TO READ: MAS data NOTE: The SD interface used by HDF requires the "libmfhdf" library. However, if HDF is to be built and used along with NetCDF, an existing namespace conflict will arise as both libraries share the same internal function names. To counteract this problem, the HDF/MFHDF distribution must be compiled/configured with the appropriate option. Setting the appropriate flag will inform the compiler to rename the HDF version of the C-interface ( nc* ) of the netCDF API tp sd_nc*. HDF4.2r1 - used the '-DHAVE_NETCDF' compiler option HDF4.2r3 - uses the '--disable-netcdf' argument to configure Check your local HDF install documentation to determine what is required for your particular version. * FITS The FITS library is required to read model output that is originally stored in the FITS format ... REQUIRED TO READ: MSFC_TVM and KPVT Solar Magnetogram Data ________________________________________________________________________________ SUPPORTED OS PLATFORMS: - Software Tested on: Sun Solaris MAC OSX Fedora RedHat Linux Cygwin ________________________________________________________________________________ BUILDING KAMELEON To build the Kameleon converter, cdf and argtable2 must already be installed. Optionally, HDF, netCDF, and FITS can also be built into kameleon to support the ingestion of data that is stored in these formats. If you need to install any of these libraries, see the "REQUIRED AND OPTIONAL LIBRARY NOTES" section of this README file. To build the kameleon converter, invoke make with the following variables and set the values = the description/location of each specific resource ie: make LIB_CDF=/Applications/cdf31-dist/lib/libcdf.a ... where ... could be a combination of the following options: Required? Description Variable Example -------- ----------------- ------------ ---------------------- - C Compiler CC CC=gcc - FORTRAN Compiler F77 F77=gfortran NO Special Library Dir LIB_PATH LIB_PATH=-L/weird_library_path/lib YES CDF Library LIB_CDF LIB_CDF=/usr/lib/libcdf.a YES CDF Include Files CDF_INC CDF_INC=/usr/cdf31-dist/include/ YES ARGTABLE2 Library LIB_ARG_TABLE_2 LIB_ARG_TABLE_2=../argtable2-6/src/libargtable2.a YES ARGTABLE2 Include Files ARG_TABLE_2_INC ARG_TABLE_2_INC=../argtable2-6/src/ NO netCDF Library LIB_NETCDF LIB_NETCDF=/usr/local/lib/libnetcdf.a NO netCDF Include Files NETCDF_INC NETCDF_INC=/usr/local/include/ NO FITS Library LIB_FITS LIB_FITS=../cfitsio/libcfitsio.a NO FITS Include Files FITS_INC FITS_INC=../cfitsio/ NO HDF4 df Library LIB_DF LIB_DF=/Applications/HDF4.2.3/lib/libdf.a NO HDF4 mfhdf Library LIB_MFHDF LIB_MFHDF=/Applications/HDF4.2.3/lib/libmfhdf.a NO HDF4 Include Files HDF_INC HDF_INC=/Applications/HDF4.2.3/include/ NO HDF4 SZ Compression Lib LIB_SZ LIB_SZ=/usr/local/lib/libsz.a NO HDF4 Z Compression Lib LIB_Z LIB_Z=/usr/local/lib/libz.a NO HDF4 JPEG Dependency LIB_JPEG LIB_JPEG=/usr/local/lib/libjpeg.a ________________________________________________________________________________ REQUIRED AND OPTIONAL LIBRARY NOTES: CDF ( REQUIRED ) : CDF needs to be installed prior to the installation of the Kameleon Conversion software ( Preferably version 3.0 or higher ). See the README.install file located in the cdf distribution for instructions. ARGTABLE 2 ( REQUIRED ) : argtable2.4 needs to be installed. See install.txt located in the argtable2.4 distribution. The KAMELEON conversion software requires the libcdf.a & libargtable2.a static libraries. Proper installation of both cdf and argtable2 should produce these two files. Take note of their locations as these two libraries will be used to build kameleon. You can either copy the libraries into kameleon's local ./lib directory, or specify the library locations during the invocation of make on kameleon. See BUILDING KAMELEON section for specific Makefile options The corresponding header/include files for the cdf and argtable2 libraries will also be required. Specifically cdf.h and artable2.h. These header files can be placed directly into kameleon's local ./inc directory, or their locations can be specified during the invocation of make on kameleon. See BUILDING KAMELEON section for specific Makefile options and/or See INCLUDE FILE section of README for more detail on incorporating external include files into kameleon build. NOTES ON Updating the cdf and argtable2 ".a" static archives if copying the libraries directly into kameleon's local ./lib directory: To update the archives table of contents if you have copied the libraries into the local lib directory for the Kameleon Converter, you should run ranlib on the newly copied/relocated libraries: ranlib libargtable2.a ranlib libcdf.a NETCDF ( OPTIONAL ) : The libnetcdf.a static library is used by the KAMELEON conversion software to read model output files that are originally stored in the netcdf .nc format. NOTE: The converter will still build if this library is not available, but you will not be able to read and convert model output files that are in the netCDF format such as enlil. See the INSTALL file located in the src directory of the netcdf distribution for details. The KAMELEON conversion software uses the libnetcdf.a static library. Proper installation of netcdf should produce this file. The netcdf.h header/include file will also be required. This header file can be placed directly into kameleon's local ./inc directory, or its location can be specified during the invocation of make on kameleon. See INCLUDE FILE section of this README for more detail on incorporating external include files into kameleon build. NOTES ON Updating the libnetcdf ".a" static archive if copying the library directly into kameleon's local ./lib directory: To update the archive table of contents if you have copied the library into the local lib directory for the Kameleon Converter, you should run ranlib on the newly copied/relocated library: ranlib libnetcdf.a HDF ( OPTIONAL ) : The HDF library is used by the KAMELEON conversion software to read model output files that are originally stored in the hdf .hdf format. NOTE: The converter will still build if this library is not available but you will not be able to read and convert model output files that are in the HDF format such as MAS. The HDF distribution yields two main libraries libmfhdf.a libdf.a HDF itself also uses three supporting shared libraries libjpeg.so libz.so libsz.so !!!NOTE: The SD interface used by HDF requires the "libmfhdf" library as noted above. However, if HDF is to be built along with NetCDF, an existing namespace conflict will arise as both libraries share the same internal function names. To counteract this problem, the HDF/MFHDF distribution must be compiled with the option '-DHAVE_NETCDF'. This will inform the compiler to rename the HDF version of the C-interface ( nc* ) of the netCDF API tp sd_nc*. Installation Notes: *For some versions of HDF ie 4.2r1 we need to compile hdf/mfhdf with the flag: -DHAVE_NETCDF Thsi may require you to add it to the os specific config file located in the config directory. For example, if building hdf on gnu linux, we edit the file: ./config/linux-gnu ie CFLAGS="-DHAVE_NETCDF" Other options include modifying the Makefiles under the mfhdf, hdf, and/or the main distribution directories. Be sure to confirm that -DHAVE_NETCDF option bieng used when the distribution is compiling. *The lastest version of HDF4 tested with Kameleon required the: --disable-netcdf option be spesified duringthe configure stage. Check your local HDF install documentation to determine what is required for your particular version of HDF. Assuming that libjpeg and libz are already installed, you can configure the hdf distribution by executing: ./configure --with-zlib=/path_2_zlib_directory \ --with-jpeg=/path_2_jpeg_directory --prefix=/path_2_install_directory \ --with-szlib=/path_to_SZIP_install_directory \ --disable-netcdf set your LD_LIBRARY_PATH=location_of_libz.so:location_of_libjpeg.so on a mac use DYLD_LIBRARY_PATH and build the libraries using gmake gmake check gmake install The corresponding header/include files for the HDF library will also be required. Since the HDF distribution has a buch of include files for itself, its best to keep them seperate and reference the group collectively with the macro LOCAL_HDF_INC which can be specified during the invocation of make on kameleon. See INCLUDE FILE section of README for more detail on incorporating external include files into kameleon build. FITS: The libcfitsio.a static library is used by the KAMELEON conversion software to read model output files that are originally stored in the fits .fts format. NOTE: The converter will still build if this library is not available but you will not be able to read and convert model/data output files that are in the FITS format. The FITS distribution yields the library libcfitsio.a The fitsio.h & longnam.h header/include files will also be required. These header files can be placed directly into kameleon's local ./inc directory, or their locations can be specified during the invocation of make on kameleon. See INCLUDE FILE section of README for more detail on incorporating external include files into kameleon build. NOTES ON Updating the libcfitsio ".a" static archive if copying the library directly into kameleon's local ./lib directory: To update the archive table of contents if you have copied the library into the local lib directory for the Kameleon Converter, you should run ranlib on the newly copied/relocated library: ranlib libcfitsio.a ________________________________________________________________________________ INCLUDE FILES: By default, the kameleon makefile is configured to look in kameleon's local inc/ directory for all of the required header files that are both specific to kameleon as well as the exernal libraries that kameleon uses. External include file can either be copied to kameleon's local inc/ directory or specified during the invocation of make using specific MACRO's **Required Include files** cdf.h - can be set using the CDF_INC macro ie. make CDF_INC=../location_of_your_local_cdf_inc_directory argtable2.h - can be set with the ARG_TABLE_2_INC macro ie make ARG_TABLE_2_INC=../location_of_your_local_argtable_inc_directory **Optional Include files** -If reading FITS files: fitsio.h longnam.h -If reading netCDF files: netcdf.h -If reading HDF files NOTE: The HDF distribution has a bunch of include files by itself so keep those seperate and reference in the MAKEFILE or during invocation of make with the macro: HDF_INC = ... ________________________________________________________________________________ MAKEFILE: The Makefile provided with the kameleon distribution should not need to be modified directly. All of the user configurable options can be specified during the invocation of make as specified under the BUILDING KAMELEON section of this README file. you can always do a: make help With the proper compilers, static libraries, header files, and library flags identified. to compile type: make CDF_INC=../your_cdf_inc_dir/ LIB_CDF=../your_dir/libcdf.a \ LIB_ARG_TABLE_2=../your_dir/libargtable2.a ARG_TABLE_2_INC=../your_argt_inc_dir/ or also include the optional libraries as well This should produce an executable named kameleon to clean the build directory type: make clean or make real_clean NOTE: make real_clean removes the libraries and executables generated by the Kameleon conversion software. ________________________________________________________________________________ USAGE: You can always type: kameleon -h to print a synopsis of the available command line options. %kameleon -h This program converts model output into a standardized format Usage: ./kameleon [--version] [-h|--help] [-v|-V|--verbose] [--nominmax] -f|--format=string -m|--model=string [-a|--aux_file=string] [-d|--database=string] [-i|--indir=string] [-o|--outdir=string] [<file>]... --version print software version number -h, --help print this help -v, -V, --verbose turn verbose output on --nominmax turn OFF actual min/max calculations for each variable -f, --format=string data format to convert to -m, --model=string model name -a, --aux_file=string auxiliary input file i.e.. a grid file separate from general data files -d, --database=string ccmc DatabaseInfo file location - for CCMC use only -i, --indir=string directory containing input files to be converted -o, --outdir=string output directory where converted files will be placed default is ./ <file> input files % example 1: converting a batsrus file kameleon -v -f cdf -m batsrus -o ./my_output_directory ./my_input_directory/my_batsrus_dot_out_file example 2: converting an open_ggcm file kameleon -v -f cdf -m ucla_ggcm -o ./my_output_directory -a ./my_grid_file ./my_input_directory/my_3df_file NOTE: The use of the -a auxiliary input flag to specify the location of the grid file To quickly browse cdf files, a useful command line tool is skeletontable. Typing: ../your_cdf_distribution/bin/skeletontable your_cdf_file.cdf produces an ascii text file named your_cdf_file.cdf.skt. You ________________________________________________________________________________ KNOWN BUGS: FIXED 1. Using the wild-card meta character * in the file name argument may cause unexpected command line parsing errors or cause the program to crash 2. OpenGGCM/UCLA-GGCM - pre 4.4.1 kameleon converter versions assumed variables where in a specific order in the original files. If/when this order changes, some variables may not have be read in. Fixed by verifying read return status and rewinding file pointer to re-read as necessary. This rewind functionality was originally available but turned off as it makes the read 4x slower. Old functionality either had rewind on or off, now it is set dynamically as required. 3. ENLIL reading with -d option: pre 4.4.1 would segfault or report a bus error when reaching the clouds_start_times entry in the CCMC DatabaseInfo file. Fixed by verifying date string length > 1 before calling the date_formatter date processing function TESTING 4. When using the -indir argument, some files may not be accessible do to system reporting "too many open files error". Files that could not be accessed will be skipped. TESTING 5. When using the -indir argument, superfluous files may not always be ignored as they should. ie. an existing .cdf file may be read into the program as input WORKAROUND 6. NEW SWMF-BATSRUS .out files were missing the NX,NY,NZ special | parameters required for number_of_blocks calculation. New | functionality searches the DatabaseInfo file as an alternative | and exits program if those parameters cannot be found. | | Also missing were the P1,P2,P3 special parameters as well. | | The workaround is to add the required info as key-value pairs to | the DatabaseInfo file. The DatabaseInfo file is used at the CCMC | for the Runs-On-Request System, but external user can generate a | simple text file as well. Either way the following lines need | to exist: | | 6 # batsrus_special_parameter_NX | 6 # batsrus_special_parameter_NY | 6 # batsrus_special_parameter_NZ | 2 # batsrus_special_parameter_P1 | 1 # batsrus_special_parameter_P2 | 1 # batsrus_special_parameter_P3 | | ...adjusting the numbers in the first column as necessary. | | You can tell the converter where the DatabaseInfo file is with | the -d option. v FIXED 6. AS of v4.7.0, the read batsrus module in now capable of calculating/determmining the missing special paramenters without having to specify them in an external DatabaseInfo file.