finishing up putting the README docs on the readthedocs page

doc/index.rst

@@ -1,8 +1,7 @@
 cdflib
 ======
 
-This is a python package to read CDF files without needing to install the
-CDF NASA library.
+This is a python package to read CDF files without needing to install the CDF NASA library.
 
 Installing
 ----------
@@ -22,6 +21,7 @@ cdflib requires python 3 and numpy. To install run
    modules/cdflib
    modules/cdfread
    modules/cdfwrite
+   modules/cdfepoch
    modules/xarray
    modules/apis
    development

doc/introduction.rst

@@ -11,11 +11,14 @@ The only module you need to install is numpy, but there are a few things you can
 While this origally started as a way to read PDS-archive compliant CDF files, thanks to many contributors, it has grown to be able to handle every type of CDF file.  
 
 
-What does it do?
+What can cdflib do?
 -------------------
 
-* Ability to read variables and attributes from CDF files
+* Ability to read variables and attributes from CDF files (see `modules\cdfread`_)
 * Writes CDF version 3 files
 * Can convert between CDF time types (EPOCH/EPOCH16/TT2000) to other common time formats
 * Can convert CDF files into XArray Dataset objects
-* Can convert XArray Dataset objects into CDF files, attempting to maintain ISTP compliance 
+* Can convert XArray Dataset objects into CDF files, attempting to maintain ISTP compliance
+
+
+

doc/modules/apis.rst

@@ -1,8 +1,7 @@
 API Reference
 =====================
 
-The following documentation is an auto-generated summary of cdflib's API.  For more details and examples, refer to other
-parts of the documentation.
+The following documentation is an auto-generated summary of cdflib's API.  For more details and examples, refer to other parts of the documentation.
 
 .. automodapi:: cdflib.cdfread
     :no-inheritance-diagram:

doc/modules/cdfepoch.rst

@@ -0,0 +1,128 @@
+**********************
+CDF Time Conversions
+**********************
+
+There are three (3) unique epoch data types in CDF: CDF_EPOCH, CDF_EPOCH16 and CDF_TIME_TT2000.
+
+- CDF_EPOCH is milliseconds since Year 0.
+- CDF_EPOCH16 is picoseconds since Year 0.
+- CDF_TIME_TT2000 (TT2000 as short) is nanoseconds since J2000 with leap seconds.
+
+The following two classes contain functions to convert those times into formats that are in more standard use.
+
+
+Epochs Class
+============
+
+There are 5 main functions in this module to help with time conversions
+
+encode (epochs, iso_8601=False)
+------------------------------------------------------------------------------
+
+Encodes the epoch(s) as read from a CDF file into UTC string(s).  Returns a list of strings.
+
+- CDF_EPOCH: The input should be either a float or list of floats (in numpy, a np.float64 or a np.ndarray of np.float64) Each epoch is encoded, by default to a ISO 8601 form: 2004-05-13T15:08:11.022 Or, if iso_8601 is set to False, 13-May-2004 15:08:11.022
+- CDF_EPOCH16: The input should be either a complex or list of complex(in numpy, a np.complex128 or a np.ndarray of np.complex128) Each epoch is encoded, by default to a ISO 8601 form: 2004-05-13T15:08:11.022033044055 Or, if iso_8601 is set to False, 13-May-2004 15:08:11.022.033.044.055
+- TT2000: The input should be either a int or list of ints (in numpy, a np.int64 or a np.ndarray of np.int64) Each epoch is encoded, by default to a ISO 8601 form: 2008-02-02T06:08:10.10.012014016 Or, if iso_8601 is set to False, 02-Feb-2008 06:08:10.012.014.016
+
+
+unixtime (epochs, to_np=False)
+------------------------------------------------------------------------------
+
+Encodes the epoch(s) as read from a CDF file into a list of seconds after 1970-01-01. Precision is only kept to the nearest microsecond.
+
+If ``to_np=True``, then the values will be returned in a numpy array.
+
+
+breakdown (epochs, to_np=False)
+------------------------------------------------------------------------------
+
+Breaks down the epoch(s) as read from a CDF file into UTC components.  This takes the form of a list, or a list of lists.
+
+- CDF_EPOCH: they are 7 date/time components: year, month, day, hour, minute, second, and millisecond
+- CDF_EPOCH16: they are 10 date/time components: year, month, day, hour, minute, second, and millisecond, microsecond, nanosecond, and picosecond.
+- TT2000: they are 9 date/time components: year, month, day, hour, minute, second, millisecond, microsecond, nanosecond.
+
+Specify ``to_np=True``, if the result should be in numpy array.
+
+compute[_epoch/_epoch16/_tt200] (datetimes, to_np=False)
+------------------------------------------------------------------------------
+
+Computes the provided date/time components into CDF epoch value(s).
+
+For CDF_EPOCH: For computing into CDF_EPOCH value, each date/time elements should have exactly seven (7) components, as year, month, day, hour, minute, second and millisecond, in a list. For example:
+
+>>> [[2017,1,1,1,1,1,111],[2017,2,2,2,2,2,222]]
+
+Or, call function compute_epoch directly, instead, with at least three (3) first (up to seven) components. The last component, if not the 7th, can be a float that can have a fraction of the unit.
+
+For CDF_EPOCH16: They should have exactly ten (10) components, as year, month, day, hour, minute, second, millisecond, microsecond, nanosecond and picosecond, in a list. For example:
+
+>>> [[2017,1,1,1,1,1,123,456,789,999],[2017,2,2,2,2,2,987,654,321,999]]
+
+Or, call function compute_epoch directly, instead, with at least three (3) first (up to ten) components. The last component, if not the 10th, can be a float that can have a fraction of the unit.
+
+For TT2000: Each TT2000 typed date/time should have exactly nine (9) components, as year, month, day, hour, minute, second, millisecond, microsecond, and nanosecond, in a list. For example:
+
+>>> [[2017,1,1,1,1,1,123,456,789],[2017,2,2,2,2,2,987,654,321]]
+
+Or, call function compute_tt2000 directly, instead, with at least three (3) first (up to nine) components. The last component, if not the 9th, can be a float that can have a fraction of the unit.
+
+Specify ``to_np=True``, if the result should be in numpy class.
+
+
+parse (datetimes, to_np=False)
+-------------------------------
+
+Parses the provided date/time string(s) into CDF epoch value(s).
+
+- CDF_EPOCH: The string has to be in the form of 'dd-mmm-yyyy hh:mm:ss.xxx' or 'yyyy-mm-ddThh:mm:ss.xxx' (in iso_8601). The string is the output from encode function.
+- CDF_EPOCH16: The string has to be in the form of 'dd-mmm-yyyy hh:mm:ss.mmm.uuu.nnn.ppp' or 'yyyy-mm-ddThh:mm:ss.mmmuuunnnppp' (in iso_8601). The string is the output from encode function.
+- TT2000: The string has to be in the form of 'dd-mmm-yyyy hh:mm:ss.mmm.uuu.nnn' or 'yyyy-mm-ddThh:mm:ss.mmmuuunnn' (in iso_8601). The string is the output from encode function.
+
+
+Specify to_``np=True``, if the result should be in numpy class.
+
+
+findepochrange (epochs, starttime=None, endtime=None)
+-------------------------------------------------------
+
+Finds the record range within the start and end time from values of a CDF epoch data type. It returns a list of record numbers. If the start time is not provided, then it is assumed to be the minimum possible value. If the end time is not provided, then the maximum possible value is assumed. The epoch is assumed to be in the chronological order. The start and end times should have the proper number of date/time components, corresponding to the epoch's data type.
+
+The start/end times should be in either be in epoch units, or in the list format described in "compute_epoch/epoch16/tt2000" section.
+
+
+getVersion ()
+-------------
+
+Shows the code version.
+
+
+getLeapSecondLastUpdated ()
+----------------------------
+
+Shows the latest date a leap second was added to the leap second table.
+
+
+Epochs Astropy
+==============
+
+If you have astropy installed, importing cdflib also imports the module cdflib.cdfastropy, which contains all of the functionality of the above module, but uses the Astropy Time class for all conversions. It can be used in the same way as the above module:
+
+>>> import cdflib
+>>> epoch_time = cdflib.cdfastropy.compute_epoch([2017,1,1,1,1,1,111])
+
+Additionally, and perhaps most importantly, there is an additonal function that converts CDF_EPOCH/EPOCH16/TT2000 times to the Astropy Time class:
+
+
+convert_to_astropy (epochs, format=None)
+--------------------------------------------
+Converts the epoch(s) into Astropy Time(s).
+
+- CDF_EPOCH: The input should be either a float or list of floats (in numpy, a np.float64 or a np.ndarray of np.float64). If you'd like to ignore the input type and convert to CDF_EPOCH directly, specify format='cdf_epoch' when you call the function.
+- CDF_EPOCH16: The input should be either a complex or list of complex(in numpy, a np.complex128 or a np.ndarray of np.complex128). If you'd like to ignore the input type and convert to CDF_EPOCH directly, specify format='cdf_epoch16' when you call the function.
+- TT2000: The input should be either a int or list of ints (in numpy, a np.int64 or a np.ndarray of np.int64). If you'd like to ignore the input type and convert to CDF_EPOCH directly, specify format='cdf_tt2000' when you call the function.
+
+For more information about Astropy Times and all the functionality it contains, take a look at the astropy documentation
+
+https://docs.astropy.org/en/stable/time/

doc/modules/cdfwrite.rst

@@ -75,15 +75,18 @@ write_var (var_spec, var_attrs=None, var_data=None)
 Writes a variable, along with variable attributes and data. ``var_spec`` is a dictionary that contains the specifications of the variable. The required/optional keys for creating a variable:
 
 Required keys:
+
 - ``Variable`` The name of the variable
 - ``Data_Type`` the CDF data type
 - ``Num_Elements`` The number of elements. Always 1 the for numeric type. The char length for string type.
 - ``Rec_Vary`` The dimensional sizes, applicable only to rVariables.
 
 For zVariables:
+
 - ``Dim_Sizes`` The dimensional sizes for zVariables only. Use [] for 0-dimension. Each and every dimension is varying for zVariables.
 
 For rVariables:
+
 - ``Dims_Vary`` The dimensional variances for rVariables only.
 
 Optional Keys:
@@ -95,16 +98,107 @@ Optional Keys:
 - ``Pad`` The padded value (in bytes, numpy.ndarray or string)
 
 ``var_attrs`` is a dictionary, with {attribute:value} pairs. The attribute is the name of a variable attribute. The value can have its data type specified for the numeric data. If not, based on Python's type, a corresponding CDF type is assumed: CDF_INT4 for int, CDF_DOUBLE for float, CDF_EPOCH16 for complex and and CDF_INT8 for long. For example:
+
 >>> var_attrs= { 'attr1': 'value1', 'attr2': 12.45, 'attr3': [3,4,5], .....} -or- var_attrs= { 'attr1': 'value1', 'attr2': [12.45, 'CDF_DOUBLE'], 'attr3': [[3,4,5], 'CDF_INT4'], ..... }
 
 ``var_data`` is the data for the variable. If the variable is a regular variable without sparse records, it must be in a single structure of bytes, or numpy.ndarray for numeric variable, or str or list of strs for string variable. If the variable has sparse records, var_data should be presented in a list/tuple with two elements, the first being a list/tuple that contains the physical record number(s), the second being the variable data in bytes, numpy.ndarray, or a list of strings. Variable data can have just physical records' data (with the same number of records as the first element) or have data from both physical records and virtual records (which with filled data). The var_data has the form:
 
 .. note::
-    The attribute entry value for the CDF epoch data type, CDF_EPOCH, CDF_EPOCH16 or CDF_TIME_TT2000, can be presented in either a numeric form, or an encoded string form. For numeric, the CDF_EPOCH data is 8-byte float, CDF_EPOCH16 16-byte complex and CDF_TIME_TT2000 8-byte long. The default encoded string for the epoch `data should have this form:
+    The attribute entry value for the CDF epoch data type, CDF_EPOCH, CDF_EPOCH16 or CDF_TIME_TT2000, can be presented in either a numeric form, or an encoded string form. For numeric, the CDF_EPOCH data is 8-byte float, CDF_EPOCH16 16-byte complex and CDF_TIME_TT2000 8-byte long. The default encoded string for the epoch data should have this form:
+
     >>> CDF_EPOCH: 'dd-mon-year hh:mm:ss.mmm'
     >>> CDF_EPOCH16: 'dd-mon-year hh:mm:ss.mmm.uuu.nnn.ppp'
     >>> CDF_TIME_TT2000: 'year-mm-ddThh:mm:ss.mmmuuunnn'
 
 getVersion()
 ------------
 Shows the code version and modified date.
+
+
+Sample Usage
+------------
+
+>>> import cdfwrite
+>>> import cdfread
+>>> import numpy as np
+>>>
+>>> cdf_master = cdfread.CDF('/path/to/master_file.cdf')
+>>> if (cdf_master.file != None):
+>>> # Get the cdf's specification
+>>> info=cdf_master.cdf_info()
+>>> cdf_file=cdfwrite.CDF('/path/to/swea_file.cdf',cdf_spec=info,delete=True)
+>>> if (cdf_file.file == None):
+>>>     cdf_master.close()
+>>>     raise OSError('Problem writing file.... Stop')
+>>>
+>>> # Get the global attributes
+>>> globalaAttrs=cdf_master.globalattsget(expand=True)
+>>> # Write the global attributes
+>>> cdf_file.write_globalattrs(globalaAttrs)
+>>> zvars=info['zVariables']
+>>> print('no of zvars=',len(zvars))
+>>> # Loop thru all the zVariables
+>>> for x in range (0, len(zvars)):
+>>>     # Get the variable's specification
+>>>     varinfo=cdf_master.varinq(zvars[x])
+>>>     #print('Z =============>',x,': ', varinfo['Variable'])
+>>>     # Get the variable's attributes
+>>>     varattrs=cdf_master.varattsget(zvars[x], expand=True)
+>>>     if (varinfo['Sparse'].lower() == 'no_sparse'):
+>>>         # A variable with no sparse records... get the variable data
+>>>         vardata=.......
+>>>         # Create the zVariable, write out the attributes and data
+>>>         cdf_file.write_var(varinfo, var_attrs=varattrs, var_data=vardata)
+>>>     else:
+>>>         # A variable with sparse records...
+>>>         # data is in this form [physical_record_numbers, data_values]
+>>>         # physical_record_numbers (0-based) contains the real record
+>>>         # numbers. For example, a variable has only 3 physical records
+>>>         # at [0, 5, 10]:
+>>>         varrecs=[0,5,10]
+>>>         # data_values could contain only the physical records' data or
+>>>         # both the physical and virtual records' data.
+>>>         # For example, a float variable of 1-D with 3 elements with only
+>>>         # 3 physical records at [0,5,10]:
+>>>         # vardata = [[  5.55000000e+01, -1.00000002e+30,  6.65999985e+01],
+>>>         #            [  6.66659973e+02,  7.77770020e+02,  8.88880005e+02],
+>>>         #            [  2.00500000e+02,  2.10600006e+02,  2.20699997e+02]]
+>>>         # Or, with virtual record data embedded in the data:
+>>>         # vardata = [[  5.55000000e+01, -1.00000002e+30,  6.65999985e+01],
+>>>         #            [ -1.00000002e+30, -1.00000002e+30, -1.00000002e+30],
+>>>         #            [ -1.00000002e+30, -1.00000002e+30, -1.00000002e+30],
+>>>         #            [ -1.00000002e+30, -1.00000002e+30, -1.00000002e+30],
+>>>         #            [ -1.00000002e+30, -1.00000002e+30, -1.00000002e+30],
+>>>         #            [  6.66659973e+02,  7.77770020e+02,  8.88880005e+02],
+>>>         #            [ -1.00000002e+30, -1.00000002e+30, -1.00000002e+30],
+>>>         #            [ -1.00000002e+30, -1.00000002e+30, -1.00000002e+30],
+>>>         #            [ -1.00000002e+30, -1.00000002e+30, -1.00000002e+30],
+>>>         #            [ -1.00000002e+30, -1.00000002e+30, -1.00000002e+30],
+>>>         #            [  2.00500000e+02,  2.10600006e+02,  2.20699997e+02]]
+>>>         # Records 1, 2, 3, 4, 6, 7, 8, 9 are all virtual records with pad
+>>>         # data (variable defined with 'pad_sparse').
+>>>         vardata=np.asarray([.,.,.,..])
+>>>         # Create the zVariable, and optionally write out the attributes
+>>>         # and data
+>>>         cdf_file.write_var(varinfo, var_attrs=varattrs,
+>>>                    var_data=[varrecs,vardata])
+>>>    rvars=info['rVariables']
+>>>    print('no of rvars=',len(rvars))
+>>>    # Loop thru all the rVariables
+>>>    for x in range (0, len(rvars)):
+>>>        varinfo=cdf_master.varinq(rvars[x])
+>>>        print('R =============>',x,': ', varinfo['Variable'])
+>>>        varattrs=cdf_master.varattsget(rvars[x], expand=True)
+>>>        if (varinfo['Sparse'].lower() == 'no_sparse'):
+>>>            vardata=.......
+>>>            # Create the rVariable, write out the attributes and data
+>>>            cdf_file.write_var(varinfo, var_attrs=varattrs, var_data=vardata)
+>>>        else:
+>>>            varrecs=[.,.,.,..]
+>>>            vardata=np.asarray([.,.,.,..])
+>>>            cdf_file.write_var(varinfo, var_attrs=varattrs,
+>>>                       var_data=[varrecs,vardata])
+>>> cdf_master.close()
+>>> cdf_file.close()
+
+

doc/modules/xarray.rst

@@ -1,12 +1,11 @@
 Working with XArray
 ===================
 
-There are two functions for working with XArray DataSets, one for converting
+There are two functions for working with XArray Datasets, one for converting
 a CDF to a DataSet, and one for going the other way.
 
 These will attempt to determine any
-`ISTP Compliance <https://spdf.gsfc.nasa.gov/istp_guide/istp_guide.html>`_ within
-the file, and incorporate that into the Dataset object.
+`ISTP Compliance <https://spdf.gsfc.nasa.gov/istp_guide/istp_guide.html>`_, and incorporate that into the output.
 
 .. autofunction:: cdflib.cdf_to_xarray