diff --git a/README.rst b/README.rst index 96e0812..adbffb3 100644 --- a/README.rst +++ b/README.rst @@ -6,7 +6,7 @@ viresclient :target: https://badge.fury.io/py/viresclient .. image:: https://readthedocs.org/projects/viresclient/badge/?version=latest - :target: http://viresclient.readthedocs.io/en/latest/?badge=latest + :target: http://viresclient.readthedocs.io/ :alt: Documentation Status .. image:: https://travis-ci.org/ESA-VirES/VirES-Python-Client.svg?branch=master @@ -22,4 +22,4 @@ Install with:: pip install viresclient -Full documentation: http://viresclient.readthedocs.io/en/latest/ +Full documentation: http://viresclient.readthedocs.io/ diff --git a/docs/api.rst b/docs/api.rst index a3f6499..eb4b226 100644 --- a/docs/api.rst +++ b/docs/api.rst @@ -1,8 +1,3 @@ -viresclient -=========== - -.. automodule:: viresclient - SwarmRequest ============ @@ -21,7 +16,7 @@ ReturnedData :show-inheritance: :inherited-members: -.. autoclass:: viresclient.ReturnedDataGroup +.. autoclass:: viresclient.ReturnedDataFile :members: :undoc-members: :show-inheritance: diff --git a/docs/available_parameters.rst b/docs/available_parameters.rst new file mode 100644 index 0000000..fc92d72 --- /dev/null +++ b/docs/available_parameters.rst @@ -0,0 +1,105 @@ +Available parameters for Swarm data +=================================== + +You can check which parameters are available with: + +.. code-block:: python + + from viresclient import SwarmRequest + req = SwarmRequest() + req.available_collections() + req.available_measurements("MAG") + req.available_models() + req.available_auxiliaries() + +---- + +``collections`` +--------------- + +(replace x with A, B, or C for Alpha, Bravo, or Charlie):: + + SW_OPER_MAGx_LR_1B + SW_OPER_EFIx_PL_1B + SW_OPER_IBIxTMS_2F + SW_OPER_TECxTMS_2F + SW_OPER_FACxTMS_2F + SW_OPER_EEFxTMS_2F + +For Alpha-Charlie FAC: ``collection="SW_OPER_FAC_TMS_2F"``. + +The ``measurements``, ``models``, and ``auxiliaries`` chosen will match the cadence of the ``collection`` chosen. + +---- + +``measurements`` +---------------- + +Choose combinations of measurements from one of the following sets, corresponding to the collection chosen above. + +For MAG:: + + F,dF_AOCS,dF_other,F_error,B_VFM,B_NEC,dB_Sun,dB_AOCS,dB_other,B_error,q_NEC_CRF,Att_error,Flags_F,Flags_B,Flags_q,Flags_Platform,ASM_Freq_Dev + +For EFI:: + + v_SC,v_ion,v_ion_error,E,E_error,dt_LP,n,n_error,T_ion,T_ion_error,T_elec,T_elec_error,U_SC,U_SC_error,v_ion_H,v_ion_H_error,v_ion_V,v_ion_V_error,rms_fit_H,rms_fit_V,var_x_H,var_y_H,var_x_V,var_y_V,dv_mtq_H,dv_mtq_V,SAA,Flags_LP,Flags_LP_n,Flags_LP_T_elec,Flags_LP_U_SC,Flags_TII,Flags_Platform,Maneuver_Id + +For IBI:: + + Bubble_Index,Bubble_Probability,Flags_Bubble,Flags_F,Flags_B,Flags_q + +For TEC:: + + GPS_Position,LEO_Position,PRN,L1,L2,P1,P2,S1,S2,Absolute_STEC,Relative_STEC,Relative_STEC_RMS,DCB,DCB_Error + +For FAC:: + + IRC,IRC_Error,FAC,FAC_Error,Flags,Flags_F,Flags_B,Flags_q + +For EEF:: + + EEF,RelErr,flags + +---- + +``models`` +---------- + +Models are evaluated along the satellite track at the positions of the measurements. + +:: + + IGRF12, SIFM, CHAOS-6-Combined, CHAOS-6-Core, CHAOS-6-Static, + MCO_SHA_2C, MCO_SHA_2D, MCO_SHA_2F, MLI_SHA_2C, MLI_SHA_2D, + MMA_SHA_2C-Primary, MMA_SHA_2C-Secondary, + MMA_SHA_2F-Primary, MMA_SHA_2F-Secondary, + MIO_SHA_2C-Primary, MIO_SHA_2C-Secondary, + MIO_SHA_2D-Primary, MIO_SHA_2D-Secondary + +(``residuals`` available when combined with MAG ``measurements`` ``F`` and/or ``B_NEC``) + +---- + +``auxiliaries`` +--------------- + +:: + + SyncStatus, Kp, Dst, IMF_BY_GSM, IMF_BZ_GSM, IMF_V, F10_INDEX, + OrbitSource, OrbitNumber, AscendingNodeTime, + AscendingNodeLongitude, QDLat, QDLon, QDBasis, MLT, SunDeclination, + SunHourAngle, SunRightAscension, SunAzimuthAngle, SunZenithAngle, + SunLongitude, SunVector, DipoleAxisVector, NGPLatitude, NGPLongitude, + DipoleTiltAngle, + + UpwardCurrent, TotalCurrent, + DivergenceFreeCurrentFunction, F_AMPS, B_NEC_AMPS + +NB: the AMPS model is currently accessible as "auxiliaries" instead of a "model". + +---- + +Standard positional variables always returned:: + + Timestamp,Latitude,Longitude,Radius,Spacecraft diff --git a/docs/index.rst b/docs/index.rst index 54202da..0841fdd 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -6,6 +6,8 @@ Welcome to VirES-Python-Client's documentation! :caption: Overview readme + available_parameters + release_notes .. toctree:: :maxdepth: 2 diff --git a/docs/readme.rst b/docs/readme.rst index 8285fd1..8e3ac31 100644 --- a/docs/readme.rst +++ b/docs/readme.rst @@ -1,11 +1,23 @@ Introduction ------------- +============ This is the documentation for the ``viresclient`` Python package. This is a tool which connects to a VirES server through the `WPS `_ interface and handles product requests and downloads. +Data can be accessed from the server as CSV or CDF files and saved to disk, or loaded directly into Python objects pandas.DataFrame_, or xarray.Dataset_. + +.. _pandas.DataFrame: https://pandas.pydata.org/pandas-docs/stable/dsintro.html#dataframe + +.. _xarray.Dataset: http://xarray.pydata.org/en/stable/data-structures.html#dataset + +The cdflib_ library is used to read CDF files. + +.. _cdflib: https://github.com/MAVENSDC/cdflib + Installation ------------ +Python ≥ 3.5 is required. (Partial?) support for 2.7 and 3.4 could be added in the future, but their usage is not recommended (https://python3statement.org/). + It can currently be installed with:: pip install viresclient @@ -21,6 +33,7 @@ Dependencies:: Example usage ------------- + Import the package and set up the connection to the server: .. code-block:: python @@ -67,89 +80,23 @@ Specify the time range from which to retrieve data, make the request to the serv filetype="cdf", asynchronous=True) -Transfer your data to a (``pandas``) dataframe or save it: +Transfer your data to a pandas.DataFrame_, or a xarray.Dataset_, or just save it as is: + +.. _pandas.DataFrame: https://pandas.pydata.org/pandas-docs/stable/dsintro.html#dataframe + +.. _xarray.Dataset: http://xarray.pydata.org/en/stable/data-structures.html#dataset .. code-block:: python df = data.as_dataframe() + ds = data.as_xarray() data.to_file('outfile.cdf', overwrite=False) The returned data has columns for: - ``Spacecraft, Timestamp, Latitude, Longitude, Radius`` - - those specified in ``measurements`` and ``auxiliaries`` above + - those specified by ``measurements`` and ``auxiliaries`` ... and model values and residuals, named as: - ``F_`` -- scalar field - ``B_NEC_`` -- vector field - ``F_res_`` -- scalar field residual (``F - F_``) - ``B_NEC_res_`` -- vector field residual (``B_NEC - B_NEC_``) - -Available parameters for Swarm data ------------------------------------ - -``collections`` (replace x with A, B, or C for Alpha, Bravo, or Charlie):: - - SW_OPER_MAGx_LR_1B - SW_OPER_EFIx_PL_1B - SW_OPER_IBIxTMS_2F - SW_OPER_TECxTMS_2F - SW_OPER_FACxTMS_2F - SW_OPER_EEFxTMS_2F - -For Alpha-Charlie FAC: ``collection="SW_OPER_FAC_TMS_2F"``. - -The ``measurements``, ``models``, and ``auxiliaries`` chosen will match the cadence of the ``collection`` chosen. - -``measurements``: - -Choose from one of the following sets, corresponding to the collection chosen above. - -For MAG:: - - F,dF_AOCS,dF_other,F_error,B_VFM,B_NEC,dB_Sun,dB_AOCS,dB_other,B_error,q_NEC_CRF,Att_error,Flags_F,Flags_B,Flags_q,Flags_Platform,ASM_Freq_Dev - -For EFI:: - - v_SC,v_ion,v_ion_error,E,E_error,dt_LP,n,n_error,T_ion,T_ion_error,T_elec,T_elec_error,U_SC,U_SC_error,v_ion_H,v_ion_H_error,v_ion_V,v_ion_V_error,rms_fit_H,rms_fit_V,var_x_H,var_y_H,var_x_V,var_y_V,dv_mtq_H,dv_mtq_V,SAA,Flags_LP,Flags_LP_n,Flags_LP_T_elec,Flags_LP_U_SC,Flags_TII,Flags_Platform,Maneuver_Id - -For IBI:: - - Bubble_Index,Bubble_Probability,Flags_Bubble,Flags_F,Flags_B,Flags_q - -For TEC:: - - GPS_Position,LEO_Position,PRN,L1,L2,P1,P2,S1,S2,Absolute_STEC,Relative_STEC,Relative_STEC_RMS,DCB,DCB_Error - -For FAC:: - - IRC,IRC_Error,FAC,FAC_Error,Flags,Flags_F,Flags_B,Flags_q - -For EEF:: - - EEF,RelErr,flags - -``models`` (``residuals`` available when combined with MAG ``measurements``):: - - IGRF12, SIFM, CHAOS-6-Combined, CHAOS-6-Core, CHAOS-6-Static, - MCO_SHA_2C, MCO_SHA_2D, MCO_SHA_2F, MLI_SHA_2C, MLI_SHA_2D, - MMA_SHA_2C-Primary, MMA_SHA_2C-Secondary, - MMA_SHA_2F-Primary, MMA_SHA_2F-Secondary, - MIO_SHA_2C-Primary, MIO_SHA_2C-Secondary, - MIO_SHA_2D-Primary, MIO_SHA_2D-Secondary - -``auxiliaries``:: - - SyncStatus, Kp, Dst, IMF_BY_GSM, IMF_BZ_GSM, IMF_V, F10_INDEX, - OrbitSource, OrbitNumber, AscendingNodeTime, - AscendingNodeLongitude, QDLat, QDLon, QDBasis, MLT, SunDeclination, - SunHourAngle, SunRightAscension, SunAzimuthAngle, SunZenithAngle, - SunLongitude, SunVector, DipoleAxisVector, NGPLatitude, NGPLongitude, - DipoleTiltAngle, - - UpwardCurrent, TotalCurrent, - DivergenceFreeCurrentFunction, F_AMPS, B_NEC_AMPS - -Standard positional variables always returned:: - - Timestamp,Latitude,Longitude,Radius,Spacecraft - -NB: the AMPS model is currently accessible as "auxiliaries" instead of a "model". diff --git a/docs/release_notes.rst b/docs/release_notes.rst new file mode 100644 index 0000000..55c936a --- /dev/null +++ b/docs/release_notes.rst @@ -0,0 +1,14 @@ +Release notes +============= + +Changes from v0.1.0 to v0.2.0 +----------------------------- + + - Now use ``SwarmRequest`` instead of ``ClientRequest``. + - kwarg ``subsample`` changed to ``sampling_step``. + - Added references to .available_collections() and .available_models(). + - User credentials are automatically stored in a configuration file ``~/.viresclient.ini``. + - Downloads are streamed to temporary files instead of being held in memory. + - Any size request is now supported. Large requests are automatically chunked up. + - Added download progress bar indicating size in MB. + - xarray added as a dependency and ``.as_xarray()`` method added. diff --git a/notebooks/simple_example.ipynb b/notebooks/simple_example.ipynb index b49aae9..f7e9b8f 100644 --- a/notebooks/simple_example.ipynb +++ b/notebooks/simple_example.ipynb @@ -4,7 +4,36 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Example Usage" + "# General example" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Connect to the server\n", + "\n", + "You need a username and password to connect to the server. You can enter these in the code directly:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "from viresclient import SwarmRequest\n", + "\n", + "request = SwarmRequest(url=\"https://staging.viresdisc.vires.services/openows\",\n", + " username=\"your username\",\n", + " password=\"your password\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "When these credentials have already been entered, you don't need to supply them again. Your username and password have now been stored (unencrypted!) in a configuration file in your home directory: ``~/.viresclient.ini``. You can update them simply by calling ``SwarmRequest()`` again with new inputs." ] }, { @@ -20,7 +49,7 @@ }, { "cell_type": "code", - "execution_count": 1, + "execution_count": 2, "metadata": {}, "outputs": [ { @@ -36,9 +65,7 @@ "from viresclient import SwarmRequest\n", "import datetime as dt\n", "\n", - "request = SwarmRequest(url=\"https://staging.viresdisc.vires.services/openows\",\n", - " username=\"your username\",\n", - " password=\"your password\")\n", + "request = SwarmRequest(url=\"https://staging.viresdisc.vires.services/openows\")\n", "\n", "request.set_collection(\"SW_OPER_MAGA_LR_1B\")\n", "\n", @@ -67,7 +94,7 @@ }, { "cell_type": "code", - "execution_count": 2, + "execution_count": 3, "metadata": {}, "outputs": [ { @@ -91,26 +118,13 @@ }, { "cell_type": "code", - "execution_count": 3, + "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", - "\n", "\n", " \n", " \n", @@ -296,7 +310,7 @@ "2016-01-01 00:28:40 -31.165012 6827978.83 A " ] }, - "execution_count": 3, + "execution_count": 4, "metadata": {}, "output_type": "execute_result" } @@ -317,7 +331,7 @@ }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 5, "metadata": {}, "outputs": [ { @@ -331,7 +345,7 @@ " 33559.6071]))" ] }, - "execution_count": 4, + "execution_count": 5, "metadata": {}, "output_type": "execute_result" } @@ -360,35 +374,35 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\n", - "Dimensions: (dim: 3, time: 4256)\n", + "Dimensions: (Timestamp: 4256, dim: 3)\n", "Coordinates:\n", - " * time (time) datetime64[ns] 2016-01-01T00:28:00 ...\n", - " * dim (dim) int64 0 1 2\n", + " * Timestamp (Timestamp) datetime64[ns] 2016-01-01T00:28:00 ...\n", + "Dimensions without coordinates: dim\n", "Data variables:\n", - " QDLon (time) float64 -30.81 -30.9 -30.99 -31.08 ...\n", - " F_MCO_SHA_2C (time) float64 2.48e+04 2.497e+04 2.516e+04 ...\n", - " F_MMA_SHA_2C-Primary (time) float64 128.3 128.3 128.4 128.4 128.5 ...\n", - " F (time) float64 2.463e+04 2.481e+04 2.499e+04 ...\n", - " Longitude (time) float64 -102.7 -102.7 -102.7 -102.7 ...\n", - " Latitude (time) float64 0.1976 0.8382 1.479 2.119 ...\n", - " B_NEC_MCO_SHA_2C (time, dim) float64 2.361e+04 2.932e+03 ...\n", - " QDLat (time) float64 8.425 9.069 9.714 10.36 11.0 ...\n", - " B_NEC (time, dim) float64 2.343e+04 2.929e+03 ...\n", - " B_NEC_MMA_SHA_2C-Primary (time, dim) float64 -126.9 -5.62 17.58 ...\n", - " B_NEC_MMA_SHA_2C-Secondary (time, dim) float64 -33.09 -1.709 -6.89 ...\n", - " F_MMA_SHA_2C-Secondary (time) float64 33.84 34.03 34.22 34.43 34.65 ...\n", - " Spacecraft (time) \n", + "\n", "array([[-13.829441, 4.811139, -12.867523],\n", " [-13.745905, 4.513371, -13.351697],\n", " [-13.673071, 4.881882, -14.067758],\n", @@ -457,11 +471,11 @@ " [ 1.096503, -2.327229, -4.433677],\n", " [ 2.135158, -3.167922, -4.39485 ]])\n", "Coordinates:\n", - " * time (time) datetime64[ns] 2016-01-01T00:28:00 2016-01-01T00:28:10 ...\n", - " * dim (dim) int64 0 1 2" + " * Timestamp (Timestamp) datetime64[ns] 2016-01-01T00:28:00 ...\n", + "Dimensions without coordinates: dim" ] }, - "execution_count": 7, + "execution_count": 8, "metadata": {}, "output_type": "execute_result" } @@ -482,7 +496,7 @@ "outputs": [], "source": [ "%matplotlib inline\n", - "B_res[:,2].plot(x='time');" + "B_res[:,2].plot(x='Timestamp');" ] }, {