Skip to content

Getting Started with pySPEDAS

Jump to bottom
Jim Lewis edited this page Dec 2, 2023 · 1 revision

Table of Contents

Welcome to PySPEDAS

PySPEDAS is an implementation of the SPEDAS framework in Python. PySPEDAS has tools for loading, analyzing, and plotting data from over 30 different directly supported satellite missions and ground magnetometer networks, as well as other sources like NASA's CDAWeb multi-mission data archive service, and HAPI (Heliophysics Application Programming Interface) servers.

PySPEDAS is a core package of the Python Heliophysics Community (pyhc.org) PyHC is working to promote standardization, interoperability, and open science principles among the various Python packages in this space (Astropy, SunPy, pysat, SpiceyPy, and others).

Requirements

As of PySPEDAS version 1.4.46, Python 3.8 or later is required. Note that PySPEDAS has dependencies on many other Python packages, some of which may need to be updated to work with newly released Python versions, OS versions, CPU architectures, etc. As of this writing (October 2023), PySPEDAS and its dependencies should be compatible with Python 3.8 through 3.10.

Certain packages may still have incompatibilities with Python 3.11 or Python 3.12. Other packages may depend on binary libraries that may not be available "out of the box" for certain OS versions or CPU architectures (for example, Mac laptops with M2 chips). It is possible that "pip install" may fail for such packages, if your environment doesn't have all the necessary compiler tools or prerequisite packages installed. If this happens to you while installing or updating PySPEDAS, please let the development team know, and we'll try to help you get things running.

pySPEDAS releases

Installing Python

If you're not already familiar with using Python, you may want to visit (python.org) and read some documentation, work through a few tutorials, and familiarize yourself a bit with the Python ecosystem, before taking the plunge and installing a Python environment. PyHC also has links to some resources for getting started with Python.

We suggest that you install Python using the Anaconda distribution; step-by-step instructions for installing Anaconda can be found at:

Once Anaconda is installed, you can use Anaconda Navigator as a graphical interface to create Python virtual environments (described more fully below), or install a variety of development environments and other useful tools.

Installing an Interactive Development Environment (IDE)

Strictly speaking, all you really need is a Python interpreter and a text editor to create and run PySPEDAS programs. But a good Python IDE will make it much easier to install, understand, navigate, and run PySPEDAS code. An IDE will generally let you create multiple independent projects, each with its own virtual environment. There will probably be a way to create a new project from a git or other version control repository. The PySPEDAS code and many usage examples are developed in several repositories on Github (for example, PySPEDAS, Pyspedas examples, MMS examples). (You may want to consider installing the git command line tools on your machine...some IDEs rely on having git available to connect your project to a Github repository.) Within a project, the IDE provide an area with editing tools where you can view or develop Python code. You will probably be able to highlight routine names and go directly to files where they're implemented, or hover over a routine name and get a tooltip showing the parameters it expects and other documentation.

Some popular IDEs for Python development include:

PyHC also has a list of suggested IDEs and other Python development tools.

Downloads, installation instructions, and other documentation is available at each of their respective web sites.

Developer install via Github, versus user install via "pip install"

When you're creating a new project, your IDE probably offers several ways to get started. The method you choose depends on how you'll be using PySPEDAS. If you'll be using PySPEDAS as more of an end user, and just want to install a stable, fully-tested package, you'll probably want to create a fresh virtual environment (described below), then install the PySPEDAS package using "pip install pyspedas".

If you are interested in contributing to PySPEDAS itself, or if you want to keep up with the "bleeding edge", and have access to new features and bug fixes as soon as they're committed to the PySPEDAS repository, you would probably want to use your IDE's option to create a project starting from a Github repository. If you take this route, you may need to have the Git command-line tools installed on your machine. See the Git web site for downloads and installation instructions. The URL for the PySPEDAS Github repo is https://github.com/spedas/pyspedas.git . Copy and paste this URL into your IDE's project setup panel, then follow the rest of your IDE's instructions for creating the project. If you use this method to get PySPEDAS, you don't need to do the "pip install pyspedas" step...you will already have the PySPEDAS source code cloned directly from the repository. However, you will still need to install several packages that PySPEDAS depends on. Your IDE may prompt you to do this as part of creating the project, or you might have to use a command like "pip install -r requirements.txt" to install the dependencies.

The PySPEDAS project has several other Github repositories containing code examples, many in the form of Jupyter notebooks:

The easiest way to work with these examples is to create a new project for each set of examples, starting from the Github URL. Then use "pip install pyspedas" to install pyspedas in this environment, and "pip install jupyter" to get the packages you'll need to run Jupyter notebooks.

Alternatively, you can download individual notebooks from Github, and save them in the same project where you installed PySPEDAS. Make sure that you're downloading the raw notebook file using the link Github offers when you navigate to the file, otherwise the notebook may download in the wrong format, and you won't be able to open it.

Creating a virtual environment

To avoid potential dependency issues with other Python packages, it’s best to create a dedicated virtual environment in Python for working with PySPEDAS.

Most IDEs will provide a way to create a new virtual environment when you start a new project. See your IDE documentation for the steps you'll need to take.

If you prefer, you can create a virtual environment in your terminal with:

<syntaxhighlight lang="cmd"> python -m venv pyspedas </syntaxhighlight>

And enter into that virtual environment by running the 'activate' script with:

macOS and Linux

<syntaxhighlight lang="cmd"> source pyspedas/bin/activate </syntaxhighlight>

Windows

<syntaxhighlight lang="cmd"> .\pyspedas\Scripts\activate </syntaxhighlight>

Installing PySPEDAS

If you're doing a user install of PySPEDAS (i.e. not from Github), the first time you enter your virtual environment, you’ll have to install pyspedas; this is as simple as using your IDE to open a terminal window (not an interactive Python window!), then typing

<syntaxhighlight lang="cmd"> pip install pyspedas </syntaxhighlight>

This should go out and find all of the required libraries and install them inside the virtual environment.

If you would like to upgrade your copy of the pySPEDAS libraries inside of your virtual environment, use:

<syntaxhighlight lang="cmd"> pip install pyspedas --upgrade </syntaxhighlight>

You'll probably want to install the packages for running Jupyter notebooks (for both user installs and Github installs):

<syntaxhighlight lang="cmd"> pip install jupyter </syntaxhighlight>

For certain PySPEDAS tools, there may be additional packages you'll need to install. For example, if you want to plot EICS/SECS data on top of a map of North America, you'll need the basemap package:

<syntaxhighlight lang="cmd"> pip install basemap </syntaxhighlight>

Setting your local data directory

The recommended way of setting your local data directory is to set the 'SPEDAS_DATA_DIR' environment variable. 'SPEDAS_DATA_DIR' acts as a root data directory for all missions, and will also be used by IDL (if you’re running a recent copy of the bleeding edge).

Mission specific data directories (e.g., 'MMS_DATA_DIR') can also be set, and these will override 'SPEDAS_DATA_DIR'

Network mirror for the MMS dataset

If you have a mirror of the MMS dataset on your local network, you may want to set the MMS_MIRROR_DATA_DIR environment variable.

If set, when you use the no_update keyword in the load routines (or if don’t have an internet connection), the load routines will check the mirror for data. Just as in IDL, data files found on the network mirror will be copied to your local data directory before loading them.

Checking that everything is working

The quickest way to check if everything is working is to load some data; to do this, use your IDE to start an interactive Python window (note: not a terminal window, as you used for the various "pip install" commands!). Then type the following Python commands:

<syntaxhighlight lang="python"> import pyspedas pyspedas.mms.fgm() </syntaxhighlight>

This should load some default data (srvy mode, probe 1) for Oct 16, 2015. You can then plot the FGM data with:

<syntaxhighlight lang="python"> from pytplot import tplot tplot('mms1_fgm_b_gse_srvy_l2') </syntaxhighlight>

Using Jupyter notebooks with your virtual environment

To get virtual environments working with Jupyter, there are a few extra steps:

If you haven't already installed the jupyter package in your virtual environment, you can get it by typing:

<syntaxhighlight lang="cmd"> pip install jupyter </syntaxhighlight>

Then, to start the Jupyter server:

<syntaxhighlight lang="cmd"> jupyter notebook </syntaxhighlight>

This should open a page in your browser, where you can navigate to the Juypyter notebook you want to run, and open it.

Your IDE may also support running notebooks directly, e.g. by double-clicking them in your project.

Recorded tutorials, presentation slides, notebooks

Google Colab Examples

Google offers a service called Google Colaboratory that lets you run Jupyter notebooks (stored in a Google Drive account) in the cloud, with no local Python installation needed. You will have to be signed in to your own Google account to use the service. The first time you try to run a Google Colab notebook, you may have to navigate to the Google Colab service from the "open with" prompt, and make sure Google Colab is enabled for your Google account.

Here are a couple of examples (caveat: at this writing, there may be package version conflicts between PySPEDAS and the Google Colab environment. We expect to have this resolved soon.)

You can also browse the README.md files using GitHub for more information and examples, e.g.,

Comparison of pySPEDAS to SPEDAS

Some examples that demonstrate how to achieve the same results using either IDL SPEDAS or python pySPEDAS.

Clone this wiki locally