Research at UC Berkeley using IPython

If you are at Berkeley and using IPython for your research, please add yourself to this list. If you have any publicly available materials, project websites, repositories or published papers or conference talks that you can link to, please do so. Also, please remember to provide us with your departmental affiliation as well as any personal site/github user you'd like to share.

• Kathryn (Katy) Huff - I use IPython and the IPython notebook for everything. In particular, I use IPython and the notebook for prototyping code that contributes to my nuclear engineering research. My most recent IPython notebook uses python packages such as PyNE and yt to generate a large 3-D regular mesh of the core of the Pebble-Bed Fluoride-Salt-Cooled High-Temperature Nuclear Reactor. This mesh and the materials described on it will be used as input for high-fidelity neutronics simulations using external codes such as Serpent, MCNP, and MOOSE.

• Min Ragan-Kelley (Plasma Theory and Simulation Group, Applied Science & Technology) We had an existing C++ plasma physics simulation application, with a primitive GUI (OOPIC, OOPD1). I removed the GUI code, and turned the application into a C++ library, which I exposed to Python via Cython wrappers. With the new Python API, I was able to use IPython notebooks to build and run simulations interactively, with immediate feedback, plotting (with matplotlib), and analysis (with numpy and scipy). This new code was applied to a system of bipolar flow of electrons and ions in a diode, allowing a automated optimization of physical parameters, a more efficient approach, which was able to reproduce results that had previously taken days, in just minutes. This work is currently under review for publication. I was also able to easily build HTML reports with nbconvert, which I frequently used to discuss results with my advisor, who had moved to a different university.

• Isaac Shivvers - I'm a graduate student in astrophysics, and I use the IPython notebook extensively in my research on the optical spectra of supernovae. I've found it incredibly useful as a lab notebook of sorts, keeping track of my ideas and calculations at the beginning of a new project as I try various approaches and try to understand the results. I've been able to run a parallel notebook cluster on one of the large astronomy computers and then access and run it through my browser on my laptop, which has made very long/difficult calculations much more friendly.

• Chris Holdgraf - I'm studying computational and cognitive neuroscience, using a method called electrocorticography to collect brain activity from surgical patients. I'm interested in understanding the mechanisms that the brain has for making sense of noisy or incomplete auditory information, and I investigate this by fitting linear regression models as well as doing classification. It's fairly code-intensive work, and IPython has drastically improved the readability and effectiveness of my codebase. I have 5 notebooks that I use for my analysis. My raw data goes into the first notebook, and then the final product of each notebook is saved to an HDF file so that it can be reopened by the next notebook. These are all version controlled, so it stores the complete history of my analysis for a particular set of data. It's also really useful because it lets me keep a visual record of my findings via the inline plots of the notebook.

• Beth Reid (Cosmology Data Science Fellow @ Berkeley Center for Cosmological Physics) - My work centers on making three-dimensional maps of the cosmos with galaxy redshift surveys, and using those maps to infer fundamental properties of our universe (dark matter, dark energy, gravity, galaxy formation, ...) I use IPython daily in my data analysis workflow. I was a Mathematica Notebook user in graduate school, but quickly decided when I saw the license fees that keeping my code in Mathematica was not sustainable through the frequent institutional shuffle of postdoc-hood. I transitioned to a C/Python combination, but really missed the notebook feature of Mathematica. It's the perfect format to save the scientific process -- wrangle/plot data and perform calculations until you conclude something ("I can ignore the XYZ effect for this analysis or Our data gives a 5 sigma detection of ZYX.") Saving the PROCESS to later look back at when you're writing a paper (or reconsidering why you made some decision 6 months ago) is invaluable. So that's why I started using IPython. However, I'm slowly learning that IPython has a lot more to offer as well -- for example, tab completion on functions and argument names makes analysis much quicker and smoother.

• Cindee Madison (Helen Wills Neuroscience Institute) - The lab uses multi-modal brain imaging to study healthy aging and dementia. We use IPython notebooks as part of our provenance tracking for many different kinds of biomarker and cognitive data processing. Most studies have an ipython notebook directory with a number of relevent notebooks for many stages of data analysis. In addition I use IPython notebooks to teach everything from image processing, to machine learning, to basic statistics in the lab.

• Dav Clark (D-Lab) - I and other instructors use IPython notebooks extensively in teaching, and we are actively building out our research arm on building effective and inclusive teaching. Likewise, for my dissertation work on the attitude & belief impacts of climate change education, I used IPython notebooks extensively to provide a narrative along with data processing steps, before evaluating models and visualizing data in R using rpy2 (again, in the notebook).

• Jesse Livezey (Redwood Center for Theoretical Neuroscience; DeWeese Lab) - I use IPython notebooks for the prototyping and analysis phases of projects. I build models of neural networks in the brain. During the initial phases of development, the notebook is very useful for quickly writing, testing, and iterating code. I usually then move this code into scripts and modules as it matures. Finally, during analysis and plotting, the ability to quickly tweak and re-run commands (e.g. matplotlib) is great.

• Justin Kitzes (Energy and Resources Group) - Although I do most of my programming in "plain" Python, I use the IPython notebooks and interpreter extensively when prototyping my code. I also use the notebooks extensively in teaching Software Carpentry bootcamps.

• Prabhavathi Matta (School of Information, UC Berkeley) - I mostly use python in my daily programming, however I tried to do web-programming on IPython notebook last semester. I could successfully manage to create and publish webpages, however found very frustrating to integrate IPython with d3 javascript visualization. I am currently taking a course on advance data analytics concepts using Pandas in IPython notebook. Hopefully, I even find a good solution to integrate d3 visualizations in IPython notebooks by the end of this semester. If anyone has already found a solution, please do share with us.

• Stephen Bailey (LBL Physics) - I use the IPython command line as my standard interactive python session for debugging development for several cosmology projects (SDSS-III/BOSS, SDSS-IV/eBOSS, CosmoData, DESI). I dabble with the IPython notebook and IPython parallelism, but don't use them extensively.

• Jan Roden (Whaley Group, theoretical quantum physics) - We use IPython Notebooks in our group as GUI for quantum dynamics and spectroscopy calculations, as well as for documentation and exchanging examples. It's a really nice tool. I like that everything, plotting, etc., is integrated in a notebook that can be used in just a browser -- really good idea and good work.

• James Gao (Gallant lab, Vision Science Program) - We use IPython extensively in our research. Using human fMRI data, we build encoding models of the brain while subjects view natural movies. This enables us to predict and decode the images that these subjects are viewing. I've used IPython to develop an MRI visualization toolkit, pycortex, for use with MRI surface-based visualization.

• Jess Hamrick (Psychology, Computational Cognitive Science Lab) - I have been using IPython for several years, and the IPython Notebook for about a year and a half. I do almost all of my prototyping now in the notebook, including everything from modeling to data analysis. My workflow usually involves writing some initial code in the notebook, and as it gets large enough or complex enough, I will break it down and move it out into modules -- the notebook makes this type of development incredibly easy! I also use the notebook extensively for data analysis; being able to display plots and results inline makes it so easy to show other people what I've been working on and makes it easier for me to keep track of where I am in a project.

• Nick Swanson-Hysell (Assistant Professor in Earth and Planetary Science (www.swanson-hysell.org)) - In my research group, we are increasingly doing data analysis and plotting within the IPython environment. One of the major advantages is the ability for us display data, generate plots and preform statistical analyses within a single notebook that can then easily be collaborated on and shared. The most recent articles we have submitted have had IPython notebooks as supplementary materials where we provide access to the data and code within a github repository (e.g. http://github.com/Swanson-Hysell/2014_Swanson-Hysell-et-al_Osler) and use nbconvert to convert the notebook to LaTEX then a PDF to be published alongside the article. We see IPython as a critical tool for doing and documenting data analysis with the goal that publishing IPython notebooks will make it possible for other researchers to easily reproduce (and see the guts of) analyses of our data.