Skip to content

hombit/scientific_python

Repository files navigation

Materials for Scientific Python Course for astronomers of Moscow University

Build Status

How to use this repository

Read files from scientific_python folder in alphabetical order and try to understand how all prints and asserts work.

Repository structure

This repository has a structure of a Python package with some additional files.

  • scientific_python folder represents top level package of the same name. This package contains several sub-packages and modules. You can start with a_intro sub-package and read its modules one by one in alphabetical order. All modules can be used as separate modules and scripts without package installation
  • bin folder contains scripts that can be used after package installation. Now they are used for testing
  • doc folder is used for documentation. Now it contains only MS Word file with course annotation (in Russian)
  • misc contains some additional course materials, e.g. sample Python project and staff related to Juyter notebooks used in class:
    • sample_project is an example of basic Python project with setup.py, a package and a subpackage, tests and a script. Students can use it as a base for their course projects
    • jupyter_notebooks contains notebooks and other files used in class sorted by date
    • share_jupyter contains little Docker project that runs my class Jupyter server and exposes notebook HTML copies to the world on sai.homb.it
  • setup.py is used to install this package
  • requirements.txt file contains Python dependencies of the project
  • virtualenv_activation.sh is a sample shell script (for *nix systems only) that can be used to activate virtualenv and install the package. Use it by typing . virtualenv_activation.sh or source virtualenv_activation.sh. For exit virtualenv type deactivate
  • Dockerfile and docker-compose.yml files can be used to run the project inside Docker container
  • .gitignore and .gitattributes are git files
  • .dockerignore is just a link to .gitignore, it used to prevent load garbage into Docker container
  • .travis.yml is a Travis configuration file. Travis is a continuous integration (CI) system used to test this project with various Python versions, 2.7 and 3.5+ are supported

Useful links

Install Python

Remember to use Python 3, 3.6 and later is good enough in 2018. You can check python version typing in console python3 --version or import sys; print(sys.version) in Python itself

All desktop platforms

  • Anaconda Python distribution is a good choice for scientific Python programming on every platform. It includes a lot of pre-compiled numerical and scientific packages and conda package manager where you can find even more packages, like astropy or scikit-learn
  • Official Python distribution: good on Windows or macOS when you'd like to build your environment from scratch

macOS

Instead of official Python distribution I recommend to use Homebrew package manager, install it and type brew install python

Linux

Probably you already have Python 3, check its version before start. If you haven't use your package manager

iOS

iOS doesn't have any application to use for scientific programming. The best choice is Pythonista paid application that can run and edit Python 3.6 code and supports numpy package, but doesn't support scipy and other useful packages

Android

PyDroid 3 looks good

Source code editors for Python

  • Spyder: the scientific Python development environment, if you have Anaconda, you probably have Spyder
  • Visual Studio Code (do not be confused with Visual Studio, they are two different products): a powerful source code editor. GitHub has integrated web-based Visual Studio Code called GitHub Codespaces
  • IDLE: a simple Python source code editor. It is a part of Python standard library, so if you have Python, you probably have IDLE
  • PyCharm: a powerful Python IDE (integrated development environment). PyCharm is closed source product, but Community edition is free to use and every student or professor can ask for a free professional version
  • Jupyter Notebook: not an editor in the usual sense, but powerful web-based tool for running Python (among with other language) code for data analysis. You can run Jupyter server on your own hardware or try it on Azure Notebooks or Google Colab
  • Almost all popular interlanguage code editors supports Python

Dive into Python

General documentation

Git

Use some version control system. Git is a good choice, see some tutorials:

Classes materials (2018)

Classes had place in classroom 48 of Sternberg Astronomical Institute MSU at 9:00 on Wednesdays from September to December 2018. Records of on-line translations of the seminars are hosted on YouTube. Development of the course in 2018 is supported by BASIS foundation.

Date Description Materials Links Video (in Russian)
2018.09.05 Introduction, coursework requirements. About Python. Numbers, lists, if-else, while, Boolean variables a_intro.basics, a_intro.sequences (the first part about lists) Python documentation, see section Dive into Python above link
2018.09.12 Built-in collections: tuples, dictionaries, sets. for-in, in. a_intro.sequences "Loop better: A deeper look at iteration in Python", "Hash table" Wikipedia article, CPython implementation of lists and tuples link
2018.09.19 Strings: unicode and formating. Functions: functions as objects, lambdas, argument packing and unpacking a_intro.strings, b_modules.functions, dump of iPython session String formating (use "new" syntax), keyword arguments link
2018.09.26 Some more details about strings: methods and re. Generators, list comprehance syntax. Python source file as module, import system. Introduction to classes. b_modules, *.py files used on seminar Regular expressions (in Russian), iteration and generators, scope and namespaces, modules link
2018.10.03 Jupyter notebook: a good way to use Python and another languages. Introduction in numpy c_numpy.arrays, Jupyter notebook Jupyter website, Jupyter guide, numpy user guide link
2018.10.10 Multidimensional arrays in numpy: reshaping, broadcasting, stacking. Review of numpy subpackages c_numpy, Jupyter notebook numpy user guide, "An introduction to Numpy and Scipy" link
2018.10.17 Read of files and cats. open() builtin, with-as statement, textual and binary files. Tabular data: np.genfromtxt and pandas Jupiter notebook and data files open() documentation, standard library modules for path and file manipulations: os.path and shutil, "10 minutes to pandas" link
2018.10.24 matplotlib: basics and examples. Introduction to scipy, scipy.integrate, scipy.optimize Jupyter notebooks "Scientific graphics in Python" (In Russian), Python matplotlib guide, "An introduction to Numpy and Scipy" link
2018.10.31 Python packages and how to prepare them. Testing, unittest. Example of class usage Sample Python project, unit test example, Jupyter notebook with Parabola class "Python modules and packages", virtualenv docs, pipenv docs, official packaging tutorial, How to Publish Your Package on PyPI, unittest docs, numpy.testing docs, pytest link
2018.11.07 Introduction to Astropy. Constants, units, quantities. Coordinates: sky coordinates, Earth coordinates, transformations between frames. Brief introduction to astropy.io: FITS ans ASCII. Jupyter notebooks astropy docs: tutorials, constants, units, io interface, coordinates link
2018.11.14 Least squares method usage with scipy.optimize. Module astroquery for web-access to astronomical data bases, e.g. Vizier and SIMBAD. Problem of cosmological parameters fit using SN Ia data. Jupyter notebooks scipy.optimize tutorial, lmfit module with pretty interface for least squares. astroquery docs: A Gallery of Queries, GAIA via TAP+, SIMBAD, Vizier link
2018.11.21 Sky coordinate match: astropy.coordinates. Problem of transient object discovery on FITS image: photutils, astroquery, astropy.wcs and astropy.coordinates Jupyter notebooks astropy tutorial Separations, Catalog Matching, and Related Functionality, photutils docs: source detection, aperture photometry link
2018.11.28 Seminar on listeners' requests. Student Alexey Nikonov tells about annotation and animation in matplotlib. Machine learning in Python Jupyter notebooks matplotlib annotation guide, Matplotlib Animation Tutorial. scikit-learn: quick start, choosing the right estimator. "Introduction to machine learning" on Coursera (in Russian) link
2018.12.05 Speed up Python code. Why Python functions and loops are slow and how to overcome it. Use the power of numpy. Gentle trick to speed up Python function or class: numba just in time (JIT) compiler. On listeners' request – web programming: flask and requests Jupyter notebooks, C++ and Python codes, f_speed "High Performance Python". numba: 5 minute guide. Web frameworks: http.server, django, flask. Web clients: urllib, requests. HTML/XML parsers: html and xml, beautifulsoap, lxml link
2018.12.12 Parallel exection of Python code. GIL (global interpreter lock) and threading, why you usually should use threads only for IO. multiprocessing and pickle: forking Python to isolate parallel workers. Pools and queues. Jupyter notebooks "It isn't Easy to Remove the GIL", "High Performance Python". Python docs: GIL multiprocessing, queue link
2018.12.19 System calls, interaction with C/C++ libraries with ctypes, Cython programming language Jupyter notebook "High Performance Python", ctypes module, Cython language link

Classes materials (2017)

Seminars had place in classroom 17 of Sternberg Astronomical Institute MSU at 13:30 on Fridays from September to December 2017. Records of on-line translations of the seminars are hosted on YouTube.

Date Description Materials Video (in Russian)
2017.09.15 Introduction, coursework requirements. Why Python 3? Numbers, lists, if-else, loops. a_intro.basics, a_intro.sequences link
2017.09.22 Boolean variables, lists, tuples, dictionaries, sets. Strings and their formatting. Functions, arguments packing and unpacking. a_intro.sequences, a_intro.strings, b_modules.functions link
2017.09.29 Functions: default values of keyword arguments, docstrings. Iterators and generators. Modules: file.py as a module. b_modules.* link
2017.10.06 Jupyter notebooks. Read and write files and cats. Introduction to numpy: one-dimensional arrays and indexing. Notebooks, c_numpy.arrays link
2017.10.13 numpy: multidimensional arrays, read tabular data files. Notebooks, c_numpy.multidim_arrays link
2017.10.20 Read tabular data files with numpy and pandas. Figure plotting with matplotlib. Notebooks link
2017.10.27 scipy: integration, interpolation, optimization. Short overview of other features. Notebook, d_scipy.* link
2017.11.03 Introduction to astropy: physical and astronomical constants, quantity calculations, sky coordinates, image manipulation, read and write data. Notebook link
2017.11.10 Packaging of Python project. Classes: example and magic methods. Unit testing. Notebook and script, setup.py of this project, e_testing.* link
2017.11.17 Two examples of astropy, astroquery and photutils usage: Hubble diagram fitting and transient object discovery. Notebooks link
2017.11.24 Dr. Ivan Zolotukhin tells about Django web framework, scientific web programming and model-template-view paradigm. Scripts
2017.12.01 Student Nikita Utkin tells about argparse. Speed up Python script: why Python is slow, why to avoid loops and why we should know how Python works, numba as a simple way to speed up calculations. Scripts and notebook, c_numpy.arrays link
2017.12.08 Parallel execution of Python code. threading and its limitations due GIL. multiprocessing and its limitations due serialisation via pickle. Notebooks link
2017.12.15 Cython language and C-code usage with Python. Python, Cython and C code, f_speed.compilers and Cython/C files for this module, setup.py link

License

Copyright (c) 2017-2018, Konstantin L. Malanchev.

All program code in this repository is distributed under the terms of the MIT license. All data files are properties of their authors, see COPYRIGHT_NOTE files in folders with data files