python_project

A template setup for a python project.

It contains the following:

1. Setting up a python package
2. Tests for unittest and integration testing
3. TravisCI for continuous integration
4. Generating documentation locally and on readthedocs.org built using Sphinx and Doxygen
5. Packaging the library for distribution on PyPi

The template documentation for this project is on readthedocs.

Usage

• Clone this project:

git clone https://github.com/wasimusu/python_project.git

• Delete *lib.py files in python_project. This is where you will write your libraries
• Delete *.py files in tests dir. Add your own tests here.
• Replace python_project with your project name

Build Status

Platform

• Linux (Xenial/Ubuntu 18.04)
• Python 3.5, 3.7

1. Setting up a python package

python_project is the name of my project as well as the package. The overall structure of the project is as follows:

python_project
    docs
    python_project
        __init.py__
        mathlib.py
        strlib.py
    tests
        test_mathlib.py
        test_strlib.py
    usage
    .gitignore
    .travis.yml
    MANIFEST.in
    readme.md
    requirements.txt
    setup.py

Note: The dirs docs and usage are not shown in expanded form.

Description of some of the files included in the project:

• .gitignore: to avoid git versioning certain types of files
• .travis.yml: Travis CI
• MANIFEST.in: control what is included in the project packaged for distribution
• readme.md : description of the project
• requirements: project dependencies
• setup.py: for packaging and distribution

2. tests for unittest and integration testing

Discovers all the files named test*.py and runs all the unittest.

python -m unittest discover tests

3. TravisCI for continuous integration

Why CI?

• To build and test everytime changes are committed and pushed to a version control system.
• To let other users/contributors know whether your project would build and run successfully on their platform.
• The green build badges look cool and comforting ;)

Basically all you need is an account on travis-ci.com and link your github account. Whenever you push a project containing .travis.yml file to github the TravisCI is triggered. It builds the projects, runs tests and/or benchmarks as defined in the .travis.yml file.

Defining .travis.yml is simple. You only need to define what versions of Python you want to test and what platforms. You also need to provide instructions on how to run your code just like you'd explain on your github readme to potential users.

4. Generating Documentation

You can only go so far without good documentation. Great library developers make sure that they have great documentation. It helps to build a user base around the library. As a library developer, you can avoid lots of frustration to users and lots of effort on your part trying to resolve user issues by having good documentation. Here's how to generate good looking documentation on the localhost.

cd docs
doxygen -g Doxyfile

Now change some things in the Doxyfile that you have just generated.

• PROJECT_NAME = "python_project"
• GENERATE_LATEX = NO

doxygen Doxyfile
sphinx-quickstart

Then you'll get some prompts.

> Separate source and build directories (y/n) [n]: y
> Project name: python_project
> Author name(s): Wasim Akram Khan
> Project release []: 0.0.1
> Project language [en]:

You can see there are four files insides docs/source generated by sphinx: _static, _templates, conf.py, index.rst

The conf.py file defines how your source files are documented. You need to add the path of your package so that its files are reachable from conf.py.

Path of my conf.py: docs/source/conf.py

Add these two statements so conf.py can access your package.

import os
import sys
sys.path.insert(0, os.path.abspath('../..'))
sys.path.insert(0, os.path.abspath('../python_project'))

'extension' tells doc generator what kind of stuff to document and what to ignore. For instance, 'sphinx.ext.viewcode' generates link to the source code. Add the following to conf.py

# First two are essential to generate the docs, others are optional.
extensions = [
    'sphinx.ext.autodoc',
    'sphinx.ext.viewcode',
    'sphinx.ext.doctest',
    'sphinx.ext.intersphinx',
    'sphinx.ext.todo',
    'sphinx.ext.ifconfig',
    'sphinx.ext.githubpages',
]

We're done with the configuration. See the source code for conf.py. I have done away with the boilerplate stuff and

docs/source/index.rst file is like the readme for the documentation. The changes you make to this file is seen in your generated documentation. In this file, you define which python source files to document. You can include installation instructions, features, usage code snippets, etc in this file. We built two libraries in this project: mathlib and strlib. We want both libs to be documented. So let's add that to the index.rst above "Indices and tables".

Math Lib
========
.. automodule:: python_project.mathlib
   :members:

String Lib
==========
.. automodule:: python_project.strlib
   :members:

View the source code for index.rst. I have done away with the boilerplate stuff and made the documentation more custom.

make html

We have now setup the documentation. Open the docs/build/html/index.html in your browser to view the documentation on localhost.

If you're not seeing your source files indexed/documented then it's probably path error. Make sure your source files are discoverable. You can resolve this by adding appropriate paths using sys.path.insert() in the conf.py file.

To re-generate documentation locally each time you change the documentation:

cd docs
make clean
doxygen Doxyfile
make html

Open the docs/build/html/index.html in your browser to view the documentation on localhost.

Readthedocs

To generate documentation on readthedocs.org, do the following:

• Make an account on readthedocs.org if you have not already.
• On readthedocs.org, select your project for generating documentation and build.

5. Packaging the python_project for distribution on pypi.org

MANIFEST.in and setup.py are files that define the packaging. If you've followed the structure of this project, you're setup for distribution.

At this point, the project is ready to be published as a package on the pypi.org. Pypi.org functions like github with full version control of your packages and accompanying documentation. You'll need an account on it and save the credentials. Remember it's just like github.

See setup.py to see how to configure your own setup.py.

python setup.py sdist

This creates the package that will be uploaded on pypi.org. Twine makes it easy to upload it.

pip install twine
twine upload dist/*

You will be prompted for your pypi.org credentials.version

You can generate the package again after making some changes. You will need to increase the version number if you want to update your package in the pypi.org and upload it using twine.

Please refer to this for more detailed instructions to build your package and upload it to pypi.org.

Congratulations you now have a python distribution that is accessible to the world.version

Let's build something great!