README.md

Continuous Integration via CircleCI

Here's a demo video for this repository. Please unmute the video if you'd like sound or watch with a transcript here. Resolution improves in full screen.

DEPP.434-55.DV2_default.mp4

continuous integration repo

in three phases: building, developing, releasing

1. building the repo

• step 01: new github repository**

• step 02: create Public with a simple README.md and .gitignore template for python

• step 03: log into https://circleci.com with github

• step 04: select your organization

• step 05: select cirepo project (circleci populates this with your github repos)

• step 06: circleci needs a .circleci/config.yml file (this is a bug later)

• step 07: from terminal or console, clone github repo and create these files:
  • Makefile
  • requirements.txt
  • cirepolib folder
    • __init__.py
    • cirepomod.py
  • tests folder
    • test_cirepo.py
  • .circleci folder
    • config.yml

• step 08: from terminal or console, edit those files with nano or vim:

• Makefile
  • setup for a python virtual environment
  • activate activate the virtual environment (does not work)
  • install required dependencies
  • test validates via unit tests of
    • python code
    • jupyter notebook
  • lint for warnings & errors in cirepolib and later 'cli.py'
    • R disables recoomendations
    • C disables configurations
  • all performs install, lint, and test

• 'requirements.txt`

• cirepomod.py
  • myfunc returns 1
  • call_web_service' will be ued by cli.py` later

• test_cirepo.py
  • imports cirepomod module from cirepolib library
  • test_func asserts that cirepomod.my_func == 1

• .circleci/config.yml is config for our SaaS
  • 3 run steps - make install
    • make test
    • make lint
  • lines with name: will show up in Circle Ci as build steps
    • install dependences
    • run tests
    • run lint

• step 12: try out make setup, make activate, and make install

• step 12 (continued): make test which fails when jupyter notebook.ipynb is not found

• step 13: start jupyter notebook

• step 14: pull down from New to Python 3 to create a new notebook

• step 15: name the notebook notebook, import cirepomod module, and populate with my_func, print_name, and fake_data

• step 16: save the notebook

• step 17: try out make test again, and try out make lint for the 1st time

• step 18: push master to origin at github
  • git status (notice anything missing?)
  • git add all with *
  • git commit -m 'initial commit'
  • 'git push`

• step 19: circleci has the build job Queued (but there is an Build Error already)

• step 19 (continued): circleci Config Processing Error because missing the .circleci/config.yml file (unsure why)

• step 20: push master again to origin at github with .circleci/config.yml file this time (it had been deleted)
  • git status
  • git add all with *
  • git commit -m 'adding circleci/config.yml'
  • git push

• step 21: circleci has the build job Queued (build indicates no error so far)

• step 21 (continued): circleci is Running

• step 21 (continued): circleci Success (notice the named steps, install dependencies, run tests, and run lint, from config.yml)

continuous integration repo

2. new repo features

• step 01: from terminal or console, create a new branch named feature_depp_20200925 via git checkout -b feature_depp_20200925

• step 02: from terminal or console, create cli.py file and start either of nano or vim editors

• step 03: copy and paste code for the command line tool, cli.py, that ...
  • imports cirepolib modules, json, click, and Flask
  • set click.option for --name
  • echo print_name module output
  • echo url's for hello, fakedata, and wname functions
  • define Flask app functions for each app.route
  • pylint fails if hello(), fakedata(), and wname() functions are not called

• step 04: test cli.py locally by running ./cli.py --name steve which returns ...
  • "steve-apple" in blue
  • one localhost url for each of 3 functions
• step 05: paste the 1st url, http://localhost:5000/, into browser

• step 06: paste the 2nd url, http://localhost:5000/fakedata, into browser

• step 07: paste the 3rd url, http://localhost:5000/wname, into browser

• step 08: from a terminal or console, open a jupyter notebook

• step 09: run cells 1 to 4, copy & paste !./cli.py --name steve into a new 5th cell, and run that 5th cell to return 3 urls
• step 10: click on the 1st url http://localhost:5000/

• step 11: click on the 2nd url http://localhost:5000/fakedata

• step 12: click on the 3rd url http://localhost:5000/wname

• step 13: remove the notebook's 5th cell created earlier; otherwise, the notebook test function pytest --nbval loops endlessly waiting for response

• step 14: in a terminal or console, cntr-c followed by y kills the notebook

• step 15: open Makefile with nano or vim, and ppend cli tests/test_cirepo to the make lint script so those functions are linted

• step 16: test linting locally with make lint

• step 17: in a terminal or console, separately add & commit the cli.py and Makefile changes with descriptive annotations to this feature_depp~20200925 branch

• step 18: to git push the locally committed features to the origin adding --set-upstream origin feature_depp20200925 so upstream origin knows to make a this branch

• step 19: observe circleci successfully building and testing feature_branch_depp_20200925

• step 20: on github's website, observe the origin repository received pushes from our remote terminal commands, and awaits a Compare & pull request; tap that green button

• step 21: annotate the pull request, here with this looks good

• step 22: observe the interacting annotations between push and pull actions; tap the Merge pull request and then Confirm merge

• step 23: observe that the Pull request successfully merged and closed indicating the 2 features are successfully INTEGRATED into the origin master branch of the cepo repository

(in summary, we have added features to feature_depp_20200925 remotely and then pushed and merged feature_depp_20200925 with the origin master repository, but the remote master repository does not know these features yet)

continuous integration repo

3. 1st production release

• step 01: in a terminal or console, switch from feature_depp_20200925 branch to master via git checkout master and then git pull the newly merged master branch with new features from origin

• step 02: from terminal or console, create a new production branch via git checkout -b production and git push that branch to origin via git push --set-upstream origin production
• step 03: siwtch from production to the master (devlopment) branch and open a nano or vim editor on README.md

• step 04: add a circleci badge to README.md to indicate build success; https://circleci.com/docs/2.0/status-badges/ explains badge configurations

• step 05: from terminal or console, git add and git commit this README.md with annotation to indicate this is the 1st production release
• step 06: switch to production branch and git merge with the master branch; then git push sending this production branch to github and circleci for building and testing

(previously, branches merged via github.com, and now branches are merged via terminal or console; fewer annotations with this latter method)

• step 07: on the circleci.com website, observe the install, test, lint and other stage successes

• step 08: back at github.com, observe the master branch with circleci badge; observe the message confirming production branch pushed

• step 09: on github.com, pull down from master to select production branch

• step 10: on github.com, in the production branch, notice the circleci badge and annotation for the 1st production release