Python Coding for Public Policy

Alternate course title: How to Use a Bit of Code to Do Things That Would Be Really Hard in Spreadsheets

This repository contains content for NYU Wagner's Python Coding for Public Policy class (PADM-GP 4506).

The materials are also available for public consumption. If you are not part of the class, see information about running the notebooks locally.

Important links

• Syllabus
• Brightspace site, which students will use for:
  • Viewing Announcements
  • Submitting Assignments
  • Viewing grades
• JupyterHub, where work will be completed

Produced and taught by Aidan Feldman. Largely based on previous iteration by Hannah Kates.

Why this class?

There are countless resources out there to learn Python and pandas — books, videos, etc. — and many are free. Relative to other Python/pandas courses, this class:

• Doesn't expect any prior technical experience
• Puts the data/code in a public policy context
  • There's a specific emphasis on learning coding for data analysis rather than software engineering
• Teaches you how to work with open data
• Optimizes for minimal setup

All the lectures and assignment templates are in this repository, so you could go through them on your own. The benefits of enrolling are:

• Additional content
  • You get access to lectures, which includes commentary that isn't in this repository
  • You get access to assignment solutions
• Support
  • There is an instructor to answer questions, both during and between lectures
  • You have peers you can work with
• You invest money and are expected to show up to class and turn in assignments on time, which makes it far more likely you will complete it

Assignments

Tips

• All lecture slides and homework templates can be found under class_materials/. The contents of this directory will be automatically updated from the GitHub repository, but should keep any changes you make.

• Look at the Table of Contents to get an overview. You can find the Table of Contents button in the sidebar.

• Read the instructions carefully. Like word problems from math class, they are very specific in what they are asking for.

• Spot check your results. If you are transforming data from a previous Step, compare the results, do a handful of the calculations manually, etc. to ensure that the results are correct.

• Don't repeat yourself (DRY). If you find yourself copying and pasting code within a notebook, there's probably a better way to do it.

• Avoid hard-coding values. Don't rely on things like row numbers or column order being stable, in case the dataset were to be updated.

• Include the boilerplate code for Plotly.

  import plotly.io as pio
  pio.renderers.default = "notebook_connected+pdf"

Kernel/memory issues

• Make sure Python [conda env:python-public-policy] is selected as the kernel.
  • Shows in the top right of the notebook interface
  • Change from Kernel menu → Change kernel → Python [conda env:python-public-policy]
• Make sure you aren't loading data sets you don't need.
• Close kernels you aren't using from the Running page.

If none of the above help, let the instructor know.

Turning in assignments

1. Ensure all the outputs are visible and the notebook is cleaned up.
   • What you see is what the instructors will see.
   • This is a good time to run the notebook end-to-end with ⏩ (Restart and run all).
   • See general scoring criteria.
2. Leave your name off the notebook filename and the notebook itself, as assignments are graded anonymously.
3. Export the notebook as a PDF. From the Jupyter interface, go to:
   1. File
   2. Download as
   3. PDF via LaTeX (PDF)
4. Glance through the PDF to ensure everything is showing up as you intend.
5. Upload the PDF to the Brightspace Assignment.

If you get a 500 error when trying to export the PDF, you may be outputting too much data. Try reducing your output (in the Jupyter sense) to smaller subsets.

After the resubmission deadline passes for each Assignment, the solutions will be posted in shared/solutions/.

Note: In-class exercises will not be graded.

Resources

• Office hours
• Wagner Quantitative Support
  • Tutoring
  • Math Review
• NYU Library Data Services
  • Consultation
  • Classes
• Stack Overflow
• GitHub Student Developer Pack
  • Includes learning resources and various tools

Pandas

• Documentation
• Cheat sheets
  • Official
  • DataCamp
  • DataQuest
• Python for Data Analysis book
• NYU's Quantitative Analysis Guide: Python

There are countless other blog posts, videos, books, etc. out there. There is no "best" resource, as individuals prefer different formats, come in with different experience, and learn at different speeds. Anything that comes up near the top of a Google search will likely be fine.

Learning more

Want to keep going after this class?

Running notebooks

• Some options:
  • Google Colab
  • Anaconda
  • Visual Studio Code
• May need to install/update packages to match those used in this class

Python fundamentals

Recommended focusing on fundamentals of Python 3. Many "learn Python" resources will be web development-oriented (they will probably mention Django/Flask), so you might want to look for ones that focus on data science or Python 3 on its own. Some that are data-oriented:

• DataCamp's Python Fundamentals or Python Programmer tracks
• Kaggle's Learn Python tutorials

Countless other "learn Python" resources/courses/videos/books out there; there isn't one right choice for everyone.

Machine learning

• Kaggle's Intro to Machine Learning
• Disco's machine learning content
• DataCamp's Natural Language Processing in Python track

NYU classes

• Advanced Data Analytics and Evidence Building
• Center for Urban Science + Progress
  • Applied Data Science
  • Machine Learning for Cities

See also

• 14 Principles of Open Government Data
• Blog post about GitHub for non-developers

Testimonials

This class has been extremely helpful and my only regret is that I didn't take it sooner in my NYU career. … In fact, I've already put python to use for my final thesis. … In writing my thesis, I used python for descriptive statistics that would have otherwise taken much longer in excel … Writing code for these analysis probably saved me about a day or two of work.

---

Thank you for an incredible semester. I truly took away a lot that I feel will help me dive deeper into a career in public service, while having unique skills that will help me deliver a much greater impact into the communities I'm serving.

---

The in-class assignments and homework provided many opportunities to practice the Python concepts we learned in class. … Professor Feldman was able to present difficult concepts in a way that was easy to digest as someone who knew very little about Python prior to this course. This course achieved its goal of removing the uneasiness I felt towards Python and coding in general, and for a 7-week course, that is no easy feat!

---

Everything controllable by the professor was fantastic. The objectives were straightforward and value-added. The professor helped make sure that we were coming away from assignments with real tangible understanding of the code rather than focusing on completion. I was challenged, but the professor was always around to answer questions.

Running the notebooks locally

Not recommended for people taking the class, unless looking for an extra challenge.

1. Install Miniconda.

2. Clone the repository.

3. Create the environment. From this directory, run:

   mamba env create --file environment.yml

4. Activate the environment:

   conda activate python-public-policy

5. Start the Jupyter server:

   jupyter notebook