COMP598_Fall2020 Software Engineering for Building Intelligent Systems

General Information

Instructor	Jin Guo
TA	Breandan Considine
Class Time	TR 11:35 am-12:55 pm
TA Office Hours	W 11:00 am-12:00 pm
Remote Lecture	Zoom (link through MyCourse)
Discussion Forum	Slack

• TODO before attending the first lecture:

1. Please fill in this Background Survey. I will try my best to accommodate your availability, background, and expectations of the course so having the input from you is extremely important.
2. Please join the Slack workspace and introduce yourself to the cohort. From here, we hope to know you and work with you as a collaborator over this semester.

• Due to the COVID-19 pandemic, this course will be taught remotely. The detailed format will be updated soon.

• I normally design many in-class activities for upper-level classes to motivate discussion and collaborative learning. Therefore, it is important to attend the lectures in order to gain the best learning experience – it cannot be replaced by watching the videos afterward. Given the special circumstance of this semester, I won't require you to attend all the lectures in real-time and try to balance the in-class and out-of-class effort. However, I would encourage you to register next year if you cannot attend most of the lectures.

Description

This course is going to explore how to build an intelligent system from a software engineering perspective, from requirement gathering and analysis to deployment and maintenance. We will also touch AI ethics and its implications to design.

Prerequisite

While there are no official prerequisite courses, you will enjoy and appreciate this course more if you have taken COMP303, COMP424 and COMP551 already.

Reference Material

We will not concentrate on any particular resources. Instead, the readings will include content from book chapters, research papers, blog posts, talks, etc. The pointers to those content will be added to the schedule later.

• Books:
  • Building Intelligent Systems A Guide to Machine Learning Engineering (access through McGill Library)
  • Thinking in Systems: A Primer (Chapter scans in Shared OneDrive folder)
  • Human Compatible: AI and the Problem of Control (Chapter scans in Shared OneDrive folder)

Assessment and Evaluation (Tentative)

Assessment Method	Weight
Participation (inclass and online)	10%
Assignment	60%
Final Project	30%

• Any form of plagiarism, cheating is strictly banned during midterm or final exam. Integrity is crucial to this course and your future career. Any violation against academic integrity will be taken very seriously. For more information, please refer here.

Schedule (Tentative)

Subject to adjustments

Lecture	Date	Content	Reading	Note	Discussant
1	3 Sep	Introduction	BIS book: Chapter 1, 2 TIS book: Intro (Onedrive)
2	8 Sep	Intro to Modern Software Engineering	GOTO 2020 Talk• Taking Back "Software Engineering" Quality Attributes
3	10 Sep	Intro to Modern Software Engineering (cont'd)
4	15 Sep	AI basics (focusing on ML)	BIS book: Chapter 16, 17, 18 Human Compatible: Intelligence (Onedrive)
5	17 Sep	Model Selection and Evaluation	BIS book: Chapter 19, 20 How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation	Assignment1 Due (20 Sep)
6	22 Sep	Model Selection and Evaluation (cont'd)	Hidden Technical Debt in Machine Learning Systems		Fuyuan Lyu
7	24 Sep	Model -> System	Software Engineering for Machine Learning: A Case Study TIS book: Chapter 4 Why Systems Suprise Us (Onedrive)
8	29 Sep	Data Acquisition & Management	BIS book: Chapter 9 A Survey on Data Collection for Machine Learning: A Big Data - AI Integration Perspective
9	1 Oct	Data Acquisition & Management (cont'd)			Kangrui Ren
10	6 Oct	Requirement for (and) AI	Requirements Engineering for Machine Learning: Perspectives from Data Scientists		Alexa Hernandez
R1	7 Oct	Project M1 Recitation	Project Intro: Movie Recommendation		Breandan Considine
11	8 Oct	Requirement for (and) AI (cont'd) Human-AI Interaction Design	How Good is 85%? A Survey Tool to Connect Classifier Evaluation to Acceptability of Accuracy		Grace Hu
12	13 Oct	Guest Lecture: Jinghui Cheng -- Designing usable machine-learning based applications	Guidelines for Human-AI Interaction		Younes Boubekeur
13	15 Oct	Human-AI Interaction Design	BIS book: Chapter 5-8 Human-Centered Artificial Intelligence: Three Fresh Ideas		Mohammad Amin Mozaffari
14	20 Oct	Data Quality Assessment	Data validation for machine learning Automating Large-Scale Data Quality Verification	Project M1 Due	Lucas Turpin
15	22 Oct	Quality Assessment	The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction BIS book: Chapter 15		Edrick Da Corte Henriquez
16	27 Oct	Continuous Delivery
R2	28 Oct	Project M2 Recitation	Concepts in Statistical and Software Testing		Breandan Considine
17	29 Oct	Class Canceled	Principled Artificial Intelligence: Mapping Consensus in Ethical and Rights-based Approaches to Principles for AI	Assignment2 Due (29 Oct)
18	3 Nov	AI principles Overview
R3	4 Nov	Project M2 Recitation	CI/CD tools and Practices for Effective Remote Development		Breandan Considine
19	5 Nov	Safety	Autonomous Vehicle Safety: An Interdisciplinary Challenge An Analysis of ISO 26262: Using Machine Learning Safely in Automotive Software	Project M2 Due (12 Nov)	Sogol Masoumzadeh
20	10 Nov	Security and Privacy	SoK: Towards the Science of Security and Privacy in Machine Learning Designing privacy-aware internet of things applications		Yechuan Shi
21	12 Nov	Accountability/Auditing	Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims (Section 2 and 3)		Jazlyn Hellman
22	17 Nov	Transparent and Explainability	Explainable machine learning in deployment Designing Theory-Driven User-Centric Explainable AI		Eddie Cai
23	19 Nov	Transparent and Explainability			Jintao Guo
24	24 Nov	Fairness	Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI	Assignment3 Due (24 Nov)	Andrea Jiang
R4	25 Nov	Project M3 Recitation	Software Release Planning and Monitoring		Breandan Considine
25	26 Nov	Value in AI Design	“The Human Body is a Black Box”: Supporting Clinical Decision-Making with Deep Learning
--	3 Dec	--		Project M3 Due
--	7 Dec	--		Project Final Report Due
--	8 Dec	Final Presentation

Credit:

The content of this course is greatly inspired by CMU 17-445/645: Software Engineering for AI-Enabled Systems which is developed by Christian Kästner et. al.

License

Unless otherwise noted, the content of this repository is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.