Citation (if you use the material): Hong-Linh Truong, Advanced Topics in Software Systems, https://version.aalto.fi/gitlab/sys4bigml/cs-e4660, 2020 BIB Entry
Copyrights/Licences: the lecture slides and course structure/info use CC BY 4.0. Individual tutorials have their own licenses (Apache Apache License 2.0)
The current focus of this course is on software systems for big data and machine learning. Complex Big Data and Machine Learning (ML) applications and services and their reliability and robustness are strongly dependent on the underlying systems empowering such applications and services. On the one hand, techniques for supporting performance engineering, configuration management, testing and debugging of Big Data and ML are extremely important. On the other hand, large-scale distributed systems and new computing models have been evolved with new hardware and infrastructure architectures, such as edge systems, tensor processing units, and quantum computing systems. Such systems and computing models are being exploited for advanced Big Data and ML applications and services. Developing and optimizing Big Data and ML applications and services in such systems and models require in-depth understanding of the systems and the roles of systems for Big Data and ML. Currently, we lack courses to deal with the above-mentioned issues, especially in-depth analysis and forward-looking research directions in (software) systems for Big Data and ML. This course will study advanced topics in systems for big data and ML/AI.
The course is for students in Doctoral and Master studies. In Aalto the course is for students in Doctoral Programme in Science and the CCIS Master Programme, especially for Big Data and Large-scale computing track and Software Systems and Technologies track.
This course provides advanced knowledge about computing and software systems that are useful for big data and machine learning domains. Therefore, it connects to various other courses, such as Big Data Platforms, Cloud Computing, Deep Learning and Master thesis, by providing complementary in-depth knowledge w.r.t system aspects.
Students should have knowledge about cloud computing, big data, operating systems, distributed systems and machine learning. Therefore, it is important that students have passed courses with these topics, such as Cloud Computing, Big Data Platforms, Operating Systems, and Machine Learning.
This course is an advanced one, aiming at supporting research topics in master theses and PhD studies. Therefore, it is not a prerequisite for any course.
First, key system requirements due to the complexity, reliability, and robustness of Big Data and ML applications and services will be analyzed and presented. Based on that we will learn techniques for supporting performance engineering, configuration management, testing and debugging of Big Data and ML. Such techniques are extremely important; they are cross-topics for the course, regardless of the underlying systems empowering Big Data and ML applications and services.
Second, selected areas in systems for Big Data and ML will be presented. For each selected area, we will examine the state-of-the-art, strengths and weakness of concepts and techniques. We will focus on engineering frameworks that can be used to development Big Data and ML, according to the above-mentioned cross-topics. The selected areas in 2020-2022 study plan will be:
- Dataflows/programming frameworks and orchestration techniques
- Edge systems and edge-cloud continuum systems
- New hardware architectures and quantum systems
Cross-topics in these selected areas will be studied. For each selected area, we will focus on the following aspects:
- understanding and applying key techniques and concepts
- analyzing/evaluating/creating (new) methods/techniques
Advanced Topics in Systems for Big Data and ML will focus on the following areas:
- Design and evaluation for systems robustness, reliability, resilience and elasticity for Big Data/ML (with also engineering work)
- Test, debug, monitoring, and configuration management (with also engineering work)
- Dataflows and Orchestration Frameworks for Big Data/ML (with also engineering work)
- Edge systems and edge-cloud continuum for Big Data/ML (with also engineering work)
We define the generic plan of the course as follows:
- Lectures given by teachers: students must provide study logs
- Hands-on tutorials given by teachers: the goal is to give some concrete examples of the techniques discussed in the lectures. However, since it is a research-oriented course, students can also practice similar problems with different software stack.
- Project topic proposal and presentation: students must identify a topic related to the content of the course and present it
- Topic implementation and demonstration: students will implement the topic and demonstrate the project
- Students will make public material about the topic project available in Git spaces (e.g., in Aalto, Github, Gitlab, ...)
As an advanced and research-oriented course, we will use the pass/fail as a way to evaluate students. Passing the course will require the students to (i) participating in lectures and hands-on, (ii) passing study logs, (iii) passing project topic presentation, and (iv) passing the final demonstration.
- Responsible teacher: Hong-Linh Truong
- Other teachers/assistants: Tri Nguyen
- Lecture 1:
- Slides: Robustness, Reliability, Resilience and Elasticity (R3E) for Big Data/Machine Learning Systems
- Key reading 1: R3E -An Approach to Robustness, Reliability, Resilience and Elasticity Engineering for End-to-End Machine Learning Systems
- Key reading 2: The New Frontier of Machine Learning Systems
- Key reading 3: Hidden Technical Debt in Machine Learning Systems
- Key reading 4: Declarative Machine Learning Systems
- Lecture 2
- Slides: Benchmarking, Monitoring, Observability and Experimenting for Big Data and Machine Learning Systems
- Key reading 1: Benchmarking big data systems: A survey
- Key reading 2: MLPERF Training Benchmark
- Key reading 3: Data Validation for Machine Learning
- Key reading 4: Developments in MLflow: A System to Accelerate the Machine Learning Lifecycle and ModelDB: a system for machine learning model management
- Key reading 5:Machine Learning Testing: Survey, Landscapes and Horizons
- Site 1: MLCommons
- The collection of Putting Machine Learning into Production Systems is also useful
- Lecture 3
- Slides: Coordination Models and Techniques for Big Data and Machine Learning Systems
- Key reading 1: Cirrus: a Serverless Framework for End-to-end ML Workflows
- Key reading 2: Towards ML Engineering: A Brief History Of TensorFlow Extended (TFX)
- Key reading 3: Orchestrating Big Data Analysis Workflows in the Cloud: Research Challenges, Survey, and Future Directions
- Key reading 4: KeystoneML: Optimizing Pipelines for Large-Scale Advanced Analytics
- Key reading 5: Jeff Smith. 2018. Machine Learning Systems: Designs that scale (1st. ed.). Manning Publications Co., USA.
- Key reading 6: Prediction-Serving Systems
- Lecture 4
- Slides: Machine Learning with Edge-centric Systems
- Key reading 1: Serving deep neural networks at the cloud edge for vision applications on mobile platforms
- Key reading 2:From the Edge to the Cloud: Model Serving in ML.NET
- Key reading 3: Machine Learning at Facebook:Understanding Inference at the Edge
- Key reading 4: Distributing Deep Neural Networks with Containerized Partitions at the Edge
- Key reading 5: A survey of federated learning for edge computing: Research problems and solutions
If you need the sources of slides for your teaching, pls. contact Linh Truong
- End-to-end ML systems development
- Observability and Monitoring
- Machine Learning Serving
- Qualty of Analytics for ML
- Edge ML Pipeline
- Common tasks with Edge ML
- Students will propose the project idea. This is an important aspect of research-oriented course. If a student cannot propose an idea, the teacher will suggest some concrete ideas for students.
- The final project demonstration should be organized like an "event" where all students can demonstrate their work and students can discuss experiences in their projects.
- List of the student projects