Architecture of ML Systems SS2021
(VU, 706.550 Architecture of Machine Learning Systems)

AMLS is a 5 ECTS master course, applicable to the master catalogs 'Data Science', 'Machine Learning', 'Software Technology', and 'Interactive and Visual Information Systems'. This course covers the architecture and essential concepts of modern ML systems for both local and large-scale machine learning (ML). These architectures include systems for data-parallel execution (e.g., Spark, Mahout, SystemML), Parameter Servers (e.g., TensorFlow, MXNet, PyTorch), ML lifecycle systems, and the integration of ML into database systems. The covered topics focus primarily on a microscopic view of internal compilation, execution, and data management techniques, but also include a macroscopic view of entire ML pipelines.


Lectures

In detail, the course covers the following topics, which also reflects the course calendar. All slides will be made available prior to the individual lectures.

A: Overview and ML System Internals

  • 01 Introduction and Overview [Mar 05, pdf, pptx]
  • 02 Languages, Architectures, and System Landscape [Mar 12, pdf, pptx]
  • 03 Size Inference, Rewrites, and Operator Selection [Mar 19, pdf, pptx]
  • 04 Operator Fusion and Runtime Adaptation [Mar 26, pdf, pptx]
  • 05 Data- and Task-Parallel Execution [Apr 16, pdf, pptx]
  • 06 Parameter Servers [Apr 23, pdf, pptx]
  • 07 Hybrid Execution and HW Accelerators [Apr 30, pdf, pptx]
  • 08 Caching, Partitioning, Indexing and Compression [May 07, pdf, pptx]

B: ML Lifecycle Systems

  • 09 Data Acquisition, Cleaning, and Preparation [May 21, pdf, pptx]
  • 10 Model Selection and Management [May 28, pdf, pptx]
  • 11 Model Debugging, Fairness, and Explainability [Jun 11, pdf, pptx]
  • 12 Model Serving Systems and Techniques [Jun 18, pdf, pptx]


Project / Exercises

The lectures are accompanied by mandatory programming projects (to the extend of 2 ECTS, i.e, roughly 50 working hours), preferrably in Apache SystemDS (an open source ML system for the end-to-end data science lifecycle).
A list of project proposals and details on an alternative exercise was made available during the second lecture:


Organization

  • Lecturer: Univ.-Prof. Dr.-Ing. Matthias Boehm, ISDS
  • Teaching Assistant: M.Sc. Sebastian Baunsgaard, ISDS
  • Final oral exam: July 7, 8, 9, 12, 13 (doodle sent out via email to all registered students)
  • Grading: 40% project, 60% final exam