Architecture of ML Systems SS2019
(VU, 706.550 Architecture of Machine Learning Systems)

AMLS is a 5 ECTS master course, applicable in the catalog 'Knowledge Technologies'. This course covers the architecture and essential concepts of modern ML systems for supporting large-scale machine learning (ML). These architectures include systems for data-parallel execution (e.g., Spark, Mahout, SystemML), Parameter Servers (e.g., TensorFlow, MXNet, PyTorch), ML lifecycle systems, and the integration of ML into database systems. The covered topics focus on a microscopic view of internal compilation, execution, and data management techniques.


Lectures

In detail, the course covers the following topics, which also reflects the course calendar. All slides will be made available prior to the individual lectures.

A: Introduction

  • 01 Introduction and Overview [Mar 15, pdf, pptx]

B: Compilation and Optimization

  • 02 Languages, Architectures, and System Landscape [Mar 22, pdf, pptx]
  • 03 Size Inference, Rewrites, and Operator Selection [Mar 29, pdf, pptx]
  • 04 Operator Fusion and Runtime Adaptation [Apr 05, pdf, pptx]

C: Execution and Data Access

  • 05 Data- and Task-Parallel Execution [Apr 12, pdf, pptx]
  • 06 Parameter Servers [May 03, pdf, pptx]
  • 07 Hybrid Execution and HW Accelerators [May 10, pdf, pptx]
  • 08 Caching, Partitioning, Indexing and Compression [May 24]

D: ML Lifecycle Systems

  • 09 Data Acquisition, Cleaning, and Preparation [Jun 07]
  • 10 Model Selection and Management [Jun 14]
  • 11 Model Deployment and Serving [Jun 21]

E: Wrap-up

  • 12 Project Presentations, Conclusions, Q&A [Jun 28]


Project

The lectures are accompanied by a mandatory open source programming project for gaining practical experience (at the extend of 2 ECTS, i.e, roughly 50 working hours). We'll make project topic suggestions in the context of SystemDS (an open source ML system for the end-to-end data science lifecycle), but your own project proposals (potentially in other open source systems) are welcome as well.


Organization

  • Lecturer: Univ.-Prof. Dr.-Ing. Matthias Boehm, ISDS
  • Final oral exam: by appointment
  • Grading: 40% project, 60% final exam