Data Integration and Large-Scale Analysis WS2021/22
(VU, 706.520 Data Integration and Large-Scale Analysis)

DIA is a 5 ECTS bachelor and master course, applicable to the bachelor programs computer science or software engineering and management, as well as the master catalog 'Data Science'. This course covers major data integration architectures, key techniques for data integration and cleaning, as well as methods for large-scale, i.e., distributed, data storage and analysis.


In detail, the course covers the following topics, which also reflects the course calendar. All slides will be made available prior to the individual lectures.

The lectures will start October 08 (not October 01) in hybrid form (in-person in HS i5 with 3G and video recording, and webex for remote live questions).

A: Data Integration and Preparation

  • 01 Introduction and Overview [Oct 08, pdf, pptx]
  • 02 Data Warehousing, ETL, and SQL/OLAP [Oct 15, pdf, pptx]
  • 03 Message-oriented Middleware, EAI, and Replication [Oct 22, pdf, pptx]
  • 04 Schema Matching and Mapping [Oct 22, pdf, pptx]
  • 05 Entity Linking and Deduplication [Nov 05]
  • 06 Data Cleaning and Data Fusion [Nov 12]
  • 07 Data Provenance and Blockchain [Nov 19]

B: Large-Scale Data Management and Analysis

  • 08 Cloud Computing Fundamentals [Nov 26]
  • 09 Cloud Resource Management and Scheduling [Dec 03]
  • 10 Distributed Data Storage [Dec 10]
  • 11 Distributed, Data-Parallel Computation [Jan 07]
  • 12 Distributed Stream Processing [Jan 14]
  • 13 Distributed Machine Learning Systems [Jan 21]


The lectures are accompanied by mandatory programming projects (to the extend of 2 ECTS, i.e, roughly 50 working hours), preferrably in Apache SystemDS (an open source ML system for the end-to-end data science lifecycle) instead.

A list of project proposals (and the details on an alternative exercise) can be found at the beginning of Lecture 03.


  • Lecturer: Univ.-Prof. Dr.-Ing. Matthias Boehm, ISDS
  • Teaching Assistant: M.Sc. Shafaq Siddiqi, ISDS
  • Final written/oral exams: TBD
  • Grading: 30% project, 70% final exam