Data Integration and Large-Scale Analysis WS2021/22
(VU, 706.520 Data Integration and Large-Scale Analysis)

DIA is a 5 ECTS bachelor and master course, applicable to the bachelor programs computer science or software engineering and management, as well as the master catalog 'Data Science'. This course covers major data integration architectures, key techniques for data integration and cleaning, as well as methods for large-scale, i.e., distributed, data storage and analysis.


Lectures

In detail, the course covers the following topics, which also reflects the course calendar. All slides will be made available prior to the individual lectures.

A: Data Integration and Preparation

  • 01 Introduction and Overview [Oct 08]
  • 02 Data Warehousing, ETL, and SQL/OLAP [Oct 15]
  • 03 Message-oriented Middleware, EAI, and Replication [Oct 22]
  • 04 Schema Matching and Mapping [Oct 29]
  • 05 Entity Linking and Deduplication [Nov 05]
  • 06 Data Cleaning and Data Fusion [Nov 12]
  • 07 Data Provenance and Blockchain [Nov 19]

B: Large-Scale Data Management and Analysis

  • 08 Cloud Computing Fundamentals [Nov 26]
  • 09 Cloud Resource Management and Scheduling [Dec 03]
  • 10 Distributed Data Storage [Dec 10]
  • 11 Distributed, Data-Parallel Computation [Jan 07]
  • 12 Distributed Stream Processing [Jan 14]
  • 13 Distributed Machine Learning Systems [Jan 21]


Exercises

The lectures are accompanied by mandatory programming projects (to the extend of 2 ECTS, i.e, roughly 50 working hours), preferrably in Apache SystemDS (an open source ML system for the end-to-end data science lifecycle) instead.

A list of project proposals (and some details on the alternative exercise) can be found at the end of Lecture 02.


Organization

  • Lecturer: Univ.-Prof. Dr.-Ing. Matthias Boehm, ISDS
  • Teaching Assistant: M.Sc. Shafaq Siddiqi, ISDS
  • Final written/oral exams: TBD
  • Grading: 30% project, 70% final exam