Highlights

In-memory Data Management

HyPer relies on in-memory data management without the ballast of traditional database systems caused by DBMS-controlled page structures and buffer management. SQL table definitions are transformed into simple vector-based virtual memory representations – which constitutes a column oriented physical storage scheme.

Data-centric Code Generation

Transactions and queries are specified in SQL or a PL/SQL-like scripting language and are efficiently compiled into efficient LLVM assembly code.

Multi-Version Concurrency Control

OLAP query processing is separated from mission-critical OLTP transaction processing using multi-version concurrency control (MVCC).

No compromises

HyPer's transaction processing is fully ACID-compliant. Queries are specified in SQL-92 plus some extensions from subsequent standards.

Team

Head: Prof. Alfons Kemper, Prof. Thomas Neumann

Senior Researcher: Prof. Dr. Viktor Leis

Ph.D. Students: Jan Böttcher, Moritz Kaufmann, Andreas Kipf, Timo Kersten, André Kohn, Harald Lang, Varun Pandey, Linnea Passing, Alexander van Renen, Wolf Rödiger, Maximilian E. Schüle, Manuel Then

Alumni: Dr. Martina Albutiu, Dr. Veneta Dobreva, Dr. Florian Funke, Dr. Jan Finis, Dr. Nina Hubig, Dr. Andrey Gubichev, Dr. Henrik Mühe, Dr. Tobias Mühlbauer, Dr. Michael Seibold

Publications

Foundational HyPer publications

List of all publications

Presentations/Mentions

Summary

The HyPer prototype demonstrates that it is indeed possible to build a main-memory database system that achieves world-record transaction processing throughput and best-of-breed OLAP query response times in one system in parallel on the same database state. The two workloads of online transaction processing (OLTP) and online analytical processing (OLAP) present different challenges for database architectures. Currently, users with high rates of mission-critical transactions have split their data into two separate systems, one database for OLTP and one so-called data warehouse for OLAP. While allowing for decent transaction rates, this separation has many disadvantages including data freshness issues due to the delay caused by only periodically initiating the Extract Transform Load-data staging and excessive resource consumption due to maintaining two separate information systems. We present an efficient hybrid system, called HyPer, that can handle both OLTP and OLAP simultaneously by using an efficient multi-version concurrency control scheme. HyPer is a main-memory database system that guarantees the full ACID properties for OLTP transactions. HyPer achieves both at the same time: unprecedentedly high transaction rates as high as 100,000 per second and very fast OLAP query response times on a single system executing both workloads in parallel. The performance analysis is based on a combined TPC-C and TPC-H benchmark. HyPer's OLTP throughput is better than VoltDB's published TPC-C performance and HyPer's OLAP query response times are superior to MonetDB's query response times. It should be emphasized that HyPer can match (or beat) these two best- of-breed transaction (VoltDB) and query (MonetDB) processing engines at the same time by performing both workloads in parallel on the same database state. HyPer's performance is due to the following design:

• HyPer relies on in-memory data management without the ballast of traditional database systems caused by DBMS-controlled page structures and buffer management. SQL table definitions are transformed into simple vector-based virtual memory representations – which constitutes a column oriented physical storage scheme.
• OLAP query processing is separated from mission-critical OLTP transaction processing using multi-version concurrency control.
• Transactions and queries are specified in SQL or a PL/SQL-like scripting language and are efficiently compiled into efficient LLVM assembly code.
• HyPer's transaction processing is fully ACID-compliant. Queries are specified in standard SQL-92 plus many extensions from following standard SQL standards.

Contact

Contact us (see Team for emails) if you are interested in a thesis, student job or even a Ph.D. position!