Data-centric Workflows with DAOS
Johann Lombardi, Intel
The Distributed Asynchronous Object Storage (DAOS) is an open-source scale-out object store designed from the ground up for massively distributed Non Volatile Memory (NVM). DAOS takes advantage of next-generation NVM technology, like Storage Class Memory (SCM) and NVM express (NVMe), and is extremely lightweight since it operates end-to-end in user space with full OS bypass. DAOS offers a shift away from an I/O model designed for block-based and high-latency storage to one that inherently supports fine-grained data access and unlocks the performance of the next-generation storage technologies. DAOS presents a key-value storage interface and features, such as transactional non-blocking I/O, advanced data protection with self-healing, end-to-end data integrity, fine-grained data control, and elastic storage.
This presentation will introduce the key concepts behind DAOS and the software ecosystem enabled around this technology. We will then share some performance numbers and explore how future HPC workflows could be accelerated via a tight integration with the storage system.
Johann Lombardi is a principal engineer in the Cloud & Enterprise Solution Group (CESG) at Intel. He started to work on Lustre in 2003 and led the sustaining team in charge of the Lustre filesystem worldwide support for more than 5 years. He then transitioned to research programs (Fast Forward, ESSIO, CORAL & Path Forward) to lead the development of a storage stack for Exascale HPC, Big Data and AI.