As filesystems become larger and larger, management of the data stored in these filesystems becomes more and more complex and time consuming. Querying filesystem metadata with standard tools can take hours or even days due to the sheer amount of metadata to be processed and how standard tools go about processing the metadata. Some filesystems provide custom tools that assist in querying their metadata, but there is no unified set of tools to do so. Additionally, such tools are usually reserved for administrators, not users. The Grand Unified File Index (GUFI) solves all these issues and more. GUFI provides a set of highly performant, parallel tools that allow for complex queries to be applied to the metadata of arbitrary filesystems. The GUFI toolset can be used by both administrators as well as users without violating permission semantics. Indices from disconnected filesystems can be combined, allowing for queries across multiple filesystems at once. This session will provide a high-level overview of GUFI as well as a tutorial on using GUFI.

ZFS is an open source volume manager and file system that provides many built-in data integrity and data transformations features. We will discuss two projects that LANL has been working on, with industry partners, to improve overall ZFS performance with NVMe devices. The first project adds support for O_DIRECT in ZFS, reducing the number of memory copies that occur in the ZFS code path which, in turn, reduces memory bandwidth pressure. The ZFS Interface for Accelerators (Z.I.A.) project offloads CPU- and memory-bandwidth-intensive operations to local and remote computational storage devices.

We seek to get a sense of what operations ZFS users would like to see offloaded to computational storage devices other than the ones that have already been offloaded.

High-performance computing applications, web-scale storage systems, and modern enterprises increasingly have the need for a data architecture that will unify at the edge, and in data centers, and clouds. These organizations with massive-scale data requirements need the performance of a parallel file system coupled with a standards-based solution that will be easy to deploy on machines with diverse security and build environments.

Join this BoF to discuss:

• How to unify data created in different clusters and locations into a single namespace, and place locally to applications and compute for processing/AI.
• Latest technologies available to deliver parallel file system performance from data sets stored in a hybrid cloud environment.
• The latest in standards-based technologies available for data orchestration and storage at mass scale.