How to Read Data into GPU Memory in Python with NVIDIA GPUDirect Storage

GPU applications are processing ever-growing datasets with increasingly powerful accelerators. According to the 2025 Stanford AI Index, compute for notable AI models has been doubling approximately every five months, while the size of datasets used to train LLMs has been doubling approximately every eight months. As computation and data scale together, moving data efficiently from storage becomes a necessity for keeping accelerators supplied.

NVIDIA Magnum IO GPUDirect Storage (GDS) addresses this challenge by enabling a direct data path between supported storage and GPU memory. This avoids unnecessary staging through CPU memory and reduces overall CPU utilization in the system. GDS is supported by a growing ecosystem that includes many major storage vendors.

At the application layer, Python plays a central role in modern AI development. As of August 2025, nearly half of all new AI projects on GitHub were built primarily in Python while the main application interface to GDS is the cuFile C API.

NVIDIA KvikIO, a Python and C++ library for high-performance file I/O, bridges this gap. It provides high-level, idiomatic Python and C++ interfaces, along with additional controls over how I/O requests are performed.