NIXL is an open source library for accelerating point-to-point data transfers in AI inference frameworks. NIXL provides a single, easy-to-use API that can be used to address a variety of data transfer challenges within these frameworks while maintaining maximum performance. This API supports multiple technologies such as RDMA, GPU-initiated networking, GPU-Direct storage, block and file storage, and advanced cloud storage options including S3 over RDMA and Azure Blob Storage. It is vendor-agnostic and can run across diverse environments. For example, it supports Amazon Web Services (AWS) with
Enhancing Distributed Inference Performance with the NVIDIA Inference Transfer Library | NVIDIA Technical Blog… This requires a library that can unify different communication and storage technologies, which ensures that frameworks can efficiently move data across various memory and storage hierarchies: GPU memory, CPU memory, and many tiers of local and distributed storage from NVMe to cloud object stores. …
… Library availability and path to production Currently, ovrtx , ovphysx , and ovstorage are in early access on GitHub and NGC. …
… GDS GDS offers a new storage technology that enables data to be read into the GPU bypassing the host CPU and memory. HDF5 supports GDS through the vfd-gds plugin, which uses the GDS cuFile library to read data directly into GPU memory, bypassing the host. …
… AI discovery and exposure management DOCA Argus also provides a foundational visibility layer for AI discovery and exposure management across the AI factory. …
… When storage gets faster, codec throughput matters Your storage speed depends on infrastructure: 2-10 GB/s for shared network filesystems Lustre, GPFS, NFS , or 15-50+ GB/s for GPUDirect Storage GDS with local NVMe. …
… Platform and Host Deployments DOCA on the BlueField Platform The NVIDIA BlueField platform, powered by the DOCA software framework, is an advanced computing platform for data center infrastructure, delivering accelerated software-defined networking, storage, security, and management services at mas… …
… In the C++ library MatX , for instance, the DSL is purely implemented through types and templating , which enables interesting compile-time optimizations for specific storage formats. …
… Einav has two decades of experience in the storage industry, spanning software development, application engineering, technical marketing, and product management View all posts by Einav Zilberstein View all posts by Einav Zilberstein About Oren Duer Oren Duer is a software architect specializing in … …
… Matrices Sparse matrix storage 2D has been studied extensively and many widely adopted sparse storage formats exist, each with a unique name. …
… 35x Improved Data Storage Efficiency Nemotron Embedding This test was conducted with the Nemotron embedding model llama-3.2-nv-embedqa-1b-v2 to show the impact on vector storage volume with long-context support, dynamic embeddings, and efficient storage for high-performance, scalable data processin… …