Build a Validated Kubernetes Runtime for NVIDIA Dynamo

Teams building AI applications need a repeatable path from a model configuration to a working inference endpoint. NVIDIA Dynamo is a distributed inference framework that coordinates model serving across GPU workers. Deploying it on Kubernetes is only a first step. A successful model deployment requires more: GPU drivers, resource allocation, scheduling, networking, and monitoring must all work together. Resolving these dependencies takes time away from testing model behavior, serving performance, and application integration.

The NVIDIA DSX AI Factory Platform brings together technologies to help the ecosystem design, build, and operate more efficient AI factories, and Dynamo is one of them. To make management of those infrastructure dependencies easier, the platform contains NVIDIA AI Cluster Runtime (AICR). It packages NVIDIA Dynamo and its supporting infrastructure into validated recipes. Each recipe defines a configuration for a specific environment and workload, including pinned component versions, compatibility constraints, and validation checks. AICR gives platform teams a repeatable way to deploy and check the runtime, providing developers with a consistent foundation for serving models and testing applications.

This walkthrough covers the infrastructure that supports Dynamo on Kubernetes, compares two routing options, and shows how to deploy and validate the runtime with AICR.

Why the underlying runtime matters for Dynamo