Introducing ModuleLens: Zooming into PyTorch Modules for Human and Agentic Engineering

Lida Li, Sr. Staff Software Engineer, Product ML Infra

Introduction

Modern recommendation systems, including those used for ads ranking and content discovery, have grown substantially in scale and complexity. This growth stems from both increasingly large model architectures. The resulting memory footprint often exceeds the capacity of a single GPU, making it necessary to shard production models across multiple GPUs using distributed training techniques such as Distributed Model Parallelism (DMP).

This evolution brings challenges to the everyday engineer work on testing and debugging the model code. The bottleneck includes: