Po-An Tsai is a Principal Research Scientist at NVIDIA Research. His research spans computer architecture and systems, with a focus on the hardware–software co-design of tensor accelerators, sparse computation and model compression, memory hierarchies and data movement, and efficient large-scale LLM inference. His work has influenced the design of hardware and software systems at NVIDIA and has been recognized with IEEE Micro Top Picks selections, best-paper award/nominations at ISCA/HPCA, and a MICRO Distinguished Artifact Award. He received his S.M. and Ph.D. degrees in Electrical Engineering and Computer Science from the Massachusetts Institute of Technology.

As deep learning models grow, sparsity remains an important tool for improving computational efficiency. However, large models challenge the familiar research cycle of proposing sparsity patterns, pruning and fine-tuning models, and co-designing hardware. This cycle increasingly demands resources beyond individual researchers and small teams, and findings from smaller models do not always carry over. Advances in low-precision quantization also raise the bar for sparsity to deliver additional benefits. In this talk, I will discuss these challenges and highlight three opportunities through recent work with my collaborators. First, inherent activation sparsity in large models offers opportunities to skip computation and reduce data movement without fine-tuning. I will explain how the latest NVIDIA Rubin GPU Architecture exploits these opportunities. Second, sparsity can complement quantization by handling the sparse outliers that complicate low-precision representations. Third, I will share how mixture-of-experts architectures make coarse-grained sparsity central to large-model execution while introducing challenges in load balancing, expert placement, and communication, with our publicly released expert activation profiles providing a foundation for future work. Looking forward, I will argue that future architecture research for DL sparsity should work alongside low-precision formats such as NVFP4, target sparsity inherent in models and algorithms, and extend beyond arithmetic to communication, data movement, and storage. Understanding how sparsity interacts with other efficiency techniques will be essential to turning its potential into practical benefits in the large model era.

Maryam Mehri Dehnavi is an Associate Professor of Computer Science and Associate Chair of Research at the University of Toronto, where she holds a Canada Research Chair in Parallel and Distributed Computing, and is a Principal Research Scientist at NVIDIA. She leads the ParaMathics group, which develops scalable numerical methods, high-performance libraries, and domain-specific languages and compilers for sparse and irregular computations on parallel, high-performance, and cloud platforms. Before joining Toronto, she was an Assistant Professor at Rutgers University and a postdoctoral researcher at MIT. She received her PhD in Electrical and Computer Engineering from McGill University, and her honors include the Ontario Early Researcher Award. She served as General Chair of PPoPP 2023 and is an Associate Editor of the Journal of Parallel and Distributed Computing.

Sparsity offers a path to more efficient language models, but reducing the number of nonzero weights does not automatically reduce inference time. Realizing these gains requires designing pruning algorithms and compiler support together, so that patterns that preserve model quality also admit efficient GPU execution. This talk explores algorithm–compiler co-design as a way to connect these objectives. Using PATCH and STOICC as a concrete example, I will show how learning a mixture of dense and 2:4 sparse tiles creates a controllable accuracy–sparsity trade-off, while compiler representations and execution schedules translate that flexibility into hardware acceleration. The resulting system achieves up to 1.38× faster end-to-end generation and 41% lower memory use for LLaMA-2-7B. More broadly, I will discuss how designing model compression alongside the software that executes it can expand the range of sparsity patterns that are both useful to models and efficient in practice.