Don’t Drop Dropout: Optimizing Layer Sparsity for Efficient LLM Training and Inference

Mostafa Elhoushi, Alex Pretko, Nolan Dey, Bin Claire Zhang, Gavia Gray, Gurpreet Gosal, Abdulrahman Mahmoud, Shane Bergsma, Joel Hestness

Published in International Conference on Machine Learning, 2026

Citation: Mostafa Elhoushi, Alex Pretko, Nolan Dey, Bin Claire Zhang, Gavia Gray, Gurpreet Gosal, Abdulrahman Mahmoud, Shane Bergsma, Joel Hestness. 2026. "Don't Drop Dropout: Optimizing Layer Sparsity for Efficient LLM Training and Inference," International Conference on Machine Learning (ICML), Seoul, South Korea, 2026.