Beidi Chen

Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, Zhangyang Wang, Beidi Chen. " H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models". NeurIPS 2023.

Yuandong Tian, Yiping Wang, Beidi Chen, Simon Shaolei Du. " Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer". NeurIPS 2023.

Stefano Massaroli, Michael Poli, Daniel Y Fu, Hermann Kumbong, David W. Romero, Rom Nishijima Parnichkun, Aman Timalsina, Quinn McIntyre, Beidi Chen, Atri Rudra, Ce Zhang, Christopher Ré, Stefano Ermon, Yoshua Bengio. " Laughing Hyena Distillery: Extracting Compact Recurrences From Convolutions". NeurIPS 2023.

Moses Charikar, Beidi Chen, Christopher Ré, Erik Waingarten. " Fast Algorithms for a New Relaxation of Optimal Transport ". Conference on Learning Theory (COLT) 2023.

Zichang Liu, Jue Wang, Tri Dao, Tianyi Zhou, Binhang Yuan, Zhao Song, Anshumali Shrivastava, Ce Zhang, Yuandong Tian, Christopher Ré, Beidi Chen. " Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time". ICML 2023. (Oral).

Ying Sheng, Lianmin Zheng, Binhang Yuan, Zhuohan Li, Max Ryabinin, Daniel Y. Fu, Zhiqiang Xie, Beidi Chen, Clark Barrett, Joseph E. Gonzalez, Percy Liang, Christopher Ré, Ion Stoica, Ce Zhang. " High-throughput Generative Inference of Large Language Models with a Single GPU". ICML 2023. (Oral). [ code ]

Jue Wang, Yucheng Lu, Binhang Yuan, Beidi Chen, Percy Liang, Christopher De Sa, Christopher Ré, Ce Zhang. " CocktailSGD: Fine-tuning Foundation Models over 500Mbps Networks ". ICML 2023.

Binhang Yuan, Yongjun He, Jared Quincy Davis, Tianyi Zhang, Tri Dao, Beidi Chen, Percy Liang, Christopher Ré, Ce Zhang. " Decentralized Training of Foundation Models in Heterogeneous Environments". NeurIPS 2022. (Oral).

Jue Wang, Binhang Yuan, Luka Rimanic, Yongjun He, Tri Dao, Beidi Chen, Christopher Ré, Ce Zhang. " Fine-tuning Language Models over Slow Networks using Activation Compression with Guarantees". NeurIPS 2022.

Beidi Chen*, Tri Dao*, Kaizhao Liang, Jiaming Yang, Zhao Song, Atri Rudra, Christopher Ré. " Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models". In Proceedings of International Conference on Learning Representations, ICLR 2022 (Spotlight). [video][ code ]

Beidi Chen*, Tri Dao*, Eric Winsor, Zhao Song, Atri Rudra, Christopher Ré. " Scatterbrain: Unifying Sparse and Low-rank Attention Approximation ". In Neural Information Processing Systems, NeurIPS 2021. [ code ]

Beidi Chen, Zichang Liu, Binghui Peng, Zhaozhuo Xu, Jonathan Lingjie Li, Tri Dao, Zhao Song, Anshumali Shrivastava, Christopher Re. "MONGOOSE: A Learnable LSH Framework for Efficient Neural Network Training". In Proceedings of International Conference on Learning Representations, ICLR 2021. (Oral) [video][ code]

Zhaozhuo Xu, Beidi Chen, Chaojian Li, Weiyang Liu, Le Song, Yingyan Lin, and Anshumali Shrivastava. " Locality Sensitive Teaching". In Neural Information Processing Systems, NeurIPS 2021.

Shabnam Daghaghi, Nicholas Meisburger, Mengnan Zhao, Beidi Chen, Tharun Medini, and Anshumali Shrivastava. "A Tale of Two Efficient and Informative Negative Sampling Distributions". In Proceedings of International Conference on Machine Learning, ICML 2021. (Long Talk)

Tharun Medini, Beidi Chen, Anshumali Shrivastava. "SOLAR: Sparse Orthogonal Learned and Random Embeddings". In Proceedings of International Conference on Learning Representations, ICLR 2021.

Beidi Chen, Weiyang Liu, Animesh Garg, Zhiding Yu, Anshumali Shrivastava, Jan Kautz, Anima Anandkumar. "Angular Visual Hardness". In Proceedings of the 37th International Conference on Machine Learning, ICML 2020. [Video]

Beidi Chen, Tharun Medini, James Farwell, Sameh Gobriel, Charlie Tai, Anshumali Shrivastava. "SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems". In Proceedings of the 3rd Conference on Machine Learning and Systems, MLSys 2020.

Beidi Chen, Yingchen Xu, and Anshumali Shrivastava. "LGD: Fast and Accurate Stochastic Gradient Estimation". In Neural Information Processing Systems, NeurIPS 2019. (LSH-Sampling Breaks the Computational Chicken-and-Egg Loop in Adaptive Stochastic Gradient Estimation in ICLR 2018 Workshop)

Beidi Chen, M. Sadegh Riazi, Anshumali Shrivastava, DanWallach, Farinaz Koushanfar. "Sub-linear Privacy-preserving Search with Untrusted Server and Semi-honest Parties". Manuscript.

Beidi Chen, Anshumali Shrivastava. " Revisiting Winner Take All (WTA) Hashing for Sparse Datasets". In Proceedings of the 34th Conference in Uncertainty in Artificial Intelligence, UAI 2018.

Beidi Chen, Anshumali Shrivastava, Rebecca C. Steorts. "Unique Entity Estimation with Application to the Syrian Conflict". The Annals of Applied Statistics 12.2 (2018). (Also Won IISA 2018 Best Student Paper in Applied Statistics with this paper.)

Kaifei Chen, Siyuan He, Beidi Chen, John Kolb, Randy H. Katz, David E. Culler. "BearLoc: A Composable Distributed Framework for Indoor Localization Systems". In Proceedings of the 2015 Workshop on IoT challenges in Mobile and Industrial Systems, IoT-Sys@MobiSys 2015, pages 7-12, May. 2015. Florence, Italy.

S. Alspaugh, Beidi Chen, Jessica Lin, Archana Ganapathi, Marti Hearst, and Randy Katz. "Analyzing Log Analysis: An Empirical Study of User Log Mining". In Proceedings of the 28th Large Installation System Administration Conference, LISA 2014. (Best Student Paper)