I’m a first year PhD student in Computer Science at University of Southern California (USC). Before that, I received master degree in Computer Science from University of California San Diego. I am broadly interested in natural language processing and theoretical computer science.

Research

Theoretical Computer Science (Author names in alphabetical order)

• Space-Efficient Interior Point Method, with applications to Linear Programming and Maximum Weight Bipartite Matching (arxiv link).
  Sixue Liu, Zhao Song, Hengjie Zhang, Lichen Zhang and Tianyi Zhou
  ICALP, 2023.
• Fast Heavy Inner Product Identification Between Weights and Inputs in Neural Network Training (arxiv link).
  Lianke Qin, Saayan Mitra, Zhao Song, Yuanyuan Yang and Tianyi Zhou
  IEEE Big Data, 2023.
• Algorithm and Hardness for Dynamic Attention Maintenance in Large Language Models (arxiv link).
  Jan van den Brand, Zhao Song and Tianyi Zhou
  NeurIPS, 2024.

Machine Learning

• Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time (arxiv link).
  Zichang Liu, Jue Wang, Tri Dao, Tianyi Zhou, Binhang Yuan, Zhao Song, Anshumali Shrivastava, Ce Zhang, Yuandong Tian, Christopher Ré, and Beidi Chen
  ICML 2023 Selected as Oral.

• H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models (arxiv link).
  Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, Zhangyang Wang and Beidi Chen
  NeurIPS 2023.