Publications

Tianyu Gao, Alexander Wettig, Luxi He, Yihe Dong, Sadhika Malladi, Danqi Chen
Metadata Conditioning Accelerates Language Model Pre-training (MeCo)
ICML, 2025 [pdf] [code]

Xi Ye, Fangcong Yin, Yinghui He, Joie Zhang, Howard Yen, Tianyu Gao, Greg Durrett, Danqi Chen
LongProc: Benchmarking Long-Context Language Models on Long Procedural Generation
Preprint, 2025 [pdf] [code]

Tianyu Gao*, Alexander Wettig*, Howard Yen, Danqi Chen
How to Train Long-Context Language Models (Effectively)
ACL, 2025 [pdf] [code] [huggingface]

Howard Yen, Tianyu Gao, Minmin Hou, Ke Ding, Daniel Fleischer, Peter Izsak, Moshe Wasserblat, Danqi Chen
HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly
Proceedings of ICLR, 2025 [pdf] [code]

Anirudh Ajith, Mengzhou Xia, Alexis Chevalier, Tanya Goyal, Danqi Chen, Tianyu Gao
LitSearch: A Retrieval Benchmark for Scientific Literature Search
Proceedings of EMNLP, 2024 [pdf] [code]

Tianyu Gao, Zirui Wang, Adithya Bhaskar, Danqi Chen
Improving Language Understanding from Screenshots (PTP)
Preprint, 2024 [pdf] [code]

Howard Yen, Tianyu Gao, Danqi Chen
Long-Context Language Modeling with Parallel Context Encoding (CEPE)
Proceedings of ACL, 2024 [pdf] [code]

Mengzhou Xia, Tianyu Gao, Zhiyuan Zeng, Danqi Chen
Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
Proceedings of ICLR, 2024 [pdf] [code] [blog]

Zhiyuan Zeng, Jiatong Yu, Tianyu Gao, Yu Meng, Tanya Goyal, Danqi Chen
Evaluating Large Language Models at Evaluating Instruction Following (LLMBar)
Proceedings of ICLR, 2024 [pdf] [code]

Tianyu Gao, Howard Yen, Jiatong Yu, Danqi Chen
Enabling Large Language Models to Generate Text with Citations (ALCE)
Proceedings of EMNLP, 2023 [pdf] [code]

Sadhika Malladi*, Tianyu Gao*, Eshaan Nichani, Alex Damian, Jason D. Lee, Danqi Chen, Sanjeev Arora
Fine-Tuning Language Models with Just Forward Passes (MeZO)
Proceedings of Neurips, 2023 (oral) [pdf] [code]

Jane Pan, Tianyu Gao, Howard Chen, Danqi Chen
What In-Context Learning "Learns" In-Context: Disentangling Task Recognition and Task Learning
Proceedings of Findings of ACL, 2023 [pdf] [code]

Leonard Adolphs, Tianyu Gao, Jing Xu, Kurt Shuster, Sainbayar Sukhbaatar, Jason Weston
The CRINGE Loss: Learning what language not to model
Proceedings of ACL, 2023 [pdf]

Alexander Wettig*, Tianyu Gao*, Zexuan Zhong, Danqi Chen
Should You Mask 15% in Masked Language Modeling?
Proceedings of EACL, 2023 [pdf] [code]

Samyak Gupta, Yangsibo Huang, Zexuan Zhong, Tianyu Gao, Kai Li, Danqi Chen
Recovering Private Text in Federated Learning of Language Models
Proceedings of NeurIPS, 2022 [pdf]

Zichun Yu, Tianyu Gao, Zhengyan Zhang, Yankai Lin, Zhiyuan Liu, Maosong Sun, Jie Zhou
Automatic Label Sequence Generation for Prompting Sequence-to-sequence Models
Proceedings of COLING, 2022 [pdf] [code]

Huihan Li*, Tianyu Gao*, Manan Goenka, Danqi Chen
Ditch the Gold Standard: Re-evaluating Conversational Question Answering
Proceedings of ACL, 2022 (Outstanding Paper Award) [pdf] [code]

Tianyu Gao*, Xingcheng Yao*, Danqi Chen
SimCSE: Simple Contrastive Learning of Sentence Embeddings
Proceedings of EMNLP, 2021 [pdf] [code]

Tianyu Gao*, Adam Fisch*, Danqi Chen
Making Pre-trained Language Models Better Few-shot Learners (LM-BFF)
Proceedings of ACL, 2021 [pdf] [code]

Tianyu Gao, Xu Han, Keyue Qiu, Yuzhuo Bai, Zhiyu Xie, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou
Manual Evaluation Matters: Reviewing Test Protocols of Distantly Supervised Relation Extraction
Proceedings of ACL Findings, 2021 [pdf] [code]

Hao Peng*, Tianyu Gao*, Xu Han, Yankai Lin, Peng Li, Zhiyuan Liu, Maosong Sun, Jie Zhou
Learning from Context or Names? An Empirical Study on Neural Relation Extraction
Proceedings of EMNLP, 2020 [pdf] [code]

Xu Han*, Tianyu Gao*, Yankai Lin*, Hao Peng, Yaoliang Yang, Chaojun Xiao, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou
More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction
Proceedings of AACL, 2020 [pdf]

Xu Han*, Yi Dai*, Tianyu Gao, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou
Continual Relation Learning via Episodic Memory Activation and Reconsolidation
Proceedings of ACL, 2020 [pdf] [code]

Meng Qu, Tianyu Gao, Louis-Pascal Xhonneux, Jian Tang
Few-shot Relation Extraction via Bayesian Meta-learning on Task Graphs
Proceedings of ICML, 2020 [pdf]

Xiaozhi Wang, Tianyu Gao, Zhaocheng Zhu, Zhiyuan Liu, Juanzi Li, Jian Tang
KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation
Proceedings of TACL, 2020 [pdf]

Tianyu Gao, Xu Han, Ruobing Xie, Zhiyuan Liu, Fen Lin, Leyu Lin, Maosong Sun
Neural Snowball for Few-Shot Relation Learning
Proceedings of AAAI, 2020 [pdf] [code]

Tianyu Gao, Xu Han, Hao Zhu, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou
FewRel 2.0: Towards More Challenging Few-Shot Relation Classification
Proceedings of EMNLP (Short Paper), 2019 [pdf] [code]

Xu Han*, Tianyu Gao*, Yuan Yao, Deming Ye, Zhiyuan Liu, Maosong Sun
OpenNRE: An Open and Extensible Toolkit for Neural Relation Extraction
Proceedings of EMNLP (Demonstration Track), 2019 [pdf] [code]

Tianyu Gao*, Xu Han*, Zhiyuan Liu and Maosong Sun.
Hybrid Attention-Based Prototypical Networks for Noisy Few-Shot Relation Classification
Proceedings of AAAI, 2019 [pdf] [code]

* indicates equal contribution.