FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference
Speeds up long-context prefill by adapting sparse attention patterns and compute budgets to each input and attention head.
Selected work
All publicationsFormalAlign: Automated Alignment Evaluation for Autoformalization
Checks whether formalized mathematical statements preserve their natural-language meaning, reducing the need for manual alignment checks.
AutoPSV: Automated Process-Supervised Verifier
Turns changes in a verifier’s confidence into automatic step-level supervision for detecting reasoning errors and selecting better answers.
* Equal contribution.
Current research
Architecture and scaling
I study which architectures and training objectives improve learning at matched compute, and whether those gains persist as models scale.
Long-context efficiency
I explore how sparse attention, memory compression, and reuse across layers can lower long-context costs while preserving the information models need.
Models and systems
I study when computational savings translate into higher throughput and lower latency, while preserving consistent model behavior across training and inference.
Publications
Google Scholar2025
- FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence InferenceICLR 2025 · Oral · Equal contribution Code
- FormalAlign: Automated Alignment Evaluation for AutoformalizationICLR 2025 · Equal contribution Code & data
- FaStFact: Faster, Stronger Long-Form Factuality Evaluations in LLMsFindings of EMNLP 2025 Code
2024
- AutoPSV: Automated Process-Supervised VerifierNeurIPS 2024 · First author Repository
- Proving Theorems RecursivelyNeurIPS 2024
- MR-Ben: A Meta-Reasoning Benchmark for Evaluating System-2 Thinking in LLMsNeurIPS 2024
- FVEL: Interactive Formal Verification Environment with Large Language Models via Theorem ProvingNeurIPS 2024 · Datasets and Benchmarks Track Project
- Online Matching Meets Sampling Without ReplacementWINE 2024 · Equal contribution
- Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex ScenariosACL 2024 Code
2023
- Improving End-to-End Speech Processing by Efficient Text Data Utilization with Latent SynthesisEMNLP 2023 · First author
Preprints (4)
- Process-Driven Autoformalization in Lean 4arXiv · 2024 · First author Code & data
- MHPP: Exploring the Capabilities and Limitations of Language Models Beyond Basic Code GenerationarXiv · 2024 · Equal contribution
- YODA: Teacher-Student Progressive Learning for Language ModelsarXiv · 2024 · First author
- SELF: Self-Evolution with Language FeedbackarXiv · 2023 · First author
Background
Research experience
- ByteDance Seed Foundation ModelResearch Engineer · 2025–presentPreviously Research Intern · 2024–2025
- Huawei Noah’s Ark LabResearch Intern · 2022–2024
Education and collaboration
PhD in Computer Science, The University of Hong Kong (2020–2025), supervised by Prof. Zhiyi Huang. Bachelor’s in Electronics, Tongji University (2015–2020).
During my PhD, I worked with Zhijiang Guo, Zhengying Liu, Yinya Huang, and Yingjia Wan on mathematical reasoning, autoformalization, and process supervision.
Away from research, I enjoy running, hiking, games, and frisbee. A little more about me