I am a Ph.D. candidate in the Department of Electronic Engineering at Tsinghua University. I received my B.E. degree from Tsinghua University in 2021.

My research focuses on LLM pre-training, specifically scaling law and data strategy. I am particularly interested in building scaling ladders, understanding the effects of multi-epoch data reuse, and optimizing data mixture ratios. My work has been published at conferences including ICML, NeurIPS, ACL, AAAI, and EMNLP.

I am currently a research intern on the Scaling Team at Moonshot AI. Previously, I worked at Tencent Hunyuan and Xiaohongshu Dots.

我目前在清华大学电子工程系攻读博士学位，于 2021 年获得清华大学工学学士学位。

我的主要研究方向是 LLM pre-training，具体包括 scaling law 和 data strategy。我尤其关注 scaling ladder 的搭建、预训练阶段 multi-epoch 的影响，以及预训练数据配比与合版。我的研究成果已发表于 ICML、NeurIPS、ACL、AAAI、EMNLP 等会议。

目前，我在 Moonshot AI 的 Scaling Team 实习；此前曾在腾讯混元和小红书 Dots实习。

💻 Internships实习经历

  • Moonshot AIMoonshot AI · Scaling Team Sept. 2026 – Present2026 年 9 月至今2026.09–Now至今
  • Tencent腾讯 · Hunyuan混元 July 2026 – Sept. 20262026 年 7 月–9 月2026.07–09
  • Xiaohongshu小红书 · Dots Mar. 2026 – June 20262026 年 3 月–6 月2026.03–06
  • JD.com, Inc. · JD Health京东集团 · 京东健康 Nov. 2023 – Aug. 20242023 年 11 月–2024 年 8 月2023.11–2024.08
  • Microsoft Research AsiaMSRA微软亚洲研究院微软 MSRA · Social Computing GroupSocial Computing Dec. 2020 – June 20212020 年 12 月–2021 年 6 月2020.12–2021.06

📝 Selected Publications代表论文

For a full list, please see my完整论文列表请见 Google Scholar.
* Equal contribution.表示共同第一作者。

NeurIPS 2026 Spotlight Learning-time estimation, data reweighting, and continued pre-training workflow

Smooth Scaling Laws Hide Stepwise Token Learning

Pingjie Wang*, Zechen Hu*, Peiru Yang*, Fu Guo, Debing Zhang

NeurIPS 2026 Spotlight

[Paper论文] [Code代码]

Smooth scaling laws emerge from localized token-level learning events; the learning-time spectrum guides more efficient training.从 token 级别的阶跃式学习解释平滑的 scaling law，并利用 learning-time spectrum 指导更高效的训练。

ICML 2026 GeoEX framework for whole-corpus reconstruction of heterogeneous RAG knowledge bases

Towards Whole-corpus Reconstruction of Heterogeneous RAG Knowledge Bases

Peiru Yang, Yi Luo, Zhenfeng Gao, Tong Ju, Haoran Zheng, Linjie Zhu, Hongke Fu, Qing Li, Shangguang Wang, Tao Qi

ICML 2026

[Paper论文] [Code代码]

A black-box framework for reconstructing heterogeneous RAG knowledge bases through exploration of the retriever embedding space.通过探索检索器的 embedding space，在黑盒条件下重建异构 RAG 知识库。

ACL 2026 M3Att framework with distribution-guided retrieval hijacking and misinformation poisoning

Knowledge Poisoning Attacks on Medical Multi-Modal Retrieval-Augmented Generation

Peiru Yang*, Haoran Zheng*, Tong Ju*, Shiting Wang, Wanchun Ni, Jiajun Liu, Shangguang Wang, Yongfeng Huang, Tao Qi

ACL 2026

[Paper论文] [Code代码]

A coordinated retrieval and generation poisoning framework for medical multimodal RAG systems.面向医疗多模态 RAG 系统的检索与生成协同知识投毒框架。

AAAI 2026 Oral OncoCoT pipeline for temporal-causal clinical reasoning data construction

OncoCoT: A Temporal-causal Chain-of-Thought Dataset for Oncologic Decision-Making

Peiru Yang, Yudong Li, Shiting Wang, Xinyi Liu, Haotian Gan, Xintian Li, Qingyu Gao, Yongfeng Huang

AAAI 2026 Oral

[Paper论文]

A temporal-causal chain-of-thought dataset for reasoning over longitudinal oncology cases.基于真实肿瘤病例的时序因果关系，构建面向临床决策的 chain-of-thought 数据集。

AAAI 2026 Oral MrM object-aware perturbation, counterfact-informed mask selection, and statistical membership inference

MrM: Black-Box Membership Inference Attacks against Multimodal RAG Systems

Peiru Yang, Jinhua Yin, Haoran Zheng, Xueying Bai, Huili Wang, Yufei Sun, Xintian Li, Songwei Pei, Yongfeng Huang, Tao Qi

AAAI 2026 Oral

[Paper论文] [Code代码]

Black-box membership inference through object-aware perturbation and counterfact-informed mask selection.结合 object-aware 数据扰动与反事实引导的掩码选择，进行黑盒多模态 RAG 成员推断。

AAAI 2026 ShieldRAG framework for safe retrieval from untrusted knowledge bases

ShieldRAG: Safeguarding Retrieval-Augmented Generation from Untrusted Knowledge Bases

Peiru Yang*, Haoran Zheng*, Yi Luo, Xinyi Liu, Jinrui Wang, Huili Wang, Xintian Li, Yongfeng Huang, Tao Qi

AAAI 2026

[Paper论文] [Code代码]

Safeguarding RAG systems against harmful content in untrusted knowledge bases.提升 RAG 系统在不可信知识库下的安全性，抵御有害知识带来的风险。

NeurIPS 2025 Knowledge-Calibrated Memory Probing with semantic masks, prior calibration, and model confidence evaluation

Black-Box Membership Inference Attack for LVLMs via Prior Knowledge-Calibrated Memory Probing

Jinhua Yin*, Peiru Yang*, Chen Yang, Huili Wang, Zhiyang Hu, Shangguang Wang, Yongfeng Huang, Tao Qi

NeurIPS 2025

[Paper论文]

Prior knowledge calibration helps distinguish memorization from general knowledge in black-box LVLMs.利用先验知识校准的 memory probing，区分黑盒 LVLM 的训练记忆与通用知识。

Knowledge-Based Systems 2024 LMKG construction framework integrating multiple medical knowledge sources

LMKG: A Large-Scale and Multi-Source Medical Knowledge Graph for Intelligent Medicine Applications

Peiru Yang, Hongjun Wang, Yingzhuo Huang, Shuai Yang, Ya Zhang, Liang Huang, Yuesong Zhang, Guoxin Wang, Shizhong Yang, Liang He, Yongfeng Huang

Knowledge-Based Systems 2024

[Paper论文] [Code代码]

A large-scale, multi-source medical knowledge graph for intelligent medicine applications.融合多来源医学知识，构建面向智能医疗应用的大规模 medical knowledge graph。

🎓 Education教育经历

  • Tsinghua University清华大学 Sept. 2021 – Present2021 年 9 月至今

    Ph.D. in Electronic Engineering电子工程，博士在读

    Award: Future Scholar Scholarship荣誉：未来学者奖学金

  • Tsinghua University清华大学 Sept. 2017 – June 20212017 年 9 月–2021 年 6 月

    B.E. in Electronic Engineering电子工程，工学学士

    GPA: 3.89/4.0 · Rank: 8/261 · Excellent Undergraduate ThesisGPA：3.89/4.0 · 排名：8/261 · 优秀本科毕业论文

  • Rice University莱斯大学 Aug. 2019 – Jan. 20202019 年 8 月–2020 年 1 月

    Exchange Student in Computer Science计算机科学，交换生

    GPA: 4.0/4.0GPA：4.0/4.0