Publications

* indicates equal contribution.

Publications

• Reverse Transition Kernel: A Flexible Framework to Accelerate Diffusion Inference
  Xunpeng Huang, Difan Zou, Hanze Dong, Yi Zhang, Yi-An Ma, Tong Zhang
  ICML Workshop on Structured Probabilistic Inference & Generative Modeling, 2024 (Oral Presentation & Best Paper Award)
  Conference on Advances in Neural Information Processing Systems (NeurIPS), 2024 (Spotlight)
  [Arxiv]

• Slight Corruption in Pre-training Data Makes Better Diffusion Models
  Hao Chen, Yujin Han, Diganta Misra, Xiang Li, Kai Hu, Difan Zou, Masashi Sugiyama, Jindong Wang, Bhiksha Raj
  Conference on Advances in Neural Information Processing Systems (NeurIPS), 2024 (Spotlight)
  [Arxiv]

• How Transformers Utilize Multi-Head Attention in In-Context Learning? A Case Study on Sparse Linear Regression
  Xingwu Chen*, Lei Zhao*, Difan Zou
  Conference on Advances in Neural Information Processing Systems (NeurIPS), 2024
  [Arxiv]

• The Implicit Bias of Adam on Separable Data
  Chenyang Zhang, Difan Zou, Yuan Cao
  Conference on Advances in Neural Information Processing Systems (NeurIPS), 2024
  [Arxiv]

• A Comprehensive Investigation of Sparse Rate Reduction in Transformer-like Models
  Yunzhe Hu, Difan Zou, Dong Xu
  Conference on Advances in Neural Information Processing Systems (NeurIPS), 2024
  [Arxiv]

• Per-Example Gradient Regularization Improves Learning Signals from Noisy Data
  Xuran Meng, Yuan Cao, Difan Zou
  Springer Machine Learning Journal, 2024 (accepted)
  [Arxiv]

• Faster Sampling without Isoperimetry via Diffusion-based Monte Carlo
  Xunpeng Huang, Difan Zou, Hanze Dong, Yian Ma, and Tong Zhang
  Annual Conference on Learning Theory (COLT), 2024
  [Paper][Arxiv]

• What Can Transformer Learn with Varying Depth? Case Studies on Sequence Learning Tasks
  Xingwu Chen, Difan Zou
  ICLR Workshop on Bridging the Gap Between Practice and Theory in Deep Learning (BPGT), 2024 (Oral Presentation)
  International Conference on Machine Learning (ICML), 2024
  [Paper][Arxiv]

• Benign Overfitting in Two-Layer ReLU Convolutional Neural Networks for XOR Data
  Xuran Meng, Difan Zou, Yuan Cao
  International Conference on Machine Learning (ICML), 2024
  [Paper][Arxiv]

• Improving Group Robustness on Spurious Correlation Requires Preciser Group Inference
  Yujin Han, Difan Zou
  International Conference on Machine Learning (ICML), 2024
  [Paper][Arxiv]

• Faster Sampling via Stochastic Gradient Proximal Sampler
  Xunpeng Huang, Difan Zou, Yian Ma, Hanze Dong, Tong Zhang
  International Conference on Machine Learning (ICML), 2024
  [Paper][Arxiv]

• On the Limitation and Experience Replay for GNNs in Continual Learning
  Junwei Su, Difan Zou, Chuan Wu
  Conference on Lifelong Learning Agents (CoLLAs), 2024

• Benign Oscillation of Stochastic Gradient Descent with Large Learning Rates
  Miao Lu, Beining Wu, Xiaodong Yang, Difan Zou
  NeurIPS Workshop on Mathematics of Modern Machine Learning (M3L), 2023
  International Conference on Learning Representations (ICLR), 2024
  [Paper][Arxiv]

• How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?
  Jingfeng Wu, Difan Zou, Zixiang Chen, Vladimir Braverman, Quanquan Gu, Peter L. Bartlett
  International Conference on Learning Representations (ICLR) (Spotlight), 2024
  [Paper][Arxiv]

• PRES: Toward Scalable Memory-Based Dynamic Graph Neural Networks
  Junwei Su, Difan Zou, Chuan Wu
  International Conference on Learning Representations (ICLR), 2024
  [Paper][Arxiv][Code]

• The Implicit Bias of Batch Normalization in Linear Models and Two-layer Linear Convolutional Neural Networks
  Yuan Cao, Difan Zou, Yuanzhi Li, Quanquan Gu
  Annual Conference on Learning Theory (COLT), 2023
  [Arxiv]

• The Benefits of Mixup for Feature Learning
  Difan Zou, Yuan Cao, Yuanzhi Li, Quanquan Gu
  International Conference on Machine Learning (ICML), 2023
  [Paper][Arxiv]

• Learning High-Dimensional Single-Neuron ReLU Networks with Finite Samples
  Jingfeng Wu*, Difan Zou*, Zixiang Chen*, Vladimir Braverman, Quanquan Gu, and Sham M. Kakade
  International Conference on Machine Learning (ICML), 2023
  [Paper][Arxiv]

• Towards Robust Graph Incremental Learning on Evolving Graphs
  Junwei Su, Difan Zou, Zijun Zhang, Chuan Wu
  International Conference on Machine Learning (ICML), 2023
  [Arxiv][Paper][Code]

• Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularization
  Difan Zou, Yuan Cao, Yuanzhi Li, and Quanquan Gu
  International Conference on Learning Representations (ICLR), 2023
  [Paper][ArXiv]

• Risk Bounds of Multi-Pass SGD for Least Squares in the Interpolation Regime
  Difan Zou*, Jingfeng Wu*, Vladimir Braverman, Quanquan Gu, and Sham M. Kakade
  Conference on Advances in Neural Information Processing Systems (NeurIPS), 2022
  [Paper][ArXiv]

• The Power and Limitation of Pretraining-Finetuning for Linear Regression under Covariate Shift
  Jingfeng Wu*, Difan Zou*, Vladimir Braverman, Quanquan Gu, and Sham M. Kakade
  Conference on Advances in Neural Information Processing Systems (NeurIPS), 2022
  [Paper][ArXiv]

• Last Iterate Risk Bounds of SGD with Decaying Stepsize for Overparameterized Linear Regression
  Jingfeng Wu*, Difan Zou*, Vladimir Braverman, Quanquan Gu, Sham M. Kakade
  International Conference on Machine Learning (ICML), 2022 (Long Presentation)
  [Paper] [ArXiv]

• Self-training Converts Weak Learners to Strong Learners in Mixture Models
  Spencer Frei*, Difan Zou*, Zixiang Chen*, Quanquan Gu
  International Conference on Artificial Intelligence and Statistics (AISTATS), 2022
  [Paper] [ArXiv]

• The Benefit of Implicit Regularization from SGD in Least Square Problems
  Difan Zou*, Jingfeng Wu*, Vladimir Braverman, Quanquan Gu, Dean P. Foster, Sham M. Kakade
  Conference on Advances in Neural Information Processing Systems (NeurIPS), 2021
  [Paper] [ArXiv]

• Benign Overfitting of Constant-Stepsize SGD for Linear Regression
  Difan Zou*, Jingfeng Wu*, Vladimir Braverman, Quanquan Gu, Sham M. Kakade
  Annual Conference on Learning Theory (COLT), 2021
  [Paper] [ArXiv]

• Faster Convergence of Stochastic Gradient Langevin Dynamics for Non-Log-Concave Sampling
  Difan Zou, Pan Xu, Quanquan Gu
  International Conference on Uncertainty in Artificial Intelligence (UAI), 2021
  [Paper] [ArXiv]

• On the Convergence of Hamiltonian Monte Carlo with Stochastic Gradients
  Difan Zou, Quanquan Gu
  International Conference on Machine Learning (ICML), 2021
  [Paper]

• Provable Robustness of Adversarial Training for Learning Halfspaces with Noise
  Difan Zou*, Spencer Frei*, Quanquan Gu
  International Conference on Machine Learning (ICML), 2021
  [Paper] [ArXiv]

• How Much Over-parameterization Is Sufficient to Learn Deep ReLU Networks?
  Zixiang Chen*, Yuan Cao*, Difan Zou* and Quanquan Gu
  International Conference on Learning Representations (ICLR), 2021
  [Paper] [ArXiv]

• Direction Matters: On the Implicit Regularization Effect of Stochastic Gradient Descent with Moderate Learning Rate
  Jingfeng Wu, Difan Zou, Vladimir Braverman and Quanquan Gu
  International Conference on Learning Representations (ICLR), 2021
  [Paper] [ArXiv]

• Laplacian Smoothing Stochastic Gradient Markov Chain Monte Carlo
  Bao Wang*, Difan Zou*, Quanquan Gu, Stanley Osher
  SIAM Journal on Scientific Computing (SISC), 2021
  [Paper] [ArXiv] [Code]

• On the Global Convergence of Training Deep Linear ResNets
  Difan Zou, Philip M. Long, Quanquan Gu
  International Conference on Learning Representations (ICLR), 2020
  [Paper]

• Improving Adversarial Robustness Requires Revisiting Misclassified Examples
  Yisen Wang*, Difan Zou*, Jinfeng Yi, James Bailey, Xingjun Ma and Quanquan Gu
  International Conference on Learning Representations (ICLR), 2020
  [Paper] [Code]

• Gradient Descent Optimizes Over-parameterized Deep ReLU Networks
  Difan Zou*, Yuan Cao*, Dongruo Zhou, Quanquan Gu
  Springer Machine Learning Journal, 2020
  [Paper] [ArXiv]

• An Improved Analysis of Training Over-parameterized Deep Neural Networks
  Difan Zou, Quanquan Gu
  Conference on Advances in Neural Information Processing Systems (NeurIPS), 2019
  [Paper] [ArXiv]

• Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networks
  Difan Zou*, Ziniu Hu*, Yewen Wang, Song Jiang, Yizhou Sun, Quanquan Gu
  Conference on Advances in Neural Information Processing Systems (NeurIPS), 2019
  [Paper] [ArXiv] [Code]

• Stochastic Gradient Hamiltonian Monte Carlo Methods with Recursive Variance Reduction
  Difan Zou, Pan Xu, Quanquan Gu
  Conference on Advances in Neural Information Processing Systems (NeurIPS), 2019
  [Paper]

• Sampling from Non-Log-Concave Distributions via Variance-Reduced Gradient Langevin Dynamics
  Difan Zou, Pan Xu, Quanquan Gu
  International Conference on Artificial Intelligence and Statistics (AISTATS), 2019
  [Paper]

• Global convergence of Langevin dynamics based algorithms for nonconvex optimization
  Pan Xu*, Jinghui Chen*, Difan Zou, Quanquan Gu
  Conference on Advances in Neural Information Processing Systems (NeurIPS), 2018, (Spotlight)
  [Paper] [ArXiv]

• Subsampled stochastic variance-reduced gradient Langevin dynamics
  Difan Zou*, Pan Xu*, Quanquan Gu
  International Conference on Uncertainty in Artificial Intelligence (UAI), 2018
  [Paper]

• Stochastic Variance-Reduced Hamilton Monte Carlo Methods
  Difan Zou*, Pan Xu*, Quanquan Gu
  International Conference on Machine Learning (ICML), 2018
  [Paper] [ArXiv]

Preprint

• Self-Control of LLM Behaviors by Compressing Suffix Gradient into Prefix Controller
  Min Cai, Yuchen Zhang, Shichang Zhang, Fan Yin, Difan Zou, Yisong Yue, Ziniu Hu
  [Arxiv][Website][Code]

• A Human-Like Reasoning Framework for Multi-Phases Planning Task with Large Language Models
  Chengxing Xie, Difan Zou
  ICML Workshop on LLMs and Cognition, 2024
  [Arxiv]

• On the Benefits of Over-parameterization for Out-of-Distribution Generalization
  Yifan Hao, Yong Lin, Difan Zou, and Tong Zhang
  [Arxiv]

• Improving Implicit Regularization of SGD with Preconditioning for Least Square Problems
  Junwei Su, Difan Zou, and Chuan Wu
  [Arxiv]

• An Improved Analysis of Langevin Algorithms with Prior Diffusion for Non-Log-Concave Sampling
  Xunpeng Huang, Hanze Dong, Difan Zou, and Tong Zhang
  [Arxiv]

• Less is More: On the Feature Redundancy of Pertrained Models When Transferring to Few-Shot Tasks
  Xu Luo, Difan Zou, Lianli Gao, Zenglin Xu, Jingkuan Song
  [Arxiv]

• Epidemic Model Guided Machine Learning for COVID-19 Forecasts in the United States
  Difan Zou, Lingxiao Wang, Pan Xu, Jinghui Chen, Weitong Zhang, and Quanquan Gu
  [MedRxiv]

• Saving Gradient and Negative Curvature Computations: Finding Local Minima More Efficiently
  Yaodong Yu*, Difan Zou*, Quanquan Gu
  [ArXiv]

Publications in Wireless Communication & Signal Processing

• An Efficient Iterative Least Square Method for Indoor Visible Light Positioning under Shot Noise
  Xiaona Liu, Difan Zou, Nuo Huang, Yang Wang
  IEEE Photonics Journal, 2023
  [Paper]

• Two-Dimensional Intensity Distribution and Adaptive Power Allocation for Ultraviolet Ad-Hoc Network
  Hong Qi, Difan Zou, Zhengyuan Xu, Chen Gong
  IEEE Transactions on Green Communications and Networking, 2022
  [Paper]

• Signal characterization and achievable transmission rate of VLC under receiver nonlinearity
  Xiaona Liu, Chen Gong, Difan Zou, Zunaira Babar, Zhengyuan Xu, Lajos Hanzo
  IEEE Access, 2019
  [Paper]

• Characterization on practical photon counting receiver in optical scattering communication
  Difan Zou, Chen Gong, Zhengyuan Xu
  IEEE Transactions on Communications, 2018 (Presented at GlobeCom 2018, Received Best Paper Award)
  [Paper]

• A 1Mbps Real-Time NLOS UV Scattering Communication System With Receiver Diversity Over 1km
  Guanchu Wang, Kun Wang, Chen Gong, Difan Zou, Zhimeng Jiang, Zhengyuan Xu
  IEEE Photonics Journal, 2018
  [Paper]

• Signal Detection Under Short-Interval Sampling of Continuous Waveforms for Optical Wireless Scattering Communication
  Difan Zou, Chen Gong, Zhengyuan Xu
  IEEE Transactions on Wireless Communication, 2018 (Presented at GlobeSip 2016)
  [Paper]

• Secrecy rate of MISO optical wireless scattering communications
  Difan Zou, Chen Gong, Zhengyuan Xu
  IEEE Transactions on Communication, 2017
  [Paper]

• Turbulence channel modeling and non-parametric estimation for optical wireless scattering communication
  Kun Wang, Chen Gong, Difan Zou, Zhengyuan Xu
  IEEE/OSA Journal of Lightwave Technology, 2017 (Presented at ICCS 2016, Received Best Paper Award)
  [Paper]

• Demonstration of a 400 kbps real-time non-line-of-sight laser-based ultraviolet communication system over 500 m
  Kun Wang, Chen Gong, Difan Zou, Xianqing Jin, Zhengyuan Xu
  OSA Chinese Optical Letters, 2017
  [Paper]

• Information security risks outside the laser beam in terrestrial free-space optical communication
  Difan Zou, Zhengyuan Xu
  IEEE Photonics Journal, 2016
  [Paper]

• Modeling of optical wireless scattering communication channels over broad spectra
  Weihao Liu, Difan Zou, Zhengyuan Xu
  OSA Journal of the Optical Society of America A, 2015
  [Paper]