Publications

* indicates equal contribution.

Publications

  • Reverse Transition Kernel: A Flexible Framework to Accelerate Diffusion Inference
    Xunpeng Huang, Difan Zou, Hanze Dong, Yi Zhang, Yi-An Ma, Tong Zhang
    ICML Workshop on Structured Probabilistic Inference & Generative Modeling, 2024 (Oral Presentation & Best Paper Award)
    Conference on Advances in Neural Information Processing Systems (NeurIPS), 2024 (Spotlight)
    [Arxiv]

  • Slight Corruption in Pre-training Data Makes Better Diffusion Models
    Hao Chen, Yujin Han, Diganta Misra, Xiang Li, Kai Hu, Difan Zou, Masashi Sugiyama, Jindong Wang, Bhiksha Raj
    Conference on Advances in Neural Information Processing Systems (NeurIPS), 2024 (Spotlight)
    [Arxiv]

  • How Transformers Utilize Multi-Head Attention in In-Context Learning? A Case Study on Sparse Linear Regression
    Xingwu Chen*, Lei Zhao*, Difan Zou
    Conference on Advances in Neural Information Processing Systems (NeurIPS), 2024
    [Arxiv]

  • The Implicit Bias of Adam on Separable Data
    Chenyang Zhang, Difan Zou, Yuan Cao
    Conference on Advances in Neural Information Processing Systems (NeurIPS), 2024
    [Arxiv]

  • A Comprehensive Investigation of Sparse Rate Reduction in Transformer-like Models
    Yunzhe Hu, Difan Zou, Dong Xu
    Conference on Advances in Neural Information Processing Systems (NeurIPS), 2024
    [Arxiv]

  • Per-Example Gradient Regularization Improves Learning Signals from Noisy Data
    Xuran Meng, Yuan Cao, Difan Zou
    Springer Machine Learning Journal, 2024 (accepted)
    [Arxiv]

  • Faster Sampling without Isoperimetry via Diffusion-based Monte Carlo
    Xunpeng Huang, Difan Zou, Hanze Dong, Yian Ma, and Tong Zhang
    Annual Conference on Learning Theory (COLT), 2024
    [Paper][Arxiv]

  • What Can Transformer Learn with Varying Depth? Case Studies on Sequence Learning Tasks
    Xingwu Chen, Difan Zou
    ICLR Workshop on Bridging the Gap Between Practice and Theory in Deep Learning (BPGT), 2024 (Oral Presentation)
    International Conference on Machine Learning (ICML), 2024
    [Paper][Arxiv]

  • Benign Overfitting in Two-Layer ReLU Convolutional Neural Networks for XOR Data
    Xuran Meng, Difan Zou, Yuan Cao
    International Conference on Machine Learning (ICML), 2024
    [Paper][Arxiv]

  • Improving Group Robustness on Spurious Correlation Requires Preciser Group Inference
    Yujin Han, Difan Zou
    International Conference on Machine Learning (ICML), 2024
    [Paper][Arxiv]

  • Faster Sampling via Stochastic Gradient Proximal Sampler
    Xunpeng Huang, Difan Zou, Yian Ma, Hanze Dong, Tong Zhang
    International Conference on Machine Learning (ICML), 2024
    [Paper][Arxiv]

  • On the Limitation and Experience Replay for GNNs in Continual Learning
    Junwei Su, Difan Zou, Chuan Wu
    Conference on Lifelong Learning Agents (CoLLAs), 2024

  • Benign Oscillation of Stochastic Gradient Descent with Large Learning Rates
    Miao Lu, Beining Wu, Xiaodong Yang, Difan Zou
    NeurIPS Workshop on Mathematics of Modern Machine Learning (M3L), 2023
    International Conference on Learning Representations (ICLR), 2024
    [Paper][Arxiv]

  • How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?
    Jingfeng Wu, Difan Zou, Zixiang Chen, Vladimir Braverman, Quanquan Gu, Peter L. Bartlett
    International Conference on Learning Representations (ICLR) (Spotlight), 2024
    [Paper][Arxiv]

  • PRES: Toward Scalable Memory-Based Dynamic Graph Neural Networks
    Junwei Su, Difan Zou, Chuan Wu
    International Conference on Learning Representations (ICLR), 2024
    [Paper][Arxiv][Code]

  • The Implicit Bias of Batch Normalization in Linear Models and Two-layer Linear Convolutional Neural Networks
    Yuan Cao, Difan Zou, Yuanzhi Li, Quanquan Gu
    Annual Conference on Learning Theory (COLT), 2023
    [Arxiv]

  • The Benefits of Mixup for Feature Learning
    Difan Zou, Yuan Cao, Yuanzhi Li, Quanquan Gu
    International Conference on Machine Learning (ICML), 2023
    [Paper][Arxiv]

  • Learning High-Dimensional Single-Neuron ReLU Networks with Finite Samples
    Jingfeng Wu*, Difan Zou*, Zixiang Chen*, Vladimir Braverman, Quanquan Gu, and Sham M. Kakade
    International Conference on Machine Learning (ICML), 2023
    [Paper][Arxiv]

  • Towards Robust Graph Incremental Learning on Evolving Graphs
    Junwei Su, Difan Zou, Zijun Zhang, Chuan Wu
    International Conference on Machine Learning (ICML), 2023
    [Arxiv][Paper][Code]

  • Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularization
    Difan Zou, Yuan Cao, Yuanzhi Li, and Quanquan Gu
    International Conference on Learning Representations (ICLR), 2023
    [Paper][ArXiv]

  • Risk Bounds of Multi-Pass SGD for Least Squares in the Interpolation Regime
    Difan Zou*, Jingfeng Wu*, Vladimir Braverman, Quanquan Gu, and Sham M. Kakade
    Conference on Advances in Neural Information Processing Systems (NeurIPS), 2022
    [Paper][ArXiv]

  • The Power and Limitation of Pretraining-Finetuning for Linear Regression under Covariate Shift
    Jingfeng Wu*, Difan Zou*, Vladimir Braverman, Quanquan Gu, and Sham M. Kakade
    Conference on Advances in Neural Information Processing Systems (NeurIPS), 2022
    [Paper][ArXiv]

  • Last Iterate Risk Bounds of SGD with Decaying Stepsize for Overparameterized Linear Regression
    Jingfeng Wu*, Difan Zou*, Vladimir Braverman, Quanquan Gu, Sham M. Kakade
    International Conference on Machine Learning (ICML), 2022 (Long Presentation)
    [Paper] [ArXiv]

  • Self-training Converts Weak Learners to Strong Learners in Mixture Models
    Spencer Frei*, Difan Zou*, Zixiang Chen*, Quanquan Gu
    International Conference on Artificial Intelligence and Statistics (AISTATS), 2022
    [Paper] [ArXiv]

  • The Benefit of Implicit Regularization from SGD in Least Square Problems
    Difan Zou*, Jingfeng Wu*, Vladimir Braverman, Quanquan Gu, Dean P. Foster, Sham M. Kakade
    Conference on Advances in Neural Information Processing Systems (NeurIPS), 2021
    [Paper] [ArXiv]

  • Benign Overfitting of Constant-Stepsize SGD for Linear Regression
    Difan Zou*, Jingfeng Wu*, Vladimir Braverman, Quanquan Gu, Sham M. Kakade
    Annual Conference on Learning Theory (COLT), 2021
    [Paper] [ArXiv]

  • Faster Convergence of Stochastic Gradient Langevin Dynamics for Non-Log-Concave Sampling
    Difan Zou, Pan Xu, Quanquan Gu
    International Conference on Uncertainty in Artificial Intelligence (UAI), 2021
    [Paper] [ArXiv]

  • On the Convergence of Hamiltonian Monte Carlo with Stochastic Gradients
    Difan Zou, Quanquan Gu
    International Conference on Machine Learning (ICML), 2021
    [Paper]

  • Provable Robustness of Adversarial Training for Learning Halfspaces with Noise
    Difan Zou*, Spencer Frei*, Quanquan Gu
    International Conference on Machine Learning (ICML), 2021
    [Paper] [ArXiv]

  • How Much Over-parameterization Is Sufficient to Learn Deep ReLU Networks?
    Zixiang Chen*, Yuan Cao*, Difan Zou* and Quanquan Gu
    International Conference on Learning Representations (ICLR), 2021
    [Paper] [ArXiv]

  • Direction Matters: On the Implicit Regularization Effect of Stochastic Gradient Descent with Moderate Learning Rate
    Jingfeng Wu, Difan Zou, Vladimir Braverman and Quanquan Gu
    International Conference on Learning Representations (ICLR), 2021
    [Paper] [ArXiv]

  • Laplacian Smoothing Stochastic Gradient Markov Chain Monte Carlo
    Bao Wang*, Difan Zou*, Quanquan Gu, Stanley Osher
    SIAM Journal on Scientific Computing (SISC), 2021
    [Paper] [ArXiv] [Code]

  • On the Global Convergence of Training Deep Linear ResNets
    Difan Zou, Philip M. Long, Quanquan Gu
    International Conference on Learning Representations (ICLR), 2020
    [Paper]

  • Improving Adversarial Robustness Requires Revisiting Misclassified Examples
    Yisen Wang*, Difan Zou*, Jinfeng Yi, James Bailey, Xingjun Ma and Quanquan Gu
    International Conference on Learning Representations (ICLR), 2020
    [Paper] [Code]

  • Gradient Descent Optimizes Over-parameterized Deep ReLU Networks
    Difan Zou*, Yuan Cao*, Dongruo Zhou, Quanquan Gu
    Springer Machine Learning Journal, 2020
    [Paper] [ArXiv]

  • An Improved Analysis of Training Over-parameterized Deep Neural Networks
    Difan Zou, Quanquan Gu
    Conference on Advances in Neural Information Processing Systems (NeurIPS), 2019
    [Paper] [ArXiv]

  • Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networks
    Difan Zou*, Ziniu Hu*, Yewen Wang, Song Jiang, Yizhou Sun, Quanquan Gu
    Conference on Advances in Neural Information Processing Systems (NeurIPS), 2019
    [Paper] [ArXiv] [Code]

  • Stochastic Gradient Hamiltonian Monte Carlo Methods with Recursive Variance Reduction
    Difan Zou, Pan Xu, Quanquan Gu
    Conference on Advances in Neural Information Processing Systems (NeurIPS), 2019
    [Paper]

  • Sampling from Non-Log-Concave Distributions via Variance-Reduced Gradient Langevin Dynamics
    Difan Zou, Pan Xu, Quanquan Gu
    International Conference on Artificial Intelligence and Statistics (AISTATS), 2019
    [Paper]

  • Global convergence of Langevin dynamics based algorithms for nonconvex optimization
    Pan Xu*, Jinghui Chen*, Difan Zou, Quanquan Gu
    Conference on Advances in Neural Information Processing Systems (NeurIPS), 2018, (Spotlight)
    [Paper] [ArXiv]

  • Subsampled stochastic variance-reduced gradient Langevin dynamics
    Difan Zou*, Pan Xu*, Quanquan Gu
    International Conference on Uncertainty in Artificial Intelligence (UAI), 2018
    [Paper]

  • Stochastic Variance-Reduced Hamilton Monte Carlo Methods
    Difan Zou*, Pan Xu*, Quanquan Gu
    International Conference on Machine Learning (ICML), 2018
    [Paper] [ArXiv]

Preprint

  • Self-Control of LLM Behaviors by Compressing Suffix Gradient into Prefix Controller
    Min Cai, Yuchen Zhang, Shichang Zhang, Fan Yin, Difan Zou, Yisong Yue, Ziniu Hu
    [Arxiv][Website][Code]

  • A Human-Like Reasoning Framework for Multi-Phases Planning Task with Large Language Models
    Chengxing Xie, Difan Zou
    ICML Workshop on LLMs and Cognition, 2024
    [Arxiv]

  • On the Benefits of Over-parameterization for Out-of-Distribution Generalization
    Yifan Hao, Yong Lin, Difan Zou, and Tong Zhang
    [Arxiv]

  • Improving Implicit Regularization of SGD with Preconditioning for Least Square Problems
    Junwei Su, Difan Zou, and Chuan Wu
    [Arxiv]

  • An Improved Analysis of Langevin Algorithms with Prior Diffusion for Non-Log-Concave Sampling
    Xunpeng Huang, Hanze Dong, Difan Zou, and Tong Zhang
    [Arxiv]

  • Less is More: On the Feature Redundancy of Pertrained Models When Transferring to Few-Shot Tasks
    Xu Luo, Difan Zou, Lianli Gao, Zenglin Xu, Jingkuan Song
    [Arxiv]

  • Epidemic Model Guided Machine Learning for COVID-19 Forecasts in the United States
    Difan Zou, Lingxiao Wang, Pan Xu, Jinghui Chen, Weitong Zhang, and Quanquan Gu
    [MedRxiv]

  • Saving Gradient and Negative Curvature Computations: Finding Local Minima More Efficiently
    Yaodong Yu*, Difan Zou*, Quanquan Gu
    [ArXiv]

Publications in Wireless Communication & Signal Processing

  • An Efficient Iterative Least Square Method for Indoor Visible Light Positioning under Shot Noise
    Xiaona Liu, Difan Zou, Nuo Huang, Yang Wang
    IEEE Photonics Journal, 2023
    [Paper]

  • Two-Dimensional Intensity Distribution and Adaptive Power Allocation for Ultraviolet Ad-Hoc Network
    Hong Qi, Difan Zou, Zhengyuan Xu, Chen Gong
    IEEE Transactions on Green Communications and Networking, 2022
    [Paper]

  • Signal characterization and achievable transmission rate of VLC under receiver nonlinearity
    Xiaona Liu, Chen Gong, Difan Zou, Zunaira Babar, Zhengyuan Xu, Lajos Hanzo
    IEEE Access, 2019
    [Paper]

  • Characterization on practical photon counting receiver in optical scattering communication
    Difan Zou, Chen Gong, Zhengyuan Xu
    IEEE Transactions on Communications, 2018 (Presented at GlobeCom 2018, Received Best Paper Award)
    [Paper]

  • A 1Mbps Real-Time NLOS UV Scattering Communication System With Receiver Diversity Over 1km
    Guanchu Wang, Kun Wang, Chen Gong, Difan Zou, Zhimeng Jiang, Zhengyuan Xu
    IEEE Photonics Journal, 2018
    [Paper]

  • Signal Detection Under Short-Interval Sampling of Continuous Waveforms for Optical Wireless Scattering Communication
    Difan Zou, Chen Gong, Zhengyuan Xu
    IEEE Transactions on Wireless Communication, 2018 (Presented at GlobeSip 2016)
    [Paper]

  • Secrecy rate of MISO optical wireless scattering communications
    Difan Zou, Chen Gong, Zhengyuan Xu
    IEEE Transactions on Communication, 2017
    [Paper]

  • Turbulence channel modeling and non-parametric estimation for optical wireless scattering communication
    Kun Wang, Chen Gong, Difan Zou, Zhengyuan Xu
    IEEE/OSA Journal of Lightwave Technology, 2017 (Presented at ICCS 2016, Received Best Paper Award)
    [Paper]

  • Demonstration of a 400 kbps real-time non-line-of-sight laser-based ultraviolet communication system over 500 m
    Kun Wang, Chen Gong, Difan Zou, Xianqing Jin, Zhengyuan Xu
    OSA Chinese Optical Letters, 2017
    [Paper]

  • Information security risks outside the laser beam in terrestrial free-space optical communication
    Difan Zou, Zhengyuan Xu
    IEEE Photonics Journal, 2016
    [Paper]

  • Modeling of optical wireless scattering communication channels over broad spectra
    Weihao Liu, Difan Zou, Zhengyuan Xu
    OSA Journal of the Optical Society of America A, 2015
    [Paper]