Zheng Zhang's Homepage

	Full Professor [IEEE-Style Short Biography, Full CV (PDF)] Department of Electrical & Computer Engineering, University of California, Santa Barbara (UCSB) Department of Computer Science (joint appointment), UCSB Department of Mathematics, (joint appointment, effective in 07/2019), UCSB
	Education Ph.D in EECS, Massachusetts Institute of Technology M.Phil in EE, The University of Hong Kong B. Eng in EE, Huazhong University of Science & Technology
	Contact Email: zhengzhang [AT] ece [dot] ucsb [dot] edu, Phone: 805-893-7294 Address: 4109 Harold Frank Hall, University of California, Santa Barbara, CA 93106

About research openings: Due to the uncertain funding landscape, we do not have postdoc openings at this moment. It is very likely that we will not have new PhD openings either. However, please feel free to submit your application via our online graduate application system and mention my name, if you are interested in our research. I will review your case if the funding situation changes before the decision-making deadline.

Checklist for paper writing: I have prepared a detailed checklist to help science/engineering graduate students improving their paper writing.

To prospective PhD students: Please read this document if you are thinking about pursuing a PhD degree. The skill sets required for PhD research are very different from those required for undergraduate study. In undergraduate study, a student learns existing knowledge that were created by others (probably a few hundred years ago). A PhD student is expected to create new knowledge. A student doesn't have to be super smart or to have a perfect GPA in order to be an excellent PhD student, but he/she may need to be self-motivated for scientific research, curious about unknown/new fields, open-minded to different opinions, and persistent when facing research challenges (or even failures).

RESEARCH INTERESTS

We work at the intersection of computational data science (e.g., uncertainty quantification, tensor computation, scientific machine learning) and hardware systems. Currently we focus on two broad directions:

Responsible AI systems: (1) hardware-efficient pre-training and fine-tuning of AI foundation models (or large language models)]; (2) memory- and energy-efficient on-device deployment and training; (3) self-healing machine learning systems.
Design automation: (1) uncertainty-aware design automation for electronics, photonics, and quantum circuits; (2) small-data operator learning for multi-physics design of 3D IC and chiplet.

Our research is supported by both government funding agencies (e.g., NSF, DOE, NIST) and industries (e.g., Meta, Intel, Amazon and Samsung). We are actively collaborating with industrial research teams (e.g., Meta, Intel, Cadence, HP, Amazon, and NVIDIA) to make practical impact.

RECENT NEWS:

[MLSys'2026]. 01/2026: We have two papers accepted to MLSys'2026. The work "BOOST: BOttleneck-Optimized Scalable Training Framework for Low-Rank Large Language Models" led by Zhengyang Wang optimized the performance of low-rank LLM pre-training frameworks on HPC. The work "SkipKV: Selective Skipping of KV Generation and Storage for Efficient Inference with Large Reasoning Models" led by Jiayi Tian proposed a new training-free KV compression method for efficient chain-of-thought (COT) reasoning of LLMs.
[ICLR'2026]. 01/2026: Jin Lee's paper "KANO: Kolmogorov-Arnold Neural Operator" is accepted to ICLR'2026. This paper proposed an interpretable and symbolic operator learning framework that is efficient to capture both the frequency- and spatial-domain behaviors.
[DeepOHeat-v1].10/2025: Our DeepOHeat-v1 framework led by Xinling Yu is accepted by IEEE Trans. Components, Packaging and Manufacturing Technology. This work presented principled approaches to improve the accuracy and training efficiency of our previous DeepOHeat. More importantly, DeepOHeat also introduced a novel framework that uses operator learning in a trustworthy way to perform fast and accurate thermal optimization of semiconductor chip design.
[NeuRIPS'2025]. We have 2 papers accepted by NeuRIPS'2025. LaX (led by Ruijie Zhang and Ziyue Liu) presents a residual connection architecture to boost the accuracy of low-rank foundation models in both pre-training and fine-tuning. SharpZO (led by Yifan Yang) proposed a sharpness-aware and hybrid approach for memory-efficient BP-free fine-tuning of vision language models.
[EMNLP'2025]. We have 4 papers accepted by EMNLP'2025. CoLA (led by Ziyue Liu, Ruijie Zhang and Zhengyang Wang) presents a novel compute- and memory-efficient low-rank pre-training method for LLM; QuZO (led by Jiajun Zhou) develop an accurate quantized BP-free training method for fine-tuning LLMs; MaZO (led by Zhen Zhang) is a multi-objective BP-free fine-tuning method for LLM; Saten (led by Ryan Solgi) improves the accuracy of tensor-compressed LLMs with sparse augmentation. CoLA is also selected as an oral paper in the main conference.
[TCAD'2025] Jiayi Tian's tensor-compressed on-FPGA training accelerator for transformers is accepted by IEEE TCAD. Link to the paper.
[ACL'2025]. We have two papers accepted to ACL'2025: Wanda++ (led by Yifan Yang) and FedTT (led by Sajjid Ghiasvand)
[NeuRIPS'2024] 09/25/2024: Our work CoMERA (see the paper), a computing- and memory-efficient rank-adaptive tensor-compressed (pre)-training method, is accepted by NeuRIPS'2024. This work was led by our former postdoc Dr. Zi Yang, in collaboration with Amazon and Meta.
[EMNLP'2024] 09/20/2024: Yifan Yang's paper (see the draft) about memory-efficient zeroth-order tensor-train adaptation method for LLM fine-tuning is accepted by EMNLP'2024. This is a collaborative work with Amazon Alexa AI.
[DeepOHeat Codes] 09/2024: our source codes of DeepOHeat for 3D-IC thermal analysis is released to the public (see the link). This is a collaborative work between our group and Cadence.
[NIST research grant] 09/2024: we got a 3-year research grant from NIST to investigate small-data and uncertainty-aware design optimization methods for analog/RF integrated circuits and systems.
[AI4Science Pre-training project] 09/2024: we will start a 3-year DOE research project to investigate the theory, algorithm and HPC implementation regarding energy-efficient pre-training of AI4Science foundation models. We will collaborate with Argonne National Labs closely on this project. Besides research funding, DOE will offer the access to hundreds to thousands of state-of-the-art GPUs for us to pre-train extreme-scale AI foundation models.
[ISIT Paper on coded tensor computation] 07/2014: our collaborative paper with Prof. Haewon Jeong is presented at IEEE International Symposium on Information Theory (ISIT) held in Athen, Greece. This work was led by Jin Lee (a PhD student of Prof. Jeong), and it investigated an interesting topic: how coded computing can be extended from matrices to tensors to help quantum circuit simulation.
[PhD defense] 07/03/2024: Zhuotong Chen finished his thesis defense, and he has joined Amazon to work on large language models (LLMs). Congratulations!
[Intel Research Project] 07/01/2024: we just started a new research project with Intel to investigate multi-physics modeling and optimization of 3D integrated circuits and systems. We have some on-going collaboration with Intel in the direction of on-device AI training, and we are excited to expand our research collaboration.
[TQE Paper] 06/2024: Zichang He's paper about quantum circuit optimization under imperfect uncertainty description is published by IEEE Trans. Quantum Engineering.
[NAACL Oral Paper] 06/16/2024: Yifan Yang and Jiajun Zhou will present their LoRETTA paper at NAACL'2024 held in Mexico City, Mexico. This paper is selected as an oral paper (top 5%) of the whole conference. This paper results from our collaboration with Amazon.
[NSF Project with HP Research Labs] 06/13/2024: we will start a 3-year NSF project to collaborate with HP Research Labs on scalable photonic on-device training for scientific computing. We look forward to the research results from this academia-industry collaboration.

SELECTED PUBLICATIONS

X. Yu, Z. Liu, H. Li, Y. Li, X. Ai, Z. Zeng, I. Young and Z. Zhang, "DeepOHeat-v1: Efficient Operator Learning for Fast and Trustworthy Thermal Simulation and Optimization in 3D-IC Design," IEEE Trans. Components, Packaging and Manufacturing Technology, vol. 16, no. 1, pp. 82-96, Jan. 2026.
R. Zhang*, Z. Liu*, Z. Wang and Z Zhang, "LaX: Boosting Low-Rank Training of Foundation Models via Latent Crossing," Annual Conference on Neural Information Processing Systems (NeuRIPS), San Diego, CA, December 2025 (*Equal contributions)
Z. Liu*, R. Zhang*, Z. Wang*, Z. Yang, P. Hovland, B. Nicolae, F.Cappello and Z. Zhang, "CoLA: Compute-Efficient Pre-Training of LLMs via Low-Rank Activation," Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP'2025), pp. 4627–4645, Suzhou, China, Nov. 2025. (*Equal contributions)
R. Solgi, K. Zhen, R. V. Swaminathan, N. Susanj, A. Mouchtaris, S. Kunzmann and Z. Zhang, "Saten: Sparse Augmented Tensor Networks for Post-Training Compression of Large Language Models," Findings of the Conference on Empirical Methods in Natural Language Processing (EMNLP'2025 Findings), pp. 23674–23683, Su Zhou, China, Nov. 2025.
J. Zhou, Y. Yang, K. Zhen, Z. Liu, Y. Zhao, E. Banijamali, A. Mouchtaris, N. Wong and Z. Zhang, "QuZO: Quantized zeroth-order fine-tuning for large language models." Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP'2025), pp. 5341–5359, Suzhou, China, Nov. 2025.
J. Tian, J. Lu, H. Li, X Wang, C.C. Hao, I. Young and Z Zhang, "Ultra Memory-Efficient On-FPGA Training of Transformers via Tensor-Compressed Optimization," IEEE Trans. Computer-Aided Design of Integrated Circuits and Systems, accepted.
Y. Zhao, H. Li, I. Young and Z. Zhang, "Poor Man's Training on MCUs: A Memory-Efficient Quantized Back-Propagation-Free Approach," ACM Trans. Design Automation of Electronic Systems (ACM TODAES), July 2025.
Z. Yang, Z. Liu, S. Choudhary, X. Xie, C. Gao, S. Kunzmann and Z. Zhang, “CoMERA: Computing- and Memory-Efficient Pre-Training via Rank-Adaptive Tensor Optimization,” Annual Conference on Neural Information Processing Systems (NeuRIPS), Vancouver, Canada, December 2024.
Y. Yang, J. Zhou, N. Wong and Z. Zhang, "LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models," Conf. Northern American Association of Computational Linguistics (NAACL), Mexico City, Mexico, June 2024.
Z. Liu, Y. Li, J. Hu, X. Yu, Xin Ai, Z. Zeng, and Z. Zhang, “DeepOHeat: Operator learning-based ultra-fast thermal simulation in 3D-IC design,” ACM/IEEE Design Automation Conference (DAC), PP. 1-6, San Francisco, CA, June 2023
Y. Zhao, X. Yu, Z. Chen, Z. Liu, S. Liu and Z. Zhang, "Tensor-compressed back-propagation-free training for (physics-informed) neural networks," arXiv:2308.09858, Aug. 2023.
Z. Chen, Q. Li and Z. Zhang, "Self-healing robust neural networks via closed-loop control," Journal of Machine Learning Research, vol. 23, no. 319, pp. 1-54, 2022. .
C. Hawkins, X. Liu and Z.Zhang, "Towards compact neural networks via end-to-end training: a Bayesian tensor approach with automatic rank determination," SIAM Journal on Mathematics of Data Science, vol. 4, no. 1, pp. 46-71, Jan. 2022.
Z. He and Z. Zhang, "High-dimensional uncertainty quantification via tensor regression with rank determination and adaptive sampling," IEEE Trans. Components, Packaging and Manufacturing Technology, vol. 11, no. 9, pp. 1317-1328, Sept. 2021. (invited paper, the conference version received the best paper award at EPEPS'2020).
Z. Chen*, Q. Li* and Z. Zhang, "Towards robust neural networks via close-loop control," International Conference on Learning Representation (ICLR) 2021 (*Equally contributing authors)
C. Cui and Z. Zhang, "Stochastic collocation with non-Gaussian correlated process variations: Theory, algorithms and applications," IEEE Trans. Components, Packaging and Manufacturing Technology, vol. 9, no. 7, pp. 1362-1375, July 2019. (arXiv:1808.09720), Matlab codes, Best Paper Award
Z. Zhang, T.-W. Weng and L. Daniel, "Big-data tensor recovery for high-dimensional uncertainty quantification of process variations," IEEE Trans. Components, Packaging and Manufacturing Technology, vol. 7, no. 5, pp. 687-697, May 2017. Best Paper Award
Z. Liu and Z. Zhang, "Quantum-inspired Hamiltonian Monte Carlo for Bayesian sampling," submitted to Journal of Machine Learning Research (arXiv:1912.01937)
Z. Zhang, K. Batselier, H. Liu, L. Daniel and N. Wong, "Tensor computation: A new framework for high-dimensional problems in EDA," IEEE Trans. Computer-Aided Design of Integrated Circuits and Systems, vol. 36, no. 4, pp. 521-536, April. 2017. Invited Keynote Paper
Z. Zhang, T. A. El-Moselhy, I. M. Elfadel and L. Daniel, "Stochastic testing method for transistor-level uncertainty quantification based on generalized polynomial chaos," IEEE Trans. Computer-Aided Design of Integrated Circuits and Systems (TCAD), vol. 32, no. 10, pp. 1533-1545, Oct. 2013. Donald O. Pederson TCAD Best Paper Award
Z. Zhang, X. Yang, I. V. Oseledets, G. E. Karniadakis and L. Daniel, "Enabling high-dimensional hierarchical uncertainty quantification by ANOVA and tensor-train decomposition," IEEE Trans. Computer-Aided Design of Integrated Circuits and Systems, vol. 34, no. 1, pp. 63-76, Jan. 2015.

More publications...

ACADEMIC AWARDS

2022: Meta Research Award.
2021: ACM SIGDA Outstanding New Faculty Award (link); IEEE CEDA Ernest S. Kuh Early Career Award (link).
2020: Best Paper Award of IEEE Trans. on Components, Packaging and Manufacturing Technology (link to paper); Facebook Research Award; Best Student Paper Award at EPEPS (by PhD advisee Zichang He, link to paper).
2019: NSF CAREER Award; Rising Stars in Computational and Data Sciences (by my advisee Chunfeng Cui); Rising Stars in EECS (by my advisee Chunfeng Cui).
2018: Best Paper Award of IEEE Transactions on Components, Packaging and Manufacturing Technology (link to ppaer); Best Conference Paper Award at IEEE EPEPS (link to paper).
2016: ACM Outstanding PhD Dissertation Award in Electronic Design Automation (link); Best Paper Award at International Workshop on Signal and Power Integrity.
2015: MIT Microsystems Technology Labs (MTL) Doctoral Dissertation Seminar Award (link).
2014: Donald O. Pederson Best Paper Award of IEEE Transactions on CAD of Integrated Circuits and Systems ( link); Best Paper Nomination at IEEE CICC.
2011: Li Ka-Shing Prize (best M.Phil/Ph.D thesis award) from the University of Hong Kong (link); best paper nominations at ICCAD2011 and ASP-DAC2011.

ACADEMIC SERVICES

Associate Editor: ACM SIGDA Newsletters (2018-2019);
TPC Member: ICCAD (2016-2018), DAC (2017, 2018);
Award Committee: ACM SIGDA Best Dissertation Award Committee (2018), DAC Best Paper Award Committee (2018), ICCAD Best Paper Award Committee (2018)