Tsui-Wei (Lily) Weng

I am an assistant professor in Halıcıoğlu Data Science Institute with affiliation to Computer Science and Engineering Department at UC San Diego.

I direct the Trustworthy Machine Learning Lab at UCSD. My research vision is to make the next generation AI systems and deep learning algorithms more robust, reliable, explainable, trustworthy and safer.

Google Scholar / Research / Teaching / Email: lweng@ucsd.edu

I’m open to collaborations with highly motivated students with strong machine learning and mathematical backgrounds!

• For UCSD (current/admitted) students: Please fill out this form and contact me with your CV and research experience.
• For Prospective (non-UCSD) students: Please apply direclty through our HDSI Graduate program, CSE Graduate program and indicate my name in your application. You are welcome to send me a note with your CV once you have applied.

Bio

I received my PhD (2020) in Electrical Engineering and Computer Science from MIT advised by Prof. Luca Daniel on the topic of Robust Machine Learning with focus on Neural Networks, and M.S. (2013) and B.S. (2011) degrees in Electrical Engineering from National Taiwan University advised by Prof. Tzong-Lin Wu and Prof. Peter Shiue on the topic of microwave circuits and combinatorics. Prior to UCSD, I spent 1 year in MIT-IBM Watson AI Lab and several research internship in Google DeepMind, IBM Research and Mitsubishi Electric Research Lab.

Research Focus: Trustworthy ML & AI Safety

• Interpretable Deep Learning
• Robust Deep Learning
• Foundations of Deep Learning

Selected Honors and Awards

• Jan 2025: Honored to receive Nvidia Academic Award to support our research on training Interpretable LLMs!
• Sep 2024: Honored to receive Intel Rising Star Faculty Award! See Announcement and UCSD News!
• Sep 2024: Honored to receive NSF CISE-IIS Award as PI to develop Foundations of Trustworthy Deep Learning! See UCSD News!
• May 2024: Honored to receive Hellman Fellowship for Junior faculty who show capacity for great distinction in research!
• Feb 2024: Honored to be selected as New Faculty Highlights in AAAI! Talk title: Towards Trustworthy Deep Learning!
• Sep 2023: Honored to receive NSF CISE-IIS Medium Award as PI for Recourse Verification!
• Sep 2021: Honored to receive NSF CISE-CCF Medium Award as Co-PI for Efficient and Robust Neural Network Design!

Group

• PhD Students: Tuomas Oikarinen, Ge Yan, Chung En Sun, Akshay Kulkarni
  
• MS Students: Tongqing Shi, Jerry John
  
• Alumni: Anvita Srinivas (MS, UCSD; First employment: Latitude AI), Avni Kothari (MS, UCSD; First employment: UCSF), Linbo Liu (PhD, UCSD; First employment: Amazon), Mohammad Ali Khan (MS, UCSD; First employment: Apple)

Teaching

• DSC 210 Numerical Linear Algebra: Fall 2022, Spring 2022, Fall 2023, Fall 2024
  
• DSC 291 Trustworthy Machine Learning: Fall 2021, Spring 2023, Spring 2024
  
• DSC 140B Representation Learning: Spring 2024
  

Preprints

• Interpretable Generative Models through Post-hoc Concept Bottlenecks | [code] | [project page]
  A. Kulkarni, G. Yan², C. Sun², T. Oikarinen, and T.-W. Weng
  CVPR 2025 (accepted)

• Concept Bottleneck Large Language Models | [code] | [project page]
  C. Sun, T. Oikarinen, B. Ustun, and T.-W. Weng
  ICLR 2025 (accepted)

• Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities
  C. Sun, X. Liu, W. Yang, T.-W. Weng, H. Cheng, A. San, M. Galley, and J. Gao
  NAACL 2025 (accepted)

• Effective Skill Unlearning through Intervention and Abstention | [code] | [project page]
  Y. Li, C. Sun, and T.-W. Weng
  NAACL 2025 (accepted)

• Interpreting Neurons in Deep Vision Networks with Language Models | [code] | [project page]
  N. Bai¹, R. A. Iyer¹, T. Oikarinen, A. Kulkarni, and T.-W. Weng
  TMLR 2025

• RAT: Boosting Misclassification Detection Ability without Extra Data | [code] | [project page]
  G. Yan and T.-W. Weng
  Preprint

Selected recent works

• VLG-CBM: Training Concept Bottleneck Models with Vision-Language Guidance | [code] | [project page]
  D. Srivastava¹, G. Yan¹, and T.-W. Weng
  NeurIPS 2024

• Abstracted Shapes as Tokens - A Generalizable and Interpretable Model for Time-series Classification | [code]
  Y. Wen, T. Ma, T.-W. Weng, L. M. Nguyen, and A. A. Julius
  NeurIPS 2024

• Probabilistic Federated Prompt-Tuning with Non-IID and Imbalanced Data | [code]
  P.-Y. Weng, M. Hoang, L. M. Nguyen, M. T. Thai, T.-W. Weng, and T. N. Hoang
  NeurIPS 2024

• Provable and Efficient Dataset Distillation for Kernel Ridge Regression | [code] | [project page]
  Y. Chen, W. Huang, and T.-W. Weng
  NeurIPS 2024

• Interpretability-Guided Test-Time Adversarial Defense | [code] | [project page]
  A. Kulkarni and T.-W. Weng
  ECCV 2024

• Concept-Driven Continual Learning | [code] | [project page]
  S.-H. Yang, T. Oikarinen, and T.-W. Weng
  TMLR 2024

• Linear Explanations for Individual Neurons | [code] | [project page]
  T. Oikarinen and T.-W. Weng
  ICML 2024

• Breaking the Barrier: Enhanced Utility and Robustness in Smoothed DRL Agents | [code] | [project page]
  C. Sun, S. Gao, and T.-W. Weng
  ICML 2024

• AND: Audio Network Dissection for Interpreting Deep Acoustic Models | [code] | [project page]
  T.-Y. Wu¹, Y.-X. Lin¹, and T.-W. Weng
  ICML 2024

• Crafting Large Language Models for Enhanced Interpretability | [code]
  C. Sun, T. Oikarinen, and T.-W. Weng
  ICML 2024 Mechanistic Interpretability workshop

• Provably Robust Conformal Prediction with Improved Efficiency | [code] | [project page]
  G. Yan, Y. Romano, and T.-W. Weng
  ICLR 2024

• Prediction without Preclusion: Recourse Verification with Reachable Sets | [code]
  A. Kothari¹, B. Kulynych¹, T.-W. Weng and B. Ustun
  ICLR 2024

• Concept-Monitor: Understanding DNN training through individual neurons
  M. A. Khan¹, T. Oikarinen¹, and T.-W. Weng
  AAAI 2024 DAI workshop

• The Importance of Prompt Tuning for Automated Neuron Explanations | [code] | [project page]
  J. Lee¹, T. Oikarinen¹, A. Chatha, K.-C. Chang³, Y. Chen³, and T.-W. Weng
  NeurIPS 2023 ATTRIB workshop

• CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networks | [code]
  T. Oikarinen and T.-W. Weng
  ICLR 2023, notable-top-25% (spotlight)

• Label-free Concept Bottleneck Models | [code]
  T. Oikarinen, Subrho Das, Lam M. Nguyen and T.-W. Weng
  ICLR 2023

• Min-Max Bilevel Multi-objective Optimization with Applications in Robust Machine Learning
  A. Gu, S. Lu, P. Ram and T.-W. Weng
  ICLR 2023

• Corrupting Neuron Explanations of Deep Visual Features | [code]
  D. Srivastava, T. Oikarinen, and T.-W. Weng
  ICCV 2023

• Analyzing Generalization of Neural Networks through Loss Path Kernels | [code]
  Y. Chen, W. Huang, H. Wang, C. Loh, A. Srivastava, L. Nguyen, and T.-W. Weng
  NeurIPS 2023

• Attacking c-MARL More Effectively: A Data Driven Approach | [code]
  N. H. Pham, L. M. Nguyen, J. Chen, H. T. Lam, S. Das, and T.-W. Weng
  ICDM 2023

• ConCerNet: A Contrastive Learning Based Framework for Automated Conservation Law Discovery and Trustworthy Dynamical System Prediction | [code]
  W. Zhang, T.-W. Weng, S. Das, A. Megretski, L. Daniel and L. Nguyen
  ICML 2023

• Quantifying Safety of Learning-based Self-Driving Control Using Almost-Barrier Functions
  Z. Qin, T.-W. Weng and S. Gao
  IROS 2022

• Revisiting Contrastive Learning through the Lens of Neighborhood Component Analysis: an Integrated Framework
  C.-Y. Ko, J. Mohapatra, S. Liu, P.-Y. Chen, L. Daniel and T.-W. Weng
  ICML 2022

• Adversarially Robust Conformal Prediction
  A. Gendler, T.-W. Weng, L. Daniel and Y. Romano
  ICLR 2022

• Robust Deep Reinforcement Learning through Adversarial Loss
  T. Oikarinen, W. Zhang, A. Megretski, L. Daniel and T.-W. Weng
  NeurIPS 2021

• On the Equivalence between Neural Network and Support Vector Machine
  Y. Chen, W. Huang, L. M. Nguyen and T.-W. Weng
  NeurIPS 2021

Publications (by topics)

Interpretable Deep Learning

• Concept Bottleneck Large Language Models | [code] | [project page]
  C. Sun, T. Oikarinen, B. Ustun, and T.-W. Weng
  ICLR 2025 (accepted)

• Interpreting Neurons in Deep Vision Networks with Language Models | [code] | [project page]
  N. Bai¹, R. A. Iyer¹, T. Oikarinen, A. Kulkarni, and T.-W. Weng
  TMLR 2025

• VLG-CBM: Training Concept Bottleneck Models with Vision-Language Guidance | [code] | [project page]
  D. Srivastava¹, G. Yan¹, and T.-W. Weng
  NeurIPS 2024

• Concept-Driven Continual Learning | [code] | [project page]
  S.-H. Yang, T. Oikarinen and T.-W. Weng
  TMLR 2024

• Linear Explanations for Individual Neurons | [code] | [project page]
  T. Oikarinen and T.-W. Weng
  ICML 2024

• AND: Audio Network Dissection for Interpreting Deep Acoustic Models | [code] | [project page]
  T.-Y. Wu¹, Y.-X. Lin¹ and T.-W. Weng
  ICML 2024

• Crafting Large Language Models for Enhanced Interpretability | [code]
  C. Sun, T. Oikarinen, and T.-W. Weng
  ICML 2024 Mechanistic Interpretability workshop

• The Importance of Prompt Tuning for Automated Neuron Explanations | [code] | [project page]
  J. Lee¹, T. Oikarinen¹, A. Chatha, K.-C. Chang³, Y. Chen³, and T.-W. Weng
  NeurIPS 2023 ATTRIB workshop

• CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networks | [code]
  T. Oikarinen and T.-W. Weng
  ICLR 2023, notable-top-25% (spotlight)

• Label-free Concept Bottleneck Models | [code]
  T. Oikarinen, Subrho Das, Lam M. Nguyen and T.-W. Weng
  ICLR 2023

• Corrupting Neuron Explanations of Deep Visual Features | [code]
  D. Srivastava, T. Oikarinen, and T.-W. Weng
  ICCV 2023

Robust Deep Learning

Enhancing robustness of deep neural networks & AI safety

• Interpretability-Guided Test-Time Adversarial Defense | [code] | [project page]
  A. Kulkarni and T.-W. Weng
  ECCV 2024

• Breaking the Barrier: Enhanced Utility and Robustness in Smoothed DRL Agents | [code] | [project page]
  C. Sun, S. Gao and T.-W. Weng
  ICML 2024

• Provably Robust Conformal Prediction with Improved Efficiency | [code] | [project page]
  G. Yan, Y. Romano and T.-W. Weng
  ICLR 2024

• Promoting Robustness of Randomized Smoothing: Two Cost-Effective Approaches
  L. Liu, T. N. Hoang, L. M. Nguyen, and T.-W. Weng
  ICDM 2023

• Quantifying Safety of Learning-based Self-Driving Control Using Almost-Barrier Functions
  Z. Qin, T.-W. Weng and S. Gao
  IROS 2022

• Adversarially Robust Conformal Prediction
  A. Gendler, T.-W. Weng, L. Daniel and Y. Romano
  ICLR 2022

• Robust Deep Reinforcement Learning through Adversarial Loss
  T. Oikarinen, W. Zhang, A. Megretski, L. Daniel and T.-W. Weng
  NeurIPS 2021

• On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta Learning
  R. Wang, K. Xu, S. Liu, P.-Y. Chen, T.-W. Weng, G. Chuang and M. Wang
  ICLR 2021

• Fast Training of Provably Robust Neural Networks by SingleProp
  A. Boopathy, T.-W. Weng, S. Liu, P.-Y. Chen, G. Zhang and L. Daniel
  AAAI 2021

• Neural Network Control Policy Verification with Persistent Adversarial Perturbations
  Y.-S. Wang, T.-W. Weng, and L. Daniel,
  ICML 2020

• Robust Deep Reinforcement Learning through Adversarial Loss
  T. Oikarinen, T.-W. Weng and L. Daniel
  ICML 2020, Uncertainty and Robustness in Deep Learning workshop

• Toward Evaluating Robustness of Deep Reinforcement Learning with Continuous Control
  T.-W. Weng, K. Dvijotham², J. Uesato², K. Xiao², S. Gowal², R. Stanforth², Pushmeet Kohli
  ICLR 2020

Neural network robustness, Deep learning theory, and Foundation models

• Analyzing Generalization of Neural Networks through Loss Path Kernels
  Y. Chen, W. Huang, H. Wang, C. Loh, A. Srivastava, L. Nguyen, and T.-W. Weng
  NeurIPS 2023

• Attacking c-MARL More Effectively: A Data Driven Approach | [code]
  N. H. Pham, L. M. Nguyen, J. Chen, H. T. Lam, S. Das, and T.-W. Weng
  ICDM 2023

• ConCerNet: A Contrastive Learning Based Framework for Automated Conservation Law Discovery and Trustworthy Dynamical System Prediction | [code]
  W. Zhang, T.-W. Weng, S. Das, A. Megretski, L. Daniel and L. Nguyen
  ICML 2023

• Revisiting Contrastive Learning through the Lens of Neighborhood Component Analysis: an Integrated Framework
  C.-Y. Ko, J. Mohapatra, S. Liu, P.-Y. Chen, L. Daniel and T.-W. Weng
  ICML 2022

• On the Equivalence between Neural Network and Support Vector Machine
  Y. Chen, W. Huang, L. M. Nguyen and T.-W. Weng
  NeurIPS 2021

Neural network robustness verification and certification (with provable guarantees)

• Hidden Cost of randomized smoothing
  J. Mohapatra, C.-Y. Ko, T.-W. Weng, P.-Y. Chen, S. Liu and L. Daniel
  AIstats 2021

• Higher order certification for randomized smoothing
  J. Mohapatra, C.-Y. Ko, T.-W. Weng, P.-Y. Chen, S. Liu and L. Daniel
  NeurIPS 2020

• Certified Interpretability Robustness for Class Activation Mapping
  A. Gu, T.-W. Weng, P.-Y. Chen, S. Liu and L. Daniel
  NeurIPS 2020, ML4AD workshop [Video]

• Towards Verifying Robustness of Neural Networks Against a Family of Semantic Perturbations
  J. Mohapatra, T.-W. Weng, P.-Y. Chen, S. Liu and L. Daniel
  CVPR 2020 [Video]

• Towards Certificated Model Robustness Against Weight Perturbations
  T.-W. Weng¹, P. Zhao¹, S. Liu, P.-Y. Chen and X. Lin, L. Daniel
  AAAI 2020

• Verification of Neural Network Control Policy Under Persistent Adversarial Perturbation
  Y.-S. Wang, T.-W. Weng and L. Daniel
  NeurIPS 2019, Safety and Robustness in Decision Making Workshop

• PROVEN: Verifying Robustness of Neural Networks with a Probabilistic Approach
  T.-W. Weng, P.-Y. Chen², L. M. Nguyen², M. S. Squillante², A. Boopathy, I. Oseledets, and L. Daniel
  ICML 2019

• POPQORN: Quantifying Robustness of Recurrent Neural Networks
  C.-Y. Ko¹, Z. Lyu¹, T.-W. Weng, L. Daniel, N. Wong, and D. Lin
  ICML 2019

• CNN-Cert: An Efficient Framework for Certifying Robustness of Convolutional Neural Networks
  A. Boopathy, T.-W. Weng, P.-Y. Chen, S. Liu and L. Daniel
  AAAI 2019

• Efficient Neural Network Robustness Certification with General Activation Functions
  H. Zhang¹, T.-W. Weng¹, P.-Y. Chen, C.-J. Hsieh and L. Daniel
  NeurIPS 2018

• Toward Fast Computation of Certified Robustness for ReLU Networks
  T.-W. Weng¹, H. Zhang¹, H. Chen, Z. Song, C.-J. Hsieh, D. Boning, I. S. Dhillon and L. Daniel
  ICML 2018

Neural network robustness evaluation and estimation

• Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach
  T.-W. Weng¹, H. Zhang¹, P.-Y Chen, J. Yi, D. Su, Y. Gao, C.-J. Hsieh and L. Daniel
  ICLR 2018

• On extensions of clever: A neural network robustness evaluation algorithm
  T.-W. Weng¹, H. Zhang¹, P.-Y Chen, A. Lozano, C.-J. Hsieh and L. Daniel
  IEEE GlobalSIP 2018

Robust Regression and Robust PCA

• Robust PCA through Robust Optimization
  T.-W. Weng and L. Daniel
  NeurIPS 2017, Women in Machine Learning Workshop (WiML)

• Computing Least Trimmed Squares Regression to Certifiable Optimality
  T.-W. Weng, R. Mazumder and L. Daniel
  NeurIPS 2017, Women in Machine Learning Workshop (WiML)

Uncertainty Quantification in silicon photonics

• Uncertainty Quantification of Silicon Photonic Devices with Correlated and Non-Gaussian Random Parameters
  T.-W. Weng¹, Z. Zhang¹, Z. Su, Y. Marzouk, A. Melloni and L. Daniel
  Optics Express, vol. 23, Issue 4, pp.4242-4254, 2015

• Stochastic Simulation and Robust Design Optimization of Integrated Photonic Filters
  T.-W. Weng, D. Melati, A. Melloni and L. Daniel
  Nanophotonics, vol. 6, Issue 1, pp. 299-308, July 2016

• A Big-Data Approach to Handle Process Variations: Uncertainty Quantification by Tensor Recovery
  Z. Zhang, T.-W. Weng, and L. Daniel
  IEEE Components, Packaging and Manufacturing Technology, Dec. 2016 (Best paper award)

Microwave circuits and Signal Integrity

• A Novel Miniaturized Bandstop Filter Using Defected Ground on System in Package(SiP)
  T.-W. Weng and T.-L. Wu
  IEEE EPEPS, Tempe, Arizona, USA, Oct. 2012

• Synthesis Model and Design of a Common-Mode Bandstop Filter (CM-BSF) with An All-Pass Characteristic for High-Speed Differential Signals
  T.-W. Weng, C.-H. Tsai, C.-H. Chen, D.-H. Han and T.-L. Wu
  IEEE Trans. Microw. Theory Tech., vol. 62, no.8, pp.1647-1656, Aug. 2014

Combinatorics

• Hyperbolic Expressions of Polynomial Sequences and Parametric Number Sequences Defined by Linear Recurrence Relations of Order 2
  T.-X. He, P. J.-S. Shiue, T.-W. Weng
  Journal of Concrete and Applicable Mathematics, vol. 12, 63-85, 2014

• Sequences of Numbers Meet the Generalized Gegenbauer-Humbert Polynomials
  T.-X. He, P. J.-S. Shiue, T.-W. Weng
  ISRN Discrete Mathematics, vol. 2011, Article ID 674167, 16 pages, 2011

• On relations of Chebyshev polynomial, Morgan-Voyce polynomial, Fibonacci number, Pell number and Lucas number (in Chinese)
  T.-W. Weng
  Mathmedia, v.34, no. 4, p.31-42, 2010