Mingyuan Ma

I spent my four years undergraduate life at UC Berkeley, with degrees in Statistics and in Computer Science. In August, 2023, I start my new journey at Harvard University to study Data Science (SM).

During summer 2023, as Machine Learning Engineer at Moonshot AI, advised by Prof. Zhilin Yang, I work on the project of Efficient Deep Learning. At Berkeley, I work with Prof. Park Sinchaisri, affiliate Berkeley Artificial Intelligence Research (BAIR), from Haas School of Business, and our research project is the Human-AI collaboration. Besides, I work closely with Prof. Yang You at HPC-AI Lab, National University of Singapore.

My current research interest is related to Efficient Deep Learning, using less resources to train our models while keeping or imporving performance. From Data perspective, I am interested in Continual Learning, Few-Shot Learning. From Algorithm perspective, I am interested in Efficient Training and Model Compression.

News

Jul 13, 2023	Our paper Preventing Zero-Shot Transfer Degradation in Continual Learning of Vision-Language Models is accepted by by ICCV 2023.
May 13, 2023	I graduate from UC Berkeley with Magna cum Laude, double majoring Statistics and Computer Science
Mar 20, 2023	I will start my Master’s in Data Science degree at Harvard SEAS
Mar 12, 2023	The first academic paper of my life with code! Thanks Zangwei and Prof. Yang You
Dec 3, 2022	I met her

Publications

Preventing Zero-Shot Transfer Degradation in Continual Learning of Vision-Language Models

Zangwei Zheng, Mingyuan Ma, Kai Wang, and 3 more authors

Accepted by ICCV 2023 , 2023

Abs HTML PDF

Continual learning (CL) can help pre-trained vision- language models efficiently adapt to new or under-trained data distributions without re-training. Nevertheless, dur- ing the continual training of the Contrastive Language- Image Pre-training (CLIP) model, we observe that the model’s zero-shot transfer ability significantly degrades due to catastrophic forgetting. Existing CL methods can miti- gate forgetting by replaying previous data. However, since the CLIP dataset is private, replay methods cannot access the pre-training dataset. In addition, replaying data of pre- viously learned downstream tasks can enhance their per- formance but comes at the cost of sacrificing zero-shot per- formance. To address this challenge, we propose a novel method ZSCL to prevent zero-shot transfer degradation in the continual learning of vision-language models in both feature and parameter space. In the feature space, a ref- erence dataset is introduced for distillation between the current and initial models. The reference dataset should have semantic diversity but no need to be labeled, seen in pre-training, or matched image-text pairs. In parame- ter space, we prevent a large parameter shift by averag- ing weights during the training. We propose a more chal- lenging Multi-domain Task Incremental Learning (MTIL) benchmark to evaluate different methods, where tasks are from various domains instead of class-separated in a sin- gle dataset. Our method outperforms other methods in the traditional class-incremental learning setting and the MTIL by 9.7% average score. Our code locates at https: //github.com/Thunderbeee/ZSCL.