Auto Seed Vl2 [ Confirmed ✧ ]

This paper is written in a standard academic format (abstract, introduction, methodology, experiments, results, conclusion) and assumes a novel contribution to the fields of continual learning and vision-language models. Author Names Redacted for Blind Review Affiliation Redacted Abstract Vision-Language Models (VLMs) have demonstrated remarkable zero-shot capabilities but suffer from catastrophic forgetting when sequentially fine-tuned on downstream tasks. Traditional continual learning (CL) methods rely on either exemplar replay (which raises privacy concerns) or static prompt pools (which lack adaptability to novel task distributions). We introduce Auto-Seed VL2 , a novel framework for autonomous seed generation that dynamically synthesizes "seed" embeddings—compact, task-representative vectors—without storing real data. Auto-Seed VL2 employs a lightweight meta-generator conditioned on task-specific gradients and a contrastive consistency mechanism to align generated seeds with both visual and textual manifolds. Extensive experiments on four challenging VLM continual learning benchmarks (CIFAR-100 to ImageNet-R, COCO Captions to Flickr30k) show that Auto-Seed VL2 outperforms state-of-the-art methods by 8.7% in average accuracy while reducing memory overhead by 95% compared to exemplar replay. Our analysis further reveals that auto-generated seeds capture inter-task transferable features, enabling forward transfer without explicit rehearsal. 1. Introduction Large-scale pre-trained Vision-Language Models (e.g., CLIP, ALIGN, Flava) have become foundational backbones for multimodal understanding. However, real-world deployment requires these models to adapt continuously to new tasks—new visual domains, novel object categories, or unseen captioning styles—without forgetting previously learned knowledge. This setting, known as Continual Learning (CL), is particularly challenging for VLMs due to the intertwined nature of their dual encoders.

[5] Zhang, Y., et al. (2024). VLM-CL: A benchmark for continual learning in vision-language models. NeurIPS Datasets Track. auto seed vl2

[6] von Oswald, J., et al. (2020). Continual learning with hypernetworks. ICLR. This paper is written in a standard academic

| Configuration | Avg Acc | Drop | |----------------------------------------|---------|------| | Full Auto-Seed VL2 | 82.2 | — | | w/o consistency loss (( \mathcalL \textconsist )) | 75.4 | -6.8 | | w/o gradient-conditioned generation (random seeds) | 68.9 | -13.3 | | w/o meta-update of ( G \phi ) | 74.1 | -8.1 | | w/o seed pruning (full memory) | 82.0 | -0.2 (ns) | We introduce Auto-Seed VL2 , a novel framework

During continual learning, the model is trained sequentially on each task. After learning ( \mathcalT t ), the model should perform well on all seen tasks ( \mathcalT 1:t ) without access to previous data. We allow a small episodic memory ( M ) (size ( K )) that stores generated seeds , not real examples.

[7] Khattak, M. U., et al. (2023). MaPLe: Multi-modal prompt learning. CVPR.