AI & Computer Science · Weekly AI Papers
CLIP: Learning Transferable Visual Models From Natural Language Supervision (Radford et al.)
LISTEN TO THIS EPISODE
CLIP asks whether natural-language descriptions can teach a visual model more flexibly than a fixed set of class labels.
Radford and colleagues train image and text encoders on 400 million image-text pairs using a contrastive objective.
Text prompts then describe new categories, allowing the model to attempt classification without task-specific labeled training examples.
The paper evaluates transfer across more than 30 vision datasets and examines robustness, prompting, and the limits of the approach.
This foundational study provides a historical counterpart to the JEPA paper: paired contrastive training offers one route to shared visual and linguistic representations.
Part of Marginalia: books, papers, and ideas for learning and debate.