Contrastive Visual Pretraining

Contrastive visual pretraining makes it possible to learn strong visual representations without labels. Different augmented views of the same image are treated as positive pairs, while unrelated examples serve as negatives. This encourages the model to learn more persistent properties such as object identity or scene structure. It is one of the most powerful routes to representation learning in settings where annotation is expensive.