Alignment and Uniformity Losses
or: Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere
Good embeddings belong on a hypersphere surface. The authors offer Alignment and Uniformity losses, which give some of the same benefits as contrastive losses, but converge faster.