Normalized Word Embedding and Orthogonal Transform for Bilingual Word Translation

Normalizing word embeddings to lie on a hypersphere, instead of scattered throughout a euclidean space, is beneficial and still allows the usual linear concept algebra.

see also Alignment and Uniformity Losses