Efficient Word Vectors for Large Datasets

This 2013 academic paper introduces two new model architectures, Continuous Bag-of-Words (CBOW) and Skip-gram, designed for efficiently computing continuous vector representations of words from vast datasets. The authors compare the quality and computational cost of these new models against existing neural network language models, demonstrating significant improvements in accuracy at a lower computational expense. A key focus is on preserving linear regularities between words, enabling the vectors to capture complex syntactic and semantic relationships that can be revealed through algebraic operations. The research highlights the scalability of these methods for large-scale parallel training, suggesting their potential to advance various Natural Language Processing (NLP) applications.

Source: https://arxiv.org/pdf/1301.3781