[Avg. reading time: 8 minutes]
Embeddings
Embeddings transform high-dimensional categorical or textual data into a compact, dense vector space.
Similar items are placed closer together in vector space -> models can understand similarity.
- These representations capture relationships and context among different entities.
- Used in Recommendation Systems, NLP, Image Search and more.
- Can be learning from data using neural networks or retrieved from pretrained models (eg: Word2Vec, FastText)
Use Cases
- Search & Retrieval: Semantic search, image search.
- NLP: Word/sentence embeddings for sentiment, chatbots, translation.
- Computer Vision: Image embeddings for similarity or classification.
Advantages over traditional encoding:
- Handle high-cardinality categorical features (e.g., millions of products).
- Capture context and semantics (“Laptop” is closer to “Computer” than “Pencil”).
- Lower-dimensional → more efficient than One-Hot or TF-IDF.
Types of Embeddings
Word Embeddings (Text)
Represent words as vectors so that semantically similar words are close together.
Examples: Word2Vec, GloVe, FastText.
“king” – “man” + “woman” = “queen”
Used in: sentiment analysis, translation, chatbots.
Sentence / Document Embeddings (Text)
Represent longer text (sentences, paragraphs, docs) in vector form.
Capture context and meaning beyond individual words.
Examples: BERT, Sentence-BERT, Universal Sentence Encoder.
“The laptop is fast” and “This computer is quick” → close vectors.
Image Embeddings (Computer Vision)
Represent images as vectors extracted from CNNs or Vision Transformers.
Capture visual similarity (shapes, colors, objects).
Examples: ResNet, CLIP (image+text).
A cheetah photo and a leopard photo → embeddings close together (both cat family).
Used in: image search, face recognition, object detection.
Audio / Speech Embeddings
Convert audio waveforms into dense vectors capturing phonetics and semantics.
Examples: wav2vec, HuBERT.
Voice saying “Laptop” → embedding close to text embedding of “Laptop”.
Used in: speech recognition, speaker identification.
Graph Embeddings
Represent nodes/edges in a graph (social networks, knowledge graphs).
Capture relationships and network structure.
Examples: Node2Vec, DeepWalk, Graph Neural Networks (GNNs).
In a product graph, Laptop node embedding will be close to Mouse if often co-purchased.
| Type | Example Algorithms | Data Type | Use Cases |
|---|---|---|---|
| Word | Word2Vec, GloVe | Text (words) | NLP basics |
| Sentence/Doc | BERT, SBERT | Text (longer) | Semantic search, QA |
| Categorical | Embedding layers | Tabular (IDs) | Recommenders, fraud detection |
| Image | ResNet, CLIP | Vision | Image search, recognition |
| Audio | wav2vec, HuBERT | Audio | Speech-to-text, voice auth |
| Graph | Node2Vec, GNNs | Graphs | Social networks, KG search |
#embeddings [#<abbr title="Bidirectional Encoder Representations from Transformers">BERT</abbr>](../tags.md#BERT "Tag: BERT") #Word2Vec #NLP