Machine Learning & AI

sentence-transformers

Teaches software that reset my password and I forgot my login basically mean the same thing.

Install it: pip install sentence-transformers

What does it do?

Sentence-transformers converts sentences and paragraphs into long lists of numbers, called embeddings, positioned so similar meanings end up near each other mathematically, even if the actual words are completely different. That’s what lets software match how do I get a refund with a customer’s typed question can I get my money back, despite sharing almost no words. It’s the engine behind semantic search and behind sorting large piles of text by meaning rather than exact keywords. Built on the same transformer technology as models like BERT, it was specifically tuned to make whole sentences comparable, not just individual words.

See it in action

This code turns two differently worded sentences into number patterns and measures how similar their meanings are, even though they share almost no words.

from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer("all-MiniLM-L6-v2")
sentences = ["How do I get a refund?", "Can I get my money back?"]
embeddings = model.encode(sentences)
similarity = util.cos_sim(embeddings[0], embeddings[1])
print(similarity)

Why would a non-developer care?

Keyword search, the old way computers matched text, fails constantly in real life since people phrase the same question a dozen different ways. Anything built on this kind of technology, from customer support search bars to AI assistants retrieving relevant documents, feels noticeably smarter than the keyword search of a decade ago.

Real-world examples

It’s a core piece of the retrieval-augmented generation setups that let chatbots search a company’s own documents to answer questions accurately instead of guessing. Search bars on many modern help centers and knowledge bases now use semantic embeddings like these instead of, or alongside, plain keyword matching.

Who uses it

Engineers building semantic search, chatbots that reference company documents, or recommendation systems based on text similarity.

How it compares to alternatives

It builds on the same underlying transformer models as Hugging Face’s Transformers library but specializes in producing high-quality sentence and paragraph embeddings, a task general transformer models handle less efficiently on their own. OpenAI and other providers also offer their own embedding APIs as a paid, hosted alternative.

Fun fact

It’s commonly known as SBERT, short for Sentence-BERT, created by researcher Nils Reimers specifically because the original BERT model was surprisingly bad at comparing whole sentences, despite excelling at other language tasks.

New to Python and want to actually try libraries like this yourself?

Find a beginner-friendly course

Related libraries