Semantic search
Semantic search is a retrieval method that matches on meaning rather than exact words. It converts the query and each product record into vectors, numeric representations of meaning, and returns the records whose vectors sit closest to the query's. A shopper searching "bolt that won't rust outdoors" can surface a 316 stainless hex cap screw even though the listing never uses the words "rust" or "outdoors."
How semantic search differs from keyword search
Keyword search is a string-matching problem. The engine indexes the tokens in your product record and returns records containing the query's tokens, weighted by frequency and field. If the shopper types "grade 8 bolt" and your title says "SAE J429 Grade 8 hex cap screw," you match. If they type "high strength bolt," you don't.
Semantic search is a distance problem. An embedding model turns text into a vector of several hundred to a few thousand numbers. Similar meanings land near each other in that space. The engine embeds the query, finds the nearest product vectors, and ranks by proximity.
The practical difference shows up on queries a buyer actually types:
| Query | Keyword search returns | Semantic search returns |
|---|---|---|
| "grade 8 hex bolt" | Exact matches on "grade 8" | Same, plus SAE J429 Grade 8 items |
| "bolt that won't rust outdoors" | Little or nothing | 316 and 304 stainless fasteners |
| "wire nut for 12 gauge" | Items with "wire nut" in the title | UL listed 600V connectors rated 18–10 AWG |
| "replacement for stock number 5RH72" | Only the exact stock-number record | Cross-referenced equivalents, only if a cross-reference field exists |
Hybrid retrieval is the common pattern: keyword (usually BM25) for precision on part numbers and exact specs, vectors for intent and paraphrase, then one ranking pass over the merged set. Semantic search does not replace keyword search. It covers the queries keyword search was never able to answer.
Why thin attribute data fails vector retrieval
An embedding model can only encode what is in the text. It does not know your product. It has no access to the spec sheet you never transcribed.
Take two records for the same physical part:
| Field | Thin record | Complete record |
|---|---|---|
| Title | HEX BOLT 3/8 X 2 G8 | 3/8-16 x 2 in. Grade 8 Hex Cap Screw, Yellow Zinc |
| Material | (blank) | Medium carbon alloy steel |
| Finish | (blank) | Yellow zinc chromate |
| Thread pitch | (blank) | 16 TPI, UNC |
| Tensile strength | (blank) | 150,000 psi |
| Drive type | (blank) | External hex |
| Description | (blank) | 40 words on application and torque |
The thin record embeds from about six tokens, half of them abbreviations. Its vector sits in a vague neighborhood near every other bolt-ish string in the catalog. A query about "high tensile fastener for a trailer hitch" will not land near it, because nothing in the record encodes tensile, alloy, or application.
The complete record embeds into a dense, specific region. It gets retrieved for tensile queries, for thread-pitch queries, for finish queries, and for the plain-English versions of all three.
This is the failure mode nobody sees in a search demo. The demo runs on twenty hand-picked SKUs with good copy. Production runs on the whole catalog, where the long tail has a title, a price, and nothing else. Swapping in a better embedding model does not fix it. The model is fine. The input is empty.
The same mechanism governs AI discovery. Retrieval-augmented generation, the machinery behind assistants that recommend products, embeds the shopper's question, retrieves candidate records, and writes an answer grounded in what came back. It reads the same vectors your site search reads. A record that cannot be retrieved on-site because it encodes no tensile strength also cannot be retrieved by an external assistant, and a record outside the context window cannot be recommended, cited, or compared.
What breaks it, what fixes it
The order matters. Many teams do this backwards. They buy the search platform first, then spend a year wondering why relevance is flat. The engine is rarely the constraint. The text it has to embed is.
| What breaks retrieval | What fixes it |
|---|---|
| Reading one catalog-wide fill rate | A healthy catalog-wide fill rate hides the categories that are empty. Measure per category. |
| Generating values before deciding the schema | Fix the attribute list for hex cap screws first: thread pitch, grade, finish, drive type, length |
| Guessed or inferred values | Complete from manufacturer spec sheets, PDFs, GDSN feeds. A wrong 600V rating is worse than a blank one. |
| Pipe-delimited spec dumps as description | Prose that carries the attributes in sentences, which is what embeddings read best |
| "3/8-16", "3/8 in-16", and "0.375-16" living as three values | One normalized value, so one thread does not occupy three neighborhoods in vector space |
| Tuning the ranking config first | Tuning it last, once the records have something to say |
Your PIM stores all of this. It does not generate any of it.
Frequently asked questions
Is semantic search better than keyword search?
Neither is better; they fail differently. Keyword search is exact and fast, and it is what you want when a buyer pastes a distributor stock number like 5RH72 or a GTIN. It returns nothing useful for a described need. Semantic search handles paraphrase and intent but can drift on precise identifiers. Most production catalogs run both and blend the results in a single ranking pass.
Do I need a vector database for semantic search?
Not necessarily. Many general-purpose search engines and databases now expose vector fields alongside keyword indexes. A dedicated vector database is a scale and latency decision, not a prerequisite. The harder question is almost never infrastructure. It is whether your product records contain enough text to embed meaningfully in the first place.
Why is our semantic search returning bad results even with a good model?
Usually the records are too thin to embed. If a SKU's text is an abbreviated nine-character title with blank material, finish, and rating fields, its vector encodes almost nothing distinguishing. It cannot be retrieved for attribute-driven queries because the attribute is not in the text. Check fill rate on the categories where search is worst before touching the model or the ranking config.
How does semantic search relate to faceted search?
They solve different halves of discovery. Semantic search gets the shopper into roughly the right set from an open-ended query. Facets let them narrow it: thread pitch, grade, finish, length. Facets require clean structured attribute values; semantic retrieval requires descriptive text. Complete product data feeds both, which is why teams that fix attributes see gains in both surfaces at once.
Does semantic search help with AI assistants and answer engines?
It is the same underlying mechanism. Retrieval-augmented generation embeds the shopper's question, pulls the nearest product records, and grounds its answer in them. A record too thin to be retrieved by your own site search is also too thin to be retrieved by an external assistant. Attribute completeness is the shared prerequisite for both surfaces.