Half of the Largest Distributor Catalogs Refuse a Plain Web Request. We Only Found Out Because We Tried to Read Them.

Part of Top Distributors 2026 — the Anglera Index: North America's largest distributors ranked, classified into six operating archetypes, and scored on the measured Digital Readiness Index.

We have argued before that agent-mediated buying is a parsing problem: an AI agent issues a query against structured data, filters on attributes, and selects the first candidate that satisfies the spec. Either your SKU survives the parse or it doesn't.

That argument contains an assumption we never examined. It assumes the agent gets the page.

This month we rebuilt the retrieval side of our own measurement pipeline, because the previous one was failing quietly. Nineteen of seventy-three storefronts returned 403 to it. Thirty-seven never reached a search results page. We had been publishing those as unknowns, which was generous to us and unfair to everyone else: a column that reads "unknown" because we never successfully asked is not a measurement, it is a filing error.

Fixing it meant escalating through four tiers of retrieval — a plain HTTP client, commercial proxy egress, a headless browser that executes JavaScript, and a second unblocking vendor for the sites that defeat the first one. Along the way we accidentally produced a more interesting number than the one we set out to collect.

The number

Of 73 distributor storefronts, 37 let a plain HTTP client reach both a homepage and a product page. The other 36 did not.

What it took to reach a product page	Storefronts
A plain HTTP client	37
Commercial proxy egress	15
A headless browser	6
A second unblocking vendor	4
No product page reachable at all	11

Four sites required a second commercial unblocking vendor because the first one was itself blocked. On two of them — Wesco and MSC Industrial — our proxy provider tripped its own circuit breaker after consecutive failures before another vendor's residential egress walked in.

Now hold that against what these same companies publish about themselves. Nine of the twenty-six that publish an XML sitemap refuse a plain client. A sitemap exists for one purpose: to tell an automated visitor where the products are. Publishing one and then blocking the visitor is not a security posture, it is a contradiction.

What actually stops a machine

The aggregate hides the interesting part. These are not sites that are merely slow or badly built. Each of the following is a specific, identifiable decision:

Hillman's product tiles are not links. Every one renders as href="javascript:void(0)" and navigates in JavaScript. A crawler following links through that catalog finds zero products. The company's own sitemaps carry roughly 64,590 product URLs. The inventory is published and the path to it is severed.

CED has no national storefront. Not a blocked one — none. The root domain refuses every client we have. The internal portal sits behind Microsoft SSO. The actual catalog is a Salesforce B2B storefront replicated per profit center across 700-plus separate subdomains, sharing one product master with no canonical entry point. No crawler, and no agent, arrives at that by inference.

Henry Schein runs a bot wall in front of pages that are entirely public. Login gates pricing, not the pages. The product pages carry structured data. The wall's only measurable effect is to stop machines from reading what any person can read freely.

McNaughton-McKay serves an empty shell. Its category grids render client-side, so the delivered HTML says "loading content." The same data is available from the storefront's own public API — which we found only because we were watching network traffic, which an agent is not.

Border States' top-level categories contain no products. They list subcategories. A crawler that samples top-level categories concludes the catalog is empty.

Optimas's webstore is dead. Its NetSuite storefront times out from a plain client, from our proxy vendor, and from the unblocker alike, while Google still serves stale index entries pointing at it. Three independent providers failing identically is what distinguishes a dead host from a blocked one.

And one that is a legitimate choice rather than an accident: Vallen is deliberately gated. Its own JavaScript bundle sets guest browsing to false and the route guard redirects anonymous visitors to a login page. Roughly 488,000 SKUs, none publicly reachable. That is a strategy, not a bug — but it is a strategy with a consequence, and the consequence is that agents cannot see the catalog at all.

Two traps for anyone measuring this

Two findings are worth passing on to anyone attempting the same exercise, because both silently corrupt results.

Kimball Midwest's mid-level category pages render only a shared thirteen-SKU "popular bought" carousel, byte-identical across Abrasives, Paint and Electrical. A crawler sampling three categories collects the same promoted SKUs three times and believes it has category spread. Anything measured from that sample describes the company's merchandising, not its catalog.

Medline's product pages are family pages — one title, one description, one specification block, over a table of many orderable item numbers. Scoring that against a rubric built around one page per purchasable SKU produces a number that means something different from every other number in the set.

Why we are not folding this into the score

The Digital Readiness Index already has a Machine and Agent Readiness pillar. Every signal in it is a declaration: structured data present, sitemap present, robots.txt stance toward AI crawlers. Those measure what a site says.

None of them measure what a site does to a machine that shows up. A company can publish flawless Product JSON-LD behind a wall that refuses the crawler which would have read it — and one of the companies here does exactly that.

So this ships as its own axis, reported next to the DRI rather than inside it. Folding it in would move published scores for reasons unrelated to anything the companies changed, and the index's entire claim is that the same site scores the same number twice. Accessibility is measured, published, and kept separate.

There is a directional relationship worth noting carefully. Median DRI runs 59 for storefronts a plain client can read, 58 where a proxy was needed, 56 where a browser was needed, and 43 for the four that needed a second unblocking vendor. The measured subsets are small — 21, 7, 6 and 3 companies — so treat the top of that range as flat and only the bottom as suggestive. What it hints at is that hostile retrieval and thin product data tend to travel together, which is what you would expect if both come from the same underinvestment.

What this is and is not

Difficulty here is measured against our ladder, not against any particular AI agent. It is a proxy, and we would rather say so than imply we tested a specific model's browsing behaviour.

What it does establish is a floor. This pipeline had commercial proxy egress, headless rendering, network-level request capture, XML sitemap parsing, and human reviewers who could work out that Hillman's real catalog lives on a different subdomain from its marketing site and that DESCOURS & CABAUD's North American flagship trades as Dillon Supply. It still needed all of it.

An AI shopping agent has a fetch tool.

Every barrier above sits upstream of the parsing problem. Attribute coverage, GTIN presence, schema markup — all of it is downstream of a request that has to succeed first. Half of these catalogs answer that request. The other half answer it only for someone willing to buy infrastructure to ask.

If you are wondering which half you are in, the test costs nothing: fetch your own product page with curl, no browser, no headers beyond a user agent, and see what comes back. Then fetch your sitemap and try to follow three links out of it. That is approximately the experience your catalog offers a machine — and increasingly, the machine is the first thing that reads it.

Methodology, per-company grades and the full barrier list are published with the Top Distributors 2026 index. Accessibility was measured in August 2026; sites change, and the next quarterly pass re-measures it.