The Hidden Number: How Grepsr Cracked Real-Time Stock Data for a Major Retail Network 

Some websites don’t hide their data behind a locked door; they hide it behind a maze. 

A major retailer’s stock levels were never listed anywhere on its site; the only way to find the real number was to keep adding items to a cart until it broke. 

This is how Grepsr turned that breaking point into a daily, store-by-store intelligence feed for a global consulting firm – at a scale no manual process could match.

About the Client

Our client is a leading global management consulting firm, engaged to advise on inventory and pricing strategy for a major national hardware & home improvement retailer. 

The retailer operates a large network of physical stores spanning multiple states, and understanding real-time stock behavior across that network was central to the engagement.

Their Requirements

The client needed daily, store-level visibility into two things: 

Exact stock quantities and pricing, across a defined list of products and a network of 200+ active stores. 

The purpose was straightforward: monitor stock depletion, identify movement trends, and track local pricing disparities between locations. 

What wasn’t straightforward was getting the data at all.

The Challenges

1. No visible stock data. The retailer’s website doesn’t expose real stock numbers anywhere in its public user-facing pages; the product pages show vague messaging like “in stock” or “usually ships in 5-9 days,” with no public API returning exact quantities.

2. The only real number is behavioral. The one way to find true stock is to test it directly: add the product to the cart, increase the quantity, and find the exact point where the site stops accepting the order. That breakpoint, which is not displayed on the product details page, is the real figure.

3. Manual workarounds don’t scale. The client’s own team had already discovered this and was working around it with the help of ChatGPT by doing batch runs in: loading a page, selecting stores from multiple addresses, and adding a product to the cart. 

Then testing a quantity, catching the quantity and messaging the quantity selection shows, catching any other site messages, clearing the cart, and repeating. It worked as a proof of concept, but doing it by hand across dozens of products and hundreds of stores, every day, wasn’t sustainable.

4. Naive automation breaks the site instead. A straightforward automated version of the same process didn’t solve the underlying problem either. 

The retailer’s cart accepts inputs up to 10,000 units, so testing quantities sequentially – 1, 2, 3, and up could require up to 10,000 requests per product just to find where it breaks. At the required scope, that volume of sequential requests would trigger server bans long before any data was delivered.

Solutions Conjured by Grepsr

Grepsr’s engineering team rebuilt the approach from the ground up around three core optimizations:

1. Binary search instead of brute force. 

Rather than counting up one unit at a time, the crawler tests the midpoint of the possible range and halves the search space based on whether that quantity succeeds or fails. 

The same logic behind any efficient search algorithm was applied to live cart behavior. A product that could theoretically require 10,000 sequential checks is now resolved in roughly 15 to 17 requests: a 99.8% reduction in server load per product, and the change that made daily automation possible at all.

2. Direct store-ID mapping. 

Instead of navigating the site’s UI store-by-store to select a location, the team pulled the retailer’s backend store directory in a single API call, matched it against the target addresses, and injected store IDs directly into each request, eliminating slow, failure-prone UI navigation entirely.

3. Multiprocessing at scale. 

With per-product requests optimized, the remaining bottleneck was volume: well over 200,000 requests needed each day across the full store and product matrix. 

Running that sequentially would have taken 200+ hours, which is unusable for a daily deliverable. A multiprocess architecture parallelized the crawl, bringing runtime down to roughly 7 hours, scheduled overnight so fresh data lands before the start of business hours.

Eventually, we delivered the result for every product, at every store, every day was an exact maximum available quantity, the site’s displayed stock status, any bulk-purchase increment rules, and error/remarks logging for full transparency on data quality.

Final Impact

The system ran daily for two consecutive weeks without missing a delivery window, adapting along the way, reactivating stores that temporarily dropped from the active list, and adjusting scope as store counts shifted.

“Regarding future work, we will be back because the work you have done was very good.”

– Engagement Manager, Client’s Consulting Team

Beyond the numbers, the engagement demonstrated something the client’s own team had already brushed up against: AI-assisted manual workflows are genuinely useful for figuring out a problem once, but they scale with the person running them, not with the problem itself.

A managed service doesn’t make that manual process faster; it replaces it entirely with an engineering team that runs crawlers at a volume and consistency no manual workflow can sustain. 

But automation alone isn’t the full story. 

Every other dataset passed through automated and customized QA rules/logics along with human monitoring before every other day’s delivery. Thus, catching edge cases like a product listing that 404s or a store that temporarily drops offline that a purely AI-automated pipeline would either miss or silently misreport. 

And the client never had to manage any of it directly: a dedicated Customer Success Manager owned the engagement end-to-end. This includes monitoring runs, delivering in the client’s data pipeline, flagging anomalies, and resolving issues before they ever became the client’s problem to notice, let alone solve. 

The binary-search framework built for this project is now a reusable template for any high-cart-limit e-commerce site where stock data isn’t otherwise accessible.

This way, we are turning a one-off engagement into a standing capability for future retail and consulting clients. 

Stop guessing what’s on the shelf.

If your data is stuck behind a login, a cart, or a broken API, chances are we’ve run into it before. Let’s figure out your data challenge and solve it together.

Talk to our team →

Data to make or break your business

Get high-priority web data for your business, when you want it.

Get started