RSD-008.11 : Field note
How accurate are shelf-scanning robots? What the published numbers mean, where the robots miss, and how to test one
Ask a shelf-scanning robot vendor how accurate the robot is and you will get a number in the high nineties. Ask a retailer that runs one and you will hear about lift: how many more empty shelves it finds than the staff did. Ask a computer vision researcher and the number drops into the eighties. None of them is wrong. They are measuring different things on different shelves. This note sorts the published figures by who measured them, explains where the robots predictably miss, and lays out how an advisor should help a retail client test accuracy in a pilot before anyone signs a chain-wide agreement. It follows our note on how shelf-scanning robots catch price tag errors.
Robotics SD : October 14, 2026
The short version
- Accuracy is several numbers, not one: whether the robot identified the right product and tag, how many empty facings it caught (recall), and how many of its alerts were right (precision). A vendor quoting a single figure is usually quoting its best one.
- Vendor-published figures cluster between 95 and 99%. Simbe reports 98.7% SKU identification and 99% shelf label identification, Badger more than 95% on out-of-stocks and 90% precision on pricing errors, Pensa 98% on out-of-stock detection, Zebra 95%, Trax 96%. Retailers speaking for themselves talk about lift instead: Schnucks reported 14 times more out-of-stock detections than manual audits.
- Peer-reviewed systems tested on other people's shelves score lower: 86.3% average precision on fully empty facings in a 2024 study across four retailers, an 89.6% success rate for on-shelf availability from a ceiling camera in a real store, 85.8% for a 2025 checker that works without planograms. The gap between those and the vendor figures is mostly tuning, planograms, and who chose the test set.
- The misses are predictable: aisles the robot cannot enter, crowded and partly hidden shelves, lookalike packaging, lighting, intentional gaps read as low stock, and fresh departments most robots do not scan at all. A pilot should be designed to find them, with a manual audit as ground truth and false alarms counted per aisle.
Field note
What accuracy means on a shelf
A shelf-scanning robot does at least three jobs on every facing, and each has its own error rate. It identifies the product, reading the package and the shelf tag. It judges the condition of the facing: stocked, low, empty, or holding the wrong item. And it decides whether to raise an alert. Identification accuracy is the share of products and tags it read correctly. Recall is the share of real problems it caught. Precision is the share of its alerts that were right. A robot can have high recall and low precision, which means it finds nearly everything and also sends staff to check shelves that were fine.
The vendors know this, and the careful ones publish the metrics separately. When Simbe announced its expanded computer vision platform in May 2025, it stated 98.7% SKU-level identification accuracy, shelf condition recall above 99.3%, and shelf condition precision above 99%, and it described what it called typical computer vision systems as averaging 85 to 90% accuracy and 75 to 85% recall. Trax, which sells image recognition to consumer goods brands and retailers, explains precision on its own blog with a worked example in which 96% of recognitions are correct. Those are the vendors' numbers, measured by the vendors. The point here is the vocabulary.
Two of the jobs are easier than the third. Empty-facing detection does not require knowing which product is missing, because an empty slot looks empty; Roboflow, a computer vision tooling company, makes exactly that point in its shelf-monitoring material. Reading a price tag is bounded by the tag: a printed price and a barcode, compared with the retailer's price file. Product identification is the hard one, because thousands of packages differ by a flavor word or a color band, and that is where most of the published accuracy gaps live.
Where the bar sits depends on who is asked. Luis Vera, the chief executive of Zippedi, told Supermarket News in 2021 that the technology of a few years earlier had not been mature enough to reach the 90%-plus accuracy an out-of-stock solution requires. Researchers tend to report average precision or F1 scores rather than a single accuracy figure, which is one reason their numbers look lower at first glance.
Field note
What the vendors publish, and what retailers say for themselves
The published vendor figures sit in a narrow band. Simbe's Tally product page states 99% shelf label identification accuracy and ten times more out-of-stocks detected than manual audits. When it launched Tally 3.0 in 2020 it said recognition accuracy was almost 99% without slowing the robot down, and a Supermarket News report on the 2022 Wakefern pilot relayed Simbe's comparison of a 99% accurate thrice-daily audit against 65% for a weekly manual one. A Simbe case study with an unnamed grocer claims 98% precision against 85% for other mobile robots and 80% for fixed cameras in a head-to-head pilot.
Badger Technologies states in its own white paper that its robots identify out-of-stock items with over 95% accuracy and detect pricing errors with 90% precision. Its 2023 press release on McCoy's Building Supply repeats the 95% figure and says McCoy's improved price tag accuracy by more than 97%, and its Woodman's Markets case study reports 95% on out-of-stocks and 90% on mispriced items. Zebra Technologies said in 2020 that its SmartSight system could provide 95% accuracy in identifying stock-outs, misplaced product, incorrect price labels, and planogram compliance. Trax states 96% accuracy on its product page. Pensa Systems, which flew drones in a Montreal IGA store with AB InBev in 2019, reported 98% out-of-stock detection accuracy over 15.9 million product images and 200 flights.
Two of the larger suppliers publish no detection rate at all. Brain Corp, whose shelf-scanning towers ride on Sam's Club floor scrubbers and whose ShelfOptix service launched in 2025, describes accurate shelf insights without a figure. Zippedi publishes outcomes instead: a 30% reduction in stock-outs, 90% fewer price errors, and a 10% gain in planogram adherence. An advisor should treat every number in this section as the supplier's own measurement, usually in the supplier's best category, often at a retailer it does not name.
Retailers speaking under their own names rarely state an accuracy rate. They state lift. Dave Steck, Schnucks' vice president of IT, told Supermarket News in 2020 that Tally provided 14 times more out-of-stock detection than manual auditing and at least a 20% reduction in out-of-stocks in the stores using it. Tyler Davis of Woodman's said in Badger's case study that the robot does the work every day and frees up 40 hours each month. Harmons' chief information officer told Grocery Dive it had reduced pricing errors by 75%, which we covered in the previous note. Lift is the honest retailer metric because the baseline is the store's own manual process, and that baseline is low.
How low is visible in the one retailer-audited comparison we found, published by Pensa in 2022 for a retailer it does not name. The retailer ran its legacy gap scan, then the Pensa scan, then audited both against the actual shelf. The legacy scans scored 35.6%, 52.3%, and 57.1% in the three stores; Pensa's scored 93.3%, 95.2%, and 93.9%, and the audit put its gap detection at 94% and overall accuracy at 99%. A robot that misses one facing in twenty still finds roughly twice what the manual scan found. That is the arithmetic behind the lift numbers.
Field note
What independent research finds
Peer-reviewed work measures the same tasks without a vendor choosing the test set, and the numbers are lower. A 2024 study in the journal Sensors by Šikić, Kalafatić, Subašić, and Lončarić trained detectors on 511 shelf images from four major Croatian retailers and reported 86.3% average precision on fully empty facings and 83.7% on facings where the front product was missing. The authors note that false positives come from gaps detected where none exist or localized too loosely, and that the size, lighting, and background of out-of-stocks vary widely.
Fixed cameras have a similar ceiling in the literature. Higa and Iwamoto, also in Sensors, measured on-shelf availability from a ceiling surveillance camera in a working store and reported an 89.6% success rate at a 10% error margin, with a method designed to be robust to shoppers standing in front of the shelf. A 2025 system called GondoCheck, presented at a Brazilian telecommunications symposium and built to audit shelves without planograms or store calibration, reported 85.8% global accuracy and a macro F1 of 0.77. A 2023 workshop paper from the University of Salerno tested an autonomous shelf-scanning robot in a supermarket during opening hours against about 2,000 manually annotated images and reported that it was eight times faster than a human operator; its abstract gives speed, not precision.
A 2025 benchmark from Chen and colleagues, built on real supermarket shelves, explains the gap. Mainstream detectors perform well in standard settings, the authors write, but their accuracy degrades under extreme product density and heavy occlusion, and methods that look for empty space are brittle under lighting changes. Their best general detector scored a mean average precision of 0.806 at the standard threshold on dense shelves.
The difference between 85 and 99 is not that one side is wrong. A commercial system runs on the client's own planogram and price file, learns the client's assortment over weeks, and reports on the categories it handles best. A research method runs on strangers' shelves with no planogram and reports on everything. The client's number will land somewhere between, and the only way to know where is to measure it in the client's stores.
Field note
Where the robots miss
The misses are not random. They cluster in a handful of situations that every pilot should include on purpose.
- Aisles the robot cannot enter. When a pallet or a cart blocks the way, Tally goes around it if it can; if it cannot, a Giant Eagle spokesperson explained in 2019, it turns around and moves on to the next aisle. That aisle is not scanned wrong. It is not scanned, and the report should say so.
- Crowded and partly hidden shelves. The 2025 benchmark found detector accuracy degrading under extreme product density and heavy occlusion. Deep shelves, overhanging promotional signage, and clip strips are the retail version.
- Lookalike packaging. The SKU-110K benchmark paper from 2019 put it plainly: sub-brands are often distinguishable only by fine-grained packaging differences. Identification errors land here, and so do wrong-item-in-slot alerts.
- Lighting and glare. Trax lists nearly identical products, partially obstructed products, and poor lighting among the hurdles its recognition has to overcome, and the Croatian study found lighting inconsistency a source of false positives. Glossy packaging under spot lighting is a known trouble spot.
- Intentional gaps and low stock. An Infosys engineering team working on retail shelf analytics with NVIDIA's models reported that the pre-trained model struggled to tell intentional display gaps from genuine low stock. Simbe says Tally uses depth perception to flag low stock before it becomes an out-of-stock, and Zippedi says it separates replenishment needs from true out-of-stocks; both are claims a pilot can check.
- Fresh departments. Simbe's chief executive, Brad Bogolea, told Fortune in 2021 that Tally did not scan fresh produce or baked goods, which he called a more difficult computer vision problem. If the client's pain is in produce, ask what the robot covers before the pilot, not after.
- The blind window between scans. A roaming robot sees a shelf two or three times a day; Focal Systems argues for its fixed cameras on the grounds that they capture the shelf every hour. An out-of-stock that opens and closes between scans is not a detection error, but it will be counted as one if the ground truth is taken at the wrong time.
There is also a quieter factor behind some of the published numbers: people. Badger has said it uses an outsourced annotation team to sharpen its detection and inventory models, and the research groups describe cross-checked labeling with arbitration and random audits. Some of what a client buys as accuracy is a model, and some of it is a review process. Neither is a problem. An advisor should simply know which one the supplier is quoting.
Field note
How to test accuracy in a pilot
No vendor or analyst we reviewed publishes a standard test protocol, and none publishes false alarms per aisle or time to detect. That leaves the design to the retailer and the advisor. The pieces are not complicated.
- Build a ground truth. Pensa's 2022 comparison is the model: run the existing manual gap scan, run the robot scan right behind it, and have store staff audit both against the actual shelf. The research groups go further, with two annotators per image, arbitration on disagreements, and a 10% random re-check. In a store, that is two people counting the same aisles on the same morning.
- Score three numbers, not one. Label match rate against the price file. Out-of-stock recall against the audit. Precision of alerts, which means counting the false alarms, because false alarms are what make staff stop working the list.
- Test repeatability. Georges Mirza wrote in Supermarket News in 2021 that for shelf robots, repeated accuracy is key. Scan the same aisles across a week and compare, rather than judging on a single good day.
- Define controllable before the pilot starts. Simbe reports that 42 to 43% of the out-of-stock instances its robots detect are controllable, meaning the store can fix them, but publishes no definition. Agree with the client and the supplier which alerts count.
- Include the hard categories on purpose. Put the deepest shelves, the promotional end caps, the lookalike sets, and a blocked aisle in the test, and write down what is excluded, such as fresh.
- Tie the pilot to the rollout decision. Simbe describes one grocer's path as four stores, then 35 more, evaluated on out-of-stock reduction, sales impact, replenishment execution, and store engagement, with a chain-wide recommendation due by the end of 2026. The pilot's numbers should be the ones the client's leadership will decide on.
Brain Corp wrote in July 2026 that many robotics prototypes impress in controlled pilots but fail in real-world deployments, and that what feels tailored at the start often becomes brittle at scale. A vendor saying that about its own industry is worth taking seriously. A pilot measured this way is how the client finds out before the rollout contract, not after.
Field note
What this means for the advisor
Accuracy is the first question a retail client will ask about a shelf-scanning robot, and the useful answer is not a number from a brochure. Three things follow.
- Translate the question. Ask which error costs the client the most: a wrong price on the tag, an empty facing nobody saw, or a wasted trip to a shelf that was fine. That decides whether label accuracy, recall, or precision is the number that matters, and which supplier's strength fits.
- Reframe the vendor claims as hypotheses. The figures in this note are what suppliers measured in their own conditions. The pilot measures the client's. Present them that way and the client stops asking which brochure to believe.
- Watch the format decision. Suppliers are converging. Simbe added fixed cameras to its roaming robots in 2025 and announced a Tally with up to twelve hours of operation and depth cameras for mid-2026, Brain Corp plans thousands of its ShelfOptix units in 2026, and Zippedi is re-entering the US. In the UK, Tesco is trialling the robots and Morrisons began the year before, to mixed public reviews. Fixed cameras, roaming robots, and scanners on existing scrubbers each miss in different places, and the pilot should test the format the store will actually run.
Register the opportunity when the client first names the accuracy question, before a supplier is engaged, and let our pre-sales engineers set up the pilot measurement with the supplier. The money follows the usual path: the supplier pays Robotics SD a commission on the revenue a registered deal produces, and Robotics SD pays the advisor from it. Shelf-scanning is usually sold per store, and a pilot that measures accuracy honestly is what turns a few stores into a chain.
FAQ
The questions that follow
How accurate are shelf-scanning robots?
Vendors publish figures between 95 and 99% for product and label identification and out-of-stock detection, retailers describe lift rather than accuracy (Schnucks reported 14 times more out-of-stock detections than manual audits), and peer-reviewed systems tested on other retailers' shelves score in the mid 80s. The honest answer for a given store is the number its own pilot measures against a manual audit, scored separately for label accuracy, out-of-stock recall, and the precision of alerts.
Are shelf-scanning robots more accurate than store employees?
On consistency, yes. In the one retailer-audited comparison we found, published by Pensa for an unnamed retailer, the store's manual gap scans scored between 35.6% and 57.1% while the robot scored between 93.3% and 95.2%. Robots also scan the whole store two or three times a day. They still miss in predictable places, including blocked aisles, crowded shelves, lookalike packaging, and fresh departments, and a person has to act on every alert.
What do shelf-scanning robots get wrong?
The published failure modes are aisles the robot cannot enter and therefore skips, dense or partly hidden shelves, sub-brands that differ only by small packaging details, poor lighting and glare, intentional display gaps misread as low stock, and fresh produce and bakery, which some robots do not scan at all. Out-of-stocks that open and close between scans are a blind window rather than an error, but they count against the robot if the audit is timed wrong.
How should a retailer test a shelf-scanning robot before buying?
Run a pilot with a manual audit as ground truth: scan, audit the same shelves right after, and score label match rate, out-of-stock recall, and the share of alerts that were correct. Count false alarms per aisle, repeat the same aisles across a week, include the hard categories on purpose, agree in advance on which alerts count as controllable, and tie the results to the measures leadership will use for the rollout decision. Our pre-sales engineers set this up with the supplier on registered deals.
Next step
You already know the client for this.
Start one conversation, register the deal, and Robotics SD engineers take it from there. Joining as a Robotics Advisor is free.