All analytics are built on a foundation of data, and retail analytics are no exception. Retail analytics began in the 1990’s with beam counters that generated data about how many people entered and left a store. Beam counters were flawed, and it wasn’t long before retailers began demanding more accuracy than beam counters could provide, and the industry moved to overhead counters. These were mainly video-based, but there was also thermal infrared radiation (IR), time of flight, Wi-Fi and others. The general idea was to put a device over the door that could automatically and accurately provide a running count of shopper traffic. Traffic counts became an indispensable data point for many retailers and, relatedly, conversion became the metric of choice.
While RetailNext was creating and deploying traffic counting solutions, a parallel technology direction was also being developed at the company: full path analysis. What if, in addition to tracking customers as they walked in and providing counts to retailers, technology could track those customers from the moment they entered the store to the moment they left and provide retailers with even more data about shopper movement than just whether they showed up?
Full path analysis (FPA) was initially built using standard IP cameras positioned overhead. Tracking was done in the view of each camera and then shopper location was “stitched” across cameras as shoppers moved throughout the store. Eventually, when the Aurora stereoscopic camera became available, FPA moved to using this more accurate stereo-based tracking solution. The net result is that today, RetailNext can generate incredibly accurate location data about shopper movements.
In-store data roadmap
As advances in computer vision continue to accelerate, we can expect this kind of full store tracking to become more accurate, less expensive and more ubiquitous. Whether tracking is performed on an edge device (as in the case of the Aurora or similar devices from Xovis and others), on an in-store appliance operating on surveillance footage or in the cloud, the result is the same: data about shopper movement is increasingly available and practical for retailers to collect at scale using computer vision.
If location data is about to be ubiquitous, what’s next?
To start, let’s consider a human analogy. Once we’ve processed what our eyes take in (vision), we begin to think about the results. Imagine our eyes, along with the visual cortex portion of our brains, as sensors that produce data about what’s happening in the physical world. Similarly, systems consisting of video cameras and computers running vision algorithms can also be thought of as sensors that produce data from a visual representation of the world. In humans, that data flows into our frontal cortex, which performs higher reasoning functions that ask “So what?” to the data provided by visual cortex. Given the sensory input of a basketball moving through the air toward him, a basketball player might reason about his position on the court and the positions of other players (also gathered by sensory inputs) and conclude that he should catch the ball and take a shot.
Higher reasoning based on raw sensor data is something that’s only now beginning to be developed in artificial intelligence (AI), because there is finally enough data available to fuel it. To understand the current cutting edge, consider the following example in the sports world.
In 2013, the National Basketball Association started tracking every player in every game using six cameras over every professional court. All this data by itself wasn’t valuable until two USC professors started Second Spectrum – they began by finding every “pick and roll” – the most common play in basketball – and have since moved on to finding and reasoning about every relevant play during the flow of the game. Their work contributes to enhancing broadcasts with overlays that describe the action on the court and auto-generating highlight films. All of this is done automatically via sophisticated AI, and points to the value that can be unlocked when higher level reasoning is applied to raw sensor data. What is the “pick and roll” equivalent in retail? And what comes after that? It seems likely that developments in basketball and other sports analytics can point the way for retailers looking to develop a similar “playbook” for managing their stores.
Potential applications of higher reasoning to retail are plentiful. Given locations of shoppers and store associates, for example, machine learning might be used to alert a nearby sales associate when it discerns a shopper is in need of help. Given thousands of examples of “normal” movement patterns through a physical store, similar machine learning might be able to pick out “abnormal” behaviors and patterns, identifying shoppers who are acting suspiciously. With facial recognition, tracking, and emotion analysis (all derived from computer vision), we can easily imagine coaching store staff in real time as to who has walked into the store and for what purpose, when and how to approach this person, and then providing headquarters with metrics about the effectiveness of such interactions.
So, if I have brick and mortar stores, why should I care about computer vision? And what should I be thinking about five years from now?
In retail, there is a growing stockpile of raw data. More sensors are being added to brick-and-mortar stores every day, from purpose-built stereoscopic tracking cameras to RFID, ultra-wideband (UWB), iBeacon and others. Existing surveillance cameras are also beginning to be utilized more readily to generate new data streams about where shoppers, associates and products are, and what they’re doing in the complex dance that takes place in a store. This is similar to the dataset that existed in the NBA in 2013. It’s now up to the technology community to build on this dataset, standing on the shoulders of giants to revolutionize the way stores operate.
- Ensuring Your Retail Analytics Platform Has the Punch to Move the Needle for Your Business Blog
- When It Comes to Shopper Traffic, Accuracy is Everything Blog