Instacart Powers Smart Cart and Drives Sales Lift With NVIDIA Jetson and Physical AI

Instacart’s answer is Caper Cart—a smart shopping cart equipped with a digital touchscreen, five cameras, a Weights and Measures-certified scale, location sensors, and an NVIDIA Jetson Orin NX module. Every cart is a mobile AI platform, processing multiple sensor streams simultaneously to recognize items as customers drop them into the basket, track quantities, detect removals, and deliver personalized recommendations—all in hundreds of milliseconds.

The architecture runs two systems in parallel. On the edge, the NVIDIA Jetson board fuses visual data from three basket-facing camera angles with weight data and location inputs to build a real-time picture of basket contents. Two additional outward-facing cameras face the store shelves. Weight serves as an X-ray of the basket: Even when cameras are blocked or items are stacked, the scale captures what vision alone cannot. In the cloud, vision-language model encoders run asynchronously for deeper context, and the two feeds combine to interpret user actions, item identity, and shelf context.

Physical AI dynamically perceives and responds to environments in real time. Unlike camera-only or barcode-scanning systems that rely on a single signal, Instacart’s approach fuses multiple sensor modalities directly on hardware embedded in the store environment. It’s what allows the system to handle the messiness of real retail: blocked cameras, stacked items, noisy weight signals from carts bumping along the floor, and tens of thousands of SKUs that shift from store to store.

Side-facing cameras on each cart also solve the location problem. A single model serves all stores with minimal store-specific retraining, which is critical for scaling across a 100,000-store footprint. On the cloud side, NVIDIA GPUs and NVIDIA Dynamo optimize ad-ranking and recommendation workloads. The result: millions of sensor inputs collected daily, feeding a data flywheel that improves with every trip.

The gap between where retailers started—navigating their own stores without real-time insight—and where Caper Cart has taken them is measurable. The AI-powered smart cart is driven by personalized, location-aware recommendations that reach customers at the moment of decision.

By incorporating ranking signals from Instacart’s online platform—drawn from 1.6 billion lifetime grocery orders—into in-store recommendations, Instacart measured more than 1% incremental sales lift in rigorous A/B testing. A pre-checkout “got everything you need?” prompt, which combines purchase history, real-time cart location, and live shelf-mapping data to remind customers of forgotten items, drove nearly 1% additional lift. On the cloud inference side, migrating ad and recommendation workloads from CPU to GPU cut whole-page ranking latency by 65% and item ranking latency by 40%, while click-through rates—a direct proxy for revenue—increased by more than 5%.

The same sensor fusion system also tackles shrink, catching item-swapping, stacking, and multi-item discrepancies that camera-only or barcode-scanning approaches miss. Meanwhile, side-facing cameras continuously build a near real-time map of each store—tracking shelf conditions, item locations, and out-of-stock situations—giving retailers operational visibility they have never had before.

Adoption is accelerating. Caper Carts are now live in more than 100 cities—across Kroger, Wegmans, Coles, Schnucks, and Wakefern—with deployment tripling year over year. What started as a convenience play has become a top-line growth driver, and retailers are scaling accordingly.

Instacart’s ambitions extend well beyond smart carts. The company is building what it calls a grocery world model—a foundational AI system that combines its commerce graph (more than 1.6 billion lifetime orders, over 2 billion product instances refined into more than 22 million unique items) with real-time sensor data from Caper Carts and Instacart shoppers scanning shelves. The goal: a continuously learning system that understands how products relate to each other, how customers make decisions, and how a store operates physically and commercially.

Instacart is already extending this intelligence into agentic workflows. An AI-powered operations assistant lets store managers query shelf conditions in natural language—surfacing out-of-stock patterns, flagging underperforming displays, and moving toward automated merchant coordination. And as the sensor network grows richer—capturing item weight, 3D representations, and precise shelf location—Instacart sees potential to unlock robotics applications in the store, providing the spatial and product intelligence that autonomous systems need to operate.

By fusing edge AI in the physical store with cloud-scale intelligence online, Instacart and NVIDIA are proving that physical AI isn’t a future concept—it’s already reshaping how grocery retail operates, one cart at a time.