Smart Cities/Spaces
How Raleigh Enhances City Safety and Mobility with Physical AI
Objective
The City of Raleigh, North Carolina is a thriving innovation hub at the heart of the famed universities “research triangle” between Durham and Chapel Hill and ranks as a top 5 metro in the U.S. for its tech impact on the local economy. With its population more than doubling over the last three decades, Raleigh faces immense challenges in scaling its infrastructure efficiently and sustainably. In response, city leaders launched a pilot initiative to aggregate and analyze data with AI through a single interactive geospatial map to accelerate decision-making. Following the NVIDIA Blueprint for smart city AI, alongside Esri’s ArcGIS geospatial platform that powers the city’s digital twin on Microsoft Azure, Raleigh is transforming real-time urban management for improved mobility, safety, and operational efficiency.
Customer
City of Raleigh
Partner
Esri, Microsoft
Use Case
Computer Vision / Video Analytics
Products
Key Takeaways
- The project achieved 95% vehicle detection and trajectory accuracy using NVIDIA DeepStream for real-time video analytics, enabling quantifiable improvement in intersection vehicle flow and resource deployment.
- Potential to save commuters an estimated $9.7 million annually in time and fuel costs
- Deliver analytics and insights 4x faster.
- The pilot demonstrated the ability to integrate video analytics AI agents built on NVIDIA Blueprint for video search and summarization (VSS), providing real-time insights and faster event alerts for quicker city responses.
- Integrating with the digital twin built on Esri’s technology gives Raleigh centralized, interactive visualization, alerts, and simulations. This transforms operations from reactive to real-time proactive management, while extending AI-powered analytics across the entire urban environment.
Taking On Safety and Congestion Challenges
Raleigh’s dramatic growth brought surging pressure on housing, transportation, and city infrastructure, while also elevating sustainability demands to maintain a high quality of life. Top community priorities included better managing development and addressing traffic congestion—areas where traditional infrastructure and manual monitoring struggled to keep pace. A core challenge was the need to support Vision Zero, a goal of eliminating fatalities involving vehicles and pedestrians, as well as anticipating rather than simply reacting to problems as population and mobility demands rise.
Esri
“Cities are continuously seeking innovative ways to enhance services. By integrating NVIDIA's smart city AI agents with Esri's interactive geospatial platform on Azure, Raleigh is committed to revolutionizing urban mobility. This initiative aims to reduce congestion, improve safety, and enable real-time, data-driven decisions that elevate the quality of life and mobility for our residents.”
Beth C. Stagner
Assistant Director - Information Technology, City of Raleigh
Real-Time Traffic Intelligence With Computer Vision
Optimizing traffic flow through well-designed signal timing plans—adjusting cycles and splits to match actual travel demand—remains crucial in a busy city looking to solve traffic congestion. On corridors like Capital Blvd, which handles up to 55,000 vehicles a day at peak, Raleigh estimated that eliminating 4.5 seconds per signal across 10 major intersections within 5.5 miles can save commuters an estimated $9.7 million annually in time and fuel costs.
Traditionally, traffic engineers relied on manual counts of vehicle turning movements—field staff counting vehicles and recording patterns at intersections—to guide their decisions about signal timing. To modernize traffic management, Raleigh adopted NVIDIA DeepStream, part of the NVIDIA Metropolis suite of vision AI tools. This real-time streaming analytics toolkit enables automated vehicle counting and trajectory tracking across key intersections. The pilot has already delivered valuable insights for the city’s traffic engineering team, with DeepStream achieving 95% accuracy in vehicle detection for turning movement counts. It also boosts the analytical capabilities of Raleigh’s team of just 2.5 dedicated staff by 4x with automatic tracking of traffic at peak hours, visualization of trajectories, object counts, and more, dramatically increasing operational efficiency and impact.
Digital Twins Enable Data-Rich City Planning and Simulation
At the same time, Raleigh started to use Esri’s widely adopted ArcGIS geospatial platform to simulate different building projects and sustainability initiatives. However, the city struggled with multiple repositories of sensor data and wanted a centralized place to aggregate, visualize, and analyze data. For example, they knew that traffic was a problem, but didn’t have the ability to visualize the data they were collecting from Metropolis on a 3D interactive map.
To move beyond static historical analysis, Raleigh has worked with Esri to incorporate these live traffic flows into their digital twin, allowing them to better understand congestion levels and signal behavior within a 3D mesh of the city. This immersive system provides engineers and planners with comprehensive, real-time visibility, not just for vehicle movement, but also pedestrian and bicycle traffic. The digital twin allows for scenario-based simulations and predictive planning (especially ahead of special events), enabling the city to identify bottlenecks, forecast impacts, and collaborate across departments for emergency response, event management, and infrastructure upgrades.
Beyond traffic, pedestrian, and bike data analytics, the city has plans to incorporate additional video analytical capabilities including flood detection, parking utilization, and asset conditions.
AI Agents Power Faster Responses and Safer Streets
As Raleigh prioritizes walkable and bikeable city design, the need for faster incident response grows. Real-time actionable insights powered by vision language models (VLMs) are essential. For example, an alert for a stalled vehicle at a busy intersection can instantly trigger video analysis using the NVIDIA Blueprint for video search and summarization (VSS). The system automatically detects safety hazards and generates text descriptions directly beneath video feeds, updating incident maps as the situation unfolds. It can also identify pedestrians and bicyclists, automating the detection and classification of critical violations and unsafe scenarios. This automation streamlines response time; when an incident is detected, city operators are notified immediately and can act before delays or safety risks escalate.
This enhanced visibility is actively helping Raleigh’s traffic engineering staff make more informed decisions using higher-fidelity, multi-day data across a wide range of conditions. As the system continues capturing baseline information on vehicles, pedestrians, and bicyclists, the city anticipates gaining a clearer understanding of how travel patterns and behaviors evolve along key corridors and across the city. This ongoing data collection provides a valuable foundation for evaluating the effectiveness of future traffic and transportation mitigation projects as they’re implemented, offering an evidence-based way to understand what’s working and where adjustments may be needed. While still in the preliminary stages, the insights gathered so far indicate the potential for meaningful improvements in mobility, safety, and overall situational awareness as the technology matures and becomes further integrated into daily operations.
Scaling AI for a Smarter, Safer Raleigh
Looking ahead, the city plans to install an additional 1,000 cameras throughout key sectors over the next 12 to 24 months, as well as evaluate cloud analytics operations platform on Microsoft Azure Cloud to scale up capabilities and reduce costs even further.
Future phases will potentially broaden this smart platform’s impact to other civic domains, such as emergency operations, water management, waste services, and fire inspections. The city plans deeper integration of advanced analytics and accelerated adoption of the NVIDIA Blueprint for smart city AI for sustainable urban growth. This will allow continuous improvement, serving as a national model for how real-time AI, powered by NVIDIA and partners, can deliver safe, efficient, and resilient city operations. For more insight into Raleigh’s smart city development, watch the replay video of the talk “Enable Safer Streets and Resilient Cities with Live Insights”
Discover how NVIDIA Metropolis accelerates smart city applications.