Objective
For disaster scenarios where every second matters, Leidos created an AI-powered, multi-agent command center that helps emergency leaders make faster, more informed decisions. By aggregating real-time data from video, audio, and sensor feeds, the system reduces cognitive load, boosts situational awareness, and enables coordinated response across multiple agencies. The result is quicker action, better resource deployment, and improved outcomes when lives are on the line.
Customer
Leidos
Key Takeaways
Faster, More Informed Emergency Decisions
- Aggregates and analyzes multi-source data in real time, enabling commanders to respond quicker, with better accuracy—saving lives and reducing damage.
Reduced Cognitive Load for Responders
- AI agents built with NVIDIA AI Enterprise handle data triage and monitoring, freeing responders to focus on the most critical incidents and manage multiple crises at once.
Rapid, Cost-Effective Deployment
- Virtualization with NVIDIA virtual GPU (vGPU) accelerates AI development and testing on modest hardware, cutting rollout time and infrastructure costs.
Complexity and Chaos in Disaster Scenarios
Disaster environments are the epitome of complexity. Consider a large-scale wildfire, a city hit by an earthquake, or a flood zone overwhelmed by rising waters. Emergency responders must quickly assess risk, coordinate resources, communicate across departments, and adapt on the fly—all while lives hang in the balance.
Traditionally, responders have relied on static models, historical data, and centralized control systems. While these have their place, they often fall short when speed, adaptability, and situational awareness are critical. Human decision-makers, no matter how skilled, are limited by information overload and the unpredictability of real-world scenarios.
Starting in 2024, Leidos—a leading systems integrator delivering smarter, more efficient digital and mission innovations to government and commercial customers—and NVIDIA collaborated to create a multi-agent architecture to reduce cognitive load for decision-makers as they respond to disaster scenarios. The solution helps to seamlessly integrate simulation and communication data with the goal of delivering actionable insights and improving overall situational awareness for human commanders who are responsible for the overall management of the incident.
Building fire simulation with chat log
AI Agents in the Command Centers
The central hub for emergency response is the command center. It continuously monitors multiple video, radio communication, and public safety asset data feeds. When an incident occurs, numerous personnel need to analyze and triage which regions or individuals require immediate assistance, allowing responders to concentrate on the most critical situations, potentially handling several emergencies simultaneously throughout the day.
The Leidos and NVIDIA collaboration produced a multi-agent architecture designed to ingest multiple radio frequency (RF), audio, and video feeds, which are aggregated and used to inform, update, and help a human commander to make faster, more informed decisions.
The AI agents processing data from roadway intersection cameras, weather data, public camera feeds, emergency vehicle locators, automated 911 transcripts, and radio communications are combined with tactical guidance software to produce actionable insights. The human commander leverages an interactive AI agent that uses the aggregated insights to orchestrate other AI agents and provide real-time context to help the human commander make decisions given rapidly developing emergency scenarios.
AI agents in command center
Building an AI Agent Stack and Running Simulation Scenarios
Leidos leveraged NVIDIA AI Enterprise software, including NVIDIA NIM™ microservices for AI agents, NVIDIA® Riva Parakeet NIM for automatic speech recognition (ASR) and NVIDIA Metropolis for video processing, to build the Command and Control Agentic AI System (C2AI) with integrated vision language models (VLMs) to enhance operational efficiency.
By leveraging the Team Awareness Kit (TAK), a situational awareness and collaboration tool developed by Air Force Research Laboratory (AFRL) and Leidos, AI agents can initiate response coordination via TAK to mobilize emergency response teams.
Synthetic data is essential for training AI models in disaster response. Leidos used Unreal Engine to produce digital twin mockups of a command center, car accidents, and structure damage scenarios from destructive weather events. 911 operators and caller exchanges were created synthetically to develop and demonstrate large language models (LLMs), ASR, and VLMs that powered the multiple AI agents in the solution workflow.
Leidos and NVIDIA developed agents for video feed processing, victim information databases, resource management, emergency response planning, asset dispatch, and human commander interaction.
Leidos
“Our collaboration with NVIDIA to develop the multi-agent command center has revolutionized emergency response. By utilizing NVIDIA AI software, organizations can build and use AI agents to process real-time data, reducing the cognitive load on human commanders and enabling quicker, more informed decisions. Additionally, NVIDIA vGPU technology allows us to simulate scenarios and scale our development process, significantly reducing development time.”
Dr. Corey Hendricks
Vice President, Chief Engineer for Commercial and International, Leidos
NVIDIA Virtualization Solutions for AI Development and Simulation
With over a dozen individual contributors, including data scientists, system administrators, developers, and artists, Leidos leveraged NVIDIA RTX™ Virtual Workstation and NVIDIA AI Enterprise to support both AI and graphics workloads while centralizing the AI models and data within the data center.
The development efforts were supported by a set of central hardware resources, including a single Dell R760XA with two NVIDIA L40S GPUs managed by VMware Cloud Foundation.
Leidos system administrators leveraged NVIDIA vGPU™ technology to efficiently share GPU power across multiple virtual machines. This ensured that Unreal Engine environments, AI agent developers, and TAK applications all had the high-performance graphics resources they needed to build and test with speed and precision. The flexible vGPU configuration also allowed the team to easily adjust resources throughout development, while providing accelerated remote graphics to enhance both developer productivity and testing performance.
Learn More and Get Started
NVIDIA RTX Virtual Workstation supports graphics and AI workloads, enabling IT to leverage virtual machines for visual computing and AI development to validate outcomes in the preproduction phase. NVIDIA AI Enterprise offers production-ready software for AI development and deployment with enterprise support and security for scalable deployment. These solutions help organizations seamlessly integrate AI into their workflows and drive innovation across various industries.
Learn more about NVIDIA RTX Virtual Workstation.