Telecommunications
AT&T Drives AI Agents’ Accuracy, Efficiency, and Performance With NVIDIA
Objective
AT&T, one of the world’s largest telecommunications companies, is reimagining customer care through the power of AI. With a massive customer base and a growing portfolio of digital services, AT&T has long prioritized innovation and operational efficiency to meet evolving customer expectations. As demand for personalized, always-on support continues to grow, the company is scaling AI-powered agents across its operations to deliver faster, more accurate service. Facing challenges like model drift, rising computational demands, and the need for real-time data access, AT&T turned to NVIDIA AI Enterprise, NVIDIA NIM, and NVIDIA NeMo™ microservices to build a feedback-driven AI platform that continuously improves performance while optimizing cost, speed, and compliance.
Scaling Customer Service AI Agents With a Data Flywheel
AT&T developed Ask AT&T AI agents to fulfill customer needs while running operations at scale. These agents significantly enhance user experiences through automated services, such as analyzing customer accounts to deliver personalized service recommendations and software upgrades while also bolstering fraud prevention and optimizing network performance leading to 84% decrease in call center analytics.
This initiative is part of AT&T’s ambitious strategy to deploy dozens of AI-powered use cases, with hundreds more in the pipeline.
To scale this vision, AT&T needed to overcome key challenges: reducing latency, lowering operational costs, and enhancing the accuracy of the models powering these AI agents.
Another hurdle is model accuracy drift. With nearly 10,000 documents updated multiple times a week, AI agents need to be updated with the most current information to remain effective.
To address this, they leveraged a data flywheel—a continuous feedback loop that ensures multi-agent systems stay up to date and consistently deliver peak performance.
Key Takeaways
From Cost to Performance: How AT&T Navigates AI Deployment Challenges
One of the biggest challenges AT&T faced was reducing latency and cost as AI adoption grew. With more applications relying on AI-driven solutions, computational demands surged, making it crucial to find a way to maintain high performance while optimizing efficiency. Striking the right balance between speed and cost became essential for sustainable AI operations.
AT&T also had to ensure the availability of high-quality data for AI training and fine-tuning. The process of curating and cleansing data became a bottleneck, and without robust data preparation, models risked being trained on outdated or irrelevant information, negatively impacting their accuracy and reliability.
Managing the complexity of multi-agent AI systems was also a pressing concern. AI agents needed to adapt and optimize continuously to align with AT&T’s dynamic business environment. Ensuring that these models remained relevant, secure, and high performing in real time required a systematic approach to ongoing refinement and evaluation.
“The successful fine-tuning story of this use case and others like it was enough evidence for us to pursue building out an entire fine-tuning platform that supports both user and differentiated fine-tuning flows across various tasks.”
Kostikey Moustakas
Director of Data Science at AT&T
Building Optimized, Top-Performing AI Agents
To overcome these challenges, AT&T worked with Quantiphi to leverage NVIDIA AI Enterprise, including NVIDIA NeMo and NIM microservices, to implement a data flywheel approach for continuously enhancing AI agent performance:
- Refining Data Quality With NeMo Curator: Using NeMo Curator, AT&T cleansed and filtered training data, leveraging domain expertise and historical application logs. Techniques like topic modeling and iterative filtering ensured only high-quality datasets were used.
- Customizing AI Models for Maximum Performance: AT&T experimented with various base models, including Mistral, Mixtral, and Llama, using NeMo Customizer. Through an iterative fine-tuning process, Mistral 7B emerged as the optimal performer, balancing accuracy and efficiency.
- Implementing Rigorous Evaluation: NVIDIA’s NeMo Evaluator enabled AT&T to establish a robust evaluation system, measuring AI agent performance through metrics like Rouge, BERT F1, question relevance, and answer quality. This ensures they are measuring the model’s performance on the most appropriate metrics.
- Acting on the Latest Information: AT&T used NeMo Retriever to set up pipelines with quick access to the company’s repository so the agents can make decisions and take actions using up-to-date information.
- Deploying on Secure Infrastructure: The models are deployed as NIM™ microservices, which deliver optimized inference performance, support industry-standard API, and provide the flexibility to run on any GPU-accelerated infrastructure.
Achieving Breakthrough Results With AI Optimization
The implementation of NVIDIA NIM and NeMo microservices yielded significant performance gains as well as substantial cost savings, while maintaining enterprise-grade scalability and compliance :
- Enhanced AI Accuracy: AI agent responses showed up to 40% improvement in accuracy in responses with post-training in key metrics, particularly in Rouge and BERT F1 scores.
- Lower Latency: Deploying lightweight, fine-tuned models enabled AT&T to lower computational overhead while maintaining high-quality responses
- Sustained AI Improvement: The data flywheel approach facilitated continuous learning, ensuring AI agents adapted to evolving business needs and user feedback.
The deployment of Ask AT&T with NVIDIA NIM and NeMo enabled AT&T to realize an 84% decrease in call center analytics cost.
The Road Ahead: Scalable, Feedback-Driven AI Innovation
Looking ahead, AT&T aims to centralize a feedback loop to refine AI agent quality continuously. By integrating a hybrid evaluation system—leveraging both human reviewers and AI models as judges—AT&T ensures its AI ecosystem remains responsive to business needs.
Additionally, AT&T, in collaboration with Arize AI, is automating the identification and handling of challenging AI interactions. This ensures that models are rigorously evaluated and fine-tuned based on real-world feedback, further enhancing AI accuracy and compliance.
By refining its AI agents through NVIDIA NeMo and NIM’s advanced capabilities, AT&T is not only improving operational efficiency but also setting a new benchmark for scalable and adaptive AI solutions.
Build and power your AI agents with a data flywheel to achieve optimized, peak performance using NVIDIA NeMo microservices.