Shell Trains Custom AI Chatbot With NVIDIA NeMo to Uplevel Operations

To achieve higher accuracy for its domain-specific LLM tailored to the energy industry, Shell focused on curating high-quality training data as the foundation of its AI solution. The development process began with the curation and preprocessing of a vast dataset of chemistry documents. Initially, Shell had access to 300,000 technical documents collected over decades. These documents cover various technical domains and were curated down to 154,000 high-quality documents using the NVIDIA NeMo Curator.

The curation process involved several steps, including exact and fuzzy deduplication to remove repeated or near-duplicate content. Shell also applied quality filters, removing documents with insufficient information or poor formatting, and used language detection to exclude non-English content. Additionally, domain classification was used to select documents for building domain-specific benchmarks.

Once the dataset was curated, Shell went beyond retrieval-augmented generation (RAG) and used the NVIDIA NeMo framework to perform domain-adaptive pretraining (DAPT) and supervised fine-tuning (SFT) to enhance the model’s domain-specific knowledge and accuracy. DAPT allowed the model to truly understand the unique context and terminology of the chemical industry. At the same time, SFT further refined the model’s performance by training it on labeled data specific to Shell’s needs. Leveraging the parallelism techniques available through NeMo, Shell accelerated the model training time (millions of GPU hours) by 20% compared to other open-sourced frameworks.

Retrieving accurate information from enterprise knowledge sources can be challenging for RAG because standard language models often misinterpret user queries, matching them with broad, generic information instead of domain-specific insights. Adapting LLMs to industry-specific language helps bridge this gap and improves answer accuracy and conversation quality. This need for precision drove Shell to develop in-house capabilities, not available in market products, for customizing LLMs, leading to the company’s collaboration with NVIDIA.

With the AI-powered chatbot developed by Shell, technology staff would have the possibility to quickly access detailed chemical documents and data, reducing the time required for these tasks and reducing the risk of errors. By streamlining knowledge retrieval, the AI chatbot can enhance gaining insights and making decisions in the R&D space, supporting both innovation and operational efficiency.

Aside from enhanced information retrieval, the custom LLM can also be utilized for technical document analysis, helping streamline workflows across departments.

By continuously refining the model through real-world interactions, Shell is positioning its AI ecosystem as an adaptive intelligence layer, transforming enterprise knowledge management into a dynamic and accessible resource.

Looking ahead, Shell plans to further improve the capability of domain-adapted LLM by expanding the training dataset and developing more diverse and challenging evaluation tasks. Together with enhancing the text-to-text model, the ambition is to unlock the multimodal capabilities of the AI chatbot. This will enable the chatbot to handle and process various types of data, including images and videos.

The addition of multimodal capabilities is expected to provide more comprehensive and contextually rich information, which can be particularly valuable for complex decision-making processes.

These enhancements are anticipated to further drive productivity and operational efficiency, solidifying Shell’s commitment to leveraging ahead-of-market advanced AI technologies for the benefit of its operations.