# Harness Agentic AI and Reasoning Models to Accelerate Scientific Discovery ## Abstract Learn how an AI co-scientist that combines powerful reasoning models with science simulations can expand the aperture for human scientists to achieve breakthroughs in domains such as fusion energy and targeted alpha therapy for cancer. This session delves into strategies for overcoming the challenge of data scarcity for training, for employing reinforcement learning leveraging physics simulations, AI surrogates, emulators, and building AI agents that can reiterate autonomously. ## AI Summary - The presentation discussed the challenges in discovering new molecules for cancer treatment, particularly focusing on the actinium element, which has special properties for releasing alpha particles to target cancer cells. - A collaborative platform called Kilitron, developed by NVIDIA and the Los Alamos National Lab, was introduced to accelerate the discovery process by integrating AI, high-throughput computing, and large language models. - The Kilitron platform used a discovery loop that included hypothesis generation, molecular structure creation, and quantum calculation verification to explore the chemical space more efficiently. - The team curated a large dataset of question-answer pairs to fine-tune a large language model called Prospero, which was designed to generate and evaluate hypotheses in the field of Inertial Confinement Fusion (ICF). - The evaluation metric for the fine-tuned model included a golden set of 250 questions from subject matter experts, a silver set of 550 questions generated by the NeMo Data Designer, and public physics benchmarks. - The presentation highlighted the ongoing process of data curation and model fine-tuning, aiming to find the optimal blend of ICF-specific and general STEM data to improve the model's performance. ## Transcript **[00:00:05 – 00:00:08]** I welcome you all for this session. **[00:00:08 – 00:00:11]** So the biggest challenge for the cancer treatment right **[00:00:11 – 00:00:17]** now is how long it will take to discover a new effective molecule. **[00:00:17 – 00:00:21]** And today I would like to discuss how agentic AI molecule **[00:00:21 – 00:00:26]** discovery can accelerate this process drastically. **[00:00:26 – 00:00:36]** I will introduce the Kilitron platform that was developed through **[00:00:36 – 00:00:41]** the collaboration between NVIDIA and the Los Alamos National Lab. **[00:00:41 – 00:00:46]** Through this, we can explore all these large chemical spaces **[00:00:46 – 00:00:50]** that were not possible before. **[00:00:50 – 00:00:54]** And this Keyletron essential is to allow us to design this **[00:00:54 – 00:00:58]** molecule, that combined molecule with a targeted property. **[00:00:58 – 00:01:02]** And here we will target for the cancer treatment. **[00:01:02 – 00:01:06]** So finding a molecule that are effective is a challenge because of **[00:01:06 – 00:01:09]** sheer volume of the chemical space. **[00:01:09 – 00:01:13]** If you see that Milky Way on the left corner, there are 10 to **[00:01:13 – 00:01:17]** the 11th star available over there. **[00:01:17 – 00:01:20]** And the already discovered chemical space in this proprietary **[00:01:20 – 00:01:24]** database is 10 to 26. **[00:01:24 – 00:01:27]** And that's not even near to the limit of the chemical **[00:01:27 – 00:01:30]** space available out there. **[00:01:31 – 00:01:35]** Everybody in this room knows that AI has transformed the **[00:01:35 – 00:01:37]** data-rich research field. **[00:01:37 – 00:01:42]** For example, AlphaFold has solved this grand challenge **[00:01:42 – 00:01:44]** for the protein folding. **[00:01:44 – 00:01:49]** This success is based on the large data curated in the Protein **[00:01:49 – 00:01:52]** Data Bank in the last 50 years. **[00:01:52 – 00:01:57]** So essentially you have 200,000 structures and a million structures **[00:01:58 – 00:02:00]** from the calculation. **[00:02:00 – 00:02:04]** However, most scientific area, or a lot of scientific area, are **[00:02:04 – 00:02:09]** in the data-scarce regime, and I will illustrate in the next slide. **[00:02:09 – 00:02:12]** So it's a natural question to ask, what AI can do **[00:02:12 – 00:02:17]** for those data-scarce challenge we are facing? **[00:02:17 – 00:02:21]** For example, in this largest database available for the **[00:02:21 – 00:02:25]** molecular structures, across the whole periodic table, **[00:02:25 – 00:02:26]** there are about 1.5 million **[00:02:26 – 00:02:30]** Five million structures available, 0.5 million for **[00:02:30 – 00:02:32]** the metal complexes. **[00:02:32 – 00:02:34]** That is on part of the protein data bank. **[00:02:34 – 00:02:37]** We will think that's enough. **[00:02:38 – 00:02:43]** Pay attention to this two row on the lowest periodic table. **[00:02:43 – 00:02:47]** These are really critical elements for the system, particularly that **[00:02:47 – 00:02:52]** enable us right now able to sit here because they are the critical **[00:02:52 – 00:02:58]** elements enable GPUs, CPUs, as well as our smartphone, let alone energy **[00:02:58 – 00:03:01]** conversion, national security. **[00:03:01 – 00:03:06]** So, however, for such important elements, they compare to the **[00:03:06 – 00:03:09]** To the data available for the transition method, as **[00:03:09 – 00:03:12]** you see in this structure here, they are minimum. **[00:03:12 – 00:03:14]** You can't even see the number. **[00:03:14 – 00:03:18]** You have to zoom in to see those numbers, and they are less than 10 **[00:03:18 – 00:03:20]** points for some critical elements. **[00:03:20 – 00:03:24]** How can we use AI directly to train and learn from those **[00:03:24 – 00:03:26]** models, from those data? **[00:03:26 – 00:03:27]** It's very challenging. **[00:03:27 – 00:03:31]** And likewise, the application of those molecules for this **[00:03:31 – 00:03:35]** neodymium and this prozinium that are available are important **[00:03:35 – 00:03:40]** for the smartphone and GPU are at the order of tens. **[00:03:40 – 00:03:44]** So we need to think a smarter way to make this available, and today **[00:03:44 – 00:03:47]** I will focus on the first element. **[00:03:47 – 00:03:51]** You see the last row is called actinium, the first element **[00:03:51 – 00:03:54]** of actinic series. **[00:03:54 – 00:03:57]** The reason is this molecule have very special property. **[00:03:57 – 00:03:59]** The isotope of this molecule **[00:04:00 – 00:04:05]** of this element, isotope 2225, can release this very **[00:04:05 – 00:04:09]** particle, a powerful particle called alpha particle. **[00:04:09 – 00:04:12]** And this is a megavolt, this is a strong alpha particle, if **[00:04:12 – 00:04:16]** you bring them next to the cancer cell, they can kill, only kill **[00:04:16 – 00:04:21]** the cancer cell, and by leaving the healthy tissue untouched. **[00:04:21 – 00:04:24]** And this is in contrast of the chemotherapy that **[00:04:24 – 00:04:25]** effective, but the **[00:04:25 – 00:04:30]** kill all the cells unanimously and which is reduced significantly **[00:04:30 – 00:04:33]** reduce the side effects. **[00:04:33 – 00:04:37]** So this will benefit to all the patients that are diagnosed every **[00:04:38 – 00:04:40]** year with 20 million patients. **[00:04:40 – 00:04:44]** So I think it's a good health challenge we can solve using **[00:04:44 – 00:04:46]** this actinium element. **[00:04:47 – 00:04:50]** However, the challenge of delivering this actinium **[00:04:50 – 00:04:53]** to the cancer cell is the biggest challenge. **[00:04:53 – 00:04:59]** And the reason is that actinium atom is huge, right? **[00:04:59 – 00:05:04]** If you imagine that a toddler holding a big yoga ball, they **[00:05:04 – 00:05:06]** cannot hold very stable. **[00:05:06 – 00:05:09]** And that's essentially the challenge we have here, because **[00:05:09 – 00:05:13]** of the ligand that you see here, that designed for the smaller **[00:05:13 – 00:05:15]** ball, for the transition metal. **[00:05:15 – 00:05:18]** So we need to make this for the larger element, **[00:05:18 – 00:05:20]** and that's much harder. **[00:05:20 – 00:05:23]** And because large, you have a lot of surface area, so we need **[00:05:23 – 00:05:29]** a lot more arm to coordinate with this element to make them stable. **[00:05:29 – 00:05:31]** And this is called high coordination numbers. **[00:05:31 – 00:05:34]** So there are a lot of arrangements involved with this. **[00:05:34 – 00:05:38]** So hence, we developed this Kilitron platform. **[00:05:38 – 00:05:41]** Specifically targeted on this problem. **[00:05:41 – 00:05:45]** So we want to design the system that combined this element **[00:05:45 – 00:05:46]** as strong as possible. **[00:05:47 – 00:05:51]** And then the short-term goal is we want to design something **[00:05:51 – 00:05:55]** 100 times faster than the state of art, that macropod that is **[00:05:55 – 00:05:59]** under clinical trial at this stage. **[00:05:59 – 00:06:05]** So I want to emphasize again why those system is such data scarce. **[00:06:05 – 00:06:08]** So you see that in the last 50 years, there are about **[00:06:08 – 00:06:12]** 100,000 molecule metal ligand complex discovered across **[00:06:12 – 00:06:13]** the whole periodic table. **[00:06:13 – 00:06:20]** So it's very slow to identify good ligand and stable complexes. **[00:06:20 – 00:06:25]** And you search all this 1.2 million data structure, you find there are **[00:06:25 – 00:06:33]** less than 400 structures available with one molecule that can hold **[00:06:33 – 00:06:38]** this together with a coordination number greater than eight. **[00:06:38 – 00:06:42]** And if you look at the periodic table, there are only six **[00:06:42 – 00:06:43]** for the actinide. **[00:06:43 – 00:06:46]** So there are very few data available. **[00:06:46 – 00:06:49]** And then you look closer of those molecules, what you see here, they **[00:06:49 – 00:06:52]** have very high chemical similarity. **[00:06:52 – 00:06:56]** That means the same color of those circles means the same structure. **[00:06:56 – 00:07:01]** So they have the same core structure with small modification **[00:07:01 – 00:07:03]** on the side substitute group. **[00:07:03 – 00:07:08]** So that means this chemical space is very limited overall. **[00:07:08 – 00:07:11]** So one natural thing is the large language model developed **[00:07:11 – 00:07:14]** in the last couple years has a show its power. **[00:07:14 – 00:07:18]** to generate a lot of new strings, give us a lot of new stuff **[00:07:18 – 00:07:22]** to think about, as we curious to test out how large language **[00:07:22 – 00:07:27]** model can directly generate those molecule for us. **[00:07:27 – 00:07:31]** We will say it's helpful, but the success is limited. **[00:07:32 – 00:07:36]** That's because it generate a lot of an invalid smile **[00:07:36 – 00:07:40]** strings, and then the structure as you see here is the improvement, **[00:07:40 – 00:07:44]** but it's quite a high similarity compared the training data. **[00:07:44 – 00:07:48]** So as a result, we want to find a new system, a new way **[00:07:48 – 00:07:53]** to explore the chemical space using this transferable knowledge. **[00:07:53 – 00:07:56]** It's just because you cannot exhaustively search all **[00:07:56 – 00:07:57]** the chemical space. **[00:07:58 – 00:08:02]** Here is our collaboration with NVIDIA come through, is to develop **[00:08:02 – 00:08:06]** this discovery loop, starting with hypothesis generation, **[00:08:06 – 00:08:10]** because hypothesis generation is when we talk chemistry **[00:08:10 – 00:08:11]** like a natural language, right? **[00:08:11 – 00:08:12]** More electron. **[00:08:12 – 00:08:15]** Donating group is better than positive group, **[00:08:15 – 00:08:17]** something like that. **[00:08:17 – 00:08:19]** And then the next step, this is using the NEMATRON **[00:08:19 – 00:08:21]** developed in NVIDIA. **[00:08:21 – 00:08:26]** And the next step is using this GMO module developed in NVIDIA to **[00:08:26 – 00:08:31]** translate those hypotheses to those candidate molecules, and then that **[00:08:31 – 00:08:35]** will feed into the architecture that developed at Leno, that **[00:08:35 – 00:08:39]** translates those from 2D structure to 3D structure, and that can **[00:08:39 – 00:08:42]** feed into the quantum calculation. **[00:08:42 – 00:08:45]** To get the high accuracy verification and then the **[00:08:45 – 00:08:50]** next step is based on all those calculated results, we can **[00:08:50 – 00:08:56]** analyze all those structures and then to see how this statistically **[00:08:56 – 00:09:01]** to identify whether those features are verified, confirmed, **[00:09:01 – 00:09:04]** or reject these hypotheses. **[00:09:04 – 00:09:06]** And all those information will feed back to the hypothesis **[00:09:06 – 00:09:10]** generation module and then that will help to generate, **[00:09:10 – 00:09:11]** guide the next AI, NVIDIA. **[00:09:11 – 00:09:13]** AI, NVIDIA. **[00:09:13 – 00:09:16]** So I will quickly walk through those each steps. **[00:09:16 – 00:09:20]** And so the first step is a hypothesis generation. **[00:09:21 – 00:09:23]** We leverage all the state-of-the-art technique **[00:09:23 – 00:09:29]** like REC and also using this to generate those interval **[00:09:29 – 00:09:32]** chemical hypothesis that can be. **[00:09:32 – 00:09:35]** Felt into this ligand design step. **[00:09:36 – 00:09:40]** So the idea here is that instead of exhausting research, all those **[00:09:40 – 00:09:44]** large chemical space, we want to, like a chemist is doing, is doing **[00:09:44 – 00:09:49]** the targeted and principle chemical principles and using that to guide **[00:09:49 – 00:09:52]** us to navigate this chemical space. **[00:09:52 – 00:09:55]** So the next step is once we generate the hypothesis, we **[00:09:55 – 00:09:59]** need to translate that into the molecules And this is **[00:09:59 – 00:10:03]** a leveraged NVIDIA's GenMol module. **[00:10:03 – 00:10:10]** So this will allow us, starting with this library, and then **[00:10:10 – 00:10:13]** decorate that with a different group, and this is done **[00:10:14 – 00:10:19]** by Logan Augustine, who is sitting in the audience. **[00:10:20 – 00:10:24]** So the next step, once we generate all those molecules, **[00:10:24 – 00:10:28]** that's what you see is from GenMol, it's kind of a graph or a string. **[00:10:29 – 00:10:33]** However, in order to do that for the quantum calculation, **[00:10:33 – 00:10:35]** is that we have to... **[00:10:35 – 00:10:39]** Generally, convert this to a 3D structure as you see here. **[00:10:39 – 00:10:42]** And this is not as easy as you would imagine. **[00:10:42 – 00:10:46]** It's similar to the alpha fold, right? **[00:10:46 – 00:10:50]** So you need to transfer from the strain of the protein, transfer **[00:10:50 – 00:10:52]** that to the 3D structures here. **[00:10:52 – 00:10:54]** We did this for metal complexes. **[00:10:54 – 00:10:57]** Because this is a large metal complex, there are a lot of **[00:10:57 – 00:11:01]** converge, a lot of different conformation can happen, and **[00:11:02 – 00:11:05]** this is really hard. **[00:11:06 – 00:11:09]** So this is done by the architecture that they developed at Los **[00:11:09 – 00:11:13]** Alamos and by Michael Taylor. **[00:11:13 – 00:11:17]** So this is essentially you translate the string from **[00:11:17 – 00:11:19]** the left box you see here. **[00:11:19 – 00:11:22]** To the 3D structure and energy ranked. **[00:11:22 – 00:11:25]** So this will give you the confirmation so you can calculate **[00:11:25 – 00:11:29]** the binding energy and the property of those complexes. **[00:11:29 – 00:11:33]** Even though it's a very hard problem, as luckily as being **[00:11:33 – 00:11:34]** automated already, so. **[00:11:35 – 00:11:38]** By considering all this gray box below, you will see here, **[00:11:38 – 00:11:41]** it's hidden from all the users. **[00:11:41 – 00:11:45]** By considering metal symmetry, ligand binding constant, all **[00:11:45 – 00:11:47]** those can be considered. **[00:11:47 – 00:11:49]** So now all those 3D structures will be **[00:11:49 – 00:11:58]** fed into this HPC cluster, either CPU-dominated or GPU clusters. **[00:11:58 – 00:12:02]** So now we can estimate the binding constant by **[00:12:02 – 00:12:06]** calculating those energy of each complex metal itself. **[00:12:06 – 00:12:09]** And this can be done at different levels. **[00:12:09 – 00:12:13]** Here I want to emphasize that this is really, this work also **[00:12:13 – 00:12:16]** part is automated by the parcel. **[00:12:16 – 00:12:23]** And this is allow us to integrate this with the rest of the workflow. **[00:12:23 – 00:12:26]** So once all this upon the complication of the **[00:12:26 – 00:12:28]** quantum calculation, **[00:12:28 – 00:12:31]** And now we can have the binding energy, we have **[00:12:31 – 00:12:34]** all the 3D structures, so we have all the features. **[00:12:34 – 00:12:39]** Now we can do statistic analysis based on all those results **[00:12:39 – 00:12:46]** to analyze whether this molecule confirm or reject the hypothesis. **[00:12:46 – 00:12:49]** And not only that, based on this, we can also mine all the **[00:12:49 – 00:12:55]** structures to identify new features that not only in current data **[00:12:55 – 00:12:58]** set that allow us to do all the **[00:12:58 – 00:13:02]** To mine the new features that fit into large language model. **[00:13:02 – 00:13:07]** So all those modules that you see now is harmoniously integrated **[00:13:07 – 00:13:18]** with agentic workflow and then by academy agentic workflow right now. **[00:13:18 – 00:13:21]** So this is right now is a part I developed by Logan **[00:13:21 – 00:13:25]** Ward in the NVIDIA. **[00:13:25 – 00:13:28]** So here is the progress you see after you go through all **[00:13:28 – 00:13:30]** this agentic workflow. **[00:13:30 – 00:13:36]** So over before, in half a century, we developed about 400 molecules **[00:13:36 – 00:13:41]** with high coordination number, and now we can explore this overnight. **[00:13:41 – 00:13:45]** And then these are a couple of good candidates, potential **[00:13:45 – 00:13:47]** promising candidates. **[00:13:47 – 00:13:50]** If you look at the circle of the rain, they are asymmetric. **[00:13:50 – 00:13:54]** So now we step in our carbon zone, we can explore new chemical **[00:13:54 – 00:13:58]** space, and then these are the promising candidate. **[00:13:58 – 00:14:01]** And this will be sent to the high-throughput lab **[00:14:01 – 00:14:06]** for verification to be synthesized and to be tested. **[00:14:06 – 00:14:11]** So with that, I think it really demonstrates, combine the **[00:14:11 – 00:14:15]** AI, high-throughput computing, a large language model, we can **[00:14:15 – 00:14:20]** explore this large chemical space that cannot be achieved before. **[00:14:20 – 00:14:23]** And I would like to finish my talk thanks to the team, because **[00:14:23 – 00:14:25]** this is a really team effort. **[00:14:25 – 00:14:30]** And Danny here is the architect of this discovery loop, and **[00:14:30 – 00:14:34]** then with all the talented postdoc, and the team member **[00:14:34 – 00:14:39]** here is Pascal, Jian, and Thomas, doing all those discovery loop. **[00:14:39 – 00:14:43]** And then we had a lot of fun working with Kitika's team, **[00:14:43 – 00:14:47]** with Logan, and Scott developed those, Logan developed the Agentic **[00:14:47 – 00:14:53]** AI, and Scott working on this large language model and container site. **[00:14:53 – 00:14:57]** And we're really lucky to have a NERSC team to help **[00:14:57 – 00:15:02]** us to integrate the whole workflow and the funding resources. **[00:15:07 – 00:15:10]** I'll talk about the science motivation for fusion and Geethika **[00:15:10 – 00:15:13]** Gupta from NVIDIA will talk about the AI aspects of it and how we are **[00:15:13 – 00:15:18]** fine tuning an LLM called Prospero to serve as a co-scientist. **[00:15:32 – 00:15:34]** Now, fusion is something that powers the stars, and it's **[00:15:34 – 00:15:37]** extremely difficult to achieve here on the Earth in a controlled **[00:15:37 – 00:15:41]** manner because fusion requires very high temperatures. **[00:15:41 – 00:15:43]** And so you need to confine this plasma, which the sun does **[00:15:43 – 00:15:45]** quite easily through its gravity. **[00:15:45 – 00:15:48]** Here on the Earth, we have a huge number of challenges **[00:15:48 – 00:15:52]** doing it, but lasers are one way of doing it here on the Earth. **[00:15:52 – 00:15:55]** So the actual ignition facility at the Lawrence Livermore **[00:15:55 – 00:15:59]** National Laboratory is the biggest laser we have here on the Earth. **[00:15:59 – 00:16:02]** And it consists of 192 beams. **[00:16:02 – 00:16:05]** It's 10 stories high. **[00:16:05 – 00:16:08]** It's about one and a half football field size laser. **[00:16:08 – 00:16:13]** So these 192 beams go back and forth through this amplifier **[00:16:13 – 00:16:17]** chain, and at the end of which a very, very high intensity laser **[00:16:17 – 00:16:22]** is inserted inside a tiny pellet, which is a few millimeters sized. **[00:16:22 – 00:16:25]** So you have something with a laser, which is 10 football fields, **[00:16:25 – 00:16:28]** which goes through this, what's called a target chamber shown **[00:16:28 – 00:16:30]** there in blue, which is 10 meters. **[00:16:30 – 00:16:33]** And inside that is this tiny pellet, which is a few millimeters. **[00:16:33 – 00:16:37]** And the precision required, you can see, is extremely enormous, right? **[00:16:37 – 00:16:41]** So you need to really be very precise in how you shoot the **[00:16:41 – 00:16:43]** laser in the bright region. **[00:16:43 – 00:16:46]** There is this can called a whole realm which holds this **[00:16:46 – 00:16:48]** tiny pellet in there. **[00:16:48 – 00:16:51]** So the best we have done so far is to get what's called **[00:16:51 – 00:16:53]** a gain of four, which is four times more energy than what **[00:16:53 – 00:16:55]** we put into the laser. **[00:16:55 – 00:16:57]** And that is quite an achievement. **[00:16:57 – 00:17:00]** It took about more than 12 years for us to reach that stage. **[00:17:00 – 00:17:03]** It took about 10 years to build the laser and 12 years **[00:17:03 – 00:17:05]** to get to this point. **[00:17:05 – 00:17:07]** And so that is an extremely challenging problem. **[00:17:07 – 00:17:09]** It's a multi-physics problem where all sorts of different **[00:17:09 – 00:17:11]** physics is involved. **[00:17:11 – 00:17:14]** And a lot of this result is achieved through basically **[00:17:14 – 00:17:20]** human intervention or insights using simulation codes and so on. **[00:17:20 – 00:17:22]** So as I mentioned, a gain of four is the best that we **[00:17:22 – 00:17:24]** have done so far. **[00:17:24 – 00:17:26]** So how do we work through this problem, right? **[00:17:26 – 00:17:29]** Right now, more or less, subject matter experts develop models, **[00:17:29 – 00:17:33]** they design experiments, and execute on them, right? **[00:17:33 – 00:17:36]** So you have a human domain expert who comes up with a hypothesis. **[00:17:36 – 00:17:39]** What about this type of a design of a laser, this type **[00:17:39 – 00:17:41]** of a design of a target? **[00:17:41 – 00:17:43]** And so there's a lot of human intervention involved. **[00:17:43 – 00:17:45]** So we perform multi-physics simulations. **[00:17:46 – 00:17:50]** We post-process them and see, okay, is this a good candidate or not? **[00:17:50 – 00:17:51]** And then we shoot the experiment. **[00:17:51 – 00:17:54]** We figure out what worked, what not. **[00:17:54 – 00:17:57]** And sometimes we get new insights into, oh, we forgot this piece **[00:17:57 – 00:17:59]** of physics, and so on, right? **[00:17:59 – 00:18:01]** So it's a very complicated process. **[00:18:01 – 00:18:05]** Over the last decade or so, machine learning has entered the picture. **[00:18:05 – 00:18:08]** We have used transfer learning to design targets **[00:18:08 – 00:18:10]** and so on and so forth. **[00:18:10 – 00:18:13]** But this is a very data-starved, by data I mean ground truth, **[00:18:13 – 00:18:18]** which is results from experimental data, a very data-starved problem. **[00:18:18 – 00:18:20]** So on the NIF you have about a few hundred of **[00:18:20 – 00:18:23]** these shots, that's about it. **[00:18:23 – 00:18:27]** So how do you accelerate discovery in something like this? **[00:18:27 – 00:18:31]** Now this field began in 1972, and it's only over the last **[00:18:31 – 00:18:34]** decade or so that we've had the NIF, which is capable of giving **[00:18:34 – 00:18:37]** us more energy than what we put in. **[00:18:37 – 00:18:39]** So there's been a lot of work that has occurred over this **[00:18:39 – 00:18:42]** field over the decades, some of it experimental, a lot **[00:18:42 – 00:18:44]** of theoretical, and so on. **[00:18:44 – 00:18:46]** And since this is a transgenerational field, **[00:18:46 – 00:18:49]** as you can see, right, because it began in 1972, the **[00:18:49 – 00:18:53]** people who did the original design of those kinds of Shiva lasers **[00:18:53 – 00:18:58]** on an Omega-24 on the University of Rochester don't exist anymore. **[00:18:58 – 00:18:59]** They're not part of the field. **[00:18:59 – 00:19:02]** So transfer of knowledge from one generation to the other **[00:19:02 – 00:19:04]** is very, very key for this. **[00:19:04 – 00:19:07]** And so what we are hoping is that Prospero will be the **[00:19:07 – 00:19:12]** tool which will allow us to not only have this ability to transfer **[00:19:12 – 00:19:15]** knowledge from one generation to the other, it can do contextual **[00:19:15 – 00:19:18]** retrieval, but we can hopefully also fine-tune it in a well **[00:19:18 – 00:19:25]** enough way that it can actually help us generate hypotheses. **[00:19:25 – 00:19:28]** So we're hoping that this will be the tool that would **[00:19:28 – 00:19:30]** be the LLM in the loop. **[00:19:30 – 00:19:32]** People say human in the loop, I would say LLM in the loop **[00:19:32 – 00:19:35]** because this is a very, very complicated field. **[00:19:36 – 00:19:38]** So we hope to build this fusion expert. **[00:19:38 – 00:19:40]** We are in the process of doing it and Geetakha will talk **[00:19:40 – 00:19:42]** more about the AI aspects of it. **[00:19:42 – 00:19:44]** We will train it. We are in the process of training it **[00:19:44 – 00:19:46]** to generate hypotheses. **[00:19:46 – 00:19:48]** And this will be the tool which will be connected to **[00:19:48 – 00:19:52]** other tools that are being developed at Los Alamos to ensure **[00:19:52 – 00:19:56]** there's a pipeline of easy use of simulations and post-processing **[00:19:56 – 00:19:57]** experiments and so on. **[00:19:58 – 00:20:00]** So this will be a hypothesis-generating **[00:20:00 – 00:20:04]** tool that will plug into the other tools, LLM-related tools, and other **[00:20:05 – 00:20:11]** AI tools at Los Alamos, which will be hopefully a seamless pipeline. **[00:20:11 – 00:20:13]** So what do we do for this, right? **[00:20:13 – 00:20:16]** So first of all, we need to teach this LLM about ICF, **[00:20:16 – 00:20:18]** and that's where the fine-tuning aspect of it comes in, and **[00:20:18 – 00:20:20]** Geetika will talk more about it. **[00:20:20 – 00:20:24]** I'll talk more about what the physics end of it is. **[00:20:24 – 00:20:28]** So we, as so-called subject matter experts, we've generated **[00:20:28 – 00:20:30]** question and answer pairs. **[00:20:30 – 00:20:34]** So there are a lot of questions, 250 questions **[00:20:34 – 00:20:37]** that we have generated as subject matter experts. **[00:20:37 – 00:20:39]** And this is going to be one of the metrics that we use **[00:20:39 – 00:20:45]** to assess what the fine-tuned LLM does before and after fine-tuning. **[00:20:45 – 00:20:46]** But that's not enough. **[00:20:46 – 00:20:49]** We need a lot more questions because there is a whole lot **[00:20:49 – 00:20:52]** of subject matter that's associated not just with fusion. **[00:20:52 – 00:20:55]** There are sub-subjects associated with it. **[00:20:55 – 00:20:56]** As I said, this is multi-physics. **[00:20:57 – 00:21:00]** So you need to understand how the laser interacts with the plasma. **[00:21:00 – 00:21:02]** You need to understand heat conduction, how **[00:21:02 – 00:21:03]** heat is transported. **[00:21:03 – 00:21:05]** You need to understand how radiation is transported. **[00:21:05 – 00:21:09]** So there are a lot of sub-subject matters in this field. **[00:21:09 – 00:21:13]** And so 250 questions ain't gonna cut it here, right? **[00:21:13 – 00:21:18]** And so what NVIDIA did was to take a subset of ICF literature and **[00:21:18 – 00:21:20]** generate a whole lot of questions. **[00:21:20 – 00:21:24]** And they gave us about, you know, 600 or so of them. **[00:21:24 – 00:21:27]** And we as subject matter experts went through all **[00:21:27 – 00:21:31]** of them and curated these questions, right, twice **[00:21:31 – 00:21:33]** independently by two scientists. **[00:21:33 – 00:21:36]** And we rewrote some of them, we eliminated some of them, **[00:21:36 – 00:21:40]** we rewrote some of the answers and so on, so we curated it. **[00:21:40 – 00:21:43]** So at the end of it all, we have about 700 questions or so, **[00:21:43 – 00:21:48]** which are very good questions that we can test a fine-tuned model on. **[00:21:48 – 00:21:52]** In addition, as subject matter experts, we had to come up with how **[00:21:52 – 00:21:57]** good is an LLM as a judge because obviously we as human beings do not **[00:21:57 – 00:22:01]** have the bandwidth to go through 700 or so of these questions **[00:22:01 – 00:22:03]** at every time it gets fine-tuned. **[00:22:03 – 00:22:07]** So we need an LLM that's capable of assessing these types of questions. **[00:22:07 – 00:22:11]** So NVIDIA again came up with a whole bunch of question and answer **[00:22:11 – 00:22:14]** pairs which were judged by an LLM. **[00:22:14 – 00:22:17]** And then we had to go in and assess whether the answers **[00:22:17 – 00:22:19]** were accurate enough or not. **[00:22:19 – 00:22:24]** And based on this, we down-selected to an LLM, which could potentially **[00:22:24 – 00:22:26]** serve as a judge. **[00:22:27 – 00:22:29]** So in addition to these metrics, there are, of course, the **[00:22:29 – 00:22:30]** standardized metrics. **[00:22:30 – 00:22:31]** So let me go through the whole series of these. **[00:22:31 – 00:22:34]** So there are these standardized benchmarks that any LLM is, **[00:22:34 – 00:22:36]** of course, assessed on. **[00:22:36 – 00:22:39]** There are these 250 subject matter expert-generated **[00:22:39 – 00:22:41]** question-answer pairs. **[00:22:41 – 00:22:44]** There are about these human-annotated question-answer **[00:22:44 – 00:22:49]** pairs, which have been down-selected from NVIDIA's list. **[00:22:49 – 00:22:53]** But the other strategy that we are following is to go through these **[00:22:53 – 00:22:58]** papers and holding back some papers and other papers that cite them. **[00:22:58 – 00:23:01]** And we have identified questions based on what we call these **[00:23:01 – 00:23:03]** hold-back papers. **[00:23:03 – 00:23:04]** And that's another metric. **[00:23:04 – 00:23:09]** Can the fine-tuned LLM actually answer questions in a conceptually **[00:23:09 – 00:23:14]** new area related to fusion, but which it hasn't learned **[00:23:14 – 00:23:17]** about explicitly through the fine-tuning process? **[00:23:17 – 00:23:20]** So there are a couple of those that we have, which are called **[00:23:20 – 00:23:21]** hold-back papers. **[00:23:21 – 00:23:25]** And they're actually pretty novel concepts in fusion that, **[00:23:25 – 00:23:28]** luckily, they're pretty recent papers, as you can see, from **[00:23:28 – 00:23:32]** 2022 and 2024, which means that they haven't quite penetrated **[00:23:32 – 00:23:35]** the landscape of AI fusion yet. **[00:23:35 – 00:23:38]** And so we're going to assess the LLM on whether it can actually **[00:23:38 – 00:23:41]** answer these questions or not. **[00:23:41 – 00:23:44]** Now the challenge with this field and the reason it takes **[00:23:44 – 00:23:48]** so long is of course partly because the laser is huge **[00:23:48 – 00:23:51]** and doesn't have great throughput, but the other reason is that **[00:23:51 – 00:23:54]** codes are not entirely predictive. **[00:23:54 – 00:23:55]** If they were predictive, we would have made progress **[00:23:56 – 00:23:56]** immediately, right? **[00:23:56 – 00:23:59]** The challenge is the codes are not predictive, largely **[00:23:59 – 00:24:03]** because the physics is not entirely known, and sometimes when you shoot **[00:24:03 – 00:24:07]** the laser, you know, not all beams fire, the target is not exactly **[00:24:07 – 00:24:09]** what you want, and so on, right? **[00:24:09 – 00:24:13]** So experiments are not entirely predicted by simulations, **[00:24:13 – 00:24:17]** and the ground truth is in the experimental data. **[00:24:17 – 00:24:21]** So what we hope to do is basically after we fine-tune this model **[00:24:21 – 00:24:24]** is to apply reinforcement learning and that will be **[00:24:24 – 00:24:29]** a very, very essential to make Prospero a unique tool for fusion. **[00:24:29 – 00:24:33]** And really, this tool would really be tested if we can design **[00:24:33 – 00:24:39]** an experiment that we can actually validate in a facility like the NIF **[00:24:39 – 00:24:42]** at the Lawrence Livermore National Laboratory or the Omega laser **[00:24:42 – 00:24:44]** at the University of Rochester. **[00:24:44 – 00:24:47]** And there are two approaches we are thinking about to do **[00:24:47 – 00:24:49]** reinforcement learning. **[00:24:49 – 00:24:50]** One is with human feedback. **[00:24:51 – 00:24:54]** Now, the other challenge about this field is that it has **[00:24:54 – 00:24:56]** grown quite organically. **[00:24:56 – 00:25:00]** So data is always not very well characterized, right? **[00:25:00 – 00:25:02]** So we do experiments and then we realize, ah, we should **[00:25:02 – 00:25:04]** have measured something else. **[00:25:04 – 00:25:07]** And then we build a new diagnostic to measure the data, right? **[00:25:07 – 00:25:09]** So data is always not very well characterized. **[00:25:09 – 00:25:12]** So that's another challenge with this field. **[00:25:12 – 00:25:15]** So we've gone through, sifted through the database that exists **[00:25:15 – 00:25:21]** and identified a few which are not quite AI ready, but we can make it **[00:25:21 – 00:25:23]** so in a reasonable amount of time. **[00:25:24 – 00:25:26]** So in the context of reinforcement learning with human feedback, **[00:25:26 – 00:25:28]** we have a whole bunch of documents. **[00:25:28 – 00:25:32]** Which outline what should be expected, which are called pre-shot **[00:25:32 – 00:25:36]** reports, we have post-shot reports, we have results from experiments. **[00:25:36 – 00:25:40]** And we hope to put these together in a framework that allows **[00:25:40 – 00:25:44]** somebody to identify what the strategy behind the campaign **[00:25:44 – 00:25:45]** is and what else we could do. **[00:25:46 – 00:25:49]** And finally, there's also this verifiable rewards, where **[00:25:49 – 00:25:55]** we intend to use experimental data to perform reinforcement learning. **[00:25:55 – 00:25:57]** And with that, I'll hand it over to Geetika, who'll talk **[00:25:57 – 00:25:59]** about the AI part of it. **[00:25:59 – 00:26:01]** Hello, everyone. **[00:26:01 – 00:26:02]** Okay, so my mic is working. **[00:26:03 – 00:26:04]** That's a good sign. **[00:26:04 – 00:26:10]** And so I have only a few minutes, so I will go over this quickly. **[00:26:10 – 00:26:13]** And I'm going to talk about, like, how we have, you know, **[00:26:13 – 00:26:18]** in this data-scarce domain, like, how do we build agentic workflows? **[00:26:18 – 00:26:21]** And the backbone for the agentic workflow is having **[00:26:21 – 00:26:24]** a domain-adapted reasoning model. **[00:26:24 – 00:26:26]** So I'm going to talk about like how you're going about building **[00:26:26 – 00:26:29]** this reasoning model for ICF. **[00:26:30 – 00:26:33]** So to build that you know the key list of ingredients **[00:26:33 – 00:26:37]** is like first you know we need like training data sets which we **[00:26:37 – 00:26:41]** can use for continual pre-training like giving knowledge back **[00:26:41 – 00:26:46]** to the model so that it understands ICF physics and like related **[00:26:46 – 00:26:52]** physics concepts data set to do SFT supervised fine tuning and then **[00:26:52 – 00:26:55]** for adding like reasoning concepts. **[00:26:55 – 00:26:59]** We rather talked about the evaluation metric, you know, **[00:26:59 – 00:27:02]** a lot of time, you know, the evaluation metric is never thought **[00:27:02 – 00:27:06]** about, like people think that all we need is a model and data set and **[00:27:06 – 00:27:10]** we train, but how do we know where we are going if we don't know? **[00:27:10 – 00:27:12]** If you don't have a clear evaluation metric. **[00:27:12 – 00:27:16]** So we spent a lot of time building this evaluation metric **[00:27:16 – 00:27:17]** along with the experts. **[00:27:17 – 00:27:21]** So the way this collaboration worked is that NVIDIA brought **[00:27:21 – 00:27:25]** their AI specialists and data scientists, and ICF specialists **[00:27:26 – 00:27:31]** came from LANL, and together we built this evaluation metric **[00:27:31 – 00:27:33]** that Radha talked about. **[00:27:33 – 00:27:36]** And we try to automate it in various spaces because **[00:27:36 – 00:27:39]** we don't want to take too many cycles from the domain specialist. **[00:27:39 – 00:27:42]** So, but, you know, this is just an example. **[00:27:42 – 00:27:45]** Like if people have to build reasoning models for other **[00:27:45 – 00:27:48]** scientific domains, like these are like some of the steps **[00:27:48 – 00:27:50]** that they will have to follow. **[00:27:50 – 00:27:54]** Then the third thing is identifying what's a good base model. **[00:27:54 – 00:27:59]** And base models, the pace of innovation in this field **[00:27:59 – 00:28:03]** is so fast, like six months or like 12 months, a new model comes up. **[00:28:03 – 00:28:08]** So even if we are starting with a particular model, by **[00:28:08 – 00:28:12]** the time we get everything else lined up, a new model may come up. **[00:28:12 – 00:28:16]** So we have to have a tool chain which allows us the flexibility **[00:28:16 – 00:28:22]** to swap in and try different models as they come along. **[00:28:22 – 00:28:24]** And that's why, you know, that brings me to the point **[00:28:24 – 00:28:27]** of, like, why having a good tool chain is important here. **[00:28:27 – 00:28:31]** And we have used extensively some of the tools which reside **[00:28:31 – 00:28:34]** inside the NeMo framework. **[00:28:34 – 00:28:39]** So we started with the data set collection and curation step. **[00:28:39 – 00:28:44]** We downloaded documents which were from archive and OSTI. **[00:28:44 – 00:28:50]** We put them through the NeMo data curator pipeline, which did the PDF **[00:28:50 – 00:28:56]** extraction, removed the duplicate documents, and then cleaned **[00:28:56 – 00:29:01]** up the data to create a token size of about 17 billion tokens. **[00:29:01 – 00:29:07]** And I would say this is a good amount of number of tokens that are **[00:29:07 – 00:29:12]** in the ballpark if one is trying to do a custom reasoning model **[00:29:12 – 00:29:15]** and going through the CPT step. **[00:29:15 – 00:29:19]** Then, you know, we worked on the evaluation metric that **[00:29:19 – 00:29:21]** Radha talked about earlier. **[00:29:21 – 00:29:23]** So we have three categories. **[00:29:23 – 00:29:27]** We have the golden set that came out of the SME **[00:29:28 – 00:29:30]** interaction, 250 questions. **[00:29:30 – 00:29:34]** We have the silver set with 550, about like 550 questions. **[00:29:34 – 00:29:39]** These were generated using the NeMo Data Designer. **[00:29:39 – 00:29:44]** Using the papers, about like 5,000 papers, prompting them **[00:29:44 – 00:29:46]** to generate question-answer pairs. **[00:29:46 – 00:29:51]** And we made sure those questions, those papers that we are using **[00:29:51 – 00:29:54]** for the evaluation are not included in the training data set. **[00:29:55 – 00:30:00]** So we have, we removed those particular papers so as to **[00:30:00 – 00:30:01]** keep the training. **[00:30:01 – 00:30:06]** Completely free of that knowledge because that's the only way to test **[00:30:06 – 00:30:09]** that the model has actually learned and is able to answer the questions **[00:30:10 – 00:30:15]** about concepts or can come up, think of answering questions about **[00:30:15 – 00:30:17]** things that it has not seen before. **[00:30:18 – 00:30:22]** And then we also have a combination of public physics benchmarks **[00:30:22 – 00:30:28]** because just having like 500 to 600 questions is sometimes not enough. **[00:30:28 – 00:30:31]** Now, we have, like, **[00:30:31 – 00:30:35]** Combining all these three, we have a really good evaluation **[00:30:35 – 00:30:40]** metric, which I would say can be used by any other model **[00:30:40 – 00:30:46]** to evaluate how good that model is for ICF reasoning. **[00:30:46 – 00:30:49]** And now, because we have this good evaluation metric, it **[00:30:49 – 00:30:52]** also enables us to swap models. **[00:30:52 – 00:30:57]** If you want to try Llama-Nemotron-49B, Or if **[00:30:57 – 00:31:00]** you want to try some other model, **[00:31:00 – 00:31:05]** Like the Nemotron Nano 3 that came out or the Nemotron 3 **[00:31:05 – 00:31:07]** Super that got released recently. **[00:31:08 – 00:31:11]** So we can try all these models because we have the pipeline **[00:31:11 – 00:31:14]** set up perfectly for that. **[00:31:14 – 00:31:17]** So this is like a summary slide which says, you know, **[00:31:17 – 00:31:20]** what are the steps that go into building a custom reasoning **[00:31:20 – 00:31:23]** model for ICF physics. **[00:31:23 – 00:31:27]** I talked about the evaluation metric and how we use the NeMo Data **[00:31:28 – 00:31:35]** Designer for it, how we took the documents and converted those into **[00:31:36 – 00:31:38]** tokens for continual pre-training. **[00:31:38 – 00:31:43]** Sometimes it's called DAPT, Domain Adaptive Pre-Training. **[00:31:43 – 00:31:47]** The step that we are doing then we also needed like SFT **[00:31:47 – 00:31:52]** data set so SFT data set is where we are teaching the model **[00:31:52 – 00:31:55]** on how to answer questions and sometimes teaching the model how **[00:31:55 – 00:32:02]** to reason so we have to build that SFT data set also so again over **[00:32:02 – 00:32:06]** here we are using the data designer to build those question answer **[00:32:06 – 00:32:11]** pairs with the reasoning traces that can be fed to the model. **[00:32:11 – 00:32:16]** So I know the slides will be available, so even though **[00:32:16 – 00:32:20]** I rush through this, you will be able to get this information. **[00:32:20 – 00:32:25]** But this is a pipeline that shows how we used the NeMo Data Designer **[00:32:25 – 00:32:26]** to build the reasoning traces. **[00:32:26 – 00:32:30]** The point over here is we used both reasoning on and **[00:32:31 – 00:32:34]** reasoning off when we were building this SFT dataset. **[00:32:34 – 00:32:36]** Because some of the questions are too basic and maybe the **[00:32:37 – 00:32:38]** model doesn't need to reason. **[00:32:38 – 00:32:41]** And in some places, it has to reason. **[00:32:41 – 00:32:45]** So that's why developing different kinds of traces and putting **[00:32:45 – 00:32:51]** the think tokens so that it can be used to impart both the knowledge **[00:32:51 – 00:32:53]** of factual and also when to reason. **[00:32:53 – 00:32:56]** So teaching the model when to reason and when not to **[00:32:56 – 00:33:00]** reason to make the best use of the number of tokens. **[00:33:01 – 00:33:04]** In terms of the data blend, you know, this is again an **[00:33:04 – 00:33:07]** art, it's almost like, okay, trying to find the right shade **[00:33:07 – 00:33:11]** of green or trying to find the right shade of orange and **[00:33:11 – 00:33:15]** we are going through this process right now, so we have generated **[00:33:15 – 00:33:21]** about like 9000 K question answer pairs, this is again just for SFT. **[00:33:21 – 00:33:23]** And we're trying different blends. **[00:33:23 – 00:33:28]** So if in the 900K question-answer pairs, if we were to say how **[00:33:28 – 00:33:32]** many are STEM-related, how many are non-STEM, or how many **[00:33:32 – 00:33:36]** are ICF-specific, and how many are other, like when I say other, it is **[00:33:36 – 00:33:42]** like other science domains and not just ICF, and then how many of them **[00:33:42 – 00:33:44]** are reasoning and non-reasoning. **[00:33:44 – 00:33:49]** Although over here it shows about 19% is ICF, we have built, I think **[00:33:49 – 00:33:55]** our goal is to get to almost like more than a million, like about **[00:33:55 – 00:33:57]** 1.1 million question-answer pairs. **[00:33:58 – 00:34:02]** So it'll be a really rich SFT data set, which will include **[00:34:03 – 00:34:05]** probably like 5,000... **[00:34:06 – 00:34:09]** 500k question-answer pairs for just ICF, like right now this data is **[00:34:10 – 00:34:15]** showing about like 130, but we have like about like we have generated **[00:34:15 – 00:34:19]** about like 550k question-answer pairs with reasoning without **[00:34:19 – 00:34:21]** reasoning just for ICF. **[00:34:21 – 00:34:26]** And again, we've been using NeMo framework to quickly **[00:34:26 – 00:34:28]** iterate on these things. **[00:34:28 – 00:34:30]** One of the interesting things we have noticed **[00:34:30 – 00:34:32]** is how the blend changes. **[00:34:32 – 00:34:35]** Like, if we have a certain blend which has more general **[00:34:35 – 00:34:41]** stem data, that model will do really good on GPQA and MMLU, **[00:34:41 – 00:34:44]** but on ICF it may not do well. **[00:34:44 – 00:34:46]** And that's what the right-hand side is showing. **[00:34:46 – 00:34:49]** But if we give it more ICF data, even if the data set **[00:34:49 – 00:34:54]** is a little less, or the ratio is right, then it does better **[00:34:54 – 00:34:58]** on the golden set that LANL will care about, but may not **[00:34:58 – 00:35:00]** do well on the public benchmarks. **[00:35:00 – 00:35:04]** So those are the kind of things that we are experimenting **[00:35:04 – 00:35:08]** and trying, and we hopefully will get, you know, a good **[00:35:08 – 00:35:11]** paper towards in the coming months. **[00:35:11 – 00:35:15]** Some of the lessons that we've learned, you know, data curation **[00:35:15 – 00:35:16]** takes a long time. **[00:35:16 – 00:35:22]** Investing in evaluation metric, you know, start that in parallel. **[00:35:22 – 00:35:26]** During the DAPT and SFT, we've had like some, you know, sometimes **[00:35:26 – 00:35:30]** the model accuracy will go down before it gets better. **[00:35:30 – 00:35:32]** So that's what we have seen. **[00:35:32 – 00:35:35]** These are the, you know, the graph over there is that's **[00:35:35 – 00:35:37]** what it's showing, like the model started at a higher **[00:35:37 – 00:35:40]** accuracy and then it went down. **[00:35:40 – 00:35:44]** And then as we gave it more SFT data, it has started rising up. **[00:35:44 – 00:35:48]** And again, you're seeing different blends of what is the right **[00:35:49 – 00:35:54]** mix of ICF versus non-ICF data, because we don't want the **[00:35:54 – 00:35:57]** model to forget the simple physics. **[00:35:57 – 00:36:01]** So we have to give it, overall, we are also giving coding **[00:36:01 – 00:36:06]** data, text data, and chemistry data, so that the model is **[00:36:06 – 00:36:10]** not forgetting the other parts of the science reasoning.