# Post-Training Open Models With Synthetic Data for Specialized Agents

## Abstract

Key takeaways Generate domain-specific training data from a schema and a few seed examples when real data is scarce, unlabeled, or too sensitive to move. Specialize an open model for a narrow task with post-training recipes, from supervised fine-tuning to RL, run as a repeatable job. Extend the same techniques to agent trajectories, generating the failure cases your logs haven’t produced yet.

## AI Summary

_No AI summary available._

## Transcript

**[00:00:05 – 00:00:08]** Welcome, everyone. I'm Rebecca, and this afternoon, we're going

**[00:00:08 – 00:00:11]** to deep dive into open models.

**[00:00:11 – 00:00:15]** Coming up next, our next session is post-training

**[00:00:15 – 00:00:19]** open models with synthetic data for specialized agents.

**[00:00:19 – 00:00:23]** And joining us are Yamini Kagal, Director of Product Management

**[00:00:23 – 00:00:29]** at NVIDIA, and Stefan Gavrilovic, Director of Engineering at NVIDIA.

**[00:00:29 – 00:00:34]** Please welcome Stefan and Yamini.

**[00:00:36 – 00:00:39]** Thanks, Rebecca. Thank you.

**[00:00:40 – 00:00:43]** Hi, everyone. Thank you, Rebecca.

**[00:00:43 – 00:00:48]** Like she said, I am Yamini Kagal.

**[00:00:48 – 00:00:54]** Me and my colleague Stefan, we work on open source AI

**[00:00:54 – 00:00:55]** libraries at NVIDIA.

**[00:00:55 – 00:00:59]** These are the NeMo family of libraries.

**[00:00:59 – 00:01:03]** We're here today to talk about using synthetic data

**[00:01:04 – 00:01:08]** to fine tune small models for specialized agents.

**[00:01:08 – 00:01:11]** Thank you for your patience.

**[00:01:12 – 00:01:16]** Let's start with what a specialized agent is.

**[00:01:16 – 00:01:22]** How many of you here are building specialized agents today?

**[00:01:23 – 00:01:27]** Okay, great. So, you know, hopefully this topic will

**[00:01:27 – 00:01:31]** resonate and that this is something that you've been experiencing.

**[00:01:31 – 00:01:33]** Obviously, you have generalized agents.

**[00:01:33 – 00:01:37]** These are your Cloud Code, Codex, Cursor.

**[00:01:37 – 00:01:43]** They're built using frontier models, which have a lot of

**[00:01:43 – 00:01:45]** general information.

**[00:01:45 – 00:01:49]** They're good, they know everything about the internet, they know

**[00:01:49 – 00:01:52]** a lot about a lot of topics, they're good at coding, but

**[00:01:52 – 00:01:58]** what they don't know is information that is specific to your company,

**[00:01:58 – 00:02:04]** to your domain, to your area of expertise, and that's when you need

**[00:02:04 – 00:02:09]** a specialized agent, when you're trying to do a lot of repeated

**[00:02:09 – 00:02:15]** tasks that are very specific to your particular problem space.

**[00:02:15 – 00:02:20]** Again, you need specialized agents when you have your

**[00:02:20 – 00:02:24]** own policy and rules that you want the agent to know something

**[00:02:24 – 00:02:28]** about, when you want to know whether it actually gets the job

**[00:02:28 – 00:02:34]** done based on your standards, not on sort of general best practices.

**[00:02:34 – 00:02:40]** So these are generally the times when you need a specialized

**[00:02:40 – 00:02:45]** agent, when the job has knowledge or capabilities that you can't

**[00:02:45 – 00:02:48]** get from a general model.

**[00:02:49 – 00:02:54]** So, basically, the practice that we've seen a lot of is

**[00:02:54 – 00:02:57]** that most companies, you know, and we do this at NVIDIA as

**[00:02:57 – 00:03:02]** well, we start off by building a general agent, we use a large

**[00:03:02 – 00:03:08]** frontier model, and it's running a bunch of jobs, but we're using the

**[00:03:08 – 00:03:11]** biggest model, and it works fine.

**[00:03:12 – 00:03:15]** When you get started, you find that for the most part,

**[00:03:15 – 00:03:18]** you know, your large model works fine until you get to

**[00:03:18 – 00:03:24]** the use cases where you actually need to start specializing.

**[00:03:24 – 00:03:28]** The most important thing really is to be able to evaluate

**[00:03:28 – 00:03:33]** your agent to understand when it's not doing the job properly.

**[00:03:33 – 00:03:40]** So every sort of practice here starts with very robust evals.

**[00:03:40 – 00:03:42]** Again, most of you are probably familiar with this.

**[00:03:42 – 00:03:46]** What we're trying to do here is set the stage so that we

**[00:03:46 – 00:03:50]** all have a common starting point as we walk through, demonstrate the

**[00:03:50 – 00:03:52]** actual fine-tuning process here.

**[00:03:52 – 00:03:56]** So, you want to be able to test your agent at every step

**[00:03:56 – 00:04:00]** to understand whether it's doing the job right, is it using

**[00:04:00 – 00:04:05]** the right model for that particular job, what are the trajectories,

**[00:04:05 – 00:04:11]** what are the tool calls that it's making, and to be able to score

**[00:04:11 – 00:04:16]** each of those steps along the way.

**[00:04:17 – 00:04:21]** There are four ways to improve a specialized agent.

**[00:04:22 – 00:04:25]** We don't want to say that fine-tuning is the only way.

**[00:04:25 – 00:04:29]** You can do prompt tuning, you can use RAG, you can use

**[00:04:29 – 00:04:30]** skills and tools.

**[00:04:30 – 00:04:33]** Each one has its place, and most of the time,

**[00:04:33 – 00:04:36]** you're using a combination

**[00:04:36 – 00:04:38]** of multiples of these tools.

**[00:04:38 – 00:04:42]** And again, what success really looks like, being able to

**[00:04:42 – 00:04:46]** evaluate all the different techniques you try.

**[00:04:46 – 00:04:49]** It's not so much about finding one technique and using that

**[00:04:49 – 00:04:51]** for all of your agents.

**[00:04:51 – 00:04:55]** You need to be able to evaluate for each agent, for each step,

**[00:04:55 – 00:05:00]** what's the right combination of techniques that you need to use.

**[00:05:01 – 00:05:05]** So today, we're going to be talking about fine-tuning.

**[00:05:05 – 00:05:08]** There are several strategies that you can use for doing

**[00:05:08 – 00:05:13]** prompt tuning and for doing retrieval, for optimizing

**[00:05:13 – 00:05:15]** skills, and so on as well.

**[00:05:16 – 00:05:19]** But fine-tuning is when you don't have written rules.

**[00:05:19 – 00:05:26]** You don't have, you know, knowledge that's stored in documents.

**[00:05:26 – 00:05:30]** You want the model to actually be able to learn and generalize

**[00:05:30 – 00:05:31]** that information.

**[00:05:31 – 00:05:35]** With RAG and with retrieval, often what happens is you're

**[00:05:35 – 00:05:37]** giving it, you know, instructions.

**[00:05:37 – 00:05:40]** It can go search and retrieve, but it tends to translate

**[00:05:40 – 00:05:42]** those very literally.

**[00:05:42 – 00:05:45]** It's not actually learning new information.

**[00:05:45 – 00:05:48]** So you want to be able to teach this model new information.

**[00:05:48 – 00:05:52]** And that's what we're going to be walking through today.

**[00:05:53 – 00:05:57]** Like I said here, you know, fine-tuning can be the right

**[00:05:57 – 00:06:01]** call when you have a task that is repeating at volume.

**[00:06:02 – 00:06:04]** The answer depends on knowledge that actually

**[00:06:04 – 00:06:05]** lives in past examples.

**[00:06:05 – 00:06:09]** So it's things that people are doing rather than telling,

**[00:06:09 – 00:06:13]** you know, you're not necessarily codifying this information

**[00:06:13 – 00:06:17]** anywhere, but it lives in traces, it lives in histories.

**[00:06:17 – 00:06:19]** And you want to be able to collect that and create your

**[00:06:19 – 00:06:25]** training data set to fine-tune a model downstream of that.

**[00:06:25 – 00:06:28]** You need it fast, you need a confidence score.

**[00:06:28 – 00:06:31]** We'll talk a little bit about that with this example that we picked

**[00:06:31 – 00:06:36]** today for doing the fine-tuning and why we picked that example.

**[00:06:36 – 00:06:40]** And sovereignty is a big reason for fine-tuning as well.

**[00:06:40 – 00:06:43]** When you want to be able to run these models on your

**[00:06:43 – 00:06:44]** own infrastructure.

**[00:06:44 – 00:06:49]** This is probably around your intellectual property, you don't

**[00:06:49 – 00:06:54]** want it leaving your environment, so this is also when you would

**[00:06:54 – 00:06:57]** probably choose fine-tuning over one of the other techniques,

**[00:06:57 – 00:07:01]** or over just using a big model.

**[00:07:02 – 00:07:05]** Here's the example that we picked.

**[00:07:05 – 00:07:07]** This is a made-up company called Meridian Compute.

**[00:07:08 – 00:07:10]** Today we're going to walk through all the steps, or rather

**[00:07:10 – 00:07:13]** Stefan's going to walk through the steps of how we did this.

**[00:07:13 – 00:07:19]** We have a repository and also we'll share these slides with

**[00:07:19 – 00:07:21]** you so, you know, you don't have to worry about that.

**[00:07:21 – 00:07:24]** We've got a repo on GitHub that you can download and

**[00:07:24 – 00:07:27]** actually go through the...

**[00:07:28 – 00:07:31]** There's a video recording.

**[00:07:31 – 00:07:33]** You just need to bring your own inference.

**[00:07:33 – 00:07:39]** This can be, you know, any OpenAI-compatible endpoints.

**[00:07:39 – 00:07:40]** Bring a GPU.

**[00:07:40 – 00:07:47]** We fine-tuned this model on a single GPU, you know, for about

**[00:07:47 – 00:07:53]** under 30 or 40 US dollars, so it was a pretty accessible experiment.

**[00:07:53 – 00:07:55]** What we wanted to do was,

**[00:07:55 – 00:07:58]** You know, we wanted to show how you could fine-tune a

**[00:07:59 – 00:08:00]** fast decision model.

**[00:08:00 – 00:08:03]** Everybody's probably heard about, you know, Jev and the model from

**[00:08:03 – 00:08:08]** TypeSafe and the System One models, that they're not traditional

**[00:08:08 – 00:08:10]** models that return text.

**[00:08:10 – 00:08:14]** They return a decision, and they give you a confidence score,

**[00:08:14 – 00:08:16]** and they do it very, very quickly.

**[00:08:16 – 00:08:22]** So, we looked at, for Meridian Compute, we said, let's imagine

**[00:08:22 – 00:08:24]** that it's got an incident agent.

**[00:08:24 – 00:08:28]** And it does triage, and incidents keep coming in, it has to

**[00:08:28 – 00:08:33]** decide whether to file a ticket or to call the, you know, to

**[00:08:33 – 00:08:38]** page the on-call engineer, and the goal here is you have a large model

**[00:08:38 – 00:08:42]** but it doesn't necessarily have the ability to make that decision of,

**[00:08:42 – 00:08:47]** you know, which team to route to and then whether or not to wake up.

**[00:08:48 – 00:08:57]** So, we wanted to see whether we could compare with a large model

**[00:08:57 – 00:09:01]** this fine-tuned decision model and see if it performed any better.

**[00:09:01 – 00:09:05]** One thing to note is, you know, everything is an experiment.

**[00:09:05 – 00:09:08]** You don't necessarily have... You don't know what the

**[00:09:08 – 00:09:09]** answer is going to be.

**[00:09:09 – 00:09:14]** So, what we encourage is do your evaluations, run experiments.

**[00:09:14 – 00:09:17]** Figure out what works best.

**[00:09:17 – 00:09:19]** It could be that fine tuning doesn't work and you're better

**[00:09:20 – 00:09:22]** off just using another small model.

**[00:09:22 – 00:09:25]** What you need is the ability to test all these different

**[00:09:25 – 00:09:30]** techniques and determine what works best for you.

**[00:09:30 – 00:09:34]** So the demo flow that we will go through here is we will

**[00:09:34 – 00:09:40]** test it with the same incident with the two agents.

**[00:09:40 – 00:09:43]** One is

**[00:09:44 – 00:09:47]** The same agent that's using the large model, and then

**[00:09:47 – 00:09:49]** the second one is the same agent for that step, which

**[00:09:49 – 00:09:51]** is using the small model.

**[00:09:51 – 00:09:52]** We'll build the data.

**[00:09:52 – 00:09:55]** We're going to build everything today using open source

**[00:09:55 – 00:09:57]** tools from NVIDIA.

**[00:09:57 – 00:09:59]** You don't have to use NVIDIA tools.

**[00:09:59 – 00:10:01]** You could also, although I hope I don't get fired for

**[00:10:01 – 00:10:07]** saying this, but you can also use other open source tools to do this.

**[00:10:07 – 00:10:11]** We want to show you how, you know, there is a process to be able

**[00:10:11 – 00:10:15]** to fine-tune your own models here.

**[00:10:15 – 00:10:16]** So we will build a data set.

**[00:10:16 – 00:10:20]** We have a open source synthetic data generation tool called

**[00:10:20 – 00:10:24]** Data Designer, which is like an agentic workflow.

**[00:10:24 – 00:10:27]** It allows you to build pretty complex data sets.

**[00:10:27 – 00:10:31]** In this instance, we're saying, let's assume we didn't have

**[00:10:31 – 00:10:32]** any data to start off with.

**[00:10:32 – 00:10:37]** If you had your own data, you had your traces from your live agent.

**[00:10:37 – 00:10:41]** You could convert those into training data, and we'll touch

**[00:10:41 – 00:10:45]** on that here at the end and use that to do your training

**[00:10:45 – 00:10:48]** as well, your fine-tuning.

**[00:10:48 – 00:10:50]** Then we're going to train the model and we're going

**[00:10:50 – 00:10:55]** to serve it up and then test it with the fine-tuned version

**[00:10:55 – 00:10:57]** and the un-fine-tuned version.

**[00:10:57 – 00:10:59]** And then finally, we'll score it.

**[00:10:59 – 00:11:03]** We'll show you how you can do evaluations.

**[00:11:04 – 00:11:08]** What we're going to check here is just making sure that the

**[00:11:08 – 00:11:12]** agent is using the right model for that step that we care about, that

**[00:11:12 – 00:11:18]** decision step, where it's routing the, it's classifying the incident

**[00:11:18 – 00:11:21]** and deciding how to route it.

**[00:11:22 – 00:11:25]** We talked about this already, you know, when do you use a chat model

**[00:11:25 – 00:11:30]** versus a decision model and, you know, what are some of the other

**[00:11:30 – 00:11:34]** decision models that are available.

**[00:11:34 – 00:11:38]** We're going to use another NVIDIA library called Helix

**[00:11:38 – 00:11:41]** today to also open source.

**[00:11:41 – 00:11:46]** It allows you to run this experiment and do the fine-tuning.

**[00:11:46 – 00:11:52]** And it exposes a lot of these other NeMo library

**[00:11:52 – 00:11:55]** functionalities as plugins.

**[00:11:55 – 00:11:59]** So you can basically plug in the functionality that you need.

**[00:11:59 – 00:12:03]** If you need data generation, you can add the data generation plugin.

**[00:12:04 – 00:12:05]** You can generate your data.

**[00:12:05 – 00:12:06]** Believe God.

**[00:12:06 – 00:12:09]** We have a plugin for post-training that actually

**[00:12:09 – 00:12:10]** uses several libraries.

**[00:12:11 – 00:12:13]** In this instance, we're going to use Unslot because we wanted

**[00:12:13 – 00:12:16]** to train on a single GPU.

**[00:12:16 – 00:12:20]** But you can also use NeMo Auto Model that allows you

**[00:12:20 – 00:12:24]** to train on, you know, if you need to scale out, depending on the

**[00:12:24 – 00:12:26]** techniques that you want to use.

**[00:12:26 – 00:12:28]** So, you know, you can kind of...

**[00:12:28 – 00:12:30]** It's almost like you can pick your own adventure,

**[00:12:30 – 00:12:32]** and you you'll get this.

**[00:12:32 – 00:12:36]** There's a backbone that makes it easy for you to wire together

**[00:12:36 – 00:12:38]** all of these steps.

**[00:12:38 – 00:12:41]** If you didn't have this, you would be basically doing all

**[00:12:42 – 00:12:46]** the wiring together yourself, using individual libraries, handing data

**[00:12:46 – 00:12:49]** back and forth, and then building your own, or using weights and

**[00:12:49 – 00:12:59]** biases or something like that to do the sweeps and to view the results.

**[00:12:59 – 00:13:04]** Quick thing here is you do not need any keys or access

**[00:13:04 – 00:13:05]** or anything like that.

**[00:13:05 – 00:13:06]** Everything is open.

**[00:13:06 – 00:13:10]** You can use our repo, just download the whole thing and run it.

**[00:13:10 – 00:13:13]** You don't have to use anything else.

**[00:13:13 – 00:13:17]** Helix is not an agent building platform or anything like that.

**[00:13:17 – 00:13:20]** You can use your existing harness of choice, you know,

**[00:13:20 – 00:13:24]** Lanchain Deep Agents or whatever else you've used.

**[00:13:24 – 00:13:26]** You build your agent wherever you want.

**[00:13:26 – 00:13:29]** Then you register it in Helix.

**[00:13:29 – 00:13:31]** Using just a simple YAML file, there's another tool

**[00:13:32 – 00:13:36]** called NeMo Fabric which helps handle the registration

**[00:13:36 – 00:13:38]** and the agent lifecycle.

**[00:13:38 – 00:13:43]** So you register it and then you deploy it.

**[00:13:43 – 00:13:45]** It gives you the ability to test your agent and so on.

**[00:13:45 – 00:13:49]** So this is almost like a way for you to test and optimize

**[00:13:49 – 00:13:53]** and secure your agents before you deploy them to production.

**[00:13:53 – 00:14:00]** It's kind of a shift left system, so it's for pre-deployment.

**[00:14:01 – 00:14:07]** Very quickly here, before I hand over to Stefan, this

**[00:14:07 – 00:14:11]** is as easy as, this is what it takes to install NeMo Helix,

**[00:14:11 – 00:14:15]** it's just a UV tool install.

**[00:14:15 – 00:14:20]** Here we are registering, it's two calls, we are registering the

**[00:14:20 – 00:14:25]** agent and then we are deploying it.

**[00:14:25 – 00:14:30]** We have a UI, it's called NeMo Studio, also open source,

**[00:14:30 – 00:14:35]** it's bundled with NeMo Helix and it just gives you a way to

**[00:14:35 – 00:14:42]** look at your agents and test them in a lightweight UI environment.

**[00:14:42 – 00:14:48]** There's also CLI, and a lot of our examples here, we were just

**[00:14:48 – 00:14:51]** using a coding assistant because, you know, that's just easy.

**[00:14:51 – 00:14:54]** You can use Py, you can use Cloud Code, but you can also

**[00:14:54 – 00:15:00]** use the CLI, and the UI is useful for visualizing all

**[00:15:00 – 00:15:04]** of your experiments and for testing your agent and so on.

**[00:15:04 – 00:15:08]** Here, I'm back to using Claude code.

**[00:15:08 – 00:15:09]** I am actually testing.

**[00:15:09 – 00:15:11]** So I brought up the agent.

**[00:15:11 – 00:15:16]** The agent is currently using Nemo Ultra as the large language

**[00:15:17 – 00:15:19]** model to do all of its tasks.

**[00:15:19 – 00:15:26]** I sent it a simple Slack message to say, one of the pods is slow.

**[00:15:26 – 00:15:28]** I need help.

**[00:15:29 – 00:15:30]** I did give it a prompt.

**[00:15:30 – 00:15:37]** So this is not a large model with no knowledge at all of

**[00:15:37 – 00:15:39]** my system, but I'm using a prompt.

**[00:15:39 – 00:15:42]** I'm using rules right now to give it a general indication

**[00:15:42 – 00:15:44]** of how to do the routing.

**[00:15:44 – 00:15:47]** In this instance, it didn't get the routing right, and

**[00:15:48 – 00:15:49]** it just paged the on-call engineer.

**[00:15:49 – 00:15:54]** So that's not the goal that we wanted.

**[00:15:54 – 00:16:00]** What we have here is the... the decision call, so this is

**[00:16:00 – 00:16:01]** the one that we're going to swap.

**[00:16:01 – 00:16:03]** We're not going to swap the rest of the agent.

**[00:16:03 – 00:16:05]** The agent has a large number of steps.

**[00:16:05 – 00:16:10]** For this experiment, we're keeping the model the same

**[00:16:10 – 00:16:12]** for all of the other steps.

**[00:16:12 – 00:16:14]** We're just swapping out that one decision call.

**[00:16:14 – 00:16:18]** This is, again, just to keep this experiment constrained so it's

**[00:16:19 – 00:16:21]** easy for you to see the benefits.

**[00:16:21 – 00:16:24]** Otherwise, we would be moving too many different things,

**[00:16:24 – 00:16:30]** and that would, you know, impact the results.

**[00:16:30 – 00:16:36]** So here, we had the same agent, but we swapped that step out

**[00:16:36 – 00:16:38]** with the fine-tuned model.

**[00:16:38 – 00:16:41]** And this opened the ticket and didn't wake up the

**[00:16:41 – 00:16:42]** on-call engineer.

**[00:16:42 – 00:16:46]** So the on-call engineer loves this new system, because it's

**[00:16:46 – 00:16:49]** not constantly waking them up.

**[00:16:50 – 00:16:54]** So this is how we're actually going to show you, what Stefan's

**[00:16:54 – 00:16:58]** going to show you is the steps that we went through to generate the

**[00:16:58 – 00:17:03]** data all the way through to running the agent using the different

**[00:17:03 – 00:17:06]** tools that we have in Nemo Helix.

**[00:17:06 – 00:17:07]** Back to you, Stefan.

**[00:17:07 – 00:17:10]** Thanks, Yamini; thanks, everyone for coming out.

**[00:17:10 – 00:17:13]** So, I'm actually gonna jump ahead to the results here

**[00:17:13 – 00:17:14]** before I walk through the method.

**[00:17:14 – 00:17:19]** And so here you can see, we tested a variety of models

**[00:17:19 – 00:17:20]** and we actually tested them on two different things.

**[00:17:20 – 00:17:23]** They select the right team to page, they select the right action.

**[00:17:23 – 00:17:26]** And 93% of the time, every single model we tested, at least

**[00:17:26 – 00:17:28]** 93% of the time got the team right.

**[00:17:28 – 00:17:29]** So that part was easy.

**[00:17:29 – 00:17:31]** The part that gets trickier is the action.

**[00:17:31 – 00:17:33]** And so the action depends on a private internal

**[00:17:33 – 00:17:35]** policy from Meridian.

**[00:17:35 – 00:17:38]** And that one, you can see here, even if we provide that

**[00:17:38 – 00:17:42]** policy to Nemotron 3 Ultra, they got it right 70% of the time.

**[00:17:42 – 00:17:45]** However, after post-training, Quen 3.5.4b as well as Nemotron Nano

**[00:17:45 – 00:17:48]** 9b, they both got over 80% there.

**[00:17:52 – 00:17:56]** And so fine-tuning here beats providing the policy directly

**[00:17:56 – 00:17:58]** even to a much larger model.

**[00:17:58 – 00:18:01]** We ran that fine-tuning three times for each model just

**[00:18:01 – 00:18:05]** to make sure we didn't get lucky, and the results held up.

**[00:18:05 – 00:18:07]** And so how do we choose that model?

**[00:18:07 – 00:18:10]** Always tricky, we sort of fine-tuned; we sort of shortlisted

**[00:18:10 – 00:18:14]** a handful that we knew were in the right size range for us.

**[00:18:14 – 00:18:17]** We measured them; they were all sort of at a similar

**[00:18:17 – 00:18:20]** baseline, around 36% for the three we looked at.

**[00:18:20 – 00:18:22]** We then looked at tool compatibility, and so for

**[00:18:22 – 00:18:26]** the trainers we had available, the decider model was not compatible.

**[00:18:26 – 00:18:28]** We sort of dropped that one from our list.

**[00:18:28 – 00:18:30]** And then we looked at, you know, of the other two, we

**[00:18:30 – 00:18:33]** tried both of those, we got both of them above 80%.

**[00:18:33 – 00:18:35]** And here we have a sort of a series of things you can

**[00:18:35 – 00:18:38]** check while you're, you know, trying to decide the model.

**[00:18:38 – 00:18:40]** Does it match the license you're looking for?

**[00:18:41 – 00:18:44]** Does it match the size and the GPU hardware that you're

**[00:18:44 – 00:18:45]** looking to run it on?

**[00:18:45 – 00:18:49]** Does it match models you're already using in your environment?

**[00:18:49 – 00:18:52]** So some general rules on how you may want to select the

**[00:18:52 – 00:18:55]** model to fine tune when you're looking to fine tune.

**[00:18:55 – 00:18:57]** So Data Designer, Yamini mentioned this.

**[00:18:57 – 00:19:01]** This is NVIDIA's open-source data generation library.

**[00:19:01 – 00:19:03]** The way it works is it breaks the data set up into columns.

**[00:19:03 – 00:19:06]** Each column is specified either by a sampler, an

**[00:19:06 – 00:19:09]** LLM, a tool, or a validator.

**[00:19:10 – 00:19:13]** And here, we're going to create two LLM-generated columns,

**[00:19:13 – 00:19:15]** which is the judge.

**[00:19:15 – 00:19:17]** And the judge judges the writer column.

**[00:19:17 – 00:19:19]** The writer column is going to be what writes the incident for us.

**[00:19:19 – 00:19:23]** So we're going to generate a data set of synthetic incidents,

**[00:19:24 – 00:19:28]** each with the action and team pairs that we're going to train with.

**[00:19:28 – 00:19:31]** And so going through the demo really quickly here,

**[00:19:31 – 00:19:34]** you can see the Data Designer recipe for the two models,

**[00:19:34 – 00:19:36]** the writer and the judge.

**[00:19:36 – 00:19:38]** Each one uses a different model under the hood to

**[00:19:38 – 00:19:41]** make sure that basically this improves data quality.

**[00:19:41 – 00:19:44]** If your judge is the same model as the writer, they

**[00:19:44 – 00:19:45]** like what they wrote.

**[00:19:45 – 00:19:48]** So you want to have two different models there.

**[00:19:48 – 00:19:50]** Here, we're showing the sampler set, and so here

**[00:19:50 – 00:19:53]** we add the runbook, the severity, the actions, and

**[00:19:53 – 00:19:55]** things like that as the samplers.

**[00:19:55 – 00:19:57]** And you actually specify the answer first.

**[00:19:57 – 00:19:59]** So here we're saying the answer is going to be this action,

**[00:19:59 – 00:20:03]** and then based on that and these other sample columns,

**[00:20:03 – 00:20:06]** following the policy, we provide all of that to a prompt instruction

**[00:20:06 – 00:20:10]** to the writer, and the writer writes the incident for us.

**[00:20:10 – 00:20:13]** We then run our checks and our judges on every row.

**[00:20:13 – 00:20:15]** And the main thing we're looking for here, and we'll go into

**[00:20:15 – 00:20:17]** more detail on this later on, is are we giving away

**[00:20:17 – 00:20:19]** the answer within the incident?

**[00:20:19 – 00:20:22]** So you don't want the incident text to say, hey, page the engineer

**[00:20:22 – 00:20:27]** for this, because then your results aren't going to be valid.

**[00:20:27 – 00:20:30]** And so here, Cloud Code is going to be using the data

**[00:20:30 – 00:20:34]** designer skill to take that recipe and run it for us.

**[00:20:35 – 00:20:37]** For this example, we're going to generate 200 incidents.

**[00:20:37 – 00:20:41]** We're going to run a job on Helix, and the judge is going

**[00:20:41 – 00:20:42]** to score those 200 rows.

**[00:20:42 – 00:20:45]** And here you can see that Cloud Code has called out that all of

**[00:20:45 – 00:20:48]** the rows with leaked data have been dropped from the data set, so that

**[00:20:48 – 00:20:52]** when we do our fine tuning, those rows don't affect the results.

**[00:20:52 – 00:20:56]** Here you can see the same data set within the Nemo Studio

**[00:20:56 – 00:20:58]** UI that Yamini showed earlier.

**[00:20:58 – 00:21:00]** You're able to pull up a single row.

**[00:21:00 – 00:21:02]** You can see all the different fields, the severity, the

**[00:21:02 – 00:21:05]** action taken, as well as the incident text at the bottom there.

**[00:21:05 – 00:21:08]** And then you can also see statistics for the entire

**[00:21:08 – 00:21:09]** synthetic data set.

**[00:21:09 – 00:21:12]** So this is all of the different rows and columns and what

**[00:21:12 – 00:21:14]** the uniqueness percentages and things, things like that

**[00:21:14 – 00:21:17]** are across that data set.

**[00:21:17 – 00:21:23]** Okay, and here we have the config that we use, so that recipe

**[00:21:23 – 00:21:26]** that we use to generate that data set is also stored in Helix, and

**[00:21:26 – 00:21:30]** so you can reference it later, you can rerun the same configuration

**[00:21:30 – 00:21:33]** for a larger data set and scale up.

**[00:21:33 – 00:21:35]** And yeah, I mentioned data validation is key.

**[00:21:35 – 00:21:39]** So we basically, the judge picks up, you know, most of

**[00:21:39 – 00:21:40]** the mistakes here.

**[00:21:40 – 00:21:42]** It catches, hey, we've given the action away, we've given the

**[00:21:42 – 00:21:44]** answer away in the question itself.

**[00:21:44 – 00:21:45]** We don't like that.

**[00:21:45 – 00:21:49]** And so it was able to catch 620 errors in the 3,600 synthetic

**[00:21:49 – 00:21:51]** records we generated.

**[00:21:51 – 00:21:55]** And then we had a deterministic check, regex at the end, that,

**[00:21:55 – 00:21:58]** that caught another 136 that the judge had missed.

**[00:21:58 – 00:22:01]** So important thing to keep in mind, don't fully trust

**[00:22:01 – 00:22:04]** the judge; do as much deterministic checking for your data as you

**[00:22:04 – 00:22:07]** possibly can for quality reasons.

**[00:22:08 – 00:22:12]** And so the synthetic data here got us somewhere day one, right?

**[00:22:12 – 00:22:14]** Like this took a few hours end to end, like Yamini said,

**[00:22:14 – 00:22:15]** for the whole thing.

**[00:22:15 – 00:22:17]** It gets us somewhere day one, it gets us pretty good results.

**[00:22:17 – 00:22:20]** If you want to take this to production, you want to hit

**[00:22:20 – 00:22:23]** a few more points here using your real production data.

**[00:22:23 – 00:22:27]** You know, you want to catch the full gamut of cases, all your edge

**[00:22:27 – 00:22:30]** cases, your real production cases.

**[00:22:31 – 00:22:33]** You want to have some level of human checks, whether it's

**[00:22:33 – 00:22:36]** spot checking or labeling, and you want to make sure

**[00:22:36 – 00:22:38]** that things evolve over time.

**[00:22:38 – 00:22:42]** So test sets that happen after your training data, holdout

**[00:22:42 – 00:22:47]** sets, retraining as policies change, that sort of thing.

**[00:22:48 – 00:22:52]** Moving on from data to fine-tuning, we were able to fine-tune

**[00:22:52 – 00:22:55]** the QEN model in 22 minutes on a single A100 GPU.

**[00:22:56 – 00:23:00]** We used the Helix post-training plugin with the unslothed backend.

**[00:23:00 – 00:23:05]** We also support the NeMo Automodel backend for SFT fine-tuning, which

**[00:23:05 – 00:23:09]** can do multi-node GPU fine-tuning.

**[00:23:09 – 00:23:15]** Here we just did a single GPU LoRa SFT tuning on that model.

**[00:23:15 – 00:23:18]** We also support NeMo RL for reinforcement learning, and

**[00:23:18 – 00:23:22]** here the loss, you can see, went from 0.47 to 0.1.

**[00:23:24 – 00:23:26]** And just walking through quickly how we did that,

**[00:23:26 – 00:23:29]** our training records are the question and answer pairs.

**[00:23:29 – 00:23:30]** The question is the incident itself.

**[00:23:30 – 00:23:33]** The answer is which action is to be taken.

**[00:23:33 – 00:23:36]** We upload that data set to Helix as a file set.

**[00:23:36 – 00:23:40]** We run a job pointing at that file set.

**[00:23:40 – 00:23:46]** And you can see here, once it's done, the loss has fallen to 0.1.

**[00:23:46 – 00:23:50]** Again, in NeMo Studio, you can see our loss graph there.

**[00:23:50 – 00:23:54]** You can see other statistics for learning rate and how long it took.

**[00:23:54 – 00:23:57]** So you can see here, 21 minutes or so.

**[00:23:57 – 00:23:59]** And then after it's been fine-tuned, we deploy

**[00:23:59 – 00:24:02]** both the original model and the LoRa adapter.

**[00:24:02 – 00:24:05]** They're deployed side by side on the same GPU, and this

**[00:24:05 – 00:24:07]** is done for evaluation purposes.

**[00:24:07 – 00:24:10]** We initially take just a single test run.

**[00:24:10 – 00:24:13]** We send it one incident, same one Yamini showed earlier.

**[00:24:13 – 00:24:15]** The untrained model takes no action, which then means

**[00:24:15 – 00:24:19]** it's escalated to the on-call engineer, and the specialized

**[00:24:19 – 00:24:23]** model files a ticket.

**[00:24:23 – 00:24:28]** So moving on to evaluation, most important step here,

**[00:24:28 – 00:24:29]** probably in my opinion.

**[00:24:29 – 00:24:32]** Some very generic steps on evaluation.

**[00:24:33 – 00:24:35]** You want to make sure that you're measuring a baseline equivalent

**[00:24:35 – 00:24:38]** to what your real production is, so as close as possible.

**[00:24:38 – 00:24:41]** You want to give the frontier model as much data as possible.

**[00:24:41 – 00:24:44]** You don't want to gimp it just to show improvement.

**[00:24:44 – 00:24:47]** And so we went from 45% to 70% here

**[00:24:47 – 00:24:50]** just by giving Ultra the policies.

**[00:24:50 – 00:24:53]** You want to make sure you're testing on scenarios that

**[00:24:53 – 00:24:54]** it never trained on.

**[00:24:54 – 00:24:58]** So make sure you have a proper holdout data set.

**[00:24:58 – 00:25:01]** I mentioned earlier, we trained three times just to make sure

**[00:25:01 – 00:25:04]** we didn't get lucky with a specific seed on our first run.

**[00:25:04 – 00:25:07]** And then you want to measure the system as a whole, so not just

**[00:25:07 – 00:25:10]** the model, but the entire agent.

**[00:25:11 – 00:25:12]** And we'll go through this super quickly.

**[00:25:12 – 00:25:15]** This is, again, Cloud Code picks up the Helix eval skill.

**[00:25:15 – 00:25:19]** It registers an experiment in Studio for comparison purposes.

**[00:25:19 – 00:25:20]** Here you can see the UI.

**[00:25:20 – 00:25:23]** Each agent is run within the same experiment.

**[00:25:23 – 00:25:26]** You can see the scores for each one.

**[00:25:26 – 00:25:30]** You can jump into the test cases for the full evaluation

**[00:25:30 – 00:25:33]** set, the individual test cases, how they score.

**[00:25:33 – 00:25:35]** And if you pull up a single test case there, you can sort

**[00:25:35 – 00:25:40]** of see side by side, on the left side, the, um, the original

**[00:25:40 – 00:25:43]** agent paged; on the right side, it filed a ticket, and so you

**[00:25:43 – 00:25:47]** can see what did it choose to do.

**[00:25:47 – 00:25:50]** Really quickly, Yamini mentioned confidence earlier, so the

**[00:25:50 – 00:25:53]** system one decision models, they operate on this idea of confidence

**[00:25:53 – 00:25:54]** and how do you leverage it.

**[00:25:54 – 00:25:56]** So the way we've leveraged it here is we were able to

**[00:25:56 – 00:25:59]** measure that if the confidence returned by the model is 90%

**[00:25:59 – 00:26:04]** or higher, then it gets the correct answer 98% of the time.

**[00:26:05 – 00:26:08]** And so what we've done here is, as a safety user trust threshold,

**[00:26:08 – 00:26:13]** we've said if the confidence is below 90%, we're going to escalate

**[00:26:13 – 00:26:15]** to the on-call engineer anyway.

**[00:26:15 – 00:26:17]** And so that basically means that the edge cases where we're not

**[00:26:17 – 00:26:20]** sure if we're getting it correct, we're going the safer route and

**[00:26:20 – 00:26:23]** escalating to the on-call engineer.

**[00:26:24 – 00:26:27]** And on top of that quality bump that we got by fine-tuning

**[00:26:27 – 00:26:29]** this model, it's actually faster and cheaper to run,

**[00:26:29 – 00:26:31]** particularly at scale.

**[00:26:31 – 00:26:34]** If you look at the model response time there, you can see the

**[00:26:34 – 00:26:35]** typical response time.

**[00:26:35 – 00:26:40]** So the P50 is cut in half, basically, for the response.

**[00:26:40 – 00:26:44]** And the P95 time is improved significantly more than that.

**[00:26:44 – 00:26:47]** This is for that model and that part of the incident

**[00:26:47 – 00:26:48]** response flow specifically.

**[00:26:48 – 00:26:51]** Within the full agent response flow, this translates

**[00:26:51 – 00:26:55]** to a 3% reduction in the P50 time and roughly a

**[00:26:55 – 00:26:58]** 15% reduction in the P95 time.

**[00:26:58 – 00:27:01]** On the cost side, you can see that hosted on OpenRouter,

**[00:27:01 – 00:27:06]** Nemotron 3 Ultra costs about$ 0.72 for 1,000 answers.

**[00:27:07 – 00:27:10]** For the custom models, their hourly rates, they need to

**[00:27:10 – 00:27:11]** be hosted on GPUs.

**[00:27:11 – 00:27:13]** Because we train such a small model, it's a four billion

**[00:27:13 – 00:27:17]** parameter model, it can be hosted on a wide variety of NVIDIA GPUs.

**[00:27:17 – 00:27:21]** I've listed three of them here with their rates from Vast AI.

**[00:27:22 – 00:27:25]** And even the slowest one here can generate well over a thousand

**[00:27:25 – 00:27:26]** answers per hour.

**[00:27:26 – 00:27:29]** And so the break-even point here is actually a relatively

**[00:27:29 – 00:27:30]** small level of scale.

**[00:27:30 – 00:27:34]** And if you're using any of the larger GPUs here, the A100s,

**[00:27:34 – 00:27:35]** you can host multiple models.

**[00:27:35 – 00:27:37]** You can host a single model with multiple LoRa adapters

**[00:27:38 – 00:27:39]** for multiple use cases.

**[00:27:39 – 00:27:44]** So that's how cost, speed, and ease serving comes into play.

**[00:27:44 – 00:27:47]** And back to Yamini.

**[00:27:50 – 00:27:52]** Thanks, Stefan.

**[00:27:53 – 00:27:56]** Just a couple of things here to, you know, and then we'll open

**[00:27:56 – 00:28:00]** it up for question and answers.

**[00:28:00 – 00:28:04]** There are a few other techniques that you can try.

**[00:28:04 – 00:28:07]** Routing is one that there has been a lot of interest in.

**[00:28:07 – 00:28:11]** Helix does support Nemo Switchyard as a plugin.

**[00:28:11 – 00:28:16]** Again, you want to be able to test your routing strategies

**[00:28:16 – 00:28:20]** before you decide to deploy it in production, so you can figure

**[00:28:20 – 00:28:25]** out which family of models you want to route between, whether you

**[00:28:25 – 00:28:30]** want to be able to route between various, like larger and smaller

**[00:28:30 – 00:28:34]** models for different types of tasks based on quality and so on.

**[00:28:34 – 00:28:38]** So this is something that we recommend that you test

**[00:28:38 – 00:28:43]** with as well, and you can do it pretty easily in Switchyard.

**[00:28:43 – 00:28:49]** We have a QR code there, so you can go check out the repo as well.

**[00:28:49 – 00:28:52]** So all of these libraries are available as individual

**[00:28:52 – 00:28:56]** libraries, or you can string them together in Nemo Helix as plug-ins.

**[00:28:56 – 00:28:59]** These are all things.

**[00:28:59 – 00:29:04]** Another library that you might be interested in that we just recently

**[00:29:04 – 00:29:07]** released is called UserSim.

**[00:29:07 – 00:29:10]** This is useful for generating a large

**[00:29:10 – 00:29:14]** number of user-simulated traces.

**[00:29:14 – 00:29:19]** Again, we find that nothing beats having live traces.

**[00:29:19 – 00:29:24]** But often, you might not have the variety of traces that you need.

**[00:29:24 – 00:29:27]** You may not have edge cases.

**[00:29:27 – 00:29:31]** Maybe as agents get more and more complex, you want to be

**[00:29:31 – 00:29:34]** able to try complex conversations.

**[00:29:34 – 00:29:38]** This is an easy way to test those out.

**[00:29:38 – 00:29:43]** Like you can see here, you have different roles that you can give

**[00:29:43 – 00:29:48]** the library, and it can generate those conversations, like, you

**[00:29:48 – 00:29:54]** know, an impatient on-call engineer or a vague manager or a new hire.

**[00:29:54 – 00:29:59]** Even for something like the incident agent, this could be

**[00:29:59 – 00:30:02]** useful to help generate some data.

**[00:30:02 – 00:30:06]** And then, finally, you would take all of these traces,

**[00:30:06 – 00:30:11]** your existing production traces, your synthetic traces, and turn

**[00:30:11 – 00:30:14]** that into your training data set.

**[00:30:14 – 00:30:18]** We showed you an example for training the fast decision

**[00:30:18 – 00:30:22]** model that was only two columns.

**[00:30:22 – 00:30:25]** It was a smaller data set.

**[00:30:25 – 00:30:30]** But obviously, for training models for different use cases,

**[00:30:30 – 00:30:36]** you might need more data, and you can follow these processes.

**[00:30:36 – 00:30:37]** We're building this functionality.

**[00:30:37 – 00:30:41]** It's not available yet, but there are individual tools for doing it.

**[00:30:41 – 00:30:45]** Data Designer is one of those for filling in gaps.

**[00:30:45 – 00:30:49]** We have a tool called NeMo Curator, which is for data

**[00:30:49 – 00:30:50]** processing and data prep.

**[00:30:50 – 00:30:55]** It can convert traces into training data as well.

**[00:30:55 – 00:30:59]** And we do recommend going through this process for generating

**[00:30:59 – 00:31:03]** your training data.

**[00:31:03 – 00:31:07]** Finally, sort of next steps is, you know, you

**[00:31:07 – 00:31:08]** can follow along yourself.

**[00:31:08 – 00:31:11]** We have a link to the repository.

**[00:31:11 – 00:31:12]** You can download it.

**[00:31:12 – 00:31:15]** Identify an agent that maybe you're interested.

**[00:31:15 – 00:31:22]** Thank you very much, everyone, for taking the time to join us today.

**[00:31:36 – 00:31:40]** Yes, so with the decision models, what I saw usually

**[00:31:40 – 00:31:44]** is one of the big features is that they are single forward pass.

**[00:31:44 – 00:31:47]** Now, from what you've shown, it looks more like you've

**[00:31:47 – 00:31:52]** fine-tuned simply the LLM into a structured output,

**[00:31:52 – 00:32:00]** and is there an option to also train, like, adapter, or with the

**[00:32:00 – 00:32:04]** structured output, how do you set up the loss that you actually get

**[00:32:04 – 00:32:06]** the confidence score that you do?

**[00:32:06 – 00:32:11]** Because with the head, you can run the typical, like,

**[00:32:11 – 00:32:16]** binary cross entropy, and that makes it easy, but just

**[00:32:16 – 00:32:19]** interested in the process there.

**[00:32:19 – 00:32:23]** Yeah, so this doesn't work identically to Jev and some

**[00:32:23 – 00:32:26]** of the models, which nobody knows exactly how they work.

**[00:32:26 – 00:32:30]** You can take a diffusion model, so a lot of the open-source decision

**[00:32:30 – 00:32:33]** models have been fine-tunes of Gemma diffusion and things

**[00:32:33 – 00:32:35]** like that, so that is supported.

**[00:32:35 – 00:32:38]** You can, through Unsloth and some of these other libraries,

**[00:32:38 – 00:32:40]** you are able to fine-tune a diffusion model as well.

**[00:32:41 – 00:32:42]** It would be a slightly different approach.

**[00:32:42 – 00:32:45]** So here we fine-tune a very small LLM to produce the same

**[00:32:45 – 00:32:49]** outputs as something like Jev, but it is, yeah, you're

**[00:32:49 – 00:32:53]** absolutely right, the architecture of the model underneath that

**[00:32:53 – 00:32:55]** is a different shape.

**[00:32:55 – 00:32:59]** Yeah, and then some of them are also like Lya and Strands.

**[00:32:59 – 00:33:01]** They have their own ways of fine-tuning.

**[00:33:01 – 00:33:06]** Our goal here was to see how close can we get to the same result.

**[00:33:06 – 00:33:10]** By the way, we did release a, one of our data scientists

**[00:33:10 – 00:33:15]** built a Nemotron-flavored decider model based on a diffusion model.

**[00:33:15 – 00:33:19]** I think it's called Nemotron Desiffusion or something like that.

**[00:33:19 – 00:33:21]** Desiffusion AP, if you want to check it out.

**[00:33:21 – 00:33:25]** So it's an internal Nemotron model, but I think it is

**[00:33:26 – 00:33:28]** publicly available.

**[00:33:28 – 00:33:30]** Hi, good afternoon. Thank you very much for the

**[00:33:30 – 00:33:33]** informative presentation.

**[00:33:33 – 00:33:36]** So I had a look at Data Designer some time ago, and it seemed

**[00:33:36 – 00:33:40]** very geared towards very structured sort of synthetic data.

**[00:33:40 – 00:33:44]** Is that the case, or are the use cases for it far more

**[00:33:44 – 00:33:46]** varied than that?

**[00:33:47 – 00:33:52]** So it is, it has been evolving.

**[00:33:52 – 00:33:56]** It is now, I mean, we're not necessarily, it's still in

**[00:33:56 – 00:34:01]** research, but we are looking at generating multimodal data as well.

**[00:34:02 – 00:34:06]** It did start off purely being for structured data because

**[00:34:06 – 00:34:09]** we found that that's actually maybe the best way to create it.

**[00:34:09 – 00:34:15]** UserSim actually uses Data Designer under the covers, but like I

**[00:34:15 – 00:34:19]** said, that generates conversational data and trajectories and

**[00:34:19 – 00:34:20]** things like that.

**[00:34:20 – 00:34:21]** So it is evolving.

**[00:34:21 – 00:34:24]** Give it a try if you haven't checked it out.

**[00:34:24 – 00:34:28]** It has, you know, like I said, it can now also generate video and

**[00:34:28 – 00:34:32]** stuff like that, but with video, it all depends on, you know, there are

**[00:34:32 – 00:34:36]** a lot of variables there, so it's still kind of in research mode.

**[00:34:36 – 00:34:38]** Okay, thank you.

**[00:34:39 – 00:34:42]** Any other questions?

**[00:34:47 – 00:34:49]** Thanks for the great talk.

**[00:34:49 – 00:34:51]** I would be interested in two parts.

**[00:34:51 – 00:34:55]** So you mentioned the evaluation data set.

**[00:34:55 – 00:34:59]** Was that also created with data designers, or was it

**[00:34:59 – 00:35:01]** a separate production data set?

**[00:35:01 – 00:35:02]** Sorry, which one?

**[00:35:02 – 00:35:07]** The eval data set you showed the results of.

**[00:35:07 – 00:35:09]** So what did we use?

**[00:35:09 – 00:35:13]** The eval data set was a holdout part of the training set.

**[00:35:13 – 00:35:15]** So it was the same. Everything came from Data Designer from the same

**[00:35:16 – 00:35:21]** recipe, and we held out, I think it was 400-ish records that it was not

**[00:35:21 – 00:35:23]** trained on as the evaluation set.

**[00:35:23 – 00:35:29]** All right. And then a second question, how How hard is

**[00:35:29 – 00:35:34]** it to keep the model from overfitting to what, so you

**[00:35:34 – 00:35:36]** showed some templating, right?

**[00:35:36 – 00:35:41]** So how hard does the model overfit to that template style,

**[00:35:41 – 00:35:47]** so that if you would deploy it to production, would it still work

**[00:35:47 – 00:35:53]** with differently worded prompts?

**[00:35:53 – 00:35:57]** I think the templating there is templating into a prompt,

**[00:35:57 – 00:36:00]** which is then sent to an LLM for the actual generation.

**[00:36:00 – 00:36:03]** So the output is less templated than the input,

**[00:36:03 – 00:36:06]** is part A of the answer.

**[00:36:06 – 00:36:08]** Part B of the answer, I think, is one of the slides I had

**[00:36:08 – 00:36:10]** on what do you need to do to get this to a production

**[00:36:10 – 00:36:15]** quality data set, and that's really collecting real traces, making

**[00:36:15 – 00:36:18]** sure you're covering all the edge cases, all the different prompt

**[00:36:18 – 00:36:19]** styles, and things like that.

**[00:36:19 – 00:36:21]** So this is, like I said, day one.

**[00:36:21 – 00:36:26]** You can get this far and maybe 80%, and the other 20% is going to be

**[00:36:26 – 00:36:29]** a longer term, collect real data, generate better synthetic data with

**[00:36:29 – 00:36:34]** things like user sim and things like that, and do more testing.

**[00:36:34 – 00:36:37]** I think also is the idea, like we said, is this is a

**[00:36:37 – 00:36:42]** little bit of a contrived example for the purposes of this, but at

**[00:36:42 – 00:36:45]** the end of the day, you'll probably need to do a lot of testing

**[00:36:45 – 00:36:49]** to make sure that you're not overfitting and stuff like that.

**[00:36:49 – 00:36:53]** We did do some things like making sure we weren't, uh,

**[00:36:53 – 00:36:56]** you know, teaching the model the right answer and that it

**[00:36:56 – 00:37:01]** was actually generalizing the, the information based on what, um, uh,

**[00:37:01 – 00:37:05]** you know, what it was learning, um, but yeah, at the end of the day,

**[00:37:05 – 00:37:07]** it's just like experimentation.

**[00:37:07 – 00:37:10]** What we're trying to show here is we want to make that

**[00:37:10 – 00:37:14]** process more accessible, so you don't have to be a data

**[00:37:14 – 00:37:18]** scientist or a machine learning engineer to fine-tune models.

**[00:37:19 – 00:37:21]** As long as you follow the process, you experiment, you

**[00:37:21 – 00:37:26]** look at the results, and then you do a lot of testing.

**[00:37:27 – 00:37:31]** There's a question here.

**[00:37:33 – 00:37:36]** You can be the next.

**[00:37:36 – 00:37:38]** OK, a quick one.

**[00:37:38 – 00:37:42]** So I think you basically provide libraries for fine-tuning

**[00:37:42 – 00:37:43]** the dance models.

**[00:37:43 – 00:37:46]** What about MOE models?

**[00:37:46 – 00:37:50]** Because fine-tuning that one would be different.

**[00:37:51 – 00:37:55]** How do you recommend, do you have a plan to release some libraries

**[00:37:55 – 00:37:58]** to help to make that easier?

**[00:37:58 – 00:38:04]** MOE models, yeah, so the libraries here do actually support MOE.

**[00:38:04 – 00:38:09]** So you can do a LoRa fine tune on MOE models using

**[00:38:09 – 00:38:13]** Helix and AutoModel SFT runner underneath the hood.

**[00:38:13 – 00:38:15]** I've done it myself. I've fine tuned some of the

**[00:38:15 – 00:38:17]** Nemotron MOE models.

**[00:38:17 – 00:38:18]** So it's already supported.

**[00:38:18 – 00:38:24]** I think everything comes down to evaluation, so you'll want

**[00:38:24 – 00:38:26]** to evaluate and see what kind of different results you're getting.

**[00:38:26 – 00:38:29]** They'll have higher speeds, different varying levels of

**[00:38:29 – 00:38:30]** quality and things like that.

**[00:38:30 – 00:38:35]** So evaluation is the most important thing here, I think.

**[00:38:37 – 00:38:40]** It is all post-training, though, just to be clear.

**[00:38:40 – 00:38:43]** I mean, you can really, this is meant for doing post-training,

**[00:38:43 – 00:38:48]** not for pre-training, so you're gonna start from a model.

**[00:38:53 – 00:38:55]** Thanks, thanks for the session.

**[00:38:55 – 00:38:56]** Just a quick question.

**[00:38:56 – 00:39:03]** So essentially, you tested the improvements in the model

**[00:39:03 – 00:39:10]** through the evaluation part of the training set that you

**[00:39:10 – 00:39:14]** kept apart just for this purpose, but did you test it even in

**[00:39:14 – 00:39:22]** a really, so a simulated customer user scenario, when actually

**[00:39:22 – 00:39:31]** a fix is applied to a live data set that is being accessed through

**[00:39:31 – 00:39:38]** different agents and then applied to fixing that particular scenario?

**[00:39:38 – 00:39:43]** So essentially, I didn't get where did you generate

**[00:39:43 – 00:39:45]** all the user inputs.

**[00:39:45 – 00:39:47]** So this was synthetically generated.

**[00:39:47 – 00:39:51]** We did not test, because this was just, this was a demo,

**[00:39:51 – 00:39:55]** so this agent is not intended to go into production, right?

**[00:39:55 – 00:39:59]** We made up a company, we made up the scenario and the rules and so

**[00:39:59 – 00:40:03]** on, all the data, because we didn't have existing traces, all the data

**[00:40:03 – 00:40:05]** here was synthetically generated.

**[00:40:05 – 00:40:08]** And then the holdout set was also from the synthetic data,

**[00:40:08 – 00:40:09]** like Stefan mentioned.

**[00:40:10 – 00:40:11]** Did that answer your question?

**[00:40:11 – 00:40:15]** Yeah, so you didn't use your

**[00:40:15 – 00:40:18]** user simulation package for?

**[00:40:18 – 00:40:19]** Oh, user sim?

**[00:40:19 – 00:40:25]** No, actually, user sim just went live, so we did not use

**[00:40:25 – 00:40:30]** the user sim data, but I wanted to show it as an option.

**[00:40:30 – 00:40:33]** The other thing, actually the reason we didn't use it

**[00:40:33 – 00:40:36]** was because for the fast decision model, we didn't need traces.

**[00:40:36 – 00:40:38]** It was a different type of data set that we needed.

**[00:40:39 – 00:40:43]** It was the incident and the action.

**[00:40:43 – 00:40:47]** So actually, user sim wasn't the right fit in that instance.

**[00:40:47 – 00:40:50]** I think we're at time.

**[00:40:50 – 00:40:51]** I think we're at time. Thank you very much.

**[00:40:51 – 00:40:56]** Thank you, everyone. And yeah, please feel free

**[00:40:57 – 00:41:02]** If you have any questions, feel free to ping us.

**[00:41:02 – 00:41:04]** And also, check out the libraries.

**[00:41:04 – 00:41:05]** You can open issues.

**[00:41:05 – 00:41:07]** We welcome your feedback.

