# Co-Engineering Grace CPU Platforms for Hyperscale

## Abstract

Meta operates one of the world’s largest and most diverse computing fleets, spanning AI training and inference, web services, ranking, caching, analytics, and video processing. In this technical session, leaders from Meta and NVIDIA share how they co-engineer NVIDIA Grace CPU–based platforms end‑to‑end to meet the performance, efficiency, and scale demands of hyperscale fleets. You’ll see how joint work across silicon, platform design, and infrastructure integration improves performance per watt, rack density, and overall fleet efficiency, and how these platforms enable the next generation of hyperscale infrastructure for AI and large‑scale services.

## AI Summary

- The partnership between NVIDIA and Meta focused on developing and optimizing the Grace CPU for data center use, transitioning from a module-based approach to a fully customizable server deployment model.
- The co-engineering efforts included optimizing the Grace CPU for low power and high density, achieving a system power of 175 watts and a CPU power of 120 watts.
- Meta’s compute portfolio approach involved a heterogeneous fleet with both x86 and ARM platforms, and the company heavily co-designed each compute platform to fit specific workloads.
- The Java Island server platform was designed to be drop-in compatible with Meta’s existing Yosemite chassis, leveraging OCP standards and optimizing for performance per watt.
- Meta used the DCPerf benchmark suite to co-optimize the software stack, from firmware to production applications, ensuring that the platform was battle-tested and reliable at scale.
- The company faced challenges with LPDDR memory latency but overcame them through iterative tuning of prefetchers and other optimizations, and continued to invest in the ARM ecosystem for future improvements.

## Transcript

**[00:00:04 – 00:00:06]** Thank you.

**[00:00:07 – 00:00:08]** So it's great to be here.

**[00:00:08 – 00:00:11]** Thanks, everyone, for bearing with the last session of the day.

**[00:00:11 – 00:00:14]** We probably saw some great content, and everyone's ready

**[00:00:14 – 00:00:17]** for dinner and beers.

**[00:00:17 – 00:00:19]** It's a privilege to get to close this out with all of you.

**[00:00:19 – 00:00:24]** So to put into context what we've done with Meta, which I'm super

**[00:00:24 – 00:00:26]** excited to have them present today.

**[00:00:26 – 00:00:30]** We have to kind of look at the genesis of the CPU business here.

**[00:00:30 – 00:00:33]** So many of you probably saw the earlier breakout sessions

**[00:00:33 – 00:00:37]** dedicated to Grace, to Vera, to what we're doing as a standalone

**[00:00:37 – 00:00:41]** CPU business and why CPUs are important in the context of AI.

**[00:00:41 – 00:00:44]** I'm going to spare a lot of the preamble.

**[00:00:44 – 00:00:47]** Around that, I just kind of focus on the platforms and

**[00:00:47 – 00:00:50]** the history of the platforms and what we've done in partnership with

**[00:00:50 – 00:00:57]** Meta to get ready for a standalone ecosystem of CPUs based on Vera.

**[00:00:57 – 00:01:02]** So way back in 2023, we launched Grace, and this was NVIDIA's

**[00:01:02 – 00:01:04]** first data center CPU.

**[00:01:04 – 00:01:09]** It's based on an Arm Neo verse core and an NVIDIA proprietary

**[00:01:09 – 00:01:12]** fabric, which we call scalable coherency fabric core part

**[00:01:12 – 00:01:15]** of our CPU design today.

**[00:01:15 – 00:01:18]** When we did this, this was actually the world's first

**[00:01:18 – 00:01:22]** data center CPU with LPDDR memory.

**[00:01:22 – 00:01:25]** So there was no established standard for how people would

**[00:01:25 – 00:01:30]** build servers based on Grace, based on the LPDDR memory

**[00:01:30 – 00:01:34]** technology, and based on, you know, how to actually package these

**[00:01:34 – 00:01:37]** things into rack-mount data center servers that people would use.

**[00:01:37 – 00:01:41]** So we did this in actually a phased approach.

**[00:01:41 – 00:01:42]** The first.

**[00:01:42 – 00:01:45]** Grace platforms were built around a module like this

**[00:01:45 – 00:01:48]** that we called C2, and it's two Graces, the memory, the voltage

**[00:01:48 – 00:01:53]** regulation, basically all the parts of a server, on this module

**[00:01:53 – 00:01:55]** that looks a lot like a GPU module.

**[00:01:55 – 00:01:58]** But it's not a standard data center-type server.

**[00:01:58 – 00:02:02]** It's certainly not something that would have been consumable

**[00:02:02 – 00:02:06]** at scale in the way that a customer like Meta operates.

**[00:02:06 – 00:02:08]** First of all, all the...

**[00:02:08 – 00:02:10]** All of the material on that board was actually handled by NVIDIA.

**[00:02:10 – 00:02:12]** The board's manufactured by NVIDIA.

**[00:02:12 – 00:02:17]** We buy the memory. It's not a great way to put a customer in

**[00:02:17 – 00:02:19]** the driver's seat on their fleet.

**[00:02:19 – 00:02:22]** Now, it did find quite a bit of success because, first

**[00:02:22 – 00:02:25]** of all, we did this because the server ecosystem around

**[00:02:25 – 00:02:26]** Grace wasn't ready yet.

**[00:02:26 – 00:02:28]** We needed to go to market. We needed people to have a way

**[00:02:28 – 00:02:31]** to deploy this new type of memory, new type of CPU, and actually

**[00:02:31 – 00:02:32]** have it in their data center.

**[00:02:32 – 00:02:33]** So we did it in a staged approach.

**[00:02:33 – 00:02:35]** It was very conscious.

**[00:02:35 – 00:02:38]** And this was what the first Grace platform looks like.

**[00:02:38 – 00:02:42]** For those of you who are server design folks who

**[00:02:42 – 00:02:45]** may be in the audience, this is really an HPC platform.

**[00:02:45 – 00:02:50]** This is not an inexpensive, scalable, CSP-type platform.

**[00:02:50 – 00:02:54]** It took four of those modules, put them into a 2U, and this was really

**[00:02:54 – 00:02:58]** a high-performance computing-type blade server for supercomputing.

**[00:02:58 – 00:02:59]** So, Grace...

**[00:02:59 – 00:03:03]** In the Falcon timeframe, just a couple years ago, was basically

**[00:03:03 – 00:03:08]** a dense HPC-type product, not a general-purpose microprocessor

**[00:03:08 – 00:03:12]** for Data Center fleet like the ones Meta operate.

**[00:03:12 – 00:03:17]** All that changed with a product that we called C1, and C1

**[00:03:17 – 00:03:20]** was a reference design.

**[00:03:20 – 00:03:23]** And it was the first time that we actually sold the chips

**[00:03:23 – 00:03:28]** themselves, let customers buy the memory, let customers customize the

**[00:03:28 – 00:03:32]** platform, build a traditional...

**[00:03:32 – 00:03:36]** Data Center, CPU type sales model around our CPU, but then

**[00:03:36 – 00:03:40]** actually extend it to be superior to the traditional Data Center, CPU

**[00:03:40 – 00:03:45]** sales model where you have sockets and DIMMs, you know, modular parts.

**[00:03:45 – 00:03:47]** This actually lets customers do intense co-design.

**[00:03:47 – 00:03:48]** You can customize the board.

**[00:03:48 – 00:03:50]** You solder the memory down.

**[00:03:50 – 00:03:53]** And really, it put the customers in control of how they built the

**[00:03:53 – 00:03:55]** server, gave them the flexibility they needed, and retained

**[00:03:55 – 00:04:00]** all the great things about having, you know, small integrated memory

**[00:04:00 – 00:04:05]** packages instead of big, dim slots, and, you know, low platform powers.

**[00:04:05 – 00:04:08]** This was a platform that we actually worked with Meta.

**[00:04:08 – 00:04:11]** We got the power per CPU down to about 140 watts.

**[00:04:11 – 00:04:14]** They're going to talk more about that in a moment.

**[00:04:14 – 00:04:18]** We only lost about 10% of the performance of running

**[00:04:18 – 00:04:20]** each of those Graces at 250 watts.

**[00:04:20 – 00:04:24]** So this was a huge shift, a huge shift in how we

**[00:04:24 – 00:04:26]** sell and deploy CPU.

**[00:04:26 – 00:04:31]** And Meta was there in the beginning doing this with us as a partner.

**[00:04:31 – 00:04:35]** And a lot of the work that went into testing this type

**[00:04:35 – 00:04:37]** of distribution model, this type of packaging model for

**[00:04:37 – 00:04:42]** CPUs, rolls forward to making sure that Vera is available

**[00:04:42 – 00:04:44]** in all of the different types of form factors and server designs

**[00:04:44 – 00:04:50]** that customers want on day zero, on launch, which by the way is today.

**[00:04:50 – 00:04:53]** Well, yesterday, I think.

**[00:04:54 – 00:04:57]** We're super grateful for the partnership with Meta.

**[00:04:57 – 00:05:00]** Very rapidly over the last two years, we've matured from

**[00:05:00 – 00:05:05]** a CPU and a memory system sold only on a module, specific

**[00:05:05 – 00:05:10]** capacity points, directly by NVIDIA, to a full customized

**[00:05:10 – 00:05:13]** customized server deployment type business where our customers buy

**[00:05:13 – 00:05:17]** bare CPUs, they buy memory on the open market, they build any type of

**[00:05:17 – 00:05:19]** server reference design they wish.

**[00:05:19 – 00:05:23]** And that is what you're about to hear about.

**[00:05:33 – 00:05:36]** Thanks again for the intro and thank you all for coming here.

**[00:05:36 – 00:05:38]** I know it's been a long day for all of you, so thanks

**[00:05:38 – 00:05:41]** for listening into this session.

**[00:05:41 – 00:05:43]** I'm going to present the rest of the talk.

**[00:05:43 – 00:05:45]** Abhishek and I will walk through this.

**[00:05:45 – 00:05:50]** We're going to talk about co-engineering NVIDIA Grace CPU.

**[00:05:50 – 00:05:53]** I think this is not about just like Ian kind of hinted, this

**[00:05:53 – 00:05:56]** isn't about just picking up the chip off the shelf and just like

**[00:05:56 – 00:05:58]** kind of using it out of the box.

**[00:05:58 – 00:05:59]** This is mostly about like trying to figure out what

**[00:05:59 – 00:06:03]** the right operation point for Meta would be and like what

**[00:06:03 – 00:06:04]** the right custom design would be.

**[00:06:04 – 00:06:07]** Something that's actually built, purpose-built and fit

**[00:06:07 – 00:06:11]** for our workloads internally.

**[00:06:11 – 00:06:13]** All right, so a quick roadmap of what we'll talk to today.

**[00:06:13 – 00:06:16]** We'll first walk through a little bit of the Meta's compute

**[00:06:17 – 00:06:21]** portfolio approach, and then talk about the way we have

**[00:06:21 – 00:06:24]** done the co-engineering effort, actually spanning multiple levels.

**[00:06:24 – 00:06:27]** So talk about the Grace CPU level, the server platform

**[00:06:28 – 00:06:32]** itself, how we use the DC per benchmarks, and how we actually

**[00:06:32 – 00:06:36]** use the production application to actually battle test this platform.

**[00:06:37 – 00:06:40]** And then we'll wrap up with what's next.

**[00:06:41 – 00:06:44]** All right, so before we get into the details for Grace,

**[00:06:44 – 00:06:48]** let's talk briefly about what Meta's compute portfolio approach

**[00:06:48 – 00:06:51]** actually consists of.

**[00:06:51 – 00:06:54]** We actually are very heterogeneous in our fleet, so we actually

**[00:06:54 – 00:06:57]** are not single source on our CPU.

**[00:06:57 – 00:07:01]** We do have multiple x86 platforms from Intel and AMD.

**[00:07:01 – 00:07:04]** We also have multiple ARM platforms and NVIDIA and ARM Inc.

**[00:07:04 – 00:07:07]** And we are actually not betting on a single

**[00:07:07 – 00:07:10]** architecture in our approach.

**[00:07:10 – 00:07:13]** I think the AR64 journey at Meta has been a little bit

**[00:07:13 – 00:07:14]** of a longer journey.

**[00:07:14 – 00:07:17]** So I'll just walk through briefly about that timeline.

**[00:07:17 – 00:07:21]** It actually started in 2017 with a lot of investment being made

**[00:07:21 – 00:07:27]** about enabling these workloads that we have to work on AR64 platform.

**[00:07:27 – 00:07:31]** However, most of our software stack remained to be predominantly

**[00:07:31 – 00:07:35]** x86 up up till 2022 is when we actually kicked up the effort

**[00:07:35 – 00:07:38]** of arm enablement in full earnest.

**[00:07:38 – 00:07:41]** We had a small team of ninja software engineers essentially

**[00:07:41 – 00:07:43]** working on like a lot of the plumbing that we need from

**[00:07:43 – 00:07:46]** the infrastructure aspects to drive the enablement and

**[00:07:46 – 00:07:51]** then they actually continued like, you know, creating like the the

**[00:07:51 – 00:07:53]** infrastructure pieces of this and also like enabling the workloads

**[00:07:53 – 00:07:57]** to actually be able to build the internal software stack and all

**[00:07:57 – 00:08:00]** of the dependencies on the AR64.

**[00:08:00 – 00:08:04]** So we got to a point where things were functional.

**[00:08:04 – 00:08:07]** And since then, we have continued to enable a lot more workloads

**[00:08:07 – 00:08:10]** and drive a lot of optimization and we are now at a point where a lot

**[00:08:10 – 00:08:15]** of the large services that we have at Meta are actually running in

**[00:08:15 – 00:08:18]** a continuous integration mode where they actually are able to build and

**[00:08:18 – 00:08:20]** deploy on Arm sort of seamlessly.

**[00:08:21 – 00:08:22]** And it's important to start with these big services because

**[00:08:22 – 00:08:26]** they actually help us pipe clean a lot of the infrastructure because

**[00:08:26 – 00:08:29]** if the big services work, then they actually can enable the medium and

**[00:08:29 – 00:08:33]** the small services very seamlessly.

**[00:08:33 – 00:08:34]** So I think, in summary, we are pretty much at parity

**[00:08:34 – 00:08:38]** today in terms of developer experience across the AR64

**[00:08:38 – 00:08:42]** and the x86 platform, which is a pretty big deal given

**[00:08:42 – 00:08:46]** the amount of services that we have and the scale at which we operate.

**[00:08:46 – 00:08:49]** So our design approach, let's talk about that a little bit, right?

**[00:08:49 – 00:08:52]** So we do heavily co-design every compute platform that

**[00:08:52 – 00:08:53]** we bring into our fleet.

**[00:08:54 – 00:08:57]** We design for both sort of scale-up workloads that actually

**[00:08:57 – 00:08:58]** end up using the entire machine.

**[00:08:58 – 00:09:00]** We do also have a bunch of mid and long tail services that actually

**[00:09:00 – 00:09:02]** are running in a multi-line mode.

**[00:09:02 – 00:09:06]** So we have to optimize for both of the sort of scenarios.

**[00:09:06 – 00:09:09]** And we also end up releasing our platforms to OCP and to

**[00:09:09 – 00:09:13]** make that actually available for the community.

**[00:09:14 – 00:09:16]** So let's start with the co-engineering effort for

**[00:09:16 – 00:09:19]** the NVIDIA Grace CPU itself.

**[00:09:19 – 00:09:21]** So like Ian mentioned, this has been a partnership with

**[00:09:21 – 00:09:25]** NVIDIA since 2022 or 2023 early.

**[00:09:25 – 00:09:28]** With respect to Grace, we made some key changes.

**[00:09:28 – 00:09:33]** We designed a one-socket system with high memory bandwidth, and

**[00:09:33 – 00:09:36]** we fine-tuned the memory-to-core ratio, achieving the sweet

**[00:09:36 – 00:09:37]** optimal point for us.

**[00:09:37 – 00:09:40]** And we wanted to really design a low-power and a

**[00:09:40 – 00:09:42]** dense rack solution.

**[00:09:42 – 00:09:45]** That was kind of the overall theme that we were going for.

**[00:09:45 – 00:09:49]** On the power side, we actually optimized the whole envelope.

**[00:09:49 – 00:09:53]** So we had 120 watts for the CPU, 154 memory included,

**[00:09:53 – 00:09:55]** and 175 for the system power.

**[00:09:55 – 00:09:57]** And this is when the CPU utilization is pretty close

**[00:09:57 – 00:10:00]** to 100% utilization.

**[00:10:00 – 00:10:04]** So, we talked about LPDDR, that's one of the key technologies that

**[00:10:05 – 00:10:07]** got enabled on the Grace platform.

**[00:10:07 – 00:10:11]** But LPDDR is very different from DDR, and deploying something

**[00:10:11 – 00:10:13]** that's that disruptive at scale requires, we had to really

**[00:10:13 – 00:10:17]** work through a lot of, you know, at scale sort of aspects of this.

**[00:10:18 – 00:10:21]** Most importantly, we had to work through the reliability

**[00:10:21 – 00:10:23]** RAS requirements, and other such aspects to make sure

**[00:10:24 – 00:10:25]** it's actually viable at our scale.

**[00:10:25 – 00:10:28]** And we actually are investing very heavily in this space and trying

**[00:10:28 – 00:10:32]** to make sure that the future LPDDR roadmap remains sort of friendly

**[00:10:32 – 00:10:36]** for like a large scale deployment that we like to work with.

**[00:10:37 – 00:10:40]** So the guiding principle around this co-engineering was essentially

**[00:10:40 – 00:10:44]** to maximize the useful compute delivered per watt.

**[00:10:44 – 00:10:46]** To the data center.

**[00:10:46 – 00:10:49]** So not just the peak performance on a spec sheet.

**[00:10:49 – 00:10:51]** So if you look at a server in isolation, you think it's

**[00:10:51 – 00:10:54]** hitting a really good sort of, any performance metric

**[00:10:54 – 00:10:54]** that you look at.

**[00:10:54 – 00:10:56]** I mean, that's interesting, but we really want to look

**[00:10:56 – 00:11:01]** at the peak performance and per-for-what at a rack granularity,

**[00:11:01 – 00:11:06]** because that's actually the right deployment unit to be considering.

**[00:11:06 – 00:11:08]** So we spent a lot of time making sure that we don't get too

**[00:11:08 – 00:11:11]** fixated in like a local maxima kind of scenario where we have really

**[00:11:11 – 00:11:14]** good perf or really good perf for whatever server level, but we

**[00:11:14 – 00:11:19]** just wanted to make sure we build a dense, power-efficient rack.

**[00:11:19 – 00:11:22]** Okay, I'll hand it off to Abhishek to walk through the

**[00:11:22 – 00:11:25]** other co-engineering aspects.

**[00:11:31 – 00:11:32]** Thanks Shobit.

**[00:11:32 – 00:11:38]** So Shobit touched upon like the key changes we did on the CPU itself.

**[00:11:38 – 00:11:42]** I'll touch upon the co-engineering efforts both on the platform

**[00:11:42 – 00:11:43]** side which we call Java Island.

**[00:11:43 – 00:11:48]** So Java Island is a custom-designed server platform specifically for

**[00:11:48 – 00:11:50]** Grace to fit on our data centers.

**[00:11:50 – 00:11:54]** Now I'll also touch upon the co-engineering efforts around

**[00:11:54 – 00:11:55]** all the software stack.

**[00:11:55 – 00:11:59]** All the way from firmware to production application that scale

**[00:11:59 – 00:12:03]** level in the subsequent slides.

**[00:12:03 – 00:12:07]** So let's start with the server blade itself.

**[00:12:07 – 00:12:10]** So as Ian highlighted, we essentially leveraged the

**[00:12:10 – 00:12:16]** C1 design for a single socket CPU, and then basically took

**[00:12:16 – 00:12:20]** that reference design and made it work in a Yosemite chassis.

**[00:12:20 – 00:12:23]** So Yosemite is our line of essentially compute server

**[00:12:23 – 00:12:26]** platforms, which is all open sourced, and what we

**[00:12:26 – 00:12:30]** typically deploy across all our racks in the fleet.

**[00:12:30 – 00:12:33]** So we essentially designed Java Island to be basically

**[00:12:33 – 00:12:39]** a drop-in compatible solution in a Yosemite v3.5 platform.

**[00:12:39 – 00:12:44]** We essentially utilized all the OCP or like standard components,

**[00:12:44 – 00:12:51]** things like OCP compliant NIC, E1S, SSD form factor

**[00:12:51 – 00:12:54]** for the boot drives.

**[00:12:54 – 00:12:59]** We essentially leveraged like everything from power

**[00:12:59 – 00:13:00]** delivery network.

**[00:13:00 – 00:13:04]** To the management interfaces, everything was basically

**[00:13:04 – 00:13:10]** leveraged from the base Yosemite design and then essentially

**[00:13:10 – 00:13:13]** optimized for Java Island.

**[00:13:13 – 00:13:16]** At a per-server level, as Shobit highlighted, we basically

**[00:13:16 – 00:13:19]** made the configuration look very similar to our other

**[00:13:19 – 00:13:21]** compute platforms in the fleet.

**[00:13:21 – 00:13:25]** So we made use of all the 72 cores, but then down-configured

**[00:13:25 – 00:13:31]** the memory to be dual-rank 256 GB to maximize the memory

**[00:13:31 – 00:13:35]** speeds that we can operate at and maximize our bandwidth per core.

**[00:13:35 – 00:13:38]** We put a two terabyte local boot drive, which essentially

**[00:13:38 – 00:13:43]** we use for all sorts of logging and essentially boot management stuff.

**[00:13:43 – 00:13:48]** Then we had a multi-host, again like OCP-compliant multi-host NIC,

**[00:13:48 – 00:13:50]** that we share among two servers.

**[00:13:50 – 00:13:55]** So overall, because of this whole drop-in compatible design,

**[00:13:55 – 00:13:59]** we are able to move really quickly in terms of taking

**[00:13:59 – 00:14:02]** the C1 reference design and build something that we could deploy

**[00:14:02 – 00:14:06]** in a production environment very, very quickly and be able to run

**[00:14:06 – 00:14:11]** production applications at scale with a pretty good time to market.

**[00:14:12 – 00:14:16]** We do intend to, if not already, I believe, like we do intend to

**[00:14:16 – 00:14:19]** open source the Java Island design itself so like others can use

**[00:14:19 – 00:14:24]** it for their applications as well.

**[00:14:24 – 00:14:28]** The other aspect of Java Island that we worked on is maximizing

**[00:14:28 – 00:14:30]** the rack-level performance.

**[00:14:30 – 00:14:33]** So this slide kind of shows not-so-pretty schematics.

**[00:14:33 – 00:14:38]** I think Ian had a better rendering of the C1 design.

**[00:14:38 – 00:14:43]** But this is more functional and a scale variant of that.

**[00:14:43 – 00:14:46]** So the top right graphic is essentially how a single socket

**[00:14:46 – 00:14:49]** Java Island blade looks like.

**[00:14:49 – 00:14:53]** And the bottom right one is the entire chassis.

**[00:14:53 – 00:14:56]** So if you can see, in the chassis we can essentially

**[00:14:56 – 00:15:00]** pack in 12 of those blades.

**[00:15:00 – 00:15:05]** And then each blade essentially, or rather two blades, share

**[00:15:05 – 00:15:09]** a multi-host NIC, they share BMC.

**[00:15:09 – 00:15:13]** Oh, yeah, I think, yeah, like, it's probably this big.

**[00:15:13 – 00:15:16]** So, like, you know, the bottom right thing you see

**[00:15:16 – 00:15:19]** is, like, literally, like, just a little bigger than

**[00:15:19 – 00:15:21]** a laptop in terms of form factor.

**[00:15:21 – 00:15:24]** And we were able to pack, I believe, like, six or seven

**[00:15:24 – 00:15:29]** of them in a rack to get to, like, 64 servers rack density.

**[00:15:30 – 00:15:34]** Our goal is always to maximize the amount of performance

**[00:15:34 – 00:15:39]** we can pack in in a given power envelope so that we can essentially

**[00:15:39 – 00:15:42]** roll out a bunch of these racks in a data center while still maxing

**[00:15:42 – 00:15:45]** out our overall power footprint.

**[00:15:45 – 00:15:52]** So essentially, if you look at the details that come out as part

**[00:15:52 – 00:15:56]** of the OCP announcements around this, you'll see that we've trimmed

**[00:15:56 – 00:16:01]** down this to the bare minimum things that we absolutely need, and

**[00:16:01 – 00:16:06]** optimized this heavily for perf per watt, perf per TCO, and basically

**[00:16:06 – 00:16:09]** maximizing overall performance.

**[00:16:09 – 00:16:12]** With that, let's move into like what we did on the software

**[00:16:12 – 00:16:14]** stack side of things.

**[00:16:14 – 00:16:17]** So we recently, actually not so recently, like maybe about

**[00:16:17 – 00:16:20]** a year, year and a half ago, we open sourced DCPerf.

**[00:16:20 – 00:16:24]** So this is like essentially our primary benchmark suite

**[00:16:25 – 00:16:27]** for all compute server designs.

**[00:16:27 – 00:16:29]** I believe this is the first.

**[00:16:29 – 00:16:34]** This is a representative benchmark suite for a hyperscaler application

**[00:16:34 – 00:16:37]** that was released when it was released in open source.

**[00:16:37 – 00:16:39]** And essentially we use this as a primary vehicle to

**[00:16:39 – 00:16:43]** collaborate on a bunch of software optimization efforts.

**[00:16:43 – 00:16:47]** So DCPerf itself has like three big components.

**[00:16:47 – 00:16:49]** There are end-to-end benchmarks.

**[00:16:49 – 00:16:53]** There are micro benchmarks for widely distributed libraries.

**[00:16:53 – 00:16:56]** And then we also recently introduced DCPerf Mini,

**[00:16:56 – 00:16:59]** which is essentially think of it as like a...

**[00:16:59 – 00:17:04]** A trimmed down version of the end-to-end benchmarks.

**[00:17:04 – 00:17:12]** Trimmed down along multiple facets including like the run length, The

**[00:17:12 – 00:17:16]** The number of codes that we need to run the benchmark itself.

**[00:17:16 – 00:17:19]** And essentially the complexity to trace and simulate it.

**[00:17:19 – 00:17:23]** So essentially we collaborated with the NVIDIA team across

**[00:17:23 – 00:17:24]** all these levers.

**[00:17:24 – 00:17:26]** Let's start with the end-to-end benchmarks.

**[00:17:26 – 00:17:30]** So essentially we have a collection of benchmarks for different

**[00:17:30 – 00:17:34]** domains of applications that we typically run in the fleet.

**[00:17:34 – 00:17:36]** I'll not go into details of individual ones.

**[00:17:36 – 00:17:40]** I believe Lucas and Mati has talked about video, which is one of

**[00:17:40 – 00:17:45]** the applications or benchmarks that we co-optimize with the NVIDIA team

**[00:17:45 – 00:17:47]** in the context of Grace and Vera.

**[00:17:47 – 00:17:51]** But in each of these cases, what essentially we did was

**[00:17:51 – 00:17:57]** we started with porting the benchmarks on AR64, essentially

**[00:17:57 – 00:18:01]** optimizing the benchmark stack itself all the way from the

**[00:18:02 – 00:18:05]** application code to the libraries to the kernel to the firmware and

**[00:18:05 – 00:18:10]** everything, basically all layers, in order to maximize performance

**[00:18:10 – 00:18:13]** on each of these benchmarks.

**[00:18:13 – 00:18:15]** The goal from that essentially was two-fold.

**[00:18:15 – 00:18:18]** One was that once we had a pretty good optimized version

**[00:18:18 – 00:18:21]** of the benchmark, we use that as a blueprint to go talk

**[00:18:21 – 00:18:25]** to the production applications, and essentially set some performance

**[00:18:25 – 00:18:27]** targets for them to show.

**[00:18:27 – 00:18:30]** What this allows us to do is have a good anchor point

**[00:18:30 – 00:18:34]** in terms of where our landing zone for the product should be.

**[00:18:34 – 00:18:38]** The second thing that we did was essentially use these end-to-end

**[00:18:38 – 00:18:44]** benchmarks to also collaborate on compiler optimizations,

**[00:18:44 – 00:18:47]** kernel flags, kernel configs, and everything in between,

**[00:18:47 – 00:18:50]** like networking stack, network drivers, all of that stuff,

**[00:18:50 – 00:18:54]** in order for us to be able to run something which is more like a

**[00:18:54 – 00:18:59]** real microservice or an end-to-end service in a scale-out environment.

**[00:18:59 – 00:19:02]** So that was all the collaboration on end-to-end benchmarks.

**[00:19:02 – 00:19:05]** The other thing we introduced were these micro benchmarks

**[00:19:05 – 00:19:07]** for widely distributed libraries.

**[00:19:07 – 00:19:12]** The goal here was essentially, as we started ramping up production

**[00:19:12 – 00:19:15]** applications on the Grace platform, whenever we would find that

**[00:19:15 – 00:19:19]** a particular commonly distributed library was not functioning well,

**[00:19:19 – 00:19:23]** we would essentially carve out a micro benchmark, but this is not.

**[00:19:23 – 00:19:25]** A typical microbenchmark in the sense that it's

**[00:19:25 – 00:19:26]** just a piece of code.

**[00:19:26 – 00:19:31]** We did release all of our working set sizes and essentially

**[00:19:31 – 00:19:35]** some sort of synthetic but representative datasets associated

**[00:19:35 – 00:19:36]** with these microbenchmarks.

**[00:19:36 – 00:19:39]** The other thing we did was essentially like kind

**[00:19:39 – 00:19:41]** of like focused on

**[00:19:41 – 00:19:44]** A specific subset of APIs within each of these libraries, right?

**[00:19:44 – 00:19:50]** I mean, as you can imagine, like, some typical RPC library

**[00:19:50 – 00:19:53]** would have, like, probably, like, 200 or 300 APIs that it supports,

**[00:19:53 – 00:19:57]** but not all of them are important from a performance standpoint.

**[00:19:57 – 00:20:00]** So essentially, what we did was made this into a more

**[00:20:00 – 00:20:04]** focused effort around a very small subset of APIs with

**[00:20:04 – 00:20:09]** representative data sets that we could actually go iterate over

**[00:20:09 – 00:20:15]** and heavily optimize all the way to the last sub-1% performance levels.

**[00:20:15 – 00:20:18]** And then I kind of touched upon DCPerf Mini already,

**[00:20:18 – 00:20:21]** but essentially like the goal from this was twofold.

**[00:20:21 – 00:20:23]** One is things that we cannot optimize just by running

**[00:20:23 – 00:20:26]** like end-to-end binaries, but something we want to

**[00:20:26 – 00:20:29]** deep dive through a simulation or emulation environment.

**[00:20:29 – 00:20:34]** We essentially use DCPerf Mini as a vehicle to go down that route.

**[00:20:34 – 00:20:38]** In addition to basically having these benchmark traces

**[00:20:38 – 00:20:43]** that essentially the external teams could also use to

**[00:20:43 – 00:20:47]** design or optimize their future microarchitectures.

**[00:20:48 – 00:20:52]** All the work that was done here is all on GitHub in open source.

**[00:20:52 – 00:20:57]** If anybody wants to check it out, the GitHub link is right here.

**[00:20:58 – 00:21:04]** So, moving on, essentially DCPerf, as I mentioned, was

**[00:21:04 – 00:21:08]** primarily a vehicle to co-optimize common set of stuff.

**[00:21:08 – 00:21:11]** But as you can imagine, in a really like a scale-out

**[00:21:11 – 00:21:16]** environment, you do anticipate more issues when you try to

**[00:21:16 – 00:21:17]** scale up applications, right?

**[00:21:18 – 00:21:21]** So this slide kind of talks about a few of the key categories

**[00:21:21 – 00:21:24]** of applications that we have enabled so far.

**[00:21:25 – 00:21:27]** All the way from things like ranking and recommendation,

**[00:21:27 – 00:21:32]** web serving, to object caching and distribution, video,

**[00:21:32 – 00:21:37]** transcode, storage, training data ingestion, and so on.

**[00:21:37 – 00:21:40]** I'll not go into details of each of these applications,

**[00:21:40 – 00:21:44]** but try to touch upon a few interesting things that we

**[00:21:44 – 00:21:49]** observed where either we were able to co-optimize this in software or

**[00:21:49 – 00:21:54]** things that we had to essentially take to the next generation.

**[00:21:54 – 00:21:56]** So let's start with ranking and recommendations.

**[00:21:56 – 00:21:59]** So the good thing was that essentially with Grace or

**[00:21:59 – 00:22:03]** the Arm v2 cores, we are able to leverage the higher single

**[00:22:03 – 00:22:08]** thread performance and higher memory bandwidth per core

**[00:22:08 – 00:22:11]** to be able to drive really high utilization on these

**[00:22:11 – 00:22:14]** boxes while meeting latency SLOs.

**[00:22:14 – 00:22:17]** So for ranking and recommendation, typically

**[00:22:17 – 00:22:20]** these are latency-critical services where But essentially,

**[00:22:20 – 00:22:22]** the overall server performance is

**[00:22:22 – 00:22:26]** And utilization is dictated by how well your single thread performance

**[00:22:26 – 00:22:28]** and memory operations are.

**[00:22:28 – 00:22:30]** So essentially, Grace did really well on these kind

**[00:22:30 – 00:22:33]** of applications, and we have been using it heavily there.

**[00:22:33 – 00:22:37]** For web serving, we did see a bunch of challenges with

**[00:22:37 – 00:22:40]** respect to being able to serve really large code footprints.

**[00:22:40 – 00:22:44]** And with LPDDR and first generation of deployment, we did see

**[00:22:44 – 00:22:47]** higher latencies, which led to...

**[00:22:47 – 00:22:50]** Not so good performance and so we have been iterating over this both

**[00:22:50 – 00:22:55]** in terms of making improvements on the software stack and also making

**[00:22:55 – 00:22:59]** improvements that could land in the next generation of processors

**[00:22:59 – 00:23:03]** so that like we can make these applications go faster as well.

**[00:23:03 – 00:23:07]** The data ingestion and pre-prog, this is like typical services that

**[00:23:07 – 00:23:10]** you would run which essentially reads data from our databases

**[00:23:10 – 00:23:14]** and then feeds all the data, does pre-processing before feeding the

**[00:23:15 – 00:23:18]** data into our training clusters.

**[00:23:18 – 00:23:20]** So in this case, the characteristics are similar

**[00:23:20 – 00:23:24]** to the ranking and recommendation use cases where essentially we

**[00:23:24 – 00:23:27]** did benefit quite a bit from better single thread performance and

**[00:23:27 – 00:23:32]** better memory performance as well.

**[00:23:33 – 00:23:35]** Storage client and object caching, in both of these

**[00:23:35 – 00:23:39]** cases, essentially the key metric that we care about is read-write

**[00:23:39 – 00:23:46]** rates and P99 latencies, because we do again care about the end-to-end

**[00:23:46 – 00:23:50]** user experience when somebody is running a storage service or object

**[00:23:50 – 00:23:52]** caching service in the fleet.

**[00:23:52 – 00:23:54]** In both of these cases,

**[00:23:54 – 00:23:57]** We did have a balance of good and bad things.

**[00:23:57 – 00:24:00]** Essentially, the good thing is we are able to drive a similar

**[00:24:00 – 00:24:03]** amount of read-write throughput and latency with fewer number of

**[00:24:03 – 00:24:08]** cores, which allow us to save power and deliver better on per watt.

**[00:24:08 – 00:24:11]** The challenges we did see are around, if we try to run

**[00:24:11 – 00:24:16]** really high throughput services, Either way, kind of like...

**[00:24:16 – 00:24:20]** See challenges because of fewer number of codes compared to some of

**[00:24:20 – 00:24:21]** the other platforms that we run on.

**[00:24:21 – 00:24:25]** Or the kernel and networking stack is not as heavily optimized

**[00:24:25 – 00:24:27]** as we would like it to be.

**[00:24:27 – 00:24:29]** But again, these are things that we are taking learnings,

**[00:24:29 – 00:24:34]** iterating, and trying to improve on the next generation.

**[00:24:34 – 00:24:38]** Finally, I would like to highlight that our approach is always

**[00:24:38 – 00:24:42]** to squeeze every last bit of performance that we can.

**[00:24:42 – 00:24:44]** We don't leave anything on the table all the way from

**[00:24:44 – 00:24:49]** core microarchitecture optimization all the way to anything, all

**[00:24:49 – 00:24:52]** the layers in between, all the way to the application stack.

**[00:24:52 – 00:24:56]** Given we run a private cloud, we do have the ability to

**[00:24:56 – 00:25:02]** actually go investigate application code, investigate kernel code,

**[00:25:02 – 00:25:03]** and all layers in between.

**[00:25:03 – 00:25:07]** And so, essentially, we truly did end-to-end work on all

**[00:25:07 – 00:25:10]** of these applications that I was talking about in the last slide.

**[00:25:10 – 00:25:14]** This slide kind of talks about some of the key technical

**[00:25:14 – 00:25:17]** issues or technical optimizations that we were able to land,

**[00:25:18 – 00:25:22]** all the way from, essentially, SOC microarchitecture level

**[00:25:22 – 00:25:23]** to the application.

**[00:25:23 – 00:25:25]** So, taking SoC architecture for instance, right, like

**[00:25:25 – 00:25:28]** when we initially got the Grace platforms, they were

**[00:25:28 – 00:25:33]** not telemetry ready for like a really like large-scale deployment.

**[00:25:33 – 00:25:35]** There were counters that essentially we had to go

**[00:25:35 – 00:25:42]** enable through like iteratively as part of this co-engineering effort.

**[00:25:42 – 00:25:47]** Similarly, some of the power telemetry, as Shobet highlighted,

**[00:25:47 – 00:25:51]** we don't run these CPUs at max TDP.

**[00:25:51 – 00:25:53]** Essentially, we have a right balance.

**[00:25:53 – 00:25:56]** And so we did have to enable quite a lot of telemetry on the power

**[00:25:56 – 00:26:00]** side for us to be able to monitor how these systems are doing across

**[00:26:00 – 00:26:05]** a variety of applications and come up with the right operating power

**[00:26:05 – 00:26:09]** point for each of these use cases.

**[00:26:10 – 00:26:13]** At the system architecture level, we work with multiple

**[00:26:13 – 00:26:16]** memory vendors to enable higher memory speeds.

**[00:26:16 – 00:26:22]** So like we operate LPDDR at 85, 33 mega transactions per second.

**[00:26:22 – 00:26:25]** Initially, when the platform came out, only one of the memory vendors

**[00:26:25 – 00:26:29]** was ready, which is not really amenable for at scale deployment.

**[00:26:29 – 00:26:31]** So we then like essentially work with pretty much all the

**[00:26:31 – 00:26:35]** other memory vendors to embrace, like have them embrace LPDDR.

**[00:26:36 – 00:26:40]** In a server configuration and then make it work for our use cases.

**[00:26:40 – 00:26:44]** It did require us to iterate also on the RAS aspects.

**[00:26:44 – 00:26:47]** There are fundamental differences between the

**[00:26:47 – 00:26:52]** RAS on the DDR5 and LPDDR side.

**[00:26:52 – 00:26:57]** We did cross-stack optimizations all the way from our scheduler

**[00:26:57 – 00:27:04]** layer to be able to trade off some of the availability For

**[00:27:04 – 00:27:07]** like in favor of like benefiting from the power efficiency

**[00:27:07 – 00:27:10]** and bandwidth advantage of LPDDR, for instance.

**[00:27:10 – 00:27:13]** So it did require some amount of work to get there, but

**[00:27:13 – 00:27:17]** like we have been like pushing heavily on supporting LPDDR broadly

**[00:27:17 – 00:27:19]** across our several platforms.

**[00:27:19 – 00:27:23]** On the kernel side, like initially, like if you look at like the

**[00:27:23 – 00:27:28]** open source kernel, it's not really bolted, PGOed, and not really like

**[00:27:28 – 00:27:31]** really fine-tuned for high packet rate, high throughput applications.

**[00:27:31 – 00:27:34]** So there is a lot of work that was done to make the

**[00:27:34 – 00:27:38]** Linux kernel, like we basically operated on Linux 6.14 and above.

**[00:27:38 – 00:27:42]** And so if you see a bunch of patches that were pushed over

**[00:27:42 – 00:27:44]** the last years, they were like in.

**[00:27:45 – 00:27:48]** As part of this effort, we were able to land a bunch of stuff.

**[00:27:48 – 00:27:52]** And of course, at the application layer, we partner very closely

**[00:27:52 – 00:27:55]** with the NVIDIA DevTech team on some of these aspects.

**[00:27:55 – 00:27:58]** Things like FPGem optimizations for FP16, which we use on

**[00:27:58 – 00:28:01]** some of our recommendation systems.

**[00:28:02 – 00:28:08]** Things like Folly is essentially the C library that Meta uses

**[00:28:08 – 00:28:09]** across the board.

**[00:28:09 – 00:28:13]** And so we essentially had a lot of x86 intrinsics that

**[00:28:13 – 00:28:18]** we ported over to ARM, or for AR64.

**[00:28:18 – 00:28:23]** Like the SVT-AV1, libx264, those kind of optimizations

**[00:28:23 – 00:28:26]** for video transcode.

**[00:28:26 – 00:28:29]** Lucas and Matthias talked about the LLVM clang and some of the

**[00:28:30 – 00:28:33]** optimizations on the compiler side.

**[00:28:34 – 00:28:36]** For us to be able to eke out every last bit of performance

**[00:28:36 – 00:28:38]** in that space, right?

**[00:28:38 – 00:28:41]** So again, like this is a sampling of the amount of work done

**[00:28:41 – 00:28:44]** like there was much more that we ended up doing over the

**[00:28:44 – 00:28:48]** last couple of years, but like as part of this like

**[00:28:48 – 00:28:52]** We were able to like essentially eke out multiple tens of percentage

**[00:28:52 – 00:28:58]** of performance over out-of-box Like over out-of-box silicon that

**[00:28:59 – 00:29:02]** we would have otherwise looked at.

**[00:29:02 – 00:29:04]** So I think that's it, right? And so in terms of what's

**[00:29:04 – 00:29:08]** next, like we are continuing to invest in the Arm ecosystem and

**[00:29:08 – 00:29:13]** like we have a broad category of applications already well optimized

**[00:29:13 – 00:29:18]** on AR64, but there is a lot more to do in that space given the variety

**[00:29:18 – 00:29:22]** of applications we run in the fleet to the order of thousands.

**[00:29:22 – 00:29:25]** And so we have been like continually

**[00:29:26 – 00:29:31]** Continuously iterating over, making each application category

**[00:29:31 – 00:29:33]** heavily optimized for AR64.

**[00:29:33 – 00:29:38]** And of course, we are continuing to incorporate learnings from

**[00:29:38 – 00:29:40]** our Grace experience into making the future platforms

**[00:29:40 – 00:29:42]** better for everyone.

**[00:29:42 – 00:29:48]** So that's all I had, and we're happy to take questions.

**[00:29:56 – 00:29:58]** Do we have any questions?

**[00:29:58 – 00:30:01]** Any challenges you faced on the higher memory latency

**[00:30:01 – 00:30:03]** of LPDDR and had to work around?

**[00:30:03 – 00:30:07]** I know bandwidth is great, but latency was higher.

**[00:30:09 – 00:30:12]** For some applications, what that meant was essentially we had to

**[00:30:12 – 00:30:17]** operate towards the left-hand side of the bandwidth latency curve.

**[00:30:17 – 00:30:21]** So on a typical DDR5 system, if you were operating at, let's say, 65%

**[00:30:21 – 00:30:25]** utilization, we had to kind of...

**[00:30:25 – 00:30:29]** Trim down the bandwidth to operate in a more like linear range.

**[00:30:29 – 00:30:32]** That was one optimization we ended up doing.

**[00:30:32 – 00:30:36]** The other was that like we did have quite a bit of like

**[00:30:36 – 00:30:41]** back or iteratively tuned a bunch of prefetchers given

**[00:30:41 – 00:30:44]** like here the bandwidth per core was significantly higher.

**[00:30:44 – 00:30:47]** We are able to essentially make the prefetching, prefetcher

**[00:30:47 – 00:30:50]** tunings more aggressive to be able to help us hide some of the

**[00:30:50 – 00:30:54]** latencies which we would otherwise not do in like a more bandwidth per

**[00:30:54 – 00:30:56]** core constraint environment, right.

**[00:30:56 – 00:30:59]** And then finally there is like, it's not saying that

**[00:30:59 – 00:31:01]** it's perfect, right, like there are

**[00:31:01 – 00:31:05]** Applications like web serving, which are typically low IPC,

**[00:31:06 – 00:31:10]** they do care quite a bit about even code getting fetched from L3 caches

**[00:31:10 – 00:31:14]** or even from memory, where we are seeing performance challenges

**[00:31:14 – 00:31:18]** in which we are trying to, which need to be addressed as part of

**[00:31:18 – 00:31:21]** the next generation optimizations.

**[00:31:36 – 00:31:38]** Apparently, we have no more questions.

**[00:31:38 – 00:31:39]** Oh, there is one.

**[00:31:39 – 00:31:45]** So Jensen has talked about the new LPX as an extreme version of CPU.

**[00:31:45 – 00:31:49]** And obviously, these days, CPUs are very tight because of agentic AI.

**[00:31:49 – 00:31:55]** Do you see the tightness sustained throughout?

**[00:31:55 – 00:31:59]** Or do you think some of the CPU role would be

**[00:31:59 – 00:32:02]** replaced by those LPX?

**[00:32:02 – 00:32:04]** Sorry, can you repeat the last part?

**[00:32:04 – 00:32:08]** Maybe I didn't get the... Do you foresee any of the existing

**[00:32:08 – 00:32:14]** CPU workload being replaced by the new LPX-like system?

**[00:32:14 – 00:32:20]** Yeah, so I think we, generally, when we are mapping our use

**[00:32:20 – 00:32:25]** cases to the underlying infra, we have a more open-ended approach.

**[00:32:25 – 00:32:29]** Like, it's not like we are, like...

**[00:32:29 – 00:32:33]** We are not necessarily optimizing everything for CPU only, right?

**[00:32:33 – 00:32:36]** Like we have a portfolio of systems, like all the way

**[00:32:36 – 00:32:43]** from, like, very low CPU to very high GPU ratio, all the way to

**[00:32:43 – 00:32:46]** CPU only use case and the spectrum.

**[00:32:46 – 00:32:49]** And so generally, whenever we have a new use case, we

**[00:32:49 – 00:32:53]** do evaluate it across that entire spectrum of server platforms that

**[00:32:53 – 00:32:57]** we build, and then find the right and most optimal home for those.

**[00:32:57 – 00:33:00]** Thank you for watching! Having said that, some of the use

**[00:33:00 – 00:33:04]** cases we do anticipate and

**[00:33:04 – 00:33:08]** I believe we already published some of our work around the Grace Hopper

**[00:33:08 – 00:33:11]** platform where we were able to offload some of the work from the

**[00:33:11 – 00:33:17]** CPU to GPU in order to do better on recommendation systems, especially

**[00:33:17 – 00:33:20]** the ones with more complex models.

**[00:33:20 – 00:33:24]** So again, it depends heavily on specific use cases and

**[00:33:24 – 00:33:26]** which platform it maps to.

**[00:33:26 – 00:33:28]** I do anticipate some of the agent tech use cases which

**[00:33:28 – 00:33:34]** will map onto CPU-only platforms.

**[00:33:34 – 00:33:36]** I want to offer one point of clarification around this.

**[00:33:36 – 00:33:40]** Like LPX, the Grok stuff, that is low latency inference.

**[00:33:40 – 00:33:43]** If you saw the earlier CPU session where we talked about

**[00:33:43 – 00:33:46]** the CPU usage driven by agentic AI, or if you look at the

**[00:33:46 – 00:33:51]** workloads in Abhishek's slides,

**[00:33:51 – 00:33:56]** Those are not things that are in conflict with the

**[00:33:56 – 00:33:58]** positioning of LPX, right?

**[00:33:58 – 00:34:00]** They're for all very different cases.

**[00:34:00 – 00:34:05]** So I don't see the Gentic CPU demand one way or

**[00:34:05 – 00:34:09]** another factoring into the LPX portion of the roadmap.

**[00:34:09 – 00:34:12]** None of these use cases that you've seen today here or

**[00:34:12 – 00:34:17]** in the previous sessions are LPX or even GPU use cases.

**[00:34:17 – 00:34:20]** We've talked about the GPU, GPU-AIML use cases.

**[00:34:20 – 00:34:22]** So if it's three sides of a triangle.

**[00:34:22 – 00:34:27]** So you mentioned about, for the optimization, you had

**[00:34:27 – 00:34:31]** to do a lot of telemetry, including power telemetry and so.

**[00:34:31 – 00:34:37]** So was this telemetry done in an eBPF kind of way, or

**[00:34:37 – 00:34:40]** did you develop something in-house?

**[00:34:40 – 00:34:44]** Sure, so generally for telemetry, we...

**[00:34:44 – 00:34:48]** We primarily lean on either PERF or eBPF.

**[00:34:48 – 00:34:51]** The actual telemetry system that we use actually is also

**[00:34:51 – 00:34:54]** open source, it's called Dynalog.

**[00:34:54 – 00:34:57]** I mean, you can, again, clone it, run it on any machine.

**[00:34:58 – 00:35:01]** So there are three levels of, or maybe four levels of

**[00:35:01 – 00:35:03]** telemetry that we do, one which is more for performance

**[00:35:03 – 00:35:09]** deep dives, one which is for fleet health, and machine checking,

**[00:35:09 – 00:35:15]** fleet health, power management at a rack or a data center level.

**[00:35:15 – 00:35:21]** And then we also have like essentially telemetry for

**[00:35:21 – 00:35:24]** like the software stack side of things, right?

**[00:35:24 – 00:35:28]** Or like kernel profiling, application level profiling, right?

**[00:35:28 – 00:35:32]** So we use a combination of...

**[00:35:32 – 00:35:36]** What's available through Perf, plus what's available through

**[00:35:36 – 00:35:39]** out-of-band, like essentially the things that we end up

**[00:35:39 – 00:35:41]** pulling through the BMC interface.

**[00:35:41 – 00:35:46]** And then we also use kernel and probes for some of the

**[00:35:47 – 00:35:49]** more system-level stuff.

**[00:35:49 – 00:35:54]** So all of that typically gives us a more holistic picture for a server

**[00:35:54 – 00:35:58]** platform and an application that is running in a scale-out environment.

**[00:35:58 – 00:36:01]** And so like in this particular case when I was referring

**[00:36:01 – 00:36:06]** to telemetry, it was more the former which is like more like what

**[00:36:06 – 00:36:11]** we can read through either Perf in band or stuff that we had to

**[00:36:11 – 00:36:14]** read out of band through the BMC.

**[00:36:15 – 00:36:17]** Which is over I3C, yes.

**[00:36:17 – 00:36:20]** EBPF, the good thing about EBPF is it runs off of standard

**[00:36:20 – 00:36:24]** Linux kernel, and so as long as you have a distribution

**[00:36:24 – 00:36:26]** Linux, it works out of the box.

**[00:36:26 – 00:36:29]** So we didn't have to really tweak it much for this platform.

**[00:36:29 – 00:36:30]** Okay, I think we're done here.

**[00:36:30 – 00:36:33]** Well, let's thank our speakers one last time.