# How Revolut Runs Production Agents on Open Models: European Infrastructure, at a Fraction of the Cost (Presented by Together AI)

## Abstract

Key takeaways Learn what changes when the codebase your coding agents touch is subject to audit. Understand that coding agents that turn written descriptions into working code and vision agents that turn mockups into working interfaces are two different deployment decisions, and one is ready before the other. Explore whether mockup-to-interface agents are ready for the work your team actually does. Consider that European data residency decides where your agents can run, and that decision shapes the whole serving stack, so it belongs in the design from day one. See how open models now carry production agent workloads that used to require closed frontier APIs, with a cost difference that is large enough to change what you build.

## AI Summary

_No AI summary available._

## Transcript

**[00:00:06 – 00:00:11]** Alright, hello everyone, hopefully you can hear me.

**[00:00:11 – 00:00:15]** We have 15 minutes together; we'll talk about high performance

**[00:00:15 – 00:00:21]** engineering and self-hosting open source model for agentic coding.

**[00:00:21 – 00:00:25]** So it's going to be a technical one, so bear with me.

**[00:00:26 – 00:00:30]** So I'm David, I work for a company called Revolut.

**[00:00:30 – 00:00:35]** You may have seen us on the way in if you were flying in the airport.

**[00:00:35 – 00:00:40]** So we're a global digital bank, we have 80 million customers

**[00:00:40 – 00:00:42]** across 45 markets.

**[00:00:42 – 00:00:46]** We're a full licensed bank in seven of those markets,

**[00:00:46 – 00:00:51]** and as of July, we're Europe's highest valued private company.

**[00:00:52 – 00:00:55]** So we're doing okay.

**[00:00:56 – 00:01:01]** Part of the success of Revolut is execution, so it's relentlessly

**[00:01:01 – 00:01:05]** shipping new products, and so there's a high emphasis

**[00:01:05 – 00:01:07]** on our engineering culture.

**[00:01:07 – 00:01:11]** So obviously, AI is now extremely relevant.

**[00:01:11 – 00:01:14]** So let's talk about AI.

**[00:01:16 – 00:01:21]** What you see here is sample data of the productivity over

**[00:01:21 – 00:01:26]** 2,000 engineers since we started adopting AI in the company.

**[00:01:26 – 00:01:29]** So you're going to have two lines.

**[00:01:29 – 00:01:32]** The green line is going to be lead time, so that's from

**[00:01:32 – 00:01:37]** the moment an engineer, a

**[00:01:38 – 00:01:40]** fundamental engineer, picks up a ticket from the moment the

**[00:01:40 – 00:01:44]** feature is shipped, and the blue line is going to be coding time, so

**[00:01:44 – 00:01:49]** that's the actual time an engineer spends developing a feature.

**[00:01:49 – 00:01:54]** So to resituate the timeline, because AI moves so fast,

**[00:01:54 – 00:01:58]** Cloud Code came out in private review around February 2025,

**[00:01:58 – 00:02:03]** and that's the Cloud Code, SONET 3.7, that would delete

**[00:02:03 – 00:02:06]** your ProdDB and say sorry later.

**[00:02:06 – 00:02:11]** So we need to remember that when we talk about adoption.

**[00:02:12 – 00:02:14]** So the first impact we had was back in January, that's

**[00:02:14 – 00:02:16]** the first we introduced Gemini.

**[00:02:16 – 00:02:20]** That was the first time engineers were able to use AI with private

**[00:02:20 – 00:02:25]** data and private code, and so we saw an impact of 10%

**[00:02:25 – 00:02:30]** productivity across lead time, 20% impact across coding time.

**[00:02:30 – 00:02:33]** That was people no longer having to look for documentation and people

**[00:02:33 – 00:02:37]** not having to read Stack Overflow.

**[00:02:38 – 00:02:40]** Around January, after some testing, that's when we started

**[00:02:40 – 00:02:45]** launching Cloud Code.

**[00:02:45 – 00:02:48]** From testing, we were already a bit skeptical about the

**[00:02:48 – 00:02:52]** cost and the ROI of Frontier model, and so we launched Cloud

**[00:02:52 – 00:02:56]** Code with limits per engineer, so engineers had a budget.

**[00:02:56 – 00:03:00]** And so what you see is very little impact in overall productivity,

**[00:03:00 – 00:03:03]** and that's because, as we think, the budget that we

**[00:03:03 – 00:03:08]** set per engineers, $300 per month, was not enough to carry an

**[00:03:08 – 00:03:11]** engineer through the whole month.

**[00:03:11 – 00:03:16]** So we were curious about open source and the promise it had.

**[00:03:17 – 00:03:21]** So around April 26, this year, is when we launched our self-hosted

**[00:03:21 – 00:03:22]** open source model.

**[00:03:22 – 00:03:27]** We used GLM, and the promise was unlimited token for all

**[00:03:27 – 00:03:29]** of our engineers.

**[00:03:30 – 00:03:33]** And here what you see is 70% improvements in lead

**[00:03:33 – 00:03:37]** time productivity and 60% improvements in coding time.

**[00:03:37 – 00:03:41]** So obviously, much better results.

**[00:03:42 – 00:03:45]** What's interesting to note is the different impacts between

**[00:03:45 – 00:03:47]** coding and lead time.

**[00:03:47 – 00:03:49]** Once you don't have to worry about what you spend your tokens

**[00:03:50 – 00:03:54]** on, you can use it for things that you think are not worth it,

**[00:03:55 – 00:03:59]** like searching confluence, dealing with JIRA tickets, or just looking

**[00:03:59 – 00:04:01]** for the latest spec across Slack.

**[00:04:02 – 00:04:04]** And so what we see with the utilization that the engineers

**[00:04:04 – 00:04:08]** had with their skill and their tools is that they use more

**[00:04:08 – 00:04:12]** these free tokens for non-coding tasks, and that shows that the

**[00:04:12 – 00:04:19]** bottleneck was no longer coding, but it's the rest of the work.

**[00:04:19 – 00:04:23]** So let's talk about how to get there.

**[00:04:25 – 00:04:30]** If you want to keep your engineers happy, what we found is, for

**[00:04:30 – 00:04:34]** the 2,000 engineers that we have, you need around 56 GPUs,

**[00:04:34 – 00:04:38]** and that's hosting a large model like GLM 5.3, right?

**[00:04:38 – 00:04:45]** So that's 200, 800 billion parameters, and that works nicely.

**[00:04:45 – 00:04:50]** What you want to aim for is a time to first token below three seconds.

**[00:04:50 – 00:04:53]** This is purely empirical.

**[00:04:53 – 00:04:56]** How we came up with this is as soon as it exceeds five seconds,

**[00:04:56 – 00:05:02]** my Slack inbox explodes, and so that's how we set the limits.

**[00:05:02 – 00:05:05]** In terms of throughput, 200 tokens per second is more than enough.

**[00:05:05 – 00:05:09]** Modern engineering is no longer looking at the terminal.

**[00:05:09 – 00:05:12]** The streaming of token is more launching parallel threads,

**[00:05:12 – 00:05:15]** and so between the threads completing in 10 minutes versus 12

**[00:05:15 – 00:05:19]** minutes, it doesn't really matter.

**[00:05:20 – 00:05:23]** The achievable result that we have is a blended cost per million token

**[00:05:23 – 00:05:29]** for GLM 5.3 of 30 cents, and so that's one-tenth the cost of what

**[00:05:29 – 00:05:35]** we are paying with the equivalent frontier model, so Opus and so on.

**[00:05:35 – 00:05:39]** Top of that, your invoice is flat, so you're trading essentially a

**[00:05:40 – 00:05:44]** spiky invoice with a spiky latency.

**[00:05:44 – 00:05:47]** Now what's important to note is we still have a very high

**[00:05:47 – 00:05:50]** usage of the frontier model, right, from OpenAI and Anthropic,

**[00:05:50 – 00:05:52]** but they're no longer the default.

**[00:05:52 – 00:05:54]** They are the reserved option with still a very small budget

**[00:05:55 – 00:06:00]** for the toughest problem, and open source is now the default.

**[00:06:04 – 00:06:06]** To get there is quite a journey.

**[00:06:06 – 00:06:09]** So out of the box, it's not gonna work, and self-hosting

**[00:06:09 – 00:06:13]** open source is not trivial at all.

**[00:06:13 – 00:06:16]** So on this chart, what you're gonna see is the blue line

**[00:06:16 – 00:06:18]** is gonna be the demand, so that's the request per second that

**[00:06:18 – 00:06:22]** we get from our engineer, sending to our infra, and the red line is

**[00:06:22 – 00:06:27]** the latency, time to first token, is what we're optimizing for.

**[00:06:27 – 00:06:29]** So we're trying to keep the red line below three second,

**[00:06:29 – 00:06:33]** and if you see back in June when we started tracking, our time

**[00:06:33 – 00:06:38]** to first token was above 10 seconds for four requests per second.

**[00:06:38 – 00:06:41]** So obviously, we're missing the mark.

**[00:06:41 – 00:06:45]** What's important to note is across the whole charts, we came

**[00:06:45 – 00:06:47]** to the number of GPU constants.

**[00:06:47 – 00:06:49]** So all of the variation in performance that you see is

**[00:06:50 – 00:06:54]** only the tuning of the parameters of the deployment.

**[00:06:54 – 00:06:59]** In the end, all the way back in August, we achieved to

**[00:06:59 – 00:07:02]** accommodate a load of more than 8 requests per second, and

**[00:07:02 – 00:07:04]** our latency is below 1.8 seconds.

**[00:07:04 – 00:07:08]** So that's 88% improvement in performance for 78%

**[00:07:08 – 00:07:11]** increase in demand.

**[00:07:11 – 00:07:15]** So the message, if anyone wants to pursue self-hosted,

**[00:07:15 – 00:07:19]** is you absolutely need to resist scaling, because if you

**[00:07:19 – 00:07:23]** scale your infrastructure as soon as your traffic increases, you're

**[00:07:23 – 00:07:26]** not going to hit your cost targets.

**[00:07:26 – 00:07:30]** So there's no magic formula for how to tune the deployment,

**[00:07:30 – 00:07:34]** but a few things that may help you.

**[00:07:34 – 00:07:37]** First thing to note is that long context is a scam.

**[00:07:37 – 00:07:42]** Even though models are available with 1 million context window,

**[00:07:42 – 00:07:46]** the cost in KV cache for having 1 million context is not worth it.

**[00:07:46 – 00:07:48]** So you want to limit to 256k.

**[00:07:48 – 00:07:50]** That's more than enough.

**[00:07:50 – 00:07:52]** We also know that intelligence drops 90% as soon as your

**[00:07:52 – 00:07:57]** context exceeds 90%, so you're just encouraging by behavior.

**[00:07:58 – 00:08:01]** If you're hosting anything on Blackwell, so B200, B200

**[00:08:01 – 00:08:05]** onwards, really important as well to consider a small quantization.

**[00:08:05 – 00:08:10]** So you want to use NVFP4, MXFP4, no longer FP8.

**[00:08:10 – 00:08:15]** So hosting FP4 versus FP8, you're saving already 40%

**[00:08:15 – 00:08:23]** of GPU, and the difference in performance is not even noticeable.

**[00:08:24 – 00:08:26]** Another thing to consider is going to be tensor parallelism.

**[00:08:26 – 00:08:31]** So even though your model will fit on a TP4, so four nodes

**[00:08:31 – 00:08:37]** GPU, you may need more KV cache headroom, and so a TP8 may give you

**[00:08:37 – 00:08:39]** the performance that you need, even though it fits, and so you need

**[00:08:39 – 00:08:42]** to test these parameters as well.

**[00:08:43 – 00:08:46]** Lastly, on the more detail, you want to check your cache hits.

**[00:08:46 – 00:08:49]** Any agent decoding, you want to be in the high 90% for the cache

**[00:08:49 – 00:08:57]** hits, and then you want to track, you can try, make sure to have

**[00:08:57 – 00:09:00]** spec decoding, and you can split your pre-fill and your decode, so

**[00:09:00 – 00:09:04]** desacpd to make sure that it works.

**[00:09:06 – 00:09:10]** So, talking about how to get this into production, you need

**[00:09:10 – 00:09:11]** a few things to get this working.

**[00:09:12 – 00:09:14]** After you have your GPU and your model, you're gonna

**[00:09:14 – 00:09:15]** need the gateway.

**[00:09:15 – 00:09:18]** The gateway is important because you wanna make sure that you

**[00:09:18 – 00:09:21]** can rate limit the traffic, to make sure you don't have a

**[00:09:21 – 00:09:25]** single engineer that's monopolizing your entire stack.

**[00:09:26 – 00:09:28]** On top of that, you want to have a local proxy.

**[00:09:29 – 00:09:32]** So a local proxy is going to allow you to do model routing, but leave

**[00:09:32 – 00:09:36]** it at the hands of an engineer to see if they want to scope

**[00:09:36 – 00:09:41]** with a frontier model and implement with an open source, or transfer

**[00:09:41 – 00:09:45]** images recognition to a vision model when your main trend is

**[00:09:45 – 00:09:48]** going to be with a text-only model.

**[00:09:50 – 00:09:53]** In terms of model, there's a way to turn it off, so the choice that we

**[00:09:53 – 00:09:58]** make systematically in Revolut is to prioritize the best intelligence

**[00:09:58 – 00:10:00]** and sacrifice the multi-modality.

**[00:10:00 – 00:10:04]** So GLM 5.3 is a text-only model, but you can circumvent

**[00:10:04 – 00:10:07]** that by just routing images automatically to a cheap model

**[00:10:07 – 00:10:11]** like Gemini Flash or Luna.

**[00:10:12 – 00:10:15]** And then lastly, what matters is going to be

**[00:10:15 – 00:10:16]** your harness or your app.

**[00:10:16 – 00:10:20]** So you're free to use open code, cloud code, Codex.

**[00:10:20 – 00:10:23]** All of these harness are going to be compatible with

**[00:10:23 – 00:10:26]** an open AI endpoint.

**[00:10:26 – 00:10:30]** What we find is, in modern developments, engineers don't

**[00:10:30 – 00:10:31]** really like the CLI anymore.

**[00:10:31 – 00:10:35]** They'd rather have an app with a nice UI, threads well-organized

**[00:10:35 – 00:10:38]** in projects and folder, so they can multitask instead

**[00:10:38 – 00:10:42]** of just watching tokens stream.

**[00:10:46 – 00:10:49]** Last word for the CIO in the room.

**[00:10:49 – 00:10:52]** Obviously, Revolut is a regulated entity, and so everything

**[00:10:52 – 00:10:57]** we do as a bank is highly scrutinized by regulators

**[00:10:57 – 00:11:01]** and auditors, and whenever we mention using Chinese models,

**[00:11:01 – 00:11:05]** we have heightened scrutiny.

**[00:11:06 – 00:11:08]** So what's important to note is, even though the model

**[00:11:08 – 00:11:12]** may have been Chinese-produced, you're just posting the weights

**[00:11:12 – 00:11:16]** on the EU or US service provider or your own data center, and

**[00:11:16 – 00:11:19]** so in terms of data privacy, this is even better than sharing

**[00:11:19 – 00:11:23]** your data with OpenAI or Anthropic.

**[00:11:24 – 00:11:27]** What matters as well in terms of auditability is that your

**[00:11:27 – 00:11:30]** agent decoding is not considered an autonomous agent, because this

**[00:11:30 – 00:11:33]** requires a lot more guardrails.

**[00:11:33 – 00:11:35]** So you want to make sure that you can prove that you have

**[00:11:35 – 00:11:39]** a human in the loop, and the way you achieve that is by having

**[00:11:39 – 00:11:44]** a sandbox that's going to allow some commands and deny some others.

**[00:11:44 – 00:11:47]** So the shortcut is you can allow grep, but you can't

**[00:11:47 – 00:11:50]** authorize gradle.

**[00:11:52 – 00:11:55]** Since this is a privacy, whether you host on some influence

**[00:11:55 – 00:12:00]** provider, some new cloud, or a local location, everyone's

**[00:12:00 – 00:12:01]** going to fall out of the box.

**[00:12:01 – 00:12:06]** Zero data retention, so on that front, you're covered, and then

**[00:12:06 – 00:12:10]** you want to make sure that you set up an open telemetry so that you

**[00:12:10 – 00:12:14]** can investigate the prompts that your engineers are sending across.

**[00:12:14 – 00:12:17]** You may be surprised to see that you may have bad actors

**[00:12:17 – 00:12:21]** inside your own organization.

**[00:12:22 – 00:12:23]** All right, thanks, everyone.