I was invited to rejoin BNR's radio show De Grote Tech Show (The Big Tech Show) about tokenmaxxing, rising AI costs, and the movement towards open weights models. BNR Nieuwsradio is a national commercial radio station that focuses on (economic) news. De Grote Tech Show is their radio show that focuses on innovation and tech developments.

You can find the podcast episode on the BNR website.

To make this interview accessible for our non-Dutch audience, we provide the translated transcript below. The transcript has been cleaned up a bit for readability.

Transcript "How to Bypass Expensive AI Tokens"

Joe van Burik: All the buzz and good ideas aside, let's head back down to earth now, for those of you who actually use AI for development and implementation in systems. Recently the major AI companies, like OpenAI and Anthropic, have raised their prices, meaning you now pay for consumed tokens rather than all-you-can-eat per month. Which caused costs to suddenly skyrocket. And AI companies for legal tools sounded the alarm about this in the FD, the Dutch financial newspaper, earlier this week. We've previously heard developers talk about this in this podcast too. So, what now? Should we start using Chinese models instead? Those are cheap and sometimes open weights so you can run them locally. But what kind of output will you get then? Our guide in all these questions about costs and values in AI use is Judith van Stegeren, machine learning engineer at Datakami and friend of our tech show. Good to have you here, Judith.

Judith van Stegeren: Yes, great to be here. Good morning.

Ben van der Burg: Good morning.

Joe van Burik: Just a few months ago, token-maxing was a thing. Running as much AI as possible to see what works. On De Technoloog, you talked a while ago about the "duct-tape phase" of AI. That's all coming to an end now, I think, or not?

Judith van Stegeren: No, not at all. Everyone is still experimenting with everything left and right.

Joe van Burik: Oh! And it doesn't matter to them what it costs?

Judith van Stegeren: People are nowhere near done with experimenting.

Ben van der Burg: And you don't see any stability anywhere? It's not like: hey, this [AI stuff] is starting to coalesce now?

Judith van Stegeren: It's only starting to coalesce a bit at the companies that knew what they were doing from the beginning. They're going about it a bit more rational. The people who are afraid of missing the boat are still "YOLO" burning an incredible amount of money left and right. And this also goes for those companies that feel they always need to use the absolute best of what's available [in AI]. So those are, for example, hedge funds, or people doing a lot of strategic work with LLMs.

Joe van Burik: So these companies are knowingly letting costs rise, out of a kind of FOMO, because they're afraid their equally wealthy competitors come up with something better using the best AI available — and then they will be left behind. So they're intentionally investing, so to speak.

Judith van Stegeren: Yes, though many companies have been a bit shocked by the price over the past few months. So now we're seeing a sort of backtracking movement. And I think [the token prices are also rising] under pressure from the IPOs, right, from the major frontier labs that suddenly want to go public. The frontier labs want to close up their money pit, right, that black hole that's eating all their cash, so [their financials] look better when they go public. So that means prices for AI are going up.

Ben van der Burg: Can you give an example of a company that knew what they were doing from the beginning, so using AI went well? And then give an example of "we have no idea what we're doing, we'll see what happens", which led to problems with AI usage later on?

Judith van Stegeren: Oh yes. Well, of course I can't name specific companies, but the companies that already had a team with machine learning engineers beforehand and who truly understand what an LLM is and does, and also what the weak sides are, yes, they know how to use such a model with precision and get a lot out of it. So: well-defined tasks, using your own data, regularly checking the quality of what comes out, and so on. And on the other hand there are companies that don't have an in-house machine learning team, so those are just... well, they might not even have engineers in-house...

Ben van der Burg: The vibecoders!

Judith van Stegeren: ...but they might be using a ChatGPT interface or something. And these people are are under the impression you can talk to a language model as if you're talking to a person. So then you typically don't restrain yourself in terms of how much data or text you throw in, right? And that costs a fortune.

Joe van Burik: Yes, okay. What I also find striking: I've also understood that the cost per token is actually going down. Is it more that the tasks themselves cost many more tokens because people want increasingly difficult, better things and therefore the net result costs more money? How does that calculation work out?

Judith van Stegeren: Well, I'll tell you something even more striking: certainly with Anthropic, the new generation of models, so Opus 4.7, 4.8, and also the new Sonnet 5 that came out yesterday, they are much less efficient because they have a new tokenizer. So, that's the piece of software that converts words into numbers that the LLM uses to compute. And those things are 40% less efficient. So, it appears that the cost per token is going down, but that's because otherwise everyone would be very angry when they find out that the less efficient tokenizer actually costs 40% more money.

Ben van der Burg: And why did they do that?

Judith van Stegeren: They probably made different design choices. Because a tokenizer, it chops words into pieces. And you can, for example, do a token per letter, or per whole word, or even per phrase. So those are probably design choices with which they hope performance goes up. And on the other hand, maybe they just want to cheaply rake in some extra revenue. That could of course also be the case.

Joe van Burik: Now for a moment, right, imagine you're a company, you're not a hedge fund. You wanted to do things with AI, you were doing well, but gosh, now your bill is running up considerably. What should you do then? Should you come up with all kinds of ways to work more efficiently? Should you actually just do some things the old-fashioned manual way instead of with AI? What advice is there?

Judith van Stegeren: Yes, well, going back to manual — I don't recommend that. Because there's probably something on the market that can perform your task just fine. We tech people always say: "Implement first, optimize later." So if you've found something that works well for you, only then do you look at: how can we make this cheaper? And that can be with an open weights model, for instance. And those don't even have to be Chinese models, right, although those are very popular. For example, Nvidia also has open weights models (Nemotron), and Mistral from France also has open weights models. So you need to look around a bit: what else is on the market, what does it cost? And also: do you have the knowledge in-house to deploy it yourself? Because not everyone can do that, of course.

Joe van Burik: But then it still feels to me like the solution to the problem, as you just described, sounds quite simple on the face of it. But to actually implement it well, because it's not plug-and-play like the API from one of the big developers, but you have to do it all yourself...

Ben van der Burg: You still need technical expertise.

Joe van Burik: Yes, you still need to know how to do this. So then they end up asking you for help, I'd imagine. Isn't this now also an opportunity for AI technical advisors in the world?

Judith van Stegeren: Yes, I think so, because for a long time it was somewhat hidden from the companies using AI — what it actually costs. And also how complicated it is to turn it on and off on demand for your company. There are also a lot of parties now that are trying to bridge that gap, right. So between the tech people who can deploy models themselves, and those big frontier labs. So you have, for example, OpenRouter, that's a good example. You can obtain an API key from them and try out all sorts of different open weights models. Only, yes, privacy and data security still pose a potential problem Because I'm really not going to just send all my company data or my clients' IP to some random inference provider on OpenRouter.

Ben van der Burg: Really, why not? [laughs]

Judith van Stegeren: Absolutely not, haha.

Ben van der Burg: Just kidding. I have another question. Earlier this week I spoke with someone who had, so to speak, run his own model locally for a specific solution — it was with video in this case, a specific solution, running locally so he no longer had token costs. Do you see that more often, or is this one in a thousand?

Judith van Stegeren: No, no, that's certainly possible too. You do need to be somewhat technical for it, it can be very tricky to use those models because they need to be set up and deployed very specifically for your hardware. So that's actually where things easily break. And the models that can run on consumer hardware are somewhat smaller and also somewhat less performant.

Ben van der Burg: Yes, he had great GPUs for that.

Judith van Stegeren: And especially if you're doing things with video models, that takes a lot of memory. So you probably need a cluster of GPUs in your shed, humming away, and then you also need the technical knowledge to get those video models running on that cluster.

Joe van Burik: I still want talk about China for a moment, even though you just said: you really don't have to go to China for an open weights model. But also partly because we first spoke with you, that was a year and a half ago with the DeepSeek moment. In the meantime we've also had Qwen and Moonshot and other Chinese alternatives. Now I want to discuss two things: firstly, I heard a few months ago that actually quite a few parties in the US also like to use Chinese models, because they're cheap. And then there's also the whole geopolitical angle. But, are there still good arguments to be made — that's my question to you — for actually using a Chinese model? Or do you say: well, choose Nvidia's or Mistral's?

Judith van Stegeren: Well, the Chinese models are pretty good for [their size]. The new models that are so good are also very large, so you still need many and expensive graphics cards, right, to be able to run them. But for example GLM-5.2, that's one of those new very good models, I hear from everyone that they're switching to it, and the performance is actually comparable to a small Anthropic model for some tasks. The only caveat you have to make is that you have to realize that because it's a Chinese model, Chinese norms and values are encoded in it, simply because of the nature of the training data.

Joe van Burik: Ah, here we go again! So, yeah, no, but that remains an issue. So we have those Chinese values, but also — Ben and I were talking about this earlier — that you have to keep moving along with the developments in AI, and that's easier for some than others. You know, you can't constantly search for a different solution every few months because it's more efficient, better, cheaper, right?

Judith van Stegeren: Yes, that's true, and that's also where the difficulty lies. But that's because the field is just still so young. So it's actually remarkable how incredibly fast the whole of society is moving along with this new development, because the technology is just still racing ahead. It's as if half of society is participating in a space race. Because it's super high-tech, and at the same time everyone is already using it.

Ben van der Burg: I have another question. Do you think, because the marginal costs are not zero, because you have these energy costs... The more I use it, the more costs I incur, you know. If we copy software, that doesn't double the costs of software, but with generative AI we do. Do you think that will ever be solved? Because [society] always had the idea: you can use an unlimited amount of AI. But, no, this doesn't hold up: the more users, the more costs. So where will we end up?

Judith van Stegeren: Yes, I think so, because new models with new architectures keep coming out, which are also more efficient, so that you can achieve very good results with a small model that fits on one GPU, for instance. And if we keep getting smaller and smaller models, then at some point they can just run on a gaming computer [or mobile phone] again. And then maybe everything we're now doing with thousands of GPUs can just be handled on much smaller computers in your own home. So I do have confidence in that.

Joe van Burik: Yes. And there's another idea I had, also because of an analysis I read this week. Look, there are a few major tech companies that are clearly lagging when it comes to their own foundation models. For example, Microsoft or Meta. Is this now an opportunity for them to undercut OpenAI, Anthropic, and Google with a more compact, efficient, better model? Or do you say: well, ultimately those big models running through their APIs remain so important that that race is simply over?

Judith van Stegeren: No, the race is far from over, because if you can generate a lot of revenue, that also attracts all kinds of competition. And there are certainly a lot of small labs now busy innovating on smaller models. And certainly Microsoft and Meta that you just mentioned have an enormous amount of data and firepower, so all kinds of innovative things can still come out of these companies. And Meta has of course already left a mark on generative AI very early on, because they really committed to open-source or open weights with the LLaMA models.

Joe van Burik: But do I hear you also saying: hey, small AI labs in the Netherlands, now is the time to build an efficient model? Or, do we still not have that capability in the Netherlands?

Judith van Stegeren: Oh, I would love that. We do of course have a lot of red tape in the Netherlands... so I'd have to see it to believe it. But yes, that would be great. To give a very concrete example of those new models that are being innovated: you have for example the models by Liquid AI. Liquid AI is also an AI lab. And they say: "You know what we do? We make our model smaller by throwing away the weights that store knowledge, and we're only going to focus on weights that deal with reasoning." Weights are the core elements of a large language model. And then we trust our users to supply all the knowledge the model needs via the context. The resulting model is quite stupid, because it doesn't know anything, but the model can still reason very well about its input. So it can still perform complex tasks, even though it's very small. Those kinds of innovations are truly fantastic, because those types of models can easily run on your mobile phone.

Ben van der Burg: Yes, fantastic. Yes, and also, then you have to maintain the knowledge yourself, because otherwise all your knowledge goes to the AI, and then you have no knowledge left yourself. So it forces you to retain knowledge.

Judith van Stegeren: Well, we're obligated to do that anyway, because...

Ben van der Burg: ...but unfortunately many people think: I'm fine with that! [laughs]

Judith van Stegeren: Keeping your own data in order is, I think, truly the most important thing. Because with that you can also stand up against Big Tech, which will soon be able to put whatever price tag they want on such a model. And as long as companies themselves have good data management and can make their own evaluation sets, so that they can very easily switch models for example, yes, then you're in a strong position.

Joe van Burik: Well, I'm a good deal wiser again and also somewhat reassured. How about you, Ben?

Ben van der Burg: Yes, wonderful. Always good talking to you, Judith.

Joe van Burik: Very nice. Thank you, Judith.