Code Mode, Two Tools, and an MCP Can Save Your LLM Context

One of the tools I'm building is a custom MCP server, based on the Cloudflare Code Mode idea, that lets me interact with local services I'm developing through their APIs as a local client, without having to do it myself.

It works so much better than I expected. I was both impressed and excited by the possibilities this approach offered. But I encountered a wall: absolutely no one I talked to about this shared my enthusiasm or had a vision of how powerful this new approach could be.

So, I did what I do best: I wrote about it.

This article is that write-up.

Context is Everything ​

One of the things we have to care about most when using LLMs is the context. The context is that window of text that the LLM sees and uses to generate its answer, filled by the provider, the user, and the model itself.

The OpenAI model spec defines five roles that can be used to fill the context:

• system: messages added by OpenAI
• developer: from the application developer (possibly also OpenAI)
• user: input from end users, or a catch-all for data we want to provide to the model
• assistant: sampled from the language model
• tool: generated by some program, such as code execution or an API call

All these roles and their messages are stacked together to form what we call the context. Think of it as a long chat history, where the LLM can see everything that has been said and done.

The context is limited by the model's maximum context length, which is usually measured in tokens. For example, GPT-5.5 has a maximum context length of up to 1M tokens. But there's a catch: the price increases after 272k tokens, and the model's performance starts to degrade after 50% of the maximum context length (which isn't really true for OpenAI models if you stay within the 272k-token limit). And if the context exceeds the maximum context length, older messages may be truncated or removed, causing the loss of valuable information that the model could have used to generate better answers.

Of course, removing older messages is not the only way to make room. An application can compact the context by asking a model to summarize those messages, then replacing them with a single compacted message.

This preserves the decisions and constraints that matter while freeing up space. This summary will omit or distort details, so guiding the LLM when summarizing is essential. You can tell the LLM what will be the next actions to help the model keep the relevant information.

So you have to be careful with the context, taking care of it as if it were the apple of one's eye.

It means that you can't just fill the context with a large amount of text and expect the model to both find the relevant information and generate a good answer. You have to provide only relevant context.

That's one of the reasons RAG, Retrieval-Augmented Generation, is so important.

But RAG can't solve everything. It performs well at adding external and specific knowledge to the context on demand, but it isn't about executing anything. That's an issue.

LLMs Know Code ​

I am a software engineer. I know how to write code, and if you're reading this, you probably know how to write code too.

Since late 2025, you can't have missed that LLMs are now able to write code, and write it well. It's gotten to the point that more than 95% of the code I deliver to production or use is generated by LLMs using an agent.

We can take advantage of this to solve the context problem. That may sound counterintuitive. But let's have a look at the Cloudflare Code Mode article.

MCP server designers are encouraged to present greatly simplified APIs as compared to the more traditional API they might expose to developers. [...] LLMs asked to write code against the full, complex APIs normally exposed to developers don't seem to have too much trouble with it. Why, then, do MCP interfaces have to "dumb it down"?

What do they mean by that?

LLMs can call multiple tools in a row, and sometimes in parallel. But when one call depends on the result of another, the LLM has to receive that result in a later turn. Imagine the following scenario: you have a tool that can read a file from a drive, and another tool that can create a new file in the drive. Then, you want to create a new file containing all the titles from a file in the drive.

What will the LLM do?

To solve this, the LLM will make two calls over two turns:

1. Call the first tool to read the file. This action will return the content of the file, which will be added to the context.
2. Call the second tool to create the new file containing all the titles. The tool call, containing the titles extracted from the first tool's output, will be added to the context.

Visually, it looks like this:

read_file(path="my-file.md")

But this is far from optimal.

The context is bloated with content from the file that the LLM does not really need to see. Extracting titles can be done without the LLM knowing the content of the file. This also means that for a confidential file, the LLM, and by extension the provider, will have access to the content of the file, which is not ideal. The non-deterministic nature of LLMs also means that title extraction may not be perfect. That could lead to real business trouble. And finally, this doesn't scale. If the document is 1M tokens long, the LLM will have to read it all, and the context will be bloated with a lot of unnecessary information.

Even though this example may sound simple and unrealistic, it is not. If we want our AI to perform complex tasks, we have to find a better way to do it. That's where Code Mode comes in.

With Code Mode, you ask the LLM to write code that will perform the task for you. To inform the LLM about the API it can use, you provide type interfaces in the tool description. The LLM will then generate code that will call the tools in the right order and with the right parameters to perform the task.

In our example, the tool description will look like this:

Execute JavaScript code against the Drive API.

Available in your code:
interface Drive {
  listFiles: () => Promise<{ title: string, filePath: string }[]>;
  readFile: (filePath: string) => Promise<string>;
  createFile: (filePath: string, content: string) => Promise<void>;
}

declare const drive: Drive;

Your code must be an async arrow function.

Example:

// Create a table of contents from all files in the drive
async () => {
  const files = await drive.listFiles();
  const titles = files.map(file => file.title);
  await drive.createFile('table-of-contents.txt', titles.join('\n'));
};

Then, the LLM will generate the following code to perform the task:

// Create a new file containing all the titles of a file in the drive
async () => {
  const content = await drive.readFile('my-file.md')
  const titles = content.split('\n').filter(line => line.startsWith('#')).map(line => line.replace(/^#+\s*/, ''))
  await drive.createFile('titles.txt', titles.join('\n'))
}

The tool returns { success: true }. So, the context of the LLM is now only the tool call and the result of the tool call, which is way better than having the whole content of the file in the context. Provided the execution environment cannot leak it, the LLM does not need access to the content of the file.

interface Drive {
  readFile: (path: string) => Promise<string>;
  writeFile: (path: string, content: string) => Promise<void>;
}

The LLM can still generate the wrong code. But the data processing is now explicit, repeatable, and testable. Once the code is validated, title extraction is no longer left to the LLM. Huge.

Two Ways to Split a Text ​

Imagine the following situation.

You're scraping a webpage to extract the content of an article to feed a news aggregator. To do so, you're using Cloudflare Browser Run, which can fetch a webpage and return its content as Markdown.

But here's the catch. The complete webpage is returned, which includes the article content, but also the header, the footer, the sidebar, and all the other stuff that isn't relevant to the context. You only want to extract the article content.

From now on, you have two ways to do it. Three ways, actually, if we consider that we can ignore the problem and just feed the whole content to the LLM.

Rewriting the Content ​

The first and simplest way would be to ask the LLM to rewrite the content and keep only the relevant part. The LLM will then output the article content. Simple. And highly inefficient.

You may have seen that output tokens are way more expensive than input tokens. For GPT-5.5, input tokens cost $5 per 1M tokens, while output tokens cost $30 per 1M tokens. So, if you have a 100k-token webpage and the article content is only 10k tokens, you will pay $0.50 for the input tokens and $0.30 for the output tokens. That's a total of $0.80 to extract 10k tokens of content, with no certainty that the LLM will perfectly rewrite the content.

Splitting the Content ​

The second way is a bit more complex, but way more efficient. The idea is to give the whole webpage to the LLM and ask it to generate a function that will extract the article content from it. That way, the LLM will only output the function, which is a few tokens, and then you can execute the function to extract the article content.

To give you an idea, the function could look like this:

() => content.split('#')[1].split('next steps.')[0].trim() // The article content is between the first H1 and the "next steps." text. That's the power of an LLM: it adapts to the content of the webpage and generates a function that extracts the article content without having to hardcode it.

This function is only 18 tokens long. Using GPT-5.5 for both approaches, you will pay $0.50 for the input tokens and $0.00054 for the output tokens. That's a total of $0.50054 to extract 10k tokens of content, with a small function that you can inspect and test.

Code Mode also opens the door to using less expensive models to write the function. For example, GPT-5.4-nano is priced at $0.20 and $1.25 per 1M input and output tokens, respectively. With this model, you will pay $0.02 for the input tokens and $0.0000225 for the output tokens. That's a total of $0.0200225 to extract 10k tokens of content, with a small function that you can inspect and test.

$0.80 for a model rewrite vs $0.50054 for a small function using the same model, or $0.0200225 using a less expensive model.

The choice is yours, but mine is clear. Code Mode is the way to go.

The OpenAPI Spec Issue ​

Imagine an OpenAPI spec containing 1.7M characters, which is roughly 425k tokens.

That's way too much to feed to the LLM, and this approach can't scale. If the OpenAPI spec exceeds the maximum context length, the LLM will not be able to see the whole spec, and even if it does, the context will be bloated with unnecessary information.

But with only the OpenAPI spec, and without any knowledge of the business logic, the LLM was able to complete quite a complex task in a couple of minutes using less than 100k tokens of context. That's insane.

That's a true story, and it was solved using the custom MCP server I mentioned earlier. But that's also Cloudflare's story with its MCP server. The first line of the README makes the context problem explicit:

A token-efficient MCP server for the entire Cloudflare API. 2500 endpoints in 1k tokens, powered by Code Mode.

Their OpenAPI spec is around 2M tokens. At that point, that's not huge. That's unusable.

Code Mode to the Rescue ​

For our OpenAPI spec problem, our Code Mode solution will consist of two tools:

1. Search, which allows the LLM to search the OpenAPI spec for relevant endpoints and parameters.
2. Execute, which allows the LLM to execute the relevant endpoints with the right parameters.

Each tool will take { code: string } as input.

A nominal workflow will look like this:

1. The LLM will search the OpenAPI spec for relevant endpoints and parameters using the Search tool. The tool will return a list of relevant endpoints and parameters, which will be added to the context. The generated code will look like this:

async () => {
  const results = []
  for (const [path, methods] of Object.entries(spec.paths)) {
    for (const [method, op] of Object.entries(methods)) {
      if (op.tags?.some(tag => tag.toLowerCase() === '<tag>')) { // In a real-world scenario, the LLM will replace <tag> with the relevant tag to search for in the OpenAPI spec.
        results.push({ method: method.toUpperCase(), path, summary: op.summary })
      }
    }
  }
  return results
}

It's just a function that will be run against the OpenAPI spec to find the relevant endpoints and parameters.

2. The LLM will generate code to execute the relevant endpoints with the right parameters using the Execute tool. The tool will return the result of the execution, which will be added to the context. The generated code will look like this:

async () => api.request({
  method: 'GET',
  path: '/api/v1/resources', // In a real-world scenario, the LLM will replace this with the relevant endpoint to execute.
  query: { max: 10 },
})

Once again, it's just a function that will be run within the MCP server to execute the relevant endpoint with the right parameters. That's it.

Under the hood, the MCP server will execute the LLM-generated code and return the result to the LLM. Of course, you have to run the code within a secure sandbox to avoid security issues. Do not do what I do here: the Function constructor is not a secure sandbox, and executing LLM-generated code with it can lead to major security issues, including arbitrary code execution and data exfiltration. I currently use it only to simplify development, but when I start sharing the MCP server, I'll use isolate-vm to run the code in a secure sandbox. A prompt injection could ask the LLM to exfiltrate data from the MCP server, so you have to be careful about that. I know that Cloudflare dynamic workers allow you to block outbound network requests.

With these two tools and all the information provided by the OpenAPI spec, such as the path, the description, the parameter name, and any other relevant human-readable information, the LLM can infer how to interact with your API, and sometimes infer parts of your business logic, without the pain of writing a custom tool for each endpoint. But an OpenAPI spec does not always describe business logic completely, and its descriptions can be incomplete or ambiguous. Combine this with an access to the business documentation, and your LLM is the perfect agent to interact with your API.

That's the power of Code Mode.

Cloudflare's MCP Showcase ​

Until now, everything was theoretical. So, let's have a look at how Cloudflare's MCP server behaves in practice:

Now, you may be wondering: "That was a great read, but how can I build my own MCP server to solve my context problem?"

Be patient. I'll have something for you in the coming weeks!