Skip to main content
Sign In
Use Cases

Agent Code Tool

Give an AI agent a code-execution tool that runs model-generated code in an isolated VM and returns structured results.

Letting a model write code instead of chaining one tool call per step costs fewer tokens and handles real data processing. The catch is that you must run code nobody reviewed. Secure Exec makes that a function call.

A run_code tool

The tool handler type-checks the model’s code, then evaluates it in a fresh VM with no network and a hard timeout. Either way, the model gets back something it can act on.

interface RunCodeResult {
	ok: boolean;
	value?: unknown;
	/** Type errors or a runtime failure for the model to fix and retry. */
	feedback?: string;
}

// The handler for a `run_code` tool. The source comes from a model, so it is
// never trusted: it runs in a fresh VM with no network and a hard timeout.
async function runCode(
	source: string,
	inputs: Record<string, number[]>,
): Promise<RunCodeResult> {
	// Type-check first. Diagnostics are cheap feedback that saves a run.
	const checked = await check(
		`declare const inputs: { prices: number[] };\n${source}`,
	);
	if (checked.outcome !== "succeeded" || checked.hasErrors) {
		const feedback = checked.diagnostics
			.map((diagnostic) => `TS${diagnostic.code}: ${diagnostic.message}`)
			.join("\n");
		return { ok: false, feedback };
	}

	const result = await evaluate(source, {
		inputs,
		timeoutMs: 5_000,
		output: { capture: "stderr" },
	});
	if (result.outcome !== "succeeded") {
		return { ok: false, feedback: result.stderr ?? result.error.message };
	}
	return { ok: true, value: result.value };
}
  • Type-check first. Diagnostics come back in milliseconds and never run the code. Most generated-code bugs stop here.
  • Pass data with inputs. Never interpolate values into source.
  • Return failures as feedback. Guest errors are results, not exceptions, so the handler stays a straight line.

The retry loop

// The model's first attempt misnames a field, so it gets the diagnostics back
// without the code ever running.
const first = await runCode(
	`inputs.price.reduce((sum, price) => sum + price, 0)`,
	{ prices: [5, 10, 27] },
);
console.log(first); // { ok: false, feedback: "TS2551: Property 'price' does not exist ..." }

// The corrected attempt runs.
const second = await runCode(
	`inputs.prices.reduce((sum, price) => sum + price, 0)`,
	{ prices: [5, 10, 27] },
);
console.log(second); // { ok: true, value: 42 }

Give the code your tools

Expose your own functions with host functions so generated code can call your APIs without ever holding a credential.

Multi-step agents

When the agent builds on its earlier steps, create one VM per conversation. Files and installed packages carry over between calls. Add a context if variables should carry over too. Dispose the VM when the conversation ends.

Locking it down

The defaults are already safe for generated code: no network, a virtual filesystem, and bounded resources. Add a timeoutMs to every call, and grant only what a task needs with permissions.

Edit this page