# Benchmarks

Secure Exec runs guest code on V8 with a JIT compiler, in a separate process.
QuickJS-based libraries run an interpreter compiled to WebAssembly, inside your
process. That trade shows up as two different numbers: how long a call takes
before any guest code runs, and how fast the guest code then goes.

## Results

Median of five runs, in milliseconds, end to end from the host.

| | No work | 50,000 records | Numeric loops | 1 MB of text |
|---|---:|---:|---:|---:|
| Secure Exec, one-shot call | 40.7 | 47.7 | 83.9 | 85.1 |
| Secure Exec, existing [VM](/secure-exec/docs/vms) | 13.7 | 23.6 | 61.4 | 69.9 |
| quickjs-emscripten | 0.4 | 43.5 | 481.3 | 177.0 |
| Vercel Run | 5.1 | 60.9 | 588.6 | 324.1 |

Subtracting the "no work" column leaves the time spent computing:

| | 50,000 records | Numeric loops | 1 MB of text |
|---|---:|---:|---:|
| Secure Exec | 9.9 | 47.7 | 56.2 |
| quickjs-emscripten | 43.1 (4.4x) | 480.9 (10.1x) | 176.6 (3.1x) |
| Vercel Run | 55.8 (5.6x) | 583.5 (12.2x) | 319.0 (5.7x) |

## What it means

- **Secure Exec has a fixed cost per call.** A one-shot call creates and disposes
  a VM, which costs about 40 ms. A call on an existing VM costs about 14 ms. The
  QuickJS libraries start in under 6 ms, because there is no process boundary to
  cross.
- **V8 computes 3 to 12 times faster.** The gap is widest on tight numeric loops
  and narrowest on string work that already runs in native code.
- **The crossover is small.** Once guest code does a few tens of milliseconds of
  real work, Secure Exec finishes first, even counting its fixed cost. A snippet
  that only calls a host function and returns will finish first on QuickJS.

## Workloads

Each workload builds its own data inside the guest, so no engine pays to move
data across its boundary.

secure-exec/examples/benchmark/src/index.ts:

```ts
const workloads: Record<string, string> = {
	// Fixed cost of one call: nothing to compute.
	baseline: "return 1;",
	// Build, filter, group, sort, and serialize 50,000 records.
	records: `
		const rows = [];
		for (let i = 0; i < 50000; i++) rows.push({ id: i, group: i % 97, price: (i * 31) % 1000, name: "item-" + i });
		const byGroup = new Map();
		for (const row of rows.filter((r) => r.price > 250)) {
			const entry = byGroup.get(row.group) ?? { count: 0, total: 0 };
			entry.count++; entry.total += row.price; byGroup.set(row.group, entry);
		}
		const top = [...byGroup.entries()].map(([group, e]) => ({ group, avg: e.total / e.count })).sort((a, b) => b.avg - a.avg).slice(0, 5);
		return JSON.parse(JSON.stringify(top)).length;`,
	// A prime sieve and a tight arithmetic loop.
	numeric: `
		let count = 0;
		const limit = 300000;
		const sieve = new Uint8Array(limit + 1);
		for (let i = 2; i <= limit; i++) { if (!sieve[i]) { count++; for (let j = i * 2; j <= limit; j += i) sieve[j] = 1; } }
		let acc = 0;
		for (let i = 0; i < 5000000; i++) acc = (acc + i * i) % 1000003;
		return count + acc;`,
	// Build about 1 MB of text, split it, count words, and run a regex replace.
	text: `
		let text = "";
		for (let i = 0; i < 20000; i++) text += "the quick brown fox " + i + " jumps over the lazy dog\\n";
		const words = text.split(/\\s+/).filter(Boolean);
		const freq = {};
		for (const w of words) freq[w] = (freq[w] ?? 0) + 1;
		return Object.keys(freq).length + text.replace(/o/g, "0").length;`,
};
```

secure-exec/examples/benchmark/src/index.ts:

```ts
const engines: Record<string, (body: string) => Promise<unknown>> = {
	"secure-exec, one-shot (fresh VM per call)": async (body) => {
		const result = await evaluate(expression(body));
		return result.outcome === "succeeded" ? result.value : result.outcome;
	},
	"secure-exec, existing VM": async (body) => {
		const result = await vm.javascript.evaluate(expression(body));
		return result.outcome === "succeeded" ? result.value : result.outcome;
	},
	"quickjs-emscripten (fresh context per call)": async (body) => {
		const context = QuickJS.newContext();
		try {
			const handle = context.unwrapResult(context.evalCode(expression(body)));
			const value = context.dump(handle);
			handle.dispose();
			return value;
		} finally {
			context.dispose();
		}
	},
	"vercel run": async (body) => {
		const result = await run({ source: body });
		return result.status === "completed" ? result.value : result.status;
	},
};
```

## Method

- Measured on 2026-09-18 on a 20-core x64 Linux machine with Node.js 22.23.
- Versions: the `secure-exec` 0.2.20 release sidecar, `quickjs-emscripten`
  0.32.0, and `run` 2.1.4.
- Every engine gets one warm-up call. quickjs-emscripten creates a fresh context
  per call, and Run creates one by design, so every row pays for a clean
  environment.
- These are small, single-threaded workloads on one machine. Run the benchmark
  on your own hardware before relying on the ratios. Inside this repository it
  uses the locally built sidecar, which is a slower debug build unless you set
  `AGENTOS_SIDECAR_BIN` to a release binary.