Marlin Jayasekera

I work on evaluation and reliability for AI agents. The hard part was never getting an agent to do something impressive once. It's knowing whether it actually succeeded, on work where a wrong answer has consequences and nobody is watching every step. That gap, between a demo that wows a room and a system you can hand a customer, is where I spend my time: building the evaluation data and verification tooling that closes it. It's in stealth, and the work is used by frontier AI labs.

Before this I built products from zero as a CTO and cofounder. Real-time analytics and attribution at Because, an embedded financing product at RenoRun that grew to about a tenth of revenue, and Storage Fellows, an on-demand storage company I built end to end. Software engineering at Concordia.

Writing

I Audited a Year of My Own Agent Sessions 2026-08

Every coding-agent session I run is archived. I scanned all 5,284 of them for actions the agent shouldn't have taken. Real overreach is rare, and the agent almost always reports it before I notice. The rule it breaks constantly is the one its own system prompt tells it to break.

Give the World Model a Truth Channel 2026-07

I trained and probed small world models on games that expose their exact internal state every frame. No hand-written verifiers, no LLM judges. What fell out: a working failure detector for JEPA latents, a decided version of the latent-vs-generative argument, and a playable dream whose bugs taught me more than its successes.

World Models Melt 2026-07

Generated video looks right frame by frame and quietly falls apart in 3D. The usual metrics miss it. Here's how I measured geometric consistency with feed-forward reconstruction, and what the numbers showed.

Why Evals Are The Moat 2026-07

The defensible layer in applied AI isn't the model or the app. It's the measurement.

The CUA Reliability Gap 2026-06

Computer-use agents can do the demo. They can't yet do the job. The gap is verification.

Data Is A Pipeline, Not A Number 2026-05

Counting tasks is a vanity metric. The pipeline that produces them is the real thing.

Building

A 6th-gen iPod Classic rebuilt with 1TB, USB-C, and Bluetooth. Plus caravan (one command provisions a whole dev workspace, for people and agent fleets), cmux-companion, and agent-telemetry. More in the lab.