Version 1 was a promise. Version 2 is running right now, on two machines, while I write this.
I’ve told this story twice before. A desktop campaign that pointed AI agents at nine open-source libraries and asked a narrow question: does the loop work? And then an early OT/ICS run that asked a harder one: does the same loop survive contact with protocol stacks that sit between a network and a physical process? Both answers were yes. This note is about the system that grew out of those answers — what it actually is now, what it gets right, and where it still bites me.
What version 1 was, and why it stayed a promise
Version 1 ran on a single desktop workstation in February 2026, under an orchestrator I built and called openclaw. openclaw was the node that coordinated a set of research agents — the “Rocket” assistants — across a six-week campaign on nine libraries. It produced real receipts: tens of billions of executions, hundreds of crashes triaged, and three findings — the headline one a confirmed remote code-execution bug in libarchive, a library that ships on virtually every operating system. The write-up is still up under agentic vuln research.
But it stayed a promise. One box, one architecture, one orchestrator carrying the whole pipeline in its head. When it worked it was impressive; when it drifted, I was the only backstop. It proved the idea and not much past it. I retired the openclaw node in August 2026 and started over with a different question: not “can agents find bugs,” but “what does this look like when it has to run every hour, on more than one machine, without me babysitting it?”
The thesis hasn’t changed
Fuzzing is maybe 20% of the work. Anybody can launch AFL++. The other 80% — picking surfaces, writing and smoke-testing harnesses, watching coverage plateaus, rotating campaigns, reproducing crashes, de-duplicating piles of near-identical faults — used to be me, at a terminal, at 2am. In version 2, most of that 80% is agents. I’m the gate. That’s the whole bet: the orchestration is the product, and a human still decides what’s real.
The fuzzer is the cheap part. The orchestration around it is the thing worth building — and the human at the end of it is not optional.
What version 2 is
An orchestrator plus a handful of specialist agents, each with one job. A harness agent surfaces a target, builds the harness, and smoke-tests it. A fleet-watch agent calls coverage plateaus and retirements. A triage agent reproduces crashes, classifies them, and de-dupes by stack signature. A status agent owns the public counters. The orchestrator plans and dispatches, and it only surfaces me the exceptions and the blockers — a healthy hour is a silent hour.
Two hosts, on purpose: an ARM64 Mac Mini and an x86_64 Linux server with 24 cores. Crashes don’t always cross architectures — endianness, alignment, ABI — so if you only fuzz one arch you’re half-testing. Each host runs its own natively-built harness; binaries are never copied between them. An hourly rotation tick is the heartbeat: it polls the live worker count, and when a campaign retires it launches the next prepared target so a host never sits idle. The queue is always stocked. That’s the never-idle rule, and it’s the single biggest difference from v1.
The loop
Every target walks the same path. It’s acquired into an inbox with a manifest and a checksum. The surfaces are ranked — file and in-memory parsers over live sockets. A harness gets built: an ASan build, a faster plain build, a dictionary where a magic value hides behind a length check. Then the gate: the harness has to print SMOKE_OK or the fleet never starts. A failed smoke gets fixed once or the target is blocked — no silent half-built campaigns.
On a clean smoke the fleet launches detached with a six-hour budget. The hourly tick watches it. New unique crashes auto-kick triage. A spent budget or a confirmed coverage plateau retires the campaign and rotates the host straight into the next prepared target. Triage reproduces with the same binary, de-dupes by signature, and writes a finding doc. Findings feed two places: a self-contained handoff package for internal validation, and the aggregate counters on the public page.
Honest tiers, and a correction
This is the part I care most about getting right, because it’s the part that’s easiest to inflate. A finding earns its way up a ladder, and the public page only ever shows the rung it’s actually standing on. First it’s a candidate. Then it’s reproduced in isolated reruns — same crash, clean container, no harness luck. Above that are the ones with outstanding PoC investigation — reachable through a library API, with the path to a running service still being worked. The top rung is confirmed via network: the lab’s best guess that a finding is reachable through a running network service. Best guess, not peer-reviewed proof — it’s a judgment call made from the crash and the code path, and I label it that way on purpose. This isn’t published research; the point is the orchestration, not any one outcome. CVSS is scored straight, with the reachability caveat written into the score line itself. No rounding up.
An earlier note said findings here are “disclosed to maintainers first.” This post makes no such claim in either direction: what has and hasn’t been reported is handled separately, by me, by hand, and it sits outside this pipeline on purpose. The counts here describe what the lab produced and assessed — not what has been reported to anyone. That same note framed the strongest tier as proven “live service-path”; here that tier is labeled honestly as the lab’s best-guess assessment — confirmed-via-network is a judgment call, not peer-reviewed proof.
That’s the whole discipline: surface candidates honestly, triage hard, label the assessment plainly, and let the next stage take it from there. The point of all this is the orchestration, not any single finding.
Receipts, not headlines
Lab only. Open-source targets. Authorized. These numbers describe what the lab produced and assessed — not what has been reported to anyone. The public page publishes totals and coverage, never crash inputs, stacks, or per-library attribution. As of October 26, 2026 (CT):
The raw crash count is noise until something survives triage, so the number I actually watch is the ladder: 24 candidates, 24 reproduced in isolated reruns, 20 with outstanding PoC investigation, and 4 confirmed via network — the best-guess strongest tier, assessed not proven. The current lead is rated High — CVSS 8.2 — and sits in the outstanding-PoC tier, cross-arch not yet re-confirmed. These counts will never be exactly right: disclosure and downstream validation happen outside the lab, by hand, so the public tally always trails the real state of any given finding. That’s fine. The orchestration is the story, not the scoreboard.
The coverage spans what you’d expect for OT and then some: EtherNet/IP and CIP (OpENer), IEC 61850 and IEC 60870, Modbus stacks, BACnet, DNP3, OPC-UA (open62541), PROFINET — plus a deliberate pivot into lighter-fuzzed embedded stacks like tinycbor, wakaama, lwIP, nanopb, QCBOR, and MQTT-SN. A handful of libraries produce almost all the crashes; most go quiet after billions of executions. Quiet isn’t a clean bill of health — it’s either solid code or a harness that isn’t hitting the interesting paths yet, and telling those apart is exactly the judgment call the loop exists to tee up.
What still breaks
A system that claims perfection invites distrust. Here’s what version 2 got wrong this month, told short — each one earns its own follow-up.
- A runaway log took a host down. A stuck worker wrote a single log until it filled the boot disk and killed the campaign. The crash evidence survived and I rotated to a fresh run, but there was no log cap and no disk guard. There should have been.
- Host disconnects stall cross-arch checks. When one host drops, I lose half the capacity and every cross-architecture confirmation that depends on it — including the lead candidate’s. Several findings sit arch-deferred with staged repro notes, waiting.
- Source-of-truth drift. The rotation state and the published summary are partly hand-maintained, and hand-maintained things drift. The public page has shown a host as “running” after it went offline, and the tier cards have carried a count I couldn’t trace back to a finding doc. Honest numbers need a single source that generates them.
- Noisy harnesses inflate raw counts. A few harnesses are unstable enough that their nondeterminism eats trust in the crash numbers until triage separates real bugs from artifacts.
None of that is a surprise. It’s the known failure surface of a system that runs every hour on real hardware, and naming it is how it gets fixed.
So
The fleet’s running. Two boxes, two architectures, an agent per job, a tick every hour, and me at the gate. The crash count isn’t the story and it never was — the orchestration is the point. Next I fix the list above. Then we start sketching what version 3 looks like.
Drew Stelly is a security researcher working on AI security, vulnerability research, and digital forensics.
Responsible disclosure: this is authorized, lab-only research against open-source targets. Disclosure is handled separately and manually, outside this pipeline; these numbers describe what the lab produced and assessed, not what has been reported to anyone. The dashboard publishes aggregate totals and coverage only — never crash inputs, stack traces, or per-library, per-vendor, or per-product attribution.