pstack: the harness that makes AI coding agents prove their work

pstack is a free, open-source add-on for Cursor, a code editor, that wraps an AI coding agent — software that writes code on instruction — in written procedures: a router that sorts a task into one of twenty-three playbooks, twenty-four principle files, and skills that force the agent to check its own work. Those layers do change how the agent behaves, and controlled tests show modest gains in finishing tasks, but nothing measured shows the code comes out safer to ship. The measurement that would settle it — a defect rate for a pstack-configured agent against a plain one — has not been made.

The plugin installs from inside the editor, and the marketplace listed 47 skill entries and two specialist subagents when the project stood at version 0.15.1 under the MIT open-source licence on 10 September 2026 (The Neuron — pstack explained: Lauren Tan’s…). At its centre is a router called /poteto-mode, which sorts a task into one of twenty-three playbooks and copies that playbook's steps into a to-do list (Cursor — pstack (README)). Around it sit twenty-four principle files.

What matters is what the agent is made to do at the end of a task. Setup offers to build a project-specific verification skill that the agent must run and pass before it reports success (Cursor — pstack guide: Set up pstack). Cursor's setup guide says the tool spends extra tokens on review panels, and calls that the price of the rigour. An agent that can check its own work keeps going until the check passes, it argues, while one that cannot hands every result back for a person to check by hand.

Controlled evidence covers the general mechanism, not this plugin. A preprint on arXiv scraped 679 rule files holding 25,532 rules and ran more than 5,000 benchmark runs, with the main comparison on 58 tasks (arXiv — Guardrails Beat Guidance: A Large-Scale…). All seven rule conditions beat the no-rule baseline of 50.0 per cent, by 6.9 to 13.8 percentage points.

SWE-bench Verified pass rate by rule condition, 2026 (arXiv preprint)

The gains did not favour curation. Randomly chosen rules and carefully curated ones scored identically, at 63.8 per cent, and no condition differed significantly (arXiv — Guardrails Beat Guidance: A Large-Scale…). Testing rules one at a time sorted the library into three that shaped behaviour — all prohibitions — four that distorted it, all positive instructions, and eleven that did nothing measurable. Removing one prohibition, "do not refactor unrelated code", cut the pass rate by 20 percentage points.

The gains were not free either. Fourteen of eighteen rules broke at least two of seventeen previously solved tasks, and the eighteen-rule set held 65.7 per cent against no-rule baselines of 68.6 to 74.3 per cent (arXiv — Guardrails Beat Guidance: A Large-Scale…). pstack's bulk is the shape that scored no better than random: twenty-four principle files telling the agent what good work looks like (Cursor — pstack (README)). Its /interrogate skill, where different agents try to break a proposed change, is the prohibition shape that did work.

The only published hands-on account ran about twice as long as a plain session — an hour against thirty minutes — and its checks caught three claims the agent had invented; one reviewer preferring the result measures trust, not defects (The Neuron — pstack explained: Lauren Tan’s…). Adoption numbers are volume. Cursor engineering had run Tan's personal skills 10,000 times in a single week before the release. She has said the tool supported 2,462 pull requests in production in August 2026.

None of that says whether the shipped code was safer. Veracode's 2026 report puts the average security pass rate across more than 100 AI models tested over four years at 56 per cent, and finds it has not moved (Veracode — 2026 GenAI Code Security Report). A Cloud Security Alliance note cites Veracode's March 2026 update at about 55 per cent, unchanged from 2025 (Cloud Security Alliance — Vibe Coding’s…). Six of the eleven new models tested in 2026 sat between 50 and 53 per cent. That report tests model output, not an agent configured with this plugin.

At company scale the record is worse. Citing Apiiro's data for Fortune 50 firms from December 2024 to June 2025, the Cloud Security Alliance reports monthly security findings rising from about 1,000 to more than 10,000 (Cloud Security Alliance — Vibe Coding’s…). Rule files add attack surface before they add safety: the same note lists 2025 Cursor vulnerabilities and hidden-Unicode injection into rule files of the kind pstack writes.

The part worth keeping is the harness rather than the shelf. A verification skill turns a passing build into a test the agent must satisfy, and the biggest single effect in the controlled work came from a prohibition rather than an instruction (arXiv — Guardrails Beat Guidance: A Large-Scale…). What would change the answer is a defect or vulnerability rate for agent-configured runs against a matched plain-agent baseline, plus a follow-rate showing agents obey instruction files at all; no source reports either.

Nothing here is financial, legal or investment advice. Every figure is published by its named source and carries the date it was published; figures may since have changed.