The Spec Was Never the Goal

GitHub Spec Kit turned one on August 21 and shipped 1.0.0 the same day. Is there still a place for it? Yes, a narrow one, and it gets narrower with every model release. Most of what Spec Kit does is tell the model how to work. That’s thousands of words of procedure loaded before your own spec gets written, and frontier models stopped needing it. What survives is a statement of intent that a person reads before the code exists, plus a test that tells the machine when it’s done. Spec Kit is a heavy way to get either one.

The new System One models, the first of which shipped last week, push the same direction from the other side. They make a single judgment cheap enough to ask for at the moment it’s needed, so there’s less reason to settle everything in prose up front. The human job left over is the goal and the definition of done. The specifics were never ours to dictate.

Spec Kit does two jobs, and only one of them aged well

The first job is instructing the model. Running /speckit.specify, /speckit.plan, /speckit.tasks and /speckit.implement loads a prompt for each step. Together they tell the agent how to ask questions, plan, break work into tasks and check its output against requirements. I counted: on main today those four prompts total about 7,000 words, and their templates add about 2,500 more. The optional steps (clarify, analyze, checklist, converge) add another 9,200.

That procedure was written for agents that didn’t plan, didn’t ask and lost the thread halfway through a feature. The agent I use every day has a plan mode, a todo list, a way to ask me questions, and subagents. Spec Kit teaches it things it already does, in more words than most of my blog posts.

The second job is making intent durable. That means a spec.md someone reviews before code gets written, a “constitution” of standing project rules, and artifacts that outlive the session and survive a switch from one agent to another. Better models don’t retire that job, because a model can’t recover a decision nobody wrote down.

The best evidence that the maintainers see the same split is their own lean preset, added in April. It cuts each command prompt to between 58 and 193 words and drops the templates. That removes about 95% of the instruction and keeps every artifact. The default went the other way. The four core prompts came to about 1,400 words in October 2025 and about 4,000 by late November. Today’s count is five times the October one. The critics said it was too heavy, and the default got heavier.

The procedure doesn’t buy quality on a strong model

The closest thing to a controlled test of the pipeline itself is a paper from April called Spec Kit Agents. The authors built their own pipeline on Spec Kit’s specify, plan, tasks, implement sequence. They compared it against going straight to implementation across 32 features in five repositories. Direct implementation scored 3.46 on a five-point quality scale. The spec pipeline scored 3.51 and took 24 minutes instead of 14.4. The two setups ran under different time limits and an LLM did the grading, so read that +0.05 as “no measurable difference.” The gain that did reach significance, +0.15, came from hooks that fed each stage evidence from the actual repository. Context moved the score. Procedure didn’t.

An August paper by Arquimedes Canedo came at the model-tier question from another direction. He gave six models the same architecture in five formats, ranging from informal prose to TypeScript interface contracts. On Sonnet 4.6 and GPT-5, format barely mattered: quality varied by 0.17 to 0.92 points on a ten-point scale depending on format. On weaker models it varied by 0.83 to 2.42 points. TypeScript contracts took the weakest model’s API route coverage from 33% to 100%. His title is “Architecture as Capability Equalizer,” and the equalizing runs one way. Structure helps weaker models most, and the structure that helps is shaped like code, not prose. The caveats are three trials per cell, no significance tests, and no baseline without a spec.

Thoughtworks said it in one sentence in its November 2025 Technology Radar, writing about spec-driven development in general: “We may be relearning a bitter lesson — that handcrafting detailed rules for AI ultimately doesn’t scale.” The bitter lesson is Rich Sutton’s 2019 essay, which opens: “The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin.” Spec Kit’s procedure is handcrafted knowledge, and the model it was written for keeps getting replaced.

Instruction bloat is the failure mode, not a side effect

The April Radar placed “agent instruction bloat” in its Caution ring. Instruction files grow as teams add rules, the rules start to conflict, and “as instructions grow, the likelihood increases that important rules are ignored.” The same edition’s entry on Spec Kit reports teams hitting exactly that, with “a growing agent instruction set and eventually context rot.” One team dug out by moving the guidance into skills that load only when they’re needed.

The generated documents pile up too. Birgitta Böckeler tried Spec Kit for martinfowler.com in October 2025 and found its files “repetitive” and “very verbose and tedious to review.” Colin Eberhardt at Scott Logic built two features with it the next month. He got 4,839 lines of markdown for 989 lines of code, and 5.5 hours of review. The same scope, done iteratively, took 24 minutes. François Zaninotto at Marmelab cited a developer who used Spec Kit to add the current date to a time-tracking app and got eight files and 1,300 lines of text. A March 2026 hands-on review at azanello.com measured 3.5 hours against 23 minutes.

Borges wrote a one-paragraph story about an empire whose cartographers drew a map the size of the empire. It was so exact that it was useless, and later generations left it to rot in the desert. A spec five times longer than the code it describes is that map. Nobody reads it, so nobody checks it, so it drifts. Spec Kit’s own documentation is candid about the drift: how specs get maintained after the first implementation is “a team convention, not a CLI setting.”

I have my own entry in this genre. Last December I let a Spec Kit-style workflow generate 40- and 50-step site-migration plans over a weekend. They fell apart around step 15. I spent more than seven hours in what Dex Horthy calls the dumb zone, correcting a plan that kept correcting itself. The fix then was smaller context and shorter plans, and it still is.

System One models move judgment to the moment it’s needed

TypeSafe released Jev on September 15. It isn’t a chat model, and it doesn’t generate text. You give it unstructured state and a typed question, and it returns a typed answer with calibrated probabilities. The answer is one of three types: a Choice from a list you define, a Score against levels you define, or a Noul, the probability that a statement is true. TypeSafe calls this class of model System One, after Daniel Kahneman’s Thinking, Fast and Slow. System 1 is fast and automatic; System 2 is slow and deliberate. An LLM agent is the System 2.

TypeSafe’s launch post quotes 70 to 500 milliseconds per call and $0.042 per million input tokens, with output free. Its headline of “193.6x faster, 444.6x cheaper” comes from the company’s own workflow evals. To its credit, the post says it expects “that these are on the higher end of real world gains.” These are vendor numbers, and there’s no independent benchmark yet.

The architecture is what matters for spec-driven development. LangChain’s post on building a harness with Jev, published two days after launch, describes the split: “use an LLM for open-ended reasoning and generation, and Jev for fast, structured decisions along the way.” Its examples are routing a request to a cheap or an expensive model, and classifying a tool call’s risk before the agent runs it. TypeSafe’s docs make the case against bloat directly: “Each question is evaluated independently, so adding more questions does not create context-rot.” They also advise: “Ask each factor as a separate question, then combine the results with logic in your code.”

Put that next to Spec Kit. A spec front-loads decisions into prose because deciding at runtime used to be slow, expensive and unreliable, so a spec-driven workflow is a bet that you can anticipate the decisions. A System One gate removes the need for that bet across a whole class of decisions. Is this change risky? Which model should take it? Does it touch the payments code? Is the task done? You ask when the question comes up, get a typed answer in under a second, and write the threshold in code, where it can be tested. The planning goes to the System 2 model, which is good at planning now. What’s left for the human is the goal.

I’ve had Jev in a triage step for about a week, and the first thing it showed me was where the spec went. My first option list was stale: it was missing most of the real options and still included one that no longer existed. Jev picked from that list without complaint, because that’s its job. The spec didn’t disappear. It shrank into a type, and a wrong type is a wrong spec with a confidence score attached.

A goal, plus a test that proves it

Stop telling the model how. Tell it what done looks like, in a form a machine can check.

The strongest evidence I found for any spec-like artifact is for tests a person wrote. TDFlow, published at EACL 2026, treats repository repair as the job of making human-written tests pass. With human-written tests, it resolved 94.3% of SWE-bench Verified issues at $1.01 per issue. When it had to write its own tests, it resolved 68.0% at $4.12. The authors conclude that the remaining obstacle is “writing successful reproduction tests,” not passing them. A person’s test is a better spec than the agent’s and cheaper to satisfy, and it’s the goal in executable form.

Spec Kit doesn’t put the test at the center. Its methodology document calls test-first an “imperative,” but the shipped tasks template says “Tests are OPTIONAL - only include them if explicitly requested in the feature specification.” The artifact with the best evidence behind it is opt-in, and the procedure isn’t. I’m not innocent here either. I run a plugin that enforces red-green-refactor on every change, and I found no study showing that the ritual helps a frontier model. The evidence supports the test, not the ceremony around it.

Tests have their own failure mode. SpecBench, from Weco AI, gave agents a spec, a set of visible tests, and a held-out set that combines the same features. Every frontier agent passed its visible suite, but held-out scores trailed, and one agent built a 2,900-line “compiler” that memorized the test inputs. The gap was larger for smaller models and grew with code size. So “goals over specifics” comes with a condition: the check has to belong to someone other than the agent being checked. That’s the argument from Verification Is the Bottleneck, arrived at from the other end.

Where Spec Kit still earns its keep

The strongest case for Spec Kit isn’t about the model. It’s about the people.

Some changes need sign-off from someone who won’t read the diff: a product owner, a compliance reviewer, another team whose API you’re about to touch. For those, a reviewed spec.md is the cheapest place to catch “that’s not what we meant.” Spec Kit’s clarify step asks up to five targeted questions and writes the answers back into the spec. That forces decisions that a goal-only prompt lets the model make silently. Thoughtworks’ April entry says its teams use Spec Kit mostly on existing systems, where “much rework stems from unclear intent, hidden assumptions and late discovery of constraints.” A stronger model doesn’t fix that problem. It only hides it faster. The same Radar singles out a lighter option, OpenSpec, for its “focus on spec deltas rather than defining a complete specification upfront,” which it calls “well-suited for existing systems.”

The model-tier case is real too, but narrower than it looks. If you hand implementation to a cheaper model, Canedo’s data says structure pays off. It also says the structure that pays is shaped like code: types, schemas, interface contracts. That’s an argument for a TypeScript interface, not for a markdown plan.

The 1.x releases also fixed some of what critics raised. Bugs get their own assess, fix and test path, so a bug fix no longer runs the full feature pipeline. A converge step checks the implementation against the artifacts and adds any missing tasks. A new guide for existing codebases advises against writing specs for the whole system first, which is sensible. And the lean preset exists. But every critical hands-on measurement I found predates the lean preset, and the one September review I found, from Wavect, reads as a desk review. It’s possible that 1.x is lighter in practice than its defaults suggest. Nobody has published numbers either way.

As for System One models, they’re new. The calibration claims haven’t been independently checked, and TypeSafe’s own docs describe the decisions they handle well as gut-check-sized. None of that writes your goal for you.

The rule I’m using

Write a spec document only when a specific person will read it before the code exists, or a cheaper model will build from it. Otherwise:

• State the goal and the constraints in a few sentences.
• Write the test that proves the goal, or have the model draft it and review the draft yourself before any code gets written.
• Let the frontier model plan. It’s better at that than a template is.
• Put fast, typed checks at the decision points: routing, risk, whether the task is done.
• Keep the tests out of the implementing agent’s reach.

If your team adopts Spec Kit anyway, start with the lean preset, add the bug extension, and pin a version. Decide how specs will be maintained before the first feature ships. Pinning matters more than usual here. The 1.0 release notes quote the lead maintainer calling 1.0 “just a number,” and ten releases have followed in the month since. It’s the first time I’ve seen a version number used as a disclaimer.

Spec-driven development was right about the problem: agents fail when intent is vague. It was wrong about the medium. Prose is how people agree with each other, and it’s a poor way to tell a machine what done means. The specs getting stronger are executable: a failing test, a TypeScript interface, the list of options a System One model is allowed to choose from. Keep the prose for the people who have to agree, and give the machine something it can check. The how was never ours to write.

---

Related Reading

• Context Engineering: How I Learned to Stop Worrying and Manage the Smart Zone: the December Spec Kit-style failure in full, and the research, plan, implement loop that replaced it.
• Verification Is the Bottleneck: why the cost of checking output, not model quality, decides where AI pays off.
• Outsource the Task, Never the Thinking: what can be delegated and what can’t, argued at length.
• Choosing the Right AI Coding Tool: the tools this post assumes you already have.

Resources

• GitHub, Spec Kit repository, v1.0.0 release notes, lean preset, tasks template, methodology (spec-driven.md), spec persistence models, agentic SDD command reference (vendor). Prompt word counts are my own, from templates/ at v0.0.55, v0.0.86 and main.
• Manfred Riem, “Spec Kit Turns One — and Ships 1.0.0”, August 21, 2026 (lead maintainer).
• Pardis Taghavi and Santosh Bhavani, “Spec Kit Agents: Context-Grounded Agentic Workflows”, arXiv, April 2026.
• Arquimedes Canedo, “Architecture as Capability Equalizer for Coding Agents”, arXiv, August 2026.
• Thoughtworks, Technology Radar, Vol. 33 (November 2025) and Vol. 34 (April 2026) (consultancy), read through the radar dataset.
• Rich Sutton, “The Bitter Lesson”, March 2019.
• Birgitta Böckeler, “Understanding Spec-Driven-Development: Kiro, spec-kit, and Tessl”, October 2025.
• Colin Eberhardt, “Putting Spec Kit Through Its Paces: Radical Idea or Reinvented Waterfall?”, Scott Logic, November 2025.
• François Zaninotto, “Spec-Driven Development: The Waterfall Strikes Back”, Marmelab, November 2025.
• azanello, “GitHub Spec Kit: A Hands-On Review for 2026”, March 2026.
• Wavect, “GitHub Spec Kit Review for Production Teams”, September 2026 (consultancy).
• TypeSafe AI, “Introducing System One Models & Jev”, September 15, 2026, and documentation (vendor).
• Sydney Runkle and Hunter Lovell, “Building a Harness with Jev”, LangChain, September 17, 2026 (ships the integration).
• Daniel Kahneman, Thinking, Fast and Slow, 2011.
• Jorge Luis Borges, “On Exactitude in Science,” 1946.
• Kevin Han et al., “TDFlow: Agentic Workflows for Test Driven Development”, EACL 2026.
• Bingchen Zhao et al., “SpecBench: Measuring Reward Hacking in Long-Horizon Coding Agents”, Weco AI, 2026 (sells an agent product).