Specifications are doing more than stating requirements
Specifications used in spec-driven development are often doing more than stating what software should do. In the judged repository sample, 77.6% also described how the software was built, while 58.2% devoted at least a section to implementation detail. Among specifications containing such detail, 75% named low-level elements such as classes, functions, paths, data structures or library versions, and 71% described components, interfaces or data models. The study's central question was how this practice differs from what SDD tools and their makers prescribe.
That pattern blurs the line between a requirement and an implementation plan. The figures are estimates from a judged sample within a filtered repository set, not a count of every specification in use.
Users do not apply the method to every task
Among interviewed tool users who described their decision rules, task size or complexity was the main guide, with large features more likely to receive the workflow. Estimated purpose labels in the repository sample were dominated by feature work at 82.0%, followed by refactoring at 24.3%, fixes at 16.5% and performance work at 11.6%. The labels could overlap, so they are not a single breakdown of all documents.
The review combined several kinds of evidence
The authors used concurrent triangulation, combining practitioner discussions, interviews and repository evidence. Their Reddit archive contained 405,163 posts and 3,710,688 comments. A selection process based on keyword matches and relative community engagement, with limits on selections per subreddit, retained 1,497 records. The interview phase included 21 practitioners, eight tool makers and 13 tool users.
Repository mining produced a broad corpus of 470,795 specification files across 73,030 repositories and 44,521 owners, plus a separate Kiro census of 98,574 artifacts across 12,910 repositories. The detailed filtered analysis covered 266 repositories and 36,890 files, including 13 Kiro repositories with 654 requirements documents. Those figures describe different parts of the dataset rather than one combined sample.
Reported benefits were paired with extra work
Some participants said agent-assisted specification drafting clarified their intent. Others reported more reliable agent work, but the study could not separate that report from concurrent improvements in the models. Cross-role collaboration appeared mainly in a few organizations where specifications moved between developers and non-developer roles.
The trade-off was visible in one Reddit comparison: spec-driven runs used two to three times as many tokens and took about twice as long as direct prompting. That was a single self-reported comparison, not a study-wide estimate.
Specifications can drift from the code
Participants described two ways the written plan and the software can separate. An agent may depart from the specification during implementation, or the specification may become stale as the code changes. The study identified these as qualitative patterns and did not directly measure behavioral decay over time.
Repository records also showed concentrated participation. Of 36,888 analyzed specification files, 98.3% had a single author; 68.5% of 1,731 inline specification comments were written by bots; and 63% of the 266 repositories had only one person touch a specification. These counts show who was recorded in the history, not how closely people read or discussed the documents.
A separate pull-request layer contained 5,992 spec-touching pull requests from 581 repositories. In 81.2% of them, code was modified in the same reviewed changeset. The paper treats that co-change rate as a heuristic rather than ground truth.
The evidence is a map of practice, not a causal test
Because the study is observational and qualitative, its comparisons cannot show that SDD caused the reported benefits, costs or drift. The authors describe their interpretations as associations across participants and repositories, and say the data do not establish direction.
Taken together, the results portray SDD as a practice in which users weigh reported clarity or reliability against planning time and the possibility that specifications and code will diverge. The findings describe the repositories and participants studied, rather than a causal effect across software work.
Paper data and sources
Original title: SPECMINE: A Large-Scale Corpus of Spec-Driven Development Artifacts
Authors: Shyam Agarwal, Bogdan Vasilescu
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-25
DOI: Not available
Original paper · Full text