IBM Developer Questions 80% AI Coding Claims as Verification and Learning Costs Mount
Updated
Updated · O'Reilly Media · Oct 29
IBM Developer Questions 80% AI Coding Claims as Verification and Learning Costs Mount
3 articles · Updated · O'Reilly Media · Oct 29
Summary
Task Viewer—an internal IBM helper with no production role—became a real-world test of how much coding an AI agent could handle before verification work outweighed the apparent speedup.
Manual checks still dominated: the developer reviewed behavior and design, added UI tests, and ran SQL queries against source records because correct-looking numbers on screen did not prove the calculations were right.
That made headline claims like an agent getting a project “80% of the way there” hard to interpret, since generated code says little about the time needed to audit assumptions, resolve data questions, and reach a result a team will trust.
DORA’s analysis of 1,110 Google engineer responses and a 52-developer study on AI-assisted learning both support the tradeoff: AI can boost output, but some saved coding time shifts to verification, and passive use can weaken understanding.
The proposed rule is task-specific delegation: give agents more autonomy when results are straightforward to verify, but keep key reasoning or first tests with developers when checks are uncertain or the work is meant to build system knowledge.