AI Software Development – What Does The Data Say?
Scored daily by a customisable AI persona to surface the most relevant engineering leadership news.
Data-driven analysis of AI in software development directly matches reader's interest in AI/ML trends and SDLC impact.
Recent studies and experiments indicate that fully autonomous, reliable long-horizon agentic software development using LLMs is effectively science fiction. Effective context limits are orders of magnitude smaller than advertised, with vendor “compression” introducing lossy summarization. LLMs cannot distinguish recency from training priors, struggle with negation, and perform better with demonstrations than descriptions. Industry data shows increased output (more code, commits) but worse outcomes (longer shipping times, lower quality), with AI amplifying existing team strengths rather than fixing weaknesses. Psychological research links high confidence in LLM output to belief in the paranormal and finds negative impacts on learning and critical thinking.