PROJECT: AI Engineering Code Summit
DOC. NO: RM-2025-HZ
AI ENGINEERING
CODE SUMMIT
REPORT
A select summit of leading coding agent builders and superusers: top AI engineers, engineering leaders (CTOs/VPs), and researchers advancing code generation and AI software engineering.
Abstract
Two days of sessions from experts at the frontier of real world AI engineering revealed a clear architecture: the entire software development life cycle is going through a period of massive transformation, capability beats scaffolding, cleanliness beats cleverness, and human judgment remains the irreducible core.
State of the Field
The AI engineering landscape is transitioning from "pilot purgatory" to production reality at various stages of maturity. Bottom up power-users are redefining what it means to be a software developer while the perennial issues of software engineering - complexity and an "accumulations of whatevers" - have not magically disappeared. The software development lifecycle is fully experiencing AI-future shock.
How much are we actually accomplishing? How much "productivity" is in fact mere reworking the slop that got churned out yesterday? The answer seems to be based on the sophistication of your verification process, and the deeper and more significant your validation process is the closer you get to this tantalizing idea of intelligent agents.
Context management was a recurring theme: text files still reign supreme (now and forever), explicit planning or research phases, either manual coaching or more of a built in framework such as Kiro Specs or Antigravity's Artifacts, and subagents as a general pattern for managing context. And they don't need to be some elaborate memory thing.
The models are so smart now that a lot of the specific tweaks of yester-month to get the harness tuned seems unnecessary, with the Claude Code crew talking about getting "bash-pilled" and Terminus being the one tool needed for these agents to do anything.
Maybe a terminal and a filesystem is all you need.
Leadership
AI rewards excellence and punishes mediocrity. Trust grows from infrastructure, not promises.
01 Trust, Role Changes, and What's Good for Humans is Good for AI
What's good for humans is good for AI - same infrastructure investments benefit both
02 Context, Memory, and The Harness: Where Product Differentiation Really Happens
76% of developers don't trust AI code because they don't trust the context
03 SDLC Evolution: Good Software Engineering Practices, Amplified
AI doesn't eliminate need for good process - it makes good process more valuable
04 The ROI Reality Check: Where AI Actually Delivers (And Where It Falls Flat)
93% of organizations stuck in 'pilot purgatory' - only 7% believe they're at scale
05 Proactive vs Reactive: The UX Evolution
Goal is reducing mental load, not just adding speed
06 The Economics of AI Engineering: New Cost Structures & Compensation Models
AI changes the cost function of engineering - previous cost-per-feature calculations obsolete
Engineering
The war on "slop" requires human thinking, environments as abstraction, and open data.
01 DO NOT OUTSOURCE THE THINKING: Context Engineering & Human-AI Collaboration
AI can only amplify the thinking you've already done
02 Environments as Universal Abstraction: The New Unit of Everything
A benchmark = environment + starting state + verifier (same as RL environments)
03 Reinforcement Learning for Specialized Models: The Economics of Domain Expertise
1000 examples can yield 10-point improvements with ARFT
03 Skills and Artifacts: The New Building Blocks for Agents
Skills are just folders - procedural knowledge packaged for agents to load dynamically
04 Model Quality Over Scaffolding: Minimalism in Agent Architecture
Use smart models to plan, fast models to execute - deploy the right capability at the right time
05 Data Collection & Quality as the New Bottleneck
Agents are collecting good data but not sharing it - keeping datasets closed slows research