Agile is Not Even Wrong
It's another centralized system for a decentralized world
Software serving the warfighter must be unbelievably responsive to evolving user needs. But when time is limited, how do organizations know which ideas to prioritize? Agile — like Scrum and Kanban and Waterfall before it — tries to answer this question but falls short.
Agile Solves the Wrong Problem
The Government Accountability Office (GAO) recently encouraged the Department of Defense to “follow Agile development principles, which deliver working software in less than a year and add capability after that based on user feedback.” Even the time frame - less than a year - tells us Agile is the wrong thing to meet the needs of the warfighter. Agile is a system modeled for organizations with one customer whose requests are centrally prioritized. Agile is not modeled for daily responsiveness to 58 Brigade Combat Teams (BCT) in the Army or 13 Air Operations Centers in the Air Force. When the user base is this diverse and distributed, we’ve merely shifted the bottleneck from Waterfall’s centralized delivery of software to Agile’s centralized prioritization of feature requests.
And in fact, there is no process or project management certification that can solve for this. Agile and Waterfall are both Soviet-style central planning processes, but we need to be embracing an entrepreneurial, American way of shipping code. The complexity of not just the user base but also the dozens of software vendors on the hook to deliver features can only be solved with a software infrastructure that is designed to massively debottleneck hyper distributed development.
We can’t just apply process to a problem that needs to be managed (at least in part) by technology. We still need project managers, but we really need the engineers — empowered by software — who work directly with users in the field to ship the most important things before they ever enter a centralized queue. The idea of content (the software + engineers) over process (Agile) is the thing that matters. Steve Jobs articulates how “people get very confused that the process is the content,” as he reflects on why Apple’s Lisa failed. He admits they hired great process people from Hewlett Packard, but it didn’t matter because “Apple did not have the caliber of people necessary.” The added benefit of content over process is it makes the defense tech ecosystem much more competitive. Companies delivering capability to the warfighter will be rewarded not for the perfection of their process but for the quality of their code in delivering outcomes.
Power Law vs Normal Distribution
The warfighter lives in a power law world, but Agile is optimized for normal distribution.
Many variables follow a normal distribution, like height, athletic ability, and software engineering talent. Agile incorrectly treats user needs like a normally distributed variable: feature requests and requirements enter a prioritization queue that biases to the average need. By managing to the mean, Agile (or any centralized management system) is designed to protect downside and avoid failure.
While some things are appropriately prioritized centrally, like a large backend effort, the most creative efforts can’t be managed this way without destroying all the value. A power law distribution roughly follows the 80/20 rule, where 80% of the outcomes result from 20% of the causes. A limited number of user needs and features will drive the majority of results.
Discerning the correct place to invest is hard and often not knowable ex-ante. It depends both on the distribution of creative commanders who are ready to wield software as a weapon system and on the real-world events catalyzing invention and innovation. The ideas and concepts from, say, a combatant command that define the next paradigm when translated to software would only be weaker if averaged against the requests from all the combatant commands.
In the case of JADC2, imagine a never-ending queue of minor improvements and adjustments that are centrally ranked and scheduled for implementation. At the same time, there are users with varying levels of connection to modern warfare who have immediate and diverse use cases and needs. By relying on a centralized prioritization system like Agile, software development will focus on the most substantial changes that cater to a homogeneous set of collective user needs, instead of addressing the most urgent, specific requirements for current day operations. Moreover, priorities might change as new commanders join the field and contribute their insights.
In adopting a dynamic, decentralized approach to software development, the warfighter is better served by enabling rapid engineering and integration of daily alterations, which would otherwise be lost in the bureaucratic process of change management cycles, no matter how agile. This way, innovation is de-bottlenecked, and the system is more responsive to the specific and evolving needs of users in the field, ensuring the most compelling concepts can scale across the enterprise regardless of how they emerge.