Wednesday, April 26, 2006

Software Success Factor #1: Manage for Agility UPDATED

The ability of an organization to Manage for Agility is one of the most important success factors for modern software development. This factor encompasses the critical role that Management--or the project Client or Sponsor--plays in Iterative Development. In this post, I will attempt to address what this factor entails and how it is accomplished. I will also briefly recap why this management style is superior. I have yet to complete the Overview of the Reference Model, but Agile Management is so important that I wanted to flesh out this success factor first.

Iterative development is perhaps the key attribute of modern software success, but without management support it is impossible. To manage for agility, leaders must go against many traditional management styles and ingrained practices. It is a natural for successful business people to try to avoid future change through detailed planning and up-front analysis, but changes are inevitable. Trusting the team and the process to produce the best solution, allowing the scope, the design, and even the process to emerge, is what it means to Manage for Agility.

To manage agile projects, sponsors must resist the desire to plan and design the entire project before starting any development. To be agile, management must instead embrace a flexible model that responds to changing requirements and an evolving understanding of the solution. Management must trust the process and the team to produce top quality software, without a relying on a rigid scope.

Often it is the "commercial Model," contracts, SOWS, etc., that most influences the interaction between management and the project team. To manage for agility, the commercial model must provide for evolving scope and design. Put another way, attempting to fix the scope in agreements up-front is simply at odds with an agile style. When using consultants or contractors, a Time-And-Materials approach allows for the most agility, or the agreement can employ separate SOWs created at the start of each iteration. Another approach offered by Kent Beck is the use of "Optional Scope Contracts."
It is also important to understand that agile does not mean that no planning or requirements are done up front. The project still needs a first cut at the inventory of requirements--e.g. a Scrum backlog or a Use-case survey--that is the source from which iteration requirements are selected, refined, and built. An agile project can also develop a detailed, "pro-forma" project plan, but managers should not feel compelled to stick to the original as the team adjusts to accumulating change.

Why should managers buy into this approach, when traditional project management techniques have worked on all kinds of projects, from construction to aerospace engineering? Because software development is fundamentally different from construction or engineering, and treating it like these disciplines magnifies the risk of failure, raises costs, and produces worse software.
Change is unavoidable in Software Development, more so than most any field. Alan Shalloway made the claim, at a recent Agile Denver meeting, that software development is not like building a bridge; it is more like building the first bridge... or the design phase of a bridge project. An analogy is also often drawn between construction and software development. However, many experts claim that software development is essentially all design work, and the only step that is really like construction is the compile/build cycle.
The highly successful Scrum process is based on proven new-product development processes used by Japanese companies. Indeed, the Scrum proponents and many others have claimed, accurately I believe, that software development is very like new product development. If we look at software development as roughly analogous to the design and development of a new kind of car, or a new consumer electronics gadget, it becomes clear that the nature of the process is unpredictable and should be managed accordingly. Agile management uses an empirical process management approach to produce a better product. One need look no further for evidence than the difference between American cars and Japanese, at least until very recently.
Alistair Cockburn expands on the idea of "responding to change over following a plan:"

...traditional process management—by continuous measurement, error identification, and process refinements—strove to drive variations out of processes. This approach assumes that variations are the result of errors. Today, while process problems certainly cause some errors, external environmental changes cause critical variations. Because we cannot eliminate these changes, driving down the cost of responding to them is the only viable strategy. Rather than eliminating rework, the new strategy is to reduce its cost.

[NOTE: I think he means code rework--Big Up-Front planning and design will result in more rework of other work products, assuming the effort is not wasted entirely as these products are abandoned.]

Business needs, priorities, understanding of user needs, expertise with the
technical platform, market forces, and available technologies... All of these
can and usually do change over the life of a project. Agile management rolls
with these changes and capitalizes on them to make the best software possible.
Smart project sponsors can use agile management to build better software for
less money.