Agile vs Waterfall. It’s one of those classic line-ups, like Caesar vs Napoleon. We can’t resist pitting two seemingly opposite contenders against one another.
But the truth is more complex and subtle. Though in principle, Agile vs Waterfall represents two apparent extremes of a spectrum, the reality is less clear. Adaptive and Predictive Project Management (to use more helpful terms) are at two ends of a spectrum. But each already incorporates many aspects of the other. And no one would sensibly say you need to stick to one extreme or another. Far less would they be wise to assert that one is ‘better’ than the other.
Both Agile and Waterfall (Predictive) PM are powerful paradigms. And a well-crafted hybrid of the two can create an optimum approach for your project. So, in this article, we’ll compare Agile vs Waterfall and also look at the subtlety underneath. My intention is to leave you better-informed and with a keener understanding of both.
So, here’s what we’ll be covering:
We cannot get into the Agile vs Waterfall debate without first understanding what each one is. So, in this section, I’ll run through the highlight. If you want to learn more about each, I suggest you take a look at our one-page learning road-map and resource guides:
Two of the problems that we have, derive from the nature of how the terminology ‘waterfall’ is in use. Firstly, many Agilists – practitioners of Agile methods – refer to anything that is not ‘Agile’ as ‘waterfall‘. And second, many of them use it as a pejorative term. So we get this false and destructive syllogism:
Waterfall = not Agile
Agile = Good
Waterfall = Bad
When Agilists do use alternative terminology for predictive projects, they are most likely to replace the term waterfall with ‘command and control’. And, while traditional project managers do crave control, I thik you’ll find few effective PMs who achieve it through command. Rather, we aspire to careful planning, effective monitoring, and inspirational leadership.
A classic (genuine) waterfall process is illustrated below.
However, one of the essential features of any predictive project management is a phased approach. We work in stages.
There are two other features I would say are absolutely characteristic of predictive project management (‘waterfall’).
There are a lot of ‘myths and misinformation’ about predictive project management, put about by some evangelical Agile practitioners (Agilists). They say Waterfall projects…
The term ‘waterfall’ is widely-used – and often, in a pejorative way – as a description of any form of Project Management that isn’t Agile. However, it was originally coined (in 1976) to refer to a specific software development framework that Winston Royce described in 1970. The principles of Waterfall’s sequential phases derived from methods used in an engineering context. Of course, in this context of physical engineering, often predictive methods will be best. You can read more about it in the Wikipedia Article.
You might ask: ‘what’s wrong with misusing ‘Waterfall?’ My concern is that is makes it all too easy to fall into the trap that predictive project management has no value. Or that a weak or absent process is also a characteristic of predictive project management. Neither is true. Predictive Project Management emphasizes strong process for good reasons. And, sometimes, we need the assurance that careful prior planning can give us.
Agile is less of a project management framework than a set of values and principles. There are, however, a number of frameworks and methodologies that do, to a greater or lesser degree, conform to these values and principles. Perhaps the best way to introduce Agile Project Management is with a short video:
The main feature of Agile is the incremental production, delivery, testing, and acceptance of deliverables. This happens around short iterations, usually 1-4 weeks, depending on the Agile method you choose. The Scrum methodology calls these iterations ‘sprints’.
In Agile, we normally define the next set of the users’ needs or wants (‘user stories’) at the start of each iteration, rather than a full set at the start of the project in a single Definition or requirements-gathering stage. Later, we can refine our understanding of the user stories during a sprint, to create formal requirements.
However, in Agile we often do define the high-level feature set at the start. We then break this down into discrete items that we can fully define and develop within the iterations. A key aim of Agile is to retain as much flexibility as we can, throughout the development cycle. And this means that we sometimes sart with nothing more precise than a brad vision.
The Agile Manifesto came forward in 2001, and was the work of 17 authors. In it, they state that they value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items onManifesto for Agile Software Development
the right, we value the items on the left more.
In support of the Agile Manifesto, the authors also set out 12 guiding principles for an Agile project, which underpin the Agile Manifesto. To summarise them:
Note that the authors referred to software projects explicitly, and did not intend their principles to other forms of project. That said, many of them are principles that all ‘traditional’ Project managers can commit to quite happily.
Another concern I have is that Agilists tend to think that, with Agile, came incremental and iterative development. Yet, as a Project Manager rooted in predictive project management, I can tell you, with assurance, that I was delivering incremental products and iterating development work long before the Agile Manifesto. And, on top of that, we also had:
Probably the best-known Agile methodology is Scrum. Its origin, however, is not in Project Management, but in Product Management. In January 1986, Harvard Business Review (HBR) published ‘The New New Product Development Game‘ by Hirotaka Takeuchi and Ikujiro Nonaka. This was about a new way to do New Product
Do read the story of Nonaka and Takeuchi, and Scrum’s origin. The upshot is that development of physical products can certainly benefit from an Agile approach, mindset, and tools.
The main difference between Agile and Waterfall is largely one of degree.
Please Note: For each of the seemingly polar opposites I offer in this table, the truth is far more nuanced. While some Platonic ideal of ‘pure’ waterfall, or ‘pure’ Agile, may conform to the descriptions here, in the real world, we see projects occupy regions of a spectrum, rather than the very endpoints.
|Traditional methods…||Agile Methods…|
|Prioritize fixing scope||Prioritise |
|Plans an integrated project before starting delivery||Develops large parts of the project during delivery|
|Starts incremental delivery after planning||Starts incremental delivery at the first iteration (and so speeds delivery of benefits and value)|
|Creates control structures for the Project Manager||Grants greater autonomy to project team members|
|Engages and consults stakeholder throughout||Explicitly puts stakeholders first|
|Controls change through structured process||Have change at the core of the approach|
|Controls schedule and cost through approved plans||Schedule and cost grow to meet the emerging scope|
|Scope is pre-committed so it is hard to terminate to manage cost and schedule over-runs||Scope is under constant review and only incremented where the value case is strong enough|
Firstly, Agile won’t be right for every circumstance, as we’ll explore soon. But secondly, while it may be right for many organizations, there is often a fear of feeling less in control. This can block the adoption of agile – as of course can fear of the unknown.
However, even if your organization does choose to make the change, shifting to Agile is a big investment… in time, effort, and money.
Both waterfall and agile project management have professional skillsets that require study, practice, and development over time. The reality is that a ‘traditional’ Project Manager cannot just read a book and then lead an Agile Project.
The principles of Agile are straightforward. The skill comes in practicing them effectively. This needs a lot of training and experience. And many organizations, project managers, and project teams fail to fully appreciate this or don’t want to put the investment or effort into it. So, they attempt to do Agile projet management by rote – following a simple guide-book approach. However, if you do not fully understand the principles and the reasons for the practices, poorly led and delivered Agile projects are typically not very successful.
My friend, the Agile expert, Chuck Cobb, points out that, in a typical Agile environment, at the team level there is no one called a ‘Project Manager’. Of course, different Agile methodologies and implementations mean that some will have a PM.
Instead, the functions that might normally be performed by a project manager are distributed among the other roles on the team. That puts a very big new load on the people on the team to actually perform some project management functions in addition to their normal role. That’s a big reason why the learning can be difficult.
Here’s an article Chuck wrote on that:
A fruitful way to compare Agile vs Waterfall is through the lens of the four constraints of Time, Cost, Quality, and Scope. Of the four, Scope carries the burden of the difference, because, in Agile, scope is the big variable. Time and cost vary to accommodate it – throughout the project.
With Predictive Project Management, we often fix the scope and then choose or time, cost, or quality priorities. Then, we optimize for what is left.
Let’s look at the four in turn. But, please do remember that I have presented as big a difference as I can to emphasize the contrasts. In reality, of course, the differences will usually be more subtle, and the overlap greater.
Predictive Project Management seeks to fix a schedule based either on:
Agile, on the other hand, offers schedule certainty per iteration, because it uses fixed time
Predictive project management makes it easier to measure and assess progress. However, Agile approaches allow users to see and test working iterations at an early stage, and then frequently through the project. This can add to the sense of ownership they feel for the newly emerging products.
The iterative nature of Agile development can easily lead to frequent revisions to the scope of the work, beyond the original definition and design. Where this happens a lot, there is a risk in significant extensions to the time scale that the client may envisage. This also has consequences for expectations of cost…
Cost mirrors schedule. In predictive projects, we plan and budget the full project at the outset. In agile projects, we make incremental decisions. This gives us good cost control for sprint-by-sprint spending, but
Without a doubt, some aspects of Agile project management do lead to a lower cost base:
Agilists also claim that Agile Teams are more productive, but I do doubt that they can support this by solid evidence. Productivity depends on many things and for every highly productive agile team, I’d expect to be able to show you an equally productive team working in a more traditional way… And also an Agile team that is wasteful and inefficient!
We can set and define quality equally well for both. But where quality depends on integrating many components, the certainty of what you
Within Agile projects, quality is built into each iteration, rather than being a sequential activity which can be lost or minimized where time pressures come to bear. There is also a sense, in Agile projects, that team members believe to a greater extent that quality is not ‘someone else’s responsibility’. In predictive projects, it does often sit with people responsible for Quality Design, Assurance, and Control.
An upshot of this, which Agilists can reasonably claim, is that customer satisfaction is particularly high after Agile Projects. Maybe that’s because of higher levels of customer involvement at all stages, though. However, there is nothing in the predictive approach that prevents extensive user and customer engagement.
In Agile, the scope grows iteratively. I would say this is the biggest difference between the two. A big advantage of Agile approaches is the incremental definition of scope. Thee is no need to identify early every part of what you will need. And it also allows for more easy adaptation to new demands, as well as re-prioritization. Finally, Agile methods make it easy to stop adding to scope when new increments would not deliver sufficient value to users.
Another benefit of Agile is the ability to rapidly bring a Minimum Viable Product to market or into beneficial use. This can have huge commercial advantages in crystallizing savings or getting ahead of competitors. And the lessons learned from each new iteration can feed into customer- and user-centric decisions about the focus of the next cycle (iteration) of work.
A challenge that Agile presents stems from this high level of user and customer involvement: what if they have neither the time nor inclination to contribute? Good agile development is highly dependent on close collaboration with users and customers.
Binary choices are foolish. Neither Agile nor Predictive Project Management is best. Each is right, within a range of contexts. And, in reality, the choice is not a simple ‘either-or’.
Agile is largely preferred by Agile-trained delivery team-members. But their clients, who need to plan integration with operational procedures to a deadline want more certainty. And when those sponsors need to set and keep to a budget, the Agile disinclination to estimate costs can be very frustrating.
However, when done well, and when used in the right context, Agile delivery can:
On the other hand, Agile methods do require an additional level of training. To embed it within an organization, will take a whole cohort of trained personnel but more than that too. It’s a significant change in how to do things, and so needs a substantial transformation effort. The downsides to this are real disruption, so if it’s not enough that you need to get the change process right; timing will be a crucial factor. And, the reality is that using agile approaches for large scale projects is still in the development phase. You would need to sign-up for early adopter status, with all of the risks (and potential rewards) that this would entail.
But some of the key indicators that will help us assess what is most appropriate are:
Let’s start with the Big Two. Predictive project management will likely be the best starting place when you:
We have a corresponding Big Two indicators for starting with an Adaptive approach to your project:
Fusion cuisine is not a matter of sticking curry on top of a pizza and expecting it to taste good. Likewise, creating a working hybrid of Agile and Predictive principles and tools is a task for a highly experienced practitioner. And by one who has a deep understanding of both traditions.
In the UK, we have the term ‘cut and shut’. This is where the remains of two or more damaged cars have been welded together to create a ‘new’ vehicle. For obvious safety reasons, this is illegal. Likewise, a cut and shut project methodology that welds together a Frankenstein’s monster of tools and techniques will not work. A subtle blend is what you need.
An example might be an outer framework of structured planning and, within that, an adaptive methodology for progressing product development. Chuck Cobb, whose Agile training we recommend, illustrates his Managed Agile Development Framework in a short article.
How you craft your hybrid of Agile and Predictive processes, mindsets, and tools must follow the needs of your project. We’ve seen in the previous section what characteristics suggest each approach. And earlier, we saw what each approach can offer. Questions to ask include:
This creates a spectrum of needs and outcomes, which you can match up with a carefully crafted hybrid. I am not qualified to advise on how to do that, but the figure below illustrates how you can wrap an Agile process within a predictive framework, at different levels.
The Project management Institute (PMI) is working on the 7th Edition of its Project Management Body of Knowledge (PMBOK Guide). I expect this to have a deep focus on selecting Agile vs Waterfall and how to create effective hybrid projects.
But, at the time of writing, we have the 6th edition of the PMBOK Guide and its accompanying Agile Practice Guide (APG). I shan’t dip into the Agile Practice Guide here save to note that Chapter 3, Life Cycle Selection, gives a lightweight introduction to hybrid solutions. To learn more about the guide, take a look at our article, ‘The PMI’s Agile Practice Guide: What You Need to Know‘.
What I will do is take a look at Appendix X3 of the PMBOK Guide: ‘Agile, Iterative, Adaptive, and Hybrid Project Environments’. It’s shorter than Chapter 3 of the APG, but for my money, it’s clearer and more helpful.
But what it also does is interpret the five PMBOK Guide Process Groups in terms of adaptive and hybrid projects.
KPMG in the Netherlands has produced a valuable survey report, ‘Agile Project Delivery’ (2017). An interesting chart on page 8 shows the uptake of Agile and hybrid approaches in different disciplines. The results are very much as you may expect if you have read this far (rather than skimmed down).
The domains where certainty is most at a premium and innovation least so, see lowest take up, and highest reliance on traditional Project Management:
At the other end, we see two disciplines that value creativity and incremental improvement, alongside IT, within which Agile emerged.
Operations sits at around 50% take up, and HR at around 35%.
I hope that in reading this, you have developed a keener appreciation for the subtleties of the, somewhat bogus, Agile vs Waterfall dilemma. You should not see them as right or wrong, nor as a binary choice. Rather, it is your job, as a project professional, to master techniques at both ends of the spectrum and to develop effective approaches to combining them to meet the demands of your project, within its context.
Please do leave your comments below, and I’ll respond to all contributions.
As you’d expect, this is a topic that a lot of people write about – often with great passion. I’ve tried to present a broadly objective overview, with a few opinions clearly identified. Other articles are more trenchant, but nonetheless worth
Dr Mike Clayton is one of the most successful and in-demand project management trainers in the UK. He is author of 13 best-selling books, including four about project management. He is also a prolific blogger and contributor to ProjectManager.com and Project, the journal of the Association for Project Management. Between 1990 and 2002, Mike was a successful project manager, leading large project teams and delivering complex projects. In 2016, Mike launched OnlinePMCourses.
Please log in again. The login page will open in a new tab. After logging in you can close it and return to this page.