Agile vs Waterfall: Which one is Right for Your Project?

Agile vs Waterfall. It’s one of those classic line-ups, like Caesar vs Napoleon. We can’t resist pitting two seemingly opposite contenders against one another.

But the truth is more complex and subtle. Though in principle, Agile vs Waterfall represents two apparent extremes of a spectrum, the reality is less clear. Adaptive and Predictive Project Management (to use more helpful terms) are at two ends of a spectrum. But each already incorporates many aspects of the other. And no one would sensibly say you need to stick to one extreme or another. Far less would they be wise to assert that one is ‘better’ than the other.

The Purpose of this Article

Both Agile and Waterfall (Predictive) PM are powerful paradigms. And a well-crafted hybrid of the two can create an optimum approach for your project. So, in this article, we’ll compare Agile vs Waterfall and also look at the subtlety underneath. My intention is to leave you better-informed and with a keener understanding of both.

So, here’s what we’ll be covering:

• What are Agile and Waterfall?
• The Principal Differences
• When to use each One
• Agile-Waterfall Hybrids
• The Impact of Domain
• Concluding Remarks

What are Agile and Waterfall?

We cannot get into the Agile vs Waterfall debate without first understanding what each one is. So, in this section, I’ll run through the highlight. If you want to learn more about each, I suggest you take a look at our one-page learning road-map and resource guides:

• I Want to Learn Project Management
• I Want to Study Agile Project Management

Waterfall: Better called Predictive or Planned Project Management

Two of the problems that we have, derive from the nature of how the terminology ‘waterfall’ is in use. Firstly, many Agilists – practitioners of Agile methods – refer to anything that is not ‘Agile’ as ‘waterfall‘. And second, many of them use it as a pejorative term. So we get this false and destructive syllogism:

Waterfall = not Agile

Agile = Good

Waterfall = Bad

When Agilists do use alternative terminology for predictive projects, they are most likely to replace the term waterfall with ‘command and control’. And, while traditional project managers do crave control, I thik you’ll find few effective PMs who achieve it through command. Rather, we aspire to careful planning, effective monitoring, and inspirational leadership.

So, what is Predictive Project Management?

A classic (genuine) waterfall process is illustrated below.

However, one of the essential features of any predictive project management is a phased approach. We work in stages.

There are two other features I would say are absolutely characteristic of predictive project management (‘waterfall’).

1. Starts with Definition
   The first is that we aim to start with a clear definition of what our project is and is not, and what the functional or capability requirements are. We need to clearly articulate our Goal, Objectives, and Scope of Requirements.
   [The Links go to short videos that explain the terms – they open new tabs so you won’t use your place.]
2. Careful Planning and Budgeting
   The second is the preparation of a comprehensive plan and budget, against a clear specification.

Not True about Waterfall

There are a lot of ‘myths and misinformation’ about predictive project management, put about by some evangelical Agile practitioners (Agilists). They say Waterfall projects…

• …have low engagement with customers and stakeholders
  In fact, the amount of stakeholder engagement is a matter for the Project Manager and your team to determine. And I am a big advocate of making it a top priority, because ‘it is your stakeholders who will determine the success (or not) of your project’.

• …take an ‘All at once, big bang approach to delivery’
  In fact, Project Managers have used phased, incremental delivery for as long as projects have been around. It de-risks the project and returns value to the client at an earlier date. Incrementalism is nothing new.
• …allow for no refinement of or change to the scope or specifications
  In fact, we have sophisticated change control processes that allow just that. And they do so in a disciplined and accountable way.

The Word: ‘Waterfall’

The term ‘waterfall’ is widely-used – and often, in a pejorative way – as a description of any form of Project Management that isn’t Agile. However, it was originally coined (in 1976) to refer to a specific software development framework that Winston Royce described in 1970. The principles of Waterfall’s sequential phases derived from methods used in an engineering context. Of course, in this context of physical engineering, often predictive methods will be best. You can read more about it in the Wikipedia Article.

You might ask: ‘what’s wrong with misusing ‘Waterfall?’ My concern is that is makes it all too easy to fall into the trap that predictive project management has no value. Or that a weak or absent process is also a characteristic of predictive project management. Neither is true. Predictive Project Management emphasizes strong process for good reasons. And, sometimes, we need the assurance that careful prior planning can give us.

Agile: Better called Adaptive Project Management

Agile is less of a project management framework than a set of values and principles. There are, however, a number of frameworks and methodologies that do, to a greater or lesser degree, conform to these values and principles. Perhaps the best way to introduce Agile Project Management is with a short video:

The main feature of Agile is the incremental production, delivery, testing, and acceptance of deliverables. This happens around short iterations, usually 1-4 weeks, depending on the Agile method you choose. The Scrum methodology calls these iterations ‘sprints’.

In Agile, we normally define the next set of the users’ needs or wants (‘user stories’) at the start of each iteration, rather than a full set at the start of the project in a single Definition or requirements-gathering stage. Later, we can refine our understanding of the user stories during a sprint, to create formal requirements.

However, in Agile we often do define the high-level feature set at the start. We then break this down into discrete items that we can fully define and develop within the iterations. A key aim of Agile is to retain as much flexibility as we can, throughout the development cycle. And this means that we sometimes sart with nothing more precise than a brad vision.

A Summary of the Agile Manifesto

The Agile Manifesto came forward in 2001, and was the work of 17 authors. In it, they state that they value:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

Manifesto for Agile Software Development

What do the Four Values of the Agile Manifesto Mean?

1. Individuals and interactions over processes and tools
   Agile places a lot of emphasis on self-managing teams and strong communication. This is not alien to predictive project management, but Agile has catalyzed the creation of new tools and methods.
2. Working software over comprehensive documentation
   Here the focus is on products/deliverables that users can start to use early on, rather than on documenting a full set of requirements. Agilists often use terms like:
   1. ‘Minimum Viable Product’ (MVP) – a product with the smallest set of features and functionality to perform a valuable function. It is used as a basis for early feedback on how to improve the MVP.
   2. ‘Shippable Product’ – products you can handover into beneficial use
3. Customer collaboration over contract negotiation
   I’d like to think any Project Manager will engage with their clients fully. For me, the big difference between Agile and Predictive PM here, is in the extent of collaboration.
4. Responding to change over following a plan
   Again, the differences here are a matter of degree. Traditional Project methodologies have Change Control processes. Agile projects are built around a constant cycle of reviewing and changing the scope and specifications of the next iteration of the end product. The key determinant of how to manage change control is the relationship with your user/customer.

12 Guiding Principles of Agile

In support of the Agile Manifesto, the authors also set out 12 guiding principles for an Agile project, which underpin the Agile Manifesto. To summarise them:

1. Top priority is customer satisfaction through early and continuous delivery of valuable software.
2. Changing requirements, even late in the project, are inevitable and a good thing.
3. Deliver working software often and on a short timescale (weeks to months max).
4. Business people and developers must work together daily throughout the project.
5. Build projects around people who are motivated to do the work. Give them the environment, resources, and support they need. Then trust them to get the job done.
6. The best way to convey information is through face-to-face conversation.
7. The primary measure of progress is working software.
8. Agile processes are sustainable and can continue until they deliver everything the client and users specify.
9. Technical excellence and good design enhance agility.
10. The way to maximize the amount of work not done is through simplification.
11. The best work is done by self-organizing teams.
12. The team reflects on how to work better regular intervals. They should then fine-tune and adjust their behavior accordingly.

Agile: What was Really New?

Note that the authors referred to software projects explicitly, and did not intend their principles to other forms of project. That said, many of them are principles that all ‘traditional’ Project managers can commit to quite happily.

Another concern I have is that Agilists tend to think that, with Agile, came incremental and iterative development. Yet, as a Project Manager rooted in predictive project management, I can tell you, with assurance, that I was delivering incremental products and iterating development work long before the Agile Manifesto. And, on top of that, we also had:

• Careful consultation with stakeholders, to prioritize delivery of what they most valued
• Effective change control processes that allowed changes throughout development
• Co-located teams discussing current issues face-to-face, as soon as they emerged
• Multi-skilled teams from across functions, with decision-makers all in the same room
• Daily team meetings – seven standing up, I recall on one project
• Regular delivery and testing cycles
• Frequent lessons learned discussions, and changes to practices accordingly

Can Agile Methods be Used for Physical Engineering Projects?

Probably the best-known Agile methodology is Scrum. Its origin, however, is not in Project Management, but in Product Management. In January 1986, Harvard Business Review (HBR) published ‘The New New Product Development Game‘ by Hirotaka Takeuchi and Ikujiro Nonaka. This was about a new way to do New Product Development, or NPD. They drew on the idea of ‘ba’ – a Japanese coinage of Nonaka’s, meaning a meeting place for minds and the energy that draws out knowledge and creates new ideas. And it used a sporting metaphor from the game of Rugby at its heart: Scrum.

Do read the story of Nonaka and Takeuchi, and Scrum’s origin. The upshot is that development of physical products can certainly benefit from an Agile approach, mindset, and tools.

Agile vs Waterfall: the Principal Differences

The main difference between Agile and Waterfall is largely one of degree.

Please Note: For each of the seemingly polar opposites I offer in this table, the truth is far more nuanced. While some Platonic ideal of ‘pure’ waterfall, or ‘pure’ Agile, may conform to the descriptions here, in the real world, we see projects occupy regions of a spectrum, rather than the very endpoints.

Agile is Sounding Good, Why is it not Universal

Firstly, Agile won’t be right for every circumstance, as we’ll explore soon. But secondly, while it may be right for many organizations, there is often a fear of feeling less in control. This can block the adoption of agile – as of course can fear of the unknown.

However, even if your organization does choose to make the change, shifting to Agile is a big investment… in time, effort, and money.

The Learning Overhead

Both waterfall and agile project management have professional skillsets that require study, practice, and development over time. The reality is that a ‘traditional’ Project Manager cannot just read a book and then lead an Agile Project.

The principles of Agile are straightforward. The skill comes in practicing them effectively. This needs a lot of training and experience. And many organizations, project managers, and project teams fail to fully appreciate this or don’t want to put the investment or effort into it. So, they attempt to do Agile projet management by rote – following a simple guide-book approach. However, if you do not fully understand the principles and the reasons for the practices, poorly led and delivered Agile projects are typically not very successful.

My friend, the Agile expert, Chuck Cobb, points out that, in a typical Agile environment, at the team level there is no one called a ‘Project Manager’. Of course, different Agile methodologies and implementations mean that some will have a PM.

Instead, the functions that might normally be performed by a project manager are distributed among the other roles on the team. That puts a very big new load on the people on the team to actually perform some project management functions in addition to their normal role. That’s a big reason why the learning can be difficult.

Here’s an article Chuck wrote on that:

Time, Cost, Quality, Scope: The Impact of Agile vs Waterfall

A fruitful way to compare Agile vs Waterfall is through the lens of the four constraints of Time, Cost, Quality, and Scope. Of the four, Scope carries the burden of the difference, because, in Agile, scope is the big variable. Time and cost vary to accommodate it – throughout the project.

With Predictive Project Management, we often fix the scope and then choose or time, cost, or quality priorities. Then, we optimize for what is left.

Let’s look at the four in turn. But, please do remember that I have presented as big a difference as I can to emphasize the contrasts. In reality, of course, the differences will usually be more subtle, and the overlap greater.

Time

Predictive Project Management seeks to fix a schedule based either on:

• a Deadline that is external to the project, or
• the achievable completion date, given the scope, quality, and resourcing constraints

Agile, on the other hand, offers schedule certainty per iteration, because it uses fixed time chunks, or sprits. But there will be no certainty for project completion.

Predictive project management makes it easier to measure and assess progress. However, Agile approaches allow users to see and test working iterations at an early stage, and then frequently through the project. This can add to the sense of ownership they feel for the newly emerging products.

The iterative nature of Agile development can easily lead to frequent revisions to the scope of the work, beyond the original definition and design. Where this happens a lot, there is a risk in significant extensions to the time scale that the client may envisage. This also has consequences for expectations of cost…

Cost

Cost mirrors schedule. In predictive projects, we plan and budget the full project at the outset. In agile projects, we make incremental decisions. This gives us good cost control for sprint-by-sprint spending, but poor estimation of the final out-turn cost at the start.

Without a doubt, some aspects of Agile project management do lead to a lower cost base:

• reduced overhead in developing documentation
• less re-work of early features that are no longer to the right specification
• avoidance of ‘feature bloat’ – the introduction of scope items that will not turn out to deliver sufficient value

Agilists also claim that Agile Teams are more productive, but I do doubt that they can support this by solid evidence. Productivity depends on many things and for every highly productive agile team, I’d expect to be able to show you an equally productive team working in a more traditional way… And also an Agile team that is wasteful and inefficient!

Quality

We can set and define quality equally well for both. But where quality depends on integrating many components, the certainty of what you wil deliver allows predictive approaches to foresee points of integration, and plan them effectively.

Within Agile projects, quality is built into each iteration, rather than being a sequential activity which can be lost or minimized where time pressures come to bear. There is also a sense, in Agile projects, that team members believe to a greater extent that quality is not ‘someone else’s responsibility’. In predictive projects, it does often sit with people responsible for Quality Design, Assurance, and Control.

An upshot of this, which Agilists can reasonably claim, is that customer satisfaction is particularly high after Agile Projects. Maybe that’s because of higher levels of customer involvement at all stages, though. However, there is nothing in the predictive approach that prevents extensive user and customer engagement.

Scope

In Agile, the scope grows iteratively. I would say this is the biggest difference between the two. A big advantage of Agile approaches is the incremental definition of scope. Thee is no need to identify early every part of what you will need. And it also allows for more easy adaptation to new demands, as well as re-prioritization. Finally, Agile methods make it easy to stop adding to scope when new increments would not deliver sufficient value to users.

Another benefit of Agile is the ability to rapidly bring a Minimum Viable Product to market or into beneficial use. This can have huge commercial advantages in crystallizing savings or getting ahead of competitors. And the lessons learned from each new iteration can feed into customer- and user-centric decisions about the focus of the next cycle (iteration) of work.

A challenge that Agile presents stems from this high level of user and customer involvement: what if they have neither the time nor inclination to contribute? Good agile development is highly dependent on close collaboration with users and customers.

Agile vs Waterfall: When to use each One

Binary choices are foolish. Neither Agile nor Predictive Project Management is best. Each is right, within a range of contexts. And, in reality, the choice is not a simple ‘either-or’.

Agile is largely preferred by Agile-trained delivery team-members. But their clients, who need to plan integration with operational procedures to a deadline want more certainty. And when those sponsors need to set and keep to a budget, the Agile disinclination to estimate costs can be very frustrating.

However, when done well, and when used in the right context, Agile delivery can:

• reduce risk
• speed time to market
• reduce costs
• respond to changes faster

On the other hand, Agile methods do require an additional level of training. To embed it within an organization, will take a whole cohort of trained personnel but more than that too. It’s a significant change in how to do things, and so needs a substantial transformation effort. The downsides to this are real disruption, so if it’s not enough that you need to get the change process right; timing will be a crucial factor. And, the reality is that using agile approaches for large scale projects is still in the development phase. You would need to sign-up for early adopter status, with all of the risks (and potential rewards) that this would entail.

But some of the key indicators that will help us assess what is most appropriate are:

• Scale of the project, in terms of:
  • budget
  • duration
  • team size
  • product set
• Complexity of the project, in terms of:
  • technology
  • interdependencies
  • team dynamics and physical constraints
  • stakeholder politics
• Sponsoring organization, and its:
  • culture
  • embedded practices
  • client preferences
  • stakeholder perceptions

Traditional, Predictive Project Management is Best when…

Let’s start with the Big Two. Predictive project management will likely be the best starting place when you:

1. …have well-defined solutions that you will need to implement is a complete fashion.
   The obvious example is a bridge. Half a bridge is useless. And you wouldn’t want an engineer to use trial and error on the strengths of trusses or pillars.
2. …need high levels of predictability and low uncertainty
   This may be for governance or political reasons, or simply because other people need to plan around your project. Where clients and sponsors need a high level of certainty of the scope of delivery, budget, and schedule, you’ll need, as a minimum, a predictive shell to your project.

Additional Circumstances that Suggest a Predictive Approach

• Projects where you need to co-ordinate with other teams, projects, functional units, or organisations
• When you have a ‘virtual team’ with remote workers
• Small, simple, well-defined projects
• Projects where your client and users are unable to contribute to frequent decision-making

Agile, Adaptive Project Management is Best when…

We have a corresponding Big Two indicators for starting with an Adaptive approach to your project:

1. Needs are poorly defined at the outset
   So, you will have to evolve the requirements through a series of incremental additions to a core requirement that drives the start of your project. This may be because people are unwilling to commit to an end state, or possibly because they cannot – they don’t have the knowledge. A good example of this is a startup launching a new software product. Customers will provide feedback to help the development team prioritize fixes and features. The uncertainty demands the flexibility of an adaptive approach.
2. When you need to get something implemented quickly…
   But it does not need to be complete at the start. An Agile approach typically creates faster development times, where products can be in use or on the market more quickly. Incremental development allows early delivery of a first portion of the solution (the MVP) without the entire solution being ready, or even fully defined.

Additional Circumstances that Suggest a Adaptive Approach

• Creativity and innovation can maximize the value of the products and solutions you create
• Projects where one small team is responsible for the whole project
• Your project is largely discrete and independent of other activities, initiatives, and projects.
• Where that team is co-located and able to communicate well
• Your client and users are ready, available, and keen to involve themselves fully
• There is no imperative to announce a firm delivery date for finished products
• There is no such thing as a complete solution

Agile-Waterfall Hybrids

Fusion cuisine is not a matter of sticking curry on top of a pizza and expecting it to taste good. Likewise, creating a working hybrid of Agile and Predictive principles and tools is a task for a highly experienced practitioner. And by one who has a deep understanding of both traditions.

In the UK, we have the term ‘cut and shut’. This is where the remains of two or more damaged cars have been welded together to create a ‘new’ vehicle. For obvious safety reasons, this is illegal. Likewise, a cut and shut project methodology that welds together a Frankenstein’s monster of tools and techniques will not work. A subtle blend is what you need.

An example might be an outer framework of structured planning and, within that, an adaptive methodology for progressing product development. Chuck Cobb, whose Agile training we recommend, illustrates his Managed Agile Development Framework in a short article.

The Spectrum from Predictive to Adaptive

How you craft your hybrid of Agile and Predictive processes, mindsets, and tools must follow the needs of your project. We’ve seen in the previous section what characteristics suggest each approach. And earlier, we saw what each approach can offer. Questions to ask include:

• How clear is the final outcome you need?
• What priority do your clients and sponsor place on certainty of functionality, budget, and schedule?
• Do you want to control risk through planning or limiting your commitment?
• What is the nature of your team?
• Where do your skillset and experience lie?
• How familiar is the task at hand, or how important is innovation?

This creates a spectrum of needs and outcomes, which you can match up with a carefully crafted hybrid. I am not qualified to advise on how to do that, but the figure below illustrates how you can wrap an Agile process within a predictive framework, at different levels.

Squeezing the PMBOK Guide Process Groups into Adaptive and Hybrid Environments

The Project management Institute (PMI) is working on the 7th Edition of its Project Management Body of Knowledge (PMBOK Guide). I expect this to have a deep focus on selecting Agile vs Waterfall and how to create effective hybrid projects.

But, at the time of writing, we have the 6th edition of the PMBOK Guide and its accompanying Agile Practice Guide (APG). I shan’t dip into the Agile Practice Guide here save to note that Chapter 3, Life Cycle Selection, gives a lightweight introduction to hybrid solutions. To learn more about the guide, take a look at our article, ‘The PMI’s Agile Practice Guide: What You Need to Know‘.

What I will do is take a look at Appendix X3 of the PMBOK Guide: ‘Agile, Iterative, Adaptive, and Hybrid Project Environments’. It’s shorter than Chapter 3 of the APG, but for my money, it’s clearer and more helpful.

But what it also does is interpret the five PMBOK Guide Process Groups in terms of adaptive and hybrid projects.

The Five Process Groups

1. Initiating Process Group
   Each iteration or sprint within an adaptive lifecycle will need its own initiation
2. Planning Process Group
   There will be high level plans around your adaptive or hybrid project, but not in the detail you’d create for a predictive project. Within the plan will sit the iterative, adaptive cycles.
3. Executing Process Group
   Each iteration will be managed according to the Agile methodology you select. But these sit within an overall project framework.
4. Monitoring and Controlling Process Group
   As with execution, monitor and control will happen both within the interactions, according to the methodology you select, and above the iterative cycles, to create overall project governance and reporting.
5. Closing Process Group
   Again, each iteration needs to be closed, as does the wider project or any phase within it.

The Impact of Domain on Agile vs Waterfall

KPMG in the Netherlands has produced a valuable survey report, ‘Agile Project Delivery’ (2017). An interesting chart on page 8 shows the uptake of Agile and hybrid approaches in different disciplines. The results are very much as you may expect if you have read this far (rather than skimmed down).

The domains where certainty is most at a premium and innovation least so, see lowest take up, and highest reliance on traditional Project Management:

• Purchasing (around 15% take-up of Agile and hybrid methods)
• Logistics (~25%)
• Finance (~30%)

At the other end, we see two disciplines that value creativity and incremental improvement, alongside IT, within which Agile emerged.

• IT (around 85% take-up of Agile and hybrid methods)
• Sales (~65%)
• Marketing (~55%)

Operations sits at around 50% take up, and HR at around 35%.

Agile vs Waterfall: Concluding Remarks

I hope that in reading this, you have developed a keener appreciation for the subtleties of the, somewhat bogus, Agile vs Waterfall dilemma. You should not see them as right or wrong, nor as a binary choice. Rather, it is your job, as a project professional, to master techniques at both ends of the spectrum and to develop effective approaches to combining them to meet the demands of your project, within its context.

What are Your Thoughts on Agile vs Waterfall?

Please do leave your comments below, and I’ll respond to all contributions.

Some Really Great Articles on this Topic

As you’d expect, this is a topic that a lot of people write about – often with great passion. I’ve tried to present a broadly objective overview, with a few opinions clearly identified. Other articles are more trenchant, but nonetheless worth reading, when they are carefully thought out. I have particularly valued articles from:

• Chuck Cobb – for example: What is the Truth of Agile vs Waterfall?
  Chuck is our recommended provider for learning Agile at all levels. His blog has a lot of great content about how agile works.
• Kevin Lonergan – for example: Why Agile can Never be a Project Management Framework
  Kevin has a lot of articles – some trenchant – on his PMIS Consulting blog.
• Ben Aston at The digital Project manager also has some excellent articles about Agile Project Management.
• And, if you are thinking of…
  Studying Agile Project Management…
  Check out our Routemap and Resources page