Late projects, man-months and the software crisis

I recently got a question recently asking about Fred Brooks’ famous quote: “Adding manpower to a late software project makes it later.” The person asking the question took it literally, and was asking about whether this meant that there’s no way any schedule estimation technique could indicate that adding manpower could shorten delivery time.

Fred Brooks wasn’t really talking about schedule estimation techniques, or writing hard-and-fast rules. He was trying to help software engineers break some very bad habits. And trying to fix a failing project by throwing bodies at it is one of the worst habits software engineers have. See, as it turns out, some tasks take however long they’re going to take, and throwing bodies at the problem won’t change that. Or, as Brooks wrote, “The bearing of a child takes nine months, no matter how many women are assigned.”

Business Plan

I think the best way to understand that Brooks quote was to understand why he first wrote “The Mythical Man-Month” back in 1975. At the time there was something called the software crisis. The field of software engineering, still young at the time, had run in to a serious problem: people were having a whole lot of trouble writing software on time and under budget.

By the time the software crisis was in full effect, the engineering world had already matured a great deal. The great civil engineering projects of the early 20th century taught us how to manage enormous projects effectively. (Hoover Dam was finished two years ahead of schedule! And Henry Gantt invented of everybody’s favorite scheduling tool around 1910.) And by the mid-1970s, the world of computers had matured to the point where most large corporations had computer programming teams and IBM “big iron” timesharing mainframes. It seemed like the tools were there, and the people had the skills to use them.

So why were all of our software projects so late?

Personally, I blame the person who coined the term “software”. The thinking, I believe, went like this: “Hardware is really expensive, and basically impossible to change. But luckily, we’ve got these programs that we can constantly tweak to suit our needs! That’s the opposite of hardware, which can’t be changed.”

By the time the mid-1990s came around, the field of software project management had matured to the point where we knew that change was the biggest factor in late software projects. Changing needs and requirements meant pulling apart the underpinnings of our code and caused us to have to do expensive (and often ineffective rework). Changing technology kept giving programmers “silver bullets” that they thought would make this next project run a whole lot more smoothly than the last one. And the (perceived) ability to change the software architecture and design in the middle of the project caused a lot of teams to fail to plan their projects well (or at all!).

The worst part of it was that it seemed like the software crisis never ended — there were still an enormous number of projects that were still coming in very late and over budget, or failing to complete at all. But at least by that point we knew the root cause of the problem and had tools to help us fix it. In fact, we were learning that adapting to change and getting those changes under control was an important part of effective software projects.

So how did we know that those things were causing the crisis in the first place, and that fixing those problems would get our software out the door on time and within budget? We can thank Fred Brooks for that. “The Mythical Man-Month” was, as far as I know, the most important software engineering book of its time. It laid out the real problems that caused software projects to run into trouble, and showed us a lot of very useful ideas about planning our software projects, structuring our teams, estimating the work, communicating with each other and our stakeholders, and writing our documentation. And a lot of what he wrote still resonates very well with modern software projects. (A lot of the tools and techniques that Jennifer Greene and I wrote about in Applied Software Project Management were designed specifically to address those root causes of software project problems.)

And that gets back to Brooks’ quote about adding manpower to a late project. When Brooks talked about a late software project, he was talking about a project that had constantly shifting requirements and needs, poor planning, and communications problems both among the team members and between the team and the stakeholders. In a situation like that, many project managers will panic and just try to throw bodies at the situation. (I’ve seen this many, many times — and so have almost all software engineering professionals.) But those extra people will just cause the bad project plan to become even more inaccurate, and they will compound the already serious communications problems. That, in turn, will cause the team to spend extra effort dealing with those problems. And that extra effort will often be larger than the man-hours that the newly added team member will add to the team.

That’s why Brooks called his book “The Mythical Man-Month”. If you need to get your two-person project done in six months, you can’t just add another person on and magically get it done in four months. Adding a person to a project for a month doesn’t automatically reduce the total effort by one man-month. The effort that one person can contribute to a project varies enormously based on the project circumstances. His “adding manpower” is saying that if the project is already late, then adding an extra person can actually subtract man-months from the project.

Useful ideas for really large projects.

How do you manage a large project? There’s a big difference between managing a project team with, say, ten people and a team with a hundred. That shouldn’t be too surprising. But if you’re managing your first large project, it can be really daunting. Luckily, there’s been a lot of good work done on understanding large projects and how to manage them. The risks are bigger, and different, than anything you’ve seen on a smaller project. But if you pay attention to the tools that you’re using, you should be able to get a handle on things.

When you’re talking about a large project, communication is the area with the biggest potential risk. That shouldn’t be a surprise; after all, some people say that 90% of project management is communication. The larger your project, the more complex the communications can get.

Crowd 2007-04-30

Anyone who’s studied for the PMP already exam knows about this stuff. Remember that formula you had to memorize for counting lines of communication? Well, this is exactly why you need it! As you add more and more people to the project, the number of potential lines of communication goes up exponentially. And if you’re not prepared for it, you can really loseWeight Exercise track of who’s talking to who, and about what. So communications management — putting together a communications plan, keeping your project teams informed, managing your stakeholders’ expectations, and generally serving as a hub of communications — really starts to make a big difference.

Recently, I had to face a situation with increasingly complex communications. I was working with a consulting company that specialized in outsourced software engineering projects. We were assigned a large part of a much larger project, one that was bigger than any other project the client company had taken on. There was another major vendor on the project as well, also doing a large amount of work. In previous projects for this client, we normally didn’t do much work alongside other vendors. But in this case, I saw that the project was divided into several teams on our side, and several more teams in the other vendor’s group. It was clear to me that there would be a large risk of communications problems. So I worked with the project manager from the other vendor to establish communications channels between our leads and theirs (getting buy-in from the client, of course). By planning for and establishing a separate line of communication, we were able to take some early steps to mitigate communications risks. This is definitely paying off now, and the project is going relatively smoothly.

As your project gets bigger, your risk management also becomes a lot more complicated. Which means you need to take more care with how you plan and track your risks. If you don’t already keep a risk register, start. And definitely spend a lot more time talking with your team about risk planning at the beginning of the project. Scott Berkun has a lot of good advice in The Art of Project Management about brainstorming and talking to your team about ideas. All of that good stuff applies to discovering and analyzing risks. Take extra time at the start of the project to work with the team to identify risks, and figure out how to handle them. Are you going to mitigate them? Which ones do you need to accept? Can you figure out ways to transfer the risks? These are all questions you should be able to answer BEFORE the risk materializes. Revisit your risks at regular team meetings. This is one of those areas that really benefits from economies of scale — the more your team is thinking about risk management, the more potential and actual project problems you’ll avoid.

Risk mitigation

Is your project REALLY large? If you’re managing a project with 1,000 people, then you really need a solid metrics system. You’re never going to get a good handle on what your team is doing by just talking to people — there are simply too many people to talk to! That doesn’t mean you don’t communicate, of course. But you do need to reliably collect data, and then analyze that data statistically. This is what tools like run charts, control charts (or Shewhart charts) and trend analysis were developed for. Here’s a tip: If you’re playing in that arena, and if you have budget for consultants, find yourself someone who’s familiar with those tools to help you set up a system to gather and analyze project data. That sort of thing is very hard to set up once the project wheels are already turning, though, so you’ve got a limited window of opportunity to get it established.

(Inicidentally, if you haven’t read Scott Berkun’s book, you definitely should. I had the pleasure of being a technical reviewer for it, and it’s one of the best books on project management I’ve ever read.)

What about Agile?

One of the best things about writing a Head First book is that you get feedback from other Head First authors. Our editor, Brett (who you may know from Head First OOA&D), got some interesting feedback from Bert Bates. Bert was going through Head First PMP, and pointed out — and rightly so — that at first glance here seems to be some distance between the PMP approach to projects and the Agile approach.So he’s right. There’s definitely some distance between what you’ll see on the PMP exam and in the PMBOK® Guide, and what you’ll see in an Agile development process like XP, SCRUM, or Crystal. There’s a lot of stuff Agile does that isn’t addressed in the PMBOK, and there’s a lot that’s on the PMP exam that Agile doesn’t address. But this shouldn’t really be a surprise to anyone. See, the PMBOK wasn’t written specifically for developers. A lot of projects that use the PMBOK processes and principles are things that you can’t do iteratively — like, say, highway construction or building a skyscraper. That’s why you see a lot of focus on things like subcontracting and procurement, risk management, communications (which you need to plan for really carefully when you’ve got a thousand people working on a project!), and budgeting. These are things that Agile doesn’t address because they’re just out of its scope.That’s not to say the PMP stuff doesn’t work on software — in fact, it works really well. So why isn’t the opposite true? Why couldn’t an Agile process work for, say, building a high-rise?Needs More WindowsJenny and I were lucky enough to have a lot of contact with some of the people who created the PMBOK while we were working on the book. And as it turns out, they were very much into iteration and iterative development. But the PMBOK and the PMP exam need to apply to all kinds of projects, including non-iterative, non-software ones. And iteration does have its place, even in construction. You should definitely approach the design and planning iteratively. But while iteration may work fine for project plans and blueprints, but it doesn’t work particularly well once you’ve broken ground.But there is one big area where Agile and the PMBOK Guide are really similar: managing change. Change management is really important in the processes you need to know for the PMP exam. They are very clear on the fact that changes happen on every project, and that you need to make sure that you plan for change and expect it to happen. Every single knowledge area you need to study stresses that no matter what sort of project you’re working on, you need to constantly look for changes, and make sure that you change course whenever changes are necessary.Sound familiar? It should! Because it’s one of the fundamental goals of Agile development — the Agile manifesto itself says that we’ve come to value “responding to change over following a plan”. And that’s really similar to what the PMBOK tells us: that when there’s a change, we need to modify our plans in order to accommodate that change. (It also wants us to make sure that we know how much the change is going to impact the project, and that everyone involved agrees that the cost of making the change is worth the benefit… which is definitely a good idea too.)When to iterateAll in all, I think that there’s a lot of value in the ideas behind the PMBOK and the PMP exam, and that an Agile shop could benefit from understanding and applying them. I definitely don’t think that the PMBOK and Agile development are incompatible. But it’s important to keep in mind that they solve different problems… and that neither Agile nor the PMBOK are intended to be a silver bullet to automatically repair all troubled projects!Oh, one more thing to remember. Jenny and I are software people, and we’ve spent most of our careers trying to figure out how to deliver software projects better. That’s what our first book was about. We spend some time in that book talking about Agile — and a lot of time talking about some specific practices that were popularized along with Agile development: refactoring, test-driven development, continuous integration, pair programming and a few others. We love those practices, and regularly use them on the job. Personally, I always do test-driven development when writing my own code. But that’s definitely an area that you simply don’t see on the PMP exam, and for good reason. What does it even mean to build, say, a highway on-ramp using test-driven development? (Actually, that question may actually turn out to have a meaningful answer. If anyone can think of one, please let us know!)