It’s not always easy to recognize when your project is in trouble. Yes, if your project crashes and burns and completely fails to deliver, it’s failed. But sometimes you don’t realize that you’ve  or you’ve built the wrong software and are about to make your users very unhappy. That’s why Jenny and I put together our Why Projects Fail talk [pdf] a few years ago: to help people recognize when their projects are starting to go off the rails, and learn a few simple techniques for fixing them. If you learn to recognize the signs of project problems, it’s easier to take action quickly and turn your project around:

Software projects are a lot like cheesy horror movies. After you’ve seen a few of them, you know that the first guy to leave the group is going to get an axe in his head. Projects are the same way. People keep making the same mistakes over and over, and it keeps getting their projects killed.

– Why Projects Fail presentation, slide 6

But what happens when your problems are bigger than just one project? What happens when you’ve got a team, a department, or a whole organization that runs into trouble repeatedly on projects?

When Jenny and I were doing research for our first book, Applied Software Project Management (O’Reilly, 2005), we talked to many different people on all kinds of teams. And when we did, we started to notice something funny. We specifically reached out to people who had bad experiences with requirements, project management, Agile development—really, any way of trying to do things better. We were surprised that we kept hearing the same things over and over again, and started to suspect that they were symptoms of deep-rooted project management problems… and teams that were resistant to changing them.

Taking the first step

A list of the most common excuses—excuses that we’ve heard dozens or even hundreds of times, over and over again, from many different teams—is a surprisingly powerful tool. There have been many times when I’ve been brought in to try to help a team improve the way they build software. I’d be in the middle of an explanation of, say, why Agile has worked well for me in the past, and someone would give me an excuse that’s almost exactly word-for-word right out of our book. It’s actually pretty uncanny: how were we able to predict exactly what he would say in advance? But there it is, written down and printed in black and white.

That was the inspiration for the chapter in our book called Understanding Change. In it, we outline the most common excuses for sticking with a poor way of building software and not adopting something better. If you’ve ever tried to help your software team adopt a new tool, technique, or practice, you’ll see some very familiar arguments there.

Here are the top excuses that we’ve heard from teams that can’t admit that they have a problem. They’re all straight out of Applied Software Project Management. Treat each of these excuses as a big red flag, and you can use this to your advantage. If you’ve ever watched an episode of Intervention, you probably know that the first step to fixing a serious issue is admitting that you have a problem. Do they sound familiar to you? Have you ever heard any of them before? If so, it’s time to take the first step.

But if you do take the step and admit that you have project problems, that’s good news! It means that there are plenty of time-tested, tried-and-true ways to help your team. (Of course, if you’re a regular reader of Building Better Software, you already know that!)

Each of the following excuses is excerpted from pages 206 through 213 of Applied Software Project Management (Stellman A and Greene J, O’Reilly 2005). You can read a lot more about each one (and what you can do when you hear them from your team) in that chapter.

We Already Build Software Well

Denial is a common response to change. You may have identified a glaring problem, but people around you fail to even recognize it (or simply refuse to acknowledge it). Many professional software engineers and managers have never experienced a project that did not have enormous delays and serious problems; it’s often assumed that this is just part of how software is built. After all, they usually delivered something—most projects were eventually completed, and the software they built is now being used. Sure, some projects seem to always be eternally 90% done (with 90% left to go), but most of them seem to get something onto the users’ desktops (although many patches and bug fixes needed to be rolled out afterward). Isn’t that good enough?

“Not Invented Here” Syndrome

“Not Invented Here” syndrome (NIH syndrome) is a name given to a common type of organizational culture where people intentionally avoid research or innovations that were not developed within the organization. When faced with a problem, the people in the organization will typically reject a solution that is known to have worked elsewhere in the industry, solely on the grounds that it did not originate from inside the organization. They opt instead to build their own solution, often at far greater cost.

It’s “Too Theoretical”

When an idea does not make intuitive sense, many people will dismiss it as a result of “academic research,” which could not possibly apply to the world they live in. For example, to someone without a project management or software  engineering background, it may not be immediately obvious that reviews reduce defects, or that it’s important to write a specification before building software. To him, these procedures are time-consuming, with no obvious benefit. They sound good in a book, but would never work in the organization. In other words, they are “too theoretical.”

It Just Adds More Bureaucracy

An especially damaging attitude in some organizations holds that programming is the only important activity in a software project. Project management tools and techniques are seen as distractions that drain energy and effort away from the programmers’ “real job” of writing code. Any project activity that takes place before the programmers begin writing code simply amounts to “spinning our wheels,” and the goal of all early project activities should be to get the programmers doing the “real work” as quickly as possible.

You Can’t Give Me More Work!

Most of the changes that a project manager makes will increase the workload of other people. Software engineers, managers, and stakeholders who were not directly involved in building software will suddenly find themselves expected to attend status and review meetings, participate in planning and estimation activities, work with someone creating a vision and scope document or eliciting requirements, or perform other tasks that were never expected of them before. If you are making these changes, then you are the person piling additional work onto someone’s already overflowing plate. Not surprisingly, there are people who will not be happy with this arrangement.

It’s Too Risky

The economist John Maynard Keynes once wrote, “Worldly wisdom teaches that it is better for the reputation to fail conventionally than to succeed unconventionally.“ Legendary investor Warren Buffett put it this way: “Lemmings as a class may be derided but never does an individual lemming get criticized.” In other words, any manager who backs a change puts his reputation on the line; if that manager does nothing, he will not be criticized for maintaining the status quo.

Apparently I’ve earned the dubious distinction of having become an expert in project failure. I’ve always had an interest in project failure—Jenny and I have been doing our “Why Projects Fail” talk for years now, and we’ve talked to many people in many different industries (like in our fourth book, Beautiful Teams) about what’s gone wrong on their projects. We’ve looked at failures on projects through the years, from small misfires on our own projects to dramatic failures like the Tacoma Narrows Bridge disaster, to try to figure out what software developers can learn from them.

One of my favorite ways that projects can fail is death by micromanagement. It’s a nasty, insidious problem for a couple of reasons. It’s easy for a boss to fall into the micromanagement trap, especially for a project that’s starting to spiral out of control, because when you feel like your project is slipping away from you, it’s hard to resist the urge to tighten your grip on every part of it that you can.

And the people on the team have trouble recognizing it because a lot of them have never worked any other way. I’ve said it before, and I’m sure I’ll say it again: I’m willing to bet that if someone was able to conduct an accurate survey, they would find that a surprisingly large number of managers are micromanagers.

On the other hand, if you’re a boss or a project manager looking for a great way to demoralize your team and cause your projects to fail, micromanagement is a great way to do it. Here are some handy tips to make sure your team hates you and your project runs into serious trouble:

• Make sure you don’t give your team enough time to do the work, and then blame them for not getting it done on time.
• Routinely ask people to stop working on whatever they’re doing right now to take care of urgent emergency work.
• Then utterly fail to follow up on that urgent emergency work.
• Never let anyone on your team release anything or even talk to a user without giving it to you to look over first.
• When they give you that work, make sure you send it back with a whole lot of vague and poorly thought-out changes – but make sure you don’t give any extra time to make them.
• In fact, try to constantly find many small changes that your team should make, just to keep them on their toes.
• Your team needs constant attention! If it’s been more than two hours since you’ve talked to someone on your team, drop by and tap one of them on the shoulder and ask for an update.
• All organizations run on status. If the status updates stop flowing, a company can crumble and perish! Also, developers feel lonely if they haven’t given a status update in the last few hours. So make sure everyone has to fill out elaborate status reports, and make sure you hold at least three two-hour-long status meetings every week.
• Did someone on your team do something differently than how you would do it? Reprimand them! They might tell you that it works just fine, and that their way is just as good. But it’s not your way, so it’s not right.
• Remember: reading your mind is part of every team member’s job. That’s how they stay proactive!

Most of all, though, remember rule #1: Nobody is ever allowed to make mistakes! If a developer makes a mistake, it reflects badly on you, and the whole team suffers. Never admit that you were wrong about anything. If you said it once, it’s now the law and should never be questioned.

If you follow this simple advice, then your team will be demoralized in no time. Also, they’ll hate you. Oddly, though, there’s a good chance that they won’t get their resumes together and start looking for new jobs. I have a theory about this: when you micromanage a team, it drains their will to such an extent that they no longer care. Psychologists call this state “learned helplessness.”  The Wikipedia article on learned helplessness has a good description of a classic experiment by Martin Seligman and Steve Maier:

In part one of Seligman and Steve Maier‘s experiment, three groups of dogs were placed in harnesses. Group One dogs were simply put in the harnesses for a period of time and later released. Groups Two and Three consisted of “yoked pairs.” A dog in Group 2 would be intentionally subjected to pain by being given electric shocks, which the dog could end by pressing a lever. A Group 3 dog was wired in parallel with a Group 2 dog, receiving shocks of identical intensity and duration, but his lever didn’t stop the electric shocks. To a dog in Group 3, it seemed that the shock ended at random, because it was his paired dog in Group 2 that was causing it to stop. For Group 3 dogs, the shock was apparently “inescapable.” Group 1 and Group 2 dogs quickly recovered from the experience, but Group 3 dogs learned to be helpless, and exhibited symptoms similar to chronic clinical depression.

In part two of the Seligman and Maier experiment, these three groups of dogs were tested in a shuttle-box apparatus, in which the dogs could escape electric shocks by jumping over a low partition. For the most part, the Group 3 dogs, who had previously “learned” that nothing they did had any effect on the shocks, simply lay down passively and whined. Even though they could have easily escaped the shocks, the dogs didn’t try.

If reading about the Group 3 dogs reminds you of work, then there’s a very good chance that you’re being micromanaged.

Why you should care about project management

I’ve had a lot of developers over the years tell me outright that they think project management is stupid.

Some of the best developers I’ve worked with had pretty negative opinions of project management. At best, they’d say something like, “Well, that sounds good for another team, especially a really big team, but our projects run just fine without the extra overhead.” At worst, they think it’s malignant bureaucracy imposed on them by pointy-haired bosses. One bad experience can sour a good developer on project management. And that’s pretty unfortunate, because good project management can make a developer’s life a lot easier.

So why should a developer care about project management? I think the answer is in the Why Projects Fail talk that Jenny and I have done many times over the years, where we outline many different ways that projects fail. We got the idea from that presentation because we’ve spent years talking to many different people about the different kinds of project problems. We had to figure out exactly what it means for a project to fail. Obviously, if your project completely crashes and burns, it’s a failure. But the funny thing about a software project is that you can almost always deliver something – even on a project that everyone agrees is a failure, you’ll have code that compiles and runs.

So we came up with this definition of project failure:

• The project costs a lot more than it should
• It takes a lot longer than anyone expected
• The product doesn’t do what it’s supposed to do
• Nobody is happy about it

When we go through these things, we always get a lot of head-nodding, especially from developers. More importantly, we’ve never met a single professional software engineer with more than a few years of experience who hasn’t been on at least one failed project. We always ask for a show of hands to see if there’s anyone who’s never had a failed project. And we’ve yet to meet anyone who has. (We did get someone who raised his hand once, but I talked to him afterwards, and he was just messing with us.)

What does this have to do with project management? Well, everything! Have a look at a typical project management textbook, and you’ll learn about the “triple constraint” (which some people call the “iron triangle”) of scope, time, and cost.

Project management really boils down to managing that triple constraint: balancing the scope of the work to be done, the cost of doing the work, and the time it takes to get the work done, all in order to get the best quality that you can. If you do that right, your project runs well, and your team is happy. If you do it badly, your project stinks, and you end up with a failure. And that’s why you, as a developer, should care.

I’ll go into a little depth about each of those things – scope, time, cost, and quality – just to give you an idea of why you should care about each of them. But the one that developers care about most is scope, so I’ll spend a little more time on that first.

Scope: the work and features that go into the project

Every project can be broken down into a small number (say, ten to twenty) features. For big projects, those features are big (“solid rocket boosters,” “reuse and reentry”); for small projects, those features are small (“holds 12 fluid ounces of liquid,” “screw-top cap”). And similarly, every project can be broken down into a small number of tasks that are needed to build those features. Project managers call the list of features the product scope, and they call the tasks the project scope – and a lot of people just talk about all of it as scope.

I’m consistently amazed by how much work a project team can put into a project without before they realize they’ve got a scope problem. Jenny and I talked about this in our book Applied Software Project Management in the section on scope problems:

A change in the project priorities partway through the project is one of the most frustrating ways that a software project can break down. After the software design and architecture is finalized, the code is under development, the testers have begun their preparation, and the rest of the organization is gearing up to use and support the software, a stakeholder “discovers” that an important feature is not being developed. This, in turn, wreaks havoc on the project schedules, causing the team to miss important deadlines as they frantically try to go back and hammer the new feature in.

Applied Software Project Management, p31

Does that sound familiar? If so, talking about the scope earlier in the project can really help. One way to help fix this is to put together a Vision and Scope document. It’s a really simple document, usually only a page or two, but it can really help prevent scope problems. It’s also a staple of project management. Once in a while, I’m pulled into a project that’s been running into trouble. I’ll spend a few hours talking to each of the team members, managers, and anyone else I can find who knows something about the project or needs it done. I’ll throw together a quick vision and scope document, and what I’ll often find is everyone had very different ideas of what would actually get built. And it’s usually a big relief, because everyone – especially the lead developers! – realizes that they were going down a path that would have had them waste a lot of time building the wrong software.

(source: Head First PMP, 1st ed., O’Reilly 2007)

Faster, better, cheaper—choose two… right?

There’s a whole lot more that I’m tempted to write about managing time, cost, and scope in order to get to the right quality level, but I really want to pare this down to the basics. There’s an old (and somewhat cynical) project management saying: “faster, cheaper, better: pick two.” What it means is that there’s no way to reduce cost, shorten the schedule, and increase quality all at the same time. At least one of those things absolutely has to give.

But there’s a reason that’s a saying and not a law! All three of those constraints are related to each other, and there’s almost never an easy, obvious trade-off where you can sacrifice one to improve the other two. If there were, project management would be a lot easier. It’s really easy to fall into a trap of trying to do the impossible, but the first step in avoiding a trap is knowing that it’s there. In this case, just knowing that every project is a balancing act between scope, time, and cost is a really good first step in making sure your project ends up running well.

I wanted to leave you with just a couple of thoughts:

• One thing that makes the “time” aspect of project management tricky is that developers really hate giving estimates – or, more specifically, they hate giving estimates because they know they’ll be misinterpreted as hard commitments. I wrote a blog post about this a while back (The Perils of a Schedule) .
• Every project needs a start, middle, and end. My old friend Scott Berkun writes about this beautifully in his book, Making Things Happen.
• Cost doesn’t have to mean dollars – it’s often more effective to measure cost in hours.
• Burndown charts really help, because they can make the cost seem real.
• Sometimes the word “quality” makes developers a little nervous, because they start to think that improving quality just means doing endless testing. Sometimes it helps to replace a phrase like “high-quality software” with “software that we can be proud of.”

Tough Programmer Interview Questions: Project Management

Programmers are often surprised when a job interview shifts from technical questions to questions about project management. I just saw a great example of this in a recent Head First C# forum post:

This is a lot more common that developers realize. I’ve seen a lot of discussion lately about how to do effective programmer job searching, especially for relatively new developers who have three years of experience or less. Did you ever ace the tech part of an interview, only to find that you didn’t get the job? I bet that a lot of the time it comes down to questions that seem peripheral or less relevant – to the a junior developer.

Not to a lot of senior developers. That’s one of the biggest differences in attitude that I’ve seen between people who are new to development and people who have been doing it for a long time. And in a lot of cases, I think it really does come down to attitude. So my goal with this post is to outline the basics of project management, the core things that really matter.

Why should you care about project management?

There’s a flip side to project management, too. A colleague of mine once asked me, “What’s the most important part of project management?” I told him that it’s managing stakeholder expectations – making sure that the people who have control of the project or are affected by it are in the loop on all the important developments as the project rolls along. If bad things happen, they know about them in advance, and are prepared. The reason for this is that some projects fail (more than you think!), often for reasons that have nothing to do with the team. If you manage everyone’s expectations, get them on the team’s side, then the developers can come out as heroes fighting a lost cause. On the other hand, a project can be a roaring success, but if everyone expects something that’s not exactly what was delivered, the developers could be blamed for something that they had no control over. Expectations matter, communication matters, and these things can have a big impact on the project and the team.

And that’s why developers should care about project management: it affects your life, even when your job is to keep your nose buried in code all day.

A lot of developers have a very poor opinion of project management and project managers. I’ve spent a lot of my career writing books and giving training to help project managers improve their skills. Over the years, I’ve met many different types of project managers. And, unfortunately, while there are plenty of great ones out there, there are a lot of really bad ones as well. In any field, there is a wide range of skill level and aptitude. If you’re a developer who’s only ever worked with poor project managers, it’s not surprising if you ended up with a dim view of the project management as a whole.

But even if you’re a developer who doesn’t have a high opinion of the field, you should at least acknowledge that learning more about it can have an impact on your own career. I’ve personally seen employers pass on good developers who didn’t know enough about project management, even ones who had the technical skills to do the job.

I’ve also conducted a lot of developer interviews, easily several hundred of them over the last decade. And one thing that I’ve noticed is that really good developers have a healthy respect for exactly the same things that really good project managers care about: the work and the features needed to create the software, the team that crafts it, the effort required to build it, and the quality of the final product. That’s why I think that learning more about project management can help make you a better developer.

In the next part of this post, I’ll outline those things, and make a case for why they should matter to you.

As a group, we developers have a project tracking problem: the problem is that we constantly, almost obsessively want to track our projects, and it’s made worse by the fact that it’s so easy to abuse otherwise great tools like JIRA and Bugzilla. Unfortunately, while tracking projects may feel useful and productive, for most teams it’s just busywork – and it can lead to a self-imposed Lose Weight Exercise in needless bureaucracy that just wastes our time. But with a few ground rules, you can escape the project tracking trap and use a tracking tool effectively.

Who needs Big Brother?

About five years ago, a small company brought me in for a few months to help them do better project management. For about a year before I got there, they’d put a couple of their best developers on the problem, and they came up with an ingenious solution for tracking projects. The team built a database to keep track of all the projects they were working on, typically about a dozen and a half simultaneous projects running at any time. This tool gave them the ability to figure out exactly what tasks had been done, who did them, and how long they took. If you wanted to know what any specific programmer was working on eight weeks ago on Tuesday afternoon, their tool would be able to tell you that.

And it all worked perfectly.

Except that their projects were still late and full of bugs, and the users were losing their last shred of patience. This team had promised their users that things would get better with their fantastic new project tracking system. But even though the system was working exactly as advertised, the software still had serious problems, and everyone was unhappy.

I’ve been thinking a lot about that company lately because I’ve been doing a lot of work with JIRA. And while JIRA is a really good tool – one that can have a very positive effect on how you manage a project – it seems to really tempt a lot of good developers down the same project tracking trap that the small company fell into.

Programmers normally hate it when their time is tracked, which is obvious to anyone who makes the mistake of suggesting to a programming team that they should start filling out timecards that list out exactly what each person did that day. That happened at my first programming job after college: the boss, a newly minted MBA, told the programmers to fill out timecards that listed what project they were working on hour by hour, and there was nearly a rebellion. But for some reason, JIRA, Bugzilla, and other issue tracking systems cause programmers to step up and volunteer for exactly that treatment. It’s an odd phenomenon, and I think it has an interesting explanation.

Let’s get something straight, here: I really like JIRA a lot. It’s a great issue tracking tool, and a very good workflow tool when it’s used right. I’ve spent some time building JIRA plugins, and the plugin API is intuitive and easy to use. Sure, I’ve got a couple of nitpicks here and there (I’m not so happy with the user interface or the filters in version 3.x, although it’s definitely improved in JIRA 4). But for the most part, JIRA is one of the best issue management and workflow tools I’ve ever used.

When it’s used right.

But when you take a perfectly good tool and abuse it, you’re bound to get bad results. And that’s especially true when it comes to project tracking tools.

Tracking projects is not a useful goal

JIRA suffers from a problem that a lot of really good issue tracking tools suffer from. It’s really good at tracking things. And for a lot of developers, that causes a “when you’re holding a hammer, everything looks like a nail” problem. Actually, scratch that. It causes a “you’re holding a screwdriver, but for some reason you want to pound nails with it” problem. It’s odd when you step back from it, but it really seems to make sense at the time.

What happens is that some developers – and I admit that I’ve been guilty of this at least once or twice myself – see a tool that gathers information about what happened on a project, and this causes something to click for us. We  start tracking everything, from requirements to goals to user requests to every little ten-minute task that needs to be done. We become our own “big brother,” doing what amounts to voluntarily filling out the most detailed timecards possible.

The “big brother” style of project tracking is especially bad when the “pointy haired boss” type of managers pick up on it. Suddenly, the team is being nagged to start entering more and more granular estimates and updating dates in tickets. For a boss who’s nervous about delivering something, it seems like the right thing to do is to start nagging programmers about every little task that’s entered into the system, or chasing down anyone who’s got a task assigned without a due date.

Which is especially bad, because that’s not really what those tools are for. There’s a big difference between tracking a task or a bug so it doesn’t slip between the cracks and tracking every little thing that a the programmers do for posterity. I think it’s especially ironic that developers often volunteer for this themselves. I think it’s because we see a tool that can track something and it causes us to want to use it. “It tracks tasks? Then it should track all the tasks!”

Better living through project tracking

A while back the New York Times ran a story on people who use everyday tools like Foursquare and Twitter, blogs, spreadsheets, and datebooks to obsessively track data about themselves. I think this excerpt is telling, especially the bit at the very end (emphasis mine):

At the center of this personal laboratory is the mobile phone. During the years that personal-data systems were making their rapid technical progress, many people started entering small reports about their lives into a phone. Sharing became the term for the quick post to a social network: a status update to Facebook, a reading list on Goodreads, a location on Dopplr, Web tags to Delicious, songs toLast.fm, your breakfast menu on Twitter. “People got used to sharing,” says David Lammers-Meis, who leads the design work on the fitness-tracking products at Garmin. “The more they want to share, the more they want to have something to share.” Personal data are ideally suited to a social life of sharing. You might not always have something to say, but you always have a number to report.

I think that the sort of person who becomes a developer is especially susceptible to whatever it is that causes some people to obsessively track things about their lives. My personal theory is that this is caused by the same itch causes us to want to obsessively collect every last episode of MST3K (or whatever it is we happen to want to obsessively collect). And like that giant collection of, say, legos or video game cartridges or Start Wars figures memorabilia or dice, at some point we can’t actually play with most of it. That’s fine for collections (assuming you have it in a cool display case, and not strewn all around your home in piles so high that they cause a fire hazard). But for tracking data from projects, it’s worse than useless, because it takes effort to keep up, and it can demoralize the software team because they start to feel like they’re creating more tickets than new lines of code.

Luckily, if you follow a few ground rules, you can break the obsessive cycle and start using project tracking the way it should be used. Here’s what I’ve recommended to people in the past, and it seems to get good results:

• Figure out in advance what you want to track. Most teams use an issue tracker for bugs. A lot of teams use it to figure out what work the developers are going to do. Some use it for new feature requests. I’ve seen people try to use an issue tracker for managing requirements (although personally, I really dislike that use of it). All of that could be fine, but any of it can potentially become obsessive. Before you start your project or the next iteration, sit down with the team for a few minutes and write down what you’ll be tracking. Stick it on a Wiki page, and add a sentence or two explaining how each one of those things needs to be used. I’m always amazed at how just a little bit of planning like this keeps the way we use a tool from creeping into “big brother” territory.

• Use post-mortem reports to track your lessons learned. One reason that “big brother” style project tracking makes intuitive sense is that we want to keep track of the lessons we learned on this project in order to make the next one run better. But there’s very little you can learn from an overwhelming pile of tracking tickets. Instead, try running a project post-mortem at the end of the project. And don’t just do it for projects that ran badly. Post-mortems are really good for making sure you take a good project and turn it into great habits for the team by writing down the lessons you learned.

• Focus on trends and simple metrics, not complete history. A good project tracking system can spit out a lot of reports. So many reports, in fact, that it’s hard to make any real sense of them. That’s why I like to try to simplify the kind of data I get out of the system. I’m a big fan of simple metrics like (% of engineering effort per project phase) that let you compare projects or iterations against each other. Also, burn-down charts or other earned value metrics can be really valuable in figuring out how your current project is going.

The bottom line is that there’s a difference between project tracking and actually managing your project. If you really want to use JIRA for project management, use a tool like Greenhopper that was built for it. (I’ve been using Greenhopper lately, and I like it a lot. But if someone has a pointer to an open source tool they really like that ties a task board or agile project management into their issue tracker, I’d love to hear about it.) The most important thing to remember is that tracking your project is a means to an end, and not a goal by itself.

We got this e-mail a few days ago from one of our readers:

Hello,

I bought your book, “Applied Software Project Management.” It seems very good overall, but I can’t get past the fact that your book seems to imply that software requirements come after the project plan/WBS/scheduling. Am I missing something?

On page 40, the script for estimating states that the input is documentation that defines the scope of the work being performed. Does this include the SRS? If so, why is this not made more explicit in your book (since requirements plays such a big role)? If not, how can a good estimate and schedule be generator before the requirements analysis has been performed?

I don’t get a solid sense of the relative timing of the activities (especially the requirements activity). Can you comment on this?

Thanks!!

– Wayne M.

That’s an excellent question. I’ve got a straightforward answer, and I’ve got a more involved answer.

The straightforward answer is yes. If you’re using Wideband Delphi (or, really, almost any estimation practice) to come up with estimates that you’ll turn into a schedule, then you need to get a handle on exactly what you’re estimating. So yes, when we say in our book that the input to the process is the “Vision and Scope document, or other documentation that defines the scope of the work product being estimated,” the “other documentation” we’re referring to definitely includes any software requirements you have. (For any readers who haven’t read our book, you can download a PDF of the estimation chapter that this reader’s referring to.)

Let me be clear about something here. You’re absolutely right that requirements analysis leads to more accurate schedules. If you’re lucky enough to have a really detailed specification at the beginning of the project that describes, in detail, all of the software that you’re going to build, then that will give you a much more accurate schedule than if you had a three-page Vision and Scope document that simply lets you know who the users and stakeholders are, explains their needs, and gives you the broad strokes about what features the team will build to meet those needs.

But when’s the last time you actually had the luxury of a complete specification before you had to deliver a schedule? And I’m stressing the word “complete” for a reason: it’s very rare that you’re done with the requirements before you’ve started building the software. So rare, in fact, that neither Jenny nor I have ever seen it in our entire careers, and I suspect very few (if any) of our readers have, either.

A good Agile developer might point out that this is the reasoning behind one of the core Agile principles: “Welcome changing requirements, even late in development.” And, in fact, that’s exactly why we dedicated so much of the chapter on software requirements in Applied Software Project Management to change control, because it’s important to not only accept that change happens, but to recognize it as a good thing. It’s better to change course partway through the project rather than to trundle on to an end goal that you know won’t actually meet your users’ needs or make your customers happy.

So with that in mind, go back to the process you mentioned. Specifically, take a look at the end of the script, because this is where it ties directly into the question you asked:

There’s an important idea there in those first six words: the project plan has been updated. That means that any time your world changes, you need to go back and update the scope, the WBS, the schedule, all the actual stuff you plan to deliverable (which might be written down in a statement of work), the list of people doing the work, a risk plan (if you built one)… all the stuff you used to plan your project.

…because there’s no single one Best Way™ to build software

And that’s why we didn’t give an explicit order to the activities. Sometimes you’ll end up planning out your project and building a schedule before you do requirements analysis; sometimes you’ll build a schedule after. Our goal was to help you do it well in either case. And by not forcing our readers into a single process or methodology, we don’t have to pretend to know all the answers… because, as far as I know, there is no single one Best Way™ to build software.  That’s the main idea, by the way, behind our “diagnose and fix” approach to improving the way you build software. Trying to overhaul your whole software process by doing a major process improvement effort is hard; adopting specific practices that make incremental improvements to the areas that hurt the most is much easier and a lot less risky.

This may sound like we’re calling for a lot of documentation, but it doesn’t have to be like that. Obviously, if you’re working on a team with dozens or even hundreds of people (like the teams Jenny often leads), this can be a pretty big task. But if you’ve got a small team working on a project that will take a few weeks to do, then this may just amount to rearranging your task board, updating your user story cards, updating a couple of Wiki pages, and having a quick stand-up meeting to make sure everyone’s in sync. That’s why people often talk about how running a really big team is like steering an aircraft carrier, because changing course requires miles of water, while running a small team is a lot more like piloting a speedboat that can change course really quickly.

So that’s the straightforward answer. But I think it’s worth delving a little deeper and asking an important question: What’s the nature of a schedule? I know, that probably seems like an odd question, but it’s actually important to understand what schedules are and how they’re used. (Technically, it’s only important if you want your projects to run well.)

A naïve answer might be to simply defer to that old project management chestnut: “Plan the work and work the plan.” If you haven’t heard that saying, take a minute and do a Google search for it. It’s one of those sayings that people love to quote, and it pretty much summarizes how a lot of people use (abuse?) project schedules.

I don’t like that saying, and there’s a reason for that.  I mean, don’t get me wrong, here. “Working the plan” is fine if it that plan accounts for changes. That’s one thing I really like about the PMBOK® and PMP approach: a whole lot of project planning revolves around how to handle changes, and specifically about dealing with change control. Also, it’s a great idea to make sure that you include a reserve for your project, and you can use risk planning to try to get a handle on the unknown.

To be continued in “The perils of a schedule, part II“

Some projects just go wrong.

It’s a fact of life. Projects go over budget, blow their schedules, squander their resources. Sometimes they go off the rails so spectacularly that there’s nothing you can do except (literally) pick up the pieces and try to learn whatever lessons you can so you don’t repeat the failure in the future.

Last week I got a phone call from a developer who was looking for some advice about exactly that. He’s being brought in to repair the damage from a disastrous software project. Apparently the project completely failed to deliver. I wasn’t 100% clear on the details—neither was he, since he’s just being brought in now—but it sounded like the final product was so utterly unusable that the company was simply scrapping the whole thing and starting over. This particular developer knows a lot about project management, and even teaches a project management course for other developers in his company. He’d heard me do a talk about project failure, and wanted to know if I had any advice, and maybe a postmortem report template or a lessons learned template.

I definitely had some advice for him, and I wanted to share it with you. Postmortem reports (reports you put together at the end of the project after taking stock of what went right and wrong) are an enormously valuable tool for any software team.

But first, let’s take a minute to talk about a bridge in the Pacific Northwest.

The tragic tale of Galloping Gertie

One of my favorite failed project case studies is Galloping Gertie, which was the nickname that nearby residents gave to the Tacoma Narrows Bridge. Jenny and I talk about it in our “Why Projects Fail” talk because it’s a great project failure example—and not just because it failed so spectacularly. It’s because the root causes for this particular project failure should sound really familiar to a lot of project managers, and especially to people who build software.

The Tacoma Narrows Bridge opened to the public on July 1, 1940. This photo was taken on November 7 of the same year:

While there were no human casualties, despite heroic attempts at a rescue the bridge disaster claimed the life of a cocker spaniel named Tubby.

Jenny and I showed a video of the bridge collapsing during a presentation of our “Why Projects Fail” talk a while back in Boston. After the talk, a woman came up to us and introduced herself as a civil engineer. She gave us a detailed explanation of the structural problems in the bridge. Apparently it’s one of the classic civil engineering project failure case studies: there were aerodynamic problems, and there were structural problems due to the size of the supports, and there were other problems that combined to cause a distinctive resonance which gave the bridge its distinctive “gallop.”

But one of the most important lessons we took away from the bridge collapse isn’t technical. It has to do with the designer.

[A]ccording to Eldridge, “eastern consulting engineers” petitioned the PWA and the Reconstruction Finance Corporation (RFC) to build the bridge for less, about which Eldridge meant the renowned New York bridge engineer Leon Moisseiff, designer and consultant engineer of the Golden Gate Bridge. Moisseiff proposed shallower supports—girders 8 feet (2.4 m) deep. His approach meant a slimmer, more elegant design and reduced construction costs compared to the Highway Department design. Moisseiff’s design won out, inasmuch as the other proposal was considered to be too expensive. On June 23, 1938, the PWA approved nearly $6 million for the Tacoma Narrows Bridge. Another $1.6 million was to be collected from tolls to cover the total $8 million cost.

(Source: Wikipedia)

Think back over your own career for a minute. Have you ever seen someone making a stupid, possibly even disastrous decision? Did you warn people around you about it until you were blue in the face, only to be ignored? Did your warnings turn out to be exactly true?

Well, from what I’ve read, that’s exactly what happened to Galloping Gertie. There was plenty of warning from many people in the civil engineering community who didn’t think this design would work. But these warnings were dismissed. After all, this was designed by the guy who designed the Golden Gate Bridge! With credentials like that, how could he possibly be wrong? And who are you, without those credentials, to question him? The pointy-haired bosses and bean counters won out. Predictably, their victory was temporary.

Incidentally, some people refer to this as one kind of halo effect: a person’s past accomplishments give others undue confidence in his performance at a different job, whether or not he’s actually doing it well. It’s a nasty little problem, and it’s a really common root cause of project failure, especially on software projects. I’ve lost count of the number of times I’ve encountered really terrible source code written by a programmer who’s been referred to by his coworkers as a “superstar.” Every time it happens, I think of the Tacoma Narrows Bridge.

But there’s a bigger lesson to learn from the disaster. When you look at the various root causes—problematic design, cocky designer, improper materials—one thing is pretty clear. The Tacoma Narrows Bridge was a failure before the first yard of concrete was poured. Failure was designed into the blueprints and materials, and even the most perfect construction would fail if it used them.

Learning from project failures

This leads me back back to the original question I was asked by that developer: how do you take stock of a failed project? (Or any project, for that matter!)

If you want to gain valuable experience from investigating a project—especially a failed one—it’s really important that you write down the lessons you learned from it. That shouldn’t be a surprise. If you want to do better software project planning tomorrow, you need to document your lessons learned today. You can think of a postmortem report as a kind of “lessons learned report” that helps you document exactly what happened on the project so you can avoid making the same missteps in the future.

So how do we take stock of a project that went wrong? How do we find root causes? How do we come up with ways to prevent this kind of problem in the future?

The first step is talking to your stakeholders… all of them. As many as you can find. You need to find everyone who was affected by the project, anyone who may have an informed opinion, and figure out what they know. This can be a surprisingly difficult thing to do, especially when you’re looking back at your own project. If people were unhappy (and people often are, even when the final product was nearly perfect), they’ll give you an earful.

This makes your life more difficult, because it’s hard to be objective when someone’s leveling criticisms at you (especially if they’re right!). But if you want to get the best information, it’s really important not to get defensive. You never know who will give you really valuable feedback until you ask them, and it often comes from the most unexpected places. As developers, we have a habit of dismissing users and business people because they don’t understand all of the technical details of the work we do. But you might be surprised at how much your users actually understand about what went wrong—and even if they don’t, you’ll often find that listening to them today can help make them more friendly and willing to listen to you in the future.

Talking to people is really important, and having discussions is a great way to get people thinking about what went wrong.  But most effective postmortem project reviews involve some sort of survey or checklist that lets you get written feedback from everyone involved in or affected by the project. Jenny and I have a section on building postmortem reports in our first book, Applied Software Project Management, that has a bunch of possible postmortem survey questions:

• Were the tasks divided well among the team?
• Were the right people assigned to each task?
• Were the reviews effective?
• Was each work product useful in the later phases of the project?
• Did the software meet the needs described in the vision and scope document?
• Did the stakeholders and users have enough input into the software?
• Were there too many changes?
• How complete is each feature?
• How useful is each feature?
• Have the users received the software?
• How is the user experience with the software?
• Are there usability or performance issues?
• Are there problems installing or configuring the software?
• Were the initial deadlines set for the project reasonable?
• How well was the overall project planned?
• Were there risks that could have been foreseen but were not planned for?
• Was the software produced in a timely manner?
• Was the software of sufficient quality?
• Do you have any suggestions for how we can improve for our next project?

We definitely recommend using a survey where the questions are grouped together and each question is scored, so that you can start your postmortem report with an overview that shows the answers in a chart. (If you’re looking for a kind of “lessons learned template,” this is a really good start.)

The rest of the report delves into each individual section, pulling out specific (anonymous) answers that people wrote down or told you. Here’s an example:

Beta
Was the beta test effective in heading off problems before clients found them?
Score: 2.28 out of 5 (12 Negative [1 to 2], 13 Neutral [3], 9 Positive [4 to 5])
All of the comments we got about the beta were negative, and only 26% (9 of 34) of the survey respondents felt that the beta exceeded their expectations. The general perception was that many obvious defects were not caught in the beta. Suggestions for improvement included lengthening the beta, expanding it to more client sites, and ensuring that the software was used as if it were in production.
Individual comments:

• I feel like Versions 2.0 and 2.1 could have been in the beta field longer so that we might have discovered the accounting bugs before many of the clients did.
• We need to have a more in-depth beta test in the future. Had the duration of the beta been longer, we would have caught more problems and headed them off before they became critical situations at the client site.
• I think that a lot of problems that were encountered were found after the beta, during the actual start of the release. Shortly thereafter, things were ironed out.
• Overall, the release has gone well. I just feel that we missed something in the beta test, particularly the performance issues we are experiencing in our Denver and Chicago branches. In the future, we can expand the beta to more sites.

(Source: Applied Software Project Management, Stellman & Greene 2005)

There’s another approach to coming up with postmortem survey results that I think can be really useful. Jenny and I have spent the last few years learning a lot about the PMBOK® Guide, since that’s what the PMP exam is based on. If you’ve studied for the PMP exam, one thing you learned is that you need to document lessons learned throughout the entire project.

The exam takes this really seriously: you’ll actually see a lot of PMP exam questions about lessons learned, and understanding where lessons learned come from is really important for PMP exam preparation.

The PMBOK® Guide categorizes the activities on a project into knowledge areas. Since there are lessons learned in every area of the project, those categories (the knowledge area definitions) give you a useful way to approach them them:

• How well you executed the project and managed changes throughout (what the PMBOK® Guide calls “Integration Management”)
• The scope, both product scope (the features you built) and project scope (the work the team planned to do)
• How well you stayed within your schedule or if you had serious scheduling problems
• Whether or not budget was tight, and if that had an effect on the decisions made during the project
• What steps you took to ensure the quality of the software
• How you managed the people on the team
• Whether communication—especially with stakeholders—was effective
• How well risks were understood and managed throughout the project
• If you worked with consultants, whether the buyer-seller relationship had an impact on the project

For each of these areas, you should ask a few basic questions:

1. How well did we plan? (Did we plan for this at all?)
2. Were there any unexpected changes? How well did we handle them?
3. Did the scope (or schedule, or staff, or our understanding of risks, etc.) look the same at the end of the project as it did at the beginning? If not, why not?

If you can get that information from your stakeholders and write it down in a way that’s meaningful and that you can come back to in the future, you’ll be in really good shape to learn the lessons you need to learn from any project. Even a failed one.

I got a great question from a software developer who also happens to be a fellow CMU alum.

I have a question related to managing scope creep with respect to “on-going”/iterative development processes.

I’m currently managing a project where we’re redesigning my application’s primary workflow. Simply put, the app is currently designed to have users to signoff all items and we’re redesigning it to be exception-based (only require certain items to be signed off).

As we’ve progressed down the path of planned iterative development, a lot of good/new ideas for future enhancements/requirements spring up. I find myself regularly working with my users and “working group” to prioritize and analyze if any of these new ideas need to be considered to build sooner rather than later (and thus triggering plan adjustments or delays).

I often feel like I’ll end up delivering a product that does deliver the initial vision, but still doesn’t make my users happy, as they’ve already shifted their expectations to wanted the “next” thing (aka phase 2).

Do you have any other tips about how to manage this process?

I’ve used things like taking a timeline-style roadmap and adjusting it by overlaying the new requirements and shifting the timeline out. Do you have an recommendations of ways to present this type of information?

Curious your thoughts. Thanks.

— Seth L.

Iterative software development can be a really useful—and highly effective—tool for software development, but it’s also one of the most abused tools I’ve seen. Even as recently as a few days ago, someone in charge of a software team that’s critical to his comapny came to me with the (incorrect) assertion that iteration just means diving into a prototype without talking to anyone or doing any investigation. Iteration done well can lead to very high quality software. But as Seth saw, iteration done poorly can lead to scope creep and serious planning issues.

Making your users “happy” means managing their expectations. They need to see exactly what you’re working on, and what’s coming next. If all people see is deadlines and they don’t have a sense of what’s going on, then they naturally start looking to the next deadline, because that’s all they see. The more visibility you can give them into the way you build software, the more understanding they’ll have – that’s just human nature.

There are a few things that work well for improving visibility into your project’s iterations. One of them is a task board, which typically involves sticking index cards with your user stories or scenarios on a whiteboard or wall. This means that you actually need to have user stories or scenarios. Scope creep is often an indication of a requirements problem, and getting consensus on at least the scenario level about exactly what’s going into the iteration. Having the index cards arrayed on a task board, with each index card showing the status (“planned”, “in development”, “completed”, “out of scope”) gives a lot more visibility into exactly what you’re building and how you’re building it. In a lot of ways, this is a sort of iterative development project plan.

Another way to prevent iteration-related scope problems is committing to delivering releasable code at the end of each iteration. Test-driven development (or, at least, developing complete unit tests) and continuous integration are effective ways to help make that happen. If your stakeholders are comfortable that you’ll deliver high-quality code at each iteration, they’ll feel less pressure to get the new features in immediately, and will be more willing to wait until the next iteration.

If this sounds like something that might help you, I definitely recommend reading the interview Jenny and I did with Mike Cohn for Beautiful Teams, who knows more about agile and iterative planning than pretty much everyone . He has a lot to say about effective iteration planning. There are pictures of task boards he’s used in the past.

That gets me back to the basic idea—one which I give Mike a whole lot of credit for helping people understand—that iterative development, especially in an Agile project, works best when we take the time to plan each iteration. It still faces the same problems: requirements need to be discovered, scope needs to be controlled, and progress needs to be communicated to everyone who cares about it… especially to anyone who can potentially make the developers’ lives more difficult. That’s why the most successful Agile development projects collect requirements and document them using user stories (or other techniques for writing down what the software needs to do). They plan their progress using task boards, forecast them using calculations and charts like project velocity and burn down rates, and constantly keep any business owners and other stakeholders up to date on their progress.

It’s a great way to develop software, and it’s been really effective for a lot of teams. And I think it shows that iterative development does not necessarily mean unplanned development.

When I sent this to the developer who sent me the question, he had an interesting follow-up, which I thought deserved a response:

So I’ve been digesting this a bit and I am curious to get your thoughts about adapting project management fundamentals into the often fluid process of app management. I manage a few virtual projects within my company and at times have struggled to keep things focused as business demands and/or interests shift over time. Similarly, the “iterative” approach has helped to clarify requirements while building out new flows/apps, but as you pointed out can be very tricky to get “right”.

It’s funny how often I hear people say, “Well, this project management stuff works in theory, but my project is fluid” (or “changing,” or “under too much pressure from the business,” or “critical,” etc.). It turns out that pretty much every project is challenging, and project management is set up specifically to deal with that kind of challenging project.

Here are two thoughts I had relating to this idea.

First, the iterative development model works very well, as long as you’re committed to delivering a high-quality product at the end of each iteration. Whether or not you develop using an iterative approach, you need to manage change: prevent unnecessary changes, and make sure you understand the impact of any change that you make. It also means that you need some sort of scope baseline, so that you know what is and is not a change. It’s faster and easier to update software on paper, before it’s written into code, so the more changes you can move to the “write it down and review it” phase of your project, the better.

And second, if your business is overly demanding, it often means that you could manage your stakeholders’ expectations better. Make sure you identify them – and write down their names and needs! – from the very beginning of the project. Talk to them… a lot. Make sure they’re in the loop. If possible, see them so often that they’re sick of seeing you. If they’re always aware of what you’re doing, there won’t be nearly as many surprises. Also, the more your stakeholders understand the details of the work that you’re doing, the more slack they’ll cut you when they ask you for changes. Often, when someone puts a lot of pressure on you to do the impossible, it’s because they don’t realize that’s what they’re asking.

I was thinking about that presentation yesterday, after getting off the phone with a manager at a company that wants to hire us to do software estimation training for their programmers. One problem that they’re having is a pretty common one. Their programmers, testers, and even project managers seem reluctant to give estimates. That reminded me of this slide from the talk:

When we get to this slide in the talk, I usually say something like this:

There’s something we call the “about three weeks” problem. Have you ever noticed that when you ask a programmer how long it’ll take to do something, it’s always going to take “about three weeks”? I’ve done it myself many times over the last fifteen years or so. How long to build a simple website to do a few calculations? About three weeks. What about a utility that will automate some file operations and generate a few reports? About three weeks. A new feature for our core product? Sure, it’ll take about three weeks. See, the problem is that three weeks seems like a long enough time to get something significant done. And if you think for thirty seconds about pretty much any programming project, you can do enough hand-waving and ignore enough details so that it’ll seem to fit into about three weeks.

What does this have to do with why programmers are so reluctant to give estimates?

There are many reasons for this, more than I’ll go into here. But a big one might just be because we’ve all quoted “about three weeks” for a programming project that ends up taking a whole lot longer than that, and we never want to be stuck in that situation again. So after we’ve been burned enough, we just stop giving estimates. I was at a job a few years ago, sitting at a full conference table with a dozen developers. The CTO — an abrasive guy who clearly went home every evening to lift Lose Weight Exercises, and spent most of his day yelling at the people who reported to him — growled an “order” at the team, demanding an estimate. Everyone at the table knew that they’d be yelled at individually, threatened with dismissal, and generally made miserable if they didn’t come up with an estimate. Yet nobody looked up and volunteered anything. Eventually a junior guy in the back cleared his throat, and in almost a whisper he said, “I’m not sure about the rest of it, but I think my piece will take about three weeks.”

And there it is. Nobody wants to go on the record and say how long they think it’ll take to do a job. We all know it’ll take as long as it takes. If the estimate is right, then there’s no great reward or recognition. But if the estimate is wrong, then we’re on the hook for it, and we get to look forward to status meetings where we get to take the blame for whatever terrible consequence happened because the project was late.

So what do we do about it?

Jenny and I put a lot of thought into this problem when we were working on our first book, Applied Software Project Management. It turns out that there’s a really effective way to get a good idea of how long the software will take without putting any one person on the hook, and it’s our favorite way of generating estimates. It’s called Wideband Delphi, and we talk a lot about it in the Estimation chapter in the book (which you can download for free [PDF]). It’s a straightforward technique — it just takes two two-hour meetings to nail down estimates for even a large project. It’s very iterative and highly interactive, which helps the team all come to a consensus and agree on the final result. It’s collaborative, so that no one person is solely responsible — usually, everyone ends up buying into the final numbers. And best of all, it doesn’t require any special expertise beyond what you need to actually do the project.

My favorite part about Wideband Delphi is that it’s really focused on assumptions. That’s another thing I like to talk about in the “Why Projects Fail” talk. If you think that building a particular program is going to take nine weeks, but I think it’s going to take four weeks, we usually aren’t disagreeing on how long it’ll take to do the task. Usually, we’re disagreeing on some basic assumption about the task. For example, you might think that we’ll be building a big GUI with printing support, while I might think that it’s just going to be a console application. That means that we can assume for the sake of the estimate that we’ll either build a GUI or build a console application, and we’ll write down that assumption. That way, if it turns out to be incorrect, we’ll know exactly why the estimate was wrong… and if someone in a meeting later wants to blame us, we can point to that assumption and give a good reason for the delay. (That’s why Delphi has two meetings: the first meeting is mostly taken up with a discussion of those assumptions.)

One nice thing about Delphi is that it’s not some esoteric, theoretical thing. Both Jenny and I have done this in the real world, many times, with all sorts of software teams. Delphi really does work, and it actually does a good job of helping team come up with numbers. And those numbers are pretty accurate, at least on the projects I’ve worked on. If you’re having trouble convincing your reluctant team to come up with an estimate, I definitely recommend giving Delphi a shot.

Here’s a PDF of the slides from the talk. The fourth slide is this video of the 1940 Tacoma Narrows Bridge disaster. And to anyone who attended, we’re always happy to answer questions — feel free to get in touch with us if you want to follow up on anything we spoke about.

We promised to upload the slides and post a couple of links, so here they are. Here’s the PowerPoint presentation [PPT] and a PDF version [PDF].You guys asked great questions — some of the best we’ve ever gotten at one of our talks. Here are a few of the answers that we promised:

• A few of you had questions about estimation, specifically the Wideband Delphi process that we’ve had a lot of success with. You can read about it in Chapter 3 of our first book, Applied Software Project Management– here’s a link to a PDF of the chapter [PDF]. We give a pretty detailed description of exactly how to hold a Wideband Delphi meeting, and how you can use it on your own projects to improve how they’re estimated.
• One person brought up a really good point about integration, and how that’s an important failure point that gets neglected, and we mentioned that we’d post a link to an article we wrote for Dr. Dobb’s Journal on integration testing called “Better, Stronger, Faster Integration Testing: Giving developers a personal stake in software quality.”
• I think a lot of the questions near the end of the talk about open source projects were answered in our ONLamp.com article called “What Corporate Projects Should Learn from Open Source”. Also, here’s a link to the “What Makes Open Source Projects Work” presentation I gave last January at the SD Best Practices India 2007 conference.
• No, we don’t have an official release date for “Head First C#” yet, but we’re definitely making good progress on it. Keep watching this blog — as soon as O’Reilly has an official release date for it, we’ll post about it. And yes, it is really fun to write a Head First book.
• After the talk, a few people asked about our availability to come in and do training. Our consulting schedule is a little tight because of our pretty aggressive writing schedule for O’Reilly, but we do have some availability. You can use our “Contact Us” page to get in touch with us about consulting and speaking — serious inquiries only, please.

We’re always happy to answer questions about anything we talk or write about. Feel free to get in touch with us any time!

Estimates have a tendency to be more accurate when they’re based on experience. Having actual data on your past projects is so important when you sit down to figure out how long it’s going to take you to do a new one. The best estimates come from people who really understand the work that’s going to get done. And practices for estimation that stay grounded in project experience are less likely to be wrong.

That’s why your project will probably be really late if you just cave in and start shaving off time arbitrarily. It might not be easy, but you need to explain the reasoning behind the estimates your team has come up with and set your boss’s expectations realistically. Too many projects are in a bad position before they ever really get started because a project manager doesn’t correct this kind of misunderstanding.

It’s one of the toughest parts of the job because your boss doesn’t want to know all the details. He just wants you to get it done faster. And sometimes you can get done faster with more resources or with a different scope of work. But it’s always a mistake to just cut an estimate because it doesn’t match what people expect. You need to turn the conversation around and help everybody get to see what the real choices are. Just saying that project will take less time, never makes it happen that way– no matter how influential the person who wants it done yesterday is.

If you estimate right, you should never go back to your team and tell them to arbitrarily cut those estimates down. Instead, you should work with your boss to understand if there might be some work that can be cut from schedule, or some other ways to accomplish what needs to be done in parallel. If you set up a relationship where you and your boss are both doing everything you can to come up with a reasonable plan together, you stand a much better chance of succeeding.

Fred Brooks wasn’t really talking about schedule estimation techniques, or writing hard-and-fast rules. He was trying to help software engineers break some very bad habits. And trying to fix a failing project by throwing bodies at it is one of the worst habits software engineers have. See, as it turns out, some tasks take however long they’re going to take, and throwing bodies at the problem won’t change that. Or, as Brooks wrote, “The bearing of a child takes nine months, no matter how many women are assigned.”

I think the best way to understand that Brooks quote was to understand why he first wrote “The Mythical Man-Month” back in 1975. At the time there was something called the software crisis. The field of software engineering, still young at the time, had run in to a serious problem: people were having a whole lot of trouble writing software on time and under budget.

By the time the software crisis was in full effect, the engineering world had already matured a great deal. The great civil engineering projects of the early 20th century taught us how to manage enormous projects effectively. (Hoover Dam was finished two years ahead of schedule! And Henry Gantt invented of everybody’s favorite scheduling tool around 1910.) And by the mid-1970s, the world of computers had matured to the point where most large corporations had computer programming teams and IBM “big iron” timesharing mainframes. It seemed like the tools were there, and the people had the skills to use them.

So why were all of our software projects so late?

Personally, I blame the person who coined the term “software”. The thinking, I believe, went like this: “Hardware is really expensive, and basically impossible to change. But luckily, we’ve got these programs that we can constantly tweak to suit our needs! That’s the opposite of hardware, which can’t be changed.”

By the time the mid-1990s came around, the field of software project management had matured to the point where we knew that change was the biggest factor in late software projects. Changing needs and requirements meant pulling apart the underpinnings of our code and caused us to have to do expensive (and often ineffective rework). Changing technology kept giving programmers “silver bullets” that they thought would make this next project run a whole lot more smoothly than the last one. And the (perceived) ability to change the software architecture and design in the middle of the project caused a lot of teams to fail to plan their projects well (or at all!).

The worst part of it was that it seemed like the software crisis never ended — there were still an enormous number of projects that were still coming in very late and over budget, or failing to complete at all. But at least by that point we knew the root cause of the problem and had tools to help us fix it. In fact, we were learning that adapting to change and getting those changes under control was an important part of effective software projects.

So how did we know that those things were causing the crisis in the first place, and that fixing those problems would get our software out the door on time and within budget? We can thank Fred Brooks for that. “The Mythical Man-Month” was, as far as I know, the most important software engineering book of its time. It laid out the real problems that caused software projects to run into trouble, and showed us a lot of very useful ideas about planning our software projects, structuring our teams, estimating the work, communicating with each other and our stakeholders, and writing our documentation. And a lot of what he wrote still resonates very well with modern software projects. (A lot of the tools and techniques that Jennifer Greene and I wrote about in Applied Software Project Management were designed specifically to address those root causes of software project problems.)

And that gets back to Brooks’ quote about adding manpower to a late project. When Brooks talked about a late software project, he was talking about a project that had constantly shifting requirements and needs, poor planning, and communications problems both among the team members and between the team and the stakeholders. In a situation like that, many project managers will panic and just try to throw bodies at the situation. (I’ve seen this many, many times — and so have almost all software engineering professionals.) But those extra people will just cause the bad project plan to become even more inaccurate, and they will compound the already serious communications problems. That, in turn, will cause the team to spend extra effort dealing with those problems. And that extra effort will often be larger than the man-hours that the newly added team member will add to the team.

That’s why Brooks called his book “The Mythical Man-Month”. If you need to get your two-person project done in six months, you can’t just add another person on and magically get it done in four months. Adding a person to a project for a month doesn’t automatically reduce the total effort by one man-month. The effort that one person can contribute to a project varies enormously based on the project circumstances. His “adding manpower” is saying that if the project is already late, then adding an extra person can actually subtract man-months from the project.

When you’re talking about a large project, communication is the area with the biggest potential risk. That shouldn’t be a surprise; after all, some people say that 90% of project management is communication. The larger your project, the more complex the communications can get.

Anyone who’s studied for the PMP already exam knows about this stuff. Remember that formula you had to memorize for counting lines of communication? Well, this is exactly why you need it! As you add more and more people to the project, the number of potential lines of communication goes up exponentially. And if you’re not prepared for it, you can really loseWeight Exercise track of who’s talking to who, and about what. So communications management — putting together a communications plan, keeping your project teams informed, managing your stakeholders’ expectations, and generally serving as a hub of communications — really starts to make a big difference.

Recently, I had to face a situation with increasingly complex communications. I was working with a consulting company that specialized in outsourced software engineering projects. We were assigned a large part of a much larger project, one that was bigger than any other project the client company had taken on. There was another major vendor on the project as well, also doing a large amount of work. In previous projects for this client, we normally didn’t do much work alongside other vendors. But in this case, I saw that the project was divided into several teams on our side, and several more teams in the other vendor’s group. It was clear to me that there would be a large risk of communications problems. So I worked with the project manager from the other vendor to establish communications channels between our leads and theirs (getting buy-in from the client, of course). By planning for and establishing a separate line of communication, we were able to take some early steps to mitigate communications risks. This is definitely paying off now, and the project is going relatively smoothly.

As your project gets bigger, your risk management also becomes a lot more complicated. Which means you need to take more care with how you plan and track your risks. If you don’t already keep a risk register, start. And definitely spend a lot more time talking with your team about risk planning at the beginning of the project. Scott Berkun has a lot of good advice in The Art of Project Management about brainstorming and talking to your team about ideas. All of that good stuff applies to discovering and analyzing risks. Take extra time at the start of the project to work with the team to identify risks, and figure out how to handle them. Are you going to mitigate them? Which ones do you need to accept? Can you figure out ways to transfer the risks? These are all questions you should be able to answer BEFORE the risk materializes. Revisit your risks at regular team meetings. This is one of those areas that really benefits from economies of scale — the more your team is thinking about risk management, the more potential and actual project problems you’ll avoid.

Is your project REALLY large? If you’re managing a project with 1,000 people, then you really need a solid metrics system. You’re never going to get a good handle on what your team is doing by just talking to people — there are simply too many people to talk to! That doesn’t mean you don’t communicate, of course. But you do need to reliably collect data, and then analyze that data statistically. This is what tools like run charts, control charts (or Shewhart charts) and trend analysis were developed for. Here’s a tip: If you’re playing in that arena, and if you have budget for consultants, find yourself someone who’s familiar with those tools to help you set up a system to gather and analyze project data. That sort of thing is very hard to set up once the project wheels are already turning, though, so you’ve got a limited window of opportunity to get it established.

(Inicidentally, if you haven’t read Scott Berkun’s book, you definitely should. I had the pleasure of being a technical reviewer for it, and it’s one of the best books on project management I’ve ever read.)