Ever since Jenny and I wrote Head First C#, I’ve gotten a lot of e-mail and questions from developers just starting their careers. (I think it has to do with the fact that our book has become one of the most popular ways to learn how to program in C#.) But there’s a big topic that we don’t cover in the book, a topic that a lot of our readers really want to know about: how to get a better development job once you’ve finished the book and feel pretty confident with C#.

We get this question every now and then on the book’s forum. And when it comes up, I usually point people to something I posted there earlier this year. I’ve gotten a lot of good feedback from people who were happy to see that advice. But I’ve been getting more e-mails from people who want a little more information. More and more people are looking to improve their job prospects, maybe because they’ve been reading headlines about the economy. So I wanted to delve a little deeper into managing your career in software development.

First things first. It sounds a little silly to state it outright, but it’s worth repeating: the most important thing that makes a good professional programmer stand out is his programming skill. Our readers already know this, because we said it in the intro:

There’s only one way to learn to program: writing a lot of code. And that’s what you’re going to do throughout this book. Coding is a skill, and the only way to get good at it is to practice.

That’s why we included a whole lot of programming puzzles and exercises throughout the whole book – not just pen-and-paper puzzles, but actual programming problems that need to be solved by writing code. One thing that a lot of junior programmers don’t realize is that programming is about more than just coding. It’s about solving problems, sometimes really tough ones. Solving the kind of problems that you come across at work is a skill, one that improves with practice. And the only way to get practice is to find problems and solve them.

Obviously, Jenny and I think that Head First C# is a great way to get that practice. But it’s certainly not the only way. There are all sorts of projects that you can do. And if you don’t have a project that needs to be done, try thinking of one. That’s what I did back in 1994 or so, when I wanted to learn Perl and I also wanted to learn about these newfangled web servers that everyone was talking about. So I decided to give myself a project: write a simple web server in Perl. Doing lots of projects like that (sometimes for myself, sometimes for others) made me a better programmer, and doing lots of projects will make you one, too.

There’s another thing that I think really helps expand your programming horizons: learn more than one language. There are a lot of peculiarities specific to any one language. On the other hand, there are many general concepts that are common to entire classes of programming languages, or all languages in general. If you’ve only worked with one language, then it’s very hard to know what’s specific to the language you know.

So if you’re starting to feel comfortable as a C# programmer, I definitely recommend giving yourself a Java project; if you’re a Java programmer, do a little C# coding. The two languages are very similar, but you’ll learn a lot from the differences. But if you really want to push yourself and learn something valuable, move away from managed languages and try writing a little C or C++ code. Try picking up a functional language like LISP. Personally, I love assembly languages, and I’ve had a lot of fun learning about microcontrollers over the years. Simply taking a few days to learn enough about a new language to do a simple project can be a very valuable experience for a programming starting out a career. Plus, it gives you something to talk about in an interview!

Coding and problem solving aren’t the only skills a professional software developer needs. There’s another necessary skill, one that’s arguably just as important: working with a team. That’s something I’ve been thinking a lot about lately (since it’s the subject of our next book): what makes a good team, and what programming team members and leaders can do to work better and build better software. There are so many facets to working with project teams, from understanding and interacting with different personalities to finding and fixing roadblocks that keep the work from getting done. And while there are certainly some great books on project teams and teamwork, a lot of them probably won’t make all that much sense to someone who’s completely green.

This used to be a catch-22: you need experience on a software team to get a job working on one, but you can’t get a job working on a software team without experience. But there’s a solution to the problem now that wasn’t available when I was first starting out. There are plenty of open source projects looking for contributors, and even a new developer can make a valuable contribution.

Even if you just volunteer as a tester, or try your hand at fixing a few bugs, you’ll learn a lot about how people work together on software teams. In fact, you can learn a lot just by lurking in the forums and learning how the project is set up. What kind of version control system does it use? Can you figure out how to check out the code and get it to compile? Do they use a continuous integration system like CruiseControl to automatically do periodic builds? How do the team members review each others’ code?

Before you start working on an open source team, I highly recommend reading Karl Fogel’s excellent book, Producing Open Source Software. You can buy it from O’Reilly, but you can also read it for free on the book’s website. It will teach you a lot about both the mechanics and culture of open source teams. (A little disclosure: I was a technical reviewer for the book, and was really impressed with it when I read it.)

I originally meant this to be advice for people looking to start a programming career. But reading back over this, I think that veteran programmers could learn from it too. Of course, if you’re looking for a programming job, clearly there’s no substitute for actually putting years working as a professional developer. But doing these things can help push your career forward, even if you’re already partway through it.

A lot of programmers feel like being asked to go to a code review is like being told by mom to eat our veggies. We’ll complain about it, and even if we do eventually swallow them we’re determined not to enjoy them.

It’s something I’ve seen over and over again: programmers groaning about having to go to a code review, usually because someone gets some big idea about making things better, and decides this is how you do it. There’s sometimes a little nervous joking at the beginning of the meeting about how nobody really wants to be there. And after it’s done, a lot of us get the distinct feeling that it was a waste of time.

The thing is, code reviews can be a really good thing. And not only that, they don’t have to be a chore. If you do them right, people on the team can start to appreciate them and even - heaven forbid - enjoy them.

So how do we make code reviews more palatable? We need to think about what motivates us as programmers. Programmers love to code. We love building things, and we love solving problems. But we hate anything that seems bureaucratic or tedious, and we definitely hate meetings. But most of all, we hate being in uncomfortable social situations that require us to confront people face-to-face. We’re not alone in that; most people hate situations like that.

I think that’s a pretty big part of why programmers intuitively dislike code reviews. It’s not that we’re afraid of putting our work out there for our peers to see. That’s actually something we look forward to: we love to show off code that we’ve worked so hard on, and we definitely appreciate the input from the people around us. But it’s almost never the person whose code is being reviewed who groans about it. No, it’s usually the people who are asked to attend the review. And I think I know why: it’s because we don’t like being asked to criticize the work of others, openly and without hesitation, for the good of the team. I think we naturally feel uncomfortable putting someone else in the position of having to openly confront their errors, because it’s so easy to imagine ourselves in that same position.

And that may be the secret to holding code reviews that people actually look forward to attending: make it about helping make the code better, not criticizing and finding its flaws. If you can come up with a way to avoid bitter arguments about tiny details and stubbornly-held opinions, and instead concentrate on helping someone improve his or her code, then people will stop thinking about code reviews as an uncomfortable meeting and start thing about them as a way to build better software.

That sounds like a tall order. How do you do it?

Well, you start out by thinking about one of the best things that can happen in a code review, something that I’ve seen many times. It usually happens about two thirds of the way through the review, about the time when the first signs of meeting fatigue are starting to set in. Someone points out another problem with the code, and a conversation starts up about an aspect of it that nobody really thought of. Uh-oh — it’s a bug, a particularly nasty one that. Then everyone kind of looks at each other with a weird mixture of relief and disgust, because we found a bug that a) definitely would have made it into production, and b) would have taken hours or days to track down and fix.

I’ve said this many times before: programmers are very practical people, especially when it comes to our own time. If something will save us time down the road, we’ll definitely do it. If you can show a programmer that a tool or technique (like a code review) saves more time than it costs, that’s a great way to get him or her to start thinking positively about it.

That doesn’t change the fact that a lot of people get nervous about code reviews, even people who have done them a bunch of times. So I spent a little time thinking of advice I’ve given about code reviews over the years. Some of this is pretty standard code review stuff, but I think it’s worth repeating because people have so many different views on how to do code reviews effectively. And I think that if you think about them, and get other people on your team thinking the same way, it could definitely help you hold effective code reviews.

So here’s some advice about holding better code reviews — you can think of them as code review tools (or even code review best practices, although I’m not a huge fan of that term) that can make your software better:

• First things first: there are a lot of different ways to do a code review. Some people like to follow a very strict process. Personally, I like to keep them informal; the more it seems like an everyday conversation, the more work we actually get done.

• It’s important to keep the meeting to under two hours — any more than that and meeting fatigue sets in. Most code review teams can handle between 200 and 400 lines of code in a two-hour review meeting. (Your mileage may vary.)

• Don’t forget to distribute the code before the review, and make sure you give everyone enough time to actually look through it. Send around a PDF of the code (a2ps is a good way to make it readable, and it’s got stylesheets for almost every language). Also make sure that everyone also has access to the source, and that they know how to build and run it. Sometimes it’s a lot easier to prepare for a code review if you can actually debug your way through the code.

• Make sure that everyone knows you appreciate their time. It’s easy to forget that, but it helps the team see the review as a useful tool and not just another boring meeting. Remember, you’re pulling half a dozen or more people into a room for two hours, plus preparation time — that’s the equivalent of taking a top developer off of all projects for two days. That’s also why it’s very important to choose a good block of code to review. Choose one that’s inherently risky: a difficult algorithm, code from a library that many other people depend on, an interface a lot of people will use, a particularly nasty bit of spaghetti code.

• Code reviews and refactoring work really well together. Look for opportunities to extract methods, rename variables, replace literals with constants, etc.

• Pay attention to OOP principles, especially encapsulation. Improving encapsulation is an easy and effective way to prevent bugs.

• The code review isn’t just about bringing up issues — it should also be about giving some indication of how to resolve those issues. Ideally, the programmer whose code is being reviewed should be able to read through the results of the review and very quickly implement the fixes, because in the meeting we wrote down exactly what needs to be fixed and how to fix it. (We don’t actually have to write down lines of code, of course — just enough information so it’s clear what to do.) A good way to do this is to make the goal of the meeting to be to produce a log, or a list of fixes that need to be made to the code.

• Instead of storing the log in a spreadsheet, Word document, or wiki page (I’ve done all three), try having the moderator put the results of the review directly into the code as comments (which includes an easily searchable unique string like ‘// %%TODO: CODE REVIEW 8/28/08%%’, so the programmer can find them all). The results of the review meeting can be e-mailed out a link to a diff report in the source repository. When the programmer goes back to update the code, he or she can alter the comments to make sure they make sense in context — but they can stay in the code because they’ll make more sense, and it’ll be clear why the code is the way it is if someone is maintaining it in the future.

• A good way to focus the discussion is to guide the conversation away from arguments about coding in general, and towards what to write down in the log to resolve the current issue. Make a good effort to figure out how to resolve the issue: most can be resolved in the meeting. Any issue that can’t be resolved in a reasonable amount of time gets added to the log as an open issue.

• I’ve always gotten a lot of mileage out of using a moderator. The moderator’s job is to keep track of the discussion, and keep the discussion on track. If people are getting off onto a tangent that won’t benefit the code, or if they’ve gotten into a disagreement where there are merits on both sides of the issue and it clearly won’t be resolved, the moderator should suggest that we log it as an open issue and move on. You can always follow up later and resolve the issue.

• Some people get very strict about making sure that the moderator stays at arm’s length, and doesn’t get involved with the review. Personally, I think code reviews are hard enough without imposing arbitrary rules like that (because we’re laying someone’s code bare and dissecting it, which can be difficult for anyone who’s not used to it). We’re all adults here, and we can trust any of us not to “abuse” a moderator role. If the moderator has something to say, he or she should say it. If it’s easier, replace “moderator” with “note-taker” or something like that.

• Don’t be pedantic, and try to avoid theoretical discussions. It’s really easy to get bogged down with a discussion that doesn’t go anywhere about whether this variable declaration should be here or there, whether this type of structure or that is slightly more efficient, if we could do something better in theory if we scrapped a large amount of code and rewrote it. If a discussion isn’t going to directly lead to a change, even if it’s an interesting topic, note it in the log as an open issue and move on. And definitely don’t point out spelling errors. A lot of grate programmers are lousy spelers.

• Make sure variable names make sense, but don’t worry about naming conventions. Some people love underscores in front of parameters, some people hate them. I’m sure you can come up with at least three different “official” conventions for any programming language. There are few things less useful during a code review than an argument over whether to use PascalCase or camelCase.

• One way people like to do reviews is to have people “read” the code - interpret it out loud. I’ve had some success with going around the table and having people take turns “reading” each chunk of code. If there’s a chunk that is difficult to “read”, it’s not clear enough and is a good candidate for refactoring.

• Before you distribute the code to the team, run it through a static code analyzer (like FindBugs or FxCop) and fix issues that are found. There’s no need to waste discussion time on problems that a tool can catch and log for us.

• I’ve had a lot of success with a kind of review called a “high impact inspection” (that’s a term that was developed at HP about fifteen years ago). Basically, it boils down to having everyone do the code review independently and e-mailing their issues to a moderator. The moderator puts the issues into one big list, sends them back out to everyone, and then the review meeting itself is focused on just going through those issues. Jenny and I developed a code review process similar to high impact inspections to let us hold inspections in teams outsourced to India (where time zone differences make it very difficult to regularly schedule two-hour meetings). We ran it a bunch of times, and it worked really well.

When Jenny and I were writing the section on code review techniques in our first book, Applied Software Project Management, we came up with a checklist of things that you should look for during a code review. That should give you a good starting point for coming up with a good review.

Good luck with your code reviews! If you end up with a good code review success (or failure!) story, I’d love to hear about it.

We’ve gotten more than a few e-mails from readers wondering where we went. First of all, don’t worry - Jenny and I are still around! We’ve just been working overtime on our latest book, Beautiful Teams (here’s the Amazon page for it). It’s coming out well, and we’re really excited about it.

Beautiful Teams is a new kind of book for us, because it’s the first time we’ve done more editing than writing. We brought in a great team of guest writers and thinkers, people we really respect and whose work we’ve been big fans of over the years: Scott Berkun, Karl Wiegers, Karl Fogel, Johanna Rothman, Barry Boehm, Steve McConnell, Grady Booch, James Grenning, Cory Doctorow and a whole lot more. A good portion of the book will be stories from their careers, which was a big departure for us: it was as much about creative writing (helping our authors make the stories fun to read) as it was about software engineering. And in addition to the stories, Jenny and I have been doing interviews with some of these great thinkers, and I’m extremely happy about how they’re coming out. We’ve put a lot of effort into finding people who are thoughtful and knowledgeable, but also really good at talking about what they do and how they think. All in all, it should be a very unique and fun to read book.

Oh, there’s one more thing I want to tell you about the project - actually, my favorite part about it. We’re donating almost all of the royalties to PlayPumps International, a truly wonderful charity that digs wells to deliver clean drinking water to rural schools and villages in sub-Saharan Africa. If you haven’t heard of them, take five minutes and watch this piece that Frontline did on them.

I’m sure we’ll give you plenty more details about it as time goes on. In the meantime, hopefully we’ll be able to post a little more here for you!

I was working on a project earlier today. Now, typically I always do test-driven development, where I’ll build unit tests that verify each class first and then build the code for the class after the tests are done. But once in a while, I’ll do a small, quick and dirty project, and I’ll think to myself, “Do I really need to write unit tests?” And then, as I start building it, it’s obvious: yes, I do. It always comes at a point where I’ve added one or two classes, and I realize that I have no idea if those classes actually work. I’ll realize that I’ve written a whole bunch of code, and I haven’t tested any of it. And that starts making me nervous. So I turn around and start writing unit tests for the classes I’ve written so far… and I always find bugs. This time was no exception.

This time, for some reason, that exercise reminded me of the story of the nearly disastrous Citicorp Center building.

Citicorp Center was one of the last skyscrapers built in the New York City skyscraper and housing boom in the 1960s and 1970s. A lot of New Yorkers today probably don’t realize that it was actually one of the more interesting feats of structural engineering at the time. The building was built on a site occupied by St. Peter’s, a turn-of-the-century Lutheran church that would have to be demolished to make way for the skyscraper. The church agreed to let Citigroup demolish it, on one condition: that it be rebuilt on the same site.

The engineer, Bill LeMessurier, came up with an ingenious plan: put the base of the building up on columns, and cantilever the edge of the building over the church. Take a look at it on Google Maps’ Street View — you can pan up, navigate around, and see just how much of a structural challenge this was.

The building was completed in 1977. A year later, LeMessurier got a call from an engineering student studying the Citicorp building. Joe Morgenstern’s excellent 1995 New Yorker article about the building describes it like this:

The student wondered about the columns–there are four–that held the building up. According to his professor, LeMessurier had put them in the wrong place.

“I was very nice to this young man,” LeMessurier recalls. “But I said, ‘Listen, I want you to tell your teacher that he doesn’t know what the hell he’s talking about, because he doesn’t know the problem that had to be solved.’ I promised to call back after my meeting and explain the whole thing.”

Unfortunately, LeMessurier was mistaken, and in the article he describes the problem in all its gory detail. It’s a fascinating story, and I definitely recommend reading it — it’s a great example of how engineering projects can go wrong. It’ll probably seem eerily familiar to most experienced developers: after a project is done, someone uncovers something that seems to be a tiny snag, which turns out to be disastrous and requires a huge amount of rework.

Rework in a building isn’t pretty. In this case, it required a team to go through and weld steel plates over hundreds of bolted joints throughout the building, all over the weekends so nobody would find out and panic.

But what I found especially interesting about the story had to do with testing the building:

On Tuesday morning, August 8th, the public-affairs department of Citibank, Citicorp’s chief subsidiary, put out the long delayed press release. In language as bland as a loan officer’s wardrobe, the three-paragraph document said unnamed “engineers who designed the building” had recommended that “certain of the connections in Citicorp Center’s wind bracing system be strengthened through additional welding.” The engineers, the press release added, “have assured us that there is no danger.” When DeFord expanded on the handout in interviews, he portrayed the bank as a corporate citizen of exemplary caution–”We wear both belts and suspenders here,” he told a reporter for the News–that had decided on the welds as soon as it learned of new data based on dynamic-wind tests conducted at the University of Western Ontario.

There was some truth in all this. During LeMessurier’s recent trip to Canada, one of Alan Davenport’s assistants had mentioned to him that probable wind velocities might be slightly higher, on a statistical basis, than predicted in 1973, during the original tests for Citicorp Center. At the time, LeMessurier viewed this piece of information as one more nail in the coffin of his career, but later, recognizing it as a blessing in disguise, he passed it on to Citicorp as the possible basis of a cover story for the press and for tenants in the building.

Tests were at the center of this whole situation. It turned out that insufficient testing was done at the beginning of the project. Now, more tests were used to figure out how to handle the situation. Tests got them into the situation, and tests got them out.

So what does this have to do with software?

I have a hunch that anyone who’s done a lot of test-driven development will see the relevance pretty quickly. The quality of your software — whether it does its job or fails dramatically — depends on the quality of your tests. It’s easy to think that you’ve done enough testing, but once in a while your tests uncover a serious problem that would be painful — even disastrous — to repair. And as LeMessurier found, it’s easy to run tests that give a false sense of security because they’re based on faulty assumptions.

I’ve had arguments many times over my career with various people about how much testing to do. I can’t say that I’ve always handled them perfectly, but I have found a tactic that works. I point to the software and ask which of the features doesn’t have to work properly. But it’s good to remind myself how easy it is to question the importance of tests. It’s so easy, in fact, that I did it myself earlier today. And that’s why it’s important to have examples like Citicorp Center to remind us of how important testing can be.

Jenny and I are at the tail end of our fourth book, Beautiful Teams, and it’s really coming out well. We’ve put together a team of authors who’ve got some really fascinating stories about building software. Each of them has written a really compelling story from his or her past career. We’ve got people writing about all sorts of industries, and running into (and, in most cases, overcoming) a lot of different kinds of problems. All together, the collection is starting to paint a picture of what it’s like working in software.

One thing that I thought would be hard about this project was the coaching. These are software people, after all, not creative writing people. We have a few veteran authors on the team, but we also have a lot of people who hadn’t really done a lot of writing. And the authors themselves, in a lot of cases, were just as apprehensive as we were. It seemed like most of the people we worked with knew they had a good story to tell, but were worried about whether or not they could really tell it well enough.

It turned out that we didn’t have anything to worry about. I was very impressed with how well everyone “got it”. We put a lot of effort into selecting people who had a good story to tell, and it only took a small amount of guidance to help the authors tell them. Even people who hadn’t really done a lot of writing before really took the time to flesh out the characters, give us all the conflict, and really draw out the stories to make them fun to read.

Another thing that Jenny and I really wanted to do with Beautiful Teams was to put storytelling ahead of teaching. There are plenty of books that will teach you about building software. And we knew that there would be a lot of good lessons about software in the book. But our goal wasn’t a book that you could pick up and suddenly do your job better. The goal was to put you in the shoes of someone who’d been there before. Or, even better, we wanted to put you in the shoes of someone who was in a really interesting situation. Or a really bad, or even unimaginable, situation. We wanted to show you teams that were great to work on, and teams that, despite being awful, managed to muddle through. Or didn’t.

We’d never done anything like this project before, and we when we started it, we weren’t sure if we’d end up with engaging stories, or if we’d just get people writing about their boring projects from work. We were lucky. It turns out that on most memorable software projects, there’s some interesting drama: a bad boss, a deceitful or weird coworker, a serious and last-minute crisis that needs to be dealt with, a loud and unreasonable client. And it’s those very things which made the projects interesting that make the stories interesting to read.

At the beginning of the project, I was worried that people who wrote good software might not necessarily be good authors. But they really took to telling their stories. A lot of the authors told us that it was actually cathartic to get their stories on paper, and most of them really seemed to enjoy doing the project.

To me, the best part of the project is the fact we’re having royalties from the book donated to PlayPumps International. It’s a great charity that came up with a novel way to provide clean drinking water to rural villages and schools in South Africa. If you haven’t heard of them, I highly recommend watching this Frontline segment about them.

I’ll post again once the project is done with details about the stories. (We’re still getting the last ones in, so it’ll have to be a surprise for now!)

I was thinking about that presentation yesterday, after getting off the phone with a manager at a company that wants to hire us to do software estimation training for their programmers. One problem that they’re having is a pretty common one. Their programmers, testers, and even project managers seem reluctant to give estimates. That reminded me of this slide from the talk:

When we get to this slide in the talk, I usually say something like this:

There’s something we call the “about three weeks” problem. Have you ever noticed that when you ask a programmer how long it’ll take to do something, it’s always going to take “about three weeks”? I’ve done it myself many times over the last fifteen years or so. How long to build a simple website to do a few calculations? About three weeks. What about a utility that will automate some file operations and generate a few reports? About three weeks. A new feature for our core product? Sure, it’ll take about three weeks. See, the problem is that three weeks seems like a long enough time to get something significant done. And if you think for thirty seconds about pretty much any programming project, you can do enough hand-waving and ignore enough details so that it’ll seem to fit into about three weeks.

What does this have to do with why programmers are so reluctant to give estimates?

There are many reasons for this, more than I’ll go into here. But a big one might just be because we’ve all quoted “about three weeks” for a programming project that ends up taking a whole lot longer than that, and we never want to be stuck in that situation again. So after we’ve been burned enough, we just stop giving estimates. I was at a job a few years ago, sitting at a full conference table with a dozen developers. The CTO — an abrasive guy who clearly went home every evening to lift weights, and spent most of his day yelling at the people who reported to him — growled an “order” at the team, demanding an estimate. Everyone at the table knew that they’d be yelled at individually, threatened with dismissal, and generally made miserable if they didn’t come up with an estimate. Yet nobody looked up and volunteered anything. Eventually a junior guy in the back cleared his throat, and in almost a whisper he said, “I’m not sure about the rest of it, but I think my piece will take about three weeks.”

And there it is. Nobody wants to go on the record and say how long they think it’ll take to do a job. We all know it’ll take as long as it takes. If the estimate is right, then there’s no great reward or recognition. But if the estimate is wrong, then we’re on the hook for it, and we get to look forward to status meetings where we get to take the blame for whatever terrible consequence happened because the project was late.

So what do we do about it?

Jenny and I put a lot of thought into this problem when we were working on our first book, Applied Software Project Management. It turns out that there’s a really effective way to get a good idea of how long the software will take without putting any one person on the hook, and it’s our favorite way of generating estimates. It’s called Wideband Delphi, and we talk a lot about it in the Estimation chapter in the book (which you can download for free [PDF]). It’s a straightforward technique — it just takes two two-hour meetings to nail down estimates for even a large project. It’s very iterative and highly interactive, which helps the team all come to a consensus and agree on the final result. It’s collaborative, so that no one person is solely responsible — usually, everyone ends up buying into the final numbers. And best of all, it doesn’t require any special expertise beyond what you need to actually do the project.

My favorite part about Wideband Delphi is that it’s really focused on assumptions. That’s another thing I like to talk about in the “Why Projects Fail” talk. If you think that building a particular program is going to take nine weeks, but I think it’s going to take four weeks, we usually aren’t disagreeing on how long it’ll take to do the task. Usually, we’re disagreeing on some basic assumption about the task. For example, you might think that we’ll be building a big GUI with printing support, while I might think that it’s just going to be a console application. That means that we can assume for the sake of the estimate that we’ll either build a GUI or build a console application, and we’ll write down that assumption. That way, if it turns out to be incorrect, we’ll know exactly why the estimate was wrong… and if someone in a meeting later wants to blame us, we can point to that assumption and give a good reason for the delay. (That’s why Delphi has two meetings: the first meeting is mostly taken up with a discussion of those assumptions.)

One nice thing about Delphi is that it’s not some esoteric, theoretical thing. Both Jenny and I have done this in the real world, many times, with all sorts of software teams. Delphi really does work, and it actually does a good job of helping team come up with numbers. And those numbers are pretty accurate, at least on the projects I’ve worked on. If you’re having trouble convincing your reluctant team to come up with an estimate, I definitely recommend giving Delphi a shot.

Remember back in January, when Tim O’Reilly blogged about great Bookscan numbers for Head First C#?

Well, Bookscan numbers don’t always match what you see on Amazon.com. Our intrepid little book has been clawing its way up the Amazon.com sales ranks. It was in the #2 position on the C# bestseller list for the last month. But when I checked a few minutes ago, it was at the top of the list!

This is a huge deal for us — it’s definitely a high-water mark for the book. And what’s really cool is that Head First C# is picking up steam. Its Amazon sales rank was under 500 all day yesterday, which means that it was one of the top 500 books sold by Amazon.com. (Not the top 500 tech books — the top 500 books.) Typically, books sell the most in the first few months, and Head First C# had a strong start out of the gate. But one of our friends at O’Reilly let us know a few days ago that we’re selling better than ever (when you discount the backorders that had to be filled, which always skew the first month’s results).

And Head First PMP is picking up steam, too! It’s been holding onto the #2 spot for PMP exam prep books, and it’s really been gaining over the last few months.

Now, Jenny and I aren’t all about sales. But we are all about helping people learn, and we see these numbers as a real win.

Check out this post on the Head First C# forum. One of our readers, Styrkar, wrote a really neat program that takes any text and turns it into a nifty little animation. He did it as a way to extend the Space Invaders Lab to make a cool animation for the “Game Over” text.

Okay, I just want to say how cool that is. When we wrote Head First C#, one of our goals was to give readers a starting point for their creativity. I figured we’d see a few little modifications to the programs in the book. But seeing stuff like this… well, it’s just really amazing to me. Not only is this a cool program, but the code is really good. It’s readable, efficient, and easy to follow. And it’s even well-encapsulated — it’s good object oriented design! I really hope Styrkar gets the rest of the lab finished — I can’t wait to see how it comes out.

Yesterday, a Slashdot article asked an age-old question:

One of the worst problems [in my company] is a lack of process documentation. All knowledge is passed down via an oral tradition. Someone gets hit by a bus and that knowledge is lost forevermore. Now I know what I’ve seen in the past. There’s the big-binder-of-crap-no-one-reads method, usually used in conjunction with nobody-updates-this-crap-so-it’s-useless-anyway approach. I’ve been hearing good things about company wikis, and mixed reviews about Sharepoint and its intranet capabilities. And yes, I know that this is all a waste of time if there’s no follow-through from management. But assuming that the required support is there, how do you guys do process documentation?

This question seems to come up over and over again. The funny thing is that it almost always leads straight to a long discussion of the technique for gathering process documentation, and then a discussion of mechanism for storing it. That’s the question I think the reader thought he was asking: how to “copy down” the process by looking at how the people on the team build the software and putting it into a “complete” set of documents, and whether to use a wiki, Sharepoint, a version control system or some other repository to hold the docuemnts. And the discussion that followed from that Slashdot post should be pretty familiar to anyone who’s tried to solve this problem in real life. Some people talk about systems to store documents, others talk about the virtues of keeping them up to date, there’s a healthy dose of “write down what you’re currently doing” or suggestions for incident logging, an apparent epidemic of bus drivers who have it in for the one guy who knows how everything in the company works, and lots of talk of cataloging, updating and verifying.

I’ve been through that before, and I’ve bought into many of those ideas in the past. And you know what? It wasn’t particularly useful. I know that in many process circles, that’s a heretical idea. In fact, if you’d told me back in 1999 that it’s not useful, I would have laughed at you. Of course you’re supposed to start by documenting the complete process! How else do you know what you’re improving? There seems to be an unspoken rule that we’re supposed to be striving for a fully documented, constantly improving process. And in some shops, that does make sense. But it’s a very, very hard thing to do, and in practice it’s almost impossible to put in place from the ground up.

This is a really hard thing for a lot of software people to accept. Our nature, as programmer types, is to strive for complete systems. When we build software, we try to handle every possible special case. We’re overly pedantic and literal; it’s like exceptions and missing cases just get under our skin. So when we’re presented with the problem of how to improve the way a team or a company builds software, the first thing we want to do is come up with a system that describes the complete process for building software, mapping out every possible special case and exception that a project might come to. And it makes sense that we’d want to test that process to make sure it’s accurate, correct any changes, and put everything in a repository that gives us complete access to the one true way that we build software.

The problem is that people don’t really work that way. Process engineering suffers from a serious problem: it seems simple when you think about it in the abstract, but once you start trying to document precisely and completely how a team builds software, you run into an enormous number of special cases. Architecture is always finished and signed off before coding begins, right? Oh, wait, except for Joe’s project, where we did 30% of the architecture and started building the code for that while Louise and Bob worked on the next piece of architecture. Oh yeah, and then there’s that project that’s going to be broken into three phases, and we don’t really know how the third part is going to work.

I’ve seen that pattern many times, and it usually plays out in one of two ways. Either you end up with really general documentation that lays out a very general process that’s trivial to follow but doesn’t really provide any useful guidance (like a big chart that shows that testing comes after coding, which comes after design), or you end up with a tangled mess of special cases and alternative paths that seems to get updated every time there’s a new project. Both of those technically fulfill the goal of process documentation — which is great if your job was to document the process. But neither is particularly useful if your goal is to actually build better software.

There’s an easy solution to this problem: don’t document your software process. Or, at least, don’t start out by documenting the complete software process. Instead, take a step back and try to figure out what problems you’re facing. What about your process needs to be fixed? Do you have too many bugs? Do you deliver the software too late? Does your CFO complain that projects are too expensive? Do you deliver a build to your users, only to have them tell you that it looks fine and all, but wasn’t it supposed to do this other thing? Those are all different problems, and they have different solutions.

That’s what Jenny and I teach in our first book, Applied Software Project Management. We call it the “diagnose and fix” approach: first you figure out what problems are plaguing your projects, and then you put in very limited fixes that address the most painful problems that keep you from building better software. People don’t just wake up one day and say, “We’ve got to totally change the way we build software.” They don’t start documenting the software process because things are going just fine. People hate change, and they don’t start making changes to the way they build software unless they have a good reason. So look for that reason, find the pain that hurts the most, and make the smallest change that you can to fix that one problem without rocking the boat. Then find the next most painful thing, and put in the smallest change that you can to fix that. This is something you can keep doing indefinitely, in a way that doesn’t disrupt those parts of your projects that are working just fine. Because the odds are that there are plenty of things that the team is doing right! If it ain’t broke, don’t fix it.

So what about the question of how to actually document the process changes that you do want to make? That’s a very practical problem, and one that we had to handle in our book. After all, we do give you processes for planning, estimating, documenting, building and testing software. And we wanted to do it in a way that was programmer-friendly, with as little cognitive overhead as we possible.

We decided to use process scripts — that’s scripts like an actor reads, not scripts like a shell runs — to describe our processes. We developed these scripts based on use cases (which we talk about in detail in the book). If you take a look at the use case page from the book’s companion website, you can see an example of a use case, followed by a typical script that you’d follow to develop use cases for your project. That particular script is very iterative, because use case development (like many great software practices) should be a highly iterative process. We’ve got examples of many scripts for the various practices and processes: ones for planning projects, reviewing deliverables, and building and testing software.

As for storing these scripts, I’ve used all sorts of ways to do it in the past. I’ve used wikis, version control systems (both Subversion and VSS, depending on what’s in place at the company), even plain old folders full of MS Word documents. The actual mechanics of storing documents aren’t particularly interesting, and are pretty much interchangeable for process documentation. Processes shouldn’t change all that often, because change is very disruptive to a company. The changes should be small, incremental, and easily understood by the team… and the team should agree that they’re useful! Because the biggest problem with process changes — and several posts in the Slashdot thread bring this up — is that they don’t “stick”. But making those changes stick is easy. Just make small changes that the team buys into, and that actually get you to build better software.

That’s easier said than done, of course. Lucky for us, too! Otherwise there wouldn’t be a market for our books or training.

We’ve been keeping ourselves busy! What’s that? You want to know more? Well, certainly. We’ve got lots of news:

• Jenny and I are doing some guest blogging on the Head First Labs website, talking about what it’s like writing a Head First book (and whatever else we feel like talking about). I’ll be doing the posts this week, starting with one called “How We Made Head First C# Different”. I’ll probably get a little more technical near the end of the week — there’s only so much anyone wants to read about writing books. (Or is there?)
• We’ve put up a new training page, because we’ve been getting a lot of questions about training. It’s a list of the various courses we offer on project management and software development. Right now, we’re mainly concentrating on training corporate teams — we’ll go into a company and do a few days of training for a team. We’ve been getting an increasing number of inquiries about putting together classes that are open to the public, though. If you’re interested in that, please drop us a note using our contacts page and we’ll let you know the next time we’re offering one.
• Last week we were invited to do our “Why Projects Fail” talk for the PMINYC Breakfast Roundtable. After the talk, one of the audience members came up to me to thank us for doing a presentation that wasn’t boring. I thought that was pretty cool! Anyway, here are the slides from the talk.
• We’ve been doing our “Why Projects Fail” talk at companies around town. If you work at a company in New York City and want some insight into why projects fail, you’ve got a brown-bag lunch program (or some other kind of program where your company brings speakers in to do a talk), and you can get a reasonably-sized audience together, get in touch with us — we’re usually happy to come in and do it as part of our New York outreach program. It’s generally pretty fun, and a good way to take your mind off of the job for an hour.

Well, we didn’t see this coming. Check out what Tim O’Reilly says in his latest blog post:

I was probably most surprised when I saw Programming WCF Services on our list of top performing books for the year. If you’re steeped in open source, you might never have heard of Windows Communications Foundation, Microsoft’s approach to building SOA applications on Windows. And you might not care. But you’d be making a mistake. Don’t count Microsoft out of the Web services game yet! They still have a brilliant, passionate developer community, and as a company have tremendous resources, persistence, and talent. And now that they have real competition, I expect them to reinvent themselves. (For that matter, Head First C# was the top selling programming language title in Bookscan last week, except for “Javascript: The Definitive Guide”. And C# continues to gain significantly on Java in terms of book sales.)

Wait, what? Let me repeat that for you. Except for a book on Javascript, Head First C# was the top selling programming language title in Bookscan last week. That’s pretty amazing to me, because we only released it at the end of November, so word of mouth hasn’t even gotten a chance to spread yet.

We took a few risks with Head First C# — we took an approach that was somewhat different than any other programming book I’ve seen. I’ve learned at least a dozen and a half programming languages over the years, and I followed the same pattern every time: I thought of a project that I wanted to do, then I figured out how to do the project using the new language. (Like back in 1994 or so, when I wanted to learn Perl and also learn about web servers, so I built a web server in Perl.) But I’m definitely not unique in this approach: pretty much every good programmer I know takes the same approach to learning a new language or technology.

Which is why I thought it was so weird that I couldn’t even find a language learning book that asked you to solve programming problems. I’ve seen a lot of “hands-on” work, where you follow step-by-step instructions to type in code to get a certain result. And certainly, an enterprising person can definitely learn that way. (If they couldn’t, nobody would buy programming books!) And certainly, we have a lot of that in Head First C#. But I just don’t think that’s enough.

I guess I have a hidden agenda here. I’ve led a lot of programming teams over the years. In 2006 and 2007, I spent a lot of time working with a consulting company, and one thing you learn working with consulting teams is that you need to work with a lot of different people. Often, you’ll have at least one very junior member on a team. And what I’ve found is that a lot of really junior developers have the capacity to be good programmers, but don’t have any experience independently solving programming projects on our own. So I decided early on that one goal of Head First C# would be to give people that experience. I wanted to teach them how to solve programming problems, rather than just follow step-by-step instructions. In essence, I wanted to create programmers who I’d be happy to have on my team in the future. (Does that sound too self-serving? I hope not!)

So when Jenny and I started working on Head First C#, it took us a few chapters to figure out exactly how to do that. It was interesting going back to the beginning of the book after we finished, to do a second pass and incorporate comments from our (really great, and really patient) tech reviewers. It really did take us about five or six chapters to work out a good way of assigning problems. We ended up co-opting the “Exercise” element that you’ll find in the other Head First books. The exercises in the other Head First books were generally pencil-and-paper oriented, although I think I remember a few step-by-step programming ones as well. That’s definitely the approach we took in Head First PMP, and it worked really well.

But with Head First C#, we took the “Exercise” element in a totally different direction. We basically used the exercise to give a programming assignment, where we’d give a problem to solve. We’d include a screenshot, define some classes, maybe give a little of the code, and then leave it up to the reader to build working software. It was lucky that I spent so much of my career writing specifications, because I think we really had to push our writing skills to the limit. It’s far too easy to write an exercise that’s ambiguous or hard to follow… and as we found out during the tech review, it’s really demoralizing to run across a poorly-written exercise. And we had to be clear that there’s no single, “correct” solution to the exericse: if it works, then you got it right. I’d be surprised if a single reader comes up with exactly the same solution that we did on any of the projects in the last third of the book.

I hope that interactive approach is what’s paying off. Word of mouth is just starting to get out, and I’m really happy to say that it seems to be positive.

One of my favorite things about Head First C# is that Jenny and I ask our readers to do a whole lot of coding. When we started doing research for the book, one thing that really struck me about other books to help you learn C# was that I couldn’t find any book that actually asked its readers to write much code. There were one or two that we saw which had some written exercises. And that’s great for a textbook, but I don’t know any programmer who learned to program with a pencil and paper. If you want to be a good programmer, you need to write a lot of code. Coding is a skill, and like any other skill it takes practice. And there are a lot of concepts that just don’t “click” until you write software that uses them.

I know that when I need to learn a new language, the first thing I do is come up with a project for myself. Like when I needed to learn Perl back in 1994 or so. I also wanted to learn about how web servers worked (they were newfangled and mysterious at the time), so I wrote a simple web server in Perl. (It turns out that it only takes about 50 lines of Perl to write a very simple HTTP/0.9 web server… probably a lot fewer, if you’re an insane Perl optimizng type.) So we decided from the beginning to make sure Head First C# has plenty of opportunities for our readers to write code. Which meant coming up with projects… a lot of projects.

Which turned out to be harder than it sounds. The instructions for the project needed to be completely self-explanatory. It’s not like an Agile project, where the programmer can talk to the customers and the software can go through several iterations. No, every project needed a complete specification, one that the reader could understand completely and build exactly the software we asked for.

And that’s where video games came in really handy.

When you ask someone to write, say, a business application, you need to explain all of the details of the business. And you need to explain why the program is needed, especially if the programmer doesn’t really have any background in that business. If a programmer doesn’t understand the reason or rationale for a particular feature, then it’s almost certain that he’ll build something other than what you’re asking for. (That’s why I always like including a rationale section when I write use cases. It prevents a lot of problems later in the project.)

Video games come “pre-loaded” with their own rationale. Nobody ever needs to ask why you’d write a video game — you do it because they’re fun. Everyone already knows why you’re writing it. It’s not hard to make a video game intuitive. One easy way to do it is to model it after a game that already exists. We have one exercise where the reader builds a Go Fish! game, where the player plays against two computer opponents. One project is Hide and Seek — the player searches through a virtual house to find an opponent who hid in a random place. We’ve got a dungeon game, a Whack-a-Mole game, and my personal favorite (and the final project in the book), Space Invaders.

There are plenty of other sorts of projects, too. Our goal was to get the readers coding from the very beginning, and keep them writing code through the whole thing. We designed the projects to be satisfying and fun. And we’ve already gotten feedback from readers who definitely enjoyed them.

Head First C# Labs — The Contest 

We’re running a little contest for our readers. We’ve included three labs in the book where the readers build larger projects. We give them a lot of design, but we don’t actually give them the code for the solutions (like we do for all of the smaller projects). Eventually, we want to post executables for the labs. But we don’t want to post our own — we want to post executables that our readers came up with! So we’re running a contest to see which readers can send in executables first. The first person to send us working code for the lab will have their executable posted on the website as the official solution, and we’ll send them some sweet O’Reilly swag, too.  Here’s the contest page — we’re looking forward to seeing what our readers come up with!

HF PMP: how does it fit with Lean Software Development…

Gian Franco asked:

How does PMP fit with practices as explained in Lean Software Development (by Poppendieck) or Agile practices in general?

I wrote a blog post about this a while back called “What About Agile?”

http://www.stellman-greene.com/2007/04/27/what-about-agile/

There are definitely a lot of things that the PMBOK(r) Guide and PMP exam cover that aren’t addressed at all with Agile — like whether a fixed price contract is better or worse for the seller than a cost-plus contract. But there’s one really important way that both the PMBOK(r) and Agile are very similar: they both recognize that managing change is an important part of a successful project. And in that respect, you’ll definitely find that your knowledge of Agile can help you when you study for the PMP exam.

And Gian had a follow-up question:

Well :), let me put it in another way…, In Lean SW development requirements churn is considered to increase costs, I don’t know much about PMP, but suppose it resembles to Prince2 (another comparable projectmanagement method) then the initial phases of specification might fall in this category of specifying too much too early.

The PMBOK(r) Guide was developed so that it can be applied to any kind of project, not just a software project. So projects that fit into its framework tend to develop all of the requirements up front, before any work begins. That’s because all of the plans for the submarine or office building need to be finalized and agreed upon before you hire the construction crew to break ground.

There will be changes that happen. But if those changes involve very large alterations to the blueprints and you need to tear out the last three floors you put in, it gets very, very expensive.

Oddly, the same is true of software in many cases. There are some changes that need to be made, and which you could have found had you “churned” through the requirements a little more before you started building the software, and now you’ve got a bunch of code you need to tear out — and it would have been a lot more efficient to take the time up front to figure out the requirements and then only build your software once. Unfortunately, many people don’t consider that “Agile”. (I think they’re wrong — I think that doing a lot of iteration before you even start writing code can be the most efficient and customer-focused way that you can build software… but that’s a story for another day.)

HF PMP: PMP is Agile!

Darya Akbari asked:

Can one say that PMP is not agile :) ?

That’s an interesting question. But I’d ask the opposite question: can an Agile shop fit into the PMBOK(r) Guide framework?

The reason is that the PMBOK(r) Guide doesn’t define one specific set of doing things. In fact, just the opposite. It says that you need to select a particular methodology based on what you know about your industry and past projects, the specific needs of the project, the deadline and milestones, etc. It doesn’t say that every project needs to include specific things. Instead, it includes things that typically happen on most projects — and practices that are most commonly found. And remember, the PMBOK(r) Guide doesn’t just apply to software: it needs to be general enough so that it can include practices for construction, industrial, civil engineering, electrical and other kinds of projects.

So can a typical Agile process fit into the PMBOK(r) Guide? As far as I can tell, the answer is yes. One hint is that when Jenny and I were working on “Head First PMP”, the PMBOK(r) Guide team members on our technical review team repeatedly stressed iteration and iterative development.

One of the strongest points in the PMBOK(r) Guide (ones that is stressed on the PMP exam) is that it really emphasizes collaboration with the stakeholders, and keeping them in the loop on all important decisions. Another thing that it really stresses is responding to change — and it’s very clear that the customers need to be involved in decisions about change.

There are definitely some things that Agile people might not agree with. It may seem very documentation-heavy, and very concerned with contracts. But the PMBOK(r) Guide was developed in a world where subcontracting is very important, and where a lack of documentation or attention to the contract can mean that the company can get sued and go out of business. I’ve spent a lot of time working in a consulting situation, and even the friendliest clients can turn into adversaries if you don’t have everything documented properly. But if you go back to the Agile manifesto, you’ll see “Individuals and interactions over processes and tools,” “Customer collaboration over contract negotiation,” and “Working software over comprehensive documentation”. While the PMBOK(r) Guide highly stresses individuals and interactions, customer collaboration, and working software (in the form of deliverables that meet the customer’s needs). But it needs to pay attention to processes and tools (since that’s what a framework is made of), contract negotiation (because a process is no good if it puts your company out of business), and comprehensive documentation (because it’s really hard to build a strip mall or highway overpass without it).

HF PMP question: SCRUM?

Rogerio Kioshi asked:

I’ve read about SCRUM. Does PMP have anything to do with this metodology?

The PMP exam won’t have any questions specifically about SCRUM. But if you have a good understanding of SCRUM, then that will give you a good leg up on studying for the PMP exam. The reason for this is that if you’ve spent time thinking about SCRUM, then you probably have a good understanding of a lot of the ideas behind team communication, project schedule constraints, and activity sequencing, and those are core concepts that you need to understand to become PMP certified.

You’ll still have studying to do, though!

Need advice on HOW to start a new Software Product

This isn’t a question about Agile specifically, but it’s still a great question. Paul Michael Laborte asked:

I hope this doesn’t sound too off topic. Anyway here it goes…

I’m currently a software developer with a Monday to Friday job.
Right now, I’m looking for opportunities which would allow me to have a 20% time (pretty much like google).

During that 20%, I’m thinking of developing my own software product.
I would be needing to wear different hats so I really think HFPMP would be of great value.

Aside from the contents of the book, would you have any other tips for entrepreneur aspirants like us

That’s a really interesting question. How do you start out a software project?

Well, if you want to get your project started out right, the first thing you need to do is figure out what it is you want to build. That might sound a little odd — of course you know what to build, right? Otherwise, why would you start a project? But if you look at a lot of projects that went off the rails at one point, one thing that you’ll see over and over again is that many problems can be traced back to the fact that one person wanted to build A, while another person wanted to build B.

There are a lot of tools and practices that can help you with this. My favorite is a Vision & Scope document — mostly because it’s very lightweight, only takes a few minutes to write, and it’s something that anyone can read if they want to learn what it is your project does. Also, it’s something that serves really well as a front page for a wiki or project website, and immediately brings people up to speed on it. Basically, the Vision & Scope tells you very briefly who needs the software, what their needs are, and how you’ll meet those needs (by explaining the features of the software you’ll be developing).

To be perfectly honest, Head First PMP may not really help you as much with this particular problem. But it’s something that Jenny and I wrote a whole lot about in our first book, Applied Software Project Management. We have a whole chapter on starting out a project using a Vision & Scope document.

Other things you need to think about when you’re starting out your project — which we also talk about in depth in Applied Software Project Management — are figuring out how your users are going to interact with the software, setting up a version control system, and doing test-driven development.

I definitely recommend taking an hour or two and really think about how you’ll handle those things. One way I’ve seen a lot of people do this is start a wiki for your project, and have a separate page that says how each of those things will be handled. It doesn’t have to be fancy or anything, and it shouldn’t take long to throw together. But just doing that will really help get your head straight about them, and set your project in the right direction from the beginning.

If you’ve got a burning question and you’re wondering what we think about it, send it to us using our “Contact Us” page! We love answering questions from our readers.

We had a great time presenting at a NYC SPIN meeting on Tuesday — it was a joint gig with the PMINYC chapter, and we had a great crowd. We got a lot of great questions, which is always great. As promised, here’s a PDF of the slides. If anyone’s got any questions, feel free to contact us.