Q&A

Iterative development is not unplanned development

~ Andrew Stellman ~ 1 Comment

What the...

I got a great question from a software developer who also happens to be a fellow CMU alum.

I have a question related to managing scope creep with respect to “on-going”/iterative development processes.

I’m currently managing a project where we’re redesigning my application’s primary workflow. Simply put, the app is currently designed to have users to signoff all items and we’re redesigning it to be exception-based (only require certain items to be signed off).

As we’ve progressed down the path of planned iterative development, a lot of good/new ideas for future enhancements/requirements spring up. I find myself regularly working with my users and “working group” to prioritize and analyze if any of these new ideas need to be considered to build sooner rather than later (and thus triggering plan adjustments or delays).

I often feel like I’ll end up delivering a product that does deliver the initial vision, but still doesn’t make my users happy, as they’ve already shifted their expectations to wanted the “next” thing (aka phase 2).

Do you have any other tips about how to manage this process?

I’ve used things like taking a timeline-style roadmap and adjusting it by overlaying the new requirements and shifting the timeline out. Do you have an recommendations of ways to present this type of information?

Curious your thoughts. Thanks.

— Seth L.

Iterative software development can be a really useful—and highly effective—tool for software development, but it’s also one of the most abused tools I’ve seen. Even as recently as a few days ago, someone in charge of a software team that’s critical to his comapny came to me with the (incorrect) assertion that iteration just means diving into a prototype without talking to anyone or doing any investigation. Iteration done well can lead to very high quality software. But as Seth saw, iteration done poorly can lead to scope creep and serious planning issues.

Making your users “happy” means managing their expectations. They need to see exactly what you’re working on, and what’s coming next. If all people see is deadlines and they don’t have a sense of what’s going on, then they naturally start looking to the next deadline, because that’s all they see. The more visibility you can give them into the way you build software, the more understanding they’ll have – that’s just human nature.

There are a few things that work well for improving visibility into your project’s iterations. One of them is a task board, which typically involves sticking index cards with your user stories or scenarios on a whiteboard or wall. This means that you actually need to have user stories or scenarios. Scope creep is often an indication of a requirements problem, and getting consensus on at least the scenario level about exactly what’s going into the iteration. Having the index cards arrayed on a task board, with each index card showing the status (“planned”, “in development”, “completed”, “out of scope”) gives a lot more visibility into exactly what you’re building and how you’re building it. In a lot of ways, this is a sort of iterative development project plan.

Another way to prevent iteration-related scope problems is committing to delivering releasable code at the end of each iteration. Test-driven development (or, at least, developing complete unit tests) and continuous integration are effective ways to help make that happen. If your stakeholders are comfortable that you’ll deliver high-quality code at each iteration, they’ll feel less pressure to get the new features in immediately, and will be more willing to wait until the next iteration.

If this sounds like something that might help you, I definitely recommend reading the interview Jenny and I did with Mike Cohn for Beautiful Teams, who knows more about agile and iterative planning than pretty much everyone . He has a lot to say about effective iteration planning. There are pictures of task boards he’s used in the past.

That gets me back to the basic idea—one which I give Mike a whole lot of credit for helping people understand—that iterative development, especially in an Agile project, works best when we take the time to plan each iteration. It still faces the same problems: requirements need to be discovered, scope needs to be controlled, and progress needs to be communicated to everyone who cares about it… especially to anyone who can potentially make the developers’ lives more difficult. That’s why the most successful Agile development projects collect requirements and document them using user stories (or other techniques for writing down what the software needs to do). They plan their progress using task boards, forecast them using calculations and charts like project velocity and burn down rates, and constantly keep any business owners and other stakeholders up to date on their progress.

It’s a great way to develop software, and it’s been really effective for a lot of teams. And I think it shows that iterative development does not necessarily mean unplanned development.

When I sent this to the developer who sent me the question, he had an interesting follow-up, which I thought deserved a response:

So I’ve been digesting this a bit and I am curious to get your thoughts about adapting project management fundamentals into the often fluid process of app management. I manage a few virtual projects within my company and at times have struggled to keep things focused as business demands and/or interests shift over time. Similarly, the “iterative” approach has helped to clarify requirements while building out new flows/apps, but as you pointed out can be very tricky to get “right”.

It’s funny how often I hear people say, “Well, this project management stuff works in theory, but my project is fluid” (or “changing,” or “under too much pressure from the business,” or “critical,” etc.). It turns out that pretty much every project is challenging, and project management is set up specifically to deal with that kind of challenging project.

Here are two thoughts I had relating to this idea.

First, the iterative development model works very well, as long as you’re committed to delivering a high-quality product at the end of each iteration. Whether or not you develop using an iterative approach, you need to manage change: prevent unnecessary changes, and make sure you understand the impact of any change that you make. It also means that you need some sort of scope baseline, so that you know what is and is not a change. It’s faster and easier to update software on paper, before it’s written into code, so the more changes you can move to the “write it down and review it” phase of your project, the better.

And second, if your business is overly demanding, it often means that you could manage your stakeholders’ expectations better. Make sure you identify them – and write down their names and needs! – from the very beginning of the project. Talk to them… a lot. Make sure they’re in the loop. If possible, see them so often that they’re sick of seeing you. If they’re always aware of what you’re doing, there won’t be nearly as many surprises. Also, the more your stakeholders understand the details of the work that you’re doing, the more slack they’ll cut you when they ask you for changes. Often, when someone puts a lot of pressure on you to do the impossible, it’s because they don’t realize that’s what they’re asking.

Check out our O’Reilly Week in Review podcast interview

~ Andrew Stellman ~ Leave a comment

Edison Phonograph

James Turner‘s weekly O’Reilly Week in Review podcast this week features an interview with Jenny and me about Beautiful Teams, and what goes into making a team work well.

I’ll transcribe a quick excerpt from the interview – we’re talking about our interview in the book with NASA manager and team leader Peter Glück:

Jenny: You hear all your life about what it’s like to work at NASA. There were so many times we’d want to put in place a practice, and you’d hear, “Well, this isn’t NASA”. So to hear about how NASA actually did stuff was really exciting. It was such a revelation that they pretty much do stuff like everybody else, you know?

Andrew: I kind of had a feeling of, “They put their pants on one leg at a time, just like everyone else.” They build the same way, they test the same way, it’s just that they put a whole lot more time, money and effort into testing. And there are some things they can’t to. We talked about continuous integration. I asked him, “Do you do continuous integration?” Well, no, they don’t do continuous integration, because that involves integrating with a hardware platform that they have to take into a clean room. If your inegration process involves a clean room, it probably can’t be automated.

If you want a little background about what goes into building an O’Reilly book, working with the folks there, and some of the thought process behind Beautiful Teams, definitely have a listen. You can hear the whole interview here!

Bringing a “teamwork feel” to your projects

~ Andrew Stellman ~ Leave a comment

When cheering fails

Jenny and I have been thinking a lot about teams lately. Working on Beautiful Teams really focused us on teamwork: what makes teams gel, what causes them to run into trouble, and what you can do to them. So when I got this question recently, it was really timely:

I’ve been working on bringing more of a teamwork feel to our projects for 2 years.  It has proven to be a non-trivial task. Any advice?

Now, just so we’re clear here, I know this guy pretty well. He’s a very good manager, and he runs a software development group with some of the smartest people I’ve met professionally.  They’re definitely a functioning group: they’ve been building software for years, and that software is used in production in critical applications at his company. By many accounts, his group is successful.

But when I took a closer look at how that particular team works day-to-day – and this should seem very familiar to a lot of developers – I definitely saw what he meant about it being a non-trivial task. There’s definitely a lack of a teamwork feel on the various projects his team works on. Everyone is really good at carving out his or her own piece of work, and is able to work on that piece independently. But even when multiple people would work on a single release of the same piece of software, they’d work independently of each other, not really talking amongst themselves. Don’t get me wrong – they definitely get their jobs done. But they’re working on individual tasks, not team projects. And even when those tasks add up to projects, it just feels like loosely connected tasks to the people working on them.

So what’s the harm in that, if they’re getting the work done? Well, for one, having a group of people working like that doesn’t exactly encourage innovation. There are some genuinely brilliant people on that team, and they do reasonably good work, but I would honestly expect more innovative work from this particular group of people. And I believe that having more of a “teamwork feel” would encourage more innovative work.

What’s more, I got the feeling that this particular team still runs into a lot of the same problems that many other, less talented teams run into, and I think those problems could be improved through better teamwork. They have some serious change management issues, where they often have to rework their code because it doesn’t quite meet the needs of the projects’ users. They have a few code quality problems – they’re trying to address them with better code reviews, which they’re having trouble getting off the ground. And they have an overall client relationship problem, where the people they build software for often feel like they’re somewhat disconnected, like they can’t quite the software that really meets their needs. Again, these are highly talented, very smart developers, with a manager who’s on the ball – and they’re still getting stuck on the same problems that hit everyone else.

Now, I don’t want to make the situation sound worse than it is. The manager has made some very good progress in getting the individuals working in a more team-like way. One thing he did that I really like (and I think it worked really well) was to really encourage everyone to write things down in the Wiki. And this wasn’t just a general edict (“Each person must add X wiki pages, or else!”). It was actual encouragement and support, treating documentation like it was a real and highly appreciated part of peoples’ jobs. Like a lot of new tools, there was a slow start to it, but once a few people dove in and started to embrace it, it started to really take off. And a funny thing happened: as people started sharing information with each other, they started to learn more about each others’ work, and some of them started to get that “teamwork feel” that they were looking for.

So what’s going on with this person’s group? And, more importantly, what can he do to help improve the “teamwork feel” for everyone?

Whenever I’m trying to help a group of effective but disconnected developers become a team, I like to start with this great analogy from Watts Humphrey:

There are different kinds of teams. In sports, for example, a basketball team’s positions are dynamic while baseball team members have more static roles. However, in both cases the members must all work together cooperatively. Conversely, wrestling and track teams are composed of individual competitors who do not dynamically interact, although the members support each other socially and emotionally.

In engineering, development teams often behave much like baseball or basketball teams. Even though they may have multiple specialties, all the members work toward a single objective. However, on systems maintenance and enhancement teams, the engineers often work relatively independently, much like wrestling and track teams.

A team is more than just a group of people who happen to work together. Teamwork takes practice and it involves special skills. Teams require common processes; they need agreed-upon goals; and they need effective guidance and leadership.

When I reread that quote before pasting it in here, something in that last sentence stood out to me: they need agreed-upon goals. Now, I’ve been referencing that quote for almost a decade. But it’s really interesting to me that I always overlooked that last bit. But working on Beautiful Teams really taught me the value of agreed-upon goals. It was pretty remarkable, actually: contributor after contributor brought up the idea that the people on a project all need to be aligned to the project’s goals, and to have a single, overarching vision that the team can rally around.

So if I had to give a single piece of advice to this manager, it would be this: try to find ways to give everyone a good sense of their goals. Two different Beautiful Teams contributors – Mike Cohn and Steve McConnell – talked about helping the team see an “elevating” goal. I really like how Steve put it:

First, the team leader needs to establish an elevating vision or goal, and I think that this is truly a leadership function, not just management. It goes beyond the management function. An awful lot of thought should go in on the part of the team leaders into what is the team’s goal and what is it about it that is not just trying to trick the team into thinking they have a goal that’s elevating or inspiring, but to come up with one that truly is.

A classic example is if you’re out digging ditches, that’s not very elevating or inspiring. But if you’re digging ditches to protect your town that’s about to be attacked by an enemy, well, that’s more inspiring, even though it’s the same activity. And so the leader’s job, really, is to try to determine or frame the activity in such a way that people can understand what the value is.

Now, that’s not an easy thing to do. But Jenny and I heard this message (or one similar to it) from so many of our Beautiful Teams contributors – Keoki Andrus, Neil Siegel and Mark Healey all come to mind, although other people talked about it too – that it actually changed the way I think about how teams work. In the past, I always looked at defining a project’s vision and scope as a straightforward tool for setting the top-level scope of the project and preventing changes. One thing I learned was that I wasn’t giving “vision” part of it nearly enough respect. Talking openly and often about that vision – and coming back to it when you do things like discuss project assumptions, come up with scenarios (or user stories, or use cases… however you figure out how your users will use the software) and plan out the work the team’s going to do – is the first step in really getting your team to have that “teamwork feel.”