Agile

Agile testing and understanding change

~ Andrew Stellman ~ 1 Comment

What the...

Tomorrow at the Agile 2009 conference, Abby Fichtner and Nate Oster are doing a workshop called Where Does Developer Testing End and Tester Testing Begin?. Jenny and I hope you can make it, because they’ll be doing a giveaway of autographed copies Beautiful Teams. Check out my O’Reilly Community posts for more information:

In that second post, I spend a little time talking about some of the reasons that programmers resist great practices like test-driven development. Writing that post reminded me of something that Jenny and I wrote about change in Applied Software Project Management:

Many project managers—especially ones who have a technical background—tend to ignore the fact that their organizations are made up of people who need to be convinced of the importance of a change before they will adopt it. Some of these people will have an emotional or even irrational response to any attempt at change; it could take a sea change in the organization before they agree to it.

Irrational attitudes about software development usually boil down to two basic beliefs. First, people believe that most or all software projects are delivered late and delivered with many bugs, and that this is just a fact of life. Second, they believe that their organization is unique, and that the problems they are experiencing are particular to their organization and have never been seen before in any other organization.

(This second belief may seem odd, considering the many thousands of software organizations around the world that have all used similar tools and techniques to fix very similar problems and make real, lasting improvements. It’s possible that the belief in uniqueness comes from the fact that the software being built truly is unique, in that it has never been built before; it’s not a leap to assume—incorrectly—that the software project and all of itsproblems are therefore also unique to that particular organization.)

Many times, resistance is not irrational at all. Anyone who has been through a change previously—possibly a passing management fad—that didn’t fix the problem (or failed outright) may be resistant to another change. It may seem unfair, but if people in your organization have previously gone through poorly planned changes, it will be harder for you to make changes of your own.

When you are introducing new tools, techniques, or practices in your organization, you may encounter resistance for a number of reasons. By exploring the feelings, fears, and justifications for resisting change that project managers commonly encounter, these reactions can be unraveled and understood.

— Stellman & Greene, Applied Software Project Management, p206 (O’Reilly 2005)

It’s really easy to forget about this when you’re pushing for a change, especially something that requires extra work and learning. (I had to learn that the hard way – hopefully you won’t have to!)

Iterative development is not unplanned development

~ Andrew Stellman ~ 1 Comment

What the...

I got a great question from a software developer who also happens to be a fellow CMU alum.

I have a question related to managing scope creep with respect to “on-going”/iterative development processes.

I’m currently managing a project where we’re redesigning my application’s primary workflow. Simply put, the app is currently designed to have users to signoff all items and we’re redesigning it to be exception-based (only require certain items to be signed off).

As we’ve progressed down the path of planned iterative development, a lot of good/new ideas for future enhancements/requirements spring up. I find myself regularly working with my users and “working group” to prioritize and analyze if any of these new ideas need to be considered to build sooner rather than later (and thus triggering plan adjustments or delays).

I often feel like I’ll end up delivering a product that does deliver the initial vision, but still doesn’t make my users happy, as they’ve already shifted their expectations to wanted the “next” thing (aka phase 2).

Do you have any other tips about how to manage this process?

I’ve used things like taking a timeline-style roadmap and adjusting it by overlaying the new requirements and shifting the timeline out. Do you have an recommendations of ways to present this type of information?

Curious your thoughts. Thanks.

— Seth L.

Iterative software development can be a really useful—and highly effective—tool for software development, but it’s also one of the most abused tools I’ve seen. Even as recently as a few days ago, someone in charge of a software team that’s critical to his comapny came to me with the (incorrect) assertion that iteration just means diving into a prototype without talking to anyone or doing any investigation. Iteration done well can lead to very high quality software. But as Seth saw, iteration done poorly can lead to scope creep and serious planning issues.

Making your users “happy” means managing their expectations. They need to see exactly what you’re working on, and what’s coming next. If all people see is deadlines and they don’t have a sense of what’s going on, then they naturally start looking to the next deadline, because that’s all they see. The more visibility you can give them into the way you build software, the more understanding they’ll have – that’s just human nature.

There are a few things that work well for improving visibility into your project’s iterations. One of them is a task board, which typically involves sticking index cards with your user stories or scenarios on a whiteboard or wall. This means that you actually need to have user stories or scenarios. Scope creep is often an indication of a requirements problem, and getting consensus on at least the scenario level about exactly what’s going into the iteration. Having the index cards arrayed on a task board, with each index card showing the status (“planned”, “in development”, “completed”, “out of scope”) gives a lot more visibility into exactly what you’re building and how you’re building it. In a lot of ways, this is a sort of iterative development project plan.

Another way to prevent iteration-related scope problems is committing to delivering releasable code at the end of each iteration. Test-driven development (or, at least, developing complete unit tests) and continuous integration are effective ways to help make that happen. If your stakeholders are comfortable that you’ll deliver high-quality code at each iteration, they’ll feel less pressure to get the new features in immediately, and will be more willing to wait until the next iteration.

If this sounds like something that might help you, I definitely recommend reading the interview Jenny and I did with Mike Cohn for Beautiful Teams, who knows more about agile and iterative planning than pretty much everyone . He has a lot to say about effective iteration planning. There are pictures of task boards he’s used in the past.

That gets me back to the basic idea—one which I give Mike a whole lot of credit for helping people understand—that iterative development, especially in an Agile project, works best when we take the time to plan each iteration. It still faces the same problems: requirements need to be discovered, scope needs to be controlled, and progress needs to be communicated to everyone who cares about it… especially to anyone who can potentially make the developers’ lives more difficult. That’s why the most successful Agile development projects collect requirements and document them using user stories (or other techniques for writing down what the software needs to do). They plan their progress using task boards, forecast them using calculations and charts like project velocity and burn down rates, and constantly keep any business owners and other stakeholders up to date on their progress.

It’s a great way to develop software, and it’s been really effective for a lot of teams. And I think it shows that iterative development does not necessarily mean unplanned development.

When I sent this to the developer who sent me the question, he had an interesting follow-up, which I thought deserved a response:

So I’ve been digesting this a bit and I am curious to get your thoughts about adapting project management fundamentals into the often fluid process of app management. I manage a few virtual projects within my company and at times have struggled to keep things focused as business demands and/or interests shift over time. Similarly, the “iterative” approach has helped to clarify requirements while building out new flows/apps, but as you pointed out can be very tricky to get “right”.

It’s funny how often I hear people say, “Well, this project management stuff works in theory, but my project is fluid” (or “changing,” or “under too much pressure from the business,” or “critical,” etc.). It turns out that pretty much every project is challenging, and project management is set up specifically to deal with that kind of challenging project.

Here are two thoughts I had relating to this idea.

First, the iterative development model works very well, as long as you’re committed to delivering a high-quality product at the end of each iteration. Whether or not you develop using an iterative approach, you need to manage change: prevent unnecessary changes, and make sure you understand the impact of any change that you make. It also means that you need some sort of scope baseline, so that you know what is and is not a change. It’s faster and easier to update software on paper, before it’s written into code, so the more changes you can move to the “write it down and review it” phase of your project, the better.

And second, if your business is overly demanding, it often means that you could manage your stakeholders’ expectations better. Make sure you identify them – and write down their names and needs! – from the very beginning of the project. Talk to them… a lot. Make sure they’re in the loop. If possible, see them so often that they’re sick of seeing you. If they’re always aware of what you’re doing, there won’t be nearly as many surprises. Also, the more your stakeholders understand the details of the work that you’re doing, the more slack they’ll cut you when they ask you for changes. Often, when someone puts a lot of pressure on you to do the impossible, it’s because they don’t realize that’s what they’re asking.

Requirements 101: User Stories vs. Use Cases

~ Andrew Stellman ~ 23 Comments

Business Analyst

Here’s a question that I get over and over again:

What’s the difference between user stories and use cases?

— Ron K.

Before I dive into an answer to that question, let’s rewind a little bit and talk about where user stories came from. I like them because they’re a great example of how the Agile movement changed the software world. Programmers used to just dive right into software projects and start coding. Whenever one of those pesky users started to tell us what they needed, we’d stop them and say something like, “Don’t worry, I totally get it. I know what you need.” The Agile folks figured out that “I know what you need” is a nasty little trap that programmers — especially good ones — fall into. We’d spend the whole project thinking that we understood our users’ needs, only to deliver software that they didn’t want. The Agile folks realized that if developers had to start working with users throughout the project to understand their needs if they wanted to avoid the code-and-fix trap.

Jenny and I teach you all about use cases and requirements in our first book, Applied Software Project Management (O'Reilly, 2005).

And that’s why I think the user story is one of the most useful tools to come out of the Agile movement. A user story — some people call it a scenario — expresses one very specific need that a user has. It’s usually written out as a couple of sentences. Most user stories are written in the language of the users, so any user should be able to read a user story and immediately understand what it means. A lot of time, user stories are written on index cards, although I’ve put them in Word documents, Excel spreadsheets and Wiki pages (depending on how the particular project is run).

A use case is similar to a user story, because it also describes one specific interaction between the user and the software. When I’m training people to improve the way they write down their project’s requirements, I often describe the use case as a “deceptively simple tool” that helps us find and write down all of the ways users interact with the software.

Looking at those definitions, I can definitely see why there’s confusion about the difference between user stories and use cases. If you look at the last two paragraphs, it might sound like I said the same thing twice! But while user stories and use cases are definitely similar, there are important differences between them. Each serves a distinct purpose, and I think they both have their place on a well-run software project.

I think the easiest way to understand the difference between use cases and user stories is to take a look at an example. Luckily, I’ve got one that I think helps make the difference clearer.

In our first book, Applied Software Project Management, Jenny and I spend a lot of time talking about how to develop use cases and use them to build better software. And as an example, we showed a use case for a software feature that everyone should be familiar with: a search and replace feature from a word processor. Comparing a user story for search and replace with a use case for the same feature helps highlight the differences.

It’s not hard to find lots of user story examples. There are lots of different ways you’ll see a user story formatted (although if you’re looking for a user story template, a 3×5 index card should be a good starting point!). So what would a user story for search and replace look like? I took a stab at writing one:

search-and-replace-user-story-card

(One thing I like to do with user stories is to use “he” or “she”, rather than try to be gender-neutral. I think this makes the user in the story easier to connect with by personifying him a bit. It it also lets me write in a more conversational tone, which makes the user story friendlier and, I think, a bit easier to read and understand.)

Now, if you’re not familiar with user stories, you might think to yourself, “Wait a minute, my word processor’s search and replace feature does a lot more than that!” And that’s okay. A typical user story will have enough information to help the user understand what it is the software needs to accomplish, but it’s not meant to be a complete description of how the software works. I’m not going to try to give a long lesson in writing effective user stories here; I highly recommend reading Mike Cohn’s excellent articles and posts aboout user stories. (Mike, incidentally, is one of the software development veterans who contributed to our latest book, Beautiful Teams [O’Reilly, 2009]. He has some really fascinating things to say about Agile planning.)

So what would a use case sample look like for search and replace? Here’s the use case example Jenny and I built to demonstrate how use cases work:

Name UC-8: Search and Replace
Summary All occurrences of a search term are replaced with replacement text.
Rationale While editing a document, many users find that there is text somewhere in the file being edited that needs to be replaced, but searching for it manually by looking through the entire document is time-consuming and ineffective. The search-and-replace function allows the user to find it automatically and replace it with specified text. Sometimes this term is repeated in many places and needs to be replaced. At other times, only the first occurrence should be replaced. The user may also wish to simply find the location of that text without replacing it.
Users All users
Preconditions A document is loaded and being edited.
Basic Course of Events
  1. The user indicates that the software is to perform a search-and-replace in the document.
  2. The software responds by requesting the search term and the replacement text.
  3. The user inputs the search term and replacement text and indicates that all occurrences are to be replaced.
  4. The software replaces all occurrences of the search term with the replacement text.
Alternative Paths
  1. In Step 3, the user indicates that only the first occurrence is to be replaced. In this case, the software finds the first occurrence of the search term in the document being edited and replaces it with the replacement text. The postcondition state is identical, except only the first occurrence is replaced, and the replacement text is highlighted.
  2. In Step 3, the user indicates that the software is only to search and not replace, and does not specify replacement text. In this case, the software highlights the first occurrence of the search term and the use case ends.
  3. The user may decide to abort the search-and-replace operation at any time during Steps 1, 2, or 3. In this case, the software returns to the precondition state.
Postconditions All occurrences of the search term have been replaced with the replacement text.

Now, if I were a developer building a word processor or text editor, I’d actually be able to write a search and replace feature that implements that particular use case. (Just to be clear: there are many different use case formats; Jenny and I use this use case template in our book because it’s stripped down to the bare minimum sections that we think an effective use case should have.)

Here’s something about use cases that I think is interesting. While you were reading through our use case example, were you thinking of something that looks like the Replace dialog in Notepad or Microsoft Word, or the Find dialog in TextEdit? If so, take another look at the sample use case. It doesn’t have any words like “window,” “button,” “click,” “field” or “checkbox”. It’s all about what actions the user takes, and how the software responds. And there are many different ways that you could build software that implements the use case. Have you ever used the search and replace feature in vi? What about the search and replace feature in Emacs? They have very different user interfaces! Who knew there were so many ways you could implement search and replace? But if you compare each of them with this use case, they all follow the same basic course of events.

So now that we’ve gone through the use case and user story examples, what’s the difference between user stories and use cases? Here’s what I think are some of the key differences:

  • User stories are about needs. When you write a user story, what you’re describing is a “raw” user need. It’s something that the user needs to do in his day-to-day job. If you never build any software for him, then that need will still exist!
  • Use cases are about the behavior you’ll build into the software to meet those needs. A developer who needs to build working software should be able to read a use case and get a good sense of what the software needs to do. It typically has a lot of detail, and describes everything that the developer needs to build in order to meet the user’s need. That’s why it needs to have a lot more detail, and be clear and unambiguous.
  • User stories are easy for users to read. When you write a user story, what you’re concentrating on is writing something that anyone can understand, in the language of the users. We all know that developers have a lot more patience for talking about details of the software they’re building than users do, which is why user stories have to be brief. A user story needs to express a complete thought in just a couple of sentences. (That’s also why it’s good to put them on index cards: somehow, that makes it clearer that it’s self-contained and independent of the other user stories.)
  • Use cases describe a complete interaction between the software and users (and possibly other systems). When you’re doing use case analysis, what you’re doing is designing a functional solution that meets the users’ needs. It needs to be something that developers can implement. It’s possible that one user story could spawn several use cases. And when you combine all of your use cases into one use case document, you’ll end up with a complete description of every interaction between the user and the software that you’re planning on building. And if your software has to interact with multiple systems, you may end up treating those other systems as actors in your use case.

Once you get a sense of how user stories and use cases differ, you can start to see what purpose they can serve on your project. And if you only use user stories, or if you only use cases, then maybe on your next project you can try using them both.