| STAQS | Software Testing and Quality Services |
| [ Docs ] > Article |
Background info: This article was originally written for a magazine in February 2006, however, the publisher ran into difficulties so it went unpublished. The issue's theme was "Learning Testing".
Until fairly recently, many people who called themselves Software Testers came by their jobs by accident rather than by design. This placed many of us in the difficult position of having to learn how to do our job while we were trying to do that job. One of my biggest challenges was trying to figure out what goes into a Test Plan.
In the first company that I worked for, the answer was straightforward enough. A Test Plan was simply a commonly-grouped set of test cases that was executed by a Tester. For example, there was a Print Test Plan, a GUI Test Plan, a Math Test Plan, and so on. No problem.
Doubt in that definition began to creep into my mind when I started to
network with other Testers in town and found out that Test Plans meant different
things to different people. When I got my first Team Lead role in the next
company I worked for, I discovered that the Test Plan I was being asked to
produce was something completely different. It was supposed to be a
planning document of some kind.
I suddenly found myself in the uncomfortable position of not really knowing something that I thought I already understood. What do I do now? Where do I start?
I began by asking people I worked with. I networked a bit and asked other testers for advice, but most of the time no one would say much because they thought it was confidential to share information like that. I even went to a day-long workshop on "Managing Systems Testing Projects". While I found the workshop to be very insightful, I didn't believe everything the instructor had said because a lot of it went against what I had learned so far on my own. For example, a Test Plan was not supposed to have any test cases in it. This just increased my doubt and anxiety.
Back at my desk, faced with an empty Microsoft Word document in front of me, I searched the Internet for Test Plan templates and examples. I needed more opinions and references than just my word against my workshop notes. It was at that time that I stumbled upon a few mailing lists focussed on Software Testing and Quality Assurance. People on those lists were very friendly and helpful and in no time I had several examples to help me develop my first test plans.
The biggest hurdle I had to overcome was acknowledging that my prior
experience and knowledge were based upon someone else's incomplete knowledge.
Until that moment, I thought that I knew what a test plan was. Then how
was it that the definition changed when I changed companies? I had to
unlearn the ways of the Dark Side.
Many of the examples and templates that I had were very similar to each other with some minor variations between them. In general, I found that a Test Plan contained elements like the following:
I became quite proficient at creating these Test Plan documents and customising them for the needs for each project. I even became familiar with some industry standards for Test Documentation1.
Then I started to notice an interesting pattern emerge with the more projects I worked on. For one thing, no one from any other department ever read any of these documents, even when they asked me to produce one. For another thing, no matter how I modified this document, I never really found it useful in the day-to-day management of the test projects I worked on. Finally, creating and maintaining the test documentation always seemed to take a significant amount of time and effort away from the actual testing activity itself.
Naturally I tried to find ways to improve the efficiency of creating and maintaining these documents, but after a while I just started to question the usefulness of the document entirely. One day I realised that I had fallen into the trap of forgetting to question why I even had to create one. This time when I asked the question the response disturbed me: "Because it is expected that you create this documentation."
Wait a minute! Hold the phone! Who
expects me to create this documentation? Why do I have to create
this documentation? Where's the value in that? I'm the one doing the
job here and I'm seeing that I'm wasting my time creating and maintaining
documents and procedures that don't actually help me do my job any better.
Where's the sense in that?
By sheer coincidence, it was at that moment that I attended a Project Management workshop. I had requested it because I was struggling to manage several different test groups and projects simultaneously. An interesting thing happened after attending that workshop: I realised that I never wanted to create a Test Plan again.
During the course I learned about the Project Management Institute2, we covered the Project Management Body of Knowledge (PMBOK), and worked through several examples in order to understand how to apply the correct knowledge, tools and techniques in order to successfully manage a project to completion. I highly recommend such a course for any Test Manager. One of the most significant aspects of the course for me was learning about the elements of a Project Work Plan:
Wow. That was it! I finally understood what it was that a Test
Plan must have originally tried to accomplish. At some time, before
Project Management became a formally recognised role in a Software Development
organisation, it must have been commonplace for each of the teams to be required
to produce their own little mini project work plans. Since that need
doesn't exist anymore, it was time to get with the program and stop wasting my
time creating test documentation that clearly no one needed anymore.
When you stop creating Test Plans, you run into all kinds of resistance, especially from Management. There are some good reasons to create extra documentation and there are many bad reasons. To be clear, "Because it's expected" is not a good reason.
One valid reason for producing some documentation is to help you learn from what you did on a test project. Being faced with all of the missed bugs now being reported by Customers through the Support department after a product ships is a harsh wake-up call. If you don't have a record of the who, what, where, when, how, or why you did or didn't test certain things, it is more difficult for you to learn how to improve and reduce the number of escaped bugs in the next project.
It is important to note that a Project Plan doesn't completely replace a Test Plan though. While the project plan may outline all the players, the general work breakdown structure and so on, there is at least one thing not captured to any useful detail by a Project Manager. One of the single most important things that a Test Team needs to own on any project is the Test Strategy.
It took me a few years to become reasonably good at developing and
implementing an effective Test Strategy. I think it is a key skill for any
Tester or Test Manager. My learning path involved "going back to square
one" so that I could really know what I needed to be effective.
Nowadays there are some good references and workshops that can help jumpstart
you to reasonable success.
In the end, it comes down to this: you will never be able to test everything; you will never find all the bugs; and you will never have as much time, people, or resources as you really want. Starting with that in mind, your Test Strategy needs to help you work with what you have, in the time you have it, to do the best job you possibly can.
For me, the best way to accomplish this kind of "plan" was to use a Risk-Based Testing approach3. A few authors in particular had the most influence on me during my on-the-job education: James Bach4, Cem Kaner5, and Ross Collard. It was at that time that I also came to learn about the Context-Driven School of Software Testing.6
Context-Driven Testing was an interesting paradigm shift that helped me to understand the last piece of the test management puzzle: why do certain test practices or approaches seem to be successful sometimes but not always? Because that's the way it is. Success depends on a great number of factors that change with almost every project. To be a successful Test Manager requires that you are a keen observer of these influencing factors and can adapt your team's dynamics and approaches to suit the needs of your current situation. Darwin had it right when he said that things in nature adapt or die. We should too.
Armed with this new knowledge, I am now quite comfortable developing Test Strategies tailored to suit the needs of each project that I work on. I take no offence when I am asked to produce a "Test Plan". Instead of providing a document that someone else thinks I need, I create something that I find useful – a usable, working Test Strategy. Instead of worrying about templates, cover pages, and document control systems, I worry about Quality Criteria, Technical Risks, and Test Techniques.
A typical Test Strategy document of mine is about one or two pages in length, and I don't worry about formatting or templates anymore. It sits on a network drive that is accessible to the whole development team, and we update our strategy as the risks and priorities change throughout the development phase. This is Agile Development. Things change and we need to adapt our testing strategy along with the new information as it arises, whenever it arises.
The next time you are asked to provide a "Test Plan", remember to focus more on the "Test" than the "Plan". If you find you are putting too much effort on the latter, you should take a serious look at the value that the documentation effort is providing your organisation. Time is money, and the last time I checked, I'm getting paid to test.
[pc]
1. An example of an Industry Standard for Test Documentation is IEEE Standard 829 - see http://standards.ieee.org/
2. To learn more about the Project Management Institute, you can find them on the web at http://www.pmi.org/
3. For an excellent article to get you started on Risk-Based Testing, read James Bach's article "Heuristic Risk-Based Testing", available on his web site at http://www.satisfice.com/articles/hrbt.pdf
4. James Bach: to find out more about his work and for additional resources on Test Methodology, explore his web site at http://www.satisfice.com/
5. Cem Kaner: author, professor, and advocate of great software testing practices, he maintains several useful web sites. His personal site is http://www.kaner.com/ but most of his work and Testing educational materials can be found for free at http://www.testingeducation.org/
6. To find out more about Context-Driven School of Software Testing, check out http://www.context-driven-testing.com/ or pick up the book "Lessons Learned in Software Testing, A Context-Driven Approach" by Kaner, Bach and Pettichord.
©2011 Paul Carvalho. Contact me at: paul [at] staqs [dot] c o m