Test documents, whose sole purpose should be to serve the work, are instead driving and constraining the work. Absurdly, form is dictating substance. When, as in this case, the form is obese and bloated, it sucks up and squanders all the energy that ought to go into the real thing.
Some testers (many, I hope) refuse to be tyrannized by the supremacy of form. I want to reach into the mainstream where form dominates and help testers there to join us. I want to help them learn to think better and think for themselves. This blog post is a step in that ongoing effort.
Another step is the webinar I presented last week for EuroSTAR: "Unburdening Testing - Finding the Balance Point for Test Documentation". (The Q&A from that session are in blog form on the EuroSTAR site.) The webinar is an introduction to the interactive tutorial I will present November 6 at EuroSTAR 2012: "Right-sizing Test Documentation".
After presenting my own webinar, I watched the recorded version of Alan Richardson's excellent webinar "Thinking Visually in Software Testing". The thinking tools and practices he describes there are so uncannily like my own that it prompted me in writing this post to think and write about my thinking. (I encourage you to watch Alan's webinar, if you haven't already done so.)
Form over Substance
When I first began working for a consulting company, my project manager gave me a mantra for billable work: “Never create anything that is not a deliverable to the customer”.
She
was a brilliant PM who became an important mentor for me, but not all her
advice was equally stellar. That statement in particular put horrible shackles on my
work that I took years to shed completely.
The
problem was that my customer deliverables were formal, standardized prose
documents. For me to show value, I needed to end each day with sections and
sub-sections populated with tidy paragraphs, building up to the finished
product.
At
first, I was lucky. My company was only dimly aware of standards purporting to govern
test documentation, and so I created my own templates. But over time, my templates became our company standards. When I used them, I was so busy trying to adapt
the structures for each project that I could no longer work easily with them. Like
the templates for test documentation imposed in many companies, I found that the
structure—the form—acted as a constraint on the substance. The tail was
wagging the dog.
We Can't Test Without Thinking
Testing is all about thinking. We think and rethink constantly. We think about how best to gather information on our projects: what to read and look at, and who to talk to and when. We think about how to approach the software and we develop test ideas as we go. We create test models, often complementary models for testing different aspects of the same piece of software. We plan and replan and then plan again. We think about what we've discovered in testing and design what we're going to do next. If we're doing a good job, we don't ever stop thinking.
Prose documents in preset patterns inhibit thinking and creativity. It’s useful sometimes to have a checklist of important things to think about, but we can’t afford to let those checklists limit our thinking.Templates are not the best checklists.
Writing in sentences can
sometimes help me to simplify and think through a tangled knot of ideas. I
do occasionally write to understand what I’m thinking. But I
never set out to think in predetermined sections of formal standardized prose.
Do you? Does anyone? Can anyone?
Visual Thinking
 |
| A couple of years ago these drawings helped me think through a problem |
Most often, I think in scribbles and doodles, beginning with notes and drawings that acquire structure as I develop the ideas and concepts. Or I might start with a tentative visual structure to generate ideas and then modify or replace the structure as needed to fit my thinking. Sometimes I scrawl ideas on coloured sticky notes and move them around over several days on a board or double-page spread of my notebook, drawing connections and annotating as I go. I often use mindmaps. I may use several different techniques to get my hands around a difficult problem.
Diagrams Emerge
What comes out of my initial thinking processes is rarely a customer deliverable. But over time, the result is usually some kind of structured diagram or set of diagrams that I can then use to communicate my ideas to other people. Rather than dictating and constraining the substance, the form of these diagrams emerges from the substance. |
| Example of a strategy diagram for an end-to-end systems integration test on a large project |
When—as is true on most projects—I must produce a formal document, I prefer to put diagrams front and centre. I want my documents to communicate, and I try to make them easy to read and understand. I use prose as sparingly as I can get away with, using tables and lists wherever possible. I don't include boilerplate, and I never copy wodges of text from one document to another. (Why ever would I waste valuable project time on such useless make-work?)
 |
| This diagram shows the division of responsibility for testing on the same big project |
Vacuous Form Tyrannizes the Mainstream
Apparently, that's not how most testers and test leads develop test documentation. In my consulting work with clients, I constantly see mammoth documents stuffed with books worth of low-content stodgy and opaque prose. Often, I search so-called test strategy documents in vain for any actual strategy. I fall asleep looking at test scripts that dictate every point and click and hideously repeat over and over again the most minute detail of so-called "test steps" and their piddling expected results. It's very hard to believe that much thinking is represented therein—or will inform the testing that must unfortunately follow.Nobody reads this stuff. It isn't useful. It certainly doesn't help anyone test well. So why do testers waste time and spirit churning it out? Why do their managers insist on the tyranny of form over the substance that only thought can produce?
Let's Take Testing Back!
I believe that thinking testers must take testing back from the process weenies and form merchants. Many testers have done this, but it has yet to happen in the mainstream—in big companies, big banks, government projects...sometimes even in startups and small companies.
In subsequent posts on this topic, I'll explore some ways we can do this.
2 comments:
Great stuff Fiona. Standardised, boiler plate documentation is used as a substitute for thought, not an aid.
I never fail to be astonished when I read through lengthy strategies and test plans, and there is hardly any worthwhile content. Sometimes there has been literally nothing of substance, yet no-one had questioned the value or the real quality of the document. The template was used, the standard document was churned out, the project milestone was hit. Whoopee! Let's go and celebrate!
I think I'm going to keep plugging away at this, like you!
My conclusion, after a year in a process-driven yet process-critical company is that the whole problem with process is this:
People don't understand what it is. It's as simple as that.
If you don't KNOW what the goals and workings of the process are, you tend to over-confine - yourself and others.
To me, a well-defined process is like a well-designed kitchen. Everything is orderly stored where I can easily reach it, and I can find my way blindfolded in the dark if I need to. This orderly environment frees my mind for the creative process of the cooking - thinking about what spice fits my meal instead of where the hell I stored it.
There are three lessons in this picture:
- If the process is to small, like a kitchen where you can't find a sharp knife, cooking is no fun.
- If the process is to big, like a kitchen where you can't move for high-end equipment (which is never used), it's no fun either.
- The kitchen equipment must fit the needs of it's owner - a bachelor may get by with a pot and some knifes, while a chef de cuisine needs more advanced appliances.
Post a Comment