Search This Blog

Tuesday, June 12, 2012

Breaking the Tyranny of Form - Part 1

Testing in many mainstream organizations is choked with low-value standardized documents that not only gobble up valuable thinking and testing time, but actively discourage thinking and impede good testing. While some testers can hope for relief from the document burden through the spread of Agile methods, this ridiculous situation isn't going away in a hurry. As a blog post by James Christie recently reminded us, the floodgates onISO/IEC 29119 Software Testing – the new international software testing standard” will soon open. I suppose it's possible that the new standard will wash away the documentation excesses we have now. I'm not holding my breath on that.

Test documents, whose sole purpose should be to serve the work, are instead driving and constraining the work. Absurdly, form is dictating substance. When, as in this case, the form is obese and bloated, it sucks up and squanders all the energy that ought to go into the real thing.

Some testers (many, I hope) refuse to be tyrannized by the supremacy of form. I want to reach into the mainstream where form dominates and help testers there to join us. I want to help them learn to think better and think for themselves. This blog post is a step in that ongoing effort.

Another step is the webinar I presented last week for EuroSTAR: "Unburdening Testing - Finding the Balance Point for Test Documentation". (The Q&A from that session are in blog form on the EuroSTAR site.) The webinar is an introduction to the interactive tutorial I will present November 6 at EuroSTAR 2012: "Right-sizing Test Documentation".

After presenting my own webinar, I watched the recorded version of Alan Richardson's excellent webinar "Thinking Visually in Software Testing". The thinking tools and practices he describes there are so uncannily like my own that it prompted me in writing this post to think and write about my thinking. (I encourage you to watch Alan's webinar, if you haven't already done so.) 

Form over Substance

When I first began working for a consulting company, my project manager gave me a mantra for billable work: “Never create anything that is not a deliverable to the customer”.

She was a brilliant PM who became an important mentor for me, but not all her advice was equally stellar. That statement in particular put horrible shackles on my work that I took years to shed completely.

The problem was that my customer deliverables were formal, standardized prose documents. For me to show value, I needed to end each day with sections and sub-sections populated with tidy paragraphs, building up to the finished product.

At first, I was lucky. My company was only dimly aware of standards purporting to govern test documentation, and so I created my own templates. But over time, my templates became our company standards. When I used them, I was so busy trying to adapt the structures for each project that I could no longer work easily with them. Like the templates for test documentation imposed in many companies, I found that the structure—the form—acted as a constraint on the substance. The tail was wagging the dog.

We Can't Test Without Thinking

Testing is all about thinking. We think and rethink constantly. We think about how best to gather information on our projects: what to read and look at, and who to talk to and when. We think about how to approach the software and we develop test ideas as we go. We create test models, often complementary models for testing different aspects of the same piece of software. We plan and replan and then plan again. We think about what we've discovered in testing and design what we're going to do next. If we're doing a good job, we don't ever stop thinking.

Prose documents in preset patterns inhibit thinking and creativity. It’s useful sometimes to have a checklist of important things to think about, but we can’t afford to let those checklists limit our thinking.Templates are not the best checklists.

Writing in sentences can sometimes help me to simplify and think through a tangled knot of ideas. I do occasionally write to understand what I’m thinking.  But I never set out to think in predetermined sections of formal standardized prose. Do you? Does anyone? Can anyone?

Visual Thinking

  A couple of years ago these drawings helped me think through a problem

Most often, I think in scribbles and doodles, beginning with notes and drawings that acquire structure as I develop the ideas and concepts. Or I might start with a tentative visual structure to generate ideas and then modify or replace the structure as needed to fit my thinking. Sometimes I scrawl ideas on coloured sticky notes and move them around over several days on a board or double-page spread of my notebook, drawing connections and annotating as I go. I often use mindmaps. I may use several different techniques to get my hands around a difficult problem. 

Diagrams Emerge

What comes out of my initial thinking processes is rarely a customer deliverable. But over time, the result is usually some kind of structured diagram or set of diagrams that I can then use to communicate my ideas to other people. Rather than dictating and constraining the substance, the form of these diagrams emerges from the substance.

Example of a strategy diagram for an end-to-end systems integration test on a large project

When—as is true on most projects—I must produce a formal document, I prefer to put diagrams front and centre. I want my documents to communicate, and I try to make them easy to read and understand. I use prose as sparingly as I can get away with, using tables and lists wherever possible.  I don't include boilerplate, and I never copy wodges of text from one document to another. (Why ever would I waste valuable project time on such useless make-work?)

This diagram shows the division of responsibility for testing on the same big project

Vacuous Form Tyrannizes the Mainstream

Apparently, that's not how most testers and test leads develop test documentation. In my consulting work with clients, I constantly see mammoth documents stuffed with books worth of low-content stodgy and opaque prose. Often, I search so-called test strategy documents in vain for any actual strategy. I fall asleep looking at test scripts that dictate every point and click and hideously repeat over and over again the most minute detail of so-called "test steps" and their piddling expected results. It's very hard to believe that much thinking is represented therein—or will inform the testing that must unfortunately follow.

Nobody reads this stuff. It isn't useful. It certainly doesn't help anyone test well. So why do testers waste time and spirit churning it out? Why do their managers insist on the tyranny of form over the substance that only thought can produce?

Let's Take Testing Back!

I believe that thinking testers must take testing back from the process weenies and form merchants.  Many testers have done this, but it has yet to happen in the mainstream—in big companies, big banks, government projects...sometimes even in startups and small companies. 

In subsequent posts on this topic, I'll explore some ways we can do this.

Wednesday, May 16, 2012

Let's Test 2012 - A Personal View

 Let's Test 2012

I'm rarely tempted to write about conferences, but for once I can't resist: the inaugural Let's Test was a wonderful experience in so many ways.

The Place

The rural setting at Runo, with its stunning "nobility of labour" art and bright, airy buildings, was an inspired choice, and no small contributor to the conference's success. We were all captive--miles from other (lesser) attractions--and there were many comfortable and inviting places to sit and confer, both indoors and out. It didn't hurt that there was a bar (with free drinks provided by sponsors on a couple of evenings), and fabulous food (the smoked salmon!). All meals were provided and the breaks were generously provisioned with treats like raspberry smoothies or pretty little macarons. When did you ever eat well, or even sit comfortably, at a conference?

Organizer Johan Jonasson, who knows I carry my own coffee to conferences and frequently joins me in a cup, promised me good Swedish coffee at Let's Test. It wasn't brilliant, but it was indeed decently strong and far more drinkable than I'd ever get in a North American hotel.

The Participants 

I use the word advisedly. The people at Let's Test were active participants throughout. Adopting the CAST facilitation model may have helped, and it certainly kept in-session discussions on track, but I don't think anything would have kept this group from lively and interesting conversations, often far into the night. And the Test Lab drew a big crowd every evening. That's what conferences are for!

The Sessions

People like me who present at a lot of conferences can become very jaded. At most, I rarely find the official sessions compelling and tend to skip them in favour of corridor conversations and networking. In contrast, the Let's Test agenda presented the best dilemma I can experience at a conference. In several time slots, I had difficulty deciding which session to go to. I was glad I chose Christin Wiedemann's "You Are a Scientist - Embracing the Scientific Method in Software Testing"  and Alan Richardson's "Testing Hypnotically", but sorry to miss several others. The real standout for me was Anne-Marie's Charrett's "Coaching Testers", especially the fascinating live session at the end, where many of us got to offer suggestions to (i.e., "coach") the coaches.

CAST is the only other test conference I know of that has this "assembly of peers" feeling, where speakers are as interested in other people's sessions as they are in their own.  


I presented a full-day tutorial on Test Leadership and a track session on managing an end-to-end systems integration test.

I felt a little trepidation before the tutorial. How would a big group (28 people), for the most part meeting me and each other for the first time, and speaking what was for most a second language, work together in an experiential session? Would they embrace exercises that might seem strange to them, and then be willing to share their experiences and learnings in a debrief?

I needn't have worried. I started with a small-group exercise that probably helped to draw participants in, particularly some of the quieter ones. In that first exercise, one of the small groups--all Swedes--sensibly worked together in Swedish, though their output was in English. Otherwise (to my shame as a less than marginally bilingual North American), this multilingual group blew me away with their facility and willingness to communicate in my language. Soon everyone was engaged and working together, and they stayed engaged throughout the day (some more quietly than others), coming up with many fascinating insights and strategies for dealing with issues in test leadership.

I am grateful to everyone in my class for making it such a great learning experience for me! So far, I've had very positive feedback from participants. If you were there, please let me know what you think. Did the session work for you? What went well? What do you think I could do better next time?

Meanwhile, here's a link to Simon Morley's blog, where he describes his reactions.

Onwards (for me)!

I'll be keynoting (Are You Managing Test - or "The Test Process") at the BCS SIGIST in London on June 21, and also doing a workshop on mind-mapping a test strategy. I look forward to working with a group of engaged British testers there.

June 26-28, I will be presenting Beyond Process: three 1-day experiential classes in London, including a reprisal of my Let's Test tutorial, "Inspiring Testing" plus "Determining Business Risks for Testing" and "The Design behind the Plan - Test Strategy workshop". Registration is through the sponsor, ElectroMind.

If you're in North America, you may want to register for Beyond Process in San Jose before CAST, July 11-13.

Then it will be time for CAST 2012, July 16-18. As Program Chair, I believe the Program Committee and I have assembled an excellent program of sessions and innovative workshops. I'm really looking forward to seeing some of my new and old friends from Let's Test, as well as many others who didn't make it to Sweden. It too is going to be a wonderful conference!

Onwards (for Let's Test)!

Let's Test 2012 was a terrific first conference. My thanks and congratulations to the organizers, and I look forward eagerly to Let's Test 2013!

And some pics from the Nature Reserve

One of the evening activities was a Nature Walk, guided by a marvelously fit 80-year-old gentleman,  recently named Sweden's Gardener of the Year. It was so beautiful, I rose early the next morning and went out again with my camera.