Signing the ISO 29119 Petition and the Tester's Manifesto

I forgot to include these links in my previous post on ISO 29119. I have signed both. I urge all testers to read, consider the arguments and, if you agree, add your signature.

http://www.professionaltestersmanifesto.org/

http://www.commonsensetesting.org/news/files/PetitionISO29119.html

Why I oppose adoption of ISO 29119

I don’t oppose the idea of a testing standard, though I’d like to see a programming standard to accompany it. But ISO 29119 and its predecessors are not testing standards. They are fundamentally standards for documentation of testing and things called “testing processes”. There is little that goes into a testing process practiced by a skilled tester that a document about documents can capture or codify.

In a long career I have yet to see any indication that so-called “test” standards have done anything to improve the skill levels of testers or the quality of their testing. Instead, I’ve seen many organizations doing mediocre rote testing with testers who are forced to produce reams of impenetrable, repetitive documents that nobody outside the company testing circle reads. I repeatedly see test strategy documents showing not an ounce of strategy yet compliant with standards such as this. Those same organizations often insist that their testers obtain certification.

Whether or not this is the intent of the ISO 29119 proponents, it is how adoption will play out in real life in many organizations. As James Christie has pointed out, contract lawyers, internal auditors and managers who know nothing about testing will insist on the grand panoply of fat documents because it’s a standard and therefore must represent “best practice”. Nervous and unskilled test managers will embrace templates based on ISO 29119 because all those documents make them feel secure and important. People on Agile projects will struggle with the conflicting demands of their projects and the standards.

I have yet to see evidence that compliance to any “testing” standard equates to good testing.

Testing is a skilled activity. (James Bach calls it a “performance art”.) The only true measurement of testing is skill exhibited in live practice. Some proponents of ISO 29119 sneer at the “craftsman” (or craftsperson) mentality espoused by many of us. I wrote in an earlier post that I grew up thinking of craft as “skill fuelled by love and integrity”. You who sneer at the idea of craft and make snide jokes about medieval guilds should take a look at some highly-skilled professions in the modern world. Do you think a surgeon never speaks of, nor works to grow, her craft? Is a person licensed to perform surgery because of the fine strategies, plans and reports he compiles in templates?

I don’t doubt that surgeons must plan and devise strategies. They have procedures they must follow and forms they must fill. But ultimately, surgeons are evaluated—and licensed by their state-sanctioned governing bodies—based on their results and the skill they exhibit on a real live human, tools of the trade in hand. They must also pass exams on their knowledge of the human body and its pathologies, as well as a range of tools and techniques. But the exams surgeons undergo are much more rigorous than anything developed so far for a testing certification. And no-one becomes a surgeon merely by passing exams. Like other craftspeople, surgeons serve an apprenticeship: studying, practicing and exhibiting on the job the skills they must have to qualify for  their profession. As do lawyers.

I’m not pretending that software testers normally require the same level of skill as surgeons, nor as extensive a education program. But I do think that scaled down the analogy holds.

I would welcome a real testing standard, though I’d like to see a programming standard to accompany it. A true testing standard would focus on demonstrated skills assessed by qualified practitioners. It might set boundaries for the levels of testing skill required to work alone or under supervision, and the types of software testers at differing levels could work on. Education to meet such a standard would combine classroom studies with on-the-job practical training, and judging of live testing. At successful conclusion of her education, a tester could be certified as a professional. Very skilled testers could become master testers, in demand for very high-risk software.

We aren’t nearly organized enough to devise a real testing standard in the near future. But I don’t see ISO 29119 as an acceptable substitute. It puts too much focus on the wrong things.

Off the top of my head - Some skills & personal qualities that a tester can benefit from

My previous blog post - on why I believe it's good for testers to learn to code - triggered discussion and some protests, especially from testers who argued that testing involves so much more than an understanding of coding. Which is indubitable (to me at least).

So I thought I'd post this mindmap, an undoubtedly partial - in both senses of the word - list of tester skills and personal qualities that I threw together a few years ago in an idle moment. These are all things I believe a tester can benefit from. It's a very high-level, i.e., superficial view. I'm sure I've missed some very important items. Of the items I did list, it's clear to me that not all testers need every item in every context.
 

Why I believe it's good for testers to learn to code

Rob Lambert recently published a blog post with the title “Why Testers Really Should Learn to Code”. http://thesocialtester.co.uk/why-testers-really-should-learn-to-code Rob’s principal argument is that “The market demands it and the supply is arriving.”

What follows is an expanded version of the comments I made on that post.

First, Rob is presumably speaking about the market he knows best, i.e., in the UK. I’m not currently seeing such a heavy emphasis on coding in the Canadian market, though I think it’s probably there in Agile circles.

But regardless of the market demands, I think there’s a larger concern: about testers growing their skills and expanding their toolkits.

Whether or not testers “should” learn to code seems to be a contentious issue in at least some parts of the testing community at the moment. I admit that I’ve been observing the controversy with amazement. I’m having trouble understanding why any tester would not want to learn to code. 

I’m now a test manager, consultant and strategist, and I haven’t done serious hands-on testing in many years. But when I was a tester I knew how to code, and I worked at learning the languages I needed to know to understand and at least read the code for the applications I was working with. Working as a technical writer before I even became a tester, I learned to code and it seemed to me then to be an essential skill.

As coding advocates keep saying, you don’t have to be able to turn out production-level code. But it’s enormously helpful for a tester to understand how a system is constructed from the inside out. When you’ve tried to write working code, you learn to know the kinds of mistakes it’s easy to make with a given language and that helps you find bugs in other people’s code. And when you can read code, you can often spot the place where the error occurred and see what else might be wrong around it.

When you can code, you can write routines to build data in bulk, and also to inject data. You can write routines that help you test (or check) faster, or make it possible for you to test a larger number of input variations than you could practically manage otherwise. You can write and run your own batch jobs. You can query a database directly, to find out what’s really getting written to it. (SQL is code, too.) You can make clever use of spreadsheets to boost your test capabilities.

There are so many things a tester can do with code. And coding is FUN, folks! In fact, executing working code that you’ve written yourself is a blast! It’s almost as much fun as testing. (Okay, that’s highly subjective. But if you love software, why wouldn’t you love building some?)

There's a social aspect too. Being able to write code helps you understand your programmer teammates and it teaches you empathy and respect for their skills. (You want programmers to empathise with you and respect your skills too, don’t you?)

 

This is not to say that you can’t be a good tester without knowing how to code. Of course you can. I know lots of excellent testers who can’t code and don’t want to learn. I also know lots of terrific testers who don’t have (or don’t believe they have) exploratory testing skills, or visual thinking skills, and don’t want to learn those either.

In my experience, these and all your other skills give you tools you can use when the context calls for them. Not every tool is appropriate or useful in every context. But the more tools and skills you have at your disposal, the more flexible you can be and the more easily you can rise to the demands of different contexts. If you don’t have a particular skill, you may not even recognize how having it could help you test better in your context.

I don’t believe that the issue comes down to should or should not. Rather, I believe it’s about expanding your skills and your toolkit. Why wouldn’t you want to do that?