In gesprek met James Whittaker

James Whittaker hield zich zijn hele leven al bezig met softwaretesten. Tijdens zijn studie schreef hij zijn afstudeeropdracht over model based testing. Hij maakte naam bij Microsoft en stapte onlangs over naar de concurrent, Google. Zijn boeken in de serie “how to break software” zijn bestsellers en zijn spectaculaire presentaties waarin hij live software of websites hacked maken diepe indruk. Zijn laatste boek over exploratory softwaretesten is eind vorig jaar uitgegeven. Speciaal voor mocht ik hem een paar vragen stellen.

1. Can you introduce yourself and explain how you already became a tester during college at the University of Tennessee-Knoxville, judging the name of your dissertation.
My name is James Whittaker and I am a Director of Engineering at Google. I own Test for a bunch of Google products including Chrome browser, Chrome operating system, Google Toolbar as well as some Search and Geo products and a bunch of back-end data center infrastructure applications. I also own Development of engineering tools including both developers and testing tools.

I got into test when I was a grad student. Mostly it was by default as my software engineering research team met on Saturday mornings and I had better things to do early Saturday than spend it with a bunch of coding nerds. My professor gave me two choices: get fired or be a tester. Neither he or I knew what a favor he did for me at the time. I really hit the ground running and did a lot of innovative work in model-based testing. In fact, I got my PhD two years before any of those developers. Testing was a great career move for me even back then.
2. How hard was it to change from a Microsoft employee towards a Google employee, and is testing very different at these two companies?
Not hard at all difficult. Microsoft was great preparation for Google. Culturally they are polar opposites with Microsoft being more top down whereas Google is more engineering driven. At Microsoft the high ratio of testers to developers is a case in point. The numbers of people on a project is made by execs and managers. At Google it is made by engineers and no engineer at Google believes a 1:1 ratio is necessary or even healthy. The fewer testers on a project means more involvement by developers for QA. I had a meeting yesterday with the development director for Chrome OS and the entire subject was what they could do to make our job easier. The director was genuinely concerned that his developers were engaging deeply enough on testing issues. A culture like that makes the 1:1 ratio irrelevant … everyone on the project is a tester.
3. I read somewhere that you are busy at Google with forging a future in which software just works. Is that possible, a world without software bugs?
Not in my lifetime. However we are getting closer. Even a few years ago I had to pull the battery on my smart phone every week or so. I’ve not even turned my current one off for 3 months and it works fine. Quality assurance for software is much like health care for humans. Humans will always get sick but with good prevention, good hygiene and regular maintenance our bodies do ok. We need to make testing like this: continuous and ongoing. One of the things that annoys me is the whole “push quality upstream” movement. Some people seem to believe that we can rig it so we just write perfect code. That’s like taking all your vitamins when you are a baby and then expecting a long healthy life. Obviously upfront debugging is good, but quality is an ongoing endeavor. It starts at the beginning and is a constant activity throughout the life of a product.
4. Patrick Copeland said that your vision on the future of testing is interesting, compelling and not just a little bit scary. Can you shortly tell us your vision, so we don’t get scared?
I am happy that it scares people and honored that it scares smart people like Patrick who thinks deeply on these matters. Too many people are dogmatic about testing. Some say “avoid rigor and do only exploratory testing” and they say it with a fervor that reminds me of religious fundamentalist who see only black and white. Others say the same of automation with the same amount of self righteousness. One thing I do know is that when you think your world view is the only view, there is a problem. People like this have stopped thinking about alternatives. They’ve stopped being open minded. They’ve definitely stopped being right.

I am also not going to stand in the middle and start every answer with ‘it depends.’ It turns out that there are some absolutes. There are some testing problems that can be driven to extinction with automation. There are some problems where exploratory testing is exactly the opposite of a good idea. I think it is smart to be problem-oriented and not solution-oriented. The latter is the proverbial hammer solution where every problem looks like a nail because you sell hammers for a living. I’ve laid out my full vision for software testing in my latest book but let me just say here that the part people find scary is that my vision requires far fewer testers than the world currently employs.
5. Your last book is about Exploratory Testing. Can you explain how taking the supermodel tour will improve our testing skills?
All the tours focus a tester’s attention. The idea is to test on purpose. Exploratory testing does not have to lack rigor and it does not have to encompass endless wandering hoping that you find a bug. It also should be about finding important bugs. I find myself endlessly annoyed by speakers who show bugs that no one would care about. Any exploratory method can find easy bugs; what about the hard ones?

Many of the tours focus on a general class of bug. The Supermodel Tour as a specific case focuses on presentation layer bugs. It asks you to first identify important properties of the UI and then choose paths that force those properties to change and then be displayed on the UI. We called it the Supermodel Tour to get the idea across that we are looking only skin-deep for bugs (only at the UI level). The tour gives both general guidance in terms of focusing on displayable properties and specific guidance about what part of the application should be visited during an exploratory session (the functions that allow you to change and then display those values). So you see that it requires some pre-work and planning but then allows for exploration once that planning is done. For example, in Maps we run the Supermodel Tour on our classification of landmarks. We make a list of all the landmarks (national parks, places of interest and so forth) in advance and explore the UI to find each location. We (actually Brendan Dhein) found a bug where Arlington National Cemetery was classified as a restaurant! It’s a subtle bug if you are just exploring. But if you are running the Supermodel Tour is jumps out at you. The idea is that a good tester can become a great tester with the right focus and by testing on purpose.
6. What will your next book be about?
I’m writing a book called How Google Test Chrome which details our testing process start to finish on Chrome OS. It’s a totally open kimono assessment of everything that we are going. Right, wrong, false starts, great ideas, cool innovation, new tools and every test artifact we generate from plans to test cases to open source automation. I am psyched about this as I don’t believe anyone has every fully documented and published a complete project before, particularly one of the complexity of an operating system.

I plan to include the browser too in this but as I have only written the first chapter on the test plan I do not want to over commit!
7. How is testing managed at Google? From one place or per country or per application or … ?
It’s divided by product lines or what we call “Focus Areas.” In my case I own the Client Focus Area. However since I am in a remote site I also have authority over all the work that goes on in Kirkland and Seattle Washington. I’m a busy boy but I like it that way. A single product would bore me.

It’s funny that on the Dev side each product has a Director in charge of it. Whereas I am the Director over many products. I have Test Managers over each product who have to interact with a Director on the dev side. So if you match us up one to one, you might have a Test Manager matching wits on a daily basis with Development Director three levels above them in rank. You talk about character building, this is the place for that. Google test managers are a breed apart. Cream of the crop.
8. Are you still collecting soccer jerseys? And if so, is there one you really want to add to your collection?
Yes and I cannot wait to wear them during the World Cup. By tradition I never wear anything else during that tournament (sorry for the visual). When people invite me to speak I often get them as gifts and I have dozens. I am hoping I get a Swiss one this trip (hint, hint) and I lost my Australian one (please don’t ask) and am looking for a replacement. But I have a lot of club jerseys too and will relish the chance to wear different colors. Send me a jersey and I’ll send you some signed books!
9. I hear you will giving a keynote at the Swiss Testing Day. Can you give us a sneak preview on what it will be about?
“Testing On Purpose” is the title. I am talking in far more depth about how we are testing Chrome at Google. I hope to see you there.

James Whittaker