Yaron Kottler is in good company. As someone who started at an entry-level position and worked his way to CEO, he shares an experience with top executives from Best Buy, GE, McDonald’s, Morgan Stanley, Nortel and Seagate.
Today, Kottler runs Quali Test US, the North American branch of the QualiTest Group. QualiTest is a global software testing and quality assurance consultancy and service provider focused exclusively on onshore QA and testing. The company employs more than 800 testing and QA professionals in 10 locations throughout the United States, Europe and the Middle East.
ANDREW MUNS: What was the first job you ever had?
YARON KOTTLER: The very first job was working at a bike shop in Tel Aviv, but my first real job—believe it or not—was at QualiTest, where I was hired as a junior testing engineer.
How did you get started in the field of software testing and what was the career path that led you to be CEO of QualiTest US?
Originally, I wanted to be a software developer, and when I joined QualiTest [10 years ago], I thought of software testing as a path toward a career as a developer. Nonetheless, I ended up staying in software testing and moved up through the ranks from senior test engineer to test lead, test manager, business manager, load testing manager, and was eventually sent to Turkey to build out the effort there. I ended up traveling back and forth between there and Israel to build that business.
I became CEO after the company acquired IBase Consulting [in 2006]. This was primarily a strategic acquisition that QualiTest used to launch its business in the US market.
What are the key differentiators of QualiTest’s testing services that allow you to compete effectively in the US market?
There are a few things that make us stand out. First, we’re completely focused on QA and testing. We don’t do test and development, we don’t sell software—we just concentrate on what we do best.
Second is our strength in the onshore market. We believe high-quality onshore teams in many cases outperform their offshore counterparts.
Having a large international practice gives us advantages as well. We have the ability to take the new technologies and practices from Europe or North America and apply them in other markets.
Lastly, we’re very good at hiring good people and developing them into world-class testers. We focus more on smarts and potential than resumes, and have a mentorship program and internal training resources that allow people to grow within the organization. Consider my career path!
You interviewed attendees of our conference last year about onshore vs. offshore testing services and found that—contrary to popular belief—onshore testing is often more cost-effective. Have trends since then shown increased interest in onshore vs. offshore testing?
Absolutely. For many reasons, a test team located onshore outperforms and is more efficient than an offshore team. Testers sharing a similar culture, language and time zone end up having a huge cost advantage that may not be apparent from a glance at the hourly rate.
There has been growth in onshore outsourcing, but I believe this is also due to the overall growth in outsourcing and not necessarily only to taking share from the offshore side.
In your opinion, what is the key to successful outsourcing of test services?
I would say the most important things are transparency and having the appropriate test management tools. Also, being able to trust your service provider and feel comfortable that they can deliver. This is key: If you are going to outsource, you have to learn to let go of some control.
Lastly, I’d say internal organization buy-in. If the operations team or the developers disagree with the outsourcing decision, it will be hard for them to work with the external test team.
What challenges arise for distributed teams using agile methodologies? Does this make outsourcing more or less appropriate for agile practitioners?
I might be old-fashioned, but in my opinion agile and distributed teams don’t work together. I see a lot of teams who describe their methodology as agile or Scrum, but who in reality have created a hybrid workflow that’s adapted to the realities of a distributed team.
As for the main challenges of agile, these aren’t usually related to applying the methodology. Barriers to successful implementation usually come from the organizational culture. After all, agile theory is simple to understand. Getting teams to embrace the necessary culture for the method to work is the hard part.
How is software testing conducted differently in the US vs. overseas? Are there cultural or regional differences in the definition of “quality”?
Differences in the definition of quality and the importance of quality seem to vary more by organization and industry than by country. Corporate culture varies widely even within a given country, and that’s a bigger driver of how much weight is put on shipping a quality product.
There are regional differences in how testing is done, however. US-based organizations are generally much further along in adopting agile and are usually more willing to try new things. The UK is more or less like the US.
In the Netherlands and Germany, testing is much more scripted and there is much more emphasis placed on metrics and measurement.
The use of the words “testing” and “QA” also varies, with many in the US using the term QA to describe what I think is really testing.
Companies overseas tend to have a better understanding of the value of testing and how testing is an activity that saves money when done correctly.
You’ve spoken a lot about “virtuology.” What does this mean and why are testers interested?
The idea is to use virtualization techniques to record a testing session in its entirety. This is analogous to a DVR for a computer where you can rewind, fast forward, pause and go live at any moment. That is basically a time machine, if you think about it. You can now record everything occurring at the time a test is conducted, including network conditions, RAM, ROM, OS conditions as well as any server conditions—not just a series of clicks and keystrokes as is done by GUI automation tools.
This is an amazing tool for debugging and resolving defects that are difficult or impossible to reproduce. A recent study of the defect life cycle in 28 testing projects, conducted by QualiTest, showed that about 9 percent of the defects receive the “can’t reproduce” status at some point of the defect life cycle. This can be attributed to either inadequate reporting or nondeterministic application behavior.
So virtuology can simplify test result documentation. It can also contribute greatly to the efficiency of the testing process, as testers can now spend more of their time actually testing and less on activities such as preparation, environment setup, running pre-steps and gathering test results. I like to summarize the benefits as the three “E’s” of virtuology—exploration, efficiency and evidence.
What are some of the most useful virtuology tools currently available?
In my opinion VMware’s products offer the most value for the money. We’ve rolled it out to clients with good success.
In addition to virtuology, how will the next generation of testers make use of cloud computing technologies?
The benefits provided in environment setup are becoming invaluable to functional and compatibility testing and I am sure load testing will continue to be an area to benefit, as well as application environments cent¬ral to this process.
Virtuology tools aren’t currently hosted in the cloud, but I hope we’ll see them start to take advantage of cloud computing environments. VMware is great, but is still in the early stages of what these tools can become as they mature.
Call me crazy, but my expectation for the future is that the world is going back to the days of the mainframe in which the cloud is replacing your two-tone computer.
How is the role of the tester within the organization changing and what will test teams look like in 10 years?
I hope testing teams will become more of an extension of operations and will work more closely with, and as advocates for, the users rather than report to development or IT as they usually do.
I also think we’ll see agile being used more widely and that we’ll see agile and Scrum teams mature.
What is the most important skill an individual tester can possess?
Good analytical skills, understanding the big picture and most importantly understanding what the end user needs and wants.
Testers must understand the business they’re working in, and for that to happen you need a dialog with operations or with end-user proxies.
Of course, testing requires a highly specialized skill set that operational people usually don’t possess, so we are talking about a multidisciplinary approach to both testing and operations.
Are there any companies that are doing this well?
I’m not sure, but this is essentially about training—not training on testing methodologies, but training that helps testers understand the business context.
What advice would you give to someone beginning a career in testing?
Try to get a job at a place that will let you experiment with lots of different technologies, tools and methodologies. Most importantly, look for a company that will give you some freedom to grow in the direction that suits you best.
I know this is talking my own book, but working with a consulting firm is a great way to start out. You get a lot of experience on a diverse range of projects. This allows you to find out what you like, and limits the time you spend on projects that aren’t a good fit.
