Exploratory testing is a powerful and convenient approach to testing. In some cases, it can be more productive than the usual scenario testing. Any tester at least once in his life, but applied research testing, at least on an unconscious level. However, very few people have studied this approach in detail, and it is not so recognized. It’s time for us to stop denying it, and publicly acknowledge the research approach as it is: scientific thinking in real time.

Parallel Design and Test Execution

The simplest definition of exploratory testing is the development and execution of tests at the same time. What is the opposite of the scenario approach (with its predetermined testing procedures, whether manual or automated). Exploratory tests, unlike scenario tests, are not defined in advance and are not performed in strict accordance with the plan. It sounds simple, but in practice everything is very vague. This is due to the fact that “certain” does not mean that we rigidly fix everything and everything. Even if all the test scenarios have been carefully defined, then the work with a lot of interesting details (for example, how fast to type on the keyboard, or what behaviors the program recognizes as erroneous) remain at the discretion of the tester. Besides, even in the free form of the search session, the test will include limitations in what part of the product to test or what strategy to use. A good research tester will write test ideas and use them in subsequent test cycles. Such notes are sometimes very similar to test scripts, even if they are not. Research testing is sometimes confused with “ad hoc” testing. Ad hoc testing usually refers to the process of improvisation, the search for a mistake by impromptu. By definition, anyone can do ad hoc testing. The term “exploratory  testing” (discovered by Cem Kaner, in the book Testing Computer Software) means a thoughtful approach to ad hoc testing. Over the past ten years, James Whittaker, Cem Kaner and James Bach have worked to identify skills and techniques allowing the effective use of research testing. For example, the processes of exploratory testing are fully defined and formulated, see the General Functionality and Stability Test Procedure for Microsoft’s Windows 2000 Compatibility Certification program.

The Balance Between Exploratory and Scenario Testing

If each next test that we perform is selected based on the results of the previous test, it means that we are using research testing. We begin to search and research when we can not tell which tests are to be performed, or when we have not yet had the opportunity these tests create, that is, the idea of writing them did not even occur to us.

If we go through the scenarios and new information comes out that offers us a better testing strategy, we can go to the search mode (as in the case of a new error that requires detailed consideration). On the other hand, we follow the script approach more: 1) the uncertainty in how we want to check is small 2) the new tests are relatively unimportant, 3) the need to ensure the efficiency and reliability in the performance of these tests is worth the effort to work with similar tests, 4) we are willing to pay for writing and maintaining tests.The results of exploratory testing do not necessarily radically differ from those that we get using scenario testing and both of these approaches to testing are fully compatible. Companies like Nortel and Microsoft usually use both approaches in the same project. Nevertheless, there are many important differences between the two approaches.

Why Conduct Research Testing?

The most discussed topics in the management of an effective research cycle of testing are the tester, testing strategies and reporting. Scenario approach to testing is an attempt to mechanize the testing process, when the idea is taken from the head of the test designer and is set out on paper. This kind of testing is very useful. But the testers who use the research approach are of the opinion that recording test scores and following them “dulls” the tester, preventing him from quickly finding key problems. The more intelligent we can do testing, the more chances we will have that we will test the application correctly and have time to do it. This is the power of exploratory testing: the wealth of this process is limited only by the breadth and depth of our imagination, as well as our understanding of the nature of the application being tested.

Scenario testing takes its own niche. We can imagine situations in testing where efficiency and reproducibility are so important that we need to write scripts for them or automate them. For example, in the case when the test platform is regularly unavailable, as in the case of client-server applications, in which there are only a few configured servers and they should be divided between development and testing teams. Common sense tells us that we need to thoroughly work out the test script in advance to get the most out of the tests during the time allotted to us. Exploratory testing is especially useful in complex testing situations, when little is known about the product, or as part of preparing a set of test scenarios. The basic rule is this: exploratory testing is used in cases where the execution of the next test is not obvious, or when you want to go beyond the obvious. In my experience, this happens in most cases.