Types of Testing

Testing Types

Functionality Testing:

During unit testing each component or object is tested for completeness and correctness of implementation of a specific feature of functionality. In integration testing we focus on functionality that requires the correct operation of two or more components or a flow of data between them.

During system testing we should consider functionality in terms of whole sequence of end user operations or an entire area of functionality. In functionality testing we should consider "right action", "wrong side effect" risks and also test with both valid and invalid data sets.

User Interface Testing

User interface of a project is the first thing a user encounters in a product. While the product may perform its intended function correctly, if the UI behaves incorrectly, does not refresh properly or over writes meaningful text on the screen inadvertently the user can be seriously annoyed.

Testing the user interface of a product usually consist of the operation or simulation of keyboard, typing and mouse clicking and movement to do menu selection, field entry, field motion, control operation, request confirmation and so on.

User interface testing during the unit testing phase often involves only a select few UI objects or specific screen or page. During integration testing we need to focus on navigation/page flow across several screens/pages. In system testing we need to test the complete navigation required to meet client requirements. In addition to testing the normal navigation with reference to various types of users, it is necessary also to test for negative behavior.

Concurrent Execution Testing

Although a product may appear to function properly when tested by a single user, when multiple instances of a product run concurrently undesirable results may occur: they may show up in the application or in the operating system. A product that can have many instances of it running at the same time in parallel should undergo these concurrent execution tests:
• Simple usage (Two users)
• Standard usage (many users)
• Boundary situations (maximum number of users plus or minus one) if this limit exists

Multiple Configuration Testing

A product may be usually tested under a default configuration, it is important to test the non-default configurations also. Sometimes new tests may have to be written to test under a different configuration. It is possible that there could be infinite possible configurations that can be set up. It is necessary to identify key configurations that are important for the customer to operate the product and test the product for these configurations

Dependency Testing

Dependency testing is recommended to test any API calls made to other products and ensure that these calls act as promised. Such tests also provide a good regression mechanism when the product being developed is integrated with the new version of products it depends on. We need to test all interactions between products, including error cases. In general, any where data or control is transferred from one component to another component (or components), either immediately or in delayed fashion, an interface exists that can cause trouble.

Forward/Backward Compatibility Testing:

The product must be compatible (to the published extent) with any previous release and supported dependencies. Backward compatibility testing ensures that no problem will arise from a user trying to use the new version of a product along with an old version and verifies integration between products.

Installation and Upgrade Testing

If a user cannot install or upgrade the product whether it is through an inability to understand documentation, a failure of the installation/upgrade program, or lack of resource, the product has failed the user.

We need to check the installation/upgrade documentation for accuracy and follow all instructions form the point of view of a naïve user. Test the entire installation and upgrade procedure, both with correct and incorrect response to all prompts. If verification scripts are provided to test the installation/upgrade, run the script as an additional check. Generate possible system error to check the response of the installation/upgrade program.

User Documentation Testing

The user documentation is part of the product. If a new user is unable to use the software because of poor documentation, the product as a whole is unusable. All sections of the user documentation should be tested for accuracy against the reality of the product. Ideally user documentation should be held under source control and made part of regression testing of the product. Verify as corrected and complete all demos, tutorials and exercise described in the user documentation.

Standards Conformance Testing

Specific set of industry standards has to be tested. Tests need to be planned and conducted to detect bugs in standards conformance that could seriously affect the prospects of the product in the market.

Capacity and Volume Testing

During unit testing, from a structural test perspective, every buffer, occurrence, storage resource, processor, bus, and I/O channel in the system has to be tested. In integration testing aspects related to capacity of network to handle traffic and its behavior under various volumes are tested. Similarly the data storage capability is tested. During system testing the capacity and volume limitations are tested from a user's point of view.

Localization

Localization typically refers to problems associated with different languages. Languages other than English may require different character sets in which data can be represented. Testing character set conversion for data conversion between different localization is a very important consideration for planning localization testing. Another important item is the provision of different message files in various languages for error messages and other system information. Testing for localization may require setting up special hardware and software.

Security Testing

Security testing attempts to verify that protection mechanism built into a system will, in fact protect it from improper penetration. During security testing the tester plays the role of the individual who desires to penetrate the system. The tester would attempt to acquire passwords, access the system through some indirect/direct means, overwhelm the system so that it is not available to others, cause system errors, etc.

Performance Testing

Any product, no matter how well constructed and reliable, is worthless if its performance does not meet the demands of its users. Testing must be done in realistic setting to ensure adequate performance. However, we have to be cost effective.

We need to test how the product performs during operations normally executed by the users regularly. If performance goals are set for various operations of the product, it is necessary to verify that these goals are met. Performance is not only "How many per second" but also" how long".

Smoke Testing

Smoke testing is an integration testing approach that is commonly used when "Shrink wrapped" software products are being developed. It is designed as a pacing mechanism for time-critical projects, allowing the software team to assess its projects on a frequent basis.

Software components produced are integrated into a "build". A series of tests are designed to expose errors that will keep the build from properly performing its function. The intent is to uncover "Show stopper" errors that have the highest likelihood of throwing the software project behind schedule. Sometimes daily builds are made and subjected to smoke testing.

The smoke test should exercise the entire system from end to end. It does not have to be exhaustive, but it should be capable of exposing major problems. The smoke test should be thorough enough that if the build passes you can assume that it is stable enough to be tested more thoroughly.

Regression Testing

Regression testing is the re-execution of some subset or all tests that have been previously conducted to ensure that changes have not propagated unintended side effects. Regression testing is the activity that helps to ensure that changes do not introduce unintended behavior or additional errors. When sub sets of test are selected for regression testing, they should contain:
• A representative sample of test that will exercise all software functions
• Additional tests that focus on software functions that are likely to be affected by the change
• Tests that focus on the software components that have been changed.

Networks and Distributed Environments

If a product works in a network or distributed environment we need to test for
• Date and time zone handling
• Communication and telephony standards
Environment factors (temperature, pressure, etc) and their impact on the product should also be tested if required.

Error / Disaster Handling and Recovery

A software system must recover from faults and resume processing within a prescribed time. In some cases, a system must be fault tolerant: that is, processing faults must not cause overall system function to cease. In other cases, a system failure must be corrected within a specific period of time.

Recovery testing forces the software to fail in a variety of ways and verifies that recovery is properly performed. If recovery is automatic then re-initialization, check pointing mechanisms, data recovery and restart are evaluated for correctness. If recovery requires human intervention, the mean-time -to -repair (MTTR) is evaluated to determine whether it is with in acceptable limits.

Quality risks in this area include unacceptable failure rates, unacceptable recovery times and the inability of the system to function under legitimate conditions without failure.


Alpha and Beta Testing

If software is developed as a product to be used by many customers, it is impractical to perform formal acceptance tests with each one. Most software product builders use a process called alpha and beta testing to uncover errors that only the end user seem to be able to find.

A customer conducts the alpha test at the developer site. The software is used in a natural setting with the developer recording the errors and usage problems. Alpha tests are conducted in a controlled environment.

The beta test is conducted at one or more customer sites by the end-user of the software unlike alpha testing the developer is not present. Beta test is conducted at live sites not controlled by the developer. The customer records all problems encountered during beta and reports them to the developer


Stress Testing

Stress testing is the search for load dependent bugs that occur only when the system is stressed and the specification of stress tests is a key activity. Stress Testing increases the overall quality and reliability of software. Stress-related bugs are the most difficult to find and exhume.

In stress testing, the system is subjected to unrealistically harsh inputs or loads with inadequate resources, with the intention of breaking it. We test to find out weather the system can operate reliably at the limit of its resources or not.

Stress testing with high background load, to a point where one or more or all resources are simultaneously saturated. It is to system testing what destructive testing is to physical objects.

Stress testing often finds bugs for the following reasons:

• It forces race conditions
(Race conditions reflect temporal boundaries. Normally event A occurs before event B; By chance if event B occurs before A, then the system may fail, giving a race condition bug)
• It totally distorts the normal order of processing, especially processing that occurs at different priority levels.
• It forces the exercise of all system limits, thresholds, or other controls designed to deal with overload situations.
• It greatly increases the number of simultaneous actions.
• It depletes resource pools in extraordinary and unexpected sequences.

Some of the reasons why stress-related bugs are encountered in system are:

• Assumptions that there will be no interrupts.
• Failure to block or unblock interrupts.
• Assumptions that code is reentrant or not reentrant.
• By passing data interlock.
• Failure to close or open interlock.
• Assumptions that a called routine is resident or non-resident.
• Assumption that the calling program is resident or non-resident.
• Assumption that registers or memory were initialized or not initialized.
• Assumptions that the registers or memory location content did not change.
• Local setting of global parameter.
• Global setting of local parameter.

PERFORMANCE TESTING

Scripting Language used for Performance Testing is VU Scripting

Cookies
Cookie is a temporary file stores the session details
set_cookie Emulation Function
Adds a cookie to the cookie cache.

SYNTAX: set_cookie(name, value, domain, path [, secure])

Syntax Element Description
name A string expression that specifies the name of the cookie.
value A string expression that specifies the value for the cookie.
domain A string expression that specifies the domain for which this cookie is valid.
path A string expression that specifies the path for which this cookie is valid.
secure An optional string expression that, if given, provides the secure modifier for the cookie. The value of this parameter should be "secure".

COMMENTS
The set_cookie function creates the named cookie with the given value. If a cookie already exists with this name for the given domain and path then set_cookie() sets the value of that cookie to value.
The expiration date of the cookie is set sufficiently in the future that it will not expire during the run.

Dynamic Data Correlation
Dynamic data correlation is a technique to supply variable data values to a script when the transactions in a script depend on values supplied from the server.
For example, when you record an http script, the Web server may send back a unique string, or session ID, to your browser. The next time your browser makes a request, it must send back the same session ID to authenticate itself with the server.
The session ID can be stored in three places:
• In the Cookie field of the HTTP header.
• In an arbitrarily named field of the HTTP header.
• In an arbitrary hidden field in an actual HTML page.

Rational Suite TestStudio finds the session IDs (and other correlated variables) and, when you run the suite, automatically generates the proper script commands to extract their actual values.

Before you record a script, you can choose whether TestStudio correlates all possible values (the default), does not correlate any values, or correlates only a specific list of variables that you provide. For Information about correlating your data, see:
Identifying data that needs to be correlated
Coding the Script to Correlate the Data

Although varying test values may work for those transactions that depend on the result of an earlier transaction, other transactions may depend on values received from the server. If a script contains these transactions, you must manually edit the script to replace some of the missing client logic so that the values correlate dynamically. This is called dynamic data correlation.