Benefits of Software Testing

Why do we test software? Not in order to make tests pass. We test software so as to find bugs, i.e. errors in code, whether those errors are in third party libraries and frameworks or in code developed by us. Testing is successful if bugs and/or security holes are found in the process of testing. If bugs are found and fixed well before the product is released to the customers, then testing has paid off.

Can we get rid of all the bugs? How long and how extensively should we keep on testing? The answer to the first question is no, some bugs will unfortunately eventually be still lurking in production code. However, minimizing the amount of production code bugs saves a lot of money and company prestige. The second one is harder to answer – the minimum amount is to ensure that functional requirements are met. Fortunately, smart people have worked on these issues for decades, so there are guidelines and practices. See examples here for software testing and here for definitions, suggestions and references. See also other suggestions.

Types and Levels of Testing

As stated in Testing methods section on Wikipedia, testing practices can be divided e.g. into white box and black box testing, where the former requires some knowledge of the source code, whereas the latter has only the user interface available. Testing levels refer mainly to how small components are tested, as in unit testing, integration testing, component interface testing, system testing, etc.

Usually, the smaller the components are that are being tested, the more white-box testing is being performed. System level tests and acceptance tests are executed to make sure that all components of a large system work together as intended, and are usually mainly black-box tests. Integration testing is usually executed to make sure that new code added to version control does not break existing interfaces and functionalities. It is often performed automatically in continuous integration servers, e.g. Jenkins. For more details and references, see here.

Test Automation

Tests can be executed manually, but repetitive, scheduled tasks should be automated whenever possible. Test automation tools also usually provide detailed test results and logs in xml or html format. See here for definitions and references. The most common test automation tool especially intended for testing the functionality of web sites and web services is Selenium. Another common tool for black box functional testing is Robot Framework, which can be used together with Selenium. Selenium can be used to simulate user input in several programming languages (Python, Java, Ruby, Javascript), on Linux, Mac and Windows platforms, with drivers for Chrome, Firefox, Internet Explorer and Edge browsers. It is practically the standard for functional testing of web sites. Robot Framework’s Selenium2Library utilizes the Python bindings of Selenium to enable using the capabilities of selenium via Robot Framework keywords.

Test-Driven-Development

The extreme form of combining testing and development is Test-Driven-Development (TDD), where unit tests are written before the code that they test. It takes getting used to, but approaching it may be useful for every developer. At least, it helps in planning classes and functions in a manner where each line of code is evaluated in terms of what might go wrong, how should the input be verified, how should the correctness of the output be verified, how to ensure graceful exits on failure, etc. Robot Framework that was mentioned above is often used in acceptance-test-driven development or ATDD. ATDD refers to a practice where the test code is designed before implementing the application code, based on the functional requirements, planned in cooperation with the customer.

Performance Testing

Performance testing is the process of determining the speed or effectiveness of a computer, network or software program or device.  It is performed to determine the system parameters in terms of responsiveness and stability under various workloads; in other words, how fast does the system respond to requests and how many simultaneous requests can the system handle (see more here and here for more details and references). These types of tests are executed to measure the responsiveness of web sites.

Security Testing

Cybercrime and cyber-espionage have become increasingly common recently, and almost all software applications are constantly connected to the Internet, so security testing has become vital in quality assurance. Comprehensive security testing has become essential, even life-saving, because the Internet of Things has exposed vital sensors and actuators in public and private infrastructure potentially open for unauthorized and malicious parties.

If security testing is ignored or if it is not extensive enough, it might be possible for malicious parties e.g. to shut down the power grid, the water supply, public rail transport, or to use security holes in people’s household appliances to gain access to personal data or confidential business data.

Read more here and here for definitions and suggestions.

Conclusion

Testing is useful and important, because finding bugs, performance issues and security holes as early as possible is much cheaper than having customers find them in the delivered product. The entire staff of a software company should be involved in testing, not just the QA personnel. Quality assurance should be the core of software development effort. Developers should write their own white box unit tests, and automate and schedule their execution. Sales personnel should be aware of the benefits of testing when they convince customers to pay for testing. Managers should always keep in mind the importance of testing. Testing provides reliability; reliability provides prestige.