Project Overview

Six decades into the computer revolution, four decades since the invention of the microprocessor, and two decades into the rise of the modern Internet, all of the technology required to transform industries through software finally works and can be widely delivered at global scale.

 — Marc Andreesen in “Why Software Is Eating The World”

Rapid market evolutions force the software industry to switch to a more agile way of working and software engineering teams typically take a leading role therein. Nevertheless, while producing goods, the manufacturing industry must adhere to safety standards such as IEC 62061 (mechatronic systems), RTCA/DO-178C (Avionics), IEC 62304 (medical devices) and ISO 26262 (automotive). But even outside the manufacturing industry —such as the financial sector and e- Government— software is a driving force in producing customer value. There as well the pressure is increasing to adhere to privacy standards; the General Data Protection Regulation GDPR 2012 is but one example of what is coming.

Unfortunately, as depicted in Figure 1, the current safety standards don’t match well with the challenges trusted on today’s software testing teams. They are based on strict processes, comprehensive documentation, and rigorous planning; valuable assets yet corresponding to a development process optimised for perfection. Agile software development on the other hand forwards a series of other values: customer value, iterative and incremental development, continuous integration, hence optimises for ease of change.

Contrast between Agile Methods and Safety-Privacy Standards
Figure 1: Contrast between Agile Methods and Safety-Privacy Standards

Partly driven by these agile development, partly driven by technological innovations, software is being released in shorter release cycles. A decade ago, people were used to install a new version of their operating system every few years; now they do it several times a year. Mobile devices on the other hand release their software by means of app stores and customers expect a notification every few weeks or otherwise don’t trust the app anymore. Companies are forced to play along these expectations or otherwise competitors will take over their market share; they have to worry about competitors leapfrogging them with ever-faster releases, made possible by technological innovations such as agile development and continuous integration.

Problem Statement. Modern software teams seek for a delicate balance between two opposing forces: striving for reliability and striving for agility. In the former, teams optimise for perfection; in the latter, they optimise for ease of change. The drive towards faster release cycles only exacerbates the problem: manual testing is not an option anymore; automating the bulk of the test load is the only way out.

Project Goal. The TESTOMAT project will allow software teams to increase the development speed without sacrificing quality. To achieve this goal, the project will advance the state-of-the-art in test automation for software teams moving towards a more agile development process.

Market Value Chain for Automated Tests
Figure 2: Market Value Chain for Automated Tests

Figure 2, shows how the TESTOMAT Project envisions its market value chain. The top and bottom illustrate the two opposing forces governing the market: the top arrow illustrates the drive towards agile methods, the bottom arrow the adherence to quality standards. At the right-hand side, we see the producers of software intensive systems, which must deliver more features, faster without sacrificing product quality. By pushing their test automation to the next level, they expect to reduce their testing costs by an order of magnitude. The middle of the figure depicts the test tool vendors and test consultants, sometimes in a hybrid combination. Their improved service offerings allow their customers to automate the bulk of the current test process, hence will allow them to gain market share in a growing but highly competitive market. The left-hand side shows the research centres and universities contributing knowledge, expertise, algorithms to have an impact down-chain.