1st IEEE Workshop on NEXt level of Test Automation 2018
NEXTA 2018

Workshop Summary

Test automation has been an acknowledged software engineering best practice for years. However, the topic involves more than the repeated execution of test cases that often comes first to mind. Simply running test cases using a unit testing framework is no longer enough for test automation to keep up with the ever-shorter release cycles driven by continuous deployment and technological innovations such as microservices and DevOps pipelines. Now test automation needs to rise to the next level by going beyond mere test execution. The NEXTA workshop will explore how to advance test automation to further contribute to software quality in the context of tomorrow's rapid release cycles. Take-aways for industry practitioners and academic researchers will encompass test case generation, automated test result analysis, test suite assessment and maintenance, and infrastructure for the future of test automation.

Keynote

Prof. Yvan Labiche

Carleton University, Canada

Test automation — Automation of what?

Abstract: Taking a birds-eye view at the different activities that take place when someone engages in software testing, we will discuss automation problems and some deployed solutions to the broad notion of software test automation. In doing so, we will discover engineering/deployment problems as well as more fundamental scientific/research issues.

Bio: Yvan Labiche received an engineering degree in Computer System Engineering in 1995, from the graduate school of engineering: CUST (Centre Universitaire des Science et Techniques, Clermont-Ferrand), France. He completed a Master of fundamental computer science and production systems in 1995 (Université Blaise Pascal, Clermont Ferrand, France). While doing his Ph.D. in Software Engineering, completed in 2000 at LAAS/CNRS in Toulouse, France, Yvan worked with Aerospatiale Matra Airbus (now EADS Airbus) on the definition of testing strategies for safety-critical, on-board object-oriented software systems. In January 2001, Dr. Yvan Labiche joined the Department of Systems and Computer Engineering at Carleton University, as an Assistant Professor. He is now Associate Professor and chairs the department of Systems and Computer Engineering at Carleton University (Ottawa, Canada).

Preprints

Preprints of the papers temporarily available in a handy zip-file [ZIP].

Tentative Program

Registration open from 8.00 at AROS CONGRESS CENTER; Västerås
  • 9.00-9.20 Introductions and Welcome from the Chairs - Session Chair: Sigrid Eldh
    • Hello - introductions!
    • Why this workshop
      • Most Viral Tweet Award [#nexta18]
      • Best Questions Award
      • Best Paper Award
  • 9.20-10.10 Test Automation - Automation of What? Keynote by Prof. Yvan Labiche, Carleton University, Canada
  • 10.10-10.30 (Short paper) Challenges in Automated Testing through Graphical User Interface - Pekka Aho and Tanja Vos

  • Coffee 10:30 - 11:00

  • 11:00-12:30 Test Maintenance - Session Chair: Markus Borg
    • (Short paper) Adam Tornhill. Uncover Your Tracks: Using CodeScene to Supervise the Quality of Automated Tests
    • (Short paper) Ralf Gerlich and Christian Prause. Evaluating Test Data Generation for Untyped Data Structures Using Genetic Algorithms
    • Martin Reider, Stephan Magnus and Jan Krause. Feature based testing by using model synthesis, test generation and parameterizable test prioritization

  • Lunch 12:30 - 14:00

  • 14.00-15.30 Automated GUI Testing - Session Chair: Serge Demeyer
    • Vahid Garousi and Erdem Yildirim. Introducing automated GUI testing and observing its benefits: an industrial case study in the context of law-practice management software
    • Toshiyuki Kurabayashi, Muneyoshi Iyama, Hiroyuki Kirinuki and Haruto Tanno. Automatically Generating Test Scripts for GUI Testing
    • Riccardo Coppola, Marco Torchiano and Maurizio Morisio. Maintenance of Android Widget-based GUI Testing: A Taxonomy of test case modification reasons

  • Cofee 15:30 - 16:00

  • 16.00-17.30 Test Impact, Metrics and Embedded Systems - Session Chair: Adnan Causevic
    • (Short paper) Kristian Wiklund and Monika Wiklund. The Next Level of Test Automation: What About the Users?
    • Gao Feng and Deng Fei. Design of High Confidence Embedded Software Hardware-in-loop Simulation Test Platform Based on Hierarchical Model
    • Leonid Borodaev, Rix Groenboom, Rein Smedinga and Alex Telea. Software metrics for policy-driven SDLC automation

  • 17:30-17:35 Closing and Awards
  • 19:00-22:00 Workshop dinner (self-paying Sponsored by the City of Västerås)
    The dinner will be in "Grekiska kolgrillsbaren Västerås". Restaurant address is: Kopparbergsvägen 27, Västerås (Google map)

Registration

Registration for the workshop should be done via ICST2018. Please referr to the following link: http://www.es.mdh.se/icst2018/registration/

Travel information

For travel information please referr to the following ICST2018 web site: http://www.es.mdh.se/icst2018/travel-and-accomodation/.
Accomodation information are presented at the following link: http://www.trippus.net/ICST2018_hotelbooking.

Important Dates

  • 19th of January, 2018

    Submission

  • 19th of February, 2018

    Notification

  • 28th of February, 2018

    Camera Ready

  • 9th of April, 2018

    Workshop

Call for Papers

NEXTA solicits contributions targeting all aspects of test automation, from initial test design to automated verdict analysis. Topics of interest include, but are not limited to, the following:

  • Test execution automation
  • Test case generation
  • Automatic test design generation
  • Analytics, learning and big data in relation to test automation
  • Automatic aspects management in test, progress, reporting, planning etc.
  • Visualization of test
  • Evolution of test automation
  • Test suite architecture and infrastructure
  • Test environment, simulation, and other contextual issues for automated testing
  • Test tools, frameworks, and general support for test automation
  • Testing in an agile and continuous integration context, and testing within DevOps
  • Orchestration of test
  • Metrics, benchmarks, and estimation on any type of test automation
  • Any type of test technologies relying on automation of test
  • Process improvements and assessments related to test automation
  • Test automation maturity and experience reports on test automation
  • Automatic retrieval of test data and test preparation aspect
  • Maintainability, monitoring and refactoring of automated test suites
  • Training and education on automated testing
  • Automated test for product lines and high-variability systems
  • Test automation patterns
  • Automated test oracles


NEXTA accepts the following types of original papers:

  • Technical Papers (max. 8 pages in IEEE format).
    Full papers presenting research results or industrial practices related to the next generation of test automation.

  • Tool Papers (max. 4 pages in IEEE format).
    Tool papers introduce tools that implement an approach to support the transition to the next generation of test automation. A tool paper submission must include either 1) a URL to a screencast of the tool in action, or 2) a runnable version of the tool for evaluation by the program committee.

  • Position and Experience Papers (max. 4 pages in IEEE format).
    Short papers introducing challenges, visions, positions or preliminary results within the scope of the workshop. Experience reports and papers on open challenges in industry are especially welcome.

  • Abstracts w. Video submission (Not exceeding more than 8 minutes)
    These videos will be shown separate - and will not be included in the publications, but as a parallel source of information. The video submission must be linked and visible on open source tools, and can include a presentation, demo or experience on test automation.

Authors should submit a PDF version of their paper through the NEXTA 2018 paper submission site: https://easychair.org/conferences/?conf=nexta2018.
All accepted papers will be part of the ICST joint workshop proceedings published in the IEEE Digital Library.

Call for Papers can be downloaded as text or as a PDF document.

Organisation

Sigrid Eldh

Ericsson, Sweden
General Chair

Adnan Causevic

Mälardalen University (MDH), Sweden
Program Co-Chair

Markus Borg

RISE SICS, Sweden
Program Co-Chair

Serge Demeyer

University of Antwerp, Belgium
Program Co-Chair

Program Committee

  • Tanja Vos, Universidad Politecnica Valencia, Spain and Open University Netherlands
  • Serge Demeyer, University of Antwerp, Belgium
  • Markus Borg, RISE Swedish Institute of Computer Science, Sweden
  • Adnan Causevic, Mälardalen University, Sweden
  • Marc-Florian Wendland, Fraunhofer Fokus, Germany
  • Michael van der Bijl, Axini, Netherlands
  • Kristian Wiklund, Ericsson, Sweden
  • Leire Etxeberria Elorza, University de Mondragon, Spain
  • Vahid Garousi, Wageningen University, Netherlands
  • Michael Felderer, University of Innsbruck, Austria
  • Ali Parsai, University of Antwerp, Belgium
  • Franz Wotawa, Graz University of Technology, Austria
  • Karl Meinke, The Royal Institute of Technology, Sweden
  • Damiano Torre, Carleton University, Canada
  • Andrew V. Jones, Vector Informatik, UK
  • Hyonsook Do, University of North Texas, US
  • Peter M. Kruse, Assystem, Germany
  • Alessandra Bagnato, Softeam, France
  • Mika Mäntylä, University of Oulu
  • Yvan Labiche, Carleton University, Canada
  • Elaine Weyuker, Rutger University, US and Mälardalen University, Sweden
  • Benoit Baudry, Royal Institute of Technology, Sweden
  • Tao Yue, Simula, Norway
  • Bernard Leger, Smarttest, France
  • Ayse Tosun, Istanbul Technical University, Turkey


Sponsored by TESTOMAT PROJECT



Sponsored by IEEE Computer Science Sweden Chapter


Sponsored by City of Västerås