3rd IEEE Workshop on NEXt level of Test Automation 2020
NEXTA 2020

Workshop Summary

Test automation has been an acknowledged software engineering best practice for years. However, the topic involves more than the repeated execution of test cases that often comes first to mind. Simply running test cases using a unit testing framework is no longer enough for test automation to keep up with the ever-shorter release cycles driven by continuous deployment and technological innovations such as microservices and DevOps pipelines. Now test automation needs to rise to the next level by going beyond mere test execution. The NEXTA workshop will explore how to advance test automation to further contribute to software quality in the context of tomorrow's rapid release cycles. Take-aways for industry practitioners and academic researchers will encompass test case generation, automated test result analysis, test suite assessment and maintenance, and infrastructure for the future of test automation.


The NEXTA workshop will be organised in a fully virtual setting. Nevertheless, we want to stay as close as possible to the spirit of the workshop (i.e. encourage discussions, active on social media). But what implications does this have for you?

(1) Live but virtual.
We ask authors to give a real virtual presentation, instead of playing a pre-recorded video. However, if you feel uncomfortable about giving a virtual presentation, contact the organizers and we will gladly accommodate.

(2) Full time slots.
We will keep the time-slots as originally planned. (Full presentation = 20 minute + 10 minutes questions; Short presentation = 15 minutes + 5 minutes questions). The keynote will be dropped however. We are fine-tuning the time-schedule right now and get back to you shortly.

(3) Date and Time zone.
The workshop itself will be held on Saturday, October 24th. We will stay in the time-zone of the conference which is (GMT+01:00) Greenwich Mean Time : Lisbon. This implies that for most of us, the actual times will differ by 1 hour. (Note that Europe switches to daylight savings time on Sunday, so watch your respective clocks)

(4) Platform.
The platform for the virtual event will most likely be zoom. Precise details will be supplied a few days before the event.

(5) Active Q&A.
Despite the virtual nature we will do our best to emulate the spirit of a live workshop. Each presenter will be asked to serve as a discussant for another paper in the same session. A discussant will read the paper beforehand and prepare a series of questions to initiate the discussion. If you are uncomfortable about serving as a discussant, contact the organisers.

(6) Awards.
NEXTA has a tradition of handing out awards to encourage interaction.
 - Most Viral Tweet Award [#nexta2020]
   Was awarded to Maaret Pyhäjärvi.
 - Best Questions Award
   Was awarded to (yes a double award) Maaret Pyhäjärvi.
 - Best Presentation Award
   Was awarded to Felix Dobslaw.
Participants will be asked to participate in voting for these awards.


All times in (GMT+01:00) Greenwich Mean Time : Lisbon

  • 9.00-9.15 Welcome + Session: Process Improvement
    • 09:00 - 09:15 — Welcome address and practical arrangements — Kristian Wiklund, Pasqualina Potena, Adnan Causevic, Serge Demeyer — Organisers
    • 09:15 - 09:45 — Test Automation Process Improvement in a DevOps Team: Experience report — Yuqing Wang, Maaret Pyhäjärvi and Mika Mäntylä — Full presentation
    • 09:45 - 10:05 — Test Automation Challenges for Application Landscape Frameworks — Nils Wild, Horst Lichter and Peter Kehren — Short presentation
    • 10:05 - 10:25 — Test Automation Improvement Model - TAIM 2.0 — Sigrid Eldh — Short Presentation
  • 10:30 - 11:00 Coffee Break
  • 9.00-9.15 Optimisation
    • 11:00 - 11:30 — Optimization of automated executions based on integration test configurations of embedded software — Masashi Mizoguchi, Takahiro Iida and Toru Irie — Full presentation
    • 11:30 - 12:00 — Selective Regression Testing based on Big Data: Comparing Feature Extraction Techniques — Khaled Al-Sabbagh, Miroslaw Staron, Regina Hebig, Mirosław Ochodekand Wilhelm Meding — Full presentation
    • 12:00 - 12:20 — Runtime Prioritization with the Classification Tree Method for Test Automation — Barbara Jung and Peter M. Kruse — Short Presentation
  • 12:30 - 14:00 Lunch Break
  • 14.00-15.30 Test Generation
    • 14:00 - 14:30 — Optimizing the Parameters of an Evolutionary Algorithm for Fuzzing and Test Data Generation — Ralf Gerlich and Christian Prause — Full presentation
    • 14:30 - 15:00 — Boundary Value Exploration for Software Analysis — Felix Dobslaw, Francisco Gomes de Oliveira Neto and Robert Feldt — Full presentation
    • 15:00 - 15:20 — Test Data Generation for MC/DC Criterion using Reinforcement Learning — Ján Čegiň and Karol Rástočný — Short Presentation
  • Cofee 15:30 - 16:00
  • 16.00-17.30 User-Interface Testing + Closing
    • 16:00 - 16:30 — On the Industrial Applicability of Augmented Testing: An Empirical Study — Michel Nass, Emil Alégroth and Robert Feldt — Full presentation
    • 16:30 - 16:50 — Session-Based Recommender Systems for Action Selection in GUI Test Generation — Varun Nayak and Daniel Kraus — Short Presentation
    • 16:50 - 17:30 — Award Ceremony: Best Question Award; Best Presentation Award; Most influential tweet award — Organisers


Registration for the workshop should be done via ICST2020. Please refer to the following link: https://icst2020.info/

Travel information

Due to the Corona pandemic, NEXTA 2020 will be organised as a fully virtual event.

Important Dates

  • 6th of January, 2020

    Abstract Submission

  • 13th of January, 2020 (hard deadline)

    Full-Paper Submission

  • 27th of January, 2020


  • 4th of February, 2020

    Camera Ready

  • 24th of October, 2020


Call for Papers

NEXTA solicits contributions targeting all aspects of test automation, from initial test design to automated verdict analysis. Topics of interest include, but are not limited to, the following:

  • Test execution automation
  • Test case generation
  • Automatic test design generation
  • Analytics, learning and big data in relation to test automation
  • Automatic aspects management in test, progress, reporting, planning etc.
  • Visualization of test
  • Evolution of test automation
  • Test suite architecture and infrastructure
  • Test environment, simulation, and other contextual issues for automated testing
  • Test tools, frameworks, and general support for test automation
  • Testing in an agile and continuous integration context, and testing within DevOps
  • Orchestration of test
  • Metrics, benchmarks, and estimation on any type of test automation
  • Any type of test technologies relying on automation of test
  • Process improvements and assessments related to test automation
  • Test automation maturity and experience reports on test automation
  • Automatic retrieval of test data and test preparation aspect
  • Maintainability, monitoring and refactoring of automated test suites
  • Training and education on automated testing
  • Automated test for product lines and high-variability systems
  • Test automation patterns
  • Automated test oracles

NEXTA accepts the following types of original papers:

  • Technical Papers (max. 8 pages in IEEE format).
    Full papers presenting research results or industrial practices related to the next generation of test automation.

  • Tool Papers (max. 4 pages in IEEE format).
    Tool papers introduce tools that implement an approach to support the transition to the next generation of test automation. A tool paper submission must include either 1) a URL to a screencast of the tool in action, or 2) a runnable version of the tool for evaluation by the program committee.

  • Position and Experience Papers (max. 4 pages in IEEE format).
    Short papers introducing challenges, visions, positions or preliminary results within the scope of the workshop. Experience reports and papers on open challenges in industry are especially welcome.

Authors should submit a PDF version of their paper through the NEXTA 2020 paper submission site: https://easychair.org/conferences/?conf=nexta2020.
All accepted papers will be part of the ICST joint workshop proceedings published in the IEEE Digital Library.

Call for Papers can be downloaded as text or as a PDF document.


Serge Demeyer

University of Antwerp, Belgium
General Chair

Adnan Causevic

Mälardalen University (MDH), Sweden
Program Co-Chair

Pasqualina Potena

RISE Research Institutes of Sweden AB, Sweden
Program Co-Chair

Kristian Wiklund

Ericsson AB, Sweden
Program Co-Chair

Program Committee (tentative)

  • Serge Demeyer, University of Antwerp, Belgium
  • Adnan Causevic, Mälardalen University, Sweden
  • Pasqualina Potena, RISE Research Institutes of Sweden AB, Sweden
  • Kristian Wiklund, Ericsson AB, Sweden
  • Markus Borg, RISE Research Institutes of Sweden AB, Sweden
  • Sigrid Eldh, Ericsson AB, Sweden
  • Tanja Vos, Universidad Politecnica Valencia, Spain and Open University Netherlands
  • Marc-Florian Wendland, Fraunhofer Fokus, Germany
  • Machiel van der Bijl, Axini, Netherlands
  • Leire Etxeberria Elorza, University de Mondragon, Spain
  • Karl Meinke, The Royal Institute of Technology, Sweden
  • Mika Mäntylä, University of Oulu
  • Maurizio Leotta, University of Genova, Italy
  • Mohammad Ghafari, University of Bern, Switzerland
  • Peter M. Kruse; Expleo, Germany
  • Magnus C Ohlsson, System Verification, Sweden
  • Pekka Aho, VTT, Finland Rix Groenboom, Parasoft, Germany