Colloquium Computer Science - Dr. A. Gambi

Title: Automated Testing of Complex and Autonomous Software Systems

Abstract:

In this talk, I will give a general overview of my research on testing complex and autonomous software systems before delving into one of my latest research topics, i.e., effectively testing self-driving car software.

My main research goal is to ensure that the software controlling complex and autonomous software systems, such as cloud-based applications and self-driving cars, is reliable and does not cause safety or other issues. In parallel to those topics, I investigate methods for optimizing continuous integration systems in the cloud, carving tests from system executions, addressing test quality issues, and improving computer science education using gamification.

To test self-driving car software, I develop techniques that combine search-based software engineering, machine learning, and procedural content generation for systematically generating realistic and relevant driving simulations that are suitable as test cases.

Currently, there is no standardized procedure to test automated vehicles, and testing self-driving cars by putting them on trial in the real world is ineffective, expensive, and dangerous. Using driving simulations to implement virtual tests is a more viable alternative for testing self-driving car software. However, manually creating suitable simulations is laborious and difficult, while automatically generating them from real-world data assumes representative data availability.

In practice, this assumption is not met, so I propose using alternative sources of data from which generate virtual tests. Specifically, I consider publicly available police reports that describe car crashes in textual form and use natural language processing to extract the necessary information to re-enact the car crashes. Next, I automatically generate critical driving simulations by combining procedural content generation and trajectory planning and derive virtual tests from the generated simulations. The extensive evaluation of the approach showed that simulated car crashes match the police reports' descriptions, and the virtual tests derived from such critical scenarios effectively stressed the test subjects in different ways compared to tests derived from non-critical scenarios.