Software systems may fail in production environments, causing system crashes, erroneous outputs, and overall system instability. Thoroughly testing software systems in development environments can reduce but not avoid failures, due to both the complexity of software applications, which may lead to a myriad of execution conditions impossible to sample exhaustively, and the many behaviors that emerge in production, which can be hardly predicted and exercised during development. This paper presents field-ready test cases, tests designed to run in production environments, aiming to proactively execute soft-ware components in yet unexplored execution scenarios, exposing error states before they result in system failures. Intuitively, the approach conceives the production environment as a testbed for opportunistically executing unit test cases that exploit the objects that become available as test data. The paper presents the results of a set of experiments with field-ready test cases that we produced for the JFreeChart and Apache Commons Lang libraries. Our field-ready test suites execute 64% of the faults that the original test suites miss, and exposes 33% of the missed faults.
Gazzola, L., Mariani, L., Orru, M., Pezze, M., Tappler, M. (2022). Testing Software in Production Environments with Data from the Field. In Proceedings - 2022 IEEE 15th International Conference on Software Testing, Verification and Validation, ICST 2022 (pp.58-69). Institute of Electrical and Electronics Engineers Inc. [10.1109/ICST53961.2022.00017].
Testing Software in Production Environments with Data from the Field
Gazzola L.;Mariani L.;Pezze M.;
2022
Abstract
Software systems may fail in production environments, causing system crashes, erroneous outputs, and overall system instability. Thoroughly testing software systems in development environments can reduce but not avoid failures, due to both the complexity of software applications, which may lead to a myriad of execution conditions impossible to sample exhaustively, and the many behaviors that emerge in production, which can be hardly predicted and exercised during development. This paper presents field-ready test cases, tests designed to run in production environments, aiming to proactively execute soft-ware components in yet unexplored execution scenarios, exposing error states before they result in system failures. Intuitively, the approach conceives the production environment as a testbed for opportunistically executing unit test cases that exploit the objects that become available as test data. The paper presents the results of a set of experiments with field-ready test cases that we produced for the JFreeChart and Apache Commons Lang libraries. Our field-ready test suites execute 64% of the faults that the original test suites miss, and exposes 33% of the missed faults.
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
