Machine learning–based decision support systems (DSS) are attracting the interest of the medical community. Their usage, however, could have deep consequences in terms of biasing the doctor’s interpretation of a case through automation bias and deskilling. In this work we address the design of DSS with the goal of minimizing these biases through the design and implementation of programmed inefficiencies (PIs), that is, features with the stated purpose of making the reliance of the human doctor on the DSS less obvious (or more difficult). We illustrate this concept by presenting a real-life medical DSS, called DataWise, embedding different PIs and currently undergoing iterative prototyping and testing with the medical users in two clinical settings. We describe the main features of DataWise, and show how different PIs have been conveyed by prompting doctors for multiple input and using qualitative visualizations instead of precise, but possibly misleading, indications. Finally, we discuss the implications of this design approach to naturalistic decision making, especially in life-saving domains like medicine is.
Cabitza, F., Campagner, A., Ciucci, D., Seveso, A. (2019). Programmed Inefficiencies in DSS-Supported Human Decision Making. In International Conference on Modeling Decisions for Artificial Intelligence (MDAI 2019) (pp.201-212). Springer Verlag [10.1007/978-3-030-26773-5_18].
Programmed Inefficiencies in DSS-Supported Human Decision Making
Cabitza F.
;Campagner A.;Ciucci D.;Seveso A.
2019
Abstract
Machine learning–based decision support systems (DSS) are attracting the interest of the medical community. Their usage, however, could have deep consequences in terms of biasing the doctor’s interpretation of a case through automation bias and deskilling. In this work we address the design of DSS with the goal of minimizing these biases through the design and implementation of programmed inefficiencies (PIs), that is, features with the stated purpose of making the reliance of the human doctor on the DSS less obvious (or more difficult). We illustrate this concept by presenting a real-life medical DSS, called DataWise, embedding different PIs and currently undergoing iterative prototyping and testing with the medical users in two clinical settings. We describe the main features of DataWise, and show how different PIs have been conveyed by prompting doctors for multiple input and using qualitative visualizations instead of precise, but possibly misleading, indications. Finally, we discuss the implications of this design approach to naturalistic decision making, especially in life-saving domains like medicine is.
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
