To Err is (only) Human. Reflections on How to Move from Accuracy to Trust for Medical AI

In this paper, we contribute to the deconstruction of the concept of accuracy with respect to machine learning systems that are used in human decision making, and specifically in medicine. We argue that, by taking a socio-technical stance, it is necessary to move from the idea that these systems are “agents that can err”, to the idea that these are just tools by which humans can interpret new cases in light of the technologically-mediated interpretation of past cases, like if they were wearing a pair of tinted glasses. In this new narrative, accuracy is a meaningless construct, while it is important that beholders can “believe in their eyes” (or spectacles), and therefore trust the tool enough to make sensible decisions.