One key challenge in medical inference are the enormous sums at stake and the hence inherent vested interests. The reliability of all items of available evidence is hence to be carefully assessed. Intuitively, the greater the variety of sources and kinds of evidence which points in the same direction, the higher the assessed reliability and thus stronger the evidence confirms a hypothesis, ceteris paribus. This intuitive thought is known as Variety of Evidence Thesis.
Previous work (Bovens and Hartmann, Bayesian Epistemology. Oxford University Press, 2003, Claveau 2013) on this thesis reports surprising failures in which the less diverse body of evidence for a hypothesis confirms the hypothesis more strongly than the more diverse body of evidence. Barbara Osimani and I have developed a framework for the formal evaluation of the Variety of Evidence Thesis under more realistic modelling assumptions regarding the reliability of scientific instruments which are closer to statistical practise. The upshot is that the Variety of Evidence Thesis fails mostly for borderline or counter-intuitive cases and holds otherwise. However, when reliability is modelled exogenously, then the Variety of Evidence Thesis does hold and it can be given a formal Bayesian justification, as I show in an — as of yet — unpublished manuscript.
These papers are embedded in an overarching framework aiming to provide a formal analysis of statistical inference, see Landes et al. 2017. The aim here is twofold: on a more theoretical level, formal epistemology should provide a sort of lingua franca, where different conceptualisations of error and different statistical techniques can be discussed and investigated; on a more practical level, we aim to use formal epistemology as a framework for the incorporation of various evidential dimensions in the overall assessment of investigated hypotheses.
Within this greater projected I visited George Masterton — based at Lund university — who is an expert in judgement aggregation and the philosophy of science. He is involved in the application of Laputa which is a research tool and a sandbox environment for simulating the attainment of knowledge in social networks, see (Masterton2013, Masterton 2014). Having successfully cooperated (Landes & Masterton 2017), I visited George Masterton in Lund to apply Laputa to the problem of modelling reliability of evidence in medical inference and how different concepts of the notion of reliability influence rational — Bayesian — beliefs in a social world.
Work on a jointly-authored manuscript has begun. Barbara Osimani has kindly agreed to participate in the joint effort. We are looking forward to joyful and labor-intensive period of readings, discussions and writing.