Presentation at the Nordic Network for the Philosophy of Science 2017
On 27-28 January 2017, the workshop Drug Safety, Probabilistic Causal Assessment, and Evidence Synthesis took place at the LMU in Munich organised by Bennett Holman, Jürgen Landes, Barbara Osimani, Roland Poellinger and David Teira. The goal of the workshop was simple, yet ambitious: to provide a platform for scholars of various disciplines and decision makers to meet and discuss issues of causal assessment in drug safety.
In confronting such a multifaceted topic, relying on just one perspective is inadvisable, if not outright dangerous. Therefore a closer collaboration between experts of different fields and regulators is pivotal for improving our policies and our answers to crucial questions in pharmaceutical risk assessment. 21 presentations from speakers based in Australia, Austria, Germany, Great Britain, Italy, Luxembourg, the Netherlands, Norway, South Korea, Spain and Sweden o ered perspectives from a wide variety of angles. Such a broad perspective is especially important for philosophers involved in a research program which strives to develop a variety of tools to address methodological and epistemological issues in causal assessment in medicine, since such a program requires the interaction with, and thus knowledge of, the work of health professionals, methodologists, statisticians and epidemiologists. It also provided non-philosophers an opportunity to learn more about current philosophical work.
Barbara Osimani (MCMP/LMU) presented an inferential framework for the purpose of probabilistic causal assessment, which she is developing together with Jürgen Landes (MCMP/LMU) and Roland Poellinger (MCMP/LMU) within the ERC project: Philosophy of Pharmacology: Safety, Statistical Standards, and Evidence Amalgamation. This consists in a Bayesian network specically adapted to model epistemic dynamics in probabilistic causal assessment that blends together different perspectives that in standard Evidence Based Medicine are working relatively independently so far. Roland Poellinger further elaborated on the concept of exploiting a Bayesian evidence amalgamation framework to formally explore the interplay between heterogeneous evidence and the different components of a causal hypothesis in pharmacological risk assessment. Jürgen Landes strengthened the case for the Variety of Evidence thesis (the thesis that more varied evidence speaking in favour of a hypothesis confirms it more strongly than less varied evidence, ceteris paribus).
The CauseHealth project contributed two presentations: Rani Lill Anjum (Norwegian University of Life Sciences) and Elena Rocca (Norwegian University of Life Sciences) suggested an integrated framework in which post market monitoring, through the study of treatment failure, feeds pre-clinical and clinical research with mechanistic hypothesis, while Stephen Mumford (University of Nottingham) presented suggestive points in support of a dispositionalist stance regarding understanding of probability and synthesis of evidence. The EBM+ consortium, which seeks to make the role of evidence of mechanisms in the evidence appraisal process more explicit was introduced by Jon Williamson (University of Kent), who spoke about two research project related to evidence of mechanisms. For the Making Scientific Inferences More Objective project Felipe Romero (University of Tilburg) presented approaches in finding a middle ground in the current reform efforts to increase replicability in sciences.
Jeff Aronson (Oxford University) gave great insight on a crucial aspect of any discussion about causality in medicine and pharmacology, namely the definition of what a signal is. Ralph Edwards (Uppsala Monitoring Center, WHO) presented on the problem of discovering causality in pharmacovigilance from real-world data.
David Teira (UNED Madrid) argued that the shift toward evidential pluralism undergoing in regulatory agencies (at least in the U.S.A.) involves a de facto relaxation of regulatory paternalism, while Bennett Holman (Yonsei University) spoke on the problematic and unavoidable deep connection that runs between industry and science and how the former shapes the latter by means more subtle than one might expect.
Regarding the hot debate that is going on over Randomized Controlled Trials and its reputation as being the cornerstone of Evidence Based Medicine, Mike Kelly (Cambridge University) presented an alternative narration in form of the Medicines Adaptive Pathways to Patients, while Ulrich Mannsmann (IBE/LMU Munich) provided a detailed analysis on how to quantify the effect of and correct design-dependent bias in RCTs. Stephen Senn (CCMS, Luxembourg Institute of Health) addressed the benefits and limits of randomization in technical research and covered some fallacious critical arguments against RCTs. Adam La Caze (University of Queensland) examined three different approaches to amalgamating drug safety evidence, pointing out that there is still room for conceptual refinement. Jacob Stegenga (Cambridge University) defended a middle view between the claim that causal inference in medicine should be based on statistical evidence and the one that identify the focus in mechanistic evidence, showing how reasoning about causal relations by appealing to both is vindicated by our best general
theory of inference.
Martin Posch (Medical University of Vienna) underlined the challenges that decision makers have to face in data interpretation, in which a balance between a qualitative and a
quantitative approach have to be reached.
Last but not the least there were the voices of the regulators and decision makers themselves, who provided invaluable insight on the mechanics and the reasoning behind the everyday decision-making processes in pharmacovigilance, which in itself was of tremendous interest to all participants. Norbert Benda (Bundesinstitut für Arzneimittel und Medizinprodukte) discussed some principles for a decision-theoretic framework in drug safety and benefit risk assessments, the related challenges and consequences and contrasted Bayesian and frequentist reasoning in risk assessment. Brigitte Keller-Stanislawski (Paul Ehrlich Institut and PRAC) discussed aspects of the European Union regulatory approach for benefit risk assessment post-authorization and gave an overview of the legal framework for decision making in the EU. Beth Shaw (NICE) outlined the use of
heterogeneous knowledge in guideline development and some challenges and potential solutions with this approach, focusing in particular on possible alternatives to Randomized Controlled Trials.
For the first time, an event brought philosophers, statisticians and decision makers together to discuss drug safety. The lively discussions throughout the workshop covered a wide range of issues which are ongoing and continue to deepen our understanding. Together we shall face arising challenges in drug safety by drawing on the expertise of researchers with a great variety of backgrounds.
— Report by Alessandro Demichelis —
Public Lecture at LSE on: “Exact replication or varied evidence? Reliability, robustness and the reproducibility problem”
Barbara Osimani will present a talk at LSE on: “Exact replication or varied evidence? Reliability, robustness and the reproducibility problem”. London, 22nd November, 5 pm – 6.30 pm.
“The “Reproducibility Project: Psychology” by the Open Science Collaboration caused some stir among psychologists, methodologists as well as scientists, since less than half of the replicated studies succeeded in reproducing the results of the original ones. The APA has attributed this result to hidden moderators that rendered the replications ineffective. Also publication bias and low power have been identified as possible sources for such mismatch. While some analysts have provided formal confirmation for the plausibility of such explanations (Etz and Vandekerkhove, 2016), others have further insisted on the problem of noisy data and suggested that “to resolve the replication crisis in science we may need to consider each individual study in the context of an implicit meta-analysis” (Andrew Gelman).
I investigate these positions through the lenses of Bayesian epistemology, and in particular of recent results on the Variety of Evidence Thesis. This approach will turn out to be fruitful in investigating the interaction of reliability, independence of evidence, and replication in scientific inference and, more broadly, will cast a new light on the debate between advocates of a pluralist methodology in medical research, who insist on supporting hypotheses through various sources of evidence, and the contending view, represented by the Evidence Based Medicine paradigm, which relies on an “elitist” approach, where “best evidence” is searched for and exact replication is highly welcome”.
Barbara Osimani will present the project at the European Medicines Agency (EMA), London, on 23rd November 2016.
[Update (16 Feb, 2017): For late submissions please contact the editors urgently.]
Samuel C. Fletcher, Jürgen Landes & Roland Poellinger are guest-editing a special issue on “Evidence Amalgamation in the Sciences” with Synthese. To submit a manuscript login into Synthese’s editorial manager before 15. FEB. 2017 and choose to submit to this special issue. Here’s the call
This proposed special issue provides a forum for researchers from diverse scientific and philosophical perspectives to discuss evidence amalgamation, its methodologies, its history, its pitfalls and its potential. In particular, we encourage critical analysis of past and current practice highlighting inadequacies and advances.
Commonalities and differences of methodologies for amalgamation across different sciences.
The roles of models and mechanisms in evidence amalgamation.
Normative – epistemological and ethical – issues concerning the amalgamation of evidence.
Basic issues in the philosophy of science relevant to evidence amalgamation such as: unification, justification, explanation, coherence, reductionism, incommensurability, inductive methods, and causal inference.
Decision and policy-making with large heterogeneous bodies of evidence.
Methodology of meta-analysis and systematic reviews.
Foundational issues of data integration and fusion within and across disciplines.
To discuss a possible contribution drop an email to the guest-editor of your choice.
Today, the American Institute of Physics published a proceedings volume containing a contribution by Jürgen Landes and Jon Williamson. In which they investigate an objective Bayesian approach to evidence amalgamation: Doi: 10.1063/1.4959048 (Open Access). See our Writings page for the full citation.