Dunja Dreesens

91 Text box 4: Frequency based induction: rolling dice (paraphrased from Hacking 2001) FREQUENCY BASED INDUCTION: ROLLING DICE The central reasoning in RCTs can be summarised simply as assuming that everything in the world (such as people, relations) behave like dice. That is, if dice roll perfectly, they are fair. If they are unfair – where one side is heavier than the others – they are biased. We’re aware of many types of bias a priori in real life situations: for example, known causes, confounders and differences between groups. An RCT of patients with therapy A versus patients with therapy B is like trying to make two similar dice as fair as possible, rolling them many times, and comparing the results of the rolls to show their tendency. The assumption is that by rolling two fair dice, one would expect to see similar results in the end. If this does not happen then the reverse must be true; one of the dice is unfair or biased; one of the faces is heavier than the other. In that case, one of the therapies was better at causing the outcome. We don’t know why (rolling dice doesn’t give us an explanatory mechanism), but we conclude that it just did, because we found an a priori ‘unknown’ bias. It is important to note that frequency-type reasoning doesn’t work for single case scenarios. A patient can get a first hip replacement only once, not a hundred times. It goes well or it goes wrong. There is no frequency about it. It’s like throwing a dice once: you’re never sure what will happen. You either throw a six or you don’t. Second, frequency-based reasoning aims to find simple causal correlations, independent of context. The question is whether these simple correlations hold true in real life. Different understandings of causality exist that could help us address this drawback (220). For instance, a network of complex causal relationships may be more realistic. This drawback is described as the efficacy paradox, where the different interference from non-specific effects (different from those controlled for between groups in a trial), measurement artefacts (that mimicked therapeutic effects in the trial) and regression patterns (such as the self-limiting nature of a disease) in real life can outweigh the specific effect found in a trial. This paradox may become especially apparent when inferring in the context of multimorbidity (221). Finally, and most importantly, although frequency-based reasoning works well for frequent events (large groups, many data points and long periods of time), such reasoning faces fundamental limitations when inferring in the single-case scenario: a single patient, a rare disease, a system intervention and a one-off event. This can be particularly challenging when recommendations based on frequency-type evidence alone are deployed to help decision making for individual patients or unique situations, such as a public health response to a disease outbreak (47). Given these drawbacks, it is worth noting that other types of reasoning to evade the epistemological problem of induction exist. In table 1, several alternative ways of reasoning are listed. They are mainly used in areas where frequency-based reasoning is particularly problematic, for instance in guidelines focusing on complex interventions, public and occupational health, rehabilitation, and social care and Chapter 5