Part one: apply the GRADE approach
Applying the GRADE approach to Dawson et al:
- Risk of bias: bias occurs when the results of a study do not represent the truth because of inherent limitations in design or conduct of a study. In the study by Dawson et al randomisation was conducted, ensuring balanced IQ and gender rations between the groups. Blinding is not possible for this intervention because both recipients and parents will notice. The assessments were performed by someone naïve of the intervention. And all data is presented in the paper. Decrease level of certainty: 0.
- Imprecision: the number of participants included in the analysis (n=45) may be small considering the differences between individuals, but results in some outcome measures show significant differences (but wide confidence intervals). Decrease level of certainty: -1.
- Inconsistency: the study used different outcome measures which show a variety of results. Moreover, the certainty of evidence is higher when it can be shown to be consistent in several studies. Decrease level of certainty: -2.
- Indirectness: evidence is most certain when studies directly compare the interventions of interest in the population of interest, and report the outcome(s) critical for decision-making. Certainty can be rated down if the patients studied are different from those whom the recommendation applies. Indirectness can also occur when the interventions studied are different than the interventions offered in real-life situations. Indirectness also occurs when the outcome studied is a surrogate for a different outcome – typically one that is more important to patients.
- Patient population: the patient population, children between 18 and 30 months of age who are diagnosed with ASD, is the population for which current recommendations advice screening for ASD and early interventions.
- Intervention: the intervention is provided in a toddler’s natural environment (at home) and provided by trained therapists and parents. The intervention is provided in consistency with the National Research Council’s recommendation.
- Outcomes: there are multiple outcome measures used, which evaluate the effects on IQ, learning, repetitive behaviour and adaptive behaviour. Although these outcomes are related to performance on behaviour that is part of the definition and diagnosis of ASD, you may question whether these are the outcomes that are relevant to patients.
Decrease level of certainty: 0
- Publication bias: difficult to assess, because you need to make inferences about missing evidence. It seems that the authors report all the data, but statistical and visual methods should be used to check this. Decrease level of certainty: 0.
- Magnitude of effect: the study shows a large effect in some outcome measures. Increase level of certainty:+2.
- Dose-response gradient: difficult to assess because all participants received the same (high) intensity of treatment. Increase level of certainty: 0.
- Residual confounding: there may be some unidentified confounding, but difficult to assess its potential influence on the magnitude of effect. Increase level of certainty: 0.
Overall, the strength of the evidence seems to be moderate.
Part two: critical appraisal of background theory (Theory of Mind and interpretive frames)
The interpretation of studies that collect evidence on the impact of ABA on children with autism as showing that ABA has the potential to cure autism depends on an underlying view on the nature of autism. The outcome measures that are currently used in assessing the effects of treatment are based on symptoms and behaviour, not on underlying neurobiological processes. Therefore, you need to infer, based on current data and ideas on the nature of autism, that these results show something about the effects on the processes that constitute autism itself.
According to the Theory of Mind (ToM) hypothesis, developed by Uta Frith, the process that is disturbed in autism is the development of the capacity to understand what other people mean when they say something or behave in a certain way. In other words, the ability to understand the world from someone else’s perspective while realizing that this may be different from your own perspective. This ability is generated by connections in neural networks. According to this hypothesis, the social, behavioural, and communicational deficiencies of children with autism are only the consequences of disturbances in this process. Therefore, treatments that aim to reduce these deficiencies are reducing symptoms but are not curing the disease. To determine whether ABA cures autism we need to study the neural networks involved and develop tests to measure changes in the activity of these networks. This is not yet done by studies that evaluate the impact of ABA, like the one by Dawson et al.
The interpretive frames of stakeholders do also have a relation with ToM. All stakeholders seem to hold to a certain background theory on what autism is (a certain set of behaviour), although they disagree on whether you should interpret this as abnormal and / or a medical condition. If they would be informed of the ToM they may acknowledge that this behaviour is the consequence of differences in neural development, which may lead them to revise their view on autism (e.g. neurodiversity advocates: autism is part of normal variation in structure and functioning of the brain, but could it partly be seen as a disturbance?; e.g. ABA advocates: autism is not just pathological, but part of a continuum of different ways of how neural networks develop?).