Rebuttal to critique of "ABA is NOT Effective: So says the Latest Report from the Department of Defense"

Author’s Note: “I’d like to thank Therapist Neurodiversity Collective for giving me the opportunity to respond to reader critique of the 31-page 2020 The Department of Defense Comprehensive Autism Care Demonstration Annual Report 2020 as well as my blog, ABA is NOT Effective: So says the Latest Report from the Department of Defense.” – Barry R. Nathan, Ph.D.

As a reminder, the 2020 Department of Defense Comprehensive Autism Care Demonstration Annual Report was commissioned in response to a request by the 2016 U.S. Senate to the Department of Defense (DoD), to study the effectiveness of the Autism Care Demonstration (ACD). The DoD spends over $1.5 BILLION on ABA services, at an average annual cost of over $23,000 per autistic participant. At the conclusion of their two-year study, the report concluded:

“… these findings demonstrate that … the delivery of ABA services, is not working for most TRICARE beneficiaries in the ACD.”; and

“ … the Department remains very concerned about these results, and whether the current design of this demonstration, as well as ABA services specifically, is providing the most appropriate and/or effective services to our beneficiaries diagnosed with ASD.”

**ABA is ineffective! And, it’s abusive!**

My blog, which summarized the conclusions of the 2020 report was disputed by many in the ABA community. For example,

“The DoD/TRICARE report isn’t a study in any real sense of that word.” This is FALSE. In fact, they included 3794 participants who had received at least 18 months of ABA services; 2,183 from the eastern region of the United States, and a replication in the western region of the United States with 1,611 participants. As such it is not merely the largest study of the effectiveness of ABA, it is the only large-scale test of ABA The conclusion that the report “isn’t a study in the real sense of the word,” seems to be predicated on objection to the report’s outcome metric, the PDD Behavior Index (more on their choice of the PDDBI subsequently). One might disagree with a conclusion based on the choice of a metric, but this does not mean it wasn’t a true research study. In fact, it was a large-scale study, something no ABA research study can claim.
“The PDDBI is “an incorrect metric…. The use of the PDDBI as a measurement should be very concerning. No conclusion should be drawn for the report.” This is WRONG. The PDDBI is a highly reliable and valid measure that is used in numerous peer-reviewed research studies with autistic children. Its publisher is WPS (formerly Western Psychological Services), with a global reputation for assessments in areas of autism, speech and language, school psychology, and occupational therapy. According to its publisher, its purpose is to assess responsiveness to interventions in autistic children. Therefore, the selection of PDDBI was appropriate given the responsibility of the DoD to evaluate the effectiveness of ABA as an intervention. It’s not used in ABA literature, but that doesn’t mean it’s an inappropriate measure for what the purpose of the DoD report was.
The DoD study “reported on a measure of SYMPTOMS and not skill acquisition.” This is MISLEADING. The study commissioned by the DoD was specifically intended to determine whether ABA impacted presented symptoms of autism. The report found that ABA did not do this. If skill acquisition from ABA in a controlled or therapeutic session does not generalize to situations outside of therapy, then what is the point of the ABA therapy? The point of any therapy is to impact symptoms in social, academic, or family situations. Showing skill acquisition alone may be a relevant outcome in ABA research, but it is not a measure of practical effectiveness. Moreover, according to Donald Kirkpatrick’s 4-level framework for evaluating training effectiveness: 1-Reaction, 2-Learning, 3-Behavior, and 4-Results, skill acquisition would be only at Level 2-Learning. Whereas reducing symptoms would be Level 4-Results.

“Additionally, the average intensity of the services for the sample was 6 hours/week — far lower than the effective intensities from the outcome literature.” This, too, is MISLEADING. Assuming the literature is correct that 35-40 hours of one-to-one ABA services is necessary for ABA to be effective, then what this report shows is that in reality, outside of research studies, actually receiving 35-40 hours of intensive ABA is extremely rare. In the 2020 Report, the average hours of ABA services were 19.9 hours in the western region of the U.S., and 23.7 hours in the East, for 1,611 and 2,183 beneficiaries, respectively. That’s a lot of 1:1 time with a certified ABA provider; yet despite this large amount of time, the researchers concluded it was not effective.

But more to the point, the researchers in the 2020 report were sensitive to this critique raised of the 2019 report. In response, they looked at the relationship between hours of ABA therapy and effectiveness scores. They found a negative correlation! In other words, the more ABA an autistic child received, the worse he scored on the PDDBI. This effect was found in both of the eastern (n=2,163) and western (n=2,183) U.S. samples. This finding though only a correlation is exactly the opposite of what one would expect if ABA was effective.

Finally, a quick comment about the CASP (Council for Autism Service Providers) letter to the Department of Defense, which was a response to the initial 2019 DoD Comprehensive Autism Care Demonstration Annual Report. The authors of the 2020 DoD Report note that they took critiques by ABA stakeholders into consideration in their 2020 data analysis, and their updated literature review. Yet despite, and more likely because of these considerations, they reached the following conclusion, “ABA services do not meet the TRICARE hierarchy of evidence standard for medical and proven care.”

Barry R. Nathan, Ph.D., is the Advocacy Initiative Co-Chair for the Therapist Neurodiversity Collective. Barry is an organizational psychologist and founding director of Concierge Pittsburgh, a regional initiative to retain and attract African American professionals in Pittsburgh.
barry@concierge-pgh.org
(412) 901.2685

The Therapist Neurodiversity Collective has strong ethical concerns and philosophical differences pertaining to the use of Applied Behavioral Analysis (ABA) on human beings, including ABA-derived therapy models and relationship-based therapy models paired with ABA. Our members pledge to provide therapy in manners that uphold the dignity of the individual while promoting self-determination. We are troubled that ABA is performed on pediatric, adult, and elderly populations without consent from the humans on the receiving end.

Autistic people also object to the use of ABA on humans.

“ASAN‘s objection is fundamentally an ethical one. The stated end goal of ABA is an autistic child who is “indistinguishable from their peers”—an autistic child who can pass as neurotypical. We don’t think that’s an acceptable goal. The end goal of all services, supports, interventions, and therapies an autistic child receives should be to support them in growing up into an autistic adult who is happy, healthy, and living a self-determined life.”
Julia Bascom, Executive Director, Autistic Self Advocacy Network

In conclusion: ABA is ineffective. The United States Department of Defense says so. ABA is inhumane and abusive. Autistic people say so. Disabled people say so. Previous clients say so. There are far more trauma-informed, ethical and respectful forms of therapy available. It’s time for therapists who are continuing to employ ABA as an intervention to just stop.

Author
Recent Posts

Guest Author