What Works for Whom? Second Edition

Roth and Fonagy have tackled an enormous and difficult question with all the nuance and intellectual muscle required in their second edition of What Works for Whom? They have not shied away from deeply examining the pros and pitfalls of quantitative psychotherapy research, and they have scoured the English language evidence thoroughly in their search for randomized controlled trials and meta-analyses. They have also organized their book by disorder, primarily relying on categorical definitions of disorder, as found in DSM-IV-TR¹, though disorders can be easily matched to their ICD-10 counterparts². The book has a tremendous potential for good use and misuse, and the authors are aware of this, taking pains in many places to point out the limits of what their findings do and do not support, especially in terms of managed care. This book is not for a lay audience, and general readers might quickly find themselves lost in a whir of scientific terminology, nuanced philosophical debate, and truckloads of statistics. For anyone with an interest in psychotherapy research of a quantitative nature, or for anyone with an interest in the philosophy of science, this is an excellent discussion of theoretical issues and a timely, exhaustive reference.

The book starts with an effort at "Defining the Psychotherapies," accepting Strupp's definition of "an interpersonal process designed to bring about modifications of feelings, cognitions, attitudes and behavior which have proved troublesome to the person seeking help from a trained professional." They next point out that Kazdin identified over 400 therapies in 1986 and state they will focus on "major orientations," defined as psychodynamic psychotherapy, behavioral and cognitive-behavioral psychotherapy, interpersonal psychotherapy, strategic or systemic psychotherapies, supportive and experiential psychotherapies, group therapies, and counseling (p. 6). The remainder of the chapter goes on to define these schools, and steers the reader toward seminal works by founders or modifiers of the various schools.

Chapter 2, "Research and Practice: Methodological Considerations and Their Influence on This Review" is, in my view, the foundation of the book and possibly its most important contribution to the theoretical literature on psychotherapy research. I would insult the chapter by attempting to summarize it, but it does tackle a number of important issues which can be made plain. Psychotherapy is a very complex interaction between two or more people. It involves a sufferer and an individual attempting to understand and bring relief. These individuals are informed by their life experiences, temperaments, expectations, and, in the case of the therapist, theoretical orientations within psychotherapy. The number of potential variables which need to be controlled in order for a quantitative approach to produce meaningful results is fiendishly large.

For instance, what and who defines suffering? The suffering reported by the individual is frequently at odds with what is observed, but quantitative methods rely on observation. How do we know when therapy proper is taking place, as opposed to say, a sympathetic hearing? Psychotherapy can be defined in a very broad way, as in the Strupp definition, but operationalizing it for research purposes is difficult. Then the issue arises, even once a technique can be recorded in a manual, and even if an inexperienced therapist is fully adherent to the letter of the manual, something of the spirit may get lost. What are the differences between people recruited into psychotherapy trials through newspapers compared with those who present with difficulties at clinics? We know very little about what motivates someone with psychological difficulties to seek help from a clinic versus responding to an ad, therefore it is difficult to generalize from research findings to clinical practice.

There are also questions about research populations. On what basis can we make generalizations from a highly selected research population to a completely unselected clinical population? Research populations need to be as homogenous as possible, for example, depressives only, with no comorbid anxiety, or substance abuse, so that the study minimizes confounding variables and maintains internal validity. However, the clinic takes all comers, with all degrees and combinations of symptomatic difficulties. Studies of efficacy, based on homogenous clinical samples studied under controlled research conditions do not establish effectiveness, which is the ability of a modality to handle the problems typically seen in a real-world clinic, where comorbidity of 50% to 90% between DSM disorders is common (p. 487).

Questions about placebo controls or other control interventions are important as well. Sitting on a waiting list or, "doing nothing," is currently one of the controls used, with assessment instruments applied to wait-list controls and individuals in therapy. Just about every psychotherapy looks good against doing nothing, and effect sizes diminish remarkably when two therapies are compared head-to-head. Allegiance effects are also known, whereby the preferred therapy, often CBT, is administered by experienced therapists, versus the "control therapy," which is often done by inexperienced therapists (p. 95). Regression analysis virtually eliminates the superiority of the preferred modality.

The questions go on and on, with very deft handling by the authors, who frequently conclude that there are neither easy nor self-evident replies. Quantitative approaches are limited in what they can tell us. Roth and Fonagy further note that there are psychotherapeutic schools, such as psychoanalytic or psychodynamic, which thoroughly repudiate quantitative approaches as wrong-headed, preferring qualitative, in-depth case studies. As a result of this bias, the voice of psychodynamic therapy is notably absent in the book. The authors appeal to psychodynamic practitioners to consider the potential usefulness of controlled trials in proving the usefulness of their approaches. Peter Fonagy, the Freud Memorial Professor of Psychoanalysis at University College of London, can hardly be said to be a detractor of psychodynamic approaches, but he clearly swims against the psychoanalytic stream with his interest in quantitative research. The second chapter will be of greatest interest to philosophers and students of quantitative methodology, and this is why it garners the lion's share of this review.

The third chapter is largely a warning to those who fund mental health services not to simplistically handle evidence. For instance, good evidence shows that depression responds to 16 sessions of CBT. A mental health planner might conclude that only CBT will be funded for depression, for up to 16 sessions. But good evidence also shows that people who have only had 16 sessions have only a 25% chance of being well one year later, while those getting maintenance therapy are more likely to avoid a relapse. But maintenance therapy is expensive. Economically, it may make more sense for an insurance company to limit treatment to 16 sessions and then not renew the individual's benefits because of their high risk of relapse.

Further, absence of evidence for a particular brand of therapy is not the same as proof of inefficacy, and what counts as evidence is very much in the eye of the beholder, whether that is the therapist, researcher, patient, or payer. Roth and Fonagy point out that rigid funding policies would eliminate second line treatments, necessary for those not responding to initial approaches. These policies would also dampen clinical innovation. There would be little point innovating if it wouldn't end up being paid for due to its non-evidence-based status.

Chapters 4 through 15 are the meat of the book, where Roth and Fonagy explore various specific disorders, looking at published controlled trials comparing various therapies to controls or else to each other head-to-head. The main message that jumps out from these chapters is that the proponents of CBT have clearly done their homework. Researchers have tried CBT in a randomized controlled trial for virtually every psychiatric disorder, and it often produces positive results, particularly for depression and anxiety, the two sorts of disorders which have always been most responsive to psychotherapy. Other modalities have a long way to go, particularly psychodynamic, that is, if their proponents ultimately decide that there is anything to be gained from a quantitative effort.

Chapter 16, "The Contributions of Therapists and Patients to Outcome," is another interesting exploration of both published literature and theoretical discussion. It turns out to be rather hard to account for why different therapists produce detectably different outcomes (p. 475). Studies looking at associations typically have too few patients per therapist to detect meaningful differences. Studies looking at therapist experience tend to confound the fact that experienced therapists usually end up with the most difficult patients, so on paper may look much less effective than novice therapists. To date, all studies of therapist characteristics have been in the secondary analysis, not built into the initial design features of the study, which severely limits their value. The study which randomizes patients to therapists, rather than to therapeutic modality, has not yet been undertaken.

The book's "Conclusions and Implications" (Chap 17) are again long, nuanced, and replete with cautions, all well justified by the discussion. Furthermore, even when only one disorder is cornered, it is clear that the symptoms themselves could be the result of numerous causal pathways, even if they look broadly similar across patient or study populations. Thus far, psychotherapy research has looked only at diagnoses, not at whether particular psychotherapies may be uniquely suited to particular causal factors such as childhood abuse. Crucially, they also note that researchers only tend to publish results showing a positive difference due to their espoused intervention. Negative or neutral studies get filed away, so the scientific literature is replete with publication bias.

There is much more I'd like to have discussed in this book, but it's just too jam-packed for a short review to do it adequate justice. The issues are of critical relevance to those who seek psychotherapy, those who provide it, and those who pay for it. There are also warnings and guidance to those who wish to conduct adequate research using an RCT paradigm. Payers like insurance companies look to shibboleths like the American Psychological Association's "Empirically Supported Therapies" for guidance and possibly as a defensive shield. To gain the APA's imprimatur, a therapy has to have been proven by RCT. Unsurprisingly, CBT has far and away been the winner, bolstered by legions of researchers who ascribe to the cognitivist presuppositions which currently undergird modern psychology yet are seldom scrutinized critically. As a result, access to psychotherapy has massively eroded across the United States, and patients are the losers. It is interesting that the U.K. Department of Health chose to reject the EST approach in its mental health policy (p. 51). A close reading of Roth and Fonagy is an important tonic, cutting through a lot of dogma and forcing the reader to confront, again and again, that the answer to the question "What works for whom?" if phrased simply would be, "Some things, some of the time, for reasons which aren't yet very clear."

Reference List

1. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision. Washington, DC: American Psychiatric Association, 2000.

2. World Health Organization. International Statistical Classification of Diseases and Related Health Problems 10th Revision [Web Page]. 2003; Available at http://www3.who.int/icd/vol1htm2003/fr-icd.htm.

Robert Tarzwell is in his 5^th and final year of psychiatry residency at Dalhousie University in Halifax, Canada. He is interested in the philosophy of psychiatry. He is also interested in critical approaches to psychiatric literature, particularly randomized controlled trials of drugs and psychotherapies. His main clinical interest is intensive dynamic psychotherapy.