A Qualitative Analysis of the Descriptions of Cognitive Behavioural Therapy (CBT) Tested in Clinical Trials of Depressed Young People

Depression in young people is a significant issue. It accounts for the greatest burden of disease in this age group [1] with adolescence and young adulthood the peak period for the emergence of new cases of depression [2]. The onset of depression in this developmental stage is associated with lifelong impairment, including poor physical health, problems with developing and maintaining good relationships, poor vocational attainment and achievement [3,4]. Further, depression is associated with an increased risk of self harm and suicide [5] and results in a reduction of potential and productivity into adulthood [68]. For many, early episodes of depression will develop into recurrent episodes in adulthood [9]. It is therefore critical to provide optimal treatment to this group [10].


Introduction
Depression in young people is a significant issue. It accounts for the greatest burden of disease in this age group [1] with adolescence and young adulthood the peak period for the emergence of new cases of depression [2]. The onset of depression in this developmental stage is associated with lifelong impairment, including poor physical health, problems with developing and maintaining good relationships, poor vocational attainment and achievement [3,4]. Further, depression is associated with an increased risk of self harm and suicide [5] and results in a reduction of potential and productivity into adulthood [6][7][8]. For many, early episodes of depression will develop into recurrent episodes in adulthood [9]. It is therefore critical to provide optimal treatment to this group [10].
Consistent with a range of international guidelines, the Evidence Based Clinical Practice Guidelines (EBCPG) for treating youth depression developed by beyondblue, The National Depression Initiative and endorsed by Australia's National Health and Medical Research Council (NHMRC) recommend that clinicians "provide cognitive behavioural therapy (CBT) or interpersonal therapy (IPT) as first line psychological treatment for moderate to severe depression" (recommendation, grade B: this indicates that there are one or two RCTs with low risk of bias or a systematic review/several pseudo RCTs with low risk of bias with generally consistent results; overall this means there is a body of evidence that can be trusted to guide practice in most situations [11]. CBT is the most frequently studied psychotherapy for depression in young people [12,13]. It aims to help clients to identify, explore and modify relationships between results. For example, The Treatment for Adolescent Depression study (TADS) [21], randomised young people to placebo, CBT, fluoxetine or combined CBT and fluoxetine. Results showed CBT alone was not statistically significantly different compared with placebo, fluoxetine or the combination of fluoxetine and CBT. Recent reviews suggest that, while still more effective than a range of comparison conditions, CBT has more modest effects than first believed [13,22].
The observed inconsistency in effect sizes may be due to the variations in treatment protocols across trials. CBT is generally regarded as a family of allied therapies [23] and it continues to evolve with a wide range of approaches and techniques variously included in different CBT treatment protocols [16,17]. Little investigation of which particular approaches and techniques are most effective, and for whom has been undertaken. A seminal review by Weisz [13,24] attempted to look at this issue specifically by examining whether interventions using a cognitive emphasis were more effective than those that did not. They found that this was not the case. At a similar time other reviews were published [22,25] that showed larger effect sizes for purely BT interventions compared with CBT. To be able to further investigate this issue, a greater understanding of the variation in the nature of CBT that is delivered is required. Our aim in this study was to examine the nature of CBT implemented across trials; this can only be done on the basis of what is described in the publication describing these trials and as such we also aimed to investigate the quality of the descriptions of CBT interventions across trials.
In undertaking this work, we are contributing to the translation of research knowledge into clinical practice in the area of CBT for treating depression in young people. In order for this body of literature to be of clinical value, it needs to be sufficiently detailed to facilitate translation and implementation. Specifically, detailed descriptions in peer reviewed literature of the interventions delivered in trials are required to be able to identify and implement interventions shown to be effective [26]. This step is essential in ensuring the translation from research into practice, in order for interventions to be tailored and contextualised to meet the needs of local populations and services. Tailoring interventions for local contexts requires an in-depth understanding of the theoretical underpinnings of the intervention and the basic processes through which the components of the interventions are proposed to operate [16]. The major aim of this study, therefore, was to examine the level of detail of descriptions of CBT interventions included in depression treatment trials in young people.

Methods
The review authors comprised of an expert in conducting systematic reviews in youth depression, experienced clinical psychologists, and a qualitative research methods expert.
The data for this study were the descriptions of CBT extracted from each of the trials of CBT for the treatment of depression in young people. These trials were all randomised controlled trials of CBT compared with any comparison condition for young people aged 12 to 25 with depression and sub-threshold depression which were selected located via electronic searches of MEDLINE, PSYCHINFO, EMBASE, and The Cochrane Central Register of Controlled Trials (CENTRAL), as well as ancestry searches of the reference lists of included trials.
To examine the level of detail used to describe CBT interventions in these trials, two independent review authors extracted data on 14 intervention description domains described below (based on the 8 item checklist developed by Hoffmann and colleagues to assess the completeness of descriptions of non-pharmacological interventions in randomised trials [27]. It should be noted that in some cases we only had access to descriptions of the intervention in the published trial, whereas in other cases the trial authors indicated in their manuscripts the availability of intervention manuals that were publically available on the Internet. We did not contact trial authors for verification of missing data or for access to intervention manuals. Domains one and two captured intervention dosage and described the number of sessions delivered and session length in minutes. Domains three and four captured the intervention schedule and described session frequency as the number of sessions per week and total intervention duration in weeks. Domain five described whether the intervention was delivered in individual or group format. Domain six and seven captured reporting of study interventionists and described whether the interventionists were qualified, students or a mix of the two and whether the interventionist's profession(s) was reported. Domains eight and nine captured fidelity to intervention and described whether it was measured and if fidelity results were reported. Domains one to nine are displayed in Table 1. Domains 10 and 11 captured whether the intervention was manualised and if so whether the manual was referenced, either published or unpublished. Domain 12 captured whether the study authors referenced a particular theorist explicitly related to the intervention delivered. Domain 13 captured whether the intervention manual had been previously tested or whether it was based on modified from a previously tested intervention manual. Domain 14 captured whether an intervention description was provided and if this was a module-by-module or overall summary description. Domains 10 to 14 are displayed in Table 2 along with a score indicating the total number of intervention description domains the trial failed to report on.

Constant comparative methods
To explore the nature of CBT interventions implemented in the included trials we used the constant comparative method. The constant comparative method is a method for analyzing data in order to develop a grounded theory. The units of data for this study were the extracted descriptions of CBT from each of the trials. We undertook a process that included open coding, axial coding and selective coding according to the method described by Strauss and Corbin [28]. Open coding involved examination of the data with reference to our aim of understanding the nature of CBT that is implemented in trials. We developed a list of provisional codes to categorise the types of techniques typically included in the CBT implemented in trials. These included cognitive restructuring, behavioural activation, relaxation, problem solving and social skills training. We then examined the shared characteristics of the descriptions with the same code and as a result categories were developed with criteria formulated for each category in order to facilitate comparisons between descriptions of CBT [28]; this is the process of axial coding. As described by Taylor and Bogdan [29] this simultaneous process of coding and analyzing units of data by continually comparing each new unit of data facilitated a process of ongoing refinement of the content and definition of our categories and how these relate to each other. Only once new data failed to reveal any new information the categories were described as saturated. Finally three core categories were identified that best allowed for a description the nature of the data in terms of the categories we started with to facilitate an explanation of the potential difficulties that might be encountered when translating CBT from research into everyday clinical practice.

Results
A summary of the information about the CBT interventions that were implemented in each trial is included in Tables 1 and 2. Table 1 describes the specifics of the delivery of the intervention while Table 2 describes whether the intervention was manualised and theory based. Across the two tables are 14 intervention description domains. There was noteworthy variability in reporting of trials across the 14 domains.
Of the 34 included studies, only two provided information for each of the 14 intervention description domains. Four of the 34 trials had missing information for at least half of the domains and most (68%) had information missing for between two and six of the domains. The most frequent missing items related to measuring and reporting treatment fidelity (56% incomplete). Six out of 34 (18%) did not state if their intervention manual had been previously tested, or if it was based on or modified from a previously tested intervention and 56% did not reference a source document for their intervention manual that could be obtained. Regarding intervention dose, 21% of trials failed to report session length. Twelve trials and nine trials referenced the work of Beck [30] and Lewinsohn [31] respectively ( Table 2).
Each of the descriptions of CBT from our corpus of included studies was subject to close reading, comparison and anlaysis according to the constant comparative method [28]. The descriptions of CBT were variously based on the desriptions in the trial publications and in some instances on the intervention manuals where these were publically available. The core categories that we identified in the analysis of the data included:

1.
Full CBT: Delivery included cognitive techniques (i.e. cognitive restructuring), behavioural techniques (i.e. behavioural activations) and additional components such as problem solving, social skills training, or relaxation were described as being delivered.

2.
Partial CBT: Delivery included cognitive techniques, and/ or behavioural techniques, and/or additional components, but not all three; these partial CBT interventions were further defined as having an emphasis on cognitive techniques or having an emphasis on behavioural techniques.
In total, 22 trials (65%) implemented a full CBT program, 9 trials (26%) implemented partial CBT with a cognitive focus, and the remaining 3 trials (9%) implemented CBT with a behavioural focus. Illustrative quotes have been included to highlight the content that relates directly to the categories of CBT described above.
Full CBT: e.g. Hamamci [32] "The fundamental goals of group therapy for depression were examination and modification of depressed individuals' maladaptive beliefs systems and dysfunctional forms of information processing. Therapist included cognitive techniques such as Socratic questioning, dysfunctional thought record, downward arrow, labeling and challenging cognitive distortions, as well as behavioural techniques such as scheduling activities, self-monitoring, behavioural rehearsal, and behavioural experiment. The content of the therapy consisted of training in systematic self-monitoring of cognitions, events, and moods; and training in strategies designed to identify and change distorted cognitive systems, mini-lectures, reading, and homework assignments… The last two sessions focused on anticipating future stress and problems in group members lives and in developing strategies for coping with these problems" p 202 [32] Partial CBT: Cognitive focus: e.g. Peden [33] "The 6-week cognitive-behavioural group intervention was designed to reduce negative thinking in depressed women… The intervention evolved from a series of studies in which women recovering from depression identified negative thinking as the most difficult symptoms to overcome [33]. Strategies or techniques used to manage negative thinking were described, including thought stopping and positive selftalk. Affirmations and direct actions were adapted from the Insight Program of Gordon and Tobin. The Depression Workbook (Copeland) provided information on thought stopping, affirmations, and distorted thinking styles." p147 [33].
"The comprehensive BATD treatment (Hopko and Lejuez, Lejuez, et al.) is based on the premise that increased activity and the resulting experience of environmental reinforcement is sufficient for the reduction of depressive symptoms and a corresponding increase in positive thoughts and feelings. The current treatment protocol represented a major modification of the original BATD intervention in that it was reduced to a one-session treatment. This decrease in therapy duration from the typical nine-session format predominantly resulted in five fewer weeks of activity scheduling (i.e., BA); a non-progressive approach to activating, in which a much greater number of behaviors were targeted for activation immediately, as opposed to the traditional graded approach to activity scheduling; and omission of behavioral contracting strategies to decrease rewards for depressive behaviors. Otherwise, all elements of the comprehensive BATD treatment were maintained." p. 471 [34].

Summary of main findings
While only two studies reported on all 14 intervention description domains thought to be important, the majority did report on more than 50% of these. Some key aspects of reporting on the detail of the intervention need improvement, and would be simple to recitify, for example, session length. It is of concern that the most frequently unreported domains related to assessment and reporting of treatment fidelity, sufficient reference to intervention manuals, availability of source documents and reporting of planned session length.
The measurement and reporting of fidelity to the intervention has implication for the potential to draw conclusions about the effectiveness of the interventions and influences the relationship between intervention and outcome. Without the reporting of fidelity, the degree to which the described intervention produced the outcome cannot be adequately determined. Without certainty with regard to the fidelity of trial therapists to the intervention manual, there is the potential that other variables may be driving the intervention effect, which is problematic for the translation and implementation of the intervention [35].
In undertaking the constant comparative method, it became evident that only two broad categories (one of which had two sub-categories) could be identified due to the large variation in the quality and detail of the description of the CBT. While word limitations for publications mean that descriptions of interventions may be somewhat truncated, we did examine whether there was reference to another publication or a published or unpublished manual that gave full description of the intervention. More than half did not have an available source document describing the intervention in full.
Given that CBT is a complex intervention that potentially includes a large range of techniques [16,17], the lack of adequate descriptions of the exact nature of the intervention being examined in a trial means that research findings can not be easily implemented into every day clinical practice, leading to barriers for both clinicians and consumers in terms of realising effective treatment outcomes [26].

Strengths and weakness of study
This is the first study of this kind to examine the quality and completeness of intervention descriptions in psychotherapy trials. The strengths of this study include the comprehensive collection of the CBT intervention descriptions gathered, facilitated by an exhaustive search for and careful inclusion of all relevant published randomised controlled trials of CBT for depression in young people. Additionally we employed independent double data extraction and used robust qualitative data analytic techniques according to the constant comparative method. The research team comprised of an expert in conducting systematic reviews in youth depression, experienced clinical psychologists, and a qualitative research methods expert.
Given the lack of reporting of or access to all source documentation, our coding system contained some inconsistency. For some trials, decisions were made based on information provided in the published manuscript while for others the actual intervention manual or source documentation was evaluated, when publically available. Further, we did not contact authors for verification of missing details and source materials (i.e., intervention manuals). While these two weaknesses are noted, they simultaneously highlight important shortcomings of published trials with regard to intervention reporting.

Implications
There is need for better standards and guidelines for reporting of trials of psychotherapy interventions to guide authors to adequately describe their work for their audience, whether that be researchers, clinicians, consumers or policy makers. Publication word limits may impinge on complete reporting of interventions and journals should more readily provide facility for supplementary information to be published or made available. Journal editors and peer reviewers should be tasked with the responsibility of ensuring interventions are adequately described and source material is provided when making review and publication recommendations.
In describing complex interventions authors should not rely solely on catchall descriptors such as 'behavioural intervention' but instead outline precisely what components constituted the intervention (e.g., behavioural activation, problem solving, social skills training) and how they were delivered. This should be in sufficient detail and supported by all relevant resources and source documents necessary to allow replication of the delivered intervention component. Unfortunately only a handful of trials included in the present review provided sufficient information to enable suitable replication.
Improving the quality and completeness of intervention reporting in psychotherapy trials has the potential to benefit the fields of research and clinical practice. Research gains may include the ability to conduct true replication and extension studies, both of which are necessary steps to establish the effectiveness of a given intervention before implementation into real world clinical settings. Additionally, by adequately describing interventions and/or making treatment manuals available, there is the potential to save an immense amount of time and resources by not creating resources/manuals from scratch. Practice gains include facilitating the translation and implementation of interventions that have been shown to be effective into routine clinical practice. Without adequate reporting, clinicians cannot reliably implement effective interventions.
A further benefit of comprehensive descriptions of interventions is the facilitation of component analyses. Component analyses are possible when we have a thorough description of an intervention, how it is delivered and in what 'dose', to enable an analysis of which components of the intervention effect change. For example, while most trials in the present study implemented a full program of CBT, preliminary analysis of outcome data from these trials shows that partial CBT with a behavioural focus is associated with more favourable patient outcomes [36]. This next critical next step is essential to better understand which types of interventions are most likely to maximise the likelihood of symptom reduction.
Overcoming the shortfalls outlined has great potential to benefit clinical research and facilitate the translation of this work into routine clinical practice. Better reporting of interventions will facilitate research gains, through replication and extension studies, as well as component analyses to determine components are integral or necessary to effect change. Further benefits will include practice gains, whereby clinicians can reliably implement interventions shown to be effective, leading to better outcomes for young people.