A CONTENT ANALYSIS OF RESEARCH IN FAMILY THERAPY JOURNALS

Journal of Marital a n d FamilyTherapy January 2000,Vol. 26, N o . l , 9 - 1 6

A CONTENT ANALYSIS OF RESEARCH IN ^FAMILY THERAPY JOURNALS

Dale R. Hawley Woodbury Church of Christ

C. Everett Bailey North Dakota State University

Koby A. Pennick Fargo, ND

In order to assess trends in family therapy research, empirical articles (N = 195) from three family therapy journals over a 5-year period were coded for several variables: authorship, external funding, methodology, sample, purpose, cost effectiveness, use of therapeutic model, and topic.

Results indicated that a large pecentage of research in these journals focused on nonclinical issues and used nonclinical samples. Authors were afiliated with a wide variety of disciplines and reported low levels of external funding for their research. While a majority of the studies used quantitative methods, there appeared to be a growing number of studies using qualitative methods.

Implications of these findngs are discussed in light of research reviews over the past two decades.

As the field of family therapy continues to mature, the role of research takes on increased importance. In an increasingly competitive market for mental health services, it is vital that the field support claims about the viability of family therapy as a clinical approach. Moreover, as family therapy evolves it is important to learn more about the therapeutic process in order to serve clients most effectively. Sprenkle and Bischoff (1995) suggest that research can strengthen the field by legitimizing family therapy as an approach to other mental health disciplines and third-party payers and by helping improve services to our consumers, the clients.

Over the past two decades a host of review articles evaluating the state of research in family therapy have been written (e.g., Gurman, Kniskem, & Pinsof, 1986; Jacobson, 1978; Piercy & Sprenkle, 1990;

Pinsof & Wynne 1995). Along with reviewing research trends and practices, these reviewers have made numerous recommendations regarding future directions for family therapy research. Some of these recommendations include an increase in articles that focus on clinical processes and outcomes (Friedlander, 1998; Pinsof & Wynne, 1995), greater use of qualitative methods (Sprenkle & Bischoff, 1995), an increase in the number of studies that examine cost effectiveness (Sprenkle & Bischoff, 1995) and compare therapy models (Piercy & Sprenkle, 1990), and more attention to the integration of theory, research, and practice (Gurman, Kniskem, & Pinsof, 1986).

While these recommendations provide a constructive path for strengthening research in the field, the degree to which they have been followed is not clear. To our knowledge, a systematic analysis of empirical

Dale R. Hawley, PhD, is Family Minister, Woodbury Church of Christ, Woodbury, MN, and adjunct faculty, Family Social Science Department, University of Minnesota. Send correspondence to: 802 3rd Street, Hudson, WI, 54016.

E-mai1:dhawley @ pressenter.com

ND 58105. E-mail: cbaileyl @prairie.nodak.edu

C. Everett Bailey, PhD, is Assistant Professor, Child Development and Family Science, North Dakota State University, Fargo, Koby A. Pennick, MS, works with US. Bank, Fargo, ND. E-mail: KF'ennick@aol.com

January 2000 JOURNAL OF MARITAL AND FAMLY THEMPY 9

work in the marriage and family therapy literature has not been conducted. This study assesses the degree to which recent marriage and family therapy research has taken note of the concerns and recommendations outlined by these reviewers. Based on a review of these articles, three questions emerged as central for evaluating current research trends in the field: (1) Who is conducting the research?; (2) What kind of research is being done?; and (3) How is this research being funded? To answer these questions, a content analysis of recent empirical articles from three leading marriage and family therapy (MFT) journals was conducted.

METHOD

All empirical articles in the 1994-1998 issues of the American Journal of Family Therapy (AJFT), Family Process (FP), and the Journal of Marital and Family Therapy (JMFT) were included in the sample.

These journals were selected because they are generally viewed as three of the leading marriage and family therapy journals and, as such, were expected to provide a representative sample of the most current research in the field. An article was judged as empirical if it analyzed quantitative or qualitative data. Of the 457 articles that appeared in these three journals over the 5-year period, 199 met the criteria for inclusion in the sample (AJFT = 75, JMFT = 62, and FP = 62).

To assess the central questions of the investigation we devised a coding scheme. In order to evaluate who is publishing in MFT journals, authors were categorized by their institutional affiliation as noted in the biographical footnote. This included separate categories for Commission on Accreditation for Marriage and Family Therapy Education- (COAMFTE) accredited programs, psychology and social work departments, medical schools and hospitals, other departments housed at universities (e.g., family science, education, etc.), therapists in private practice, and authors who were from nonacademic institutions (e.g., agencies, training programs), or whose institutional affiliation was not clearly identified. Several items were coded to evaluate the type of research included in this sample. Articles were categorized by method (quantitative, qualitative, or both) and sample type (clients, therapists, supervisors, nonclinical, content analysis). Articles were also categorized according to their primary purpose based on a coding scheme developed by the authors from the literature review. The following categories were used: (1) family process/individual issues, which included studies that addressed individual or family dynamics outside a clinical context; (2) clinical process, which were articles dealing with interactions occurring during therapy; (3) clinical outcome, which included studies evaluating the effectiveness of therapy; (4) training issues, including articles addressing issues in the supervision and the formal development of therapists; ( 5 ) therapist characteristics, including studies that evaluated ideologies or amibutes influencing how therapists worked; (6) prevention, or studies focused on preventive interventions and psychoeducation; (7) descriptive, such as surveys and content analyses; and (8) psychometric/methods, including studies addressing reliability and validity of instruments or testing means of conducting research. Articles were also evaluated as to whether they addressed cost- effectiveness issues or compared therapeutic models. In addition, articles were coded according to topical descriptors, based on a modified list used by Sprenkle and Bailey (1997). To assess the question of funding, acknowledgements of funding listed in footnotes or endnotes were categorized according to whether they were from sources internal or external to the authors’ institution.

The fEst author coded all the articles, while the second and third authors coded approximately 40% and 60% respectively. Interrater reliability on the categories ranged from .80 to 1.0, with an overall average of .90.

RESULTS

Who is Publishing in MFT Journals?

Table 1 lists the different authors according to their institutional affiliation. First authors who identified themselves as faculty in COAMFTE-accredited marriage and family therapy programs were the most frequent publishers (30%). Of the 59 first authors from these programs, 34 (17% of total) were from PhD programs and 25 (13% of the total) were from Master’s programs. Results also showed that 43% of the articles had at least one author from a COAMFTE program, while 57% did not.

10 JOUWAL OF MARITAL AND F M I L Y THERAPY January 2000

TABLE 1

Author Affiliation and Sample Type

2

1

1

1 Author Clients Affiliation

PhD (MFT)

1 0 5 2 2 0 0

1 0 14 5 1 0 0

0 0 16 4 2 0 0

0 0 4 1 2 1 0

Master’s (MFT)

---F

University other

I

Psychology

3

l 3

Social Work

Medical school/

i

Other

Totals

i

Totals

34 (17%)

25 (13%)

40 (20%)

15 (7.5%)

29 (14.5%)

26 (13%)

10 (5%)

20 (10%)

199 (100%)

What Kind _of Research is Being Done?

Table 1 lists the different samples used in the research articles. The most frequently used sample consisted of nonclinical subjects (44%), followed by clinical subjects (36%, combining client only, clinicallnonclincial, and clientherapist categories). Of all studies examined, 80% (n = 160) used quanti- tative methodologies, 12% (n = 23) used qualitative methods, and 8% (n = 16) used both. The clinical process research was fairly balanced, using quantitative methods 4 1 % of the time and qualitative methods 46%, with 14% of studies using both. The clinical outcome research is predominantly quantitative (95%), with one study employing both quantitative and qualitative methods. Fifty percent of the research articles on training and supervision used quantitative methods, 25% used qualitative methods, and 25% employed both methods.

According to Table 2, most of marriage and family therapy research examined family process or individual issues (43%). Clinical process (11%) and clinical outcome (10%) research combined for a total of 21% of all the research articles. Categories relating to clinical issues (i.e., clinical process, clinical outcome, preventive education, training, and therapist characteristics) combined for a total of 33% of all research articles. Only 3 of the 20 clinical outcome studies (15%) used both a nonclinical control group and clinical subjects, while 60% (12 of the 20) used clients only. In addition, only 2% ( n = 4) of all the research articles published in the three MFT journals contained a cost-effectiveness component. Nineteen percent (n

January 2000 OURNAL OF MARITAL A N D FAMILY THERAPY 11

Author Affiliation and Purpose of Research Author Family Clinical Clinical Prevention Train / Therapists’ Descriptive Affiliation process process outcome supervise char.

PhD (MFT) 9 5 2 0 1 4 10

Master’s (MFT) 6 4 2 1 5 3 2

Psychology 17 4 5 0 0 1 1

Social work 6 2 2 0 0 1 2

Medical school/ 20 0 4 0 0 1 1

hospital

University 12 4 3 0 0 0 4

other

Private practice 3 1 1 1 2 1 1

Other 12 2 1 1 0 1 1

Totals 85 22 20 3 8 12 22

I

I

I

I

= 37) of the research articles identified a specific therapy model associated with the research, but only 3%

( n = 6) of the studies included a component that compared the effectiveness of different family therapy models. The articles were also coded into 67 different categories describing a content area addressed in the research. Articles were coded in a category if they were related or had implications for the content area. The top 10 are shown in Table 3.

Methods/ Totals psychometrics

3 34

(17%)

2 25

(13%)

12 40

(20%)

2 15

(7.5%)

3 29

(14.5%)

3 26

(13%)

0 10

(5%)

2 20

(10%)

27 199

(13.5%)

1

What Kind of Research is Being Done in COAMFTE-Accredited Programs?

Cross-tabulated results (Table 2) showed that researchers from COAMFTE-accredited MFT programs most often conducted clinical research. Forty-five percent of the research articles done by MFTs were on clinical topics such as clinical process/outcome, supervision or training issues, or therapist characteristics.

Authors from MFT programs published 4 clinical outcome studies (2 from PhD programs and 2 from Master’s programs) and 9 clinical process studies. In addition 25% (n = 15) of MFT research articles were on family procedindividual issues and 20% (n = 12) were descriptive in nature. Compared to other profes- sionals, MFT authors had the lowest percentage of articles on family processhndividual issues while authors from medical schools or hospitals had the highest percentage (69%) of articles dealing with these issues.

Psychologists had the highest percentage (30%) of articles on methods and psychometrics. Finally, h@T authors had the highest percentage (20%) of research articles that were descriptive in nature.

12 JOURVAL OF MARITAL AND F M I L Y THERAPY January 2000

Overall, 44% of the data collected were from nonclinical subjects and 36% were from samples that included clinical subjects. MFT authors collected data most often from therapists and supervisors (33%), followed by clinical samples (28%), and nonclinical subjects (26%). MFT researchers had the lowest percentage of research using a clinical sample (28%) compared to 48% of studies done by psychologists, 46% by social workers, 45% by professionals in medical schools or hospitals, and 35%

by other university authors. Of the 59 research articles by MFTs, two (3%) used both a treatment and control group, 15 (25%) used a nonclinical sample only, and 14 (24%) were based on a sample of therapists only. Conversely, MFT authors had the highest percentage of samples utilizing therapists and supervisors (33%), compared with 20% for social workers, 7% for authors from medical schools or hospitals, and none for psychologists. Furthermore, MFTs also had the highest percentage of content analysis articles (l2%), while social workers, medical school or hospital professionals, university other, and those authors in private practice had none. In addition, MFT researchers had one of the lowest percentages of samples using both clients and therapists (7%), compared with 20% for those in private practice, 13% for social workers, and 13% for psychologists. Finally, MFT researchers had the lowest rate of studies using only nonclinical subjects (26%) compared to 62% for university other, 50% for psychologists, 48% for medical school or hospital, and 33% for social workers.

Most of the research used quantitative methods. For example, 98% of the research published by psychologists was quantitative, 97% by authors from medical school or hospital settings, and 85% by authors from other university settings. MFT researchers also primarily relied only on quantitative methods in their studies (70%) but, along with social workers and authors in private practice, incorporated qualitative methods in some of their studies. Twenty percent of articles from MFT authors were entirely qualitative in methodology and another 10% used both quantitative and qualitative methods.

To What Degree is MFT Research Being Funded?

Seventy-one percent (n = 141) of the research articles did not indicate any funding sources for the research conducted (see Table 4). However, 22% (n = 43) of the articles acknowledged funding from external sources and 7% (n = 15) stated the study was supported by internal grants. Six studies by MFTs (two authors from PhD programs, four authors from Master’s programs) received external funding. Aside from private practice authors (none of whom indicated they had received any funding for their research) MFT researchers received the least amount (10%) of external funding. In comparison, 41% of medical school or hospital authors received external funding.

Of the 43 studies receiving external funding, 58% were research on family processhndividual issues, 23% on methods, 7% on clinical outcome, and 5% on clinical process. Furthermore, 9% of the clinical process research articles (n ⁼ 2) and 15% of the clinical outcome research articles ( n = 3) received external funding. Eighty-one percent of the research receiving external funding used quanti- tative methods, 5% used qualitative methods, and 14% used both methods. In terms of methods, 22% of the research utilizing quantitative methods was funded externally, as well as 9% of qualitative studies and 38% of research employing both methods.

How Do the MFT Journals Compare?

Authors from COAMFTE-accredited programs published more frequently in JMFT and AJFT than in FP. Only 11% of first authors in FP articles came from these programs as opposed to 36% for AFJT and 40% for JMFT. Similarly, 58% of articles in JMFT and 5 1% of AJFT articles had at least one author from a COAMFTE-accredited program, as compared with 14% for FP. Articles in AJFT (88%) and FP (84%) tended to rely solely on quantitiative methods. Although JMFT articles were predominantly quantitative, a higher percentage of articles used qualitative methods (19% qualitative only; 13% combined methods).

Several differences were also found when articles were coded for purpose. The most prominent category in AJFT (49%) and FP (61%) was family processhndividual issues. In addition, about a quarter (23%) of the articles in FP were focused on psychometrics/methods. Clinical outcome studies were most likely to be found in AJFT, where 11 articles (15%) had this purpose. There was a wider range of purposes evident in JMFT articles, with descriptive studies (27%), clinical process (16%), and family processhndividual issues

January 2000 JOURNAL OF MARITAL AND FAMLY THERAPY 13

TABLE 3

Description of Research Articles (N = 199)

Category n %

Couples

Assessmentshnstruments Training/supervision Children/adolescents Techniques

Chronichnpatientlschizophrenia Models of family therapy Professional issues Self-of-therapist Violencelabusehncest

41 38 19 19 16 14 13 12 12 12

20.6 19.1 9.5 9.5 8.0 7.0 6.5 6.0 6.0 6.0 Note. An article may be coded into more than one category

I

(16%) being most common. Samples in AJFT and FP were most commonly drawn from nonclinical populations (AJFT = 47%; FP 61%) followed by clients (AJFT = 29%; FP = 32%). Articles in JMFT tended to focus on clients and therapists/supervisors. Forty-four percent of articles included therapists in the sample and 39% included clients. Finally, studies published in FP (52%) were much more likely to have external funding than those in JMFT (1 1%) and AJFT (5%).

DISCUSSION

Results of this study indicate that authors from a variety of disciplines publish in these three journals.

Although researchers associated with COAMFTE-accredited programs constitute the largest group, they are by no means the dominant producers of research in this sample. Authors associated with other university departments (most notably psychology) and medical schools or hospitals wrote a substantial number of articles. Authors from COAMFI'E-accredited programs were much more likely to publish articles in JMFT and AJFT than in FP, which had a high percentage of authors from psychology and the medical field.

These findings are not surprising given the multidisciplinary nature of marriage and family therapy.

The pioneers of the field came from a variety of disciplines including psychiatry, psychology, and social work. Moreover, many members of AAMFT have multiple professional allegiances, with MFT often being a secondary professional identification. Pinsof and Wynne ( I 995) have argued that a multidisciplinary focus strengthens research efforts in the field. At the same time, it is notable that less than half of the articles in this sample of three leading journals primarily associated with the field of marriage and family therapy have any author affiliated with a COAMFTE-accredited program.

Repeated calls (Gurman, Kniskern, & Pinsof; 1986; Piercy & Sprenkle, 1990; Wynne, 1988) over the years for an increase in clinical studies, especially those focusing on the therapeutic process, seem to have had little impact, judging from this sample. The largest category of studies in this analysis focused on family process and individual issues (43%) without a clinical context. Overall, we found that only 11% of the articles addressed clinical process and 10% assessed clinical outcome. This nonclinical focus was especially evident in FP, where less than 15% of the articles were classified as clinical. Furthermore, nonclinical samples were more common than those involving clients. Although researchers from COAMFTE- accredited programs had a relatively low percentage of studies using only a nonclinical sample, they had a greater tendency to draw their samples from therapists and supervisors rather than clients. It is also

14 JOURNAL OF MARITAL AND FAMILY THERAPY January ²⁰⁰⁰

TABLE 4

Author Affiliation and Research Funding

Other

12

2

6

20 (10%)

I

Totals

141 (71%)

15 (7%)

43 (22%)

199 (100%) Funding

I

Research

I

No Funding

Internal

External

Totals

(17%) 17

4

4

25 (13%)

'sychology

32

0

8

40 (20%)

Social work

11

2

2

15 (7.5%)

M e I a l

1

(15%)

1

interesting to note that there are relatively few research articles directly dealing with training or supervision issues (see Table 2). Given that this is a distinguishing facet of MFT training when compared to other disciplines, it would seem that more research in this area is needed. Finally, there appears to be a paucity of research that addresses issues of interest to managed care. Only a handful of articles studied cost effectiveness or compared therapy models to assess their effectiveness.

It is unclear why such a high percentage of articles appearing in these clinical journals draw on nonclinical samples and seem to focus on nonclinical issues. It may be that such samples are easier to access than clinical populations or that the nature of clinical process or outcome research makes it more difficult to produce. In academic settings where the pressure to publish is intense, faculty may have a tendency to pursue research interests that will more quickly become published. Regardless of the reason, it appears that the call for more clinical research needs to be reiterated.

The recommendation for an increase in qualitative methods (Piercy & Sprenkle, 1990; Sprenkle &

Bischoff, 1995) seems to have received more attention. Although the vast majority of studies still employ only quantitative analyses, one-fifth of the studies in this sample used qualitative methods, sometimes in combination with quantitative methods. There was a clear distinction among disciplines in this regard.

Qualitative analyses were virtually nonexistent for psychologists and researchers in medical settings. Yet, a substantial number of articles from COAMFTE authors, social workers, and therapists in private practice used qualitative methods, especially in studies focused on clinical processes.

Qualitative research designs have been gaining in popularity in recent years, especially among researchers who favor a postpositivist paradigm. However, a concern that is commonly raised is whether such research will be acceptable for publication in scholarly outlets. This study suggests that the credibility of qualitative research in the MFT literature is rising and that such efforts are increasingly welcomed in the field.

Our findings also indicate that external funding is the exception rather than the rule for studies appearing in these journals (see Table 4). Less than one-fourth of the authors reported receiving external funding. Moreover, authors from primary affiliations other than marriage and family therapy conducted the majority of funded studies. Only 10% ( n = 6) of first authors from COAMFTE-accredited programs reported external funding, with an additional 14% (n = 8) reporting funding sources from within their institutions.

When funding was reported, it was usually for articles focusing on family process/individual issues or psychometrics/methods. Only five out of the 42 articles addressing clinical outcome or process reported receiving external funding.

These results highlight a critical concern for MFT research. Consistent, on-going research programs

January2000 JOURNAL OF MARITAL AND F M I L Y THERAPY 15

are needed if the field is to make strides in learning about effective processes and outcomes in family therapy. In other disciplines, such programs require a great deal of time and effort on the part of researchers and significant funding from external sources generally supports them. At this point it seems that research programs like these in the MFT field are rare. This may be partially due to the strong emphasis placed on clinical training in COAMFTE-accredited programs. MFI' faculty may lack the time and resources to teach academic classes, supervise and administer on-site clinics, and conduct a rigorous research agenda. In any event, one may wonder whether efforts for increased clinical research will materialize unless the funding issue is also addressed.

Several limitations need to be considered in interpreting these results. While these three journals are generally considered to be leaders in the field, a number of other outlets for MFT research also exist. A sample drawn from a wider range of journals in the field might yield a different picture than the one presented from these results. Also, in a few cases institutional aftiliations of authors was not clear from the biographical footnote. In particular, authors from departments that housed a COAMlTE-accredited program were categorized as such, although we had no way of determining whether they were clinical faculty. Finally, coding was dependent on the judgment of the raters. Although we worked to develop categories that were clear and had acceptable to high levels of interrater reliability, there is a degree of subjectivity in a content analysis like this.

Although it appears that some strides have been made in response to recommendations of reviewers, the MFT field still has significant room for improvement in order to establish a credible research base that will justify its practices. We acknowledge the efforts of the AAMFT to conduct research conferences in order to address the research needs of the field and see it as a good start. Perhaps as clinical research is emphasized more at our professional conferences and in MFT training programs, more studies establishing the process and effectiveness of marriage and family therapy will be produced.

REFERENCES

Friedlander, M. L. (1998). Family therapy research Science into practice, practice into science. In M. P. Nichols & R. C.

S c h w a (Eds.), Family therapy: Concept.7 and methods (pp. 503-533). Boston, M A Allyn & Bacon.

Gurman, A,, Kniskem, D., & Pinsof, W. (1986). Research on the process and outcome of marital and family therapy. In S.

Garfield &A. Bergin (Eds.), Handbook ofpsychotherapy and behavior change (3rd ed.), (pp. 565424). New York Wiley.

Jacobson, N. (1978). A review of the research on the effectiveness of marital therapy. In T. Paolino & B. McCrady (Eds.), Marriage and marital therapy: Psychoanalytic, behavioral, and systems-theory perspectives (pp. 395444). New York BrunnerMazel.

Piercy, F. P., & Sprenkle, D. H. (1990). Marriage and family therapy: A decade review. Journal of Marriage and the Family, 50, 1 1 1 6 1 126.

Pinsof, W. M., & Wynne, L. C. (1995) The efficacy of marital and family therapy: An empirical overview, conclusions, and recommendations. Journal of Marital and Family Therapy, 21, 585413.

Sprenkle, D. & Bailey, C. E.(1997). Who is publishing in JMFT? An examination of author characteristics. Journal of Marital and Family Therapy, 23, 381-388.

Sprenkle, D., & Bischoff, R. (1995). Research in family therapy: Trends, issues, and recommendations. In M. Nichols & R.

Schwartz (Eds.), Family therapy: Concepts and methods (3rd ed.), @p. 542-580). Needham Heights, MA: Allyn &Bacon.

16 JOURNAL OF MARITAL AND F M I L Y THERAPY January 2000