Content Analysis

Content Analysis

Leaders: Dr. Erik Bean,

Dr. Liz Johnston 

Welcome to the Content Analysis Research Method Resource Site!

MOST USED FOR COMPARING MEDIA CONTENT IN THE HUMANITIES FIELDS

For those interested in this alternative research method, its roots generally go back to the humanities and study of journalism, media, and society. This can encompass a broad range of studies in business, communications, and education. More than 20 years ago under the tutelage of Stephen Lacy, Ph.D. at Michigan State University, Erik conducted a thesis study to compare a published environmental priorities state report and the degree of newspaper coverage devoted to the most dangerous health risks it reported. Lacy, who has co-authored content analysis textbooks and who has amassed dozens upon dozens of rigorous such studies maintains that this quantitative method of analysis is most successful when a carefully defined coding book of definitions is set up prior to any investigation.

A QUANTITATIVE & QUALITATIVE RESEARCH METHOD

Often confused as just qualitative research method, content analysis also is quantitative since instances of words, images, measurements, and published pieces, for example, are counted and compared to one another based on size, frequency, prominence (how likely they would be seen by the target audience) and by whatever research question is posed.  Dr. Bean and Dr. Johnston are happy to assist those who may need help in developing a rigorous coding book of definitions which must be in place before any of the research starts. Below are some recommended texts you might find useful too.

Best regards to all SAS faculty and students!

Dr. Erik Bean (Quantitative Content Analysis) 
deadline@email.phoenix.edu

Dr. Liz Johnston (Qualitative Content Analysis)
ljohnston@email.phoenix.edu 

SUGGESTED RESOURCES

Bean, E. (2017). An academy customer experience benchmark observation. Internet Learning Journal, 5(1), 17-31.

Bean, E. (2015). Assessing the degree of homogeneous online teaching textbook Infancy from 1999 to 2007 using the immediacy principle. Internet Learning Journal, 4(1). 88-125.

Creswell, J. W., Clark, V. L., Gutmann, M. L., & Hanson, W. E. (2003). Advanced mixed methods research designs. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed method research in social and behavioral research (pp. 209-240). Thousand Oaks, CA: Sage.

Riffe, D., Fico, F., & Lacy, S. (2005). Analyzing media messages: Using quantitative content analysis in research (2nd ed.). Mahwah, NJ: Lawrence Erlbaum.

2013 version available from Amazon Applying Media Messages - Quantitative Communications by Stephen Lacy.  

Also recommended, A Content Analysis Guidebook by Kimberly Neuendorf.

Articles that apply Content Analysis to specific problem

Creamer, E. G., & Ghoston, M. (2013). Using a mixed methods content analysis to analyze mission statements from colleges of engineering. Journal of Mixed Methods Research, 7(2), 110-120. doi:10.1177/1558689812458976

Abstract

A mixed method design was used to conduct a content analysis of the mission statements of colleges of engineering to map inductively derived codes with the EC 2000 outcomes and to test if any of the codes were significantly associated with institutions with reasonably strong representation of women. Most institution’s (25 of 48) mission statement had two or fewer of the outcomes endorsed by the accrediting agency. The diversity code was significantly related to the representation of women, but is not one of the outcomes identified by the accrediting agency. The research demonstrates how mixed methods can be applied to content analysis.

Holman, D. K., & Banning, J. H. (2012). Honors dissertation abstracts: A bounded qualitative Meta-Study. Journal of the National Collegiate Honors Council, 13(1), 41-61.

Abstract

A potential source of useful information about undergraduate honors education can be found in doctoral dissertation abstracts that focus on honors. Debra Holman and James Banning of Colorado State University sought to explore this resource by undertaking a bounded qualitative meta-study of such abstracts using document analysis. Three sub-questions focused their inquiry: (1) What are the general attributes of dissertations on honors education?; (2) What are the thematic subjects and topics associated with the dissertations?; and (3) Have these dissertation findings been published in higher education journals or books? This article provides an account of their research, including information on the meta-study framework they used, their selection of the dissertations for analysis, and their methods and procedures for analyzing the dissertations. Findings, a summary of publication-related trends for dissertations on honors education, and recommendations for future research are provided.

Stepchenkova, S., Kirilenko, A.,   Morrison, A. Facilitating Content Analysis in Tourism Research. SAGE Secondary Data Analysis. SAGE Publications Ltd. London: SAGE Publications Ltd.

Abstract

This article proposes a methodological approach to analyzing multiple files of textual data typical in tourism studies in a transparent, replicable, and effective way. The approach proceeds with data preparation, identification of key variables, obtaining the word-frequency matrix, and subsequent dimensional reduction of word-frequency data. Obtaining a matrix of word frequencies from multiple units of qualitative data allows more sophisticated statistical analyses of data and, ultimately, hypothesis testing. The approach uses an efficient combination of two computer programs, CATPAC and WORDER; however, the methodology is not dependent on this particular software tandem. Other programs that perform the same functions can be used, and the choice, as always, is with the researcher. The objective of the article is to show that the proposed methodology is firmly grounded in the theory and practices of content analysis and is both simple and efficient enough to facilitate statistical data analysis in tourism studies.

Yalçın, S., ÇiğdemY., & Dibek, M. (2015). Content analysis of papers published in educational journals with high impact factors. Education & Science / Egitim ve Bilim, 40(182), 1-28. doi:10.15390/EB.2015.4868

Abstract

The purpose of this study was to carry out a content analysis on the papers published in high-impact educational journals between 2009 and 2014 and to identify the trends over the selected years. The criteria for the analysis were; number of authors, time between the submission and publication of the papers, keywords, the field and rationale of the study, sample size, descriptions of participants, data collection tools and analysis, and software. The current study was designed as a descriptive content analysis study and utilized a purposive sampling technique. A total of 789 papers were selected from the "Journal of Educational Psychology", "Educational Psychologist", "Educational Researcher" and "American Education Research Journal". Content analysis was employed to analyze the collected data. The results of the analyses showed that the most commonly studied fields were educational psychology, linguistic properties and mathematics in the four journals. In terms of the content, the rationales for writing the papers were generally related to gaps in the literature and theoretical discussion. It was found that generally, studies involving research were conducted with elementary/high school students and data was collected from large samples (larger than 10000) using achievement tests and questionnaires. It was revealed that concerning the trends in data analysis methods, there was a similar pattern from 1970s to date, and generally, multilevel modelling was used when appropriate to the data sets. This situation indicates standard data analyses are essential for researchers. From the findings of the current study, it is recommended for researchers to work with heterogenous sample and various types of participants together (family, teacher, peer, etc.)

Conceptual articles on methodology

Hsiu-Fang, H., & Shannon, S. (2005) Three approaches to qualitative content analysis. Qualitative Heath Research. 15:1277-1288, doi:10.1177/1049732305276687

Abstract

Content analysis is a widely used qualitative research technique. Rather than being a single method, current applications of content analysis show three distinct approaches: conventional, directed, or summative. All three approaches are used to interpret meaning from the content of text data and, hence, adhere to the naturalistic paradigm. The major differences among the approaches are coding schemes, origins of codes, and threats to trustworthiness.

In conventional content analysis, coding categories are derived directly from the text data. With a directed approach, analysis starts with a theory or relevant research findings as guidance for initial codes. A summative content analysis involves counting and comparisons, usually of keywords or content, followed by the interpretation of the underlying context. The authors delineate analytic procedures specific to each approach and techniques addressing trustworthiness with hypothetical examples drawn from the area of end-of-life care.

Guest, G. (2013). Describing mixed methods research: An alternative to typologies. Journal of Mixed Methods Research, 7(2), 141-151. doi:10.1177/1558689812461179

Abstract

Scholars have created a variety of typologies to describe and simplify mixed methods research designs. In this article, I review the rationale for using these typologies and discuss some shortcomings of the existing methods of classification. I argue that current systems of classification, although useful for simple and less fluid types of mixed methods research, are not capable of capturing the complexity and iterative nature of larger, more intricate research projects. I suggest an alternative way of viewing and describing mixed methods research for studies that resist simple classification. This alternative perspective shifts the unit of reference to the point of interface—where QUAL and QUAN data are integrated—and reduces the number of descriptive dimensions to two—the timing and the purpose of data integration.

Mayring, P., (2000). Qualitative Content Analysis. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 1(2), Art. 20, http://nbn-resolving.de/urn:nbn:de:0114-fqs0002204.

Excerpt

This introduction criticizes the methodological dichotomization of qualitative and quantitative research, defines Qualitative Content Analysis as a mixed methods approach (containing qualitative and quantitative steps of analysis) and advocates common research criteria for qualitative and quantitative research. Finally, a step-by-step model of the (qualitative-quantitative) research process is presented. http://www.qualitative-research.net/index.php/fqs/article/view/1089/2385

Hartman, R., Johnston, E., & Hill, M., (in press) Empathetic design: A sustainable approach to school change. Discourse and Communication for Sustainable Education

Abstract: A descriptive case study approach employing a content analysis of the blogs of 36 school leaders who took part in the 2016-2017 Shadow a Student Challenge project and follow up interviews of five of the school leaders were used to explore and describe how an empathetic design approach provided a greater vantage point for school leaders to gain a deeper insight into the school experience by viewing school through the eyes of students.  School leaders play a key role in establishing the culture, climate, and educational outcomes within their school environments.  An empathetic design approach to school reform provides a stronger point for initiating change than an atmosphere focused on evaluation and accountability.  Taking a school-based approach, encouraging school leaders to immerse themselves in the school experience, provides an opportunity for school leaders to gain empathy and insights foundational in generating meaningful and innovative change leading to sustainable education. Three major themes emerged from the findings: 1) pace and structure of the school day, 2) student learning experience, and 3) learning environment influences student motivation, energy, and openness to learning.  

Johnston, E., Olivas, G., Steele, P., Smith, C., & Bailey, L. (2017). Exploring Pedagogical Foundations of Existing Virtual Reality Educational Applications: A Content Analysis Study. Journal of Educational Technology Systems, 0047239517745560. doi:10.1177/0047239517745560 

Abstract: New virtual reality (VR) applications for education appear frequently in the marketplace but rarely contain explicit pedagogies. The research objective of this study was to identify and categorize principles and practices of pedagogy that are evident but not articulated in selected VR applications for education. Analysis of public content for the VR applications showed most were experiential while others were categorized as discovery learning, constructivism, situated cognition, direct instruction, or unclassified approaches. Educators and VR designers could use explicit pedagogical frameworks to support faculty development, construct extended, and congruent curricular options that stimulate reflections, build insights, and insure innovative and measurable outcomes.

Johnston, E., Rasmusson, X., Foyil, B., & Shopland P., (2017) Witnesses to transformation: Family member experiences providingindividualized music to their relatives with dementia. Cogent Education. Taylor and Francis Publishinghttp://dx.doi.org/10.1080/2331186X.2017.1362888

Abstract: Content analysis of 35 family members stories found that sharing individualized music enhanced memory, mood, and provided interactive opportunities,where family members connected and communicated with relatives who had dementia. Technology supports a positive new role for family members, who often use MP3 players (e.g. iPods), headphones, splitters, and other technical devices to share preferred, culturally relevant, and digitized music with relatives with dementia. Family members, who provided individualized music described new positive memories and transformed beliefs about the disease and their loved one. Educational leaders in general and music educators specifically, have an opportunity to add renewed emphasis to the culture and value of music as a lifelong resource. Music appreciation courses that include technological skills for creating playlists could support a lifetime of access to individualized music for young and older students, who are also prepared to address needs of those with neurological disorders, such as Alzheimer’s disease.

Image is Petal Perfect, original water media artwork by Elizabeth Johnston (used with permission of the artist)