example of findings in qualitative research

How To Write The Results/Findings Chapter

For qualitative studies (dissertations & theses).

By: Jenna Crossley (PhD). Expert Reviewed By: Dr. Eunice Rautenbach | August 2021

So, you’ve collected and analysed your qualitative data, and it’s time to write up your results chapter. But where do you start? In this post, we’ll guide you through the qualitative results chapter (also called the findings chapter), step by step. 

Overview: Qualitative Results Chapter

• What (exactly) the qualitative results chapter is
• What to include in your results chapter
• How to write up your results chapter
• A few tips and tricks to help you along the way
• Free results chapter template

What exactly is the results chapter?

The results chapter in a dissertation or thesis (or any formal academic research piece) is where you objectively and neutrally present the findings of your qualitative analysis (or analyses if you used multiple qualitative analysis methods ). This chapter can sometimes be combined with the discussion chapter (where you interpret the data and discuss its meaning), depending on your university’s preference.  We’ll treat the two chapters as separate, as that’s the most common approach.

In contrast to a quantitative results chapter that presents numbers and statistics, a qualitative results chapter presents data primarily in the form of words . But this doesn’t mean that a qualitative study can’t have quantitative elements – you could, for example, present the number of times a theme or topic pops up in your data, depending on the analysis method(s) you adopt.

Adding a quantitative element to your study can add some rigour, which strengthens your results by providing more evidence for your claims. This is particularly common when using qualitative content analysis. Keep in mind though that qualitative research aims to achieve depth, richness and identify nuances , so don’t get tunnel vision by focusing on the numbers. They’re just cream on top in a qualitative analysis.

So, to recap, the results chapter is where you objectively present the findings of your analysis, without interpreting them (you’ll save that for the discussion chapter). With that out the way, let’s take a look at what you should include in your results chapter.

What should you include in the results chapter?

As we’ve mentioned, your qualitative results chapter should purely present and describe your results , not interpret them in relation to the existing literature or your research questions . Any speculations or discussion about the implications of your findings should be reserved for your discussion chapter.

In your results chapter, you’ll want to talk about your analysis findings and whether or not they support your hypotheses (if you have any). Naturally, the exact contents of your results chapter will depend on which qualitative analysis method (or methods) you use. For example, if you were to use thematic analysis, you’d detail the themes identified in your analysis, using extracts from the transcripts or text to support your claims.

While you do need to present your analysis findings in some detail, you should avoid dumping large amounts of raw data in this chapter. Instead, focus on presenting the key findings and using a handful of select quotes or text extracts to support each finding . The reams of data and analysis can be relegated to your appendices.

While it’s tempting to include every last detail you found in your qualitative analysis, it is important to make sure that you report only that which is relevant to your research aims, objectives and research questions .  Always keep these three components, as well as your hypotheses (if you have any) front of mind when writing the chapter and use them as a filter to decide what’s relevant and what’s not.

Need a helping hand?

How do I write the results chapter?

Now that we’ve covered the basics, it’s time to look at how to structure your chapter. Broadly speaking, the results chapter needs to contain three core components – the introduction, the body and the concluding summary. Let’s take a look at each of these.

Section 1: Introduction

The first step is to craft a brief introduction to the chapter. This intro is vital as it provides some context for your findings. In your introduction, you should begin by reiterating your problem statement and research questions and highlight the purpose of your research . Make sure that you spell this out for the reader so that the rest of your chapter is well contextualised.

The next step is to briefly outline the structure of your results chapter. In other words, explain what’s included in the chapter and what the reader can expect. In the results chapter, you want to tell a story that is coherent, flows logically, and is easy to follow , so make sure that you plan your structure out well and convey that structure (at a high level), so that your reader is well oriented.

The introduction section shouldn’t be lengthy. Two or three short paragraphs should be more than adequate. It is merely an introduction and overview, not a summary of the chapter.

Pro Tip – To help you structure your chapter, it can be useful to set up an initial draft with (sub)section headings so that you’re able to easily (re)arrange parts of your chapter. This will also help your reader to follow your results and give your chapter some coherence.  Be sure to use level-based heading styles (e.g. Heading 1, 2, 3 styles) to help the reader differentiate between levels visually. You can find these options in Word (example below).

Section 2: Body

Before we get started on what to include in the body of your chapter, it’s vital to remember that a results section should be completely objective and descriptive, not interpretive . So, be careful not to use words such as, “suggests” or “implies”, as these usually accompany some form of interpretation – that’s reserved for your discussion chapter.

The structure of your body section is very important , so make sure that you plan it out well. When planning out your qualitative results chapter, create sections and subsections so that you can maintain the flow of the story you’re trying to tell. Be sure to systematically and consistently describe each portion of results. Try to adopt a standardised structure for each portion so that you achieve a high level of consistency throughout the chapter.

For qualitative studies, results chapters tend to be structured according to themes , which makes it easier for readers to follow. However, keep in mind that not all results chapters have to be structured in this manner. For example, if you’re conducting a longitudinal study, you may want to structure your chapter chronologically. Similarly, you might structure this chapter based on your theoretical framework . The exact structure of your chapter will depend on the nature of your study , especially your research questions.

As you work through the body of your chapter, make sure that you use quotes to substantiate every one of your claims . You can present these quotes in italics to differentiate them from your own words. A general rule of thumb is to use at least two pieces of evidence per claim, and these should be linked directly to your data. Also, remember that you need to include all relevant results , not just the ones that support your assumptions or initial leanings.

In addition to including quotes, you can also link your claims to the data by using appendices , which you should reference throughout your text. When you reference, make sure that you include both the name/number of the appendix , as well as the line(s) from which you drew your data.

As referencing styles can vary greatly, be sure to look up the appendix referencing conventions of your university’s prescribed style (e.g. APA , Harvard, etc) and keep this consistent throughout your chapter.

Section 3: Concluding summary

The concluding summary is very important because it summarises your key findings and lays the foundation for the discussion chapter . Keep in mind that some readers may skip directly to this section (from the introduction section), so make sure that it can be read and understood well in isolation.

In this section, you need to remind the reader of the key findings. That is, the results that directly relate to your research questions and that you will build upon in your discussion chapter. Remember, your reader has digested a lot of information in this chapter, so you need to use this section to remind them of the most important takeaways.

Importantly, the concluding summary should not present any new information and should only describe what you’ve already presented in your chapter. Keep it concise – you’re not summarising the whole chapter, just the essentials.

Tips for writing an A-grade results chapter

Now that you’ve got a clear picture of what the qualitative results chapter is all about, here are some quick tips and reminders to help you craft a high-quality chapter:

• Your results chapter should be written in the past tense . You’ve done the work already, so you want to tell the reader what you found , not what you are currently finding .
• Make sure that you review your work multiple times and check that every claim is adequately backed up by evidence . Aim for at least two examples per claim, and make use of an appendix to reference these.
• When writing up your results, make sure that you stick to only what is relevant . Don’t waste time on data that are not relevant to your research objectives and research questions.
• Use headings and subheadings to create an intuitive, easy to follow piece of writing. Make use of Microsoft Word’s “heading styles” and be sure to use them consistently.
• When referring to numerical data, tables and figures can provide a useful visual aid. When using these, make sure that they can be read and understood independent of your body text (i.e. that they can stand-alone). To this end, use clear, concise labels for each of your tables or figures and make use of colours to code indicate differences or hierarchy.
• Similarly, when you’re writing up your chapter, it can be useful to highlight topics and themes in different colours . This can help you to differentiate between your data if you get a bit overwhelmed and will also help you to ensure that your results flow logically and coherently.

If you have any questions, leave a comment below and we’ll do our best to help. If you’d like 1-on-1 help with your results chapter (or any chapter of your dissertation or thesis), check out our private dissertation coaching service here or book a free initial consultation to discuss how we can help you.

Psst... there’s more!

This post was based on one of our popular Research Bootcamps . If you're working on a research project, you'll definitely want to check this out ...

22 Comments

This was extremely helpful. Thanks a lot guys

Hi, thanks for the great research support platform created by the gradcoach team!

I wanted to ask- While “suggests” or “implies” are interpretive terms, what terms could we use for the results chapter? Could you share some examples of descriptive terms?

I think that instead of saying, ‘The data suggested, or The data implied,’ you can say, ‘The Data showed or revealed, or illustrated or outlined’…If interview data, you may say Jane Doe illuminated or elaborated, or Jane Doe described… or Jane Doe expressed or stated.

I found this article very useful. Thank you very much for the outstanding work you are doing.

What if i have 3 different interviewees answering the same interview questions? Should i then present the results in form of the table with the division on the 3 perspectives or rather give a results in form of the text and highlight who said what?

I think this tabular representation of results is a great idea. I am doing it too along with the text. Thanks

That was helpful was struggling to separate the discussion from the findings

this was very useful, Thank you.

Very helpful, I am confident to write my results chapter now.

It is so helpful! It is a good job. Thank you very much!

Very useful, well explained. Many thanks.

Hello, I appreciate the way you provided a supportive comments about qualitative results presenting tips

I loved this! It explains everything needed, and it has helped me better organize my thoughts. What words should I not use while writing my results section, other than subjective ones.

Thanks a lot, it is really helpful

Thank you so much dear, i really appropriate your nice explanations about this.

Thank you so much for this! I was wondering if anyone could help with how to prproperly integrate quotations (Excerpts) from interviews in the finding chapter in a qualitative research. Please GradCoach, address this issue and provide examples.

what if I’m not doing any interviews myself and all the information is coming from case studies that have already done the research.

Very helpful thank you.

This was very helpful as I was wondering how to structure this part of my dissertation, to include the quotes… Thanks for this explanation

This is very helpful, thanks! I am required to write up my results chapters with the discussion in each of them – any tips and tricks for this strategy?

For qualitative studies, can the findings be structured according to the Research questions? Thank you.

Do I need to include literature/references in my findings chapter?

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

• Dissertation Coaching
• Qualitative Data Analysis
• Statistical Consulting
• Dissertation Editing
• On-Demand Courses

How To Write the Findings Chapter for Qualitative Studies

Writing the findings chapter for qualitative studies is a critical step in the research process. This chapter allows researchers to present their findings, analyze the data collected, and draw conclusions based on the study’s objectives. In this blog post, I’lll explore the purpose, key elements, preparation, writing, and presentation of the findings chapter in qualitative studies.

Introduction: Contextualizing Your Findings

The introductory section serves as the gateway to your qualitative findings chapter. Begin by reiterating your problem statement and research questions, setting the stage for the data that will be presented. Highlighting the purpose of your research is crucial at this point to give context to the reader.

Example: The primary aim of this case study is to explore how educators perceive the integration of artificial intelligence (AI) tools in K-12 education settings. This research seeks to understand the perceptions and experiences of educators who have implemented AI technologies in their classrooms, framed within the context of Technological Pedagogical Content Knowledge (TPACK). In this chapter, data are systematically organized into three overarching themes, each comprising sub-themes that provide a nuanced understanding of the participants’ perspectives. The research questions guiding this investigation were: (1) What are educators’ perceptions of the role of AI tools in K-12 classrooms? (2) How do educators navigate the challenges of integrating AI tools into their teaching practices? The answers to these questions are integral to understanding the complex dynamics of AI implementation in educational contexts.  

Overview of Findings

Following this, offer a brief overview of your main findings. While you should not delve into the details here, giving the reader an idea of what to expect can be helpful. Explain the overall structure of your results chapter and how you’ve organized it to maintain coherence and logical flow.

The Heart of the Matter: The Body of Your Chapter

Presentation.

The body of the chapter is where you lay out your data for the reader. In qualitative research, this usually means dividing your data into themes or categories, which should be clearly described and substantiated with quotes and examples from your dataset. These themes provide the skeletal framework upon which your narrative is built.

Structure and Flow

When planning your qualitative findings chapter, carefully outline the sections and subsections to maintain the flow of the writing and improve readability. You can structure your chapter based on themes, which is often the case in qualitative research, but other formats like chronological or framework-based structures may be more appropriate depending on your specific research design.

✅ Consistency is Key: Make sure each portion of findings adheres to a standardized structure. This enhances consistency and enables the reader to follow your line of reasoning.

Objective and Descriptive Language

While your narrative might touch upon individual experiences and perspectives, remember to maintain an objective tone. Your task is to describe, not interpret—that comes later, in the Discussion chapter. Thus, avoid phrases that suggest interpretation, such as “suggests” or “implies.”

Visual Aids

Tables, figures, and other visual aids can add another layer of comprehension and break up the text, but make sure they can be understood independently of your body text. Label them clearly and use color coding judiciously to indicate differences or hierarchy.

Data Analysis and Interpretation

As you delve into the data, aim to narrate a coherent story. Interpret your findings in light of the literature in the field and your theoretical framework, but remember to clearly differentiate between your descriptions and your interpretations.

✅ References and Appendices: When using quotes or data excerpts, reference them appropriately. Use appendices to present additional data and ensure that you cite them according to the referencing style prescribed by your institution (e.g., APA, Harvard).

Bringing It All Together: The Concluding Summary

This is the section where you summarize your key findings in a concise manner, reiterating points that directly relate to your research questions. It serves as a stepping stone to the Discussion chapter, providing the reader with the essential takeaways. As a rule of thumb, this section should contain no new information.

Additional Tips and Tricks

– Write in the past tense, as you present findings that have already been gathered.

– Review your work multiple times, ensuring each theme or finding is backed by sufficient data.

– Use Microsoft Word’s “heading styles” for consistency.

Final Thoughts

With the right approach, writing the Findings chapter can be an enriching experience that showcases your research and prepares you for discussions and conclusions that follow. The tips and guidelines presented here are meant to make this crucial chapter as clear and impactful as possible, helping you make a valuable contribution to your field of study.

Need Further Assistance?

Dissertation by Design is here to support you every step of the way. Our team of experienced academic coaches and methodologists are dedicated to helping you confidently navigate through each stage of your dissertation journey.  Book a free, 30-minute consult today.

Happy writing!

Author:  Jessica Parker, EdD

Related posts.

Download our free guide on how to overcome the top 10 challenges common to doctoral candidates and graduate sooner.

Thank You 🙌

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is Qualitative Research? | Methods & Examples

What Is Qualitative Research? | Methods & Examples

Published on June 19, 2020 by Pritha Bhandari . Revised on June 22, 2023.

Qualitative research involves collecting and analyzing non-numerical data (e.g., text, video, or audio) to understand concepts, opinions, or experiences. It can be used to gather in-depth insights into a problem or generate new ideas for research.

Qualitative research is the opposite of quantitative research , which involves collecting and analyzing numerical data for statistical analysis.

Qualitative research is commonly used in the humanities and social sciences, in subjects such as anthropology, sociology, education, health sciences, history, etc.

• How does social media shape body image in teenagers?
• How do children and adults interpret healthy eating in the UK?
• What factors influence employee retention in a large organization?
• How is anxiety experienced around the world?
• How can teachers integrate social issues into science curriculums?

Table of contents

Approaches to qualitative research, qualitative research methods, qualitative data analysis, advantages of qualitative research, disadvantages of qualitative research, other interesting articles, frequently asked questions about qualitative research.

Qualitative research is used to understand how people experience the world. While there are many approaches to qualitative research, they tend to be flexible and focus on retaining rich meaning when interpreting data.

Common approaches include grounded theory, ethnography , action research , phenomenological research, and narrative research. They share some similarities, but emphasize different aims and perspectives.

Note that qualitative research is at risk for certain research biases including the Hawthorne effect , observer bias , recall bias , and social desirability bias . While not always totally avoidable, awareness of potential biases as you collect and analyze your data can prevent them from impacting your work too much.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Each of the research approaches involve using one or more data collection methods . These are some of the most common qualitative methods:

• Observations: recording what you have seen, heard, or encountered in detailed field notes.
• Interviews:  personally asking people questions in one-on-one conversations.
• Focus groups: asking questions and generating discussion among a group of people.
• Surveys : distributing questionnaires with open-ended questions.
• Secondary research: collecting existing data in the form of texts, images, audio or video recordings, etc.
• You take field notes with observations and reflect on your own experiences of the company culture.
• You distribute open-ended surveys to employees across all the company’s offices by email to find out if the culture varies across locations.
• You conduct in-depth interviews with employees in your office to learn about their experiences and perspectives in greater detail.

Qualitative researchers often consider themselves “instruments” in research because all observations, interpretations and analyses are filtered through their own personal lens.

For this reason, when writing up your methodology for qualitative research, it’s important to reflect on your approach and to thoroughly explain the choices you made in collecting and analyzing the data.

Qualitative data can take the form of texts, photos, videos and audio. For example, you might be working with interview transcripts, survey responses, fieldnotes, or recordings from natural settings.

Most types of qualitative data analysis share the same five steps:

• Prepare and organize your data. This may mean transcribing interviews or typing up fieldnotes.
• Review and explore your data. Examine the data for patterns or repeated ideas that emerge.
• Develop a data coding system. Based on your initial ideas, establish a set of codes that you can apply to categorize your data.
• Assign codes to the data. For example, in qualitative survey analysis, this may mean going through each participant’s responses and tagging them with codes in a spreadsheet. As you go through your data, you can create new codes to add to your system if necessary.
• Identify recurring themes. Link codes together into cohesive, overarching themes.

There are several specific approaches to analyzing qualitative data. Although these methods share similar processes, they emphasize different concepts.

Qualitative research often tries to preserve the voice and perspective of participants and can be adjusted as new research questions arise. Qualitative research is good for:

• Flexibility

The data collection and analysis process can be adapted as new ideas or patterns emerge. They are not rigidly decided beforehand.

• Natural settings

Data collection occurs in real-world contexts or in naturalistic ways.

• Meaningful insights

Detailed descriptions of people’s experiences, feelings and perceptions can be used in designing, testing or improving systems or products.

• Generation of new ideas

Open-ended responses mean that researchers can uncover novel problems or opportunities that they wouldn’t have thought of otherwise.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Researchers must consider practical and theoretical limitations in analyzing and interpreting their data. Qualitative research suffers from:

• Unreliability

The real-world setting often makes qualitative research unreliable because of uncontrolled factors that affect the data.

• Subjectivity

Due to the researcher’s primary role in analyzing and interpreting data, qualitative research cannot be replicated . The researcher decides what is important and what is irrelevant in data analysis, so interpretations of the same data can vary greatly.

• Limited generalizability

Small samples are often used to gather detailed data about specific contexts. Despite rigorous analysis procedures, it is difficult to draw generalizable conclusions because the data may be biased and unrepresentative of the wider population .

• Labor-intensive

Although software can be used to manage and record large amounts of text, data analysis often has to be checked or performed manually.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Chi square goodness of fit test
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Inclusion and exclusion criteria

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to systematically measure variables and test hypotheses . Qualitative methods allow you to explore concepts and experiences in more detail.

There are five common approaches to qualitative research :

• Grounded theory involves collecting data in order to develop new theories.
• Ethnography involves immersing yourself in a group or organization to understand its culture.
• Narrative research involves interpreting stories to understand how people make sense of their experiences and perceptions.
• Phenomenological research involves investigating phenomena through people’s lived experiences.
• Action research links theory and practice in several cycles to drive innovative changes.

Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organizations.

There are various approaches to qualitative data analysis , but they all share five steps in common:

• Prepare and organize your data.
• Review and explore your data.
• Develop a data coding system.
• Assign codes to the data.
• Identify recurring themes.

The specifics of each step depend on the focus of the analysis. Some common approaches include textual analysis , thematic analysis , and discourse analysis .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bhandari, P. (2023, June 22). What Is Qualitative Research? | Methods & Examples. Scribbr. Retrieved September 2, 2024, from https://www.scribbr.com/methodology/qualitative-research/

Is this article helpful?

Pritha Bhandari

Other students also liked, qualitative vs. quantitative research | differences, examples & methods, how to do thematic analysis | step-by-step guide & examples, what is your plagiarism score.

• Privacy Policy

Home » Qualitative Research – Methods, Analysis Types and Guide

Qualitative Research – Methods, Analysis Types and Guide

Table of Contents

Qualitative Research

Qualitative research is a type of research methodology that focuses on exploring and understanding people’s beliefs, attitudes, behaviors, and experiences through the collection and analysis of non-numerical data. It seeks to answer research questions through the examination of subjective data, such as interviews, focus groups, observations, and textual analysis.

Qualitative research aims to uncover the meaning and significance of social phenomena, and it typically involves a more flexible and iterative approach to data collection and analysis compared to quantitative research. Qualitative research is often used in fields such as sociology, anthropology, psychology, and education.

Qualitative Research Methods

Qualitative Research Methods are as follows:

One-to-One Interview

This method involves conducting an interview with a single participant to gain a detailed understanding of their experiences, attitudes, and beliefs. One-to-one interviews can be conducted in-person, over the phone, or through video conferencing. The interviewer typically uses open-ended questions to encourage the participant to share their thoughts and feelings. One-to-one interviews are useful for gaining detailed insights into individual experiences.

Focus Groups

This method involves bringing together a group of people to discuss a specific topic in a structured setting. The focus group is led by a moderator who guides the discussion and encourages participants to share their thoughts and opinions. Focus groups are useful for generating ideas and insights, exploring social norms and attitudes, and understanding group dynamics.

Ethnographic Studies

This method involves immersing oneself in a culture or community to gain a deep understanding of its norms, beliefs, and practices. Ethnographic studies typically involve long-term fieldwork and observation, as well as interviews and document analysis. Ethnographic studies are useful for understanding the cultural context of social phenomena and for gaining a holistic understanding of complex social processes.

Text Analysis

This method involves analyzing written or spoken language to identify patterns and themes. Text analysis can be quantitative or qualitative. Qualitative text analysis involves close reading and interpretation of texts to identify recurring themes, concepts, and patterns. Text analysis is useful for understanding media messages, public discourse, and cultural trends.

This method involves an in-depth examination of a single person, group, or event to gain an understanding of complex phenomena. Case studies typically involve a combination of data collection methods, such as interviews, observations, and document analysis, to provide a comprehensive understanding of the case. Case studies are useful for exploring unique or rare cases, and for generating hypotheses for further research.

Process of Observation

This method involves systematically observing and recording behaviors and interactions in natural settings. The observer may take notes, use audio or video recordings, or use other methods to document what they see. Process of observation is useful for understanding social interactions, cultural practices, and the context in which behaviors occur.

Record Keeping

This method involves keeping detailed records of observations, interviews, and other data collected during the research process. Record keeping is essential for ensuring the accuracy and reliability of the data, and for providing a basis for analysis and interpretation.

This method involves collecting data from a large sample of participants through a structured questionnaire. Surveys can be conducted in person, over the phone, through mail, or online. Surveys are useful for collecting data on attitudes, beliefs, and behaviors, and for identifying patterns and trends in a population.

Qualitative data analysis is a process of turning unstructured data into meaningful insights. It involves extracting and organizing information from sources like interviews, focus groups, and surveys. The goal is to understand people’s attitudes, behaviors, and motivations

Qualitative Research Analysis Methods

Qualitative Research analysis methods involve a systematic approach to interpreting and making sense of the data collected in qualitative research. Here are some common qualitative data analysis methods:

Thematic Analysis

This method involves identifying patterns or themes in the data that are relevant to the research question. The researcher reviews the data, identifies keywords or phrases, and groups them into categories or themes. Thematic analysis is useful for identifying patterns across multiple data sources and for generating new insights into the research topic.

Content Analysis

This method involves analyzing the content of written or spoken language to identify key themes or concepts. Content analysis can be quantitative or qualitative. Qualitative content analysis involves close reading and interpretation of texts to identify recurring themes, concepts, and patterns. Content analysis is useful for identifying patterns in media messages, public discourse, and cultural trends.

Discourse Analysis

This method involves analyzing language to understand how it constructs meaning and shapes social interactions. Discourse analysis can involve a variety of methods, such as conversation analysis, critical discourse analysis, and narrative analysis. Discourse analysis is useful for understanding how language shapes social interactions, cultural norms, and power relationships.

Grounded Theory Analysis

This method involves developing a theory or explanation based on the data collected. Grounded theory analysis starts with the data and uses an iterative process of coding and analysis to identify patterns and themes in the data. The theory or explanation that emerges is grounded in the data, rather than preconceived hypotheses. Grounded theory analysis is useful for understanding complex social phenomena and for generating new theoretical insights.

Narrative Analysis

This method involves analyzing the stories or narratives that participants share to gain insights into their experiences, attitudes, and beliefs. Narrative analysis can involve a variety of methods, such as structural analysis, thematic analysis, and discourse analysis. Narrative analysis is useful for understanding how individuals construct their identities, make sense of their experiences, and communicate their values and beliefs.

Phenomenological Analysis

This method involves analyzing how individuals make sense of their experiences and the meanings they attach to them. Phenomenological analysis typically involves in-depth interviews with participants to explore their experiences in detail. Phenomenological analysis is useful for understanding subjective experiences and for developing a rich understanding of human consciousness.

Comparative Analysis

This method involves comparing and contrasting data across different cases or groups to identify similarities and differences. Comparative analysis can be used to identify patterns or themes that are common across multiple cases, as well as to identify unique or distinctive features of individual cases. Comparative analysis is useful for understanding how social phenomena vary across different contexts and groups.

Applications of Qualitative Research

Qualitative research has many applications across different fields and industries. Here are some examples of how qualitative research is used:

• Market Research: Qualitative research is often used in market research to understand consumer attitudes, behaviors, and preferences. Researchers conduct focus groups and one-on-one interviews with consumers to gather insights into their experiences and perceptions of products and services.
• Health Care: Qualitative research is used in health care to explore patient experiences and perspectives on health and illness. Researchers conduct in-depth interviews with patients and their families to gather information on their experiences with different health care providers and treatments.
• Education: Qualitative research is used in education to understand student experiences and to develop effective teaching strategies. Researchers conduct classroom observations and interviews with students and teachers to gather insights into classroom dynamics and instructional practices.
• Social Work : Qualitative research is used in social work to explore social problems and to develop interventions to address them. Researchers conduct in-depth interviews with individuals and families to understand their experiences with poverty, discrimination, and other social problems.
• Anthropology : Qualitative research is used in anthropology to understand different cultures and societies. Researchers conduct ethnographic studies and observe and interview members of different cultural groups to gain insights into their beliefs, practices, and social structures.
• Psychology : Qualitative research is used in psychology to understand human behavior and mental processes. Researchers conduct in-depth interviews with individuals to explore their thoughts, feelings, and experiences.
• Public Policy : Qualitative research is used in public policy to explore public attitudes and to inform policy decisions. Researchers conduct focus groups and one-on-one interviews with members of the public to gather insights into their perspectives on different policy issues.

How to Conduct Qualitative Research

Here are some general steps for conducting qualitative research:

• Identify your research question: Qualitative research starts with a research question or set of questions that you want to explore. This question should be focused and specific, but also broad enough to allow for exploration and discovery.
• Select your research design: There are different types of qualitative research designs, including ethnography, case study, grounded theory, and phenomenology. You should select a design that aligns with your research question and that will allow you to gather the data you need to answer your research question.
• Recruit participants: Once you have your research question and design, you need to recruit participants. The number of participants you need will depend on your research design and the scope of your research. You can recruit participants through advertisements, social media, or through personal networks.
• Collect data: There are different methods for collecting qualitative data, including interviews, focus groups, observation, and document analysis. You should select the method or methods that align with your research design and that will allow you to gather the data you need to answer your research question.
• Analyze data: Once you have collected your data, you need to analyze it. This involves reviewing your data, identifying patterns and themes, and developing codes to organize your data. You can use different software programs to help you analyze your data, or you can do it manually.
• Interpret data: Once you have analyzed your data, you need to interpret it. This involves making sense of the patterns and themes you have identified, and developing insights and conclusions that answer your research question. You should be guided by your research question and use your data to support your conclusions.
• Communicate results: Once you have interpreted your data, you need to communicate your results. This can be done through academic papers, presentations, or reports. You should be clear and concise in your communication, and use examples and quotes from your data to support your findings.

Examples of Qualitative Research

Here are some real-time examples of qualitative research:

• Customer Feedback: A company may conduct qualitative research to understand the feedback and experiences of its customers. This may involve conducting focus groups or one-on-one interviews with customers to gather insights into their attitudes, behaviors, and preferences.
• Healthcare : A healthcare provider may conduct qualitative research to explore patient experiences and perspectives on health and illness. This may involve conducting in-depth interviews with patients and their families to gather information on their experiences with different health care providers and treatments.
• Education : An educational institution may conduct qualitative research to understand student experiences and to develop effective teaching strategies. This may involve conducting classroom observations and interviews with students and teachers to gather insights into classroom dynamics and instructional practices.
• Social Work: A social worker may conduct qualitative research to explore social problems and to develop interventions to address them. This may involve conducting in-depth interviews with individuals and families to understand their experiences with poverty, discrimination, and other social problems.
• Anthropology : An anthropologist may conduct qualitative research to understand different cultures and societies. This may involve conducting ethnographic studies and observing and interviewing members of different cultural groups to gain insights into their beliefs, practices, and social structures.
• Psychology : A psychologist may conduct qualitative research to understand human behavior and mental processes. This may involve conducting in-depth interviews with individuals to explore their thoughts, feelings, and experiences.
• Public Policy: A government agency or non-profit organization may conduct qualitative research to explore public attitudes and to inform policy decisions. This may involve conducting focus groups and one-on-one interviews with members of the public to gather insights into their perspectives on different policy issues.

Purpose of Qualitative Research

The purpose of qualitative research is to explore and understand the subjective experiences, behaviors, and perspectives of individuals or groups in a particular context. Unlike quantitative research, which focuses on numerical data and statistical analysis, qualitative research aims to provide in-depth, descriptive information that can help researchers develop insights and theories about complex social phenomena.

Qualitative research can serve multiple purposes, including:

• Exploring new or emerging phenomena : Qualitative research can be useful for exploring new or emerging phenomena, such as new technologies or social trends. This type of research can help researchers develop a deeper understanding of these phenomena and identify potential areas for further study.
• Understanding complex social phenomena : Qualitative research can be useful for exploring complex social phenomena, such as cultural beliefs, social norms, or political processes. This type of research can help researchers develop a more nuanced understanding of these phenomena and identify factors that may influence them.
• Generating new theories or hypotheses: Qualitative research can be useful for generating new theories or hypotheses about social phenomena. By gathering rich, detailed data about individuals’ experiences and perspectives, researchers can develop insights that may challenge existing theories or lead to new lines of inquiry.
• Providing context for quantitative data: Qualitative research can be useful for providing context for quantitative data. By gathering qualitative data alongside quantitative data, researchers can develop a more complete understanding of complex social phenomena and identify potential explanations for quantitative findings.

When to use Qualitative Research

Here are some situations where qualitative research may be appropriate:

• Exploring a new area: If little is known about a particular topic, qualitative research can help to identify key issues, generate hypotheses, and develop new theories.
• Understanding complex phenomena: Qualitative research can be used to investigate complex social, cultural, or organizational phenomena that are difficult to measure quantitatively.
• Investigating subjective experiences: Qualitative research is particularly useful for investigating the subjective experiences of individuals or groups, such as their attitudes, beliefs, values, or emotions.
• Conducting formative research: Qualitative research can be used in the early stages of a research project to develop research questions, identify potential research participants, and refine research methods.
• Evaluating interventions or programs: Qualitative research can be used to evaluate the effectiveness of interventions or programs by collecting data on participants’ experiences, attitudes, and behaviors.

Characteristics of Qualitative Research

Qualitative research is characterized by several key features, including:

• Focus on subjective experience: Qualitative research is concerned with understanding the subjective experiences, beliefs, and perspectives of individuals or groups in a particular context. Researchers aim to explore the meanings that people attach to their experiences and to understand the social and cultural factors that shape these meanings.
• Use of open-ended questions: Qualitative research relies on open-ended questions that allow participants to provide detailed, in-depth responses. Researchers seek to elicit rich, descriptive data that can provide insights into participants’ experiences and perspectives.
• Sampling-based on purpose and diversity: Qualitative research often involves purposive sampling, in which participants are selected based on specific criteria related to the research question. Researchers may also seek to include participants with diverse experiences and perspectives to capture a range of viewpoints.
• Data collection through multiple methods: Qualitative research typically involves the use of multiple data collection methods, such as in-depth interviews, focus groups, and observation. This allows researchers to gather rich, detailed data from multiple sources, which can provide a more complete picture of participants’ experiences and perspectives.
• Inductive data analysis: Qualitative research relies on inductive data analysis, in which researchers develop theories and insights based on the data rather than testing pre-existing hypotheses. Researchers use coding and thematic analysis to identify patterns and themes in the data and to develop theories and explanations based on these patterns.
• Emphasis on researcher reflexivity: Qualitative research recognizes the importance of the researcher’s role in shaping the research process and outcomes. Researchers are encouraged to reflect on their own biases and assumptions and to be transparent about their role in the research process.

Advantages of Qualitative Research

Qualitative research offers several advantages over other research methods, including:

• Depth and detail: Qualitative research allows researchers to gather rich, detailed data that provides a deeper understanding of complex social phenomena. Through in-depth interviews, focus groups, and observation, researchers can gather detailed information about participants’ experiences and perspectives that may be missed by other research methods.
• Flexibility : Qualitative research is a flexible approach that allows researchers to adapt their methods to the research question and context. Researchers can adjust their research methods in real-time to gather more information or explore unexpected findings.
• Contextual understanding: Qualitative research is well-suited to exploring the social and cultural context in which individuals or groups are situated. Researchers can gather information about cultural norms, social structures, and historical events that may influence participants’ experiences and perspectives.
• Participant perspective : Qualitative research prioritizes the perspective of participants, allowing researchers to explore subjective experiences and understand the meanings that participants attach to their experiences.
• Theory development: Qualitative research can contribute to the development of new theories and insights about complex social phenomena. By gathering rich, detailed data and using inductive data analysis, researchers can develop new theories and explanations that may challenge existing understandings.
• Validity : Qualitative research can offer high validity by using multiple data collection methods, purposive and diverse sampling, and researcher reflexivity. This can help ensure that findings are credible and trustworthy.

Limitations of Qualitative Research

Qualitative research also has some limitations, including:

• Subjectivity : Qualitative research relies on the subjective interpretation of researchers, which can introduce bias into the research process. The researcher’s perspective, beliefs, and experiences can influence the way data is collected, analyzed, and interpreted.
• Limited generalizability: Qualitative research typically involves small, purposive samples that may not be representative of larger populations. This limits the generalizability of findings to other contexts or populations.
• Time-consuming: Qualitative research can be a time-consuming process, requiring significant resources for data collection, analysis, and interpretation.
• Resource-intensive: Qualitative research may require more resources than other research methods, including specialized training for researchers, specialized software for data analysis, and transcription services.
• Limited reliability: Qualitative research may be less reliable than quantitative research, as it relies on the subjective interpretation of researchers. This can make it difficult to replicate findings or compare results across different studies.
• Ethics and confidentiality: Qualitative research involves collecting sensitive information from participants, which raises ethical concerns about confidentiality and informed consent. Researchers must take care to protect the privacy and confidentiality of participants and obtain informed consent.

Also see Research Methods

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Ethnographic Research -Types, Methods and Guide

Applied Research – Types, Methods and Examples

Focus Groups – Steps, Examples and Guide

Experimental Design – Types, Methods, Guide

Case Study – Methods, Examples and Guide

Qualitative Research Methods

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Qualitative Data Analysis

23 Presenting the Results of Qualitative Analysis

Mikaila Mariel Lemonik Arthur

Qualitative research is not finished just because you have determined the main findings or conclusions of your study. Indeed, disseminating the results is an essential part of the research process. By sharing your results with others, whether in written form as scholarly paper or an applied report or in some alternative format like an oral presentation, an infographic, or a video, you ensure that your findings become part of the ongoing conversation of scholarship in your field, forming part of the foundation for future researchers. This chapter provides an introduction to writing about qualitative research findings. It will outline how writing continues to contribute to the analysis process, what concerns researchers should keep in mind as they draft their presentations of findings, and how best to organize qualitative research writing

As you move through the research process, it is essential to keep yourself organized. Organizing your data, memos, and notes aids both the analytical and the writing processes. Whether you use electronic or physical, real-world filing and organizational systems, these systems help make sense of the mountains of data you have and assure you focus your attention on the themes and ideas you have determined are important (Warren and Karner 2015). Be sure that you have kept detailed notes on all of the decisions you have made and procedures you have followed in carrying out research design, data collection, and analysis, as these will guide your ultimate write-up.

First and foremost, researchers should keep in mind that writing is in fact a form of thinking. Writing is an excellent way to discover ideas and arguments and to further develop an analysis. As you write, more ideas will occur to you, things that were previously confusing will start to make sense, and arguments will take a clear shape rather than being amorphous and poorly-organized. However, writing-as-thinking cannot be the final version that you share with others. Good-quality writing does not display the workings of your thought process. It is reorganized and revised (more on that later) to present the data and arguments important in a particular piece. And revision is totally normal! No one expects the first draft of a piece of writing to be ready for prime time. So write rough drafts and memos and notes to yourself and use them to think, and then revise them until the piece is the way you want it to be for sharing.

Bergin (2018) lays out a set of key concerns for appropriate writing about research. First, present your results accurately, without exaggerating or misrepresenting. It is very easy to overstate your findings by accident if you are enthusiastic about what you have found, so it is important to take care and use appropriate cautions about the limitations of the research. You also need to work to ensure that you communicate your findings in a way people can understand, using clear and appropriate language that is adjusted to the level of those you are communicating with. And you must be clear and transparent about the methodological strategies employed in the research. Remember, the goal is, as much as possible, to describe your research in a way that would permit others to replicate the study. There are a variety of other concerns and decision points that qualitative researchers must keep in mind, including the extent to which to include quantification in their presentation of results, ethics, considerations of audience and voice, and how to bring the richness of qualitative data to life.

Quantification, as you have learned, refers to the process of turning data into numbers. It can indeed be very useful to count and tabulate quantitative data drawn from qualitative research. For instance, if you were doing a study of dual-earner households and wanted to know how many had an equal division of household labor and how many did not, you might want to count those numbers up and include them as part of the final write-up. However, researchers need to take care when they are writing about quantified qualitative data. Qualitative data is not as generalizable as quantitative data, so quantification can be very misleading. Thus, qualitative researchers should strive to use raw numbers instead of the percentages that are more appropriate for quantitative research. Writing, for instance, “15 of the 20 people I interviewed prefer pancakes to waffles” is a simple description of the data; writing “75% of people prefer pancakes” suggests a generalizable claim that is not likely supported by the data. Note that mixing numbers with qualitative data is really a type of mixed-methods approach. Mixed-methods approaches are good, but sometimes they seduce researchers into focusing on the persuasive power of numbers and tables rather than capitalizing on the inherent richness of their qualitative data.

A variety of issues of scholarly ethics and research integrity are raised by the writing process. Some of these are unique to qualitative research, while others are more universal concerns for all academic and professional writing. For example, it is essential to avoid plagiarism and misuse of sources. All quotations that appear in a text must be properly cited, whether with in-text and bibliographic citations to the source or with an attribution to the research participant (or the participant’s pseudonym or description in order to protect confidentiality) who said those words. Where writers will paraphrase a text or a participant’s words, they need to make sure that the paraphrase they develop accurately reflects the meaning of the original words. Thus, some scholars suggest that participants should have the opportunity to read (or to have read to them, if they cannot read the text themselves) all sections of the text in which they, their words, or their ideas are presented to ensure accuracy and enable participants to maintain control over their lives.

Audience and Voice

When writing, researchers must consider their audience(s) and the effects they want their writing to have on these audiences. The designated audience will dictate the voice used in the writing, or the individual style and personality of a piece of text. Keep in mind that the potential audience for qualitative research is often much more diverse than that for quantitative research because of the accessibility of the data and the extent to which the writing can be accessible and interesting. Yet individual pieces of writing are typically pitched to a more specific subset of the audience.

Let us consider one potential research study, an ethnography involving participant-observation of the same children both when they are at daycare facility and when they are at home with their families to try to understand how daycare might impact behavior and social development. The findings of this study might be of interest to a wide variety of potential audiences: academic peers, whether at your own academic institution, in your broader discipline, or multidisciplinary; people responsible for creating laws and policies; practitioners who run or teach at day care centers; and the general public, including both people who are interested in child development more generally and those who are themselves parents making decisions about child care for their own children. And the way you write for each of these audiences will be somewhat different. Take a moment and think through what some of these differences might look like.

If you are writing to academic audiences, using specialized academic language and working within the typical constraints of scholarly genres, as will be discussed below, can be an important part of convincing others that your work is legitimate and should be taken seriously. Your writing will be formal. Even if you are writing for students and faculty you already know—your classmates, for instance—you are often asked to imitate the style of academic writing that is used in publications, as this is part of learning to become part of the scholarly conversation. When speaking to academic audiences outside your discipline, you may need to be more careful about jargon and specialized language, as disciplines do not always share the same key terms. For instance, in sociology, scholars use the term diffusion to refer to the way new ideas or practices spread from organization to organization. In the field of international relations, scholars often used the term cascade to refer to the way ideas or practices spread from nation to nation. These terms are describing what is fundamentally the same concept, but they are different terms—and a scholar from one field might have no idea what a scholar from a different field is talking about! Therefore, while the formality and academic structure of the text would stay the same, a writer with a multidisciplinary audience might need to pay more attention to defining their terms in the body of the text.

It is not only other academic scholars who expect to see formal writing. Policymakers tend to expect formality when ideas are presented to them, as well. However, the content and style of the writing will be different. Much less academic jargon should be used, and the most important findings and policy implications should be emphasized right from the start rather than initially focusing on prior literature and theoretical models as you might for an academic audience. Long discussions of research methods should also be minimized. Similarly, when you write for practitioners, the findings and implications for practice should be highlighted. The reading level of the text will vary depending on the typical background of the practitioners to whom you are writing—you can make very different assumptions about the general knowledge and reading abilities of a group of hospital medical directors with MDs than you can about a group of case workers who have a post-high-school certificate. Consider the primary language of your audience as well. The fact that someone can get by in spoken English does not mean they have the vocabulary or English reading skills to digest a complex report. But the fact that someone’s vocabulary is limited says little about their intellectual abilities, so try your best to convey the important complexity of the ideas and findings from your research without dumbing them down—even if you must limit your vocabulary usage.

When writing for the general public, you will want to move even further towards emphasizing key findings and policy implications, but you also want to draw on the most interesting aspects of your data. General readers will read sociological texts that are rich with ethnographic or other kinds of detail—it is almost like reality television on a page! And this is a contrast to busy policymakers and practitioners, who probably want to learn the main findings as quickly as possible so they can go about their busy lives. But also keep in mind that there is a wide variation in reading levels. Journalists at publications pegged to the general public are often advised to write at about a tenth-grade reading level, which would leave most of the specialized terminology we develop in our research fields out of reach. If you want to be accessible to even more people, your vocabulary must be even more limited. The excellent exercise of trying to write using the 1,000 most common English words, available at the Up-Goer Five website ( https://www.splasho.com/upgoer5/ ) does a good job of illustrating this challenge (Sanderson n.d.).

Another element of voice is whether to write in the first person. While many students are instructed to avoid the use of the first person in academic writing, this advice needs to be taken with a grain of salt. There are indeed many contexts in which the first person is best avoided, at least as long as writers can find ways to build strong, comprehensible sentences without its use, including most quantitative research writing. However, if the alternative to using the first person is crafting a sentence like “it is proposed that the researcher will conduct interviews,” it is preferable to write “I propose to conduct interviews.” In qualitative research, in fact, the use of the first person is far more common. This is because the researcher is central to the research project. Qualitative researchers can themselves be understood as research instruments, and thus eliminating the use of the first person in writing is in a sense eliminating information about the conduct of the researchers themselves.

But the question really extends beyond the issue of first-person or third-person. Qualitative researchers have choices about how and whether to foreground themselves in their writing, not just in terms of using the first person, but also in terms of whether to emphasize their own subjectivity and reflexivity, their impressions and ideas, and their role in the setting. In contrast, conventional quantitative research in the positivist tradition really tries to eliminate the author from the study—which indeed is exactly why typical quantitative research avoids the use of the first person. Keep in mind that emphasizing researchers’ roles and reflexivity and using the first person does not mean crafting articles that provide overwhelming detail about the author’s thoughts and practices. Readers do not need to hear, and should not be told, which database you used to search for journal articles, how many hours you spent transcribing, or whether the research process was stressful—save these things for the memos you write to yourself. Rather, readers need to hear how you interacted with research participants, how your standpoint may have shaped the findings, and what analytical procedures you carried out.

Making Data Come Alive

One of the most important parts of writing about qualitative research is presenting the data in a way that makes its richness and value accessible to readers. As the discussion of analysis in the prior chapter suggests, there are a variety of ways to do this. Researchers may select key quotes or images to illustrate points, write up specific case studies that exemplify their argument, or develop vignettes (little stories) that illustrate ideas and themes, all drawing directly on the research data. Researchers can also write more lengthy summaries, narratives, and thick descriptions.

Nearly all qualitative work includes quotes from research participants or documents to some extent, though ethnographic work may focus more on thick description than on relaying participants’ own words. When quotes are presented, they must be explained and interpreted—they cannot stand on their own. This is one of the ways in which qualitative research can be distinguished from journalism. Journalism presents what happened, but social science needs to present the “why,” and the why is best explained by the researcher.

So how do authors go about integrating quotes into their written work? Julie Posselt (2017), a sociologist who studies graduate education, provides a set of instructions. First of all, authors need to remain focused on the core questions of their research, and avoid getting distracted by quotes that are interesting or attention-grabbing but not so relevant to the research question. Selecting the right quotes, those that illustrate the ideas and arguments of the paper, is an important part of the writing process. Second, not all quotes should be the same length (just like not all sentences or paragraphs in a paper should be the same length). Include some quotes that are just phrases, others that are a sentence or so, and others that are longer. We call longer quotes, generally those more than about three lines long, block quotes , and they are typically indented on both sides to set them off from the surrounding text. For all quotes, be sure to summarize what the quote should be telling or showing the reader, connect this quote to other quotes that are similar or different, and provide transitions in the discussion to move from quote to quote and from topic to topic. Especially for longer quotes, it is helpful to do some of this writing before the quote to preview what is coming and other writing after the quote to make clear what readers should have come to understand. Remember, it is always the author’s job to interpret the data. Presenting excerpts of the data, like quotes, in a form the reader can access does not minimize the importance of this job. Be sure that you are explaining the meaning of the data you present.

A few more notes about writing with quotes: avoid patchwriting, whether in your literature review or the section of your paper in which quotes from respondents are presented. Patchwriting is a writing practice wherein the author lightly paraphrases original texts but stays so close to those texts that there is little the author has added. Sometimes, this even takes the form of presenting a series of quotes, properly documented, with nothing much in the way of text generated by the author. A patchwriting approach does not build the scholarly conversation forward, as it does not represent any kind of new contribution on the part of the author. It is of course fine to paraphrase quotes, as long as the meaning is not changed. But if you use direct quotes, do not edit the text of the quotes unless how you edit them does not change the meaning and you have made clear through the use of ellipses (…) and brackets ([])what kinds of edits have been made. For example, consider this exchange from Matthew Desmond’s (2012:1317) research on evictions:

The thing was, I wasn’t never gonna let Crystal come and stay with me from the get go. I just told her that to throw her off. And she wasn’t fittin’ to come stay with me with no money…No. Nope. You might as well stay in that shelter.

A paraphrase of this exchange might read “She said that she was going to let Crystal stay with her if Crystal did not have any money.” Paraphrases like that are fine. What is not fine is rewording the statement but treating it like a quote, for instance writing:

The thing was, I was not going to let Crystal come and stay with me from beginning. I just told her that to throw her off. And it was not proper for her to come stay with me without any money…No. Nope. You might as well stay in that shelter.

But as you can see, the change in language and style removes some of the distinct meaning of the original quote. Instead, writers should leave as much of the original language as possible. If some text in the middle of the quote needs to be removed, as in this example, ellipses are used to show that this has occurred. And if a word needs to be added to clarify, it is placed in square brackets to show that it was not part of the original quote.

Data can also be presented through the use of data displays like tables, charts, graphs, diagrams, and infographics created for publication or presentation, as well as through the use of visual material collected during the research process. Note that if visuals are used, the author must have the legal right to use them. Photographs or diagrams created by the author themselves—or by research participants who have signed consent forms for their work to be used, are fine. But photographs, and sometimes even excerpts from archival documents, may be owned by others from whom researchers must get permission in order to use them.

A large percentage of qualitative research does not include any data displays or visualizations. Therefore, researchers should carefully consider whether the use of data displays will help the reader understand the data. One of the most common types of data displays used by qualitative researchers are simple tables. These might include tables summarizing key data about cases included in the study; tables laying out the characteristics of different taxonomic elements or types developed as part of the analysis; tables counting the incidence of various elements; and 2×2 tables (two columns and two rows) illuminating a theory. Basic network or process diagrams are also commonly included. If data displays are used, it is essential that researchers include context and analysis alongside data displays rather than letting them stand by themselves, and it is preferable to continue to present excerpts and examples from the data rather than just relying on summaries in the tables.

If you will be using graphs, infographics, or other data visualizations, it is important that you attend to making them useful and accurate (Bergin 2018). Think about the viewer or user as your audience and ensure the data visualizations will be comprehensible. You may need to include more detail or labels than you might think. Ensure that data visualizations are laid out and labeled clearly and that you make visual choices that enhance viewers’ ability to understand the points you intend to communicate using the visual in question. Finally, given the ease with which it is possible to design visuals that are deceptive or misleading, it is essential to make ethical and responsible choices in the construction of visualization so that viewers will interpret them in accurate ways.

The Genre of Research Writing

As discussed above, the style and format in which results are presented depends on the audience they are intended for. These differences in styles and format are part of the genre of writing. Genre is a term referring to the rules of a specific form of creative or productive work. Thus, the academic journal article—and student papers based on this form—is one genre. A report or policy paper is another. The discussion below will focus on the academic journal article, but note that reports and policy papers follow somewhat different formats. They might begin with an executive summary of one or a few pages, include minimal background, focus on key findings, and conclude with policy implications, shifting methods and details about the data to an appendix. But both academic journal articles and policy papers share some things in common, for instance the necessity for clear writing, a well-organized structure, and the use of headings.

So what factors make up the genre of the academic journal article in sociology? While there is some flexibility, particularly for ethnographic work, academic journal articles tend to follow a fairly standard format. They begin with a “title page” that includes the article title (often witty and involving scholarly inside jokes, but more importantly clearly describing the content of the article); the authors’ names and institutional affiliations, an abstract , and sometimes keywords designed to help others find the article in databases. An abstract is a short summary of the article that appears both at the very beginning of the article and in search databases. Abstracts are designed to aid readers by giving them the opportunity to learn enough about an article that they can determine whether it is worth their time to read the complete text. They are written about the article, and thus not in the first person, and clearly summarize the research question, methodological approach, main findings, and often the implications of the research.

After the abstract comes an “introduction” of a page or two that details the research question, why it matters, and what approach the paper will take. This is followed by a literature review of about a quarter to a third the length of the entire paper. The literature review is often divided, with headings, into topical subsections, and is designed to provide a clear, thorough overview of the prior research literature on which a paper has built—including prior literature the new paper contradicts. At the end of the literature review it should be made clear what researchers know about the research topic and question, what they do not know, and what this new paper aims to do to address what is not known.

The next major section of the paper is the section that describes research design, data collection, and data analysis, often referred to as “research methods” or “methodology.” This section is an essential part of any written or oral presentation of your research. Here, you tell your readers or listeners “how you collected and interpreted your data” (Taylor, Bogdan, and DeVault 2016:215). Taylor, Bogdan, and DeVault suggest that the discussion of your research methods include the following:

• The particular approach to data collection used in the study;
• Any theoretical perspective(s) that shaped your data collection and analytical approach;
• When the study occurred, over how long, and where (concealing identifiable details as needed);
• A description of the setting and participants, including sampling and selection criteria (if an interview-based study, the number of participants should be clearly stated);
• The researcher’s perspective in carrying out the study, including relevant elements of their identity and standpoint, as well as their role (if any) in research settings; and
• The approach to analyzing the data.

After the methods section comes a section, variously titled but often called “data,” that takes readers through the analysis. This section is where the thick description narrative; the quotes, broken up by theme or topic, with their interpretation; the discussions of case studies; most data displays (other than perhaps those outlining a theoretical model or summarizing descriptive data about cases); and other similar material appears. The idea of the data section is to give readers the ability to see the data for themselves and to understand how this data supports the ultimate conclusions. Note that all tables and figures included in formal publications should be titled and numbered.

At the end of the paper come one or two summary sections, often called “discussion” and/or “conclusion.” If there is a separate discussion section, it will focus on exploring the overall themes and findings of the paper. The conclusion clearly and succinctly summarizes the findings and conclusions of the paper, the limitations of the research and analysis, any suggestions for future research building on the paper or addressing these limitations, and implications, be they for scholarship and theory or policy and practice.

After the end of the textual material in the paper comes the bibliography, typically called “works cited” or “references.” The references should appear in a consistent citation style—in sociology, we often use the American Sociological Association format (American Sociological Association 2019), but other formats may be used depending on where the piece will eventually be published. Care should be taken to ensure that in-text citations also reflect the chosen citation style. In some papers, there may be an appendix containing supplemental information such as a list of interview questions or an additional data visualization.

Note that when researchers give presentations to scholarly audiences, the presentations typically follow a format similar to that of scholarly papers, though given time limitations they are compressed. Abstracts and works cited are often not part of the presentation, though in-text citations are still used. The literature review presented will be shortened to only focus on the most important aspects of the prior literature, and only key examples from the discussion of data will be included. For long or complex papers, sometimes only one of several findings is the focus of the presentation. Of course, presentations for other audiences may be constructed differently, with greater attention to interesting elements of the data and findings as well as implications and less to the literature review and methods.

Concluding Your Work

After you have written a complete draft of the paper, be sure you take the time to revise and edit your work. There are several important strategies for revision. First, put your work away for a little while. Even waiting a day to revise is better than nothing, but it is best, if possible, to take much more time away from the text. This helps you forget what your writing looks like and makes it easier to find errors, mistakes, and omissions. Second, show your work to others. Ask them to read your work and critique it, pointing out places where the argument is weak, where you may have overlooked alternative explanations, where the writing could be improved, and what else you need to work on. Finally, read your work out loud to yourself (or, if you really need an audience, try reading to some stuffed animals). Reading out loud helps you catch wrong words, tricky sentences, and many other issues. But as important as revision is, try to avoid perfectionism in writing (Warren and Karner 2015). Writing can always be improved, no matter how much time you spend on it. Those improvements, however, have diminishing returns, and at some point the writing process needs to conclude so the writing can be shared with the world.

Of course, the main goal of writing up the results of a research project is to share with others. Thus, researchers should be considering how they intend to disseminate their results. What conferences might be appropriate? Where can the paper be submitted? Note that if you are an undergraduate student, there are a wide variety of journals that accept and publish research conducted by undergraduates. Some publish across disciplines, while others are specific to disciplines. Other work, such as reports, may be best disseminated by publication online on relevant organizational websites.

After a project is completed, be sure to take some time to organize your research materials and archive them for longer-term storage. Some Institutional Review Board (IRB) protocols require that original data, such as interview recordings, transcripts, and field notes, be preserved for a specific number of years in a protected (locked for paper or password-protected for digital) form and then destroyed, so be sure that your plans adhere to the IRB requirements. Be sure you keep any materials that might be relevant for future related research or for answering questions people may ask later about your project.

And then what? Well, then it is time to move on to your next research project. Research is a long-term endeavor, not a one-time-only activity. We build our skills and our expertise as we continue to pursue research. So keep at it.

• Find a short article that uses qualitative methods. The sociological magazine Contexts is a good place to find such pieces. Write an abstract of the article.
• Choose a sociological journal article on a topic you are interested in that uses some form of qualitative methods and is at least 20 pages long. Rewrite the article as a five-page research summary accessible to non-scholarly audiences.
• Choose a concept or idea you have learned in this course and write an explanation of it using the Up-Goer Five Text Editor ( https://www.splasho.com/upgoer5/ ), a website that restricts your writing to the 1,000 most common English words. What was this experience like? What did it teach you about communicating with people who have a more limited English-language vocabulary—and what did it teach you about the utility of having access to complex academic language?
• Select five or more sociological journal articles that all use the same basic type of qualitative methods (interviewing, ethnography, documents, or visual sociology). Using what you have learned about coding, code the methods sections of each article, and use your coding to figure out what is common in how such articles discuss their research design, data collection, and analysis methods.
• Return to an exercise you completed earlier in this course and revise your work. What did you change? How did revising impact the final product?
• Find a quote from the transcript of an interview, a social media post, or elsewhere that has not yet been interpreted or explained. Write a paragraph that includes the quote along with an explanation of its sociological meaning or significance.

The style or personality of a piece of writing, including such elements as tone, word choice, syntax, and rhythm.

A quotation, usually one of some length, which is set off from the main text by being indented on both sides rather than being placed in quotation marks.

A classification of written or artistic work based on form, content, and style.

A short summary of a text written from the perspective of a reader rather than from the perspective of an author.

Social Data Analysis Copyright © 2021 by Mikaila Mariel Lemonik Arthur is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

blog @ precision

Presenting your qualitative analysis findings: tables to include in chapter 4.

The earliest stages of developing a doctoral dissertation—most specifically the topic development  and literature review  stages—require that you immerse yourself in a ton of existing research related to your potential topic. If you have begun writing your dissertation proposal, you have undoubtedly reviewed countless results and findings sections of studies in order to help gain an understanding of what is currently known about your topic. 

In this process, we’re guessing that you observed a distinct pattern: Results sections are full of tables. Indeed, the results chapter for your own dissertation will need to be similarly packed with tables. So, if you’re preparing to write up the results of your statistical analysis or qualitative analysis, it will probably help to review your APA editing  manual to brush up on your table formatting skills. But, aside from formatting, how should you develop the tables in your results chapter?

In quantitative studies, tables are a handy way of presenting the variety of statistical analysis results in a form that readers can easily process. You’ve probably noticed that quantitative studies present descriptive results like mean, mode, range, standard deviation, etc., as well the inferential results that indicate whether significant relationships or differences were found through the statistical analysis . These are pretty standard tables that you probably learned about in your pre-dissertation statistics courses.

But, what if you are conducting qualitative analysis? What tables are appropriate for this type of study? This is a question we hear often from our dissertation assistance  clients, and with good reason. University guidelines for results chapters often contain vague instructions that guide you to include “appropriate tables” without specifying what exactly those are. To help clarify on this point, we asked our qualitative analysis experts to share their recommendations for tables to include in your Chapter 4.

Demographics Tables

As with studies using quantitative methods , presenting an overview of your sample demographics is useful in studies that use qualitative research methods. The standard demographics table in a quantitative study provides aggregate information for what are often large samples. In other words, such tables present totals and percentages for demographic categories within the sample that are relevant to the study (e.g., age, gender, job title). 

If conducting qualitative research  for your dissertation, however, you will use a smaller sample and obtain richer data from each participant than in quantitative studies. To enhance thick description—a dimension of trustworthiness—it will help to present sample demographics in a table that includes information on each participant. Remember that ethical standards of research require that all participant information be deidentified, so use participant identification numbers or pseudonyms for each participant, and do not present any personal information that would allow others to identify the participant (Blignault & Ritchie, 2009). Table 1 provides participant demographics for a hypothetical qualitative research study exploring the perspectives of persons who were formerly homeless regarding their experiences of transitioning into stable housing and obtaining employment.

Participant Demographics

Tables to Illustrate Initial Codes

Most of our dissertation consulting clients who are conducting qualitative research choose a form of thematic analysis . Qualitative analysis to identify themes in the data typically involves a progression from (a) identifying surface-level codes to (b) developing themes by combining codes based on shared similarities. As this process is inherently subjective, it is important that readers be able to evaluate the correspondence between the data and your findings (Anfara et al., 2002). This supports confirmability, another dimension of trustworthiness .

A great way to illustrate the trustworthiness of your qualitative analysis is to create a table that displays quotes from the data that exemplify each of your initial codes. Providing a sample quote for each of your codes can help the reader to assess whether your coding was faithful to the meanings in the data, and it can also help to create clarity about each code’s meaning and bring the voices of your participants into your work (Blignault & Ritchie, 2009).

Table 2 is an example of how you might present information regarding initial codes. Depending on your preference or your dissertation committee’s preference, you might also present percentages of the sample that expressed each code. Another common piece of information to include is which actual participants expressed each code. Note that if your qualitative analysis yields a high volume of codes, it may be appropriate to present the table as an appendix.

Initial Codes

Tables to Present the Groups of Codes That Form Each Theme

As noted previously, most of our dissertation assistance clients use a thematic analysis approach, which involves multiple phases of qualitative analysis  that eventually result in themes that answer the dissertation’s research questions. After initial coding is completed, the analysis process involves (a) examining what different codes have in common and then (b) grouping similar codes together in ways that are meaningful given your research questions. In other words, the common threads that you identify across multiple codes become the theme that holds them all together—and that theme answers one of your research questions.

As with initial coding, grouping codes together into themes involves your own subjective interpretations, even when aided by qualitative analysis software such as NVivo  or MAXQDA. In fact, our dissertation assistance clients are often surprised to learn that qualitative analysis software does not complete the analysis in the same ways that statistical analysis software such as SPSS does. While statistical analysis software completes the computations for you, qualitative analysis software does not have such analysis capabilities. Software such as NVivo provides a set of organizational tools that make the qualitative analysis far more convenient, but the analysis itself is still a very human process (Burnard et al., 2008).

Because of the subjective nature of qualitative analysis, it is important to show the underlying logic behind your thematic analysis in tables—such tables help readers to assess the trustworthiness of your analysis. Table 3 provides an example of how to present the codes that were grouped together to create themes, and you can modify the specifics of the table based on your preferences or your dissertation committee’s requirements. For example, this type of table might be presented to illustrate the codes associated with themes that answer each research question. 

Grouping of Initial Codes to Form Themes

Tables to Illustrate the Themes That Answer Each Research Question

Creating alignment throughout your dissertation is an important objective, and to maintain alignment in your results chapter, the themes you present must clearly answer your research questions. Conducting qualitative analysis is an in-depth process of immersion in the data, and many of our dissertation consulting  clients have shared that it’s easy to lose your direction during the process. So, it is important to stay focused on your research questions during the qualitative analysis and also to show the reader exactly which themes—and subthemes, as applicable—answered each of the research questions.

Below, Table 4 provides an example of how to display the thematic findings of your study in table form. Depending on your dissertation committee’s preference or your own, you might present all research questions and all themes and subthemes in a single table. Or, you might provide separate tables to introduce the themes for each research question as you progress through your presentation of the findings in the chapter.

Emergent Themes and Research Questions

Bonus Tip! Figures to Spice Up Your Results

Although dissertation committees most often wish to see tables such as the above in qualitative results chapters, some also like to see figures that illustrate the data. Qualitative software packages such as NVivo offer many options for visualizing your data, such as mind maps, concept maps, charts, and cluster diagrams. A common choice for this type of figure among our dissertation assistance clients is a tree diagram, which shows the connections between specified words and the words or phrases that participants shared most often in the same context. Another common choice of figure is the word cloud, as depicted in Figure 1. The word cloud simply reflects frequencies of words in the data, which may provide an indication of the importance of related concepts for the participants.

As you move forward with your qualitative analysis and development of your results chapter, we hope that this brief overview of useful tables and figures helps you to decide on an ideal presentation to showcase the trustworthiness your findings. Completing a rigorous qualitative analysis for your dissertation requires many hours of careful interpretation of your data, and your end product should be a rich and detailed results presentation that you can be proud of. Reach out if we can help  in any way, as our dissertation coaches would be thrilled to assist as you move through this exciting stage of your dissertation journey!

Anfara Jr., V. A., Brown, K. M., & Mangione, T. L. (2002). Qualitative analysis on stage: Making the research process more public.  Educational Researcher ,  31 (7), 28-38. https://doi.org/10.3102/0013189X031007028

Blignault, I., & Ritchie, J. (2009). Revealing the wood and the trees: Reporting qualitative research.  Health Promotion Journal of Australia ,  20 (2), 140-145. https://doi.org/10.1071/HE09140

Burnard, P., Gill, P., Stewart, K., Treasure, E., & Chadwick, B. (2008). Analysing and presenting qualitative data.  British Dental Journal ,  204 (8), 429-432. https://doi.org/10.1038/sj.bdj.2008.292

Qualitative Research: Characteristics, Design, Methods & Examples

Lauren McCall

MSc Health Psychology Graduate

MSc, Health Psychology, University of Nottingham

Lauren obtained an MSc in Health Psychology from The University of Nottingham with a distinction classification.

Learn about our Editorial Process

Saul McLeod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul McLeod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

Qualitative research is a type of research methodology that focuses on gathering and analyzing non-numerical data to gain a deeper understanding of human behavior, experiences, and perspectives.

It aims to explore the “why” and “how” of a phenomenon rather than the “what,” “where,” and “when” typically addressed by quantitative research.

Unlike quantitative research, which focuses on gathering and analyzing numerical data for statistical analysis, qualitative research involves researchers interpreting data to identify themes, patterns, and meanings.

Qualitative research can be used to:

• Gain deep contextual understandings of the subjective social reality of individuals
• To answer questions about experience and meaning from the participant’s perspective
• To design hypotheses, theory must be researched using qualitative methods to determine what is important before research can begin. 

Examples of qualitative research questions include: 

• How does stress influence young adults’ behavior?
• What factors influence students’ school attendance rates in developed countries?
• How do adults interpret binge drinking in the UK?
• What are the psychological impacts of cervical cancer screening in women?
• How can mental health lessons be integrated into the school curriculum? 

Characteristics 

Naturalistic setting.

Individuals are studied in their natural setting to gain a deeper understanding of how people experience the world. This enables the researcher to understand a phenomenon close to how participants experience it. 

Naturalistic settings provide valuable contextual information to help researchers better understand and interpret the data they collect.

The environment, social interactions, and cultural factors can all influence behavior and experiences, and these elements are more easily observed in real-world settings.

Reality is socially constructed

Qualitative research aims to understand how participants make meaning of their experiences – individually or in social contexts. It assumes there is no objective reality and that the social world is interpreted (Yilmaz, 2013). 

The primacy of subject matter 

The primary aim of qualitative research is to understand the perspectives, experiences, and beliefs of individuals who have experienced the phenomenon selected for research rather than the average experiences of groups of people (Minichiello, 1990).

An in-depth understanding is attained since qualitative techniques allow participants to freely disclose their experiences, thoughts, and feelings without constraint (Tenny et al., 2022). 

Variables are complex, interwoven, and difficult to measure

Factors such as experiences, behaviors, and attitudes are complex and interwoven, so they cannot be reduced to isolated variables , making them difficult to measure quantitatively.

However, a qualitative approach enables participants to describe what, why, or how they were thinking/ feeling during a phenomenon being studied (Yilmaz, 2013). 

Emic (insider’s point of view)

The phenomenon being studied is centered on the participants’ point of view (Minichiello, 1990).

Emic is used to describe how participants interact, communicate, and behave in the research setting (Scarduzio, 2017).

Interpretive analysis

In qualitative research, interpretive analysis is crucial in making sense of the collected data.

This process involves examining the raw data, such as interview transcripts, field notes, or documents, and identifying the underlying themes, patterns, and meanings that emerge from the participants’ experiences and perspectives.

Collecting Qualitative Data

There are four main research design methods used to collect qualitative data: observations, interviews,  focus groups, and ethnography.

Observations

This method involves watching and recording phenomena as they occur in nature. Observation can be divided into two types: participant and non-participant observation.

In participant observation, the researcher actively participates in the situation/events being observed.

In non-participant observation, the researcher is not an active part of the observation and tries not to influence the behaviors they are observing (Busetto et al., 2020). 

Observations can be covert (participants are unaware that a researcher is observing them) or overt (participants are aware of the researcher’s presence and know they are being observed).

However, awareness of an observer’s presence may influence participants’ behavior. 

Interviews give researchers a window into the world of a participant by seeking their account of an event, situation, or phenomenon. They are usually conducted on a one-to-one basis and can be distinguished according to the level at which they are structured (Punch, 2013). 

Structured interviews involve predetermined questions and sequences to ensure replicability and comparability. However, they are unable to explore emerging issues.

Informal interviews consist of spontaneous, casual conversations which are closer to the truth of a phenomenon. However, information is gathered using quick notes made by the researcher and is therefore subject to recall bias. 

Semi-structured interviews have a flexible structure, phrasing, and placement so emerging issues can be explored (Denny & Weckesser, 2022).

The use of probing questions and clarification can lead to a detailed understanding, but semi-structured interviews can be time-consuming and subject to interviewer bias. 

Focus groups 

Similar to interviews, focus groups elicit a rich and detailed account of an experience. However, focus groups are more dynamic since participants with shared characteristics construct this account together (Denny & Weckesser, 2022).

A shared narrative is built between participants to capture a group experience shaped by a shared context. 

The researcher takes on the role of a moderator, who will establish ground rules and guide the discussion by following a topic guide to focus the group discussions.

Typically, focus groups have 4-10 participants as a discussion can be difficult to facilitate with more than this, and this number allows everyone the time to speak.

Ethnography

Ethnography is a methodology used to study a group of people’s behaviors and social interactions in their environment (Reeves et al., 2008).

Data are collected using methods such as observations, field notes, or structured/ unstructured interviews.

The aim of ethnography is to provide detailed, holistic insights into people’s behavior and perspectives within their natural setting. In order to achieve this, researchers immerse themselves in a community or organization. 

Due to the flexibility and real-world focus of ethnography, researchers are able to gather an in-depth, nuanced understanding of people’s experiences, knowledge and perspectives that are influenced by culture and society.

In order to develop a representative picture of a particular culture/ context, researchers must conduct extensive field work. 

This can be time-consuming as researchers may need to immerse themselves into a community/ culture for a few days, or possibly a few years.

Qualitative Data Analysis Methods

Different methods can be used for analyzing qualitative data. The researcher chooses based on the objectives of their study. 

The researcher plays a key role in the interpretation of data, making decisions about the coding, theming, decontextualizing, and recontextualizing of data (Starks & Trinidad, 2007). 

Grounded theory

Grounded theory is a qualitative method specifically designed to inductively generate theory from data. It was developed by Glaser and Strauss in 1967 (Glaser & Strauss, 2017).

This methodology aims to develop theories (rather than test hypotheses) that explain a social process, action, or interaction (Petty et al., 2012). To inform the developing theory, data collection and analysis run simultaneously. 

There are three key types of coding used in grounded theory: initial (open), intermediate (axial), and advanced (selective) coding. 

Throughout the analysis, memos should be created to document methodological and theoretical ideas about the data. Data should be collected and analyzed until data saturation is reached and a theory is developed. 

Content analysis

Content analysis was first used in the early twentieth century to analyze textual materials such as newspapers and political speeches.

Content analysis is a research method used to identify and analyze the presence and patterns of themes, concepts, or words in data (Vaismoradi et al., 2013). 

This research method can be used to analyze data in different formats, which can be written, oral, or visual. 

The goal of content analysis is to develop themes that capture the underlying meanings of data (Schreier, 2012). 

Qualitative content analysis can be used to validate existing theories, support the development of new models and theories, and provide in-depth descriptions of particular settings or experiences.

The following six steps provide a guideline for how to conduct qualitative content analysis.

• Define a Research Question : To start content analysis, a clear research question should be developed.
• Identify and Collect Data : Establish the inclusion criteria for your data. Find the relevant sources to analyze.
• Define the Unit or Theme of Analysis : Categorize the content into themes. Themes can be a word, phrase, or sentence.
• Develop Rules for Coding your Data : Define a set of coding rules to ensure that all data are coded consistently.
• Code the Data : Follow the coding rules to categorize data into themes.
• Analyze the Results and Draw Conclusions : Examine the data to identify patterns and draw conclusions in relation to your research question.

Discourse analysis

Discourse analysis is a research method used to study written/ spoken language in relation to its social context (Wood & Kroger, 2000).

In discourse analysis, the researcher interprets details of language materials and the context in which it is situated.

Discourse analysis aims to understand the functions of language (how language is used in real life) and how meaning is conveyed by language in different contexts. Researchers use discourse analysis to investigate social groups and how language is used to achieve specific communication goals.

Different methods of discourse analysis can be used depending on the aims and objectives of a study. However, the following steps provide a guideline on how to conduct discourse analysis.

• Define the Research Question : Develop a relevant research question to frame the analysis.
• Gather Data and Establish the Context : Collect research materials (e.g., interview transcripts, documents). Gather factual details and review the literature to construct a theory about the social and historical context of your study.
• Analyze the Content : Closely examine various components of the text, such as the vocabulary, sentences, paragraphs, and structure of the text. Identify patterns relevant to the research question to create codes, then group these into themes.
• Review the Results : Reflect on the findings to examine the function of the language, and the meaning and context of the discourse. 

Thematic analysis

Thematic analysis is a method used to identify, interpret, and report patterns in data, such as commonalities or contrasts. 

Although the origin of thematic analysis can be traced back to the early twentieth century, understanding and clarity of thematic analysis is attributed to Braun and Clarke (2006).

Thematic analysis aims to develop themes (patterns of meaning) across a dataset to address a research question. 

In thematic analysis, qualitative data is gathered using techniques such as interviews, focus groups, and questionnaires. Audio recordings are transcribed. The dataset is then explored and interpreted by a researcher to identify patterns. 

This occurs through the rigorous process of data familiarisation, coding, theme development, and revision. These identified patterns provide a summary of the dataset and can be used to address a research question.

Themes are developed by exploring the implicit and explicit meanings within the data. Two different approaches are used to generate themes: inductive and deductive. 

An inductive approach allows themes to emerge from the data. In contrast, a deductive approach uses existing theories or knowledge to apply preconceived ideas to the data.

Phases of Thematic Analysis

Braun and Clarke (2006) provide a guide of the six phases of thematic analysis. These phases can be applied flexibly to fit research questions and data. 

Template analysis

Template analysis refers to a specific method of thematic analysis which uses hierarchical coding (Brooks et al., 2014).

Template analysis is used to analyze textual data, for example, interview transcripts or open-ended responses on a written questionnaire.

To conduct template analysis, a coding template must be developed (usually from a subset of the data) and subsequently revised and refined. This template represents the themes identified by researchers as important in the dataset. 

Codes are ordered hierarchically within the template, with the highest-level codes demonstrating overarching themes in the data and lower-level codes representing constituent themes with a narrower focus.

A guideline for the main procedural steps for conducting template analysis is outlined below.

• Familiarization with the Data : Read (and reread) the dataset in full. Engage, reflect, and take notes on data that may be relevant to the research question.
• Preliminary Coding : Identify initial codes using guidance from the a priori codes, identified before the analysis as likely to be beneficial and relevant to the analysis.
• Organize Themes : Organize themes into meaningful clusters. Consider the relationships between the themes both within and between clusters.
• Produce an Initial Template : Develop an initial template. This may be based on a subset of the data.
• Apply and Develop the Template : Apply the initial template to further data and make any necessary modifications. Refinements of the template may include adding themes, removing themes, or changing the scope/title of themes. 
• Finalize Template : Finalize the template, then apply it to the entire dataset. 

Frame analysis

Frame analysis is a comparative form of thematic analysis which systematically analyzes data using a matrix output.

Ritchie and Spencer (1994) developed this set of techniques to analyze qualitative data in applied policy research. Frame analysis aims to generate theory from data.

Frame analysis encourages researchers to organize and manage their data using summarization.

This results in a flexible and unique matrix output, in which individual participants (or cases) are represented by rows and themes are represented by columns. 

Each intersecting cell is used to summarize findings relating to the corresponding participant and theme.

Frame analysis has five distinct phases which are interrelated, forming a methodical and rigorous framework.

• Familiarization with the Data : Familiarize yourself with all the transcripts. Immerse yourself in the details of each transcript and start to note recurring themes.
• Develop a Theoretical Framework : Identify recurrent/ important themes and add them to a chart. Provide a framework/ structure for the analysis.
• Indexing : Apply the framework systematically to the entire study data.
• Summarize Data in Analytical Framework : Reduce the data into brief summaries of participants’ accounts.
• Mapping and Interpretation : Compare themes and subthemes and check against the original transcripts. Group the data into categories and provide an explanation for them.

Preventing Bias in Qualitative Research

To evaluate qualitative studies, the CASP (Critical Appraisal Skills Programme) checklist for qualitative studies can be used to ensure all aspects of a study have been considered (CASP, 2018).

The quality of research can be enhanced and assessed using criteria such as checklists, reflexivity, co-coding, and member-checking. 

Co-coding 

Relying on only one researcher to interpret rich and complex data may risk key insights and alternative viewpoints being missed. Therefore, coding is often performed by multiple researchers.

A common strategy must be defined at the beginning of the coding process  (Busetto et al., 2020). This includes establishing a useful coding list and finding a common definition of individual codes.

Transcripts are initially coded independently by researchers and then compared and consolidated to minimize error or bias and to bring confirmation of findings. 

Member checking

Member checking (or respondent validation) involves checking back with participants to see if the research resonates with their experiences (Russell & Gregory, 2003).

Data can be returned to participants after data collection or when results are first available. For example, participants may be provided with their interview transcript and asked to verify whether this is a complete and accurate representation of their views.

Participants may then clarify or elaborate on their responses to ensure they align with their views (Shenton, 2004).

This feedback becomes part of data collection and ensures accurate descriptions/ interpretations of phenomena (Mays & Pope, 2000). 

Reflexivity in qualitative research

Reflexivity typically involves examining your own judgments, practices, and belief systems during data collection and analysis. It aims to identify any personal beliefs which may affect the research. 

Reflexivity is essential in qualitative research to ensure methodological transparency and complete reporting. This enables readers to understand how the interaction between the researcher and participant shapes the data.

Depending on the research question and population being researched, factors that need to be considered include the experience of the researcher, how the contact was established and maintained, age, gender, and ethnicity.

These details are important because, in qualitative research, the researcher is a dynamic part of the research process and actively influences the outcome of the research (Boeije, 2014). 

Reflexivity Example

Who you are and your characteristics influence how you collect and analyze data. Here is an example of a reflexivity statement for research on smoking. I am a 30-year-old white female from a middle-class background. I live in the southwest of England and have been educated to master’s level. I have been involved in two research projects on oral health. I have never smoked, but I have witnessed how smoking can cause ill health from my volunteering in a smoking cessation clinic. My research aspirations are to help to develop interventions to help smokers quit.

Establishing Trustworthiness in Qualitative Research

Trustworthiness is a concept used to assess the quality and rigor of qualitative research. Four criteria are used to assess a study’s trustworthiness: credibility, transferability, dependability, and confirmability.

1. Credibility in Qualitative Research

Credibility refers to how accurately the results represent the reality and viewpoints of the participants.

To establish credibility in research, participants’ views and the researcher’s representation of their views need to align (Tobin & Begley, 2004).

To increase the credibility of findings, researchers may use data source triangulation, investigator triangulation, peer debriefing, or member checking (Lincoln & Guba, 1985). 

2. Transferability in Qualitative Research

Transferability refers to how generalizable the findings are: whether the findings may be applied to another context, setting, or group (Tobin & Begley, 2004).

Transferability can be enhanced by giving thorough and in-depth descriptions of the research setting, sample, and methods (Nowell et al., 2017). 

3. Dependability in Qualitative Research

Dependability is the extent to which the study could be replicated under similar conditions and the findings would be consistent.

Researchers can establish dependability using methods such as audit trails so readers can see the research process is logical and traceable (Koch, 1994).

4. Confirmability in Qualitative Research

Confirmability is concerned with establishing that there is a clear link between the researcher’s interpretations/ findings and the data.

Researchers can achieve confirmability by demonstrating how conclusions and interpretations were arrived at (Nowell et al., 2017).

This enables readers to understand the reasoning behind the decisions made. 

Audit Trails in Qualitative Research

An audit trail provides evidence of the decisions made by the researcher regarding theory, research design, and data collection, as well as the steps they have chosen to manage, analyze, and report data. 

The researcher must provide a clear rationale to demonstrate how conclusions were reached in their study.

A clear description of the research path must be provided to enable readers to trace through the researcher’s logic (Halpren, 1983).

Researchers should maintain records of the raw data, field notes, transcripts, and a reflective journal in order to provide a clear audit trail. 

Discovery of unexpected data

Open-ended questions in qualitative research mean the researcher can probe an interview topic and enable the participant to elaborate on responses in an unrestricted manner.

This allows unexpected data to emerge, which can lead to further research into that topic. 

The exploratory nature of qualitative research helps generate hypotheses that can be tested quantitatively (Busetto et al., 2020).

Flexibility

Data collection and analysis can be modified and adapted to take the research in a different direction if new ideas or patterns emerge in the data.

This enables researchers to investigate new opportunities while firmly maintaining their research goals. 

Naturalistic settings

The behaviors of participants are recorded in real-world settings. Studies that use real-world settings have high ecological validity since participants behave more authentically. 

Limitations

Time-consuming .

Qualitative research results in large amounts of data which often need to be transcribed and analyzed manually.

Even when software is used, transcription can be inaccurate, and using software for analysis can result in many codes which need to be condensed into themes. 

Subjectivity 

The researcher has an integral role in collecting and interpreting qualitative data. Therefore, the conclusions reached are from their perspective and experience.

Consequently, interpretations of data from another researcher may vary greatly. 

Limited generalizability

The aim of qualitative research is to provide a detailed, contextualized understanding of an aspect of the human experience from a relatively small sample size.

Despite rigorous analysis procedures, conclusions drawn cannot be generalized to the wider population since data may be biased or unrepresentative.

Therefore, results are only applicable to a small group of the population. 

While individual qualitative studies are often limited in their generalizability due to factors such as sample size and context, metasynthesis enables researchers to synthesize findings from multiple studies, potentially leading to more generalizable conclusions.

By integrating findings from studies conducted in diverse settings and with different populations, metasynthesis can provide broader insights into the phenomenon of interest.

Extraneous variables

Qualitative research is often conducted in real-world settings. This may cause results to be unreliable since extraneous variables may affect the data, for example:

• Situational variables : different environmental conditions may influence participants’ behavior in a study. The random variation in factors (such as noise or lighting) may be difficult to control in real-world settings.
• Participant characteristics : this includes any characteristics that may influence how a participant answers/ behaves in a study. This may include a participant’s mood, gender, age, ethnicity, sexual identity, IQ, etc.
• Experimenter effect : experimenter effect refers to how a researcher’s unintentional influence can change the outcome of a study. This occurs when (i) their interactions with participants unintentionally change participants’ behaviors or (ii) due to errors in observation, interpretation, or analysis. 

What sample size should qualitative research be?

The sample size for qualitative studies has been recommended to include a minimum of 12 participants to reach data saturation (Braun, 2013).

Are surveys qualitative or quantitative?

Surveys can be used to gather information from a sample qualitatively or quantitatively. Qualitative surveys use open-ended questions to gather detailed information from a large sample using free text responses.

The use of open-ended questions allows for unrestricted responses where participants use their own words, enabling the collection of more in-depth information than closed-ended questions.

In contrast, quantitative surveys consist of closed-ended questions with multiple-choice answer options. Quantitative surveys are ideal to gather a statistical representation of a population.

What are the ethical considerations of qualitative research?

Before conducting a study, you must think about any risks that could occur and take steps to prevent them. Participant Protection : Researchers must protect participants from physical and mental harm. This means you must not embarrass, frighten, offend, or harm participants. Transparency : Researchers are obligated to clearly communicate how they will collect, store, analyze, use, and share the data. Confidentiality : You need to consider how to maintain the confidentiality and anonymity of participants’ data.

What is triangulation in qualitative research?

Triangulation refers to the use of several approaches in a study to comprehensively understand phenomena. This method helps to increase the validity and credibility of research findings. 

Types of triangulation include method triangulation (using multiple methods to gather data); investigator triangulation (multiple researchers for collecting/ analyzing data), theory triangulation (comparing several theoretical perspectives to explain a phenomenon), and data source triangulation (using data from various times, locations, and people; Carter et al., 2014).

Why is qualitative research important?

Qualitative research allows researchers to describe and explain the social world. The exploratory nature of qualitative research helps to generate hypotheses that can then be tested quantitatively.

In qualitative research, participants are able to express their thoughts, experiences, and feelings without constraint.

Additionally, researchers are able to follow up on participants’ answers in real-time, generating valuable discussion around a topic. This enables researchers to gain a nuanced understanding of phenomena which is difficult to attain using quantitative methods.

What is coding data in qualitative research?

Coding data is a qualitative data analysis strategy in which a section of text is assigned with a label that describes its content.

These labels may be words or phrases which represent important (and recurring) patterns in the data.

This process enables researchers to identify related content across the dataset. Codes can then be used to group similar types of data to generate themes.

What is the difference between qualitative and quantitative research?

Qualitative research involves the collection and analysis of non-numerical data in order to understand experiences and meanings from the participant’s perspective.

This can provide rich, in-depth insights on complicated phenomena. Qualitative data may be collected using interviews, focus groups, or observations.

In contrast, quantitative research involves the collection and analysis of numerical data to measure the frequency, magnitude, or relationships of variables. This can provide objective and reliable evidence that can be generalized to the wider population.

Quantitative data may be collected using closed-ended questionnaires or experiments.

What is trustworthiness in qualitative research?

Trustworthiness is a concept used to assess the quality and rigor of qualitative research. Four criteria are used to assess a study’s trustworthiness: credibility, transferability, dependability, and confirmability. 

Credibility refers to how accurately the results represent the reality and viewpoints of the participants. Transferability refers to whether the findings may be applied to another context, setting, or group.

Dependability is the extent to which the findings are consistent and reliable. Confirmability refers to the objectivity of findings (not influenced by the bias or assumptions of researchers).

What is data saturation in qualitative research?

Data saturation is a methodological principle used to guide the sample size of a qualitative research study.

Data saturation is proposed as a necessary methodological component in qualitative research (Saunders et al., 2018) as it is a vital criterion for discontinuing data collection and/or analysis. 

The intention of data saturation is to find “no new data, no new themes, no new coding, and ability to replicate the study” (Guest et al., 2006). Therefore, enough data has been gathered to make conclusions.

Why is sampling in qualitative research important?

In quantitative research, large sample sizes are used to provide statistically significant quantitative estimates.

This is because quantitative research aims to provide generalizable conclusions that represent populations.

However, the aim of sampling in qualitative research is to gather data that will help the researcher understand the depth, complexity, variation, or context of a phenomenon. The small sample sizes in qualitative studies support the depth of case-oriented analysis.

What is narrative analysis?

Narrative analysis is a qualitative research method used to understand how individuals create stories from their personal experiences.

There is an emphasis on understanding the context in which a narrative is constructed, recognizing the influence of historical, cultural, and social factors on storytelling.

Researchers can use different methods together to explore a research question.

Some narrative researchers focus on the content of what is said, using thematic narrative analysis, while others focus on the structure, such as holistic-form or categorical-form structural narrative analysis. Others focus on how the narrative is produced and performed.

Boeije, H. (2014). Analysis in qualitative research. Sage.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative research in psychology , 3 (2), 77-101. https://doi.org/10.1191/1478088706qp063oa

Brooks, J., McCluskey, S., Turley, E., & King, N. (2014). The utility of template analysis in qualitative psychology research. Qualitative Research in Psychology , 12 (2), 202–222. https://doi.org/10.1080/14780887.2014.955224

Busetto, L., Wick, W., & Gumbinger, C. (2020). How to use and assess qualitative research methods. Neurological research and practice , 2 (1), 14-14. https://doi.org/10.1186/s42466-020-00059-z 

Carter, N., Bryant-Lukosius, D., DiCenso, A., Blythe, J., & Neville, A. J. (2014). The use of triangulation in qualitative research. Oncology nursing forum , 41 (5), 545–547. https://doi.org/10.1188/14.ONF.545-547

Critical Appraisal Skills Programme. (2018). CASP Checklist: 10 questions to help you make sense of a Qualitative research. https://casp-uk.net/images/checklist/documents/CASP-Qualitative-Studies-Checklist/CASP-Qualitative-Checklist-2018_fillable_form.pdf Accessed: March 15 2023

Clarke, V., & Braun, V. (2013). Successful qualitative research: A practical guide for beginners. Successful Qualitative Research , 1-400.

Denny, E., & Weckesser, A. (2022). How to do qualitative research?: Qualitative research methods. BJOG : an international journal of obstetrics and gynaecology , 129 (7), 1166-1167. https://doi.org/10.1111/1471-0528.17150 

Glaser, B. G., & Strauss, A. L. (2017). The discovery of grounded theory. The Discovery of Grounded Theory , 1–18. https://doi.org/10.4324/9780203793206-1

Guest, G., Bunce, A., & Johnson, L. (2006). How many interviews are enough? An experiment with data saturation and variability. Field Methods, 18 (1), 59-82. doi:10.1177/1525822X05279903

Halpren, E. S. (1983). Auditing naturalistic inquiries: The development and application of a model (Unpublished doctoral dissertation). Indiana University, Bloomington.

Hammarberg, K., Kirkman, M., & de Lacey, S. (2016). Qualitative research methods: When to use them and how to judge them. Human Reproduction , 31 (3), 498–501. https://doi.org/10.1093/humrep/dev334

Koch, T. (1994). Establishing rigour in qualitative research: The decision trail. Journal of Advanced Nursing, 19, 976–986. doi:10.1111/ j.1365-2648.1994.tb01177.x

Lincoln, Y., & Guba, E. G. (1985). Naturalistic inquiry. Newbury Park, CA: Sage.

Mays, N., & Pope, C. (2000). Assessing quality in qualitative research. BMJ, 320(7226), 50–52.

Minichiello, V. (1990). In-Depth Interviewing: Researching People. Longman Cheshire.

Nowell, L. S., Norris, J. M., White, D. E., & Moules, N. J. (2017). Thematic Analysis: Striving to Meet the Trustworthiness Criteria. International Journal of Qualitative Methods, 16 (1). https://doi.org/10.1177/1609406917733847

Petty, N. J., Thomson, O. P., & Stew, G. (2012). Ready for a paradigm shift? part 2: Introducing qualitative research methodologies and methods. Manual Therapy , 17 (5), 378–384. https://doi.org/10.1016/j.math.2012.03.004

Punch, K. F. (2013). Introduction to social research: Quantitative and qualitative approaches. London: Sage

Reeves, S., Kuper, A., & Hodges, B. D. (2008). Qualitative research methodologies: Ethnography. BMJ , 337 (aug07 3). https://doi.org/10.1136/bmj.a1020

Russell, C. K., & Gregory, D. M. (2003). Evaluation of qualitative research studies. Evidence Based Nursing, 6 (2), 36–40.

Saunders, B., Sim, J., Kingstone, T., Baker, S., Waterfield, J., Bartlam, B., Burroughs, H., & Jinks, C. (2018). Saturation in qualitative research: exploring its conceptualization and operationalization. Quality & quantity , 52 (4), 1893–1907. https://doi.org/10.1007/s11135-017-0574-8

Scarduzio, J. A. (2017). Emic approach to qualitative research. The International Encyclopedia of Communication Research Methods, 1–2 . https://doi.org/10.1002/9781118901731.iecrm0082

Schreier, M. (2012). Qualitative content analysis in practice / Margrit Schreier.

Shenton, A. K. (2004). Strategies for ensuring trustworthiness in qualitative research projects. Education for Information, 22 , 63–75.

Starks, H., & Trinidad, S. B. (2007). Choose your method: a comparison of phenomenology, discourse analysis, and grounded theory. Qualitative health research , 17 (10), 1372–1380. https://doi.org/10.1177/1049732307307031

Tenny, S., Brannan, J. M., & Brannan, G. D. (2022). Qualitative Study. In StatPearls. StatPearls Publishing.

Tobin, G. A., & Begley, C. M. (2004). Methodological rigour within a qualitative framework. Journal of Advanced Nursing, 48, 388–396. doi:10.1111/j.1365-2648.2004.03207.x

Vaismoradi, M., Turunen, H., & Bondas, T. (2013). Content analysis and thematic analysis: Implications for conducting a qualitative descriptive study. Nursing & health sciences , 15 (3), 398-405. https://doi.org/10.1111/nhs.12048

Wood L. A., Kroger R. O. (2000). Doing discourse analysis: Methods for studying action in talk and text. Sage.

Yilmaz, K. (2013). Comparison of Quantitative and Qualitative Research Traditions: epistemological, theoretical, and methodological differences. European journal of education , 48 (2), 311-325. https://doi.org/10.1111/ejed.12014

No products in the cart.

Preparing the presentation of qualitative findings: considering your roles and goals

Dr. Philip Adu is a Methodology Expert at The Chicago School of Professional Psychology (TCSPP). In this post he explains the things to consider when presenting your research findings.

This post follows on from his previous blog post “Perfecting the art of qualitative coding” in which he took us through the stages of qualitative coding and, along the way, outlined the features he found most useful.

In my previous blog post, I presented on making good use of the innovative features of NVivo across the three main stages of qualitative analysis. Expounding on the third stage which is the ‘ Post-Coding stage (Presenting your findings) ’, I want to throw light on things to consider when drafting and refining your presentation. The moment you reach a milestone of successfully using NVivo 12 (Version 12.1.249; QSR International Pty Ltd, 2018) to complete the data analysis process, the reality of preparing all of this data so you can present your findings sets in (Adu, 2016). Your methodical review of the qualitative data and development of codes, categories and themes has yielded massive and interesting NVivo outputs. The outcomes include but are not limited to; codes/nodes, categories/themes, Word Clouds, Word Tree, Framework Matrices, Cluster Tree, code-case matrices, and code-attribute matrices (see Figure 1). These findings need to be carefully examined – selecting the ones that will be useful in drafting a meaningful presentation. You can watch the presentation I developed below:

Source: https://www.youtube.com/watch?v=xEyGGFtVQFw

Note, not all of this information (i.e. the outcomes) needs to be presented to your audience (see Adu, 2019 ). Other questions that may arise as you develop your presentation include; what kind of results should you present? How do you engage with your audience when presenting your findings? How would you help your audience to understand and believe your findings?

In this post, I will discuss the three pertinent components a good presentation of qualitative findings should have. They are; background information, data analysis process and main findings.

Figure 1. Presentation of findings

Presenting background information

Participants’ past and current situations influence the information they provide to you. Due to this, there is the need to provide readers a summary of who participants are and any background information which may help them to put the findings into the proper context. Also, as a researcher analyzing qualitative data, there is the likelihood of your own background impacting the data analysis process. In the same way, you need to let readers know who you are, what your background is and how you ‘bracketed’ them from not having an effect on the findings ( Adu, 2019 ).

Presenting the data analysis process

Qualitative analysis doesn’t only involve engaging in subjective development of codes and categories, but also promoting transparency in the coding and categorization process (Greckhamer & Cilesiz, 2014). Due to this, you are expected to describe the main and detailed steps you took to analyze your data to arrive at your findings and their respective outcomes. Addressing the following questions would be great:

• What coding strategy did you use?
• What kinds of codes did you assign to relevant excerpts of the data?
• What are the examples of codes you generated?
• What categorization technique did you use?
• How did you develop categories/themes out of the codes?

Your audience’s aim is not only consuming what you found but also learning more about how you came up with the results.

Presenting main findings

When it comes to the presentation of findings, there are two main structures you could choose from. You could present them based on the themes generated or based on the cases (participants or groups of participants) you have. The decision to either structure depends on the kind of research question(s) or the research purpose you have. For a detailed explanation of the types of presentation formats and how to select an appropriate structure, see Chapter 13 of the book, “ A Step-by-Step Guide to Qualitative Data Coding ”.

Considering your roles and goals

As you plan on how to communicate the above components, make sure you accomplish your goals and carry out your role as a communicator of qualitative data analysis outcomes (See Figure 1). Your roles are; to thoughtfully arrange the data analysis outcomes and to adequately address your research questions.

Liken the presentation of your findings to sharing a puzzle which has been solved. Your goal is to prevent a situation where the burden is put on the audience to piece together the puzzle of findings. In other words, you are expected to present the findings in a meaningful way that would enhance the audience’s understanding of the data analysis outcomes (Adu, 2016 & 2019). By so doing, they are more likely to trust what you found.

Let’s summarize the action items:

• Out of a pool of qualitative analysis outcomes, select the ones that would allow you to address your research questions and meaningfully communicate your findings.
• Decide on how you want to structure the presentation of the findings.
• Irrespective of the presentation format you choose, make sure you include background information, the data analysis process and main findings in your presentation.
• Make sure you are ‘narrating’ participants’ stories or what you found – making the numeric outputs include the tables and charts generated play a supporting role when presenting the main findings.

Adu, P. (2016). Presenting Qualitative Findings Using NVivo Output to Tell the Story. [PowerPoint slides]. SlideShare. Retrieved from https://www.slideshare.net/kontorphilip/presenting-qualitative-findings-using-nvivo-output-to-tell-the-story

QSR International Pty Ltd. (2018). NVivo 12. Version 12.1.249 [Computer software]. Retrieved from https://qsrinternational.com/nvivo-qualitative-data-analysis-software

Adu, P. (2019). A Step-by-Step Guide to Qualitative Data Coding . Oxford: Routledge

Greckhamer, T., & Cilesiz, S. (2014). Rigor, Transparency, Evidence, and Representation in Discourse Analysis: Challenges and Recommendations. International Journal of Qualitative Methods, 13(1), 422-443. doi:10.1177/160940691401300123

ABOUT THE AUTHOR

Dr. Philip Adu is a Methodology Expert at The Chicago School of Professional Psychology (TCSPP). His role is to provide support to dissertating students in TCSPP addressing their methodology related concerns. You could access some of his webinars at the ‘Methodology Related Presentations – TCSPP’ YouTube Channel. He completed his Doctoral degree in Education with a concentration in Learning, Instructional Design and Technology from West Virginia University (WVU). Dr. Adu recently authored a book titled, “A Step-by-Step Guide to Qualitative Data Coding” (available on routledge.com or amazon.com ). You could reach Dr. Adu at [email protected] and @drphilipadu on twitter.

Recent Articles

Jump to navigation

Cochrane Training

Chapter 14: completing ‘summary of findings’ tables and grading the certainty of the evidence.

Holger J Schünemann, Julian PT Higgins, Gunn E Vist, Paul Glasziou, Elie A Akl, Nicole Skoetz, Gordon H Guyatt; on behalf of the Cochrane GRADEing Methods Group (formerly Applicability and Recommendations Methods Group) and the Cochrane Statistical Methods Group

Key Points:

• A ‘Summary of findings’ table for a given comparison of interventions provides key information concerning the magnitudes of relative and absolute effects of the interventions examined, the amount of available evidence and the certainty (or quality) of available evidence.
• ‘Summary of findings’ tables include a row for each important outcome (up to a maximum of seven). Accepted formats of ‘Summary of findings’ tables and interactive ‘Summary of findings’ tables can be produced using GRADE’s software GRADEpro GDT.
• Cochrane has adopted the GRADE approach (Grading of Recommendations Assessment, Development and Evaluation) for assessing certainty (or quality) of a body of evidence.
• The GRADE approach specifies four levels of the certainty for a body of evidence for a given outcome: high, moderate, low and very low.
• GRADE assessments of certainty are determined through consideration of five domains: risk of bias, inconsistency, indirectness, imprecision and publication bias. For evidence from non-randomized studies and rarely randomized studies, assessments can then be upgraded through consideration of three further domains.

Cite this chapter as: Schünemann HJ, Higgins JPT, Vist GE, Glasziou P, Akl EA, Skoetz N, Guyatt GH. Chapter 14: Completing ‘Summary of findings’ tables and grading the certainty of the evidence. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.4 (updated August 2023). Cochrane, 2023. Available from www.training.cochrane.org/handbook .

14.1 ‘Summary of findings’ tables

14.1.1 introduction to ‘summary of findings’ tables.

‘Summary of findings’ tables present the main findings of a review in a transparent, structured and simple tabular format. In particular, they provide key information concerning the certainty or quality of evidence (i.e. the confidence or certainty in the range of an effect estimate or an association), the magnitude of effect of the interventions examined, and the sum of available data on the main outcomes. Cochrane Reviews should incorporate ‘Summary of findings’ tables during planning and publication, and should have at least one key ‘Summary of findings’ table representing the most important comparisons. Some reviews may include more than one ‘Summary of findings’ table, for example if the review addresses more than one major comparison, or includes substantially different populations that require separate tables (e.g. because the effects differ or it is important to show results separately). In the Cochrane Database of Systematic Reviews (CDSR),  all ‘Summary of findings’ tables for a review appear at the beginning, before the Background section.

14.1.2 Selecting outcomes for ‘Summary of findings’ tables

Planning for the ‘Summary of findings’ table starts early in the systematic review, with the selection of the outcomes to be included in: (i) the review; and (ii) the ‘Summary of findings’ table. This is a crucial step, and one that review authors need to address carefully.

To ensure production of optimally useful information, Cochrane Reviews begin by developing a review question and by listing all main outcomes that are important to patients and other decision makers (see Chapter 2 and Chapter 3 ). The GRADE approach to assessing the certainty of the evidence (see Section 14.2 ) defines and operationalizes a rating process that helps separate outcomes into those that are critical, important or not important for decision making. Consultation and feedback on the review protocol, including from consumers and other decision makers, can enhance this process.

Critical outcomes are likely to include clearly important endpoints; typical examples include mortality and major morbidity (such as strokes and myocardial infarction). However, they may also represent frequent minor and rare major side effects, symptoms, quality of life, burdens associated with treatment, and resource issues (costs). Burdens represent the impact of healthcare workload on patient function and well-being, and include the demands of adhering to an intervention that patients or caregivers (e.g. family) may dislike, such as having to undergo more frequent tests, or the restrictions on lifestyle that certain interventions require (Spencer-Bonilla et al 2017).

Frequently, when formulating questions that include all patient-important outcomes for decision making, review authors will confront reports of studies that have not included all these outcomes. This is particularly true for adverse outcomes. For instance, randomized trials might contribute evidence on intended effects, and on frequent, relatively minor side effects, but not report on rare adverse outcomes such as suicide attempts. Chapter 19 discusses strategies for addressing adverse effects. To obtain data for all important outcomes it may be necessary to examine the results of non-randomized studies (see Chapter 24 ). Cochrane, in collaboration with others, has developed guidance for review authors to support their decision about when to look for and include non-randomized studies (Schünemann et al 2013).

If a review includes only randomized trials, these trials may not address all important outcomes and it may therefore not be possible to address these outcomes within the constraints of the review. Review authors should acknowledge these limitations and make them transparent to readers. Review authors are encouraged to include non-randomized studies to examine rare or long-term adverse effects that may not adequately be studied in randomized trials. This raises the possibility that harm outcomes may come from studies in which participants differ from those in studies used in the analysis of benefit. Review authors will then need to consider how much such differences are likely to impact on the findings, and this will influence the certainty of evidence because of concerns about indirectness related to the population (see Section 14.2.2 ).

Non-randomized studies can provide important information not only when randomized trials do not report on an outcome or randomized trials suffer from indirectness, but also when the evidence from randomized trials is rated as very low and non-randomized studies provide evidence of higher certainty. Further discussion of these issues appears also in Chapter 24 .

14.1.3 General template for ‘Summary of findings’ tables

Several alternative standard versions of ‘Summary of findings’ tables have been developed to ensure consistency and ease of use across reviews, inclusion of the most important information needed by decision makers, and optimal presentation (see examples at Figures 14.1.a and 14.1.b ). These formats are supported by research that focused on improved understanding of the information they intend to convey (Carrasco-Labra et al 2016, Langendam et al 2016, Santesso et al 2016). They are available through GRADE’s official software package developed to support the GRADE approach: GRADEpro GDT (www.gradepro.org).

Standard Cochrane ‘Summary of findings’ tables include the following elements using one of the accepted formats. Further guidance on each of these is provided in Section 14.1.6 .

• A brief description of the population and setting addressed by the available evidence (which may be slightly different to or narrower than those defined by the review question).
• A brief description of the comparison addressed in the ‘Summary of findings’ table, including both the experimental and comparison interventions.
• A list of the most critical and/or important health outcomes, both desirable and undesirable, limited to seven or fewer outcomes.
• A measure of the typical burden of each outcomes (e.g. illustrative risk, or illustrative mean, on comparator intervention).
• The absolute and relative magnitude of effect measured for each (if both are appropriate).
• The numbers of participants and studies contributing to the analysis of each outcomes.
• A GRADE assessment of the overall certainty of the body of evidence for each outcome (which may vary by outcome).
• Space for comments.
• Explanations (formerly known as footnotes).

Ideally, ‘Summary of findings’ tables are supported by more detailed tables (known as ‘evidence profiles’) to which the review may be linked, which provide more detailed explanations. Evidence profiles include the same important health outcomes, and provide greater detail than ‘Summary of findings’ tables of both of the individual considerations feeding into the grading of certainty and of the results of the studies (Guyatt et al 2011a). They ensure that a structured approach is used to rating the certainty of evidence. Although they are rarely published in Cochrane Reviews, evidence profiles are often used, for example, by guideline developers in considering the certainty of the evidence to support guideline recommendations. Review authors will find it easier to develop the ‘Summary of findings’ table by completing the rating of the certainty of evidence in the evidence profile first in GRADEpro GDT. They can then automatically convert this to one of the ‘Summary of findings’ formats in GRADEpro GDT, including an interactive ‘Summary of findings’ for publication.

As a measure of the magnitude of effect for dichotomous outcomes, the ‘Summary of findings’ table should provide a relative measure of effect (e.g. risk ratio, odds ratio, hazard) and measures of absolute risk. For other types of data, an absolute measure alone (such as a difference in means for continuous data) might be sufficient. It is important that the magnitude of effect is presented in a meaningful way, which may require some transformation of the result of a meta-analysis (see also Chapter 15, Section 15.4 and Section 15.5 ). Reviews with more than one main comparison should include a separate ‘Summary of findings’ table for each comparison.

Figure 14.1.a provides an example of a ‘Summary of findings’ table. Figure 15.1.b  provides an alternative format that may further facilitate users’ understanding and interpretation of the review’s findings. Evidence evaluating different formats suggests that the ‘Summary of findings’ table should include a risk difference as a measure of the absolute effect and authors should preferably use a format that includes a risk difference .

A detailed description of the contents of a ‘Summary of findings’ table appears in Section 14.1.6 .

Figure 14.1.a Example of a ‘Summary of findings’ table

Summary of findings (for interactive version click here )

a All the stockings in the nine studies included in this review were below-knee compression stockings. In four studies the compression strength was 20 mmHg to 30 mmHg at the ankle. It was 10 mmHg to 20 mmHg in the other four studies. Stockings come in different sizes. If a stocking is too tight around the knee it can prevent essential venous return causing the blood to pool around the knee. Compression stockings should be fitted properly. A stocking that is too tight could cut into the skin on a long flight and potentially cause ulceration and increased risk of DVT. Some stockings can be slightly thicker than normal leg covering and can be potentially restrictive with tight foot wear. It is a good idea to wear stockings around the house prior to travel to ensure a good, comfortable fit. Participants put their stockings on two to three hours before the flight in most of the studies. The availability and cost of stockings can vary.

b Two studies recruited high risk participants defined as those with previous episodes of DVT, coagulation disorders, severe obesity, limited mobility due to bone or joint problems, neoplastic disease within the previous two years, large varicose veins or, in one of the studies, participants taller than 190 cm and heavier than 90 kg. The incidence for the seven studies that excluded high risk participants was 1.45% and the incidence for the two studies that recruited high-risk participants (with at least one risk factor) was 2.43%. We have used 10 and 30 per 1000 to express different risk strata, respectively.

c The confidence interval crosses no difference and does not rule out a small increase.

d The measurement of oedema was not validated (indirectness of the outcome) or blinded to the intervention (risk of bias).

e If there are very few or no events and the number of participants is large, judgement about the certainty of evidence (particularly judgements about imprecision) may be based on the absolute effect. Here the certainty rating may be considered ‘high’ if the outcome was appropriately assessed and the event, in fact, did not occur in 2821 studied participants.

f None of the other studies reported adverse effects, apart from four cases of superficial vein thrombosis in varicose veins in the knee region that were compressed by the upper edge of the stocking in one study.

Figure 14.1.b Example of alternative ‘Summary of findings’ table

14.1.4 Producing ‘Summary of findings’ tables

The GRADE Working Group’s software, GRADEpro GDT ( www.gradepro.org ), including GRADE’s interactive handbook, is available to assist review authors in the preparation of ‘Summary of findings’ tables. GRADEpro can use data on the comparator group risk and the effect estimate (entered by the review authors or imported from files generated in RevMan) to produce the relative effects and absolute risks associated with experimental interventions. In addition, it leads the user through the process of a GRADE assessment, and produces a table that can be used as a standalone table in a review (including by direct import into software such as RevMan or integration with RevMan Web), or an interactive ‘Summary of findings’ table (see help resources in GRADEpro).

14.1.5 Statistical considerations in ‘Summary of findings’ tables

14.1.5.1 dichotomous outcomes.

‘Summary of findings’ tables should include both absolute and relative measures of effect for dichotomous outcomes. Risk ratios, odds ratios and risk differences are different ways of comparing two groups with dichotomous outcome data (see Chapter 6, Section 6.4.1 ). Furthermore, there are two distinct risk ratios, depending on which event (e.g. ‘yes’ or ‘no’) is the focus of the analysis (see Chapter 6, Section 6.4.1.5 ). In the presence of a non-zero intervention effect, any variation across studies in the comparator group risks (i.e. variation in the risk of the event occurring without the intervention of interest, for example in different populations) makes it impossible for more than one of these measures to be truly the same in every study.

It has long been assumed in epidemiology that relative measures of effect are more consistent than absolute measures of effect from one scenario to another. There is empirical evidence to support this assumption (Engels et al 2000, Deeks and Altman 2001, Furukawa et al 2002). For this reason, meta-analyses should generally use either a risk ratio or an odds ratio as a measure of effect (see Chapter 10, Section 10.4.3 ). Correspondingly, a single estimate of relative effect is likely to be a more appropriate summary than a single estimate of absolute effect. If a relative effect is indeed consistent across studies, then different comparator group risks will have different implications for absolute benefit. For instance, if the risk ratio is consistently 0.75, then the experimental intervention would reduce a comparator group risk of 80% to 60% in the intervention group (an absolute risk reduction of 20 percentage points), but would also reduce a comparator group risk of 20% to 15% in the intervention group (an absolute risk reduction of 5 percentage points).

‘Summary of findings’ tables are built around the assumption of a consistent relative effect. It is therefore important to consider the implications of this effect for different comparator group risks (these can be derived or estimated from a number of sources, see Section 14.1.6.3 ), which may require an assessment of the certainty of evidence for prognostic evidence (Spencer et al 2012, Iorio et al 2015). For any comparator group risk, it is possible to estimate a corresponding intervention group risk (i.e. the absolute risk with the intervention) from the meta-analytic risk ratio or odds ratio. Note that the numbers provided in the ‘Corresponding risk’ column are specific to the ‘risks’ in the adjacent column.

For the meta-analytic risk ratio (RR) and assumed comparator risk (ACR) the corresponding intervention risk is obtained as:

As an example, in Figure 14.1.a , the meta-analytic risk ratio for symptomless deep vein thrombosis (DVT) is RR = 0.10 (95% CI 0.04 to 0.26). Assuming a comparator risk of ACR = 10 per 1000 = 0.01, we obtain:

For the meta-analytic odds ratio (OR) and assumed comparator risk, ACR, the corresponding intervention risk is obtained as:

Upper and lower confidence limits for the corresponding intervention risk are obtained by replacing RR or OR by their upper and lower confidence limits, respectively (e.g. replacing 0.10 with 0.04, then with 0.26, in the example). Such confidence intervals do not incorporate uncertainty in the assumed comparator risks.

When dealing with risk ratios, it is critical that the same definition of ‘event’ is used as was used for the meta-analysis. For example, if the meta-analysis focused on ‘death’ (as opposed to survival) as the event, then corresponding risks in the ‘Summary of findings’ table must also refer to ‘death’.

In (rare) circumstances in which there is clear rationale to assume a consistent risk difference in the meta-analysis, in principle it is possible to present this for relevant ‘assumed risks’ and their corresponding risks, and to present the corresponding (different) relative effects for each assumed risk.

The risk difference expresses the difference between the ACR and the corresponding intervention risk (or the difference between the experimental and the comparator intervention).

For the meta-analytic risk ratio (RR) and assumed comparator risk (ACR) the corresponding risk difference is obtained as (note that risks can also be expressed using percentage or percentage points):

As an example, in Figure 14.1.b the meta-analytic risk ratio is 0.41 (95% CI 0.29 to 0.55) for diarrhoea in children less than 5 years of age. Assuming a comparator group risk of 22.3% we obtain:

For the meta-analytic odds ratio (OR) and assumed comparator risk (ACR) the absolute risk difference is obtained as (percentage points):

Upper and lower confidence limits for the absolute risk difference are obtained by re-running the calculation above while replacing RR or OR by their upper and lower confidence limits, respectively (e.g. replacing 0.41 with 0.28, then with 0.55, in the example). Such confidence intervals do not incorporate uncertainty in the assumed comparator risks.

14.1.5.2 Time-to-event outcomes

Time-to-event outcomes measure whether and when a particular event (e.g. death) occurs (van Dalen et al 2007). The impact of the experimental intervention relative to the comparison group on time-to-event outcomes is usually measured using a hazard ratio (HR) (see Chapter 6, Section 6.8.1 ).

A hazard ratio expresses a relative effect estimate. It may be used in various ways to obtain absolute risks and other interpretable quantities for a specific population. Here we describe how to re-express hazard ratios in terms of: (i) absolute risk of event-free survival within a particular period of time; (ii) absolute risk of an event within a particular period of time; and (iii) median time to the event. All methods are built on an assumption of consistent relative effects (i.e. that the hazard ratio does not vary over time).

(i) Absolute risk of event-free survival within a particular period of time Event-free survival (e.g. overall survival) is commonly reported by individual studies. To obtain absolute effects for time-to-event outcomes measured as event-free survival, the summary HR can be used in conjunction with an assumed proportion of patients who are event-free in the comparator group (Tierney et al 2007). This proportion of patients will be specific to a period of time of observation. However, it is not strictly necessary to specify this period of time. For instance, a proportion of 50% of event-free patients might apply to patients with a high event rate observed over 1 year, or to patients with a low event rate observed over 2 years.

As an example, suppose the meta-analytic hazard ratio is 0.42 (95% CI 0.25 to 0.72). Assuming a comparator group risk of event-free survival (e.g. for overall survival people being alive) at 2 years of ACR = 900 per 1000 = 0.9 we obtain:

so that that 956 per 1000 people will be alive with the experimental intervention at 2 years. The derivation of the risk should be explained in a comment or footnote.

(ii) Absolute risk of an event within a particular period of time To obtain this absolute effect, again the summary HR can be used (Tierney et al 2007):

In the example, suppose we assume a comparator group risk of events (e.g. for mortality, people being dead) at 2 years of ACR = 100 per 1000 = 0.1. We obtain:

so that that 44 per 1000 people will be dead with the experimental intervention at 2 years.

(iii) Median time to the event Instead of absolute numbers, the time to the event in the intervention and comparison groups can be expressed as median survival time in months or years. To obtain median survival time the pooled HR can be applied to an assumed median survival time in the comparator group (Tierney et al 2007):

In the example, assuming a comparator group median survival time of 80 months, we obtain:

For all three of these options for re-expressing results of time-to-event analyses, upper and lower confidence limits for the corresponding intervention risk are obtained by replacing HR by its upper and lower confidence limits, respectively (e.g. replacing 0.42 with 0.25, then with 0.72, in the example). Again, as for dichotomous outcomes, such confidence intervals do not incorporate uncertainty in the assumed comparator group risks. This is of special concern for long-term survival with a low or moderate mortality rate and a corresponding high number of censored patients (i.e. a low number of patients under risk and a high censoring rate).

14.1.6 Detailed contents of a ‘Summary of findings’ table

14.1.6.1 table title and header.

The title of each ‘Summary of findings’ table should specify the healthcare question, framed in terms of the population and making it clear exactly what comparison of interventions are made. In Figure 14.1.a , the population is people taking long aeroplane flights, the intervention is compression stockings, and the control is no compression stockings.

The first rows of each ‘Summary of findings’ table should provide the following ‘header’ information:

Patients or population This further clarifies the population (and possibly the subpopulations) of interest and ideally the magnitude of risk of the most crucial adverse outcome at which an intervention is directed. For instance, people on a long-haul flight may be at different risks for DVT; those using selective serotonin reuptake inhibitors (SSRIs) might be at different risk for side effects; while those with atrial fibrillation may be at low (< 1%), moderate (1% to 4%) or high (> 4%) yearly risk of stroke.

Setting This should state any specific characteristics of the settings of the healthcare question that might limit the applicability of the summary of findings to other settings (e.g. primary care in Europe and North America).

Intervention The experimental intervention.

Comparison The comparator intervention (including no specific intervention).

14.1.6.2 Outcomes

The rows of a ‘Summary of findings’ table should include all desirable and undesirable health outcomes (listed in order of importance) that are essential for decision making, up to a maximum of seven outcomes. If there are more outcomes in the review, review authors will need to omit the less important outcomes from the table, and the decision selecting which outcomes are critical or important to the review should be made during protocol development (see Chapter 3 ). Review authors should provide time frames for the measurement of the outcomes (e.g. 90 days or 12 months) and the type of instrument scores (e.g. ranging from 0 to 100).

Note that review authors should include the pre-specified critical and important outcomes in the table whether data are available or not. However, they should be alert to the possibility that the importance of an outcome (e.g. a serious adverse effect) may only become known after the protocol was written or the analysis was carried out, and should take appropriate actions to include these in the ‘Summary of findings’ table.

The ‘Summary of findings’ table can include effects in subgroups of the population for different comparator risks and effect sizes separately. For instance, in Figure 14.1.b effects are presented for children younger and older than 5 years separately. Review authors may also opt to produce separate ‘Summary of findings’ tables for different populations.

Review authors should include serious adverse events, but it might be possible to combine minor adverse events as a single outcome, and describe this in an explanatory footnote (note that it is not appropriate to add events together unless they are independent, that is, a participant who has experienced one adverse event has an unaffected chance of experiencing the other adverse event).

Outcomes measured at multiple time points represent a particular problem. In general, to keep the table simple, review authors should present multiple time points only for outcomes critical to decision making, where either the result or the decision made are likely to vary over time. The remainder should be presented at a common time point where possible.

Review authors can present continuous outcome measures in the ‘Summary of findings’ table and should endeavour to make these interpretable to the target audience. This requires that the units are clear and readily interpretable, for example, days of pain, or frequency of headache, and the name and scale of any measurement tools used should be stated (e.g. a Visual Analogue Scale, ranging from 0 to 100). However, many measurement instruments are not readily interpretable by non-specialist clinicians or patients, for example, points on a Beck Depression Inventory or quality of life score. For these, a more interpretable presentation might involve converting a continuous to a dichotomous outcome, such as >50% improvement (see Chapter 15, Section 15.5 ).

14.1.6.3 Best estimate of risk with comparator intervention

Review authors should provide up to three typical risks for participants receiving the comparator intervention. For dichotomous outcomes, we recommend that these be presented in the form of the number of people experiencing the event per 100 or 1000 people (natural frequency) depending on the frequency of the outcome. For continuous outcomes, this would be stated as a mean or median value of the outcome measured.

Estimated or assumed comparator intervention risks could be based on assessments of typical risks in different patient groups derived from the review itself, individual representative studies in the review, or risks derived from a systematic review of prognosis studies or other sources of evidence which may in turn require an assessment of the certainty for the prognostic evidence (Spencer et al 2012, Iorio et al 2015). Ideally, risks would reflect groups that clinicians can easily identify on the basis of their presenting features.

An explanatory footnote should specify the source or rationale for each comparator group risk, including the time period to which it corresponds where appropriate. In Figure 14.1.a , clinicians can easily differentiate individuals with risk factors for deep venous thrombosis from those without. If there is known to be little variation in baseline risk then review authors may use the median comparator group risk across studies. If typical risks are not known, an option is to choose the risk from the included studies, providing the second highest for a high and the second lowest for a low risk population.

14.1.6.4 Risk with intervention

For dichotomous outcomes, review authors should provide a corresponding absolute risk for each comparator group risk, along with a confidence interval. This absolute risk with the (experimental) intervention will usually be derived from the meta-analysis result presented in the relative effect column (see Section 14.1.6.6 ). Formulae are provided in Section 14.1.5 . Review authors should present the absolute effect in the same format as the risks with comparator intervention (see Section 14.1.6.3 ), for example as the number of people experiencing the event per 1000 people.

For continuous outcomes, a difference in means or standardized difference in means should be presented with its confidence interval. These will typically be obtained directly from a meta-analysis. Explanatory text should be used to clarify the meaning, as in Figures 14.1.a and 14.1.b .

14.1.6.5 Risk difference

For dichotomous outcomes, the risk difference can be provided using one of the ‘Summary of findings’ table formats as an additional option (see Figure 14.1.b ). This risk difference expresses the difference between the experimental and comparator intervention and will usually be derived from the meta-analysis result presented in the relative effect column (see Section 14.1.6.6 ). Formulae are provided in Section 14.1.5 . Review authors should present the risk difference in the same format as assumed and corresponding risks with comparator intervention (see Section 14.1.6.3 ); for example, as the number of people experiencing the event per 1000 people or as percentage points if the assumed and corresponding risks are expressed in percentage.

For continuous outcomes, if the ‘Summary of findings’ table includes this option, the mean difference can be presented here and the ‘corresponding risk’ column left blank (see Figure 14.1.b ).

14.1.6.6 Relative effect (95% CI)

The relative effect will typically be a risk ratio or odds ratio (or occasionally a hazard ratio) with its accompanying 95% confidence interval, obtained from a meta-analysis performed on the basis of the same effect measure. Risk ratios and odds ratios are similar when the comparator intervention risks are low and effects are small, but may differ considerably when comparator group risks increase. The meta-analysis may involve an assumption of either fixed or random effects, depending on what the review authors consider appropriate, and implying that the relative effect is either an estimate of the effect of the intervention, or an estimate of the average effect of the intervention across studies, respectively.

14.1.6.7 Number of participants (studies)

This column should include the number of participants assessed in the included studies for each outcome and the corresponding number of studies that contributed these participants.

14.1.6.8 Certainty of the evidence (GRADE)

Review authors should comment on the certainty of the evidence (also known as quality of the body of evidence or confidence in the effect estimates). Review authors should use the specific evidence grading system developed by the GRADE Working Group (Atkins et al 2004, Guyatt et al 2008, Guyatt et al 2011a), which is described in detail in Section 14.2 . The GRADE approach categorizes the certainty in a body of evidence as ‘high’, ‘moderate’, ‘low’ or ‘very low’ by outcome. This is a result of judgement, but the judgement process operates within a transparent structure. As an example, the certainty would be ‘high’ if the summary were of several randomized trials with low risk of bias, but the rating of certainty becomes lower if there are concerns about risk of bias, inconsistency, indirectness, imprecision or publication bias. Judgements other than of ‘high’ certainty should be made transparent using explanatory footnotes or the ‘Comments’ column in the ‘Summary of findings’ table (see Section 14.1.6.10 ).

14.1.6.9 Comments

The aim of the ‘Comments’ field is to help interpret the information or data identified in the row. For example, this may be on the validity of the outcome measure or the presence of variables that are associated with the magnitude of effect. Important caveats about the results should be flagged here. Not all rows will need comments, and it is best to leave a blank if there is nothing warranting a comment.

14.1.6.10 Explanations

Detailed explanations should be included as footnotes to support the judgements in the ‘Summary of findings’ table, such as the overall GRADE assessment. The explanations should describe the rationale for important aspects of the content. Table 14.1.a lists guidance for useful explanations. Explanations should be concise, informative, relevant, easy to understand and accurate. If explanations cannot be sufficiently described in footnotes, review authors should provide further details of the issues in the Results and Discussion sections of the review.

Table 14.1.a Guidance for providing useful explanations in ‘Summary of findings’ (SoF) tables. Adapted from Santesso et al (2016)

14.2 Assessing the certainty or quality of a body of evidence

14.2.1 the grade approach.

The Grades of Recommendation, Assessment, Development and Evaluation Working Group (GRADE Working Group) has developed a system for grading the certainty of evidence (Schünemann et al 2003, Atkins et al 2004, Schünemann et al 2006, Guyatt et al 2008, Guyatt et al 2011a). Over 100 organizations including the World Health Organization (WHO), the American College of Physicians, the American Society of Hematology (ASH), the Canadian Agency for Drugs and Technology in Health (CADTH) and the National Institutes of Health and Clinical Excellence (NICE) in the UK have adopted the GRADE system ( www.gradeworkinggroup.org ).

Cochrane has also formally adopted this approach, and all Cochrane Reviews should use GRADE to evaluate the certainty of evidence for important outcomes (see MECIR Box 14.2.a ).

MECIR Box 14.2.a Relevant expectations for conduct of intervention reviews

For systematic reviews, the GRADE approach defines the certainty of a body of evidence as the extent to which one can be confident that an estimate of effect or association is close to the quantity of specific interest. Assessing the certainty of a body of evidence involves consideration of within- and across-study risk of bias (limitations in study design and execution or methodological quality), inconsistency (or heterogeneity), indirectness of evidence, imprecision of the effect estimates and risk of publication bias (see Section 14.2.2 ), as well as domains that may increase our confidence in the effect estimate (as described in Section 14.2.3 ). The GRADE system entails an assessment of the certainty of a body of evidence for each individual outcome. Judgements about the domains that determine the certainty of evidence should be described in the results or discussion section and as part of the ‘Summary of findings’ table.

The GRADE approach specifies four levels of certainty ( Figure 14.2.a ). For interventions, including diagnostic and other tests that are evaluated as interventions (Schünemann et al 2008b, Schünemann et al 2008a, Balshem et al 2011, Schünemann et al 2012), the starting point for rating the certainty of evidence is categorized into two types:

• randomized trials; and
• non-randomized studies of interventions (NRSI), including observational studies (including but not limited to cohort studies, and case-control studies, cross-sectional studies, case series and case reports, although not all of these designs are usually included in Cochrane Reviews).

There are many instances in which review authors rely on information from NRSI, in particular to evaluate potential harms (see Chapter 24 ). In addition, review authors can obtain relevant data from both randomized trials and NRSI, with each type of evidence complementing the other (Schünemann et al 2013).

In GRADE, a body of evidence from randomized trials begins with a high-certainty rating while a body of evidence from NRSI begins with a low-certainty rating. The lower rating with NRSI is the result of the potential bias induced by the lack of randomization (i.e. confounding and selection bias).

However, when using the new Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) tool (Sterne et al 2016), an assessment tool that covers the risk of bias due to lack of randomization, all studies may start as high certainty of the evidence (Schünemann et al 2018). The approach of starting all study designs (including NRSI) as high certainty does not conflict with the initial GRADE approach of starting the rating of NRSI as low certainty evidence. This is because a body of evidence from NRSI should generally be downgraded by two levels due to the inherent risk of bias associated with the lack of randomization, namely confounding and selection bias. Not downgrading NRSI from high to low certainty needs transparent and detailed justification for what mitigates concerns about confounding and selection bias (Schünemann et al 2018). Very few examples of where not rating down by two levels is appropriate currently exist.

The highest certainty rating is a body of evidence when there are no concerns in any of the GRADE factors listed in Figure 14.2.a . Review authors often downgrade evidence to moderate, low or even very low certainty evidence, depending on the presence of the five factors in Figure 14.2.a . Usually, certainty rating will fall by one level for each factor, up to a maximum of three levels for all factors. If there are very severe problems for any one domain (e.g. when assessing risk of bias, all studies were unconcealed, unblinded and lost over 50% of their patients to follow-up), evidence may fall by two levels due to that factor alone. It is not possible to rate lower than ‘very low certainty’ evidence.

Review authors will generally grade evidence from sound non-randomized studies as low certainty, even if ROBINS-I is used. If, however, such studies yield large effects and there is no obvious bias explaining those effects, review authors may rate the evidence as moderate or – if the effect is large enough – even as high certainty ( Figure 14.2.a ). The very low certainty level is appropriate for, but is not limited to, studies with critical problems and unsystematic clinical observations (e.g. case series or case reports).

Figure 14.2.a Levels of the certainty of a body of evidence in the GRADE approach. *Upgrading criteria are usually applicable to non-randomized studies only (but exceptions exist).

14.2.2 Domains that can lead to decreasing the certainty level of a body of evidence   

We now describe in more detail the five reasons (or domains) for downgrading the certainty of a body of evidence for a specific outcome. In each case, if no reason is found for downgrading the evidence, it should be classified as 'no limitation or not serious' (not important enough to warrant downgrading). If a reason is found for downgrading the evidence, it should be classified as 'serious' (downgrading the certainty rating by one level) or 'very serious' (downgrading the certainty grade by two levels). For non-randomized studies assessed with ROBINS-I, rating down by three levels should be classified as 'extremely' serious.

(1) Risk of bias or limitations in the detailed design and implementation

Our confidence in an estimate of effect decreases if studies suffer from major limitations that are likely to result in a biased assessment of the intervention effect. For randomized trials, these methodological limitations include failure to generate a random sequence, lack of allocation sequence concealment, lack of blinding (particularly with subjective outcomes that are highly susceptible to biased assessment), a large loss to follow-up or selective reporting of outcomes. Chapter 8 provides a discussion of study-level assessments of risk of bias in the context of a Cochrane Review, and proposes an approach to assessing the risk of bias for an outcome across studies as ‘Low’ risk of bias, ‘Some concerns’ and ‘High’ risk of bias for randomized trials. Levels of ‘Low’. ‘Moderate’, ‘Serious’ and ‘Critical’ risk of bias arise for non-randomized studies assessed with ROBINS-I ( Chapter 25 ). These assessments should feed directly into this GRADE domain. In particular, ‘Low’ risk of bias would indicate ‘no limitation’; ‘Some concerns’ would indicate either ‘no limitation’ or ‘serious limitation’; and ‘High’ risk of bias would indicate either ‘serious limitation’ or ‘very serious limitation’. ‘Critical’ risk of bias on ROBINS-I would indicate extremely serious limitations in GRADE. Review authors should use their judgement to decide between alternative categories, depending on the likely magnitude of the potential biases.

Every study addressing a particular outcome will differ, to some degree, in the risk of bias. Review authors should make an overall judgement on whether the certainty of evidence for an outcome warrants downgrading on the basis of study limitations. The assessment of study limitations should apply to the studies contributing to the results in the ‘Summary of findings’ table, rather than to all studies that could potentially be included in the analysis. We have argued in Chapter 7, Section 7.6.2 , that the primary analysis should be restricted to studies at low (or low and unclear) risk of bias where possible.

Table 14.2.a presents the judgements that must be made in going from assessments of the risk of bias to judgements about study limitations for each outcome included in a ‘Summary of findings’ table. A rating of high certainty evidence can be achieved only when most evidence comes from studies that met the criteria for low risk of bias. For example, of the 22 studies addressing the impact of beta-blockers on mortality in patients with heart failure, most probably or certainly used concealed allocation of the sequence, all blinded at least some key groups and follow-up of randomized patients was almost complete (Brophy et al 2001). The certainty of evidence might be downgraded by one level when most of the evidence comes from individual studies either with a crucial limitation for one item, or with some limitations for multiple items. An example of very serious limitations, warranting downgrading by two levels, is provided by evidence on surgery versus conservative treatment in the management of patients with lumbar disc prolapse (Gibson and Waddell 2007). We are uncertain of the benefit of surgery in reducing symptoms after one year or longer, because the one study included in the analysis had inadequate concealment of the allocation sequence and the outcome was assessed using a crude rating by the surgeon without blinding.

(2) Unexplained heterogeneity or inconsistency of results

When studies yield widely differing estimates of effect (heterogeneity or variability in results), investigators should look for robust explanations for that heterogeneity. For instance, drugs may have larger relative effects in sicker populations or when given in larger doses. A detailed discussion of heterogeneity and its investigation is provided in Chapter 10, Section 10.10 and Section 10.11 . If an important modifier exists, with good evidence that important outcomes are different in different subgroups (which would ideally be pre-specified), then a separate ‘Summary of findings’ table may be considered for a separate population. For instance, a separate ‘Summary of findings’ table would be used for carotid endarterectomy in symptomatic patients with high grade stenosis (70% to 99%) in which the intervention is, in the hands of the right surgeons, beneficial, and another (if review authors considered it relevant) for asymptomatic patients with low grade stenosis (less than 30%) in which surgery appears harmful (Orrapin and Rerkasem 2017). When heterogeneity exists and affects the interpretation of results, but review authors are unable to identify a plausible explanation with the data available, the certainty of the evidence decreases.

(3) Indirectness of evidence

Two types of indirectness are relevant. First, a review comparing the effectiveness of alternative interventions (say A and B) may find that randomized trials are available, but they have compared A with placebo and B with placebo. Thus, the evidence is restricted to indirect comparisons between A and B. Where indirect comparisons are undertaken within a network meta-analysis context, GRADE for network meta-analysis should be used (see Chapter 11, Section 11.5 ).

Second, a review may find randomized trials that meet eligibility criteria but address a restricted version of the main review question in terms of population, intervention, comparator or outcomes. For example, suppose that in a review addressing an intervention for secondary prevention of coronary heart disease, most identified studies happened to be in people who also had diabetes. Then the evidence may be regarded as indirect in relation to the broader question of interest because the population is primarily related to people with diabetes. The opposite scenario can equally apply: a review addressing the effect of a preventive strategy for coronary heart disease in people with diabetes may consider studies in people without diabetes to provide relevant, albeit indirect, evidence. This would be particularly likely if investigators had conducted few if any randomized trials in the target population (e.g. people with diabetes). Other sources of indirectness may arise from interventions studied (e.g. if in all included studies a technical intervention was implemented by expert, highly trained specialists in specialist centres, then evidence on the effects of the intervention outside these centres may be indirect), comparators used (e.g. if the comparator groups received an intervention that is less effective than standard treatment in most settings) and outcomes assessed (e.g. indirectness due to surrogate outcomes when data on patient-important outcomes are not available, or when investigators seek data on quality of life but only symptoms are reported). Review authors should make judgements transparent when they believe downgrading is justified, based on differences in anticipated effects in the group of primary interest. Review authors may be aided and increase transparency of their judgements about indirectness if they use Table 14.2.b available in the GRADEpro GDT software (Schünemann et al 2013).

(4) Imprecision of results

When studies include few participants or few events, and thus have wide confidence intervals, review authors can lower their rating of the certainty of the evidence. The confidence intervals included in the ‘Summary of findings’ table will provide readers with information that allows them to make, to some extent, their own rating of precision. Review authors can use a calculation of the optimal information size (OIS) or review information size (RIS), similar to sample size calculations, to make judgements about imprecision (Guyatt et al 2011b, Schünemann 2016). The OIS or RIS is calculated on the basis of the number of participants required for an adequately powered individual study. If the 95% confidence interval excludes a risk ratio (RR) of 1.0, and the total number of events or patients exceeds the OIS criterion, precision is adequate. If the 95% CI includes appreciable benefit or harm (an RR of under 0.75 or over 1.25 is often suggested as a very rough guide) downgrading for imprecision may be appropriate even if OIS criteria are met (Guyatt et al 2011b, Schünemann 2016).

(5) High probability of publication bias

The certainty of evidence level may be downgraded if investigators fail to report studies on the basis of results (typically those that show no effect: publication bias) or outcomes (typically those that may be harmful or for which no effect was observed: selective outcome non-reporting bias). Selective reporting of outcomes from among multiple outcomes measured is assessed at the study level as part of the assessment of risk of bias (see Chapter 8, Section 8.7 ), so for the studies contributing to the outcome in the ‘Summary of findings’ table this is addressed by domain 1 above (limitations in the design and implementation). If a large number of studies included in the review do not contribute to an outcome, or if there is evidence of publication bias, the certainty of the evidence may be downgraded. Chapter 13 provides a detailed discussion of reporting biases, including publication bias, and how it may be tackled in a Cochrane Review. A prototypical situation that may elicit suspicion of publication bias is when published evidence includes a number of small studies, all of which are industry-funded (Bhandari et al 2004). For example, 14 studies of flavanoids in patients with haemorrhoids have shown apparent large benefits, but enrolled a total of only 1432 patients (i.e. each study enrolled relatively few patients) (Alonso-Coello et al 2006). The heavy involvement of sponsors in most of these studies raises questions of whether unpublished studies that suggest no benefit exist (publication bias).

A particular body of evidence can suffer from problems associated with more than one of the five factors listed here, and the greater the problems, the lower the certainty of evidence rating that should result. One could imagine a situation in which randomized trials were available, but all or virtually all of these limitations would be present, and in serious form. A very low certainty of evidence rating would result.

Table 14.2.a Further guidelines for domain 1 (of 5) in a GRADE assessment: going from assessments of risk of bias in studies to judgements about study limitations for main outcomes across studies

Table 14.2.b Judgements about indirectness by outcome (available in GRADEpro GDT)

Intervention:

Comparator:

Direct comparison:

Final judgement about indirectness across domains:

14.2.3 Domains that may lead to increasing the certainty level of a body of evidence

Although NRSI and downgraded randomized trials will generally yield a low rating for certainty of evidence, there will be unusual circumstances in which review authors could ‘upgrade’ such evidence to moderate or even high certainty ( Table 14.3.a ).

• Large effects On rare occasions when methodologically well-done observational studies yield large, consistent and precise estimates of the magnitude of an intervention effect, one may be particularly confident in the results. A large estimated effect (e.g. RR >2 or RR <0.5) in the absence of plausible confounders, or a very large effect (e.g. RR >5 or RR <0.2) in studies with no major threats to validity, might qualify for this. In these situations, while the NRSI may possibly have provided an over-estimate of the true effect, the weak study design may not explain all of the apparent observed benefit. Thus, despite reservations based on the observational study design, review authors are confident that the effect exists. The magnitude of the effect in these studies may move the assigned certainty of evidence from low to moderate (if the effect is large in the absence of other methodological limitations). For example, a meta-analysis of observational studies showed that bicycle helmets reduce the risk of head injuries in cyclists by a large margin (odds ratio (OR) 0.31, 95% CI 0.26 to 0.37) (Thompson et al 2000). This large effect, in the absence of obvious bias that could create the association, suggests a rating of moderate-certainty evidence.  Note : GRADE guidance suggests the possibility of rating up one level for a large effect if the relative effect is greater than 2.0. However, if the point estimate of the relative effect is greater than 2.0, but the confidence interval is appreciably below 2.0, then some hesitation would be appropriate in the decision to rate up for a large effect. Another situation allows inference of a strong association without a formal comparative study. Consider the question of the impact of routine colonoscopy versus no screening for colon cancer on the rate of perforation associated with colonoscopy. Here, a large series of representative patients undergoing colonoscopy may provide high certainty evidence about the risk of perforation associated with colonoscopy. When the risk of the event among patients receiving the relevant comparator is known to be near 0 (i.e. we are certain that the incidence of spontaneous colon perforation in patients not undergoing colonoscopy is extremely low), case series or cohort studies of representative patients can provide high certainty evidence of adverse effects associated with an intervention, thereby allowing us to infer a strong association from even a limited number of events.
• Dose-response The presence of a dose-response gradient may increase our confidence in the findings of observational studies and thereby enhance the assigned certainty of evidence. For example, our confidence in the result of observational studies that show an increased risk of bleeding in patients who have supratherapeutic anticoagulation levels is increased by the observation that there is a dose-response gradient between the length of time needed for blood to clot (as measured by the international normalized ratio (INR)) and an increased risk of bleeding (Levine et al 2004). A systematic review of NRSI investigating the effect of cyclooxygenase-2 inhibitors on cardiovascular events found that the summary estimate (RR) with rofecoxib was 1.33 (95% CI 1.00 to 1.79) with doses less than 25mg/d, and 2.19 (95% CI 1.64 to 2.91) with doses more than 25mg/d. Although residual confounding is likely to exist in the NRSI that address this issue, the existence of a dose-response gradient and the large apparent effect of higher doses of rofecoxib markedly increase our strength of inference that the association cannot be explained by residual confounding, and is therefore likely to be both causal and, at high levels of exposure, substantial.  Note : GRADE guidance suggests the possibility of rating up one level for a large effect if the relative effect is greater than 2.0. Here, the fact that the point estimate of the relative effect is greater than 2.0, but the confidence interval is appreciably below 2.0 might make some hesitate in the decision to rate up for a large effect
• Plausible confounding On occasion, all plausible biases from randomized or non-randomized studies may be working to under-estimate an apparent intervention effect. For example, if only sicker patients receive an experimental intervention or exposure, yet they still fare better, it is likely that the actual intervention or exposure effect is larger than the data suggest. For instance, a rigorous systematic review of observational studies including a total of 38 million patients demonstrated higher death rates in private for-profit versus private not-for-profit hospitals (Devereaux et al 2002). One possible bias relates to different disease severity in patients in the two hospital types. It is likely, however, that patients in the not-for-profit hospitals were sicker than those in the for-profit hospitals. Thus, to the extent that residual confounding existed, it would bias results against the not-for-profit hospitals. The second likely bias was the possibility that higher numbers of patients with excellent private insurance coverage could lead to a hospital having more resources and a spill-over effect that would benefit those without such coverage. Since for-profit hospitals are likely to admit a larger proportion of such well-insured patients than not-for-profit hospitals, the bias is once again against the not-for-profit hospitals. Since the plausible biases would all diminish the demonstrated intervention effect, one might consider the evidence from these observational studies as moderate rather than low certainty. A parallel situation exists when observational studies have failed to demonstrate an association, but all plausible biases would have increased an intervention effect. This situation will usually arise in the exploration of apparent harmful effects. For example, because the hypoglycaemic drug phenformin causes lactic acidosis, the related agent metformin was under suspicion for the same toxicity. Nevertheless, very large observational studies have failed to demonstrate an association (Salpeter et al 2007). Given the likelihood that clinicians would be more alert to lactic acidosis in the presence of the agent and over-report its occurrence, one might consider this moderate, or even high certainty, evidence refuting a causal relationship between typical therapeutic doses of metformin and lactic acidosis.

14.3 Describing the assessment of the certainty of a body of evidence using the GRADE framework

Review authors should report the grading of the certainty of evidence in the Results section for each outcome for which this has been performed, providing the rationale for downgrading or upgrading the evidence, and referring to the ‘Summary of findings’ table where applicable.

Table 14.3.a provides a framework and examples for how review authors can justify their judgements about the certainty of evidence in each domain. These justifications should also be included in explanatory notes to the ‘Summary of Findings’ table (see Section 14.1.6.10 ).

Chapter 15, Section 15.6 , describes in more detail how the overall GRADE assessment across all domains can be used to draw conclusions about the effects of the intervention, as well as providing implications for future research.

Table 14.3.a Framework for describing the certainty of evidence and justifying downgrading or upgrading

14.4 Chapter information

Authors: Holger J Schünemann, Julian PT Higgins, Gunn E Vist, Paul Glasziou, Elie A Akl, Nicole Skoetz, Gordon H Guyatt; on behalf of the Cochrane GRADEing Methods Group (formerly Applicability and Recommendations Methods Group) and the Cochrane Statistical Methods Group

Acknowledgements: Andrew D Oxman contributed to earlier versions. Professor Penny Hawe contributed to the text on adverse effects in earlier versions. Jon Deeks provided helpful contributions on an earlier version of this chapter. For details of previous authors and editors of the Handbook , please refer to the Preface.

Funding: This work was in part supported by funding from the Michael G DeGroote Cochrane Canada Centre and the Ontario Ministry of Health.

14.5 References

Alonso-Coello P, Zhou Q, Martinez-Zapata MJ, Mills E, Heels-Ansdell D, Johanson JF, Guyatt G. Meta-analysis of flavonoids for the treatment of haemorrhoids. British Journal of Surgery 2006; 93 : 909-920.

Atkins D, Best D, Briss PA, Eccles M, Falck-Ytter Y, Flottorp S, Guyatt GH, Harbour RT, Haugh MC, Henry D, Hill S, Jaeschke R, Leng G, Liberati A, Magrini N, Mason J, Middleton P, Mrukowicz J, O'Connell D, Oxman AD, Phillips B, Schünemann HJ, Edejer TT, Varonen H, Vist GE, Williams JW, Jr., Zaza S. Grading quality of evidence and strength of recommendations. BMJ 2004; 328 : 1490.

Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz R, Brozek J, Vist GE, Falck-Ytter Y, Meerpohl J, Norris S, Guyatt GH. GRADE guidelines: 3. Rating the quality of evidence. Journal of Clinical Epidemiology 2011; 64 : 401-406.

Bhandari M, Busse JW, Jackowski D, Montori VM, Schünemann H, Sprague S, Mears D, Schemitsch EH, Heels-Ansdell D, Devereaux PJ. Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials. Canadian Medical Association Journal 2004; 170 : 477-480.

Brophy JM, Joseph L, Rouleau JL. Beta-blockers in congestive heart failure. A Bayesian meta-analysis. Annals of Internal Medicine 2001; 134 : 550-560.

Carrasco-Labra A, Brignardello-Petersen R, Santesso N, Neumann I, Mustafa RA, Mbuagbaw L, Etxeandia Ikobaltzeta I, De Stio C, McCullagh LJ, Alonso-Coello P, Meerpohl JJ, Vandvik PO, Brozek JL, Akl EA, Bossuyt P, Churchill R, Glenton C, Rosenbaum S, Tugwell P, Welch V, Garner P, Guyatt G, Schünemann HJ. Improving GRADE evidence tables part 1: a randomized trial shows improved understanding of content in summary of findings tables with a new format. Journal of Clinical Epidemiology 2016; 74 : 7-18.

Deeks JJ, Altman DG. Effect measures for meta-analysis of trials with binary outcomes. In: Egger M, Davey Smith G, Altman DG, editors. Systematic Reviews in Health Care: Meta-analysis in Context . 2nd ed. London (UK): BMJ Publication Group; 2001. p. 313-335.

Devereaux PJ, Choi PT, Lacchetti C, Weaver B, Schünemann HJ, Haines T, Lavis JN, Grant BJ, Haslam DR, Bhandari M, Sullivan T, Cook DJ, Walter SD, Meade M, Khan H, Bhatnagar N, Guyatt GH. A systematic review and meta-analysis of studies comparing mortality rates of private for-profit and private not-for-profit hospitals. Canadian Medical Association Journal 2002; 166 : 1399-1406.

Engels EA, Schmid CH, Terrin N, Olkin I, Lau J. Heterogeneity and statistical significance in meta-analysis: an empirical study of 125 meta-analyses. Statistics in Medicine 2000; 19 : 1707-1728.

Furukawa TA, Guyatt GH, Griffith LE. Can we individualize the 'number needed to treat'? An empirical study of summary effect measures in meta-analyses. International Journal of Epidemiology 2002; 31 : 72-76.

Gibson JN, Waddell G. Surgical interventions for lumbar disc prolapse: updated Cochrane Review. Spine 2007; 32 : 1735-1747.

Guyatt G, Oxman A, Vist G, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schünemann H. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008; 336 : 3.

Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, Norris S, Falck-Ytter Y, Glasziou P, DeBeer H, Jaeschke R, Rind D, Meerpohl J, Dahm P, Schünemann HJ. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. Journal of Clinical Epidemiology 2011a; 64 : 383-394.

Guyatt GH, Oxman AD, Kunz R, Brozek J, Alonso-Coello P, Rind D, Devereaux PJ, Montori VM, Freyschuss B, Vist G, Jaeschke R, Williams JW, Jr., Murad MH, Sinclair D, Falck-Ytter Y, Meerpohl J, Whittington C, Thorlund K, Andrews J, Schünemann HJ. GRADE guidelines 6. Rating the quality of evidence--imprecision. Journal of Clinical Epidemiology 2011b; 64 : 1283-1293.

Iorio A, Spencer FA, Falavigna M, Alba C, Lang E, Burnand B, McGinn T, Hayden J, Williams K, Shea B, Wolff R, Kujpers T, Perel P, Vandvik PO, Glasziou P, Schünemann H, Guyatt G. Use of GRADE for assessment of evidence about prognosis: rating confidence in estimates of event rates in broad categories of patients. BMJ 2015; 350 : h870.

Langendam M, Carrasco-Labra A, Santesso N, Mustafa RA, Brignardello-Petersen R, Ventresca M, Heus P, Lasserson T, Moustgaard R, Brozek J, Schünemann HJ. Improving GRADE evidence tables part 2: a systematic survey of explanatory notes shows more guidance is needed. Journal of Clinical Epidemiology 2016; 74 : 19-27.

Levine MN, Raskob G, Landefeld S, Kearon C, Schulman S. Hemorrhagic complications of anticoagulant treatment: the Seventh ACCP Conference on Antithrombotic and Thrombolytic Therapy. Chest 2004; 126 : 287S-310S.

Orrapin S, Rerkasem K. Carotid endarterectomy for symptomatic carotid stenosis. Cochrane Database of Systematic Reviews 2017; 6 : CD001081.

Salpeter S, Greyber E, Pasternak G, Salpeter E. Risk of fatal and nonfatal lactic acidosis with metformin use in type 2 diabetes mellitus. Cochrane Database of Systematic Reviews 2007; 4 : CD002967.

Santesso N, Carrasco-Labra A, Langendam M, Brignardello-Petersen R, Mustafa RA, Heus P, Lasserson T, Opiyo N, Kunnamo I, Sinclair D, Garner P, Treweek S, Tovey D, Akl EA, Tugwell P, Brozek JL, Guyatt G, Schünemann HJ. Improving GRADE evidence tables part 3: detailed guidance for explanatory footnotes supports creating and understanding GRADE certainty in the evidence judgments. Journal of Clinical Epidemiology 2016; 74 : 28-39.

Schünemann HJ, Best D, Vist G, Oxman AD, Group GW. Letters, numbers, symbols and words: how to communicate grades of evidence and recommendations. Canadian Medical Association Journal 2003; 169 : 677-680.

Schünemann HJ, Jaeschke R, Cook DJ, Bria WF, El-Solh AA, Ernst A, Fahy BF, Gould MK, Horan KL, Krishnan JA, Manthous CA, Maurer JR, McNicholas WT, Oxman AD, Rubenfeld G, Turino GM, Guyatt G. An official ATS statement: grading the quality of evidence and strength of recommendations in ATS guidelines and recommendations. American Journal of Respiratory and Critical Care Medicine 2006; 174 : 605-614.

Schünemann HJ, Oxman AD, Brozek J, Glasziou P, Jaeschke R, Vist GE, Williams JW, Jr., Kunz R, Craig J, Montori VM, Bossuyt P, Guyatt GH. Grading quality of evidence and strength of recommendations for diagnostic tests and strategies. BMJ 2008a; 336 : 1106-1110.

Schünemann HJ, Oxman AD, Brozek J, Glasziou P, Bossuyt P, Chang S, Muti P, Jaeschke R, Guyatt GH. GRADE: assessing the quality of evidence for diagnostic recommendations. ACP Journal Club 2008b; 149 : 2.

Schünemann HJ, Mustafa R, Brozek J. [Diagnostic accuracy and linked evidence--testing the chain]. Zeitschrift für Evidenz, Fortbildung und Qualität im Gesundheitswesen 2012; 106 : 153-160.

Schünemann HJ, Tugwell P, Reeves BC, Akl EA, Santesso N, Spencer FA, Shea B, Wells G, Helfand M. Non-randomized studies as a source of complementary, sequential or replacement evidence for randomized controlled trials in systematic reviews on the effects of interventions. Research Synthesis Methods 2013; 4 : 49-62.

Schünemann HJ. Interpreting GRADE's levels of certainty or quality of the evidence: GRADE for statisticians, considering review information size or less emphasis on imprecision? Journal of Clinical Epidemiology 2016; 75 : 6-15.

Schünemann HJ, Cuello C, Akl EA, Mustafa RA, Meerpohl JJ, Thayer K, Morgan RL, Gartlehner G, Kunz R, Katikireddi SV, Sterne J, Higgins JPT, Guyatt G, Group GW. GRADE guidelines: 18. How ROBINS-I and other tools to assess risk of bias in nonrandomized studies should be used to rate the certainty of a body of evidence. Journal of Clinical Epidemiology 2018.

Spencer-Bonilla G, Quinones AR, Montori VM, International Minimally Disruptive Medicine W. Assessing the Burden of Treatment. Journal of General Internal Medicine 2017; 32 : 1141-1145.

Spencer FA, Iorio A, You J, Murad MH, Schünemann HJ, Vandvik PO, Crowther MA, Pottie K, Lang ES, Meerpohl JJ, Falck-Ytter Y, Alonso-Coello P, Guyatt GH. Uncertainties in baseline risk estimates and confidence in treatment effects. BMJ 2012; 345 : e7401.

Sterne JAC, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, Henry D, Altman DG, Ansari MT, Boutron I, Carpenter JR, Chan AW, Churchill R, Deeks JJ, Hróbjartsson A, Kirkham J, Jüni P, Loke YK, Pigott TD, Ramsay CR, Regidor D, Rothstein HR, Sandhu L, Santaguida PL, Schünemann HJ, Shea B, Shrier I, Tugwell P, Turner L, Valentine JC, Waddington H, Waters E, Wells GA, Whiting PF, Higgins JPT. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ 2016; 355 : i4919.

Thompson DC, Rivara FP, Thompson R. Helmets for preventing head and facial injuries in bicyclists. Cochrane Database of Systematic Reviews 2000; 2 : CD001855.

Tierney JF, Stewart LA, Ghersi D, Burdett S, Sydes MR. Practical methods for incorporating summary time-to-event data into meta-analysis. Trials 2007; 8 .

van Dalen EC, Tierney JF, Kremer LCM. Tips and tricks for understanding and using SR results. No. 7: time‐to‐event data. Evidence-Based Child Health 2007; 2 : 1089-1090.

For permission to re-use material from the Handbook (either academic or commercial), please see here for full details.

Presenting Findings from Qualitative Research: One Size Does Not Fit All!

• Research in the Sociology of Organizations

• University of Alberta

• Brock University

• Copenhagen Business School

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations

• Belinda Dentaa Oduro
• Christopher Neil Makanga

• John C. Kigozi Munene
• J CLEAN PROD

• Nicolás Pedemonte‐Rojas
• Diego Gálvez

• J SERV MANAGE
• Aku Valtakoski

• Anita Garvey
• Ewan MacKenzie

• J MANAGE STUD

• J MANAGE INQUIRY

• Kathleen M. Eisenhardt

• Daisy E. Chung
• SOC STUD SCI

• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

Home > Blog >

Data analysis in qualitative research, theertha raj, august 30, 2024.

While numbers tell us "what" and "how much," qualitative data reveals the crucial "why" and "how." But let's face it - turning mountains of text, images, and observations into meaningful insights can be daunting.

This guide dives deep into the art and science of how to analyze qualitative data. We'll explore cutting-edge techniques, free qualitative data analysis software, and strategies to make your analysis more rigorous and insightful. Expect practical, actionable advice on qualitative data analysis methods, whether you're a seasoned researcher looking to refine your skills or a team leader aiming to extract more value from your qualitative data.

What is qualitative data?

Qualitative data is non-numerical information that describes qualities or characteristics. It includes text, images, audio, and video. 

This data type captures complex human experiences, behaviors, and opinions that numbers alone can't express.

A qualitative data example can include interview transcripts, open-ended survey responses, field notes from observations, social media posts and customer reviews

Importance of qualitative data

Qualitative data is vital for several reasons:

• It provides a deep, nuanced understanding of complex phenomena.
• It captures the 'why' behind behaviors and opinions.
• It allows for unexpected discoveries and new research directions.
• It puts people's experiences and perspectives at the forefront.
• It enhances quantitative findings with depth and detail.

What is data analysis in qualitative research?

Data analysis in qualitative research is the process of examining and interpreting non-numerical data to uncover patterns, themes, and insights. It aims to make sense of rich, detailed information gathered through methods like interviews, focus groups, or observations.

This analysis moves beyond simple description. It seeks to understand the underlying meanings, contexts, and relationships within the data. The goal is to create a coherent narrative that answers research questions and generates new knowledge.

How is qualitative data analysis different from quantitative data analysis?

Qualitative and quantitative data analyses differ in several key ways:

• Data type: Qualitative analysis uses non-numerical data (text, images), while quantitative analysis uses numerical data.
• Approach: Qualitative analysis is inductive and exploratory. Quantitative analysis is deductive and confirmatory.
• Sample size: Qualitative studies often use smaller samples. Quantitative studies typically need larger samples for statistical validity.
• Depth vs. breadth: Qualitative analysis provides in-depth insights about a few cases. Quantitative analysis offers broader insights across many cases.
• Subjectivity: Qualitative analysis involves more subjective interpretation. Quantitative analysis aims for objective, statistical measures.

What are the 3 main components of qualitative data analysis?

The three main components of qualitative data analysis are:

• Data reduction: Simplifying and focusing the raw data through coding and categorization.
• Data display: Organizing the reduced data into visual formats like matrices, charts, or networks.
• Conclusion drawing/verification: Interpreting the displayed data and verifying the conclusions.

These components aren't linear steps. Instead, they form an iterative process where researchers move back and forth between them throughout the analysis.

How do you write a qualitative analysis?

Step 1: organize your data.

Start with bringing all your qualitative research data in one place. A repository can be of immense help here. Transcribe interviews , compile field notes, and gather all relevant materials.

Immerse yourself in the data. Read through everything multiple times.

Step 2: Code & identify themes

Identify and label key concepts, themes, or patterns. Group related codes into broader themes or categories. Try to connect themes to tell a coherent story that answers your research questions.

Pick out direct quotes from your data to illustrate key points.

Step 3: Interpret and reflect

Explain what your results mean in the context of your research and existing literature.

Als discuss, identify and try to eliminate potential biases or limitations in your analysis. 

Summarize main insights and their implications.

What are the 5 qualitative data analysis methods?

Thematic Analysis Identifying, analyzing, and reporting patterns (themes) within data.

Content Analysis Systematically categorizing and counting the occurrence of specific elements in text.

Grounded Theory Developing theory from data through iterative coding and analysis.

Discourse Analysis Examining language use and meaning in social contexts.

Narrative Analysis Interpreting stories and personal accounts to understand experiences and meanings.

Each method suits different research goals and data types. Researchers often combine methods for comprehensive analysis.

What are the 4 data collection methods in qualitative research?

When it comes to collecting qualitative data, researchers primarily rely on four methods.

• Interviews : One-on-one conversations to gather in-depth information.
• Focus Groups : Group discussions to explore collective opinions and experiences.
• Observations : Watching and recording behaviors in natural settings.
• Document Analysis : Examining existing texts, images, or artifacts.

Researchers often use multiple methods to gain a comprehensive understanding of their topic.

How is qualitative data analysis measured?

Unlike quantitative data, qualitative data analysis isn't measured in traditional numerical terms. Instead, its quality is evaluated based on several criteria. 

Trustworthiness is key, encompassing the credibility, transferability, dependability, and confirmability of the findings. The rigor of the analysis - the thoroughness and care taken in data collection and analysis - is another crucial factor. 

Transparency in documenting the analysis process and decision-making is essential, as is reflexivity - acknowledging and examining the researcher's own biases and influences. 

Employing techniques like member checking and triangulation all contribute to the strength of qualitative analysis.

Benefits of qualitative data analysis

The benefits of qualitative data analysis are numerous. It uncovers rich, nuanced understanding of complex phenomena and allows for unexpected discoveries and new research directions. 

By capturing the 'why' behind behaviors and opinions, qualitative data analysis methods provide crucial context. 

Qualitative analysis can also lead to new theoretical frameworks or hypotheses and enhances quantitative findings with depth and detail. It's particularly adept at capturing cultural nuances that might be missed in quantitative studies.

Challenges of Qualitative Data Analysis

Researchers face several challenges when conducting qualitative data analysis. 

Managing and making sense of large volumes of rich, complex data can lead to data overload. Maintaining consistent coding across large datasets or between multiple coders can be difficult. 

There's a delicate balance to strike between providing enough context and maintaining focus on analysis. Recognizing and mitigating researcher biases in data interpretation is an ongoing challenge. 

The learning curve for qualitative data analysis software can be steep and time-consuming. Ethical considerations, particularly around protecting participant anonymity while presenting rich, detailed data, require careful navigation. Integrating different types of data from various sources can be complex. Time management is crucial, as researchers must balance the depth of analysis with project timelines and resources. Finally, communicating complex qualitative insights in clear, compelling ways can be challenging.

Best Software to Analyze Qualitative Data

G2 rating: 4.6/5

Pricing: Starts at $30 monthly.

Looppanel is an AI-powered research assistant and repository platform that can make it 5x faster to get to insights, by automating all the manual, tedious parts of your job. 

Here’s how Looppanel’s features can help with qualitative data analysis:

• Automatic Transcription: Quickly turn speech into accurate text; it works across 8 languages and even heavy accents, with over 90% accuracy.
• AI Note-Taking: The research assistant can join you on calls and take notes, as well as automatically sort your notes based on your interview questions.
• Automatic Tagging: Easily tag and organize your data with free AI tools.
• Insight Generation: Create shareable insights that fit right into your other tools.
• Repository Search: Run Google-like searches within your projects and calls to find a data snippet/quote in seconds
• Smart Summary: Ask the AI a question on your research, and it will give you an answer, using extracts from your data as citations.

Looppanel’s focus on automating research tasks makes it perfect for researchers who want to save time and work smarter.

G2 rating: 4.7/5

Pricing: Free version available, with the Plus version costing $20 monthly.

ChatGPT, developed by OpenAI, offers a range of capabilities for qualitative data analysis including:

• Document analysis : It can easily extract and analyze text from various file formats.
• Summarization : GPT can condense lengthy documents into concise summaries.
• Advanced Data Analysis (ADA) : For paid users, Chat-GPT offers quantitative analysis of data documents.
• Sentiment analysis: Although not Chat-GPT’s specialty, it can still perform basic sentiment analysis on text data.

ChatGPT's versatility makes it valuable for researchers who need quick insights from diverse text sources.

How to use ChatGPT for qualitative data analysis

ChatGPT can be a handy sidekick in your qualitative analysis, if you do the following:

• Use it to summarize long documents or transcripts
• Ask it to identify key themes in your data
• Use it for basic sentiment analysis
• Have it generate potential codes based on your research questions
• Use it to brainstorm interpretations of your findings

G2 rating: 4.7/5 Pricing: Custom

Atlas.ti is a powerful platform built for detailed qualitative and mixed-methods research, offering a lot of capabilities for running both quantitative and qualitative research.

It’s key data analysis features include:

• Multi-format Support: Analyze text, PDFs, images, audio, video, and geo data all within one platform.
• AI-Powered Coding: Uses AI to suggest codes and summarize documents.
• Collaboration Tools: Ideal for teams working on complex research projects.
• Data Visualization: Create network views and other visualizations to showcase relationships in your data.

G2 rating: 4.1/5 Pricing: Custom

NVivo is another powerful platform for qualitative and mixed-methods research. It’s analysis features include:

• Data Import and Organization: Easily manage different data types, including text, audio, and video.
• AI-Powered Coding: Speeds up the coding process with machine learning.
• Visualization Tools: Create charts, graphs, and diagrams to represent your findings.
• Collaboration Features: Suitable for team-based research projects.

NVivo combines AI capabilities with traditional qualitative analysis tools, making it versatile for various research needs.

Can Excel do qualitative data analysis?

Excel can be a handy tool for qualitative data analysis, especially if you're just starting out or working on a smaller project. While it's not specialized qualitative data analysis software, you can use it to organize your data, maybe putting different themes in different columns. It's good for basic coding, where you label bits of text with keywords. You can use its filter feature to focus on specific themes. Excel can also create simple charts to visualize your findings. But for bigger or more complex projects, you might want to look into software designed specifically for qualitative data analysis. These tools often have more advanced features that can save you time and help you dig deeper into your data.

How do you show qualitative analysis?

Showing qualitative data analysis is about telling the story of your data. In qualitative data analysis methods, we use quotes from interviews or documents to back up our points. Create charts or mind maps to show how different ideas connect, which is a common practice in data analysis in qualitative research. Group your findings into themes that make sense. Then, write it all up in a way that flows, explaining what you found and why it matters.

What is the best way to analyze qualitative data?

There's no one-size-fits-all approach to how to analyze qualitative data, but there are some tried-and-true steps. 

Start by getting your data in order. Then, read through it a few times to get familiar with it. As you go, start marking important bits with codes - this is a fundamental qualitative data analysis method. Group similar codes into bigger themes. Look for patterns in these themes - how do they connect? 

Finally, think about what it all means in the bigger picture of your research. Remember, it's okay to go back and forth between these steps as you dig deeper into your data. Qualitative data analysis software can be a big help in this process, especially for managing large amounts of data.

In qualitative methods of test analysis, what do test developers do to generate data?

Test developers in qualitative research might sit down with people for in-depth chats or run group discussions, which are key qualitative data analysis methods. They often use surveys with open-ended questions that let people express themselves freely. Sometimes, they'll observe people in their natural environment, taking notes on what they see. They might also dig into existing documents or artifacts that relate to their topic. The goal is to gather rich, detailed information that helps them understand the full picture, which is crucial in data analysis in qualitative research.

Which is not a purpose of reflexivity during qualitative data analysis?

Reflexivity in qualitative data analysis isn't about proving you're completely objective. That's not the goal. Instead, it's about being honest about who you are as a researcher. It's recognizing that your own experiences and views might influence how you see the data. By being upfront about this, you actually make your research more trustworthy. It's also a way to dig deeper into your data, seeing things you might have missed at first glance. This self-awareness is a crucial part of qualitative data analysis methods.

What is a qualitative data analysis example?

A simple example is analyzing customer feedback for a new product. You might collect feedback, read through responses, create codes like "ease of use" or "design," and group similar codes into themes. You'd then identify patterns and support findings with specific quotes. This process helps transform raw feedback into actionable insights.

How to analyze qualitative data from a survey?

First, gather all your responses in one place. Read through them to get a feel for what people are saying. Then, start labeling responses with codes - short descriptions of what each bit is about. This coding process is a fundamental qualitative data analysis method. Group similar codes into bigger themes. Look for patterns in these themes. Are certain ideas coming up a lot? Do different groups of people have different views? Use actual quotes from your survey to back up what you're seeing. Think about how your findings relate to your original research questions. 

Which one is better, NVivo or Atlas.ti?

NVivo is known for being user-friendly and great for team projects. Atlas.ti shines when it comes to visual mapping of concepts and handling geographic data. Both can handle a variety of data types and have powerful tools for qualitative data analysis. The best way to decide is to try out both if you can. 

While these are powerful tools, the core of qualitative data analysis still relies on your analytical skills and understanding of qualitative data analysis methods.

Do I need to use NVivo for qualitative data analysis?

You don't necessarily need NVivo for qualitative data analysis, but it can definitely make your life easier, especially for bigger projects. Think of it like using a power tool versus a hand tool - you can get the job done either way, but the power tool might save you time and effort. For smaller projects or if you're just starting out, you might be fine with simpler tools or even free qualitative data analysis software. But if you're dealing with lots of data, or if you need to collaborate with a team, or if you want to do more complex analysis, then specialized qualitative data analysis software like NVivo can be a big help. It's all about finding the right tool for your specific research needs and the qualitative data analysis methods you're using.

Here’s a guide that can help you decide.

How to use NVivo for qualitative data analysis

First, you import all your data - interviews, documents, videos, whatever you've got. Then you start creating "nodes," which are like folders for different themes or ideas in your data. As you read through your material, you highlight bits that relate to these themes and file them under the right nodes. NVivo lets you easily search through all this organized data, find connections between different themes, and even create visual maps of how everything relates.

How much does NVivo cost?

NVivo's pricing isn't one-size-fits-all. They offer different plans for individuals, teams, and large organizations, but they don't publish their prices openly. Contact the team here for a custom quote.

What are the four steps of qualitative data analysis?

While qualitative data analysis is often iterative, it generally follows these four main steps:

1. Data Collection: Gathering raw data through interviews, observations, or documents.

2. Data Preparation: Organizing and transcribing the collected data.

3. Data Coding: Identifying and labeling important concepts or themes in the data.

4. Interpretation: Drawing meaning from the coded data and developing insights.

Follow us on

Get the best resources for ux research, in your inbox, related articles.

Resources & Guides

February 15, 2024

How to use AI for Qualitative Data Analysis

August 15, 2024

Transcription in Qualitative Research: A Comprehensive Guide for UX Researchers

May 22, 2024

Triangulation in Qualitative Research: A Comprehensive Guide [2024]

Looppanel automatically records your calls, transcribes them, and centralizes all your research data in one place

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Write for Us
• BMJ Journals

You are here

• Volume 22, Issue 1
• How to appraise qualitative research
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Calvin Moorley 1 ,
• Xabi Cathala 2
• 1 Nursing Research and Diversity in Care, School of Health and Social Care , London South Bank University , London , UK
• 2 Institute of Vocational Learning , School of Health and Social Care, London South Bank University , London , UK
• Correspondence to Dr Calvin Moorley, Nursing Research and Diversity in Care, School of Health and Social Care, London South Bank University, London SE1 0AA, UK; Moorleyc{at}lsbu.ac.uk

https://doi.org/10.1136/ebnurs-2018-103044

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

In order to make a decision about implementing evidence into practice, nurses need to be able to critically appraise research. Nurses also have a professional responsibility to maintain up-to-date practice. 1 This paper provides a guide on how to critically appraise a qualitative research paper.

What is qualitative research?

• View inline

Useful terms

Some of the qualitative approaches used in nursing research include grounded theory, phenomenology, ethnography, case study (can lend itself to mixed methods) and narrative analysis. The data collection methods used in qualitative research include in depth interviews, focus groups, observations and stories in the form of diaries or other documents. 3

Authenticity

Title, keywords, authors and abstract.

In a previous paper, we discussed how the title, keywords, authors’ positions and affiliations and abstract can influence the authenticity and readability of quantitative research papers, 4 the same applies to qualitative research. However, other areas such as the purpose of the study and the research question, theoretical and conceptual frameworks, sampling and methodology also need consideration when appraising a qualitative paper.

Purpose and question

The topic under investigation in the study should be guided by a clear research question or a statement of the problem or purpose. An example of a statement can be seen in table 2 . Unlike most quantitative studies, qualitative research does not seek to test a hypothesis. The research statement should be specific to the problem and should be reflected in the design. This will inform the reader of what will be studied and justify the purpose of the study. 5

Example of research question and problem statement

An appropriate literature review should have been conducted and summarised in the paper. It should be linked to the subject, using peer-reviewed primary research which is up to date. We suggest papers with a age limit of 5–8 years excluding original work. The literature review should give the reader a balanced view on what has been written on the subject. It is worth noting that for some qualitative approaches some literature reviews are conducted after the data collection to minimise bias, for example, in grounded theory studies. In phenomenological studies, the review sometimes occurs after the data analysis. If this is the case, the author(s) should make this clear.

Theoretical and conceptual frameworks

Most authors use the terms theoretical and conceptual frameworks interchangeably. Usually, a theoretical framework is used when research is underpinned by one theory that aims to help predict, explain and understand the topic investigated. A theoretical framework is the blueprint that can hold or scaffold a study’s theory. Conceptual frameworks are based on concepts from various theories and findings which help to guide the research. 6 It is the researcher’s understanding of how different variables are connected in the study, for example, the literature review and research question. Theoretical and conceptual frameworks connect the researcher to existing knowledge and these are used in a study to help to explain and understand what is being investigated. A framework is the design or map for a study. When you are appraising a qualitative paper, you should be able to see how the framework helped with (1) providing a rationale and (2) the development of research questions or statements. 7 You should be able to identify how the framework, research question, purpose and literature review all complement each other.

There remains an ongoing debate in relation to what an appropriate sample size should be for a qualitative study. We hold the view that qualitative research does not seek to power and a sample size can be as small as one (eg, a single case study) or any number above one (a grounded theory study) providing that it is appropriate and answers the research problem. Shorten and Moorley 8 explain that three main types of sampling exist in qualitative research: (1) convenience (2) judgement or (3) theoretical. In the paper , the sample size should be stated and a rationale for how it was decided should be clear.

Methodology

Qualitative research encompasses a variety of methods and designs. Based on the chosen method or design, the findings may be reported in a variety of different formats. Table 3 provides the main qualitative approaches used in nursing with a short description.

Different qualitative approaches

The authors should make it clear why they are using a qualitative methodology and the chosen theoretical approach or framework. The paper should provide details of participant inclusion and exclusion criteria as well as recruitment sites where the sample was drawn from, for example, urban, rural, hospital inpatient or community. Methods of data collection should be identified and be appropriate for the research statement/question.

Data collection

Overall there should be a clear trail of data collection. The paper should explain when and how the study was advertised, participants were recruited and consented. it should also state when and where the data collection took place. Data collection methods include interviews, this can be structured or unstructured and in depth one to one or group. 9 Group interviews are often referred to as focus group interviews these are often voice recorded and transcribed verbatim. It should be clear if these were conducted face to face, telephone or any other type of media used. Table 3 includes some data collection methods. Other collection methods not included in table 3 examples are observation, diaries, video recording, photographs, documents or objects (artefacts). The schedule of questions for interview or the protocol for non-interview data collection should be provided, available or discussed in the paper. Some authors may use the term ‘recruitment ended once data saturation was reached’. This simply mean that the researchers were not gaining any new information at subsequent interviews, so they stopped data collection.

The data collection section should include details of the ethical approval gained to carry out the study. For example, the strategies used to gain participants’ consent to take part in the study. The authors should make clear if any ethical issues arose and how these were resolved or managed.

The approach to data analysis (see ref  10 ) needs to be clearly articulated, for example, was there more than one person responsible for analysing the data? How were any discrepancies in findings resolved? An audit trail of how the data were analysed including its management should be documented. If member checking was used this should also be reported. This level of transparency contributes to the trustworthiness and credibility of qualitative research. Some researchers provide a diagram of how they approached data analysis to demonstrate the rigour applied ( figure 1 ).

• Download figure
• Open in new tab
• Download powerpoint

Example of data analysis diagram.

Validity and rigour

The study’s validity is reliant on the statement of the question/problem, theoretical/conceptual framework, design, method, sample and data analysis. When critiquing qualitative research, these elements will help you to determine the study’s reliability. Noble and Smith 11 explain that validity is the integrity of data methods applied and that findings should accurately reflect the data. Rigour should acknowledge the researcher’s role and involvement as well as any biases. Essentially it should focus on truth value, consistency and neutrality and applicability. 11 The authors should discuss if they used triangulation (see table 2 ) to develop the best possible understanding of the phenomena.

Themes and interpretations and implications for practice

In qualitative research no hypothesis is tested, therefore, there is no specific result. Instead, qualitative findings are often reported in themes based on the data analysed. The findings should be clearly linked to, and reflect, the data. This contributes to the soundness of the research. 11 The researchers should make it clear how they arrived at the interpretations of the findings. The theoretical or conceptual framework used should be discussed aiding the rigour of the study. The implications of the findings need to be made clear and where appropriate their applicability or transferability should be identified. 12

Discussions, recommendations and conclusions

The discussion should relate to the research findings as the authors seek to make connections with the literature reviewed earlier in the paper to contextualise their work. A strong discussion will connect the research aims and objectives to the findings and will be supported with literature if possible. A paper that seeks to influence nursing practice will have a recommendations section for clinical practice and research. A good conclusion will focus on the findings and discussion of the phenomena investigated.

Qualitative research has much to offer nursing and healthcare, in terms of understanding patients’ experience of illness, treatment and recovery, it can also help to understand better areas of healthcare practice. However, it must be done with rigour and this paper provides some guidance for appraising such research. To help you critique a qualitative research paper some guidance is provided in table 4 .

Some guidance for critiquing qualitative research

• ↵ Nursing and Midwifery Council . The code: Standard of conduct, performance and ethics for nurses and midwives . 2015 https://www.nmc.org.uk/globalassets/sitedocuments/nmc-publications/nmc-code.pdf ( accessed 21 Aug 18 ).
• Barrett D ,
• Cathala X ,
• Shorten A ,

Patient consent for publication Not required.

Competing interests None declared.

Provenance and peer review Commissioned; internally peer reviewed.

Read the full text or download the PDF:

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Can J Hosp Pharm
• v.68(3); May-Jun 2015

Qualitative Research: Data Collection, Analysis, and Management

Introduction.

In an earlier paper, 1 we presented an introduction to using qualitative research methods in pharmacy practice. In this article, we review some principles of the collection, analysis, and management of qualitative data to help pharmacists interested in doing research in their practice to continue their learning in this area. Qualitative research can help researchers to access the thoughts and feelings of research participants, which can enable development of an understanding of the meaning that people ascribe to their experiences. Whereas quantitative research methods can be used to determine how many people undertake particular behaviours, qualitative methods can help researchers to understand how and why such behaviours take place. Within the context of pharmacy practice research, qualitative approaches have been used to examine a diverse array of topics, including the perceptions of key stakeholders regarding prescribing by pharmacists and the postgraduation employment experiences of young pharmacists (see “Further Reading” section at the end of this article).

In the previous paper, 1 we outlined 3 commonly used methodologies: ethnography 2 , grounded theory 3 , and phenomenology. 4 Briefly, ethnography involves researchers using direct observation to study participants in their “real life” environment, sometimes over extended periods. Grounded theory and its later modified versions (e.g., Strauss and Corbin 5 ) use face-to-face interviews and interactions such as focus groups to explore a particular research phenomenon and may help in clarifying a less-well-understood problem, situation, or context. Phenomenology shares some features with grounded theory (such as an exploration of participants’ behaviour) and uses similar techniques to collect data, but it focuses on understanding how human beings experience their world. It gives researchers the opportunity to put themselves in another person’s shoes and to understand the subjective experiences of participants. 6 Some researchers use qualitative methodologies but adopt a different standpoint, and an example of this appears in the work of Thurston and others, 7 discussed later in this paper.

Qualitative work requires reflection on the part of researchers, both before and during the research process, as a way of providing context and understanding for readers. When being reflexive, researchers should not try to simply ignore or avoid their own biases (as this would likely be impossible); instead, reflexivity requires researchers to reflect upon and clearly articulate their position and subjectivities (world view, perspectives, biases), so that readers can better understand the filters through which questions were asked, data were gathered and analyzed, and findings were reported. From this perspective, bias and subjectivity are not inherently negative but they are unavoidable; as a result, it is best that they be articulated up-front in a manner that is clear and coherent for readers.

THE PARTICIPANT’S VIEWPOINT

What qualitative study seeks to convey is why people have thoughts and feelings that might affect the way they behave. Such study may occur in any number of contexts, but here, we focus on pharmacy practice and the way people behave with regard to medicines use (e.g., to understand patients’ reasons for nonadherence with medication therapy or to explore physicians’ resistance to pharmacists’ clinical suggestions). As we suggested in our earlier article, 1 an important point about qualitative research is that there is no attempt to generalize the findings to a wider population. Qualitative research is used to gain insights into people’s feelings and thoughts, which may provide the basis for a future stand-alone qualitative study or may help researchers to map out survey instruments for use in a quantitative study. It is also possible to use different types of research in the same study, an approach known as “mixed methods” research, and further reading on this topic may be found at the end of this paper.

The role of the researcher in qualitative research is to attempt to access the thoughts and feelings of study participants. This is not an easy task, as it involves asking people to talk about things that may be very personal to them. Sometimes the experiences being explored are fresh in the participant’s mind, whereas on other occasions reliving past experiences may be difficult. However the data are being collected, a primary responsibility of the researcher is to safeguard participants and their data. Mechanisms for such safeguarding must be clearly articulated to participants and must be approved by a relevant research ethics review board before the research begins. Researchers and practitioners new to qualitative research should seek advice from an experienced qualitative researcher before embarking on their project.

DATA COLLECTION

Whatever philosophical standpoint the researcher is taking and whatever the data collection method (e.g., focus group, one-to-one interviews), the process will involve the generation of large amounts of data. In addition to the variety of study methodologies available, there are also different ways of making a record of what is said and done during an interview or focus group, such as taking handwritten notes or video-recording. If the researcher is audio- or video-recording data collection, then the recordings must be transcribed verbatim before data analysis can begin. As a rough guide, it can take an experienced researcher/transcriber 8 hours to transcribe one 45-minute audio-recorded interview, a process than will generate 20–30 pages of written dialogue.

Many researchers will also maintain a folder of “field notes” to complement audio-taped interviews. Field notes allow the researcher to maintain and comment upon impressions, environmental contexts, behaviours, and nonverbal cues that may not be adequately captured through the audio-recording; they are typically handwritten in a small notebook at the same time the interview takes place. Field notes can provide important context to the interpretation of audio-taped data and can help remind the researcher of situational factors that may be important during data analysis. Such notes need not be formal, but they should be maintained and secured in a similar manner to audio tapes and transcripts, as they contain sensitive information and are relevant to the research. For more information about collecting qualitative data, please see the “Further Reading” section at the end of this paper.

DATA ANALYSIS AND MANAGEMENT

If, as suggested earlier, doing qualitative research is about putting oneself in another person’s shoes and seeing the world from that person’s perspective, the most important part of data analysis and management is to be true to the participants. It is their voices that the researcher is trying to hear, so that they can be interpreted and reported on for others to read and learn from. To illustrate this point, consider the anonymized transcript excerpt presented in Appendix 1 , which is taken from a research interview conducted by one of the authors (J.S.). We refer to this excerpt throughout the remainder of this paper to illustrate how data can be managed, analyzed, and presented.

Interpretation of Data

Interpretation of the data will depend on the theoretical standpoint taken by researchers. For example, the title of the research report by Thurston and others, 7 “Discordant indigenous and provider frames explain challenges in improving access to arthritis care: a qualitative study using constructivist grounded theory,” indicates at least 2 theoretical standpoints. The first is the culture of the indigenous population of Canada and the place of this population in society, and the second is the social constructivist theory used in the constructivist grounded theory method. With regard to the first standpoint, it can be surmised that, to have decided to conduct the research, the researchers must have felt that there was anecdotal evidence of differences in access to arthritis care for patients from indigenous and non-indigenous backgrounds. With regard to the second standpoint, it can be surmised that the researchers used social constructivist theory because it assumes that behaviour is socially constructed; in other words, people do things because of the expectations of those in their personal world or in the wider society in which they live. (Please see the “Further Reading” section for resources providing more information about social constructivist theory and reflexivity.) Thus, these 2 standpoints (and there may have been others relevant to the research of Thurston and others 7 ) will have affected the way in which these researchers interpreted the experiences of the indigenous population participants and those providing their care. Another standpoint is feminist standpoint theory which, among other things, focuses on marginalized groups in society. Such theories are helpful to researchers, as they enable us to think about things from a different perspective. Being aware of the standpoints you are taking in your own research is one of the foundations of qualitative work. Without such awareness, it is easy to slip into interpreting other people’s narratives from your own viewpoint, rather than that of the participants.

To analyze the example in Appendix 1 , we will adopt a phenomenological approach because we want to understand how the participant experienced the illness and we want to try to see the experience from that person’s perspective. It is important for the researcher to reflect upon and articulate his or her starting point for such analysis; for example, in the example, the coder could reflect upon her own experience as a female of a majority ethnocultural group who has lived within middle class and upper middle class settings. This personal history therefore forms the filter through which the data will be examined. This filter does not diminish the quality or significance of the analysis, since every researcher has his or her own filters; however, by explicitly stating and acknowledging what these filters are, the researcher makes it easer for readers to contextualize the work.

Transcribing and Checking

For the purposes of this paper it is assumed that interviews or focus groups have been audio-recorded. As mentioned above, transcribing is an arduous process, even for the most experienced transcribers, but it must be done to convert the spoken word to the written word to facilitate analysis. For anyone new to conducting qualitative research, it is beneficial to transcribe at least one interview and one focus group. It is only by doing this that researchers realize how difficult the task is, and this realization affects their expectations when asking others to transcribe. If the research project has sufficient funding, then a professional transcriber can be hired to do the work. If this is the case, then it is a good idea to sit down with the transcriber, if possible, and talk through the research and what the participants were talking about. This background knowledge for the transcriber is especially important in research in which people are using jargon or medical terms (as in pharmacy practice). Involving your transcriber in this way makes the work both easier and more rewarding, as he or she will feel part of the team. Transcription editing software is also available, but it is expensive. For example, ELAN (more formally known as EUDICO Linguistic Annotator, developed at the Technical University of Berlin) 8 is a tool that can help keep data organized by linking media and data files (particularly valuable if, for example, video-taping of interviews is complemented by transcriptions). It can also be helpful in searching complex data sets. Products such as ELAN do not actually automatically transcribe interviews or complete analyses, and they do require some time and effort to learn; nonetheless, for some research applications, it may be a valuable to consider such software tools.

All audio recordings should be transcribed verbatim, regardless of how intelligible the transcript may be when it is read back. Lines of text should be numbered. Once the transcription is complete, the researcher should read it while listening to the recording and do the following: correct any spelling or other errors; anonymize the transcript so that the participant cannot be identified from anything that is said (e.g., names, places, significant events); insert notations for pauses, laughter, looks of discomfort; insert any punctuation, such as commas and full stops (periods) (see Appendix 1 for examples of inserted punctuation), and include any other contextual information that might have affected the participant (e.g., temperature or comfort of the room).

Dealing with the transcription of a focus group is slightly more difficult, as multiple voices are involved. One way of transcribing such data is to “tag” each voice (e.g., Voice A, Voice B). In addition, the focus group will usually have 2 facilitators, whose respective roles will help in making sense of the data. While one facilitator guides participants through the topic, the other can make notes about context and group dynamics. More information about group dynamics and focus groups can be found in resources listed in the “Further Reading” section.

Reading between the Lines

During the process outlined above, the researcher can begin to get a feel for the participant’s experience of the phenomenon in question and can start to think about things that could be pursued in subsequent interviews or focus groups (if appropriate). In this way, one participant’s narrative informs the next, and the researcher can continue to interview until nothing new is being heard or, as it says in the text books, “saturation is reached”. While continuing with the processes of coding and theming (described in the next 2 sections), it is important to consider not just what the person is saying but also what they are not saying. For example, is a lengthy pause an indication that the participant is finding the subject difficult, or is the person simply deciding what to say? The aim of the whole process from data collection to presentation is to tell the participants’ stories using exemplars from their own narratives, thus grounding the research findings in the participants’ lived experiences.

Smith 9 suggested a qualitative research method known as interpretative phenomenological analysis, which has 2 basic tenets: first, that it is rooted in phenomenology, attempting to understand the meaning that individuals ascribe to their lived experiences, and second, that the researcher must attempt to interpret this meaning in the context of the research. That the researcher has some knowledge and expertise in the subject of the research means that he or she can have considerable scope in interpreting the participant’s experiences. Larkin and others 10 discussed the importance of not just providing a description of what participants say. Rather, interpretative phenomenological analysis is about getting underneath what a person is saying to try to truly understand the world from his or her perspective.

Once all of the research interviews have been transcribed and checked, it is time to begin coding. Field notes compiled during an interview can be a useful complementary source of information to facilitate this process, as the gap in time between an interview, transcribing, and coding can result in memory bias regarding nonverbal or environmental context issues that may affect interpretation of data.

Coding refers to the identification of topics, issues, similarities, and differences that are revealed through the participants’ narratives and interpreted by the researcher. This process enables the researcher to begin to understand the world from each participant’s perspective. Coding can be done by hand on a hard copy of the transcript, by making notes in the margin or by highlighting and naming sections of text. More commonly, researchers use qualitative research software (e.g., NVivo, QSR International Pty Ltd; www.qsrinternational.com/products_nvivo.aspx ) to help manage their transcriptions. It is advised that researchers undertake a formal course in the use of such software or seek supervision from a researcher experienced in these tools.

Returning to Appendix 1 and reading from lines 8–11, a code for this section might be “diagnosis of mental health condition”, but this would just be a description of what the participant is talking about at that point. If we read a little more deeply, we can ask ourselves how the participant might have come to feel that the doctor assumed he or she was aware of the diagnosis or indeed that they had only just been told the diagnosis. There are a number of pauses in the narrative that might suggest the participant is finding it difficult to recall that experience. Later in the text, the participant says “nobody asked me any questions about my life” (line 19). This could be coded simply as “health care professionals’ consultation skills”, but that would not reflect how the participant must have felt never to be asked anything about his or her personal life, about the participant as a human being. At the end of this excerpt, the participant just trails off, recalling that no-one showed any interest, which makes for very moving reading. For practitioners in pharmacy, it might also be pertinent to explore the participant’s experience of akathisia and why this was left untreated for 20 years.

One of the questions that arises about qualitative research relates to the reliability of the interpretation and representation of the participants’ narratives. There are no statistical tests that can be used to check reliability and validity as there are in quantitative research. However, work by Lincoln and Guba 11 suggests that there are other ways to “establish confidence in the ‘truth’ of the findings” (p. 218). They call this confidence “trustworthiness” and suggest that there are 4 criteria of trustworthiness: credibility (confidence in the “truth” of the findings), transferability (showing that the findings have applicability in other contexts), dependability (showing that the findings are consistent and could be repeated), and confirmability (the extent to which the findings of a study are shaped by the respondents and not researcher bias, motivation, or interest).

One way of establishing the “credibility” of the coding is to ask another researcher to code the same transcript and then to discuss any similarities and differences in the 2 resulting sets of codes. This simple act can result in revisions to the codes and can help to clarify and confirm the research findings.

Theming refers to the drawing together of codes from one or more transcripts to present the findings of qualitative research in a coherent and meaningful way. For example, there may be examples across participants’ narratives of the way in which they were treated in hospital, such as “not being listened to” or “lack of interest in personal experiences” (see Appendix 1 ). These may be drawn together as a theme running through the narratives that could be named “the patient’s experience of hospital care”. The importance of going through this process is that at its conclusion, it will be possible to present the data from the interviews using quotations from the individual transcripts to illustrate the source of the researchers’ interpretations. Thus, when the findings are organized for presentation, each theme can become the heading of a section in the report or presentation. Underneath each theme will be the codes, examples from the transcripts, and the researcher’s own interpretation of what the themes mean. Implications for real life (e.g., the treatment of people with chronic mental health problems) should also be given.

DATA SYNTHESIS

In this final section of this paper, we describe some ways of drawing together or “synthesizing” research findings to represent, as faithfully as possible, the meaning that participants ascribe to their life experiences. This synthesis is the aim of the final stage of qualitative research. For most readers, the synthesis of data presented by the researcher is of crucial significance—this is usually where “the story” of the participants can be distilled, summarized, and told in a manner that is both respectful to those participants and meaningful to readers. There are a number of ways in which researchers can synthesize and present their findings, but any conclusions drawn by the researchers must be supported by direct quotations from the participants. In this way, it is made clear to the reader that the themes under discussion have emerged from the participants’ interviews and not the mind of the researcher. The work of Latif and others 12 gives an example of how qualitative research findings might be presented.

Planning and Writing the Report

As has been suggested above, if researchers code and theme their material appropriately, they will naturally find the headings for sections of their report. Qualitative researchers tend to report “findings” rather than “results”, as the latter term typically implies that the data have come from a quantitative source. The final presentation of the research will usually be in the form of a report or a paper and so should follow accepted academic guidelines. In particular, the article should begin with an introduction, including a literature review and rationale for the research. There should be a section on the chosen methodology and a brief discussion about why qualitative methodology was most appropriate for the study question and why one particular methodology (e.g., interpretative phenomenological analysis rather than grounded theory) was selected to guide the research. The method itself should then be described, including ethics approval, choice of participants, mode of recruitment, and method of data collection (e.g., semistructured interviews or focus groups), followed by the research findings, which will be the main body of the report or paper. The findings should be written as if a story is being told; as such, it is not necessary to have a lengthy discussion section at the end. This is because much of the discussion will take place around the participants’ quotes, such that all that is needed to close the report or paper is a summary, limitations of the research, and the implications that the research has for practice. As stated earlier, it is not the intention of qualitative research to allow the findings to be generalized, and therefore this is not, in itself, a limitation.

Planning out the way that findings are to be presented is helpful. It is useful to insert the headings of the sections (the themes) and then make a note of the codes that exemplify the thoughts and feelings of your participants. It is generally advisable to put in the quotations that you want to use for each theme, using each quotation only once. After all this is done, the telling of the story can begin as you give your voice to the experiences of the participants, writing around their quotations. Do not be afraid to draw assumptions from the participants’ narratives, as this is necessary to give an in-depth account of the phenomena in question. Discuss these assumptions, drawing on your participants’ words to support you as you move from one code to another and from one theme to the next. Finally, as appropriate, it is possible to include examples from literature or policy documents that add support for your findings. As an exercise, you may wish to code and theme the sample excerpt in Appendix 1 and tell the participant’s story in your own way. Further reading about “doing” qualitative research can be found at the end of this paper.

CONCLUSIONS

Qualitative research can help researchers to access the thoughts and feelings of research participants, which can enable development of an understanding of the meaning that people ascribe to their experiences. It can be used in pharmacy practice research to explore how patients feel about their health and their treatment. Qualitative research has been used by pharmacists to explore a variety of questions and problems (see the “Further Reading” section for examples). An understanding of these issues can help pharmacists and other health care professionals to tailor health care to match the individual needs of patients and to develop a concordant relationship. Doing qualitative research is not easy and may require a complete rethink of how research is conducted, particularly for researchers who are more familiar with quantitative approaches. There are many ways of conducting qualitative research, and this paper has covered some of the practical issues regarding data collection, analysis, and management. Further reading around the subject will be essential to truly understand this method of accessing peoples’ thoughts and feelings to enable researchers to tell participants’ stories.

Appendix 1. Excerpt from a sample transcript

The participant (age late 50s) had suffered from a chronic mental health illness for 30 years. The participant had become a “revolving door patient,” someone who is frequently in and out of hospital. As the participant talked about past experiences, the researcher asked:

• What was treatment like 30 years ago?
• Umm—well it was pretty much they could do what they wanted with you because I was put into the er, the er kind of system er, I was just on
• endless section threes.
• Really…
• But what I didn’t realize until later was that if you haven’t actually posed a threat to someone or yourself they can’t really do that but I didn’t know
• that. So wh-when I first went into hospital they put me on the forensic ward ’cause they said, “We don’t think you’ll stay here we think you’ll just
• run-run away.” So they put me then onto the acute admissions ward and – er – I can remember one of the first things I recall when I got onto that
• ward was sitting down with a er a Dr XXX. He had a book this thick [gestures] and on each page it was like three questions and he went through
• all these questions and I answered all these questions. So we’re there for I don’t maybe two hours doing all that and he asked me he said “well
• when did somebody tell you then that you have schizophrenia” I said “well nobody’s told me that” so he seemed very surprised but nobody had
• actually [pause] whe-when I first went up there under police escort erm the senior kind of consultants people I’d been to where I was staying and
• ermm so er [pause] I . . . the, I can remember the very first night that I was there and given this injection in this muscle here [gestures] and just
• having dreadful side effects the next day I woke up [pause]
• . . . and I suffered that akathesia I swear to you, every minute of every day for about 20 years.
• Oh how awful.
• And that side of it just makes life impossible so the care on the wards [pause] umm I don’t know it’s kind of, it’s kind of hard to put into words
• [pause]. Because I’m not saying they were sort of like not friendly or interested but then nobody ever seemed to want to talk about your life [pause]
• nobody asked me any questions about my life. The only questions that came into was they asked me if I’d be a volunteer for these student exams
• and things and I said “yeah” so all the questions were like “oh what jobs have you done,” er about your relationships and things and er but
• nobody actually sat down and had a talk and showed some interest in you as a person you were just there basically [pause] um labelled and you
• know there was there was [pause] but umm [pause] yeah . . .

This article is the 10th in the CJHP Research Primer Series, an initiative of the CJHP Editorial Board and the CSHP Research Committee. The planned 2-year series is intended to appeal to relatively inexperienced researchers, with the goal of building research capacity among practising pharmacists. The articles, presenting simple but rigorous guidance to encourage and support novice researchers, are being solicited from authors with appropriate expertise.

Previous articles in this series:

Bond CM. The research jigsaw: how to get started. Can J Hosp Pharm . 2014;67(1):28–30.

Tully MP. Research: articulating questions, generating hypotheses, and choosing study designs. Can J Hosp Pharm . 2014;67(1):31–4.

Loewen P. Ethical issues in pharmacy practice research: an introductory guide. Can J Hosp Pharm. 2014;67(2):133–7.

Tsuyuki RT. Designing pharmacy practice research trials. Can J Hosp Pharm . 2014;67(3):226–9.

Bresee LC. An introduction to developing surveys for pharmacy practice research. Can J Hosp Pharm . 2014;67(4):286–91.

Gamble JM. An introduction to the fundamentals of cohort and case–control studies. Can J Hosp Pharm . 2014;67(5):366–72.

Austin Z, Sutton J. Qualitative research: getting started. C an J Hosp Pharm . 2014;67(6):436–40.

Houle S. An introduction to the fundamentals of randomized controlled trials in pharmacy research. Can J Hosp Pharm . 2014; 68(1):28–32.

Charrois TL. Systematic reviews: What do you need to know to get started? Can J Hosp Pharm . 2014;68(2):144–8.

Competing interests: None declared.

Further Reading

Examples of qualitative research in pharmacy practice.

• Farrell B, Pottie K, Woodend K, Yao V, Dolovich L, Kennie N, et al. Shifts in expectations: evaluating physicians’ perceptions as pharmacists integrated into family practice. J Interprof Care. 2010; 24 (1):80–9. [ PubMed ] [ Google Scholar ]
• Gregory P, Austin Z. Postgraduation employment experiences of new pharmacists in Ontario in 2012–2013. Can Pharm J. 2014; 147 (5):290–9. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Marks PZ, Jennnings B, Farrell B, Kennie-Kaulbach N, Jorgenson D, Pearson-Sharpe J, et al. “I gained a skill and a change in attitude”: a case study describing how an online continuing professional education course for pharmacists supported achievement of its transfer to practice outcomes. Can J Univ Contin Educ. 2014; 40 (2):1–18. [ Google Scholar ]
• Nair KM, Dolovich L, Brazil K, Raina P. It’s all about relationships: a qualitative study of health researchers’ perspectives on interdisciplinary research. BMC Health Serv Res. 2008; 8 :110. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Pojskic N, MacKeigan L, Boon H, Austin Z. Initial perceptions of key stakeholders in Ontario regarding independent prescriptive authority for pharmacists. Res Soc Adm Pharm. 2014; 10 (2):341–54. [ PubMed ] [ Google Scholar ]

Qualitative Research in General

• Breakwell GM, Hammond S, Fife-Schaw C. Research methods in psychology. Thousand Oaks (CA): Sage Publications; 1995. [ Google Scholar ]
• Given LM. 100 questions (and answers) about qualitative research. Thousand Oaks (CA): Sage Publications; 2015. [ Google Scholar ]
• Miles B, Huberman AM. Qualitative data analysis. Thousand Oaks (CA): Sage Publications; 2009. [ Google Scholar ]
• Patton M. Qualitative research and evaluation methods. Thousand Oaks (CA): Sage Publications; 2002. [ Google Scholar ]
• Willig C. Introducing qualitative research in psychology. Buckingham (UK): Open University Press; 2001. [ Google Scholar ]

Group Dynamics in Focus Groups

• Farnsworth J, Boon B. Analysing group dynamics within the focus group. Qual Res. 2010; 10 (5):605–24. [ Google Scholar ]

Social Constructivism

• Social constructivism. Berkeley (CA): University of California, Berkeley, Berkeley Graduate Division, Graduate Student Instruction Teaching & Resource Center; [cited 2015 June 4]. Available from: http://gsi.berkeley.edu/gsi-guide-contents/learning-theory-research/social-constructivism/ [ Google Scholar ]

Mixed Methods

• Creswell J. Research design: qualitative, quantitative, and mixed methods approaches. Thousand Oaks (CA): Sage Publications; 2009. [ Google Scholar ]

Collecting Qualitative Data

• Arksey H, Knight P. Interviewing for social scientists: an introductory resource with examples. Thousand Oaks (CA): Sage Publications; 1999. [ Google Scholar ]
• Guest G, Namey EE, Mitchel ML. Collecting qualitative data: a field manual for applied research. Thousand Oaks (CA): Sage Publications; 2013. [ Google Scholar ]

Constructivist Grounded Theory

• Charmaz K. Grounded theory: objectivist and constructivist methods. In: Denzin N, Lincoln Y, editors. Handbook of qualitative research. 2nd ed. Thousand Oaks (CA): Sage Publications; 2000. pp. 509–35. [ Google Scholar ]

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

A fuzzy-set qualitative comparative analysis for understanding the interactive effects of good governance practices and ceo profiles on esg performance.

1. Introduction

2. literature review, 2.1. attributes included in the ggc and esg performance, 2.2. csr committee and esg performance, 2.3. ceo profile and esg performance, 3. materials and methods, 3.2. variables, 3.3. methodology, 4.1. descriptive statistics and correlation analysis, 4.2. contrarian case analysis, 4.3. analysis of necessary conditions, 4.4. sufficiency analysis for high esg performance, 4.5. bundles of corporate governance practices, 4.6. sufficiency analysis for non-high esg performance, 4.7. robustness analysis for sufficiency, 5. discussion, 5.1. csr committee as the critical component, 5.2. the complementary relationship between the creation of a csr committee and high ggc compliance, 5.3. the substituted relationship between ceo duality and ceo tenure, 5.4. the relative importance of antecedent conditions configuring corporate governance bundles to improve e-s-g practices, 5.5. contributions and implications, 5.6. limitations and future directions, 6. conclusions, author contributions, data availability statement, conflicts of interest.

Click here to enlarge figure

• Harjoto, M.; Laksmana, I.; Lee, R. Board Diversity and Corporate Social Responsibility. J. Bus. Ethics 2015 , 132 , 641–660. [ Google Scholar ] [ CrossRef ]
• Burke, J.J.; Hoitash, R.; Hoitash, U. The Heterogeneity of Board-Level Sustainability Committees and Corporate Social Performance. J. Bus. Ethics 2019 , 154 , 1161–1186. [ Google Scholar ] [ CrossRef ]
• OECD. G20/OECD Principles of Corporate Governance 2023 ; OECD: Paris, France, 2023; ISBN 9789264451681. [ Google Scholar ]
• European Parliament and of the Council Directive 2022/2464/EU of the European Parliament and of the Council of 14 December 2022 Amending Regulation (EU) No 537/2014, Directive 2004/109/EC, Directive 2006/43/EC and Directive 2013/34/EU, as Regards Corporate Sustainability Reporting. Off. J. Eur. Union 2022 , L 322 , 15–80.
• CNMV. Unified Good Governance Code of Listed Companies ; Comisión Nacional del Mercado de Valores: Madrid, Spain, 2006; Available online: https://www.cnmv.es/DocPortal/Publicaciones/CodigoGov/Codigo_unificado_Ing_04en.pdf (accessed on 17 April 2024).
• CNMV. Good Governance Code of Listed Companies ; Comisión Nacional del Mercado de Valores: Madrid, Spain, 2020; Available online: https://www.cnmv.es/DocPortal/Publicaciones/CodigoGov/CBG_2020_ENen.PDF (accessed on 17 April 2024).
• Villalba-Ríos, P.; Barroso-Castro, C.; Vecino-Gravel, J.D. The Influence of CEO Profile on Corporate Social Responsibility Companies. A Qualitative Comparative Analysis. Corp. Soc. Responsib. Environ. Manag. 2022 , 29 , 356–366. [ Google Scholar ] [ CrossRef ]
• Endrikat, J.; de Villiers, C.; Guenther, T.W.; Guenther, E.M. Board Characteristics and Corporate Social Responsibility: A Meta-Analytic Investigation. Bus. Soc. 2021 , 60 , 2099–2135. [ Google Scholar ] [ CrossRef ]
• Zaman, R.; Jain, T.; Samara, G.; Jamali, D. Corporate Governance Meets Corporate Social Responsibility: Mapping the Interface. Bus. Soc. 2022 , 61 , 690–752. [ Google Scholar ] [ CrossRef ]
• Yang, Z.; Na, J.; Dong, X. Corporate Governance for Sustainable Development: A Study on Mechanism Configuration. J. Clean. Prod. 2024 , 458 , 142509. [ Google Scholar ] [ CrossRef ]
• Lewellyn, K.; Muller-Kahle, M. ESG Leaders or Laggards? A Configurational Analysis of ESG Performance. Bus. Soc. 2023 , 63 , 1149–1202. [ Google Scholar ] [ CrossRef ]
• Jain, T.; Jamali, D. Looking Inside the Black Box: The Effect of Corporate Governance on Corporate Social Responsibility. Corp. Gov. Int. Rev. 2016 , 24 , 253–273. [ Google Scholar ] [ CrossRef ]
• Zaman, R.; Nadeem, M.; Carvajal, M. Corporate Governance and Corporate Social Responsibility Synergies: Evidence from New Zealand. Meditari Account. Res. 2020 , 29 , 135–160. [ Google Scholar ] [ CrossRef ]
• Aguilera, R.V.; Desender, K.A.; de Castro, L.R.K. A Bundle Perspective to Comparative Corporate Governance. In The SAGE Handbook of Corporate Governance ; Sage Publications: Thousand Oaks, CA, USA, 2012. [ Google Scholar ]
• Saridakis, C.; Angelidou, S.; Woodside, A.G. What Type of CSR Engagement Suits My Firm Best? Evidence from an Abductively-Derived Typology. J. Bus. Res. 2020 , 108 , 174–187. [ Google Scholar ] [ CrossRef ]
• Bolourian, S.; Alinaghian, L.; Angus, A. Exploring the Role of Board-Level Corporate Social Responsibility Committees in Corporate Social Responsibility Performance: A Configurational Approach. J. Bus. Res. 2023 , 169 , 114280. [ Google Scholar ] [ CrossRef ]
• Jensen, M.C.; Meckling, W.H. Theory of the Firm: Managerial Behavior, Agency Costs and Ownership Structure. J. Financ. Econ. 1976 , 3 , 305–360. [ Google Scholar ] [ CrossRef ]
• Jizi, M.; Salama, A.; Dixon, R.; Stratling, R. Corporate Governance and Corporate Social Responsibility Disclosure: Evidence from the US Banking Sector. J. Bus. Ethics 2014 , 125 , 601–615. [ Google Scholar ] [ CrossRef ]
• Pfeffer, J.; Salancik, G. The External Control of Organizations: A Resource Dependence Perspective ; Stanford Business Press: Stanford, CA, USA, 1978; ISBN 080474789X. [ Google Scholar ]
• Bear, S.; Rahman, N.; Post, C. The Impact of Board Diversity and Gender Composition on Corporate Social Responsibility and Firm Reputation. J. Bus. Ethics 2010 , 97 , 207–221. [ Google Scholar ] [ CrossRef ]
• Freeman, R.E. Strategic Management: A Stakeholder Approach ; Cambridge University Press: Cambridge, UK, 1984. [ Google Scholar ]
• Jamali, D.; Safieddine, A.M.; Rabbath, M. Corporate Governance and Corporate Social Responsibility Synergies and Interrelationships. Corp. Gov. Int. Rev. 2008 , 16 , 443–459. [ Google Scholar ] [ CrossRef ]
• Konrad, A.M.; Kramer, V.; Erkut, S. Critical mass: The impact of three or more women on corporate boards. Organ. Dyn. 2008 , 37 , 145–164. [ Google Scholar ] [ CrossRef ]
• Hambrick, D.C.; Mason, P.A. Upper Echelons: The Organization as a Reflection of Its Top Managers. Acad. Manag. Rev. 1984 , 9 , 193. [ Google Scholar ] [ CrossRef ]
• Haniffa, R.M.; Cooke, T.E. The Impact of Culture and Governance on Corporate Social Reporting. J. Account. Public Policy 2005 , 24 , 391–430. [ Google Scholar ] [ CrossRef ]
• Oh, W.-Y.; Chang, Y.K.; Jung, R. Experience-Based Human Capital or Fixed Paradigm Problem? CEO Tenure, Contextual Influences, and Corporate Social (Ir)Responsibility. J. Bus. Res. 2018 , 90 , 325–333. [ Google Scholar ] [ CrossRef ]
• Nguyen, L.T.M.; Nguyen, P.T. The Board Profiles That Promote Environmental, Social, and Governance Disclosure–Evidence from S&P 500 Firms. Financ. Res. Lett. 2023 , 55 , 103925. [ Google Scholar ] [ CrossRef ]
• Dwekat, A.; Seguí-Mas, E.; Zaid, M.A.A.; Tormo-Carbó, G. Corporate Governance and Corporate Social Responsibility: Mapping the Most Critical Drivers in the Board Academic Literature. Meditari Account. Res. 2022 , 30 , 1705–1739. [ Google Scholar ] [ CrossRef ]
• Ponomareva, Y.; Federo, R.; Aguilera, R.V.; Collin, S. The Cost of Conformity to Good Governance: Board Design and Compensation. Corp. Gov. Int. Rev. 2022 , 30 , 399–420. [ Google Scholar ] [ CrossRef ]
• Harjoto, M.; Jo, H. Corporate Governance and CSR Nexus. J. Bus. Ethics 2011 , 100 , 45–67. [ Google Scholar ] [ CrossRef ]
• Hillman, A.J.; Withers, M.C.; Collins, B.J. Resource Dependence Theory: A Review. J. Manag. 2009 , 35 , 1404–1427. [ Google Scholar ] [ CrossRef ]
• Khan, A.; Muttakin, M.B.; Siddiqui, J. Corporate Governance and Corporate Social Responsibility Disclosures: Evidence from an Emerging Economy. J. Bus. Ethics 2013 , 114 , 207–223. [ Google Scholar ] [ CrossRef ]
• Ibrahim, N.A.; Howard, D.P.; Angelidis, J.P. Board Members in the Service Industry: An Empirical Examination of the Relationship Between Corporate Social Responsibility Orientation and Directorial Type. J. Bus. Ethics 2003 , 47 , 393–401. [ Google Scholar ] [ CrossRef ]
• Majeed, S.; Aziz, T.; Saleem, S. The Effect of Corporate Governance Elements on Corporate Social Responsibility (CSR) Disclosure: An Empirical Evidence from Listed Companies at KSE Pakistan. Int. J. Financ. Stud. 2015 , 3 , 530–556. [ Google Scholar ] [ CrossRef ]
• Katmon, N.; Mohamad, Z.Z.; Norwani, N.M.; Farooque, O. Al Comprehensive Board Diversity and Quality of Corporate Social Responsibility Disclosure: Evidence from an Emerging Market. J. Bus. Ethics 2019 , 157 , 447–481. [ Google Scholar ] [ CrossRef ]
• Huang, S.; Hilary, G. Zombie Board: Board Tenure and Firm Performance. J. Account. Res. 2018 , 56 , 1285–1329. [ Google Scholar ] [ CrossRef ]
• Commission of the European Communities. Commission Recommendation of 15 February 2005 on the Role of Non-Executive or Supervisory Directors of Listed Companies and on the Committees of the (Supervisory) Board. Off. J. Eur. Union 2005 , L 52 , 51–63.
• Radu, C.; Smaili, N. Alignment Versus Monitoring: An Examination of the Effect of the CSR Committee and CSR-Linked Executive Compensation on CSR Performance. J. Bus. Ethics 2022 , 180 , 145–163. [ Google Scholar ] [ CrossRef ]
• Baraibar-Diez, E.; Odriozola, M.D. CSR Committees and Their Effect on ESG Performance in UK, France, Germany, and Spain. Sustainability 2019 , 11 , 5077. [ Google Scholar ] [ CrossRef ]
• Gennari, F.; Salvioni, D. CSR Committees on Boards: The Impact of the External Country Level Factors. J. Manag. Gov. 2019 , 23 , 759–785. [ Google Scholar ] [ CrossRef ]
• Michelon, G.; Parbonetti, A. The Effect of Corporate Governance on Sustainability Disclosure. J. Manag. Gov. 2012 , 16 , 1–33. [ Google Scholar ] [ CrossRef ]
• Chams, N.; García-Blandón, J. Sustainable or Not Sustainable? The Role of the Board of Directors. J. Clean. Prod. 2019 , 226 , 1067–1081. [ Google Scholar ] [ CrossRef ]
• Rodrigue, M.; Magnan, M.; Cho, C.H. Is Environmental Governance Substantive or Symbolic? An Empirical Investigation. J. Bus. Ethics 2013 , 114 , 107–129. [ Google Scholar ] [ CrossRef ]
• Hu, Z.; Ma, L.; Xu, X. The Impact Path of Executive Team Heterogeneity and Environmental-Social-Governance on Corporate Performance. Technol. Invest. 2023 , 14 , 279–292. [ Google Scholar ] [ CrossRef ]
• Wernicke, G.; Sajko, M.; Boone, C. How Much Influence Do CEOs Have on Company Actions and Outcomes? The Example of Corporate Social Responsibility. Acad. Manag. Discov. 2022 , 8 , 36–55. [ Google Scholar ] [ CrossRef ]
• Güngör, N.; Şeker, Y. The Relationship between Board Characteristics and ESG Performance: Evidence from the Oil, Gas and Coal Sector. Stratejik ve Sosyal Araştırmalar Dergisi 2022 , 6 , 17–37. [ Google Scholar ] [ CrossRef ]
• Uyar, A.; Kuzey, C.; Kilic, M.; Karaman, A.S. Board Structure, Financial Performance, Corporate Social Responsibility Performance, CSR Committee, and CEO Duality: Disentangling the Connection in Healthcare. Corp. Soc. Responsib. Environ. Manag. 2021 , 28 , 1730–1748. [ Google Scholar ] [ CrossRef ]
• Borghesi, R.; Houston, J.F.; Naranjo, A. Corporate Socially Responsible Investments: CEO Altruism, Reputation, and Shareholder Interests. J. Corp. Financ. 2014 , 26 , 164–181. [ Google Scholar ] [ CrossRef ]
• Hussain, M.J.; Tian, G.; Ayaz, M.; Ashraf, A. CEO Career Horizon and Corporate Social Responsibility Assurance. Span. J. Financ. Account. 2023 , 52 , 384–411. [ Google Scholar ] [ CrossRef ]
• García-Blandón, J.; Argilés-Bosch, J.M.; Ravenda, D. Exploring the Relationship between CEO Characteristics and Performance. J. Bus. Econ. Manag. 2019 , 20 , 1064–1082. [ Google Scholar ] [ CrossRef ]
• Chen, L.; Liao, C.; Tsang, A.; Yu, L. CEO Career Concerns in Early Tenure and Corporate Social Responsibility Reporting. Contemp. Account. Res. 2023 , 40 , 1545–1575. [ Google Scholar ] [ CrossRef ]
• Shahab, Y.; Ntim, C.G.; Chen, Y.; Ullah, F.; Li, H.; Ye, Z. Chief Executive Officer Attributes, Sustainable Performance, Environmental Performance, and Environmental Reporting: New Insights from Upper Echelons Perspective. Bus. Strategy Environ. 2020 , 29 , 1–16. [ Google Scholar ] [ CrossRef ]
• Bolourian, S.; Angus, A.; Alinaghian, L. The Impact of Corporate Governance on Corporate Social Responsibility at the Board-Level: A Critical Assessment. J. Clean. Prod. 2021 , 291 , 125752. [ Google Scholar ] [ CrossRef ]
• Krause, R.; Semadeni, M.; Withers, M.C. That Special Someone: When the Board Views Its Chair as a Resource. Strateg. Manag. J. 2016 , 37 , 1990–2002. [ Google Scholar ] [ CrossRef ]
• Miller, D. Stale in the Saddle: CEO Tenure and the Match between Organization and Environment. Manag. Sci. 1991 , 37 , 34–52. [ Google Scholar ] [ CrossRef ]
• McClelland, P.L.; Barker, V.L.; Oh, W.Y. CEO Career Horizon and Tenure: Future Performance Implications under Different Contingencies. J. Bus. Res. 2012 , 65 , 1387–1393. [ Google Scholar ] [ CrossRef ]
• Becker, G.S. Human Capital: A Theoretical and Empirical Analysis, with Special Reference to Education ; University of Chicago Press: Chicago, IL, USA, 2009. [ Google Scholar ]
• Kim, K.; Kim, T.-N. CEO Career Concerns and ESG Investments. Financ. Res. Lett. 2023 , 55 , 103819. [ Google Scholar ] [ CrossRef ]
• Fiss, P.C. A Set-Theoretic Approach to Organizational Configurations. Acad. Manag. Rev. 2007 , 32 , 1180–1198. [ Google Scholar ] [ CrossRef ]
• Paolone, F.; Sardi, A.; Sorano, E.; Ferraris, A. Integrated Processing of Sustainability Accounting Reports: A Multi-Utility Company Case Study. Meditari Account. Res. 2021 , 29 , 985–1004. [ Google Scholar ] [ CrossRef ]
• Liao, L.; Lin, T.; Zhang, Y. Corporate Board and Corporate Social Responsibility Assurance: Evidence from China. J. Bus. Ethics 2018 , 150 , 211–225. [ Google Scholar ] [ CrossRef ]
• Woodside, A.G. Moving beyond Multiple Regression Analysis to Algorithms: Calling for a Paradigm Shift from Symmetric to Asymmetric Thinking in Data Analysis and Crafting Theory. J. Bus. Res. 2013 , 66 , 463–472. [ Google Scholar ] [ CrossRef ]
• Ragin, C.C. The Comparative Method: Moving Beyond Qualitative and Quantitative Strategies ; University of California Press: Berkeley, CA, USA; Los Angeles, CA, USA, 1987; ISBN 0520909240. [ Google Scholar ]
• Schneider, C.Q.; Wagemann, C. Standards of Good Practice in Qualitative Comparative Analysis (QCA) and Fuzzy-Sets. Comp. Sociol. 2010 , 9 , 397–418. [ Google Scholar ] [ CrossRef ]
• Fiss, P.C. Building Better Causal Theories: A Fuzzy Set Approach to Typologies in Organization Research. Acad. Manag. J. 2011 , 54 , 393–420. [ Google Scholar ] [ CrossRef ]
• Zadeh, L.A. Fuzzy Sets. Inf. Control 1965 , 8 , 338–353. [ Google Scholar ] [ CrossRef ]
• Pappas, I.O.; Woodside, A.G. Fuzzy-Set Qualitative Comparative Analysis (FsQCA): Guidelines for Research Practice in Information Systems and Marketing. Int. J. Inf. Manag. 2021 , 58 , 102310. [ Google Scholar ] [ CrossRef ]
• Ragin, C.C. Redesigning Social Inquiry ; University of Chicago Press: Chicago, IL, USA; London, UK, 2008; ISBN 9780226702759. [ Google Scholar ]
• Rihoux, B.; Ragin, C.C. Configurational Comparative Methods: Qualitative Comparative Analysis (QCA) and Related Techniques ; Sage Publications: Thousand Oaks, CA, USA, 2009; ISBN 9781412942355. [ Google Scholar ]
• Mendel, J.M.; Korjani, M.M. A New Method for Calibrating the Fuzzy Sets Used in FsQCA. Inf. Sci. (N. Y.) 2018 , 468 , 155–171. [ Google Scholar ] [ CrossRef ]
• Misangyi, V.F.; Acharya, A.G. Substitutes or Complements? A Configurational Examination of Corporate Governance Mechanisms. Acad. Manag. J. 2014 , 57 , 1681–1705. [ Google Scholar ] [ CrossRef ]
• Misangyi, V.F.; Greckhamer, T.; Furnari, S.; Fiss, P.C.; Crilly, D.; Aguilera, R. Embracing Causal Complexity: The Emergence of a Neo-Configurational Perspective. J. Manag. 2016 , 43 , 255–282. [ Google Scholar ] [ CrossRef ]
• Greckhamer, T.; Furnari, S.; Fiss, P.C.; Aguilera, R.V. Studying Configurations with Qualitative Comparative Analysis: Best Practices in Strategy and Organization Research. Strateg. Organ. 2018 , 16 , 482–495. [ Google Scholar ] [ CrossRef ]
• Ragin, C.C.; Davey, S. Fuzzy-Set/Qualitative Comparative Analysis 4.0 ; Department of Sociology, University of California: Irvine, CA, USA, 2022. [ Google Scholar ]
• Russo, I.; Confente, I. From Dataset to Qualitative Comparative Analysis (QCA)—Challenges and Tricky Points: A Research Note on Contrarian Case Analysis and Data Calibration. Australas. Mark. J. (AMJ) 2019 , 27 , 129–135. [ Google Scholar ] [ CrossRef ]
• Woodside, A.G. Embrace Perform Model: Complexity Theory, Contrarian Case Analysis, and Multiple Realities. J. Bus. Res. 2014 , 67 , 2495–2503. [ Google Scholar ] [ CrossRef ]
• Schneider, C.Q.; Wagemann, C. Set-Theoretic Methods for the Social Sciences: A Guide to Qualitative Comparative Analysis ; Cambridge University Press: Cambridge, UK, 2012; ISBN 9781107013520. [ Google Scholar ]
• Mattke, J.; Maier, C.; Weitzel, T.; Thatcher, J.B. Qualitative Comparative Analysis in the Information Systems Discipline: A Literature Review and Methodological Recommendations. Internet Res. 2021 , 31 , 1493–1517. [ Google Scholar ] [ CrossRef ]
• Dwekat, A.; Seguí-Mas, E.; Tormo-Carbó, G.; Carmona, P. Corporate Governance Configurations and Corporate Social Responsibility Disclosure: Qualitative Comparative Analysis of Audit Committee and Board Characteristics. Corp. Soc. Responsib. Environ. Manag. 2020 , 27 , 2879–2892. [ Google Scholar ] [ CrossRef ]
• Thiem, A.; Duşa, A. Qualitative Comparative Analysis with R. A User’s Guide ; Springer: New York, NY, USA, 2013; ISBN 978-1-4614-4583-8. [ Google Scholar ]
• Witt, M.A.; Fainshmidt, S.; Aguilera, R.V. Our Board, Our Rules: Nonconformity to Global Corporate Governance Norms. Adm. Sci. Q. 2022 , 67 , 131–166. [ Google Scholar ] [ CrossRef ]
• Aguilera, R.V.; Marano, V.; Haxhi, I. International Corporate Governance: A Review and Opportunities for Future Research. J. Int. Bus. Stud. 2019 , 50 , 457–498. [ Google Scholar ] [ CrossRef ]
• Chen, W.T.; Zhou, G.S.; Zhu, X.K. CEO Tenure and Corporate Social Responsibility Performance. J. Bus. Res. 2019 , 95 , 292–302. [ Google Scholar ] [ CrossRef ]
• Gurol, B.; Lagasio, V. Women Board Members’ Impact on ESG Disclosure with Environment and Social Dimensions: Evidence from the European Banking Sector. Soc. Responsib. J. 2023 , 19 , 211–228. [ Google Scholar ] [ CrossRef ]
• Pathan, S. Strong Boards, CEO Power and Bank Risk-Taking. J. Bank Financ. 2009 , 33 , 1340–1350. [ Google Scholar ] [ CrossRef ]
• Oh, W.-Y.; Chang, Y.K.; Cheng, Z. When CEO Career Horizon Problems Matter for Corporate Social Responsibility: The Moderating Roles of Industry-Level Discretion and Blockholder Ownership. J. Bus. Ethics 2016 , 133 , 279–291. [ Google Scholar ] [ CrossRef ]
• Yoo, S.; Managi, S. Disclosure or Action: Evaluating ESG Behavior towards Financial Performance. Financ. Res. Lett. 2022 , 44 , 102108. [ Google Scholar ] [ CrossRef ]
• García-Castro, R.; Aguilera, R.V.; Ariño, M.A. Bundles of Firm Corporate Governance Practices: A Fuzzy Set Analysis. Corp. Gov. Int. Rev. 2013 , 21 , 390–407. [ Google Scholar ] [ CrossRef ]
• Schiehll, E.; Lewellyn, K.B.; Muller-Kahle, M.I. Pilot, Pivot and Advisory Boards: The Role of Governance Configurations in Innovation Commitment. Organ. Stud. 2018 , 39 , 1449–1472. [ Google Scholar ] [ CrossRef ]

Share and Cite

Remo-Diez, N.; Mendaña-Cuervo, C.; Arenas-Parra, M. A Fuzzy-Set Qualitative Comparative Analysis for Understanding the Interactive Effects of Good Governance Practices and CEO Profiles on ESG Performance. Mathematics 2024 , 12 , 2726. https://doi.org/10.3390/math12172726

Remo-Diez N, Mendaña-Cuervo C, Arenas-Parra M. A Fuzzy-Set Qualitative Comparative Analysis for Understanding the Interactive Effects of Good Governance Practices and CEO Profiles on ESG Performance. Mathematics . 2024; 12(17):2726. https://doi.org/10.3390/math12172726

Remo-Diez, Nieves, Cristina Mendaña-Cuervo, and Mar Arenas-Parra. 2024. "A Fuzzy-Set Qualitative Comparative Analysis for Understanding the Interactive Effects of Good Governance Practices and CEO Profiles on ESG Performance" Mathematics 12, no. 17: 2726. https://doi.org/10.3390/math12172726

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 02 September 2024

Propaganda in focus: decoding the media strategy of ISIS

• Yuanbo Qi   ORCID: orcid.org/0000-0001-9541-8220 1  

Humanities and Social Sciences Communications volume  11 , Article number:  1123 ( 2024 ) Cite this article

1 Altmetric

Metrics details

• Cultural and media studies
• Politics and international relations

This investigation employs the analytical framework established by Braddock and Horgan to conduct a comprehensive content analysis of 79 official English-language propaganda videos disseminated by ISIS, with the objective of quantifying the thematic composition and the evolutionary trajectory of ISIS’s international media operations and propaganda machinery from 2014 to 2017. The findings reveal that a predominant portion of the videos articulate narratives extensively centred around themes of the adversary and religious discourse, with the Sharia (Islamic law) emerging as the most prevalent theme. This research concludes that at a global scale, the propaganda apparatus of ISIS has orchestrated an intricate narrative, incorporating adversarial, theological, and emotional elements, thereby delineating the advanced sophistication of ISIS’s global propaganda endeavours.

Similar content being viewed by others

Visual propaganda in chinese central and local news agencies: a douyin case study

The portrayal of Pakistan as whited sepulture against the Taliban: a case study of American media

Transmediatisation of the Covid-19 crisis in Brazil: The emergence of (bio-/geo-)political repertoires of (re-)interpretation

Introduction.

Between 2014 and 2017, pivotal years in the trajectory of the Islamic State (ISIS), the group witnessed a meteoric rise and subsequent decline in territorial control (al-Lami, 2019 ; Barnard and Saad, 2017 ; Chulov, 2019 ; Curry, 2014a ; Damon et al. 2017 ; Gilsinan, 2014 ; Phippen, 2017 ). Throughout this period, ISIS strategically utilised media, particularly through its official English-language videos, to propagate its message worldwide. Abu Bakr al-Baghdadi’s proclamation of a caliphate at Mosul’s al-Nuri Mosque marked a significant milestone for ISIS, symbolically hearkening back to a perceived Islamic golden age and galvanising Muslims to join their cause (al-Lami, 2019 ; Chulov, 2019 ). The extreme measures employed by the group, along with the global military response, accentuated the multifaceted nature of contemporary extremist movements (Curry, 2014b ; Gilsinan, 2014 ; Malik, 2014b ).

Understanding ISIS’s media strategy is a vital piece of the puzzle in the broader fight against global terrorism. The English-language videos produced by ISIS from 2014 to 2017 underscore not just the group’s media acumen but also their ideological engagement with a global audience. These videos aimed to intimidate adversaries, recruit sympathisers, and affirm the group’s narrative, showcasing a new dimension of digital terrorism that manipulates psychological, rhetorical, and theological elements to wield influence (Dearden, 2015 ).

This study examines ISIS’s video propaganda to decode its key narratives, rhetoric device, and implications for counter narratives. The increased reliance of ISIS on propaganda as their territorial grasp expanded underscores the imperative to scrutinise and interpret their communicative strategies. By delving into the content of these videos, this research seeks insights into how ISIS’s messaging evolved in response to military setbacks, territorial losses, and shifts in the geopolitical landscape.

This enquiry is of paramount importance for scholarly exploration and practical counter-terrorism measures. Recognising the patterns, themes, and shifts in ISIS’s propagated content enables security analysts and policymakers to anticipate and counteract the tactics of ISIS and similar entities. The insight derived from this study has the potential to inform counter-narratives and deconstruct extremist messaging strategies, thus curtailing the reach and impact of jihadist propaganda on a global scale (Gerges, 2019 ).

The confines of this study are set strictly within the official English-language video propaganda disseminated by ISIS from 2014 to 2017, a period marking the zenith of the group’s influence (al-Lami, 2019 ; Barnard and Saad, 2017 ; Chulov, 2019 ; Curry, 2014a ; Damon et al. 2017 ; Gilsinan, 2014 ; Phippen, 2017 ), thus permitting an analysis of its narrative amidst changing geopolitical realities. By focusing solely on these materials, the research delves into the intricacies of ISIS’s marketing strategies and the ideological underpinnings aimed at a global, primarily English-speaking, audience. The study’s deliberate temporal and linguistic boundaries enable a concentrated examination of the group’s communication tactics during a critical juncture of its existence. The study posits the following specific inquiries:

What intrinsic narrative motifs emerge with the greatest prominence in the videos?

The lens of the videos’ intrinsic narratives, in what manner is the worldview inherent to ISIS ideology articulated and represented?

How does the thematic distribution within these videos reflect an evolution or transformation in the period from 2014 to 2017, and what does this shift signify about ISIS media strategy?

A thorough examination of the corpus of research reveals a trend of tactical and thematic complexity in ISIS’s English-language propaganda (Colas, 2017 ; Winter, 2015 ; Winter, 2018 ). First, many studies do not differentiate their examination of ISIS propaganda between Arabic and English-language content, thereby overlooking the group’s nuanced and targeted messaging aimed at reaching a global audience (Abrahms et al. 2017 ; Fisher, 2015 ; Lakomy, 2021a , 2021b ; Salem et al. 2008 ). Assessments like those by Qi ( 2020a , 2020b ) focus on English-language propaganda, primarily highlighting production aspects or performed speech acts without exploring the thematic richness of the content (Colas, 2017 ). Secondly, there is a lack of studies documenting how these themes have evolved in response to the group’s changing circumstances and the global environment. The thematic evolution within the English text remains unexplored (Al-Rawi, 2018 ; Colas, 2017 ; Fisher, 2015 ; Kuznar, 2017 ; Qi, 2020b ; Winter, 2018 ). By providing a detailed study of the themes and substance found in ISIS’s English-language videos, this research bridges the gaps by analysing how these videos have changed to reflect global dynamics and the group circumstances. This study narrows its focus to provide a necessary perspective on ISIS’s strategic use of English-language materials aimed at global audiences, highlighting the specificity of their media strategy and deliberate use of language in terrorist propaganda.

Literature review

The evolution of isis media strategy and its historical context.

ISIS, also known as the Islamic State, surged to global prominence under the leadership of Abu Bakr al-Baghdadi, who proclaimed a caliphate in 2014, with the ambition of restoring what they considered the golden age of Islam and calling for global Jihad (al-Lami, 2019 ; Chulov, 2019 ). This group quickly gained infamy for its brutal tactics, including the persecution of minorities and conducting high-profile terror attacks, while seizing vast territories in Syria and Iraq (Curry, 2014b ; Gilsinan, 2014 ; Malik, 2014a , 2014b ). By 2017, concerted military efforts significantly diminished ISIS’s territorial control, leading to the loss of their critical strongholds, Mosul and Raqqa (Barnard and Saad, 2017 ; Chulov, 2019 ; Damon et al. 2017 ; Phippen, 2017 ). Despite their territorial defeat and the declaration of the caliphate’s end in 2019, ISIS continues to represent a threat through dispersed networks and sleeper cells globally (Forrest, 2019 ). For instance, nearly a decade after the 2015 terror attacks in Paris, an ISIS affiliate, known as the Islamic State Khorasan Province (ISIS-Khorasan), claimed responsibility for a devastating attack at the Crocus City concert hall in Moscow, which killed 137 people in 2024 (Roth and Sauer, 2024 ; Schmitt, 2024 ). The persistence of the group’s radical ideology suggests that, without addressing the root causes of its emergence, the potential for resurgence remains (Gerges, 2019 ). The transformation of ISIS into a more diffuse entity, which inspires global violence through its propaganda, underscores the enduring challenge of neutralising its impact (al-Lami, 2019 ; Votel et al. 2017 ).

ISIS’s media strategy has a complex history that has developed over time due to strategic adjustments and technology breakthroughs (Gerges, 2016 ). ISIS has recognised the power of the media from the beginning, using it as a recruiting tool and a psychological warfare weapon (Atwan, 2015 ). Their activities at first resembled conventional jihadist communication channels, but they quickly changed into an advanced media machine that made use of social media and excellent video production.

ISIS propagandised mostly in Arabic in the phases, focusing on the local populace as well as the larger Middle East. There was a noticeable change, though, as the group looked to broaden its international appeal and attract members from Western nations. The group’s magazine “Dabiq,” and subsequently “Rumiyah,” as well as a number of videos meant for Western audiences, signalled the appearance of English-language material (Milton, 2020 ). The deliberate change in strategy to add English-language content indicated that the campaign was intended to be multinational.

In addition to being linguistic, the shift from local to international media was both thematic and stylistic. In their analysis of the narrative structures and cinematic devices used in these videos, Venkatesh et al. ( 2020 ) highlight how the “Cinemas of Attraction” and “Pornography of violence” models were created with the intention of shocking as well as attracting viewers. Meanwhile, Sweeney et al. ( 2020 ) emphasised the positive relationship that exists between ISIS’s ability to govern territory and the complexity of the media that it produces, directly connecting the group’s perceived legitimacy and power to the calibre of its output.

ISIS media strategy analyses have changed in tandem with the group’s output. According to Kruglova ( 2020 ), ISIS propaganda utilised narrative advertising by appropriating marketing research, especially on social media platforms. This change is especially significant in light of the group’s deliberate use of stories to elicit strong feelings from the audience; these stories are made much more engaging when they are told in the language of the intended audience. Furthermore, an examination of how the group’s employment of cutting-edge tactics, such as drone images, improved the perceived power and legitimacy of the ISIS brand in these propaganda operations was presented (Archambault and Veilleux-Lepage, 2020 ).

A crucial element in the development of ISIS’s tactics is the interaction between the medium and message content. Toguslu ( 2019 ) examined the ways in which ISIS propaganda—particularly that seen in their magazines—constructs and presents storylines to support their ideology. ISIS’s media activities have seen a substantial metamorphosis with the conversion of these tales into video format and English translation (Fisher, 2015 ). These kinds of materials aim to appeal to Western audiences, frequently imitating Western media styles to give the propaganda a more recognisable sentiment (Qi and He, 2023 ).

The development of ISIS’s media strategy demonstrates a purposeful and strategic shift from local Arabic material to a more inclusive, wider media strategy that aims to interact with English-speaking viewers. This shift highlights how terrorist organisations are adaptable in the digital era and emphasises how crucial it is to thoroughly examine their media outputs to comprehend their influence and reach. Even if they are comprehensive, the earlier studies on ISIS’s media tactics have gaps that need for more research, especially when it comes to the topic of English-language video propaganda and its peculiar characteristics (Cottee and Cunliffe, 2020 ). Comprehending this evolutionary process is essential to crafting counter-narrative tactics and reducing the group’s impact on vulnerable English-speaking communities.

ISIS Propaganda’s thematic and tactical development

The tactical strategies and subject matter of ISIS’s English-language propaganda videos evolved significantly as their media apparatus grew (Winter, 2015 ). After analysing these themes, academics found recurring themes including victimisation, cruelty, utopianism, martyrdom, and apocalypse, all of which were intended to accomplish certain tactical goals (Johnston, 2022 ; Lakomy, 2020 ; Price and Mooney, 2022 ; Winter, 2015 ).

Early examination of the content of ISIS revealed a duality between images of horror and utopia—a dualism meant to arouse and terrify. The contrast of violent activities against the backdrop of an Islamic utopia promised was noticed by Venkatesh et al. ( 2020 ). ‘Cinematic charms’ combined with pictures of a dreamy caliphate lifestyle were intended to justify violence by painting it as a means of achieving a holy purpose.

In their investigation of the “Theatre of Terror,” Sweeney et al. ( 2020 ) and Qi ( 2020b ) contended that the staged violence in ISIS films was a deliberate strategy to represent authority and engender terror rather than being merely for spectacular. According to their study, those videos demonstrated the group’s ability to avenge its adversaries and were an example of a low-cost, high-effect tactic that maximised the symbolic value of violence and self-justification.

This topic was expanded upon by Kruglova ( 2020 ) to include the marketing-like techniques employed in these videos. She emphasised how skilled ISIS is at using social media as a platform to attract and radicalise potential recruits by creating narratives that play on emotions and identity. ISIS was able to connect with a larger audience by using English to tell a compelling tale that spoke to the needs and grievances of those who were remote from the fighting.

Milton ( 2020 ) looked into another facet of ISIS’s propaganda, which involved the deliberate manipulation of pictures. Using a dataset of 1700 ISIS images, he concluded that violent images, especially those of adversaries, greatly boosted attention. In addition to showcasing the group’s military might, the carefully chosen video also showed ISIS government and everyday life, appealing to viewers’ feeling of order and community.

With time, there was a noticeable change in the quality of ISIS propaganda—from widely circulated messages to more specialised information. In their audience perception research, Cottee and Cunliffe ( 2020 ) brought to light ISIS’s acute comprehension of its Western audience. ISIS’s English-language videos were an effective recruiting tool because they were crafted with tales that spoke to certain frustrations or ideological inclinations.

Qi and He ( 2023 ) has conducted an evaluation of English-language videos with an emphasis on their production and semantic attributes. These studies provide insight into how the videos’ production value and thematic distribution strategies have changed over time, despite criticism for their cursory presentation of the subject matter. This kind of study is essential to comprehending how, despite its military decline on the ground, ISIS managed to stay relevant and active in the digital sphere.

By concentrating on the performative element of ISIS narratives, Toguslu ( 2019 ) exposed the group’s deft use of religious texts to justify its crimes. ISIS attempted to provide its supporters with a spiritual purpose and a theological rationale for their atrocities by utilising passages from the Quran and Hadith in their propaganda.

ISIS propaganda’s thematic and tactical growth demonstrates a deliberate progression that aims to shock and persuade. By skilfully utilising English-language videos, the group was able to reach a wider audience and have a more profound effect, appealing to deeper themes of identity, religion, and political grievances than the surface-level appeal of violence. Therefore, analysing these advancements offers crucial insights into the workings of contemporary terrorist propaganda and serves as a foundation for developing potent counterstrategies.

Theoretical framework

In this study, we adopt the theoretical framework of content analysis as outlined by Braddock and Horgan ( 2016 ), which serves as a methodological cornerstone for dissecting the narratives utilised by extremist groups like ISIS. This framework builds upon the understanding that these groups use specific communicative strategies, including narratives imbued with extremist ideologies, values, and beliefs, to achieve strategic objectives and potentially foster radicalisation (Braddock and Horgan, 2016 ; Braddock and Dillard, 2016 ). The persuasive power of extremist media, and its role in radicalisation, has been acknowledged in various studies (Horgan, 2014 ; Jackson, 2007 ), highlighting the urgency of crafting counter-narratives based on a profound understanding of terrorist narratives.

Braddock and Horgan ( 2016 ) advocate for content analysis as a pivotal tool for this endeavour, enabling researchers to identify themes central to an extremist group’s ideology through a detailed examination of their media productions. Their proposed method includes both quantitative assessments of overt message characteristics and a more nuanced thematic analysis aimed at uncovering underlying values, views, and ideologies (Krippendorff, 2012 ). This approach facilitates the recognition of patterns within texts, serving as a crucial step in understanding extremist narratives.

Following the analytic procedures suggested by Braddock and Horgan ( 2016 , pp. 387–388), our study undertakes a systematic exploration of ISIS and other jihadi groups’ narratives. This involves multiple readings of the narratives to grasp their theme, style, and meaning; generating and consolidating codes that reflect the terrorist group’s ideology; sorting codes into overarching themes to identify higher-order concepts; and quantifying these thematic elements to ascertain the most prevalent themes. Such a structured analysis allows for a comprehensive understanding of the narratives, supported by a pilot-coding to ensure objectivity (Boyatzis and E, 1995 ; Patton, 2002 ).

Sampling rationale

Since there has been a substantial quantity of ISIS media production (Atwan, 2015 ; Colas, 2016a ; Cottee, 2015 ; Stern and Berger, 2016 ; Winkler et al. 2016 ; Winter, 2015 ; Zelin, 2015 ), there must be a feasible solution for sampling the data into a manageable corpus (Colas, 2016a ). Through the existing literature, it is observable that, first, studies have largely focused on the written texts, even though empirical studies on ISIS media output have clearly shown that ISIS relies more on visual propaganda than on written propaganda (Zelin, 2015 ). Second, the corpus might need to vary chronologically in terms of release dates to comprehend the evolution and changing dynamics of ISIS media in response to real-world events (Kuznar, 2017 ). Third, English is the second most commonly used language next to Arabic in ISIS propaganda and is the most commonly used foreign language (Fisher, 2015 ). Finally, in its intentional use of such a worldwide, accessible language, the official English-language video, from the organisation’s perspective, represents ISIS’s global ambitions and central strategies. This, in turn, sheds light upon ISIS’s worldview, how ISIS sees itself, and how ISIS wishes to be seen (Colas, 2016a ; Fisher, 2015 ). Ultimately, this fourfold rationale that leads to the sampling criteria sharpens data into a manageable size while remaining quantifiable and comparable with others’ studies of ISIS media production, leading to a more comprehensive, if counter-intuitive, study.

Sampling criteria

The 79 official English-language videos from ISIS were selected based on the following criteria: (1) timing: the video productions must have been released from April 2014 to July 2017, a timeframe that fully captures the Fall and the Liberation of Mosul (10 June 2014–10 July 2017), which symbolises the geographic controllability and territorial power of ISIS (Burke, 2017 ; Forrest, 2019 ; Gamal-Gabriel and Dunlop, 2017 ); (2) language: the video must either be narrated in English or have subtitles in English; (3) sources [for selecting those that represent official ISIS material]: the video productions must be from official ISIS media centres or from provincial-level centres accredited by official media centres. The criteria were implemented to collect English-narrated/subtitled videos released within the established time period that were produced/recognised by the official media centres at al-Hayat , al-Furqan , and al-I’tisam (Barr and Herfroy-Mischler, 2017 ; Zelin, 2015 ). To ensure that the English-language used in the videos released from provincial media centres was officially authorised by ISIS, as opposed to being a private translating effort from pro-ISIS supporters, the videos must have been promoted by the video series Selected 10 and Featured 3 , both of which represent or highlight periodic exaltations of exemplary provincial videos productions by al-Hayat media centre.

The criteria of timing in this study might be worth particular attention to further clarify. The designation of 2014 to 2017 as the peak period of ISIS activities is substantiated by a detailed examination of their territorial control and pivotal events, with a significant focus on the strategic city of Mosul. This era marks ISIS’s swift territorial expansion, reaching its apogee in 2014, characterised by the capture of Mosul, a major urban centre that symbolised their operational and administrative capabilities (Chulov, 2019 ; Curry, 2014b ; Gilsinan, 2014 ). The occupation of Mosul not only demonstrated ISIS’s military prowess but also established a geographical and ideological centre for the caliphate (al-Lami, 2019 ; Boffey and Jalabi, 2014 ; Dearden, 2014 ).

The subsequent decline of ISIS, leading to the liberation of Mosul in July 2017, underscores the importance of this timeframe. The battle for Mosul, which began in October 2016, represented a turning point in the international effort to dismantle ISIS’s territorial hold, highlighting a concerted counter-terrorism strategy that significantly diminished their control and influence (Barnard and Saad, 2017 ; Chulov, 2019 ; Damon et al. 2017 ; Phippen, 2017 )). The liberation of Mosul is widely regarded as a critical indicator of ISIS’s waning power, marking the end of their most significant territorial possession (Burke, 2017 ; Forrest, 2019 ).

Given these considerations, the period between 2014 and 2017 is selected as the focal point of this study, reflecting the zenith and subsequent reduction of ISIS’s territorial and operational command. This timeframe is crucial for understanding the dynamics of ISIS’s rise and fall, providing a comprehensive overview of their impact and the global response to their activities (Burke, 2017 ; Forrest, 2019 ).

Data collection

It is worth noting the distinction between the period of data collection (October 2015 to August 2017) and the video release date criteria (April 2014 to July 2017) for the sake of clarity. This study collected data between 1 October 2015, and 1 August 2017, leveraging Jihadology.net , a renowned repository for jihadi primary materials. During this period, MP4 files of ISIS videos were gathered from digital archives curated by scholars. The collection prioritised anonymity in sourcing to safeguard security while ensuring the authenticity and reliability of the data through cross-verification. Among a broad dataset of 1025 videos, 79 official English-language ISIS videos were chosen based on stringent criteria. Empirical evidence supports data collection via digital media, establishing them as promising research channels for the social sciences (Okereka et al. 2024 ).

Analytical procedure

Extremist organisations use varied communication strategies, including crafting narratives to embed ideologies and values (Braddock and Horgan, 2016 ). The effectiveness of these media in radicalisation is debated. Certain studies suggest narrative exposure can be persuasive, while others see no consistent link (Hong and Park, 2012 ; Peracchio and Meyers-Levy, 1997 ). However, it’s recognised that extremist narratives might potentially lead to radicalisation (Horgan, 2014 ).

Braddock and Horgan focus on developing counter-narratives to extremist ideologies. Understanding terrorist narratives is crucial for crafting effective counter-narratives and strategic communication to prevent radicalisation. These narratives, complex in ideological and emotional content, fulfil several roles: identity, justification for violence, and presenting a skewed reality that influences behaviour. Dissecting them is key to understanding their resonance and potential to foster extremist ideologies.

Development of the coding instrument

The initial phase of the analytical process was the development and enhancement of the coding scheme, which serves as the backbone for thematic analysis. This began with the construction of a provisional list of codes, which are essentially interpretative tags assigned to segments of the meaningful organisations within the videos. These segments to which the codes are applied could vary in length, thereby providing the flexibility to code discrete elements or broader sections of the narrative as necessary.

The analyst used their expertise and preliminary observations to form an initial list of themes present in the videos. This list was dynamically refined to align with established thematic frameworks in extremist propaganda research, ensuring a scholarly foundation for the coding instrument.

Relevant literature, including works by Winter ( 2015 ), Pelletier et al. ( 2016 ), and Gråtrud ( 2016 ), contributed established thematic codes to the analysis. This comparative approach refined the coding list, eliminating redundancy and ensuring a robust, comprehensive coding structure.

The analysis then shifted to a quantitative phase, systematically applying the refined codes to the video narratives. This quantification measured the frequency and prominence of themes, providing empirical insights into ISIS’s strategic messaging priorities during the study period. This approach moved the analysis beyond subjective interpretation towards a data-driven understanding of the thematic content in the ISIS videos.

Application of codes and content analysis

The analytical stage for examining ISIS videos involved a detailed and systematic coding process. The analyst analysed 79 videos, totalling 915 min, by breaking them down into one-minute increments, resulting in 915 distinct units for granular analysis.

Each minute unit was scrutinised using a set of thematic codes, identifying, and recording occurrences of specific themes, termed ‘segments.’ This led to the cataloguing of 799 segments of varying lengths. The prevalence of themes was assessed by calculating the cumulative duration of these segments, quantifying both the frequency and the temporal span of themes in the dataset. The total duration of all segments was 1707 units. Themes with a significance level of 0.06 or higher, roughly equivalent to 100 units or more, were considered substantially prevalent.

Additionally, the analysis explored the ‘asymmetric nature’ of ISIS media operations. This involved using the SKEW function, a statistical measure of distribution asymmetry, to understand the uneven thematic distribution over time, highlighting the dynamic nature of ISIS’s propaganda focus.

The outcome was a comprehensive thematic overview, showing both the frequency and variability of themes in ISIS’s video propaganda. The results were then visually represented in tables and graphs for clearer interpretation and discussion. At the conclusion of this rigorous process, the analyst had at their disposal a comprehensive list of themes, along with detailed insights into the frequency and changing patterns of these themes within ISIS’s video propaganda. The findings from this stage of analysis were then translated into tables and graphs, which facilitated a clear visual representation of the data, allowing for more accessible interpretation and discussion of the results.

Inter-coder reliability

The methodology for analysing ISIS video narratives involved enhancing reliability through an independent expert coder’s review, aligning with Schreier’s ( 2012 ) conflict resolution guidelines. The initial thematic categorisation has been scrutinised to ensure balanced and accurate coding. Braddock and Dillard’s ( 2016 ) methodical evaluation approach guided the determination of theme presence, with coder reviews forming the basis of final decisions.

To verify coding consistency, a pilot test aimed for at least 0.80 inter-coder reliability, following Cohen’s ( 1960 ) benchmark for high reliability. This standard reduces subjective bias, ensuring systematic and replicable coding. After pilot testing and discussions, two significant coding instrument revisions were made, leading to a final list of 26 thematic codes. This process established the credibility and rigour of the analysis.

The research includes three appendices in its online archive for transparency and replication. Appendix 1 details the data collection sources, Appendix 2 presents the final 26 thematic codes, and Appendix 3 contains example tables showing segment cataloguing and duration calculations. These appendices underpin the methodology and analysis, offering detailed insights into the study’s mechanics and coding process.

Thematic dissection of ISIS propaganda: enemy, religious, and emotive narratives

Figure 1 in the study categorises primary narratives in ISIS’s English-language videos into three groups: enemy, religious, and emotive, based on 26 thematic elements.

This figure presents the statistical compositions of the narratives identified in ISIS English-language videos, detailing the specific prevalence of each associated theme.

Figure 2 shows the ‘enemy’ narrative, comprising 40.83% of the content, focuses mainly on portraying ISIS at war (9.02%), captives confessing ‘sins’ (7.26%), and depicting the West as aggressive and oppressive (6.09%). It also highlights Western failures (4.22%) and alliances against ISIS (2.69%), with lesser emphasis on terror attacks (1.52%) and domestic vice and punishment (1.17%).

This figure illustrates the statistical thematic distribution of the enemy narrative within ISIS English-language videos, showing the relative frequency of each theme.

Figure 3 indicates the ‘religious’ narrative forms a substantial portion, led by themes of Sharia law enforcement (9.31%). Other key themes include incitement for jihad (6.39%), Islamic references (6.27%), and calls for emigration (4.16%). Lesser themes include allegiance to the leader (1.29%) and apocalypse (1.05%).

This figure shows the statistical thematic distribution of the religious narrative in ISIS English-language videos, highlighting the prevalence of each associated theme.

Figure 4 signifies the ‘emotive’ narrative, at 21.15%, highlights ‘happiness’ living within ISIS territory (5.74%) and victories at battlefield (4.98%). It also covers martyrdom and Muslim suffering (3.81%; 2.46%; 2.05%), with infrequent mentions of restoring honour of Islam (1.23%) or feelings of humiliation (0.88%).

This figure depicts the statistical thematic distribution of the emotive narrative in ISIS English-language videos, indicating the frequency of different themes.

Prevalent themes in ISIS propaganda: a detailed thematic breakdown

In the detailed analysis of ISIS English-language videos, ‘sharia and governance’ was the most prevalent theme, accounting for 9.31% of the content. This was followed by ‘combat’ (9.02%), ‘captives and confession’ (7.26%), ‘jihad’ (6.39%), ‘support from Quran and Sunnah’ (6.27%), and ‘Western malevolence’ (6.09%). Other notable themes included ‘happiness and wellbeing’ (5.74%), ‘strength and victory’ (4.98%), and ‘execution’ (4.45%).

Figure 5 in the study ranks these 26 themes based on their segment duration in the video corpus. Themes with a significance level of 0.06 or higher, such as ‘sharia and governance’, ‘combat’, and ‘captives and confession’, are highlighted, indicating their central role in ISIS propaganda.

This figure ranks the prevalence of 26 individual themes according to the total duration of segments in 79 ISIS English-language videos, presenting the relative importance of each theme.

Temporal shifts in ISIS narrative focus: analysing the stability of thematic content

The temporal analysis of ISIS English-language videos from 2014 to 2017 reveals fluctuating narrative themes, with six—‘vice and punishment’, ‘terror attack’, ‘apocalypse and prophecy’, ‘support from scholars’, ‘combat’, and ‘obedience to God’—showing significant variability. For example, ‘Terror attack’ was minimal until mid-2015, then became frequent, peaking in January 2016 and coinciding with portrayals of the November 2015 Paris attacks in ISIS videos.

However, as represented by Fig. 6 , themes like ‘jihad’, ‘West colluding with enemies’, ‘happiness and wellbeing’, ‘captives and confession’, ‘support from Quran and Sunnah’, and ‘sharia and governance’ remained stable and recurrent, reflecting ISIS’s core ideological appeals.

This figure displays the skewness in the distribution of the 26 themes in ISIS English-language videos. From left to right, the figure ranks the themes based on their instability, from the highest to the lowest.

Binary worldviews and theological legitimacy in ISIS propaganda

The prevalence of enemy narratives in ISIS propaganda creates a stark ‘us versus them’ dichotomy. Gerges ( 2009 , 2016 ) and Mahood and Rane ( 2016 ) discuss how ISIS portrays itself as the ‘good’—upholders of Sharia and the true path of jihad—while anyone opposing them is depicted as the ‘evil’ doomed to fail due to their disbelief. The narrative is given legitimacy by anchoring it in the historical and contemporary experiences of Muslims. Such binary opposition is a classic psychological warfare technique, fostering a collective identity among ISIS followers and justifying the group’s violent actions (Cantey, 2017 ; Gråtrud, 2016 ).

ISIS ideologues use theological language to assert that the group is on a divine mission, with violent jihad being the sole path to rectify the world (Mahood and Rane, 2016 ). They claim to be guided by a ‘prophetic methodology,’ deriving their understanding from the Quran and Sunnah, and present their jihadists as ‘lions of the caliphate’ and ‘warriors in upholding the rules of God’ (Gerges, 2016 ; Mahood and Rane, 2016 ). This religious narrative is fundamental in legitimising ISIS’s actions and in recruiting followers by weaving theological justifications into its narrative framework.

The strategic use of emotive content in ISIS propaganda is discussed as being less prevalent compared to the enemy and religious narratives. This strategic choice could indicate a focus on ideological and combative aspects, particularly in content aimed at Western audiences (Colas, 2016b ; Spier, 2018 ). However, when emotive content is utilised, it is designed to resonate with feelings of injustice and discrimination, appealing to a sense of identity and grievance (Mahood and Rane, 2016 ; Olidort and McCants, 2015 ).

The discussion further situates ISIS’s propaganda strategy within the broader debate on the role of Islam in its ideology, referencing Graeme Wood’s influential essay (Wood, 2015 ) that contends ISIS is intrinsically Islamic, sparking a debate on the relationship between Islamism and terrorism (Cottee, 2017a ). This debate pits those who view ISIS as representing true Islam (Ali et al. 2020 ) against those who vehemently disagree (Coles, 2015 ; Hasan and Mehdi, 2015 ; Tharoor, 2016 ). This ongoing argument examines whether the violence enacted in the name of religion is inherently religious or if it is politically motivated and sometimes can be secular in character.

Wood’s essay challenges the notion that ISIS’s violence is purely psychopathic, suggesting instead that it is rooted in early medieval Islamic ideology (Remnick, 2014 ). Meanwhile, critics like Coles ( 2015 ) argue that ISIS’s interpretation of Islam is a deviation, and others like Coolsaet ( 2016 ) and Roy ( 2016 ) attribute the violence to political rather than religious motivations.

The empirical evidence from this study, which shows a significant emphasis on religious narrative in ISIS’s English-language videos, adds a critical dimension to this debate. The frequent recurrence of themes such as sharia law and violent jihad in official media suggests that ISIS’s theological underpinnings are significant and that the group’s ideological foundations are vital to understanding its allure and the motivations of its adherents (Makdisi and John, 1985 ; Mutahhari, 2014 ).

The integration of empirical findings with the broader discourse on ISIS’s propaganda strategies provides a more comprehensive understanding of how the group uses enemy and religious narratives to construct a worldview that legitimises its actions, while also participating in a broader debate about the role of religion in political violence. This complex narrative strategy serves multiple functions within the group’s ideological battle, solidifying its identity, justifying its violent actions, and recruiting followers.

Strategic emphasis and media diversity in ISIS’s propaganda narrative

The findings in ISIS’s English-language videos, as identified in the study, affirm the thematic elements highlighted in other research on ISIS’s propaganda (Gråtrud and Henrik, 2016 ; Kuznar, 2017 ). Emotive language, moderately employed in ISIS’s videos, is a common thread throughout jihadist propaganda, which is also prevalent among groups like the Taliban, Al-Qaeda, and its affiliates AQIM and AQAP. However, ISIS’s unique emphasis on certain themes distinguishes its propaganda from others (Abrahms et al. 2017 ; Gendron and Angela, 2016 ; Salem et al. 2008 ).

The study supports Kuznar’s observation that the thematic elements of ISIS propaganda are present in other jihadi propaganda but emphasises that ISIS has a distinctive approach to these themes. While the general message across ISIS’s various media formats—magazines, leaders’ speeches, public statements, and Nasheeds —remains coherent, the intensity with which certain themes are highlighted varies. ISIS’s English-language videos, in particular, consistently emphasise religious and enemy narratives in line with ‘Dabiq,’ the group’s official English-language magazine, while emotive narratives are less pronounced (Colas, 2016a ).

In contrast, the leaders’ speeches seem to focus more on emotion-provoking themes, suggesting a strategic use of emotional appeal to strengthen the group’s core narratives, with religious and enemy narratives taking a secondary role (Spier, 2018 ). Gråtrud’s analysis suggests that the effectiveness of ISIS’s media, such as Nasheeds , could be attributed to its emphasis on a select number of broadly appealing themes. This targeted approach likely extends beyond Nasheeds to other media productions, indicating a nuanced strategy to engage with various target audiences effectively.

When we consider these findings alongside the comparative analysis of other extremist groups, it becomes evident that ISIS has carved out a unique space in jihadist media strategy. While the shared use of recruitment, indoctrination, enemy construction, religious justification, and calls to action are common jihadist media narratives, ISIS’s distinct approach lies in its media production quality, modern communication tool utilisation, apocalyptic messaging, and tailored language use.

The thematic emphasis and diversity in ISIS’s media productions, juxtaposed with the broader landscape of jihadist propaganda, underscore the group’s sophisticated media strategy. ISIS’s ability to maintain thematic consistency across different media forms while varying the intensity of certain themes reveals an intention to optimise the impact of its messaging. This adaptability and tailored emphasis not only differentiate ISIS’s propaganda from other groups but also potentially enhance its effectiveness in recruitment and ideological dissemination.

By understanding these nuances, counter-terrorism efforts can be better tailored to address the specific strategies employed by ISIS and other extremist groups, acknowledging the shared tactics while targeting the unique aspects of each group’s propaganda approach.

Adaptive themes and consistent ideology in ISIS propaganda

The research of Pelletier et al. ( 2016 ) aligns with the findings from this study, suggesting that jihadist groups like AQAP and ISIS exhibit major thematic shifts in response to real-world events while maintaining a consistent overarching thematic structure. In the case of ISIS, the primary and most fundamental themes—those at the core of ISIS’s ideology—tend to remain stable over time. Conversely, the less recurrent themes display more dynamism, often aligning with specific geopolitical or operational developments that ISIS encounters.

For example, the theme of ‘Terror Attack’ in ISIS videos became more prominent following high-profile attacks that ISIS claimed responsibility for, such as the Paris attacks in November 2015. Foster ( 2014 ) notes the depiction of the perpetrators as heroes in ISIS’s narrative, which marked a peak in the terror attack theme’s prominence. The ‘Apocalypse and Prophecy’ theme’s activity aligns with the group’s control over the town of Dabiq, believed to be a prophesied battlefield, and its eventual loss of the town in 2016 (Withnall, 2016 ). These shifts illustrate how ISIS’s media strategy is interwoven with its operational successes and setbacks, using thematic content to reflect and amplify its real-world narrative.

Despite the responsiveness to events, the more recurrent themes, particularly those propagating violent jihad and Islamic law—pillars of the ISIS ideology—remained consistent. These themes are critical for maintaining a steady ideological message for recruitment, indoctrination, and asserting the group’s identity.

However, the theme of ‘Combat’ presents an interesting case. Although it is one of the most recurrent themes, it exhibited significant instability. The two major peaks in this theme’s prominence not only reflect specific events but also disproportionately affect the theme’s overall statistical stability. This instability may serve a strategic purpose, as Zelin ( 2015 ) indicates, potentially highlighting the asymmetric nature of ISIS media operations. The aim could be to project an image of ongoing struggle and resilience despite real-world setbacks, thereby maintaining morale and commitment among its followers.

The skewness in the distribution of themes across ISIS’s videos supports the notion of an asymmetric media strategy. This asymmetry is not arbitrary but appears to be a calculated response to real-world events. Themes that exhibit significant shifts correspond to specific incidents, underscoring ISIS’s intent to manipulate media narratives in line with operational objectives and challenges.

Counter-terrorism strategies implications

Counter-narrative campaigns are crucial in combating the binary enemy narratives that ISIS propagates. ISIS frames the world in terms of black and white, good and evil, believers and non-believers. To counter this, it’s important to develop narratives that showcase the complexity and diversity of Muslim identities and the peaceful, pluralistic nature of Islamic teachings. Educational initiatives can play a pivotal role in this area, as they can foster a more nuanced understanding of Islam that goes beyond the simplistic and extremist interpretations offered by ISIS (Gerges, 2016 ; Mahood and Rane, 2016 ).

By highlighting the rich tapestry of Islamic scholarship and the diversity of interpretations that have coexisted within Islamic history, these campaigns can undermine the theological foundations upon which ISIS builds its legitimacy. It is also essential to promote voices within the Muslim community that speak to the religion’s core messages of peace and compassion, drawing on both historical and contemporary sources of Islamic thought (Makdisi and John 1985 ).

ISIS has demonstrated an ability to adapt its messages in response to changing circumstances, be they losses on the battlefield or shifts in geopolitical alliances. A successful counter-terrorism approach must be equally flexible, employing real-time intelligence to detect and respond to changes in ISIS’s narrative strategies (Pelletier et al. 2016 ). Developing predictive models based on this intelligence can help anticipate the group’s future shifts in narrative and allow counter-terrorism efforts to stay one step ahead.

At its core, radicalisation often stems from socio-political factors such as alienation, discrimination, and injustice—elements that ISIS exploits to recruit and radicalise individuals. Counter-terrorism efforts must, therefore, also focus on the root causes of radicalisation. This involves creating inclusive policies that address unemployment, provide educational opportunities, and promote social cohesion within marginalised communities (Coolsaet, 2016 ; Roy, 2016 ). Programs that target these areas can reduce the susceptibility of individuals to extremist ideologies by improving their socioeconomic conditions and fostering a sense of belonging within their societies.

In summary, ISIS has created a complex global propaganda apparatus comprising comprehensive narrative themes that span adversarial, theological, and emotional artefacts. The primary conclusion is fourfold: first, within the scope of ISIS propaganda, the extent to which ISIS emphasises certain themes is distinctive from that of other media releases; second, the two most important narratives for ISIS propagandists are the enemy and religious narratives, which reflect a binary worldview of ISIS ideology. ISIS represents the ‘good’ whereas those who oppose ISIS are its enemies and the ‘evil’; third, the ratio of the religious narrative in ISIS English-language videos elucidates one of the hottest debates regarding ISIS’s Islamic nature by supporting and reinforcing the arguments that the religious artefacts of ISIS are important and cannot be neglected if the narrative and underlying ideologies are to be understood (Pelletier et al. 2016 ; Wood, 2015 ); finally, the most fundamental themes promoted by ISIS remain consistent over time whereas the least recurrent themes are more dynamic and might shift significantly in response to a series of real-world events that ISIS faces on the ground.

However, this study is merely the first step. There have been some limitations and many other research trajectories of which future studies might be aware. Most apparently, due to the resource restraints and unstable circumstances in the region, it has been difficult to claim the complete collection of ISIS-produced videos has been archived in this field. Second, this study does not account for the audience perception of ISIS videos – it demands a separate study, although some research in this domain is already underway, and such work is beneficial for our understanding of ISIS propaganda (Cottee, 2017b ). Third, the effectiveness of ISIS narratives might not only depend on what the narrative contains but also on the style in which the content is vividly presented (O’Keefe, 1997 ). Further studies could provide another perspective by, for instance, examining cinematography or semiotics. Finally, the use of content analysis might restrain our understanding of the sophistication of ISIS rhetoric and reasoning devices; further research could offer an interpretation of ISIS English videos beyond locution and thematic analysis of extremists’ messages. For example, ISIS’s use of language as a means to achieve objectives through words and deeds.

Nevertheless, contributing significantly to the discourse on jihadist media strategy, this study undertakes a detailed examination of narrative motifs found in the official English-language videos of ISIS. It renders an original, comprehensive content analysis of ISIS’s propaganda, and facilitates a highly inclusive range of thematic elements that are also applicable to other extremists’ visual texts. Dissecting these videos’ narrative constructs enables a deeper counterpoint to the narratives that have found traction in jihadist online propaganda. By doing so, insights into the group’s strategic narrative constructions and worldviews are gleaned. These insights are crucial for demystifying ISIS: discerning its self-image, presentation style, and desired perception among international audiences. Moreover, this research augments existing studies on ISIS’s global media reach by providing an exhaustive analysis of its official English-language videos and adopts a dynamic perspective on the group’s media offerings, tracking how ISIS tailored its propagandist responses to various global events during the important period. The methodological rigour applied herein lays the groundwork for future explorations into the propaganda of other terrorist organisations. Exposure to jihadist propaganda might not be a sole radicalising force; rather, it is the confluence of ideological currents within broader social, political, and cultural frameworks that is critical (Winter, 2015). ISIS’s media arsenal, encompassing literature, videos, social platforms, and discussion forums, is curated to mirror these undercurrents. The analytical method developed through this study’s examination of ISIS’s videos paves the way for scrutinising a broader array of extremist communications.

Data availability

The dataset generated during and/or analysed during the current study is submitted as a supplementary file and can also be obtained from the corresponding author upon reasonable request.

Abrahms M, Beauchamp N, Mroszczyk J (2017) What terrorist leaders want: a content analysis of terrorist propaganda videos. Stud Confl Terror 40(11):899–916. https://doi.org/10.1080/1057610X.2016.1248666

Article   Google Scholar  

al-Lami, M (2019). Analysis: will disillusionment with the IS ‘caliphate’ prevent its revival? – BBC Monitoring. https://monitoring.bbc.co.uk/product/c200ntjp

Al-Rawi A (2018) Video games, terrorism, and ISIS’s Jihad 3.0. Terror Polit Viol 30(4):740–760. https://doi.org/10.1080/09546553.2016.1207633

Ali H, Khan HA, Pecht MG (2020) Evaluation of Li-based battery current, voltage, and temperature profiles for in-service mobile phones. IEEE Access 8:73665–73676. https://doi.org/10.1109/access.2020.2988728

Archambault E, Veilleux-Lepage Y (2020) Drone imagery in Islamic state propaganda: flying like a state. Int Aff 96(4):955. https://doi.org/10.1093/ia/iiaa014

Atwan, A-B (2015). Islamic state: the digital caliphate (New edition edition ed.). Saqi Books. https://www.amazon.co.uk/Islamic-State-Caliphate-Abdel-Bari-Atwan/dp/0863561349/ref=sr_1_sc_1?ie=UTF8&qid=1474825938&sr=8-1-spell&keywords=the+digital+canliphate

Barnard, A, & Saad, H (2017). Raqqa, ISIS ‘capital,’ is captured, U.S.-backed forces say. In: The New York Times. https://www.nytimes.com/2017/10/17/world/middleeast/isis-syria-raqqa.html

Barr A, Herfroy-Mischler A (2017) ISIL’s execution videos: audience segmentation and terrorist communication in the digital age. Stud Confl Terror 0(0):1–22. https://doi.org/10.1080/1057610X.2017.1361282

Boffey, D, & Jalabi, R (2014). ISIS video claims to show beheading of British hostage David Haines. In: The Guardian. http://www.theguardian.com/world/2014/sep/13/isis-video-david-haines-beheading , https://www.theguardian.com/world/2014/sep/13/isis-video-david-haines-beheading

Boyatzis, & E, R. (1995). Transforming qualitative information: thematic analysis and code development. In: Sage Publications, Inc

Braddock K, Dillard JP (2016) Meta-analytic evidence for the persuasive effect of narratives on beliefs, attitudes, intentions, and behaviors. Commun Monogr 83(4):446–467. https://doi.org/10.1080/03637751.2015.1128555

Braddock K, Horgan J (2016) Towards a guide for constructing and disseminating counternarratives to reduce support for terrorism. Stud Confl Terror 39(5):381–404. https://doi.org/10.1080/1057610X.2015.1116277

Burke, J (2017). Rise and fall of Isis: its dream of a caliphate is over, so what now? In: The Observer. http://www.theguardian.com/world/2017/oct/21/isis-caliphate-islamic-state-raqqa-iraq-islamist

Cantey S (2017) Beyond the Pale? Exploring prospects for negotiations with Al Qaeda and the Islamic State. Stud Confl Terror 0(0):1–19. https://doi.org/10.1080/1057610X.2017.1348096

Chulov, M (2019). The rise and fall of the Isis ‘caliphate’. In: The Guardian. https://www.theguardian.com/world/2019/mar/23/the-rise-and-fall-of-the-isis-caliphate

Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20(1):37–46. http://journals.sagepub.com/doi/abs/10.1177/001316446002000104files/1191/001316446002000104.html

Colas B (2016a) What does Dabiq do? ISIS hermeneutics and organizational fractures within dabiq magazine. Stud Confl Terror 0:1–18

Google Scholar  

Colas B (2016b) What does Dabiq do? ISIS hermeneutics and organizational fractures within Dabiq magazine. Stud Confl Terror 0(0):1–18. https://doi.org/10.1080/1057610X.2016.1184062

Colas B (2017) What does Dabiq do? ISIS hermeneutics and organizational fractures within Dabiq magazine. Stud Confl Terror 40(3):173–190. https://doi.org/10.1080/1057610x.2016.1184062

Coles I (2015) Kurdish forces seize Iraq’s Sinjar town from Islamic State. In: Reuters. https://www.reuters.com/article/us-mideast-crisis-iraq-sinjar/kurds-expect-to-enter-and-clear-sinjar-soon-idUSKCN0T10AL20151113 , https://www.reuters.com/article/us-mideast-crisis-iraq-sinjar/kurdish-forces-seize-iraqs-sinjar-town-from-islamic-state-idUSKCN0T10AL20151113

Coolsaet, R (2016). Facing the fourth foreign fighters wave. In: Egmont Paper 81 . http://www.egmontinstitute.be/wp-content/uploads/2016/02/egmont.papers.81_online-versie.pdf

Cottee, S (2015). Why it’s so hard to stop ISIS propaganda. In: The Atlantic. http://www.theatlantic.com/international/archive/2015/03/why-its-so-hard-to-stop-isis-propaganda/386216/

Cottee S (2017a) “What ISIS really wants” revisited: religion matters in jihadist violence, but how? Stud Confl Terror 40(6):439–454. https://doi.org/10.1080/1057610x.2016.1221258

Cottee S (2017b) “What ISIS really wants” revisited: religion matters in jihadist violence, but how? Stud Confl Terror 40(6):439–454. https://doi.org/10.1080/1057610X.2016.1221258

Cottee S, Cunliffe J (2020) Watching ISIS: how young adults engage with official english-language ISIS videos. Stud Confl Terror 43(3):183–207. https://doi.org/10.1080/1057610x.2018.1444955

Curry, C (2014a). A simple, useful guide to the conflict in Iraq. ABC News . https://abcnews.go.com/International/simple-guide-understanding-conflict-iraq/story?id=24113794

Curry, C (2014b). This is the Militant Islamic group taking over Iraq. ABC News . https://abcnews.go.com/International/militant-islamic-group-taking-iraq/story?id=24122981

Damon, A, Ghazi, B, & Laura, S-S (2017). Raqqa: SDF declare ‘total liberation’ of ISIS stronghold. In: CNN. https://www.cnn.com/2017/10/20/middleeast/raqqa-syria-isis-total-liberation/index.html

Dearden, L (2014). Steven Sotloff ‘beheading’: mother’s tragic appeal to Isis to ‘follow. In: The Independent. http://www.independent.co.uk/news/world/middle-east/steven-sotloff-beheading-mothers-tragic-appeal-to-isis-to-follow-mohameds-example-and-show-mercy-on-9707645.html

Dearden, L (2015). Alan Henning’s daughter found out he was dead when she saw his body on Instagram. In: The Independent. http://www.independent.co.uk/news/uk/home-news/alan-hennings-daughter-found-out-isis-had-beheaded-father-when-she-saw-photo-of-body-on-instagram-a6733326.html

Fisher, A (2015). How Jihadist networks maintain a persistent online presence. In: Perspect Terror 9. http://terrorismanalysts.com/pt/index.php/pot/article/view/426 , http://terrorismanalysts.com/pt/index.php/pot/article/view/426/html

Forrest, A (2019). British ‘spy’ known as Osama bin Bieber executed by Isis after joining group. In: The Independent. https://www.independent.co.uk/news/uk/home-news/isis-british-spy-osama-bin-bieber-executed-death-mohammed-ismail-a8848171.html

Foster, P (2014). Jihadists from around the world flock to fight with Isil: UN. https://www.telegraph.co.uk/news/worldnews/islamic-state/11200701/Jihadists-from-around-the-world-flock-to-fight-with-Isil-UN.html

Gamal-Gabriel, T, & Dunlop, WG (2017). Iraq’s PM hails victory over ‘brutality and terrorism’ in Mosul. http://www.timesofisrael.com/iraqs-pm-hails-victory-over-brutality-and-terrorism-in-mosul/ , https://www.timesofisrael.com/iraqs-pm-hails-victory-over-brutality-and-terrorism-in-mosul/

Gendron, Angela (2016) The Call to Jihad: charismatic preachers and the internet. Stud Confl Terror 40(1):44–61

Gerges, F (2019). Opinion | The Islamic state has not been defeated—the New York Times. https://www.nytimes.com/2019/03/23/opinion/isis-defeated.html

Gerges, FA (2009). The far enemy: why jihad went global (2 edition ed.). Cambridge University Press. https://www.amazon.co.uk/Far-Enemy-Jihad-Went-Global/dp/0521737435

Gerges, FA (2016). ISIS: a history. Princeton University Press

Gilsinan, K (2014). The many ways to map the Islamic ‘State’. In: The Atlantic. https://www.theatlantic.com/international/archive/2014/08/the-many-ways-to-map-the-islamic-state/379196/

Gråtrud, Henrik (2016) Islamic state nasheeds as messaging tools. Stud Confl Terror 39(12):1050–1070

Gråtrud H (2016) Islamic state nasheeds as messaging tools. Stud Confl Terror 39(12):1050–1070. https://doi.org/10.1080/1057610X.2016.1159429

Hasan M (2015) Mehdi Hasan: how islamic is Islamic state?

Hong S, Park HS (2012) Computer-mediated Persuasion in Online Reviews: Statistical Versus Narrative Evidence. Comput Hum Behav 28(3):906–919. https://doi.org/10.1016/j.chb.2011.12.011

Horgan, J (2014). The psychology of terrorism (2 edition ed.). Routledge. https://www.amazon.co.uk/Psychology-Terrorism-Political-Violence/dp/0415698022

Jackson R (2007) Constructing enemies: ‘Islamic terrorism’ in political and academic discourse [Review]. Gov Oppos 42(3):394–426. https://doi.org/10.1111/j.1477-7053.2007.00229.x

Johnston N (2022) Selling terror: a multidimensional analysis of the Islamic State’s recruitment propaganda. Aust J Int Aff 76(2):194–218. https://doi.org/10.1080/10357718.2021.1970714

Krippendorff, K (2012). Content analysis: an introduction to its methodology (3 edition ed.). Sage publications, Inc. https://www.amazon.co.uk/Content-Analysis-Introduction-Its-Methodology/dp/1412983150/ref=sr_1_1?s=books&ie=UTF8&qid=1548460171&sr=1-1&keywords=Content+Analysis%3A+An+Introduction+to+Its+Methodology

Kruglova A (2020) “I will tell you a story about Jihad”: ISIS’s propaganda and narrative advertising. Stud Confl Terror 44(2):115–137. https://doi.org/10.1080/1057610x.2020.1799519

Kuznar L (2017) The stability of the Islamic State (IS) narrative: implications for the future. Dyn Asymmetric Confl 10(1):40–53

Lakomy M (2020) “One of the two good outcomes”: turning defeats into victories in the Islamic State’s flagship magazine Rumiyah. Terror Polit Viol 32(8):1712–1730. <Go to ISI>://WOS:000589862200005

Lakomy M (2021a) Mapping the online presence and activities of the Islamic State’s unofficial propaganda cell: Ahlut-Tawhid publications. Secur J 34(2):358–384. <Go to ISI>://WOS:000516044500001

Lakomy M (2021b) Recruitment and incitement to violence in the Islamic state’s online propaganda: comparative analysis of Dabiq and Rumiyah. Stud Confl Terror 44(7):565–580. <Go to ISI>://WOS:000648296800002

Mahood S, Rane H (2016) Islamist narratives in ISIS recruitment propaganda. J Int Commun 0(0):1–21. https://doi.org/10.1080/13216597.2016.1263231

Article   ADS   Google Scholar  

Makdisi J (1985) Legal logic and equity in Islamic law. Am J Comp Law 33:63–92. https://doi.org/10.2307/840118

Malik, S (2014a). French Isis fighters filmed burning passports and calling for terror at home. In: The Guardian. https://www.theguardian.com/world/2014/nov/20/french-isis-fighters-filmed-burning-passports-calling-for-terrorfiles/1657/french-isis-fighters-filmed-burning-passports-calling-for-terror.html

Malik, S (2014b). Isis video appears to show hostage Peter Kassig has been killed. In: The Guardian. http://www.theguardian.com/world/2014/nov/16/isis-beheads-peter-kassig-reports

Milton D (2020) The ISIS reader: milestone texts of the Islamic state movement. Terror Polit Viol 32(5):1121–1122. https://doi.org/10.1080/09546553.2020.1776989

Mutahhari AM (2014) Jurisprudence and its principles. CreateSpace independent publishing platform. https://www.amazon.co.uk/Jurisprudence-Principles-Ayatullah-Murtadha-Mutahhari/dp/1502538695/ref=sr_1_1?ie=UTF8&qid=1513251433&sr=8-1&keywords=Jurisprudence+and+Its+Principles

O’Keefe DJ (1997) Standpoint explicitness and persuasive effect: a meta-analytic review of the effects of varying conclusion articulation in persuasive messages. Argum Advocacy 34:1–12. https://www.scholars.northwestern.edu/en/publications/standpoint-explicitness-and-persuasive-effect-a-meta-analytic-rev

Okereka OP, Orhero AE, Okolie UC (2024) Digital media and data collection in social and management sciences research in Nigeria. Ianna J Interdiscip Stud 6(1):76–89. https://iannajournalofinterdisciplinarystudies.com/index.php/1/article/view/176

Olidort, J, & McCants, W (2015). Is quietist Salafism the antidote to ISIS? In: Brookings. https://www.brookings.edu/blog/markaz/2015/03/13/is-quietist-salafism-the-antidote-to-isis/

Patton, MQ (2002). Qualitative research & evaluation methods (Third edition ed.). SAGE Publications, Inc. https://www.amazon.co.uk/Qualitative-Research-Evaluation-Methods-Michael/dp/0761919716

Pelletier IR, Lundmark L, Gardner R, Ligon GS, Kilinc R (2016) Why ISIS’s message resonates: leveraging Islam, sociopolitical catalysts, and adaptive messaging. Stud Confl Terror 39(10):871–899. https://doi.org/10.1080/1057610X.2016.1139373

Peracchio LA, Meyers-Levy J (1997) Evaluating persuasion-enhancing techniques from a resource-matching perspective. J Consum Res 24(2):178–191. https://doi.org/10.1086/209503

Phippen, JW (2017). Iraqi forces take Mosul. The Atlantic. https://www.theatlantic.com/news/archive/2017/07/iraqi-forces-take-mosul/533055/

Price G, Mooney T (2022) Utopia, war, and justice: the discursive construction of the state in ISIS’ political communications. J Lang Polit 21(5):675–696. https://doi.org/10.1075/jlp.21016.moo

Qi Y (2020a) Illuminating terror: content analysis of official ISIS English-language videos from 2014 to 2017. Behav Sci Terrorism Polit Aggres. 14(3):187–215. https://doi.org/10.1080/19434472.2020.1841266

Qi Y (2020b) The language of terror: exploring speech acts in official English-language ISIS videos, 2014-2017. Small Wars Insur 31(6):1196–1241. https://doi.org/10.1080/09592318.2020.1775055

Qi YB, He J (2023) Toward a Skinnerian interpretivist methodological approach for terrorist propaganda. Soc Sci Q 104(5):1020–1033. https://doi.org/10.1111/ssqu.13296

Remnick, D (2014). Going the distance: on and off the road with Barack Obama. In: The New Yorker. http://www.newyorker.com/magazine/2014/01/27/going-the-distance-david-remnick

Roth, A, & Sauer, P (2024). Four suspects in Moscow concert hall terror attack appear in court. In: The Guardian. https://www.theguardian.com/world/2024/mar/24/new-islamic-state-videos-back-claim-it-carried-out-moscow-concert-hall-attack

Roy, O (2016). France’s Oedipal Islamist complex. In: Foreign Policy. http://foreignpolicy.com/2016/01/07/frances-oedipal-islamist-complex-charlie-hebdo-islamic-state-isis/

Salem A, Reid E, Chen H (2008) Multimedia content coding and analysis: unraveling the content of Jihadi extremist groups’ Videos. Stud Confl Terror 31(7):605–626

Schmitt, E (2024). ISIS affiliate linked to moscow attack has global ambitions. In: The New York Times. https://www.nytimes.com/2024/03/25/us/politics/moscow-attack-isis.html , https://www.nytimes.com/2024/03/25/us/politics/moscow-attack-isis.html?auth=login-google1tap&login=google1tap

Schreier, M (2012). Qualitative Content Analysis in Practice (1 edition ed.). Sage Publications Ltd. https://www.amazon.co.uk/Qualitative-Content-Analysis-Practice-Schreier/dp/1849205930

Spier TE (2018) Extremist propaganda and Qur’anic scripture: a “radical’ corpus-based study of the Dabiq. Discourse Soc 29(5):553–567. https://doi.org/10.1177/0957926518770265

Stern, J, & Berger, JM (2016). ISIS: The State of Terror (01 edition ed.). William Collins. https://www.amazon.co.uk/ISIS-State-Terror-Jessica-Stern/dp/000812096X/ref=pd_bxgy_14_img_3?_encoding=UTF8&psc=1&refRID=ZVWYN5QPN5NKCE33GH0J

Sweeney, MM, Perliger, A, & Pedahzur, A (2020). Reconstructing the theater of terror. Small Wars Insur. https://doi.org/10.1080/09592318.2020.1794176

Tharoor, I (2016). Belgium’s big problem with radical Islam. In: The Washington Post. https://www.washingtonpost.com/news/worldviews/wp/2016/03/22/belgiums-big-problem-with-radical-islam/

Toguslu E (2019) Caliphate, Hijrah and martyrdom as performative narrative in ISIS Dabiq magazine. Politics Relig Ideol 20(1):94–120. https://doi.org/10.1080/21567689.2018.1554480

Venkatesh V, Podoshen JS, Wallin J, Rabah J, Glass D (2020) Promoting extreme violence: visual and narrative analysis of select ultraviolent terror propaganda videos produced by the Islamic State of Iraq and Syria (ISIS) in 2015 and 2016. Terror Polit Viol 32(8):1753–1775. https://doi.org/10.1080/09546553.2018.1516209

Votel, JL, Bembenek, C, Spencer, A, Hans, C, & Mouton, J (2017). Virtual caliphate: defeating ISIL on the physical battlefield is not enough. https://www.cnas.org/publications/reports/virtual-caliphate

Winkler CK, Damanhoury KE, Dicker A, Lemieux AF (2016) The medium is terrorism: transformation of the about to die trope in Dabiq. Terror Polit Viol 0(0):1–20. https://doi.org/10.1080/09546553.2016.1211526

Winter, C (2015). The virtual ‘Caliphate’: understanding Islamic State’s propaganda strategy. Quilliam

Winter C (2018) Apocalypse, later: a longitudinal study of the Islamic State brand. Crit Stud Media Commun 35(1):103–121. https://doi.org/10.1080/15295036.2017.1393094

Withnall, A (2016). Isis loses ‘prophesied’ town of Dabiq to Syrian rebels in short battle. In: The Independent. http://www.independent.co.uk/news/world/middle-east/isis-dabiq-loses-apocalyptic-prophesy-town-of-dabiq-to-syria-rebels-short-battle-a7363931.html

Wood, G (2015). What ISIS really wants. In: The Atlantic. http://www.theatlantic.com/magazine/archive/2015/03/what-isis-really-wants/384980/

Zelin, A (2015). Picture or it didn’t happen: a snapshot of the Islamic state’s official media output. In: Perspect Terror, 9. http://www.terrorismanalysts.com/pt/index.php/pot/article/view/445 , http://www.terrorismanalysts.com/pt/index.php/pot/article/view/445/html

Download references

Acknowledgements

The research is funded by the “Fundamental Research Funds for the Central Universities” (2232024E-10).

Author information

Authors and affiliations.

College of Humanities, Donghua University, Shanghai, 200051, China

You can also search for this author in PubMed   Google Scholar

Contributions

The author contributes solely to this work.

Corresponding author

Correspondence to Yuanbo Qi .

Ethics declarations

Competing interests.

The author declares no competing interests.

Ethical approval

This article does not contain any studies with human participants performed by the author.

Informed consent

Ethical approval was not required as the study did not involve human participants.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Extraction of segments, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Qi, Y. Propaganda in focus: decoding the media strategy of ISIS. Humanit Soc Sci Commun 11 , 1123 (2024). https://doi.org/10.1057/s41599-024-03608-y

Download citation

Received : 30 November 2023

Accepted : 15 August 2024

Published : 02 September 2024

DOI : https://doi.org/10.1057/s41599-024-03608-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies