in depth interview in research methodology

Chapter 11. Interviewing

Introduction.

Interviewing people is at the heart of qualitative research. It is not merely a way to collect data but an intrinsically rewarding activity—an interaction between two people that holds the potential for greater understanding and interpersonal development. Unlike many of our daily interactions with others that are fairly shallow and mundane, sitting down with a person for an hour or two and really listening to what they have to say is a profound and deep enterprise, one that can provide not only “data” for you, the interviewer, but also self-understanding and a feeling of being heard for the interviewee. I always approach interviewing with a deep appreciation for the opportunity it gives me to understand how other people experience the world. That said, there is not one kind of interview but many, and some of these are shallower than others. This chapter will provide you with an overview of interview techniques but with a special focus on the in-depth semistructured interview guide approach, which is the approach most widely used in social science research.

An interview can be variously defined as “a conversation with a purpose” ( Lune and Berg 2018 ) and an attempt to understand the world from the point of view of the person being interviewed: “to unfold the meaning of peoples’ experiences, to uncover their lived world prior to scientific explanations” ( Kvale 2007 ). It is a form of active listening in which the interviewer steers the conversation to subjects and topics of interest to their research but also manages to leave enough space for those interviewed to say surprising things. Achieving that balance is a tricky thing, which is why most practitioners believe interviewing is both an art and a science. In my experience as a teacher, there are some students who are “natural” interviewers (often they are introverts), but anyone can learn to conduct interviews, and everyone, even those of us who have been doing this for years, can improve their interviewing skills. This might be a good time to highlight the fact that the interview is a product between interviewer and interviewee and that this product is only as good as the rapport established between the two participants. Active listening is the key to establishing this necessary rapport.

Patton ( 2002 ) makes the argument that we use interviews because there are certain things that are not observable. In particular, “we cannot observe feelings, thoughts, and intentions. We cannot observe behaviors that took place at some previous point in time. We cannot observe situations that preclude the presence of an observer. We cannot observe how people have organized the world and the meanings they attach to what goes on in the world. We have to ask people questions about those things” ( 341 ).

Types of Interviews

There are several distinct types of interviews. Imagine a continuum (figure 11.1). On one side are unstructured conversations—the kind you have with your friends. No one is in control of those conversations, and what you talk about is often random—whatever pops into your head. There is no secret, underlying purpose to your talking—if anything, the purpose is to talk to and engage with each other, and the words you use and the things you talk about are a little beside the point. An unstructured interview is a little like this informal conversation, except that one of the parties to the conversation (you, the researcher) does have an underlying purpose, and that is to understand the other person. You are not friends speaking for no purpose, but it might feel just as unstructured to the “interviewee” in this scenario. That is one side of the continuum. On the other side are fully structured and standardized survey-type questions asked face-to-face. Here it is very clear who is asking the questions and who is answering them. This doesn’t feel like a conversation at all! A lot of people new to interviewing have this ( erroneously !) in mind when they think about interviews as data collection. Somewhere in the middle of these two extreme cases is the “ semistructured” interview , in which the researcher uses an “interview guide” to gently move the conversation to certain topics and issues. This is the primary form of interviewing for qualitative social scientists and will be what I refer to as interviewing for the rest of this chapter, unless otherwise specified.

Informal (unstructured conversations). This is the most “open-ended” approach to interviewing. It is particularly useful in conjunction with observational methods (see chapters 13 and 14). There are no predetermined questions. Each interview will be different. Imagine you are researching the Oregon Country Fair, an annual event in Veneta, Oregon, that includes live music, artisan craft booths, face painting, and a lot of people walking through forest paths. It’s unlikely that you will be able to get a person to sit down with you and talk intensely about a set of questions for an hour and a half. But you might be able to sidle up to several people and engage with them about their experiences at the fair. You might have a general interest in what attracts people to these events, so you could start a conversation by asking strangers why they are here or why they come back every year. That’s it. Then you have a conversation that may lead you anywhere. Maybe one person tells a long story about how their parents brought them here when they were a kid. A second person talks about how this is better than Burning Man. A third person shares their favorite traveling band. And yet another enthuses about the public library in the woods. During your conversations, you also talk about a lot of other things—the weather, the utilikilts for sale, the fact that a favorite food booth has disappeared. It’s all good. You may not be able to record these conversations. Instead, you might jot down notes on the spot and then, when you have the time, write down as much as you can remember about the conversations in long fieldnotes. Later, you will have to sit down with these fieldnotes and try to make sense of all the information (see chapters 18 and 19).

Interview guide ( semistructured interview ). This is the primary type employed by social science qualitative researchers. The researcher creates an “interview guide” in advance, which she uses in every interview. In theory, every person interviewed is asked the same questions. In practice, every person interviewed is asked mostly the same topics but not always the same questions, as the whole point of a “guide” is that it guides the direction of the conversation but does not command it. The guide is typically between five and ten questions or question areas, sometimes with suggested follow-ups or prompts . For example, one question might be “What was it like growing up in Eastern Oregon?” with prompts such as “Did you live in a rural area? What kind of high school did you attend?” to help the conversation develop. These interviews generally take place in a quiet place (not a busy walkway during a festival) and are recorded. The recordings are transcribed, and those transcriptions then become the “data” that is analyzed (see chapters 18 and 19). The conventional length of one of these types of interviews is between one hour and two hours, optimally ninety minutes. Less than one hour doesn’t allow for much development of questions and thoughts, and two hours (or more) is a lot of time to ask someone to sit still and answer questions. If you have a lot of ground to cover, and the person is willing, I highly recommend two separate interview sessions, with the second session being slightly shorter than the first (e.g., ninety minutes the first day, sixty minutes the second). There are lots of good reasons for this, but the most compelling one is that this allows you to listen to the first day’s recording and catch anything interesting you might have missed in the moment and so develop follow-up questions that can probe further. This also allows the person being interviewed to have some time to think about the issues raised in the interview and go a little deeper with their answers.

Standardized questionnaire with open responses ( structured interview ). This is the type of interview a lot of people have in mind when they hear “interview”: a researcher comes to your door with a clipboard and proceeds to ask you a series of questions. These questions are all the same whoever answers the door; they are “standardized.” Both the wording and the exact order are important, as people’s responses may vary depending on how and when a question is asked. These are qualitative only in that the questions allow for “open-ended responses”: people can say whatever they want rather than select from a predetermined menu of responses. For example, a survey I collaborated on included this open-ended response question: “How does class affect one’s career success in sociology?” Some of the answers were simply one word long (e.g., “debt”), and others were long statements with stories and personal anecdotes. It is possible to be surprised by the responses. Although it’s a stretch to call this kind of questioning a conversation, it does allow the person answering the question some degree of freedom in how they answer.

Survey questionnaire with closed responses (not an interview!). Standardized survey questions with specific answer options (e.g., closed responses) are not really interviews at all, and they do not generate qualitative data. For example, if we included five options for the question “How does class affect one’s career success in sociology?”—(1) debt, (2) social networks, (3) alienation, (4) family doesn’t understand, (5) type of grad program—we leave no room for surprises at all. Instead, we would most likely look at patterns around these responses, thinking quantitatively rather than qualitatively (e.g., using regression analysis techniques, we might find that working-class sociologists were twice as likely to bring up alienation). It can sometimes be confusing for new students because the very same survey can include both closed-ended and open-ended questions. The key is to think about how these will be analyzed and to what level surprises are possible. If your plan is to turn all responses into a number and make predictions about correlations and relationships, you are no longer conducting qualitative research. This is true even if you are conducting this survey face-to-face with a real live human. Closed-response questions are not conversations of any kind, purposeful or not.

In summary, the semistructured interview guide approach is the predominant form of interviewing for social science qualitative researchers because it allows a high degree of freedom of responses from those interviewed (thus allowing for novel discoveries) while still maintaining some connection to a research question area or topic of interest. The rest of the chapter assumes the employment of this form.

Creating an Interview Guide

Your interview guide is the instrument used to bridge your research question(s) and what the people you are interviewing want to tell you. Unlike a standardized questionnaire, the questions actually asked do not need to be exactly what you have written down in your guide. The guide is meant to create space for those you are interviewing to talk about the phenomenon of interest, but sometimes you are not even sure what that phenomenon is until you start asking questions. A priority in creating an interview guide is to ensure it offers space. One of the worst mistakes is to create questions that are so specific that the person answering them will not stray. Relatedly, questions that sound “academic” will shut down a lot of respondents. A good interview guide invites respondents to talk about what is important to them, not feel like they are performing or being evaluated by you.

Good interview questions should not sound like your “research question” at all. For example, let’s say your research question is “How do patriarchal assumptions influence men’s understanding of climate change and responses to climate change?” It would be worse than unhelpful to ask a respondent, “How do your assumptions about the role of men affect your understanding of climate change?” You need to unpack this into manageable nuggets that pull your respondent into the area of interest without leading him anywhere. You could start by asking him what he thinks about climate change in general. Or, even better, whether he has any concerns about heatwaves or increased tornadoes or polar icecaps melting. Once he starts talking about that, you can ask follow-up questions that bring in issues around gendered roles, perhaps asking if he is married (to a woman) and whether his wife shares his thoughts and, if not, how they negotiate that difference. The fact is, you won’t really know the right questions to ask until he starts talking.

There are several distinct types of questions that can be used in your interview guide, either as main questions or as follow-up probes. If you remember that the point is to leave space for the respondent, you will craft a much more effective interview guide! You will also want to think about the place of time in both the questions themselves (past, present, future orientations) and the sequencing of the questions.

Researcher Note

Suggestion : As you read the next three sections (types of questions, temporality, question sequence), have in mind a particular research question, and try to draft questions and sequence them in a way that opens space for a discussion that helps you answer your research question.

Type of Questions

Experience and behavior questions ask about what a respondent does regularly (their behavior) or has done (their experience). These are relatively easy questions for people to answer because they appear more “factual” and less subjective. This makes them good opening questions. For the study on climate change above, you might ask, “Have you ever experienced an unusual weather event? What happened?” Or “You said you work outside? What is a typical summer workday like for you? How do you protect yourself from the heat?”

Opinion and values questions , in contrast, ask questions that get inside the minds of those you are interviewing. “Do you think climate change is real? Who or what is responsible for it?” are two such questions. Note that you don’t have to literally ask, “What is your opinion of X?” but you can find a way to ask the specific question relevant to the conversation you are having. These questions are a bit trickier to ask because the answers you get may depend in part on how your respondent perceives you and whether they want to please you or not. We’ve talked a fair amount about being reflective. Here is another place where this comes into play. You need to be aware of the effect your presence might have on the answers you are receiving and adjust accordingly. If you are a woman who is perceived as liberal asking a man who identifies as conservative about climate change, there is a lot of subtext that can be going on in the interview. There is no one right way to resolve this, but you must at least be aware of it.

Feeling questions are questions that ask respondents to draw on their emotional responses. It’s pretty common for academic researchers to forget that we have bodies and emotions, but people’s understandings of the world often operate at this affective level, sometimes unconsciously or barely consciously. It is a good idea to include questions that leave space for respondents to remember, imagine, or relive emotional responses to particular phenomena. “What was it like when you heard your cousin’s house burned down in that wildfire?” doesn’t explicitly use any emotion words, but it allows your respondent to remember what was probably a pretty emotional day. And if they respond emotionally neutral, that is pretty interesting data too. Note that asking someone “How do you feel about X” is not always going to evoke an emotional response, as they might simply turn around and respond with “I think that…” It is better to craft a question that actually pushes the respondent into the affective category. This might be a specific follow-up to an experience and behavior question —for example, “You just told me about your daily routine during the summer heat. Do you worry it is going to get worse?” or “Have you ever been afraid it will be too hot to get your work accomplished?”

Knowledge questions ask respondents what they actually know about something factual. We have to be careful when we ask these types of questions so that respondents do not feel like we are evaluating them (which would shut them down), but, for example, it is helpful to know when you are having a conversation about climate change that your respondent does in fact know that unusual weather events have increased and that these have been attributed to climate change! Asking these questions can set the stage for deeper questions and can ensure that the conversation makes the same kind of sense to both participants. For example, a conversation about political polarization can be put back on track once you realize that the respondent doesn’t really have a clear understanding that there are two parties in the US. Instead of asking a series of questions about Republicans and Democrats, you might shift your questions to talk more generally about political disagreements (e.g., “people against abortion”). And sometimes what you do want to know is the level of knowledge about a particular program or event (e.g., “Are you aware you can discharge your student loans through the Public Service Loan Forgiveness program?”).

Sensory questions call on all senses of the respondent to capture deeper responses. These are particularly helpful in sparking memory. “Think back to your childhood in Eastern Oregon. Describe the smells, the sounds…” Or you could use these questions to help a person access the full experience of a setting they customarily inhabit: “When you walk through the doors to your office building, what do you see? Hear? Smell?” As with feeling questions , these questions often supplement experience and behavior questions . They are another way of allowing your respondent to report fully and deeply rather than remain on the surface.

Creative questions employ illustrative examples, suggested scenarios, or simulations to get respondents to think more deeply about an issue, topic, or experience. There are many options here. In The Trouble with Passion , Erin Cech ( 2021 ) provides a scenario in which “Joe” is trying to decide whether to stay at his decent but boring computer job or follow his passion by opening a restaurant. She asks respondents, “What should Joe do?” Their answers illuminate the attraction of “passion” in job selection. In my own work, I have used a news story about an upwardly mobile young man who no longer has time to see his mother and sisters to probe respondents’ feelings about the costs of social mobility. Jessi Streib and Betsy Leondar-Wright have used single-page cartoon “scenes” to elicit evaluations of potential racial discrimination, sexual harassment, and classism. Barbara Sutton ( 2010 ) has employed lists of words (“strong,” “mother,” “victim”) on notecards she fans out and asks her female respondents to select and discuss.

Background/Demographic Questions

You most definitely will want to know more about the person you are interviewing in terms of conventional demographic information, such as age, race, gender identity, occupation, and educational attainment. These are not questions that normally open up inquiry. [1] For this reason, my practice has been to include a separate “demographic questionnaire” sheet that I ask each respondent to fill out at the conclusion of the interview. Only include those aspects that are relevant to your study. For example, if you are not exploring religion or religious affiliation, do not include questions about a person’s religion on the demographic sheet. See the example provided at the end of this chapter.

Temporality

Any type of question can have a past, present, or future orientation. For example, if you are asking a behavior question about workplace routine, you might ask the respondent to talk about past work, present work, and ideal (future) work. Similarly, if you want to understand how people cope with natural disasters, you might ask your respondent how they felt then during the wildfire and now in retrospect and whether and to what extent they have concerns for future wildfire disasters. It’s a relatively simple suggestion—don’t forget to ask about past, present, and future—but it can have a big impact on the quality of the responses you receive.

Question Sequence

Having a list of good questions or good question areas is not enough to make a good interview guide. You will want to pay attention to the order in which you ask your questions. Even though any one respondent can derail this order (perhaps by jumping to answer a question you haven’t yet asked), a good advance plan is always helpful. When thinking about sequence, remember that your goal is to get your respondent to open up to you and to say things that might surprise you. To establish rapport, it is best to start with nonthreatening questions. Asking about the present is often the safest place to begin, followed by the past (they have to know you a little bit to get there), and lastly, the future (talking about hopes and fears requires the most rapport). To allow for surprises, it is best to move from very general questions to more particular questions only later in the interview. This ensures that respondents have the freedom to bring up the topics that are relevant to them rather than feel like they are constrained to answer you narrowly. For example, refrain from asking about particular emotions until these have come up previously—don’t lead with them. Often, your more particular questions will emerge only during the course of the interview, tailored to what is emerging in conversation.

Once you have a set of questions, read through them aloud and imagine you are being asked the same questions. Does the set of questions have a natural flow? Would you be willing to answer the very first question to a total stranger? Does your sequence establish facts and experiences before moving on to opinions and values? Did you include prefatory statements, where necessary; transitions; and other announcements? These can be as simple as “Hey, we talked a lot about your experiences as a barista while in college.… Now I am turning to something completely different: how you managed friendships in college.” That is an abrupt transition, but it has been softened by your acknowledgment of that.

Probes and Flexibility

Once you have the interview guide, you will also want to leave room for probes and follow-up questions. As in the sample probe included here, you can write out the obvious probes and follow-up questions in advance. You might not need them, as your respondent might anticipate them and include full responses to the original question. Or you might need to tailor them to how your respondent answered the question. Some common probes and follow-up questions include asking for more details (When did that happen? Who else was there?), asking for elaboration (Could you say more about that?), asking for clarification (Does that mean what I think it means or something else? I understand what you mean, but someone else reading the transcript might not), and asking for contrast or comparison (How did this experience compare with last year’s event?). “Probing is a skill that comes from knowing what to look for in the interview, listening carefully to what is being said and what is not said, and being sensitive to the feedback needs of the person being interviewed” ( Patton 2002:374 ). It takes work! And energy. I and many other interviewers I know report feeling emotionally and even physically drained after conducting an interview. You are tasked with active listening and rearranging your interview guide as needed on the fly. If you only ask the questions written down in your interview guide with no deviations, you are doing it wrong. [2]

The Final Question

Every interview guide should include a very open-ended final question that allows for the respondent to say whatever it is they have been dying to tell you but you’ve forgotten to ask. About half the time they are tired too and will tell you they have nothing else to say. But incredibly, some of the most honest and complete responses take place here, at the end of a long interview. You have to realize that the person being interviewed is often discovering things about themselves as they talk to you and that this process of discovery can lead to new insights for them. Making space at the end is therefore crucial. Be sure you convey that you actually do want them to tell you more, that the offer of “anything else?” is not read as an empty convention where the polite response is no. Here is where you can pull from that active listening and tailor the final question to the particular person. For example, “I’ve asked you a lot of questions about what it was like to live through that wildfire. I’m wondering if there is anything I’ve forgotten to ask, especially because I haven’t had that experience myself” is a much more inviting final question than “Great. Anything you want to add?” It’s also helpful to convey to the person that you have the time to listen to their full answer, even if the allotted time is at the end. After all, there are no more questions to ask, so the respondent knows exactly how much time is left. Do them the courtesy of listening to them!

Conducting the Interview

Once you have your interview guide, you are on your way to conducting your first interview. I always practice my interview guide with a friend or family member. I do this even when the questions don’t make perfect sense for them, as it still helps me realize which questions make no sense, are poorly worded (too academic), or don’t follow sequentially. I also practice the routine I will use for interviewing, which goes something like this:

• Introduce myself and reintroduce the study
• Provide consent form and ask them to sign and retain/return copy
• Ask if they have any questions about the study before we begin
• Ask if I can begin recording
• Ask questions (from interview guide)
• Turn off the recording device
• Ask if they are willing to fill out my demographic questionnaire
• Collect questionnaire and, without looking at the answers, place in same folder as signed consent form
• Thank them and depart

A note on remote interviewing: Interviews have traditionally been conducted face-to-face in a private or quiet public setting. You don’t want a lot of background noise, as this will make transcriptions difficult. During the recent global pandemic, many interviewers, myself included, learned the benefits of interviewing remotely. Although face-to-face is still preferable for many reasons, Zoom interviewing is not a bad alternative, and it does allow more interviews across great distances. Zoom also includes automatic transcription, which significantly cuts down on the time it normally takes to convert our conversations into “data” to be analyzed. These automatic transcriptions are not perfect, however, and you will still need to listen to the recording and clarify and clean up the transcription. Nor do automatic transcriptions include notations of body language or change of tone, which you may want to include. When interviewing remotely, you will want to collect the consent form before you meet: ask them to read, sign, and return it as an email attachment. I think it is better to ask for the demographic questionnaire after the interview, but because some respondents may never return it then, it is probably best to ask for this at the same time as the consent form, in advance of the interview.

What should you bring to the interview? I would recommend bringing two copies of the consent form (one for you and one for the respondent), a demographic questionnaire, a manila folder in which to place the signed consent form and filled-out demographic questionnaire, a printed copy of your interview guide (I print with three-inch right margins so I can jot down notes on the page next to relevant questions), a pen, a recording device, and water.

After the interview, you will want to secure the signed consent form in a locked filing cabinet (if in print) or a password-protected folder on your computer. Using Excel or a similar program that allows tables/spreadsheets, create an identifying number for your interview that links to the consent form without using the name of your respondent. For example, let’s say that I conduct interviews with US politicians, and the first person I meet with is George W. Bush. I will assign the transcription the number “INT#001” and add it to the signed consent form. [3] The signed consent form goes into a locked filing cabinet, and I never use the name “George W. Bush” again. I take the information from the demographic sheet, open my Excel spreadsheet, and add the relevant information in separate columns for the row INT#001: White, male, Republican. When I interview Bill Clinton as my second interview, I include a second row: INT#002: White, male, Democrat. And so on. The only link to the actual name of the respondent and this information is the fact that the consent form (unavailable to anyone but me) has stamped on it the interview number.

Many students get very nervous before their first interview. Actually, many of us are always nervous before the interview! But do not worry—this is normal, and it does pass. Chances are, you will be pleasantly surprised at how comfortable it begins to feel. These “purposeful conversations” are often a delight for both participants. This is not to say that sometimes things go wrong. I often have my students practice several “bad scenarios” (e.g., a respondent that you cannot get to open up; a respondent who is too talkative and dominates the conversation, steering it away from the topics you are interested in; emotions that completely take over; or shocking disclosures you are ill-prepared to handle), but most of the time, things go quite well. Be prepared for the unexpected, but know that the reason interviews are so popular as a technique of data collection is that they are usually richly rewarding for both participants.

One thing that I stress to my methods students and remind myself about is that interviews are still conversations between people. If there’s something you might feel uncomfortable asking someone about in a “normal” conversation, you will likely also feel a bit of discomfort asking it in an interview. Maybe more importantly, your respondent may feel uncomfortable. Social research—especially about inequality—can be uncomfortable. And it’s easy to slip into an abstract, intellectualized, or removed perspective as an interviewer. This is one reason trying out interview questions is important. Another is that sometimes the question sounds good in your head but doesn’t work as well out loud in practice. I learned this the hard way when a respondent asked me how I would answer the question I had just posed, and I realized that not only did I not really know how I would answer it, but I also wasn’t quite as sure I knew what I was asking as I had thought.

—Elizabeth M. Lee, Associate Professor of Sociology at Saint Joseph’s University, author of Class and Campus Life , and co-author of Geographies of Campus Inequality

How Many Interviews?

Your research design has included a targeted number of interviews and a recruitment plan (see chapter 5). Follow your plan, but remember that “ saturation ” is your goal. You interview as many people as you can until you reach a point at which you are no longer surprised by what they tell you. This means not that no one after your first twenty interviews will have surprising, interesting stories to tell you but rather that the picture you are forming about the phenomenon of interest to you from a research perspective has come into focus, and none of the interviews are substantially refocusing that picture. That is when you should stop collecting interviews. Note that to know when you have reached this, you will need to read your transcripts as you go. More about this in chapters 18 and 19.

Your Final Product: The Ideal Interview Transcript

A good interview transcript will demonstrate a subtly controlled conversation by the skillful interviewer. In general, you want to see replies that are about one paragraph long, not short sentences and not running on for several pages. Although it is sometimes necessary to follow respondents down tangents, it is also often necessary to pull them back to the questions that form the basis of your research study. This is not really a free conversation, although it may feel like that to the person you are interviewing.

Final Tips from an Interview Master

Annette Lareau is arguably one of the masters of the trade. In Listening to People , she provides several guidelines for good interviews and then offers a detailed example of an interview gone wrong and how it could be addressed (please see the “Further Readings” at the end of this chapter). Here is an abbreviated version of her set of guidelines: (1) interview respondents who are experts on the subjects of most interest to you (as a corollary, don’t ask people about things they don’t know); (2) listen carefully and talk as little as possible; (3) keep in mind what you want to know and why you want to know it; (4) be a proactive interviewer (subtly guide the conversation); (5) assure respondents that there aren’t any right or wrong answers; (6) use the respondent’s own words to probe further (this both allows you to accurately identify what you heard and pushes the respondent to explain further); (7) reuse effective probes (don’t reinvent the wheel as you go—if repeating the words back works, do it again and again); (8) focus on learning the subjective meanings that events or experiences have for a respondent; (9) don’t be afraid to ask a question that draws on your own knowledge (unlike trial lawyers who are trained never to ask a question for which they don’t already know the answer, sometimes it’s worth it to ask risky questions based on your hypotheses or just plain hunches); (10) keep thinking while you are listening (so difficult…and important); (11) return to a theme raised by a respondent if you want further information; (12) be mindful of power inequalities (and never ever coerce a respondent to continue the interview if they want out); (13) take control with overly talkative respondents; (14) expect overly succinct responses, and develop strategies for probing further; (15) balance digging deep and moving on; (16) develop a plan to deflect questions (e.g., let them know you are happy to answer any questions at the end of the interview, but you don’t want to take time away from them now); and at the end, (17) check to see whether you have asked all your questions. You don’t always have to ask everyone the same set of questions, but if there is a big area you have forgotten to cover, now is the time to recover ( Lareau 2021:93–103 ).

Sample: Demographic Questionnaire

ASA Taskforce on First-Generation and Working-Class Persons in Sociology – Class Effects on Career Success

Supplementary Demographic Questionnaire

Thank you for your participation in this interview project. We would like to collect a few pieces of key demographic information from you to supplement our analyses. Your answers to these questions will be kept confidential and stored by ID number. All of your responses here are entirely voluntary!

What best captures your race/ethnicity? (please check any/all that apply)

• White (Non Hispanic/Latina/o/x)
• Black or African American
• Hispanic, Latino/a/x of Spanish
• Asian or Asian American
• American Indian or Alaska Native
• Middle Eastern or North African
• Native Hawaiian or Pacific Islander
• Other : (Please write in: ________________)

What is your current position?

• Grad Student
• Full Professor

Please check any and all of the following that apply to you:

• I identify as a working-class academic
• I was the first in my family to graduate from college
• I grew up poor

What best reflects your gender?

• Transgender female/Transgender woman
• Transgender male/Transgender man
• Gender queer/ Gender nonconforming

Anything else you would like us to know about you?

Example: Interview Guide

In this example, follow-up prompts are italicized.  Note the sequence of questions.  That second question often elicits an entire life history , answering several later questions in advance.

Introduction Script/Question

Thank you for participating in our survey of ASA members who identify as first-generation or working-class.  As you may have heard, ASA has sponsored a taskforce on first-generation and working-class persons in sociology and we are interested in hearing from those who so identify.  Your participation in this interview will help advance our knowledge in this area.

• The first thing we would like to as you is why you have volunteered to be part of this study? What does it mean to you be first-gen or working class?  Why were you willing to be interviewed?
• How did you decide to become a sociologist?
• Can you tell me a little bit about where you grew up? ( prompts: what did your parent(s) do for a living?  What kind of high school did you attend?)
• Has this identity been salient to your experience? (how? How much?)
• How welcoming was your grad program? Your first academic employer?
• Why did you decide to pursue sociology at the graduate level?
• Did you experience culture shock in college? In graduate school?
• Has your FGWC status shaped how you’ve thought about where you went to school? debt? etc?
• Were you mentored? How did this work (not work)?  How might it?
• What did you consider when deciding where to go to grad school? Where to apply for your first position?
• What, to you, is a mark of career success? Have you achieved that success?  What has helped or hindered your pursuit of success?
• Do you think sociology, as a field, cares about prestige?
• Let’s talk a little bit about intersectionality. How does being first-gen/working class work alongside other identities that are important to you?
• What do your friends and family think about your career? Have you had any difficulty relating to family members or past friends since becoming highly educated?
• Do you have any debt from college/grad school? Are you concerned about this?  Could you explain more about how you paid for college/grad school?  (here, include assistance from family, fellowships, scholarships, etc.)
• (You’ve mentioned issues or obstacles you had because of your background.) What could have helped?  Or, who or what did? Can you think of fortuitous moments in your career?
• Do you have any regrets about the path you took?
• Is there anything else you would like to add? Anything that the Taskforce should take note of, that we did not ask you about here?

Further Readings

Britten, Nicky. 1995. “Qualitative Interviews in Medical Research.” BMJ: British Medical Journal 31(6999):251–253. A good basic overview of interviewing particularly useful for students of public health and medical research generally.

Corbin, Juliet, and Janice M. Morse. 2003. “The Unstructured Interactive Interview: Issues of Reciprocity and Risks When Dealing with Sensitive Topics.” Qualitative Inquiry 9(3):335–354. Weighs the potential benefits and harms of conducting interviews on topics that may cause emotional distress. Argues that the researcher’s skills and code of ethics should ensure that the interviewing process provides more of a benefit to both participant and researcher than a harm to the former.

Gerson, Kathleen, and Sarah Damaske. 2020. The Science and Art of Interviewing . New York: Oxford University Press. A useful guidebook/textbook for both undergraduates and graduate students, written by sociologists.

Kvale, Steiner. 2007. Doing Interviews . London: SAGE. An easy-to-follow guide to conducting and analyzing interviews by psychologists.

Lamont, Michèle, and Ann Swidler. 2014. “Methodological Pluralism and the Possibilities and Limits of Interviewing.” Qualitative Sociology 37(2):153–171. Written as a response to various debates surrounding the relative value of interview-based studies and ethnographic studies defending the particular strengths of interviewing. This is a must-read article for anyone seriously engaging in qualitative research!

Pugh, Allison J. 2013. “What Good Are Interviews for Thinking about Culture? Demystifying Interpretive Analysis.” American Journal of Cultural Sociology 1(1):42–68. Another defense of interviewing written against those who champion ethnographic methods as superior, particularly in the area of studying culture. A classic.

Rapley, Timothy John. 2001. “The ‘Artfulness’ of Open-Ended Interviewing: Some considerations in analyzing interviews.” Qualitative Research 1(3):303–323. Argues for the importance of “local context” of data production (the relationship built between interviewer and interviewee, for example) in properly analyzing interview data.

Weiss, Robert S. 1995. Learning from Strangers: The Art and Method of Qualitative Interview Studies . New York: Simon and Schuster. A classic and well-regarded textbook on interviewing. Because Weiss has extensive experience conducting surveys, he contrasts the qualitative interview with the survey questionnaire well; particularly useful for those trained in the latter.

• I say “normally” because how people understand their various identities can itself be an expansive topic of inquiry. Here, I am merely talking about collecting otherwise unexamined demographic data, similar to how we ask people to check boxes on surveys. ↵
• Again, this applies to “semistructured in-depth interviewing.” When conducting standardized questionnaires, you will want to ask each question exactly as written, without deviations! ↵
• I always include “INT” in the number because I sometimes have other kinds of data with their own numbering: FG#001 would mean the first focus group, for example. I also always include three-digit spaces, as this allows for up to 999 interviews (or, more realistically, allows for me to interview up to one hundred persons without having to reset my numbering system). ↵

A method of data collection in which the researcher asks the participant questions; the answers to these questions are often recorded and transcribed verbatim. There are many different kinds of interviews - see also semistructured interview , structured interview , and unstructured interview .

A document listing key questions and question areas for use during an interview.  It is used most often for semi-structured interviews.  A good interview guide may have no more than ten primary questions for two hours of interviewing, but these ten questions will be supplemented by probes and relevant follow-ups throughout the interview.  Most IRBs require the inclusion of the interview guide in applications for review.  See also interview and  semi-structured interview .

A data-collection method that relies on casual, conversational, and informal interviewing.  Despite its apparent conversational nature, the researcher usually has a set of particular questions or question areas in mind but allows the interview to unfold spontaneously.  This is a common data-collection technique among ethnographers.  Compare to the semi-structured or in-depth interview .

A form of interview that follows a standard guide of questions asked, although the order of the questions may change to match the particular needs of each individual interview subject, and probing “follow-up” questions are often added during the course of the interview.  The semi-structured interview is the primary form of interviewing used by qualitative researchers in the social sciences.  It is sometimes referred to as an “in-depth” interview.  See also interview and  interview guide .

The cluster of data-collection tools and techniques that involve observing interactions between people, the behaviors, and practices of individuals (sometimes in contrast to what they say about how they act and behave), and cultures in context.  Observational methods are the key tools employed by ethnographers and Grounded Theory .

Follow-up questions used in a semi-structured interview  to elicit further elaboration.  Suggested prompts can be included in the interview guide  to be used/deployed depending on how the initial question was answered or if the topic of the prompt does not emerge spontaneously.

A form of interview that follows a strict set of questions, asked in a particular order, for all interview subjects.  The questions are also the kind that elicits short answers, and the data is more “informative” than probing.  This is often used in mixed-methods studies, accompanying a survey instrument.  Because there is no room for nuance or the exploration of meaning in structured interviews, qualitative researchers tend to employ semi-structured interviews instead.  See also interview.

The point at which you can conclude data collection because every person you are interviewing, the interaction you are observing, or content you are analyzing merely confirms what you have already noted.  Achieving saturation is often used as the justification for the final sample size.

An interview variant in which a person’s life story is elicited in a narrative form.  Turning points and key themes are established by the researcher and used as data points for further analysis.

Introduction to Qualitative Research Methods Copyright © 2023 by Allison Hurst is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License , except where otherwise noted.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Types of Interviews in Research | Guide & Examples

Types of Interviews in Research | Guide & Examples

Published on March 10, 2022 by Tegan George . Revised on June 22, 2023.

An interview is a qualitative research method that relies on asking questions in order to collect data . Interviews involve two or more people, one of whom is the interviewer asking the questions.

There are several types of interviews, often differentiated by their level of structure.

• Structured interviews have predetermined questions asked in a predetermined order.
• Unstructured interviews are more free-flowing.
• Semi-structured interviews fall in between.

Interviews are commonly used in market research, social science, and ethnographic research .

Table of contents

What is a structured interview, what is a semi-structured interview, what is an unstructured interview, what is a focus group, examples of interview questions, advantages and disadvantages of interviews, other interesting articles, frequently asked questions about types of interviews.

Structured interviews have predetermined questions in a set order. They are often closed-ended, featuring dichotomous (yes/no) or multiple-choice questions. While open-ended structured interviews exist, they are much less common. The types of questions asked make structured interviews a predominantly quantitative tool.

Asking set questions in a set order can help you see patterns among responses, and it allows you to easily compare responses between participants while keeping other factors constant. This can mitigate   research biases and lead to higher reliability and validity. However, structured interviews can be overly formal, as well as limited in scope and flexibility.

• You feel very comfortable with your topic. This will help you formulate your questions most effectively.
• You have limited time or resources. Structured interviews are a bit more straightforward to analyze because of their closed-ended nature, and can be a doable undertaking for an individual.
• Your research question depends on holding environmental conditions between participants constant.

Prevent plagiarism. Run a free check.

Semi-structured interviews are a blend of structured and unstructured interviews. While the interviewer has a general plan for what they want to ask, the questions do not have to follow a particular phrasing or order.

Semi-structured interviews are often open-ended, allowing for flexibility, but follow a predetermined thematic framework, giving a sense of order. For this reason, they are often considered “the best of both worlds.”

However, if the questions differ substantially between participants, it can be challenging to look for patterns, lessening the generalizability and validity of your results.

• You have prior interview experience. It’s easier than you think to accidentally ask a leading question when coming up with questions on the fly. Overall, spontaneous questions are much more difficult than they may seem.
• Your research question is exploratory in nature. The answers you receive can help guide your future research.

An unstructured interview is the most flexible type of interview. The questions and the order in which they are asked are not set. Instead, the interview can proceed more spontaneously, based on the participant’s previous answers.

Unstructured interviews are by definition open-ended. This flexibility can help you gather detailed information on your topic, while still allowing you to observe patterns between participants.

However, so much flexibility means that they can be very challenging to conduct properly. You must be very careful not to ask leading questions, as biased responses can lead to lower reliability or even invalidate your research.

• You have a solid background in your research topic and have conducted interviews before.
• Your research question is exploratory in nature, and you are seeking descriptive data that will deepen and contextualize your initial hypotheses.
• Your research necessitates forming a deeper connection with your participants, encouraging them to feel comfortable revealing their true opinions and emotions.

A focus group brings together a group of participants to answer questions on a topic of interest in a moderated setting. Focus groups are qualitative in nature and often study the group’s dynamic and body language in addition to their answers. Responses can guide future research on consumer products and services, human behavior, or controversial topics.

Focus groups can provide more nuanced and unfiltered feedback than individual interviews and are easier to organize than experiments or large surveys . However, their small size leads to low external validity and the temptation as a researcher to “cherry-pick” responses that fit your hypotheses.

• Your research focuses on the dynamics of group discussion or real-time responses to your topic.
• Your questions are complex and rooted in feelings, opinions, and perceptions that cannot be answered with a “yes” or “no.”
• Your topic is exploratory in nature, and you are seeking information that will help you uncover new questions or future research ideas.

Depending on the type of interview you are conducting, your questions will differ in style, phrasing, and intention. Structured interview questions are set and precise, while the other types of interviews allow for more open-endedness and flexibility.

Here are some examples.

• Semi-structured
• Unstructured
• Focus group
• Do you like dogs? Yes/No
• Do you associate dogs with feeling: happy; somewhat happy; neutral; somewhat unhappy; unhappy
• If yes, name one attribute of dogs that you like.
• If no, name one attribute of dogs that you don’t like.
• What feelings do dogs bring out in you?
• When you think more deeply about this, what experiences would you say your feelings are rooted in?

Interviews are a great research tool. They allow you to gather rich information and draw more detailed conclusions than other research methods, taking into consideration nonverbal cues, off-the-cuff reactions, and emotional responses.

However, they can also be time-consuming and deceptively challenging to conduct properly. Smaller sample sizes can cause their validity and reliability to suffer, and there is an inherent risk of interviewer effect arising from accidentally leading questions.

Here are some advantages and disadvantages of each type of interview that can help you decide if you’d like to utilize this research method.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Student’s  t -distribution
• Normal distribution
• Null and Alternative Hypotheses
• Chi square tests
• Confidence interval
• Quartiles & Quantiles
• Cluster sampling
• Stratified sampling
• Data cleansing
• Reproducibility vs Replicability
• Peer review
• Prospective cohort study

Research bias

• Implicit bias
• Cognitive bias
• Placebo effect
• Hawthorne effect
• Hindsight bias
• Affect heuristic
• Social desirability bias

The four most common types of interviews are:

• Structured interviews : The questions are predetermined in both topic and order. 
• Semi-structured interviews : A few questions are predetermined, but other questions aren’t planned.
• Unstructured interviews : None of the questions are predetermined.
• Focus group interviews : The questions are presented to a group instead of one individual.

The interviewer effect is a type of bias that emerges when a characteristic of an interviewer (race, age, gender identity, etc.) influences the responses given by the interviewee.

There is a risk of an interviewer effect in all types of interviews , but it can be mitigated by writing really high-quality interview questions.

Social desirability bias is the tendency for interview participants to give responses that will be viewed favorably by the interviewer or other participants. It occurs in all types of interviews and surveys , but is most common in semi-structured interviews , unstructured interviews , and focus groups .

Social desirability bias can be mitigated by ensuring participants feel at ease and comfortable sharing their views. Make sure to pay attention to your own body language and any physical or verbal cues, such as nodding or widening your eyes.

This type of bias can also occur in observations if the participants know they’re being observed. They might alter their behavior accordingly.

A focus group is a research method that brings together a small group of people to answer questions in a moderated setting. The group is chosen due to predefined demographic traits, and the questions are designed to shed light on a topic of interest. It is one of 4 types of interviews .

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to systematically measure variables and test hypotheses . Qualitative methods allow you to explore concepts and experiences in more detail.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

George, T. (2023, June 22). Types of Interviews in Research | Guide & Examples. Scribbr. Retrieved August 30, 2024, from https://www.scribbr.com/methodology/interviews-research/

Is this article helpful?

Tegan George

Other students also liked, unstructured interview | definition, guide & examples, structured interview | definition, guide & examples, semi-structured interview | definition, guide & examples, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

A comprehensive guide to in-depth interviews (IDIs)

UserTesting

You might have user data. But do you fully understand the why behind the data? Do you know who they are as people? If the answer is no, you're not alone.

A 2019 study found that more than half of researchers would like to use more in-person interviews as a UX research method . The good news? Most people are actually willing to give you their undivided attention to actively improve your app, site, or service user experience. As a UX researcher, you’re sitting on a potential goldmine of information and insights from your customer base. 

To pull that insightful data from participants, you’ll need to conduct in-depth interviews (IDIs) . These interviews require more planning and resources than other data collection methods. You’re also asking for more of your user’s time. 

For those reasons, it's essential to ensure you go into the process with a clear idea of what data you hope to get out of one. 

What is an in-depth interview? 

An in-depth interview is a qualitative research technique that involves conducting multiple individual interviews. They involve one-on-one engagement with participants, usually taking place face-to-face, either remotely or in-person. 

Unlike other research methods, in-depth interviews have a more flexible structure than moderated usability studies .

IDIs are used to get a more detailed and well-rounded perspective of users’ opinions, experiences, and feelings about a product's UX. 

Instead of more general qualitative or quantitative questionnaires that are sent out to a larger group of customers, IDI questions can be tailored to the interviewee and their individual usage. 

In these more intensive interviews, organizations usually ask a smaller number of customers to take part. This means that responses to different ideas, features, services, or future plans are more deeply investigated. 

You could consider using in-depth interviews for the following reasons: 

• To get feedback on a new product or service your business has launched 
• As a way of understanding the needs and expectations of your customers during persona gathering sessions
• For coming up with new ideas on how customers would make improvements to an existing product or service 
• Following a usability study to better understand how users intend to use your platform
• To get insight into how a customer thinks about design elements on certain pages

As you can see, they’re most effective when used in combination with other research methods like online surveys and usability testing .

Why are in-depth interviews important? 

There’s a large gap between the consumer's experience of brands and the marketer's confidence in their own branding. While most marketers are confident they can meet their target market's level of expectation, just under half of consumers say brands fail to meet their expectations.

Most users will switch to a competitor if they have just one bad experience with a brand they typically like.

What does this mean? To put it simply: creating a positive user experience is key for encouraging people to interact with your app, service, or site. 

In-depth interviews are one way of bridging this gap between consumer experience and business confidence. You can get insight into a users’ thoughts and feelings—and use that qualitative data to improve design, product launches, and key messaging.

What are the benefits of performing in-depth interviews? 

IDIs should show you how users feel about specific elements of your UX. They can also help you gain confidence in making future decisions. 

Here are a few of the top benefits of using IDIs in your UX process. 

Smaller sample size

Given the higher quality relevant insights, researchers require fewer participants to take part in in-depth interviews. 

Lower drop-off rates mean that interviewers can conduct fewer IDIs and still collect rich data. For instance, online questionnaires have a higher drop-off rate, so they require a larger sampling. But with IDIs, you can get a lot of data from each individual participant. 

Get honest feedback

One-on-one in-depth interviews are free from possible peer-pressure dynamics or distractions that are sometimes present in larger focus groups . By taking an hour to chat with a participant directly, the two-way conversation leaves zero space for other users’ influence.

Some people may also feel more comfortable providing honest feedback in conversation instead of through a written questionnaire. 

Gain a deeper understanding of user behavior

Face-to-face in-depth interviews, whether remote or in-person, allow researchers to interpret body language . Interviewers can also analyze changes in tone of voice and word choice. 

These nuances help interviewers build a complete picture of user behavior that isn’t possible through other online or offline feedback channels. 

Build a stronger understanding of user expectations and motivations 

It’s easier to ask follow-up questions, request more detailed information, and explore particular topics in more depth with an IDI. They’re suited to asking open-ended questions that encourage longer and more detailed responses from the participant. 

As a researcher, you should take advantage of having participants’ undivided attention. Take the time to explore their opinions more deeply, beyond the surface level, for the most useful qualitative data .

What are the challenges? 

IDIs can give you valuable insight into users’ expectations and actual uses of your site. But, there are some challenges.

They’re time-consuming

Every interview you conduct will need to be transcribed, analyzed, organized, and properly stored. Multiple team members may need to be involved in the process.

While often more informative, IDIs require more time and preparation than other research methods—including simple written surveys.

Interviewers or moderators require thorough training and briefing 

You need skilled interviewers to ask the right research questions and properly engage with participants to obtain valuable insights. 

Successful IDIs depend on an interviewer's ability to ask thoughtful questions at the right time. They need to give the participants space to think and talk—while making them feel comfortable enough to do so. It takes training for an interviewer to hone in on this skill set.

Participants require careful vetting 

To gather valuable and balanced insights, it’s essential to use random sampling to gather a group of participants that accurately represent your organization’s user base. Once you have a random set of participants, you should check that they represent your user base’s different groups. 

Depending on the size of your customer base, it likely won’t be possible to interview all your customers. That being said, it's important to interview a variety of different users. 

For instance, you may want to interview a group of users who are unfamiliar with your site design, those who have been using your site for six months, and another group who left for a competitor. 

The best way to do this would be to  segment your customer base , then randomly generate a sample of participants to invite to an IDI.

How do you structure IDIs? 

When it comes to how structured your IDIs are, you have several options: 

• Structured interviews  are fixed in their methodology. The interviewer would only ask predetermined questions and target specific experiences. A structured approach limits the scope for exploring discussed topics in more depth. 
• Unstructured interviews  aren’t defined and don’t include pre-planned questions. It’s more like a conversation between the researcher and the respondent.
• Semi-structured interviews  follow some protocols to guide the process. While it’s a conversation between two individuals, and the interviewer can ask for more details, most of the questions are scripted. Interviewers will plan some initial questions and themes to cover, but allow the respondents’ answers to guide the interview direction. 

Generally, the most valuable in-depth interviews are semi-structured. 

These IDIs have a loose structure, but remain adaptable to the participant’s issues and ideas. This flexibility enables interviewers to explore each response fully and better engage with the user. 

When to use unstructured interviews: "If you’re at the beginning of a design phase, just trying to path-find for innovation, or trying to really dig into another layer of how your users use your product, that's when you're going to want to be more unstructured. When you've already got the product or the prototype, and you're wanting to validate, and you want to make sure that you're designing it the right way." - Julie Strubel, UserZoom Senior UX Researcher

How do you conduct a good in-depth interview?

Some preparation is key for conducting an insightful in-depth interview. Planning topics and conversation starters in advance will help you use your participants’ time more wisely. If your customers feel that their time was wasted or the IDI was simply too long, they may be reluctant to participate in future drives for customer insights. 

Keeping your IDIs brief and well-structured will also help your participants maintain focus until the end. Longer interviews with less clear objectives run the risk of tiring customers out and reducing the quality of their answers.

In addition to the tips we just mentioned, we've pulled together 9 best practices to keep in mind when you're running IDIs:

• Know your aims
• Define your scope
• Set a time limit
• Ask the right questions
• Remove bias
• Make questions actionable
• Test your questions
• Create an IDI guide
• Put your insights into action

1. Know your aims

Before you plan on conducting any in-depth interviews, it’s important to know what you’re aiming to get out of the process. This helps guide your questions—and ultimately, the conversation.  

Perhaps you’re looking to understand how customers feel about your site’s new design. In that case, you need to ask open-ended questions like, “What do you think of our new site design when compared to our previous version?”

If you want to find out if participants find your checkout page to be intuitive, ask questions like, “How did you feel about the navigation experience of our checkout page?”

When you identify your goals , it’ll be easier to plan questions to help build your understanding of what your end-user is looking to achieve. 

For this reason, Hector Harris-Burton of Imaginaire recommends having some structure.

“Have a loose structure so that you can cover topics that you'd like to hear about. While an IDI is supposed to be a freeform conversation, one of the best ways to get the most information out of the interviewee is to ensure that you have talking points and some kind of structure.  By having a structure, not only are you able to cover the topics you need, but you can also aid the conversation. No matter who you talk to, there is always the potential for the conversation to run dry. But with a structure and talking points, you're able to lead the conversation and move forward, rather than struggling.”

2. Define your scope 

Always clarify the extent of your research before you begin interviewing people. 

Decide how much time you have to spend on conducting IDIs, and the minimum number of respondents you need to consider any themes as standard. That should define the number of users you need to interview.

There’s no magic number when it comes to determining the size of your sample . Always prioritize quality over quantity, and check that you can spend a reasonable amount of time in each interview. 

For instance, there’s not much point in interviewing 120 people for five minutes each. It would be better to spend the same amount of time interviewing 20 people for 30 minutes each. That way, you’d have time to explore topics more deeply with each participant instead of rushing through a list of questions. 

3. Set a time limit

When conducting an IDI, make sure you’re mindful of your participants’ time. Remember: their answer quality may drop off towards the end of the conversation if you’ve been talking for hours. 

Let participants know how much time each interview will take and stick to it. 

Keeping each interview to a maximum of one hour will allow you to ask participants plenty of questions without going off-topic. It’s also short enough for them to commit to the interview. 

4. Ask the right questions 

Asking the right questions will encourage respondents to share their honest points of view. 

In your in-depth interviews, add in a mix of question types, such as:

• General ice-breaking questions.  Ease people into the discussion by asking light-hearted questions first. For example, “Tell me about your biggest challenges right now.”
• More specific detail-oriented questions.  Start to explore subjects that are more closely linked to your research goals. For instance, “What are you hoping to achieve by using our app?”
• Insight-based questions.  Ask more specific questions about existing or new features. Use your last questions to find out how users feel about your future plans. For example, “How useful would this new calendar function be for you?” 

Using the right questions will help you better listen to users and then effectively implement their feedback. You’ll also gather qualitative data that helps you make smarter UX decisions. 

5. Remove bias 

It’s easy to accidentally influence customers’ answers without intending to. Take care with how you phrase each of your questions to make sure you’re not accidentally influencing their responses. 

Consider the following question:  “What do you like about this new service?”  

The phrasing of the question restricts users to only talking about what they like about the service as opposed to providing a more balanced answer. 

To make the question more neutral and bias-free , ask something along the lines of:  “What do you think about our new service?” 

This simple change of phrasing leaves the customer free to provide an honest perspective (as opposed to just listing what they like about the service).

The key is to collect valuable, actionable feedback that isn’t shaped by your organization’s expectations or agenda. So, always ask users open-ended questions and avoid leading questions that influence participants’ answers. 

Read up on other examples of leading questions so you know what to avoid.

Don't ask them leading questions designed to elicit a certain response. You want them to answer truthfully, so it's best to ask questions straight up and don't suggest an answer. For example, avoid questions like “What do you think is helpful in the new package we offered? The new payment methods?” Ask open-ended questions that will lead to expansive responses. Remember, you want to find out what they are trying to do and what their problems are. - Lauri Kinkar, Messente CEO

6. Make questions actionable 

When conducting IDIs, only ask users questions that are actionable. That way, you’ll be able to directly use their answers to improve user experience. 

Here’s an example of an actionable question: “Is there anything you would change about our checkout page?” Any answers you receive to this question will improve your current page based on users’ current sentiment around it. 

If you’re in doubt about asking a particular question, think about whether you could use the answer to improve your UX. If you can’t find a way of using the response to improve your user experience, it’s best to ask another question. 

7. Test your interview questions

Test your questions on teammates and ask for feedback on whether your questions are straightforward. 

Do they give the answers you were expecting? Or bounce the questions back to you, asking what you meant?

Your users’ responses should give you a clear idea of what needs to be changed or improved moving forwards. 

8. Create an IDI guide

IDI guides are an informative document that outlines the interview process from start to finish.

 It should act as a to-do list that you refer to throughout the interview process, as UX Specialist Andreas Johansson explains:

"As part of the user research plan, I also create an interview guide. This is basically a rough structure for me to refer to when I do the interviews. I tend to do semi-structured interviews. That means that I refer to the interview guide if I get stuck, but I don't tend to be too rigid about it. Sometimes it's good if the discussions go off on a tangent, for instance."

First, state your objectives and then outline the general flow of the interview. Include all the topics you want to talk about and in what order—remembering  why  you’re asking them in the first place. 

This interview guide will stop you from getting side-tracked during the conversation, helping you create the best possible experience for your participants. 

Consider giving your colleagues a copy of the IDI guide, too. They can provide you with feedback on what you’re planning to ask. Knowing what insights you’re planning to pull will also help them anticipate the data they’ll later analyze and store.

9. Put your insights into action

It’s all well and good to have a jam-packed day of IDIs. Once you’ve collected that data, though, you need to turn it into actionable insights.

Olga Kimalana, Senior Conversion Strategist at Scandiweb, explains:

“There’s no point in conducting user interviews if you don’t act upon the insights you gathered, so make sure to present your insights and plans of action to stakeholders to kick off the improvements.”

The simplest way to do this is to listen back through each interview. Create a transcription of the conversion, and flag different parts of the conversation that are interesting. This can include snippets you want to re-listen or pay closer attention to.

We recommend using a professional UX platform (instead of Zoom or Google Meet) for this reason. UserTesting, for example, has a note-taking feature. You can annotate transcripts, and add hashtags to certain topics, to spot themes across several IDIs.

Five in-depth interview best practices 

Following a few best practices will ensure you make the most of each interview and collect data for improving your UX. Here are five to start with. 

1. Ensure participants feel comfortable 

In-depth interviews are voluntary, so it’s important to make customers feel comfortable enough to share their honest opinions. 

As an interviewer, you should approach each interview with an approachable, friendly, and open-minded attitude. Avoid making the interview feel too formal–you don’t want users to feel under pressure or stressed. 

"You're setting the stage, you want people to feel relaxed. Even if they're just going through and doing a talk-out-loud for a usability session, you want to put them at ease, and that's not just about reading the script. It's about making that human connection in the first three minutes." - Julie Strubel, UserZoom Senior UX Researcher

If customers have a positive experience during your interview, they’ll be more likely to readily offer feedback if you request it again in the future. You’ll also get better data from respondents who were genuinely interested in the conversation. 

2. Properly engage with your interviewees

Effectively engaging with interviewees is sometimes easier said than done—especially when you have a busy day full of in-depth interviews. 

Taking detailed notes may distract you from what your customer is saying or take you out of the moment, so you may want to use an audio or video recording device instead. That way, you can give your interviewee 100% of your attention and remain responsive to their answers.

Making recordings of each interview will enable you to fully engage with participants without worrying about forgetting critical data or insights later on. Simply refer to your recordings afterward and collect all of the relevant insights. 

3. Follow-up on user responses

It’s vital to understand what your customers mean by their responses and what’s behind their opinion of your app, site, or service. Try clarifying their responses by summarizing their thoughts. If you’re not sure, always ask for clarification. 

Whenever users share an opinion, follow-up on their response by asking why they feel the way they do.

For instance, if a participant says they don’t find your checkout page intuitive, you should follow-up by asking them what it is that they don’t find user-friendly. Is it the layout? Are the payment instructions unclear? Do they have to click on too many buttons?

Avoid putting words into participants’ mouths, but make sure to find out what it is precisely about the page they find unintuitive. 

Alternatively, if a user says they prefer your old site design, make sure to pinpoint why. Was there a specific menu flow they found easy to use? Was the search capability stronger? 

Asking why will help you go beyond surface-level responses and genuinely engage with your customer base. 

4. Provide consent forms

You should always provide your participants with consent forms that outline the purpose of the interview. 

To legally use the participants’ responses and details, you need to make sure that everyone signs an agreement as to how the information gathered from the interview will be used. 

Consent always needs to be:

• Based on clearly explained information.  Participants need to know what exactly is being researched. Provide a detailed information sheet for them to read before signing.  
• Given freely.  Consent can’t be dependent on receiving a benefit. If you plan on offering incentives, make sure you do this before asking for consent. 
• Given separately from other information.  Consent forms need to be given separately to an NDA form, for example. 
• Given for a specific purpose . If you plan on using the research in multiple ways, consent needs to be given for all of them.
• Able to be refused and withdrawn.  The interviewee needs to know how they can withdraw consent at any moment during the process. 

5. Say thank you 

Participants are giving up their time to share their opinions and perspectives on your product or service with you. Make it a priority to thank your customers for their time, so they feel appreciated. 

You could also provide incentives as a way of saying thank you to participants. For example, you could automatically enter participants into an Amazon gift card giveaway. 

Michael Margolis, a UX research partner at Google Ventures, offers different incentives depending on the customer type:

“I typically offer a $100 gift card for customer interviews. [...] Some people may be enticed by a larger “honorarium” or charitable donation made in their names. But experts often respond more to professional incentives, such as sharing a version of the research results, previewing a new or advanced technology, or giving them credit in a public way.”

Conduct more insightful IDIs today

In-depth interviews are a great source of original user insights as part of a broader UX research process. 

When planned and conducted correctly, IDIs make it easier to understand your users’ expectations and pain points. Unlike other qualitative data collection methods, you’ll be able to interact with users directly and dig deeper into how they feel about your UX.

Try UserTesting for free today!

In this article.

Get started now

About the author(s)

With UserTesting’s on-demand platform, you uncover ‘the why’ behind customer interactions. In just a few hours, you can capture the critical human insights you need to confidently deliver what your customers want and expect.

Related Blog Posts

Continuous discovery: all your questions answered

5 tips for retailers preparing for the 2024 holiday shopping season

How and why you should test your product description pages

Human understanding. Human experiences.

Get the latest news on events, research, and product launches

Oh no! We're unable to display this form.

Please check that you’re not running an adblocker and if you are please whitelist usertesting.com.

If you’re still having problems please drop us an email .

By submitting the form, I agree to the Privacy Policy and Terms of Use .

• Skip to main content
• Skip to primary sidebar
• Skip to footer
• QuestionPro

• Solutions Industries Gaming Automotive Sports and events Education Government Travel & Hospitality Financial Services Healthcare Cannabis Technology Use Case AskWhy Communities Audience Contactless surveys Mobile LivePolls Member Experience GDPR Positive People Science 360 Feedback Surveys
• Resources Blog eBooks Survey Templates Case Studies Training Help center

Home Market Research

In-depth Interviews: Definition and how to conduct them

Online surveys, user review sites and focus groups can be great methods for collecting data. However, another method of gathering data that is sometimes overlooked are the in-depth interviews.

All of these methods can be used in your comprehensive customer experience management strategy, but in-depth interviews can help you collect data that can offer rich insights into your target audience’s experience and preferences from a broad sample.

In this article you will discover the main characteristics of in-depth interviews as a great tool for your qualitative research and gather better insights from your objects of study.

LEARN ABOUT: Behavioral Research

What are in-depth interviews?

In-depth interviews are a qualitative data collection method that allows for the collection of a large amount of information about the behavior, attitude and perception of the interviewees.

LEARN ABOUT: Best Data Collection Tools

During in-depth interviews, researchers and participants have the freedom to explore additional points and change the direction of the process when necessary. It is an independent research method that can adopt multiple strategies according to the needs of the research.

Characteristics of in-depth interviews

There are many types of interviews , each with its particularities, in this case the most important characteristics of in-depth interviews are:

• Flexible structure: Although it is not very structured, it covers a few topics based on a guide, which allows the interviewer to cover areas appropriate for the interviewee.
• Interactive: The interviewer processes the material that is produced during the interview. During the interaction the interviewer poses initial questions in a positive manner, so that the respondent is encouraged to answer. The complete process is very human, and so less mundane and dull.
• Deep: Many probing techniques are used in in-depth interviews, so that results are understood through exploration and explanation. The interviewer asks follow-up questions to gain a deeper perspective and understand the participant’s viewpoint.
• Generative: Often interacting with your target audience creates new knowledge. For instance, if you are talking to your customers, you learn more about the purchase behavior. Researchers and participants present ideas for a specific topic and solutions to the problems posed.

To learn more about the characteristics of in-depth interviews, check out our blog on interview questions .

Importance of conducting in-depth interviews

As an in-depth interview is a one-on-one conversation, you get enough opportunities to get to the root causes of likes/dislikes, perceptions, or beliefs. 

Generally, questions are open-ended questions and can be customized as per the particular situation. You can use single ease questions . A single-ease question is a straightforward query that elicits a concise and uncomplicated response. The interviewer gets an opportunity to develop a rapport with the participant, thereby making them feel comfortable. Thus, they can bring out honest feedback and also note their expressions and body language. Such cues can amount to rich qualitative data.

LEARN ABOUT: Selection Bias

With surveys, there are chances that the respondents may select answers in a rush, but in case of in-depth interviews it’s hardly the worry of researchers. 

Conversations can prove to be an excellent method to collect data. In fact, people might be reluctant to answer questions in written format, but given the nature of an interview, participants might agree giving information verbally. You can also discuss with the interviewees if they want to keep their identity confidential.

In-depth interviews are aimed at uncovering the issues in order to obtain detailed results. This method allows you to gain insight into the experiences, feelings and perspectives of the interviewees.

When conducting the initial stage of a large research project, in-depth interviews prove to be useful to narrow down and focus on important research details.

When you want to have the context of a problem, in-depth interviews allow you to evaluate different solutions to manage the research process while assisting in in-depth data analysis .

LEARN ABOUT: Research Process Steps

Steps to conduct in-depth interviews

• Obtain the necessary information about the respondents and the context in which they operate.
• Make a script or a list of topics you want to cover. This will make it easy to add secondary questions.
• Schedule an interview at a time and date of the respondent’s choice.
• Ask questions confidently and let the interviewees feel comfortable, so that they too are confident and can answer difficult questions with ease.
• Set a maximum duration such that it doesn’t feel exhaustive.
• Observe and make notes on the interviewee’s body expressions and gestures.
• It is important to maintain ethics throughout the process.
• Transcribe the recordings and verify them with the interviewee.

Advantages of in-depth interviews

The benefits of conducting an in-depth interview include the following:

• They allow the researcher and participants to have a comfortable relationship to generate more in-depth responses regarding sensitive topics.
• Researchers can ask follow-up questions , obtain additional information, and return to key questions to gain a better understanding of the participants’ attitudes.
• The sampling is more accurate than other data collection methods .
• Researchers can monitor changes in tone and word choice of participants to gain a better understanding of opinions.
• Fewer participants are needed to obtain useful information. 
• In-depth interviews can be very beneficial when a detailed report on a person’s opinion and behavior is needed. In addition, it explores new ideas and contexts that give the researcher a complete picture of the phenomena that occurred.

Disadvantages 

The disadvantages of in-depth interviews are:

• They are time-consuming, as they must be transcribed, organized, analyzed in detail.
• If the interviewer is inexperienced, it affects the complete process.
• It is a costly research method compared to other methods.
• Participants must be chosen carefully to avoid bias, otherwise it can lengthen the process.
• Generally, participants decide to collaborate only when they receive an incentive in return.

LEARN ABOUT: Self-Selection Bias

What is the purpose of in-depth interviews?

The main purpose of in-depth interviews is to understand the consumer behavior and make well-informed decisions. Organizations can formulate their marketing strategies based on the information received from the respondents. They can also gain insights into the probable demand and know consumer pulse.

In the case of B2B businesses, researchers can understand the demand in more detail and can ask questions targeted for the experts. Interviews offer a chance to understand the customer’s thought process and design products that have higher chances of being accepted in the market.

LEARN ABOUT: 12 Best Tools for Researchers

Final words

An in-depth interview should follow all the steps of the process to collect meaningful data. Hope this blog helps you decide whether you should conduct a detailed interview with your target audience, keeping in mind the pros and cons of it.

If you want to get started with conducting research online, we suggest using an online survey software that offers features like designing a questionnaire , customized look and feel, distributing to your contacts and data analytics. Create an account with QuestionPro Surveys and explore the tool. If you need any help with research or data collection, feel free to connect with us.

Create a free account

MORE LIKE THIS

Why You Should Attend XDAY 2024

Aug 30, 2024

Alchemer vs Qualtrics: Find out which one you should choose

Target Population: What It Is + Strategies for Targeting

Aug 29, 2024

Microsoft Customer Voice vs QuestionPro: Choosing the Best

Other categories.

• Academic Research
• Artificial Intelligence
• Assessments
• Brand Awareness
• Case Studies
• Communities
• Consumer Insights
• Customer effort score
• Customer Engagement
• Customer Experience
• Customer Loyalty
• Customer Research
• Customer Satisfaction
• Employee Benefits
• Employee Engagement
• Employee Retention
• Friday Five
• General Data Protection Regulation
• Insights Hub
• Life@QuestionPro
• Market Research
• Mobile diaries
• Mobile Surveys
• New Features
• Online Communities
• Question Types
• Questionnaire
• QuestionPro Products
• Release Notes
• Research Tools and Apps
• Revenue at Risk
• Survey Templates
• Training Tips
• Tuesday CX Thoughts (TCXT)
• Uncategorized
• What’s Coming Up
• Workforce Intelligence

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Basic Clin Pharm
• v.5(4); September 2014-November 2014

Qualitative research method-interviewing and observation

Shazia jamshed.

Department of Pharmacy Practice, Kulliyyah of Pharmacy, International Islamic University Malaysia, Kuantan Campus, Pahang, Malaysia

Buckley and Chiang define research methodology as “a strategy or architectural design by which the researcher maps out an approach to problem-finding or problem-solving.”[ 1 ] According to Crotty, research methodology is a comprehensive strategy ‘that silhouettes our choice and use of specific methods relating them to the anticipated outcomes,[ 2 ] but the choice of research methodology is based upon the type and features of the research problem.[ 3 ] According to Johnson et al . mixed method research is “a class of research where the researcher mixes or combines quantitative and qualitative research techniques, methods, approaches, theories and or language into a single study.[ 4 ] In order to have diverse opinions and views, qualitative findings need to be supplemented with quantitative results.[ 5 ] Therefore, these research methodologies are considered to be complementary to each other rather than incompatible to each other.[ 6 ]

Qualitative research methodology is considered to be suitable when the researcher or the investigator either investigates new field of study or intends to ascertain and theorize prominent issues.[ 6 , 7 ] There are many qualitative methods which are developed to have an in depth and extensive understanding of the issues by means of their textual interpretation and the most common types are interviewing and observation.[ 7 ]

Interviewing

This is the most common format of data collection in qualitative research. According to Oakley, qualitative interview is a type of framework in which the practices and standards be not only recorded, but also achieved, challenged and as well as reinforced.[ 8 ] As no research interview lacks structure[ 9 ] most of the qualitative research interviews are either semi-structured, lightly structured or in-depth.[ 9 ] Unstructured interviews are generally suggested in conducting long-term field work and allow respondents to let them express in their own ways and pace, with minimal hold on respondents’ responses.[ 10 ]

Pioneers of ethnography developed the use of unstructured interviews with local key informants that is., by collecting the data through observation and record field notes as well as to involve themselves with study participants. To be precise, unstructured interview resembles a conversation more than an interview and is always thought to be a “controlled conversation,” which is skewed towards the interests of the interviewer.[ 11 ] Non-directive interviews, form of unstructured interviews are aimed to gather in-depth information and usually do not have pre-planned set of questions.[ 11 ] Another type of the unstructured interview is the focused interview in which the interviewer is well aware of the respondent and in times of deviating away from the main issue the interviewer generally refocuses the respondent towards key subject.[ 11 ] Another type of the unstructured interview is an informal, conversational interview, based on unplanned set of questions that are generated instantaneously during the interview.[ 11 ]

In contrast, semi-structured interviews are those in-depth interviews where the respondents have to answer preset open-ended questions and thus are widely employed by different healthcare professionals in their research. Semi-structured, in-depth interviews are utilized extensively as interviewing format possibly with an individual or sometimes even with a group.[ 6 ] These types of interviews are conducted once only, with an individual or with a group and generally cover the duration of 30 min to more than an hour.[ 12 ] Semi-structured interviews are based on semi-structured interview guide, which is a schematic presentation of questions or topics and need to be explored by the interviewer.[ 12 ] To achieve optimum use of interview time, interview guides serve the useful purpose of exploring many respondents more systematically and comprehensively as well as to keep the interview focused on the desired line of action.[ 12 ] The questions in the interview guide comprise of the core question and many associated questions related to the central question, which in turn, improve further through pilot testing of the interview guide.[ 7 ] In order to have the interview data captured more effectively, recording of the interviews is considered an appropriate choice but sometimes a matter of controversy among the researcher and the respondent. Hand written notes during the interview are relatively unreliable, and the researcher might miss some key points. The recording of the interview makes it easier for the researcher to focus on the interview content and the verbal prompts and thus enables the transcriptionist to generate “verbatim transcript” of the interview.

Similarly, in focus groups, invited groups of people are interviewed in a discussion setting in the presence of the session moderator and generally these discussions last for 90 min.[ 7 ] Like every research technique having its own merits and demerits, group discussions have some intrinsic worth of expressing the opinions openly by the participants. On the contrary in these types of discussion settings, limited issues can be focused, and this may lead to the generation of fewer initiatives and suggestions about research topic.

Observation

Observation is a type of qualitative research method which not only included participant's observation, but also covered ethnography and research work in the field. In the observational research design, multiple study sites are involved. Observational data can be integrated as auxiliary or confirmatory research.[ 11 ]

Research can be visualized and perceived as painstaking methodical efforts to examine, investigate as well as restructure the realities, theories and applications. Research methods reflect the approach to tackling the research problem. Depending upon the need, research method could be either an amalgam of both qualitative and quantitative or qualitative or quantitative independently. By adopting qualitative methodology, a prospective researcher is going to fine-tune the pre-conceived notions as well as extrapolate the thought process, analyzing and estimating the issues from an in-depth perspective. This could be carried out by one-to-one interviews or as issue-directed discussions. Observational methods are, sometimes, supplemental means for corroborating research findings.

Interviews can be defined as a qualitative research technique which involves “conducting intensive individual interviews with a small number of respondents to explore their perspectives on a particular idea, program or situation.” [1]

There are three different formats of interviews: structured, semi-structured and unstructured.

Structured interviews consist of a series of pre-determined questions that all interviewees answer in the same order. Data analysis usually tends to be more straightforward because researcher can compare and contrast different answers given to the same questions.

Unstructured interviews are usually the least reliable from research viewpoint, because no questions are prepared prior to the interview and data collection is conducted in an informal manner. Unstructured interviews can be associated with a high level of bias and comparison of answers given by different respondents tends to be difficult due to the differences in formulation of questions.

Semi-structured interviews contain the components of both, structured and unstructured interviews. In semi-structured interviews, interviewer prepares a set of same questions to be answered by all interviewees. At the same time, additional questions might be asked during interviews to clarify and/or further expand certain issues.

Advantages of interviews include possibilities of collecting detailed information about research questions.  Moreover, in in this type of primary data collection researcher has direct control over the flow of process and she has a chance to clarify certain issues during the process if needed. Disadvantages, on the other hand, include longer time requirements and difficulties associated with arranging an appropriate time with perspective sample group members to conduct interviews.

When conducting interviews you should have an open mind and refrain from displaying disagreements in any forms when viewpoints expressed by interviewees contradict your own ideas. Moreover, timing and environment for interviews need to be scheduled effectively. Specifically, interviews need to be conducted in a relaxed environment, free of any forms of pressure for interviewees whatsoever.

Respected scholars warn that “in conducting an interview the interviewer should attempt to create a friendly, non-threatening atmosphere. Much as one does with a cover letter, the interviewer should give a brief, casual introduction to the study; stress the importance of the person’s participation; and assure anonymity, or at least confidentiality, when possible.” [2]

There is a risk of interviewee bias during the primary data collection process and this would seriously compromise the validity of the project findings. Some interviewer bias can be avoided by ensuring that the interviewer does not overreact to responses of the interviewee. Other steps that can be taken to help avoid or reduce interviewer bias include having the interviewer dress inconspicuously and appropriately for the environment and holding the interview in a private setting.  [3]

My e-book, The Ultimate Guide to Writing a Dissertation in Business Studies: a step by step assistance offers practical assistance to complete a dissertation with minimum or no stress. The e-book covers all stages of writing a dissertation starting from the selection to the research area to submitting the completed version of the work within the deadline.John Dudovskiy

[1] Boyce, C. & Neale, P. (2006) “Conducting in-depth Interviews: A Guide for Designing and Conducting In-Depth Interviews”, Pathfinder International Tool Series

[2] Connaway, L.S.& Powell, R.P.(2010) “Basic Research Methods for Librarians” ABC-CLIO

[3] Connaway, L.S.& Powell, R.P.(2010) “Basic Research Methods for Librarians” ABC-CLIO

Research Design Review

A discussion of qualitative & quantitative research design, strengths & limitations of the in-depth interview method: an overview.

The following is a modified excerpt from Applied Qualitative Research Design: A Total Quality Framework Approach (Roller & Lavrakas, 2015, pp. 56-57).

An additional strength of the IDI method is the flexibility of the interview format, which allows the interviewer to tailor the order in which questions are asked, modify the question wording as appropriate, ask follow-up questions to clarify interviewees’ responses, and use indirect questions (e.g., the use of projective techniques ) to stimulate subconscious opinions or recall. It should be noted, however, that “flexibility” does not mean a willy-nilly approach to interviewing, and, indeed, the interviewer should employ quality measures such as those outlined in “Applying a Quality Framework to the In-depth Interview Method.”

A third key strength of the IDI method—analyzability of the data—is a byproduct of the interviewer–interviewee relationship and the depth of interviewing techniques, which produce a granularity in the IDI data that is rich in fine details and serves as the basis for deciphering the narrative within each interview. These details also enable researchers to readily identify where they agree or disagree with the meanings of codes and themes associated with specific responses, which ultimately leads to the identification of themes and connections across interview participants.

Limitations

The IDI method also presents challenges and limitations that deserve the researcher’s attention. The most important, from a Total Quality Framework standpoint, has to do with what is also considered a key strength of the IDI method: the interviewer–interviewee relationship. There are two key aspects of the relationship that can potentially limit (or even undermine) the effectiveness of the IDI method: the interviewer and the social context. The main issue with respect to the interviewer is his/her potential for biasing the information that is gathered. This can happen due to  (a) personal characteristics such as gender, age, race, ethnicity, and education (e.g., a 60-year-old Caucasian male interviewer may stifle or skew responses from young, female, African American participants); (b) personal values or beliefs (e.g., an interviewer with strongly held beliefs about global warming and its damaging impact on the environment may “tune out” or misconstrue the comments from interviewees who believe global warming is a myth); and/or (c) other factors (e.g., an interviewer’s stereotyping, misinterpreting, and/or presumptions about the interviewee based solely on the interviewee’s outward appearance). Any of these characteristics may negatively influence an interviewee’s responses to the researcher’s questions and/or the accuracy of the interviewer’s data gathering. A result of these interviewer effects may be the “difficulty of seeing the people as complex, and . . . a reduction of their humanity to a stereotypical, flat, one-dimensional paradigm” (Krumer-Nevo, 2002, p. 315).

The second key area of concern with the IDI method is related to the broader social context of the relationship, particularly what Kvale (2006) calls the “power dynamics” within the interview environment, characterized by the possibility of “a one-way dialogue” whereby “the interviewer rules the interview” (p. 484). It is important, therefore, for the researcher to carefully consider the social interactions that are integral to the interviewing process and the possible impact these interactions may have on the credibility of an IDI study. For example, the trained interviewer will maximize the social interaction by utilizing positive engagement techniques such as establishing rapport (i.e., being approachable), asking thoughtful questions that indicate the interviewer is listening carefully to the interviewee, and knowing when to stay silent and let the interviewee talk freely.

Krumer-Nevo, M. (2002). The arena of othering: A life-story study with women living in poverty and social marginality. Qualitative Social Work , 1 (3), 303–318.

Kvale, S. (2006). Dominance through interviews and dialogues. Qualitative Inquiry , 12 (3), 480–500.

Image captured from: https://upgradedhumans.com/2015/10/21/a-mile-wide-and-an-inch-deep/

Share this:

• Click to share on Reddit (Opens in new window)
• Click to share on Twitter (Opens in new window)
• Click to share on LinkedIn (Opens in new window)
• Click to share on Facebook (Opens in new window)
• Click to share on Tumblr (Opens in new window)
• Click to email a link to a friend (Opens in new window)
• Click to print (Opens in new window)

Leave a comment Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed .

• Already have a WordPress.com account? Log in now.
• Subscribe Subscribed
• Copy shortlink
• Report this content
• View post in Reader
• Manage subscriptions
• Collapse this bar

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

The qualitative research interview

Affiliation.

• 1 Department of Family Medicine, University of Medicine and Dentistry at Robert Wood Johnson Medical School, Somerset, New Jersey 08873, USA. [email protected]
• PMID: 16573666
• DOI: 10.1111/j.1365-2929.2006.02418.x

Background: Interviews are among the most familiar strategies for collecting qualitative data. The different qualitative interviewing strategies in common use emerged from diverse disciplinary perspectives resulting in a wide variation among interviewing approaches. Unlike the highly structured survey interviews and questionnaires used in epidemiology and most health services research, we examine less structured interview strategies in which the person interviewed is more a participant in meaning making than a conduit from which information is retrieved.

Purpose: In this article we briefly review the more common qualitative interview methods and then focus on the widely used individual face-to-face in-depth interview, which seeks to foster learning about individual experiences and perspectives on a given set of issues. We discuss methods for conducting in-depth interviews and consider relevant ethical issues with particular regard to the rights and protection of the participants.

PubMed Disclaimer

Similar articles

• Breaching the wall: interviewing people from other cultures. Birks MJ, Chapman Y, Francis K. Birks MJ, et al. J Transcult Nurs. 2007 Apr;18(2):150-6. doi: 10.1177/1043659606298617. J Transcult Nurs. 2007. PMID: 17416717 Review.
• Methodological and ethical issues related to qualitative telephone interviews on sensitive topics. Mealer M, Jones Rn J. Mealer M, et al. Nurse Res. 2014 Mar;21(4):32-7. doi: 10.7748/nr2014.03.21.4.32.e1229. Nurse Res. 2014. PMID: 24673351
• Skype interviewing: the new generation of online synchronous interview in qualitative research. Janghorban R, Latifnejad Roudsari R, Taghipour A. Janghorban R, et al. Int J Qual Stud Health Well-being. 2014 Apr 15;9:24152. doi: 10.3402/qhw.v9.24152. eCollection 2014. Int J Qual Stud Health Well-being. 2014. PMID: 24746247 Free PMC article. Review.
• Interviewing to develop Patient-Reported Outcome (PRO) measures for clinical research: eliciting patients' experience. Brédart A, Marrel A, Abetz-Webb L, Lasch K, Acquadro C. Brédart A, et al. Health Qual Life Outcomes. 2014 Feb 5;12:15. doi: 10.1186/1477-7525-12-15. Health Qual Life Outcomes. 2014. PMID: 24499454 Free PMC article. Review.
• Use of interviews in nursing research. Mitchell G. Mitchell G. Nurs Stand. 2015 Jun 24;29(43):44-8. doi: 10.7748/ns.29.43.44.e8905. Nurs Stand. 2015. PMID: 26103871
• Human errors in emergency medical services: a qualitative analysis of contributing factors. Poranen A, Kouvonen A, Nordquist H. Poranen A, et al. Scand J Trauma Resusc Emerg Med. 2024 Aug 30;32(1):78. doi: 10.1186/s13049-024-01253-7. Scand J Trauma Resusc Emerg Med. 2024. PMID: 39215372
• Negotiating grey areas: an interview-based analysis of paramedic uncertainty and decision-making in cardiac arrest events. Gardiner G, Eli K, Huxley CJ, Fothergill R, Perkins GD, Smyth MA, Griffiths F, Slowther AM. Gardiner G, et al. BMC Emerg Med. 2024 Aug 29;24(1):154. doi: 10.1186/s12873-024-01057-z. BMC Emerg Med. 2024. PMID: 39198771
• Dispatcher nurses' experiences of handling drones equipped with automated external defibrillators in suspected out-of-hospital cardiac arrest - a qualitative study. Hanna DP, Erika B, Ellinor B, Sofia S, Leif S, Anette N, Jacob H, Andreas C. Hanna DP, et al. Scand J Trauma Resusc Emerg Med. 2024 Aug 21;32(1):74. doi: 10.1186/s13049-024-01246-6. Scand J Trauma Resusc Emerg Med. 2024. PMID: 39169425 Free PMC article.
• Reducing asthma attacks in disadvantaged school children with asthma: study protocol for a type 2 hybrid implementation-effectiveness trial (Better Asthma Control for Kids, BACK). Huebschmann AG, Wagner NM, Gleason M, Brinton JT, Brtnikova M, Brewer SE, Begum A, Armstrong R, DeCamp LR, McFarlane A, DeKeyser H, Coleman H, Federico MJ, Szefler SJ, Cicutto LC. Huebschmann AG, et al. Implement Sci. 2024 Aug 15;19(1):60. doi: 10.1186/s13012-024-01387-3. Implement Sci. 2024. PMID: 39148094 Free PMC article.
• Dental care staff's experience with risk assessment of dental erosion: a qualitative study. Todorov J, Shmarina E, Johannsen A. Todorov J, et al. BMC Oral Health. 2024 Aug 11;24(1):933. doi: 10.1186/s12903-024-04700-0. BMC Oral Health. 2024. PMID: 39129015 Free PMC article.

Publication types

• Search in MeSH

Related information

• Cited in Books

Grants and funding

• 1D12HP00167/PHS HHS/United States

LinkOut - more resources

Full text sources.

• Ovid Technologies, Inc.

• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Pardon Our Interruption

As you were browsing something about your browser made us think you were a bot. There are a few reasons this might happen:

• You've disabled JavaScript in your web browser.
• You're a power user moving through this website with super-human speed.
• You've disabled cookies in your web browser.
• A third-party browser plugin, such as Ghostery or NoScript, is preventing JavaScript from running. Additional information is available in this support article .

To regain access, please make sure that cookies and JavaScript are enabled before reloading the page.

Qualitative Interviewing

• Reference work entry
• First Online: 13 January 2019
• Cite this reference work entry

• Sally Nathan 2 ,
• Christy Newman 3 &
• Kari Lancaster 3  

4684 Accesses

26 Citations

8 Altmetric

Qualitative interviewing is a foundational method in qualitative research and is widely used in health research and the social sciences. Both qualitative semi-structured and in-depth unstructured interviews use verbal communication, mostly in face-to-face interactions, to collect data about the attitudes, beliefs, and experiences of participants. Interviews are an accessible, often affordable, and effective method to understand the socially situated world of research participants. The approach is typically informed by an interpretive framework where the data collected is not viewed as evidence of the truth or reality of a situation or experience but rather a context-bound subjective insight from the participants. The researcher needs to be open to new insights and to privilege the participant’s experience in data collection. The data from qualitative interviews is not generalizable, but its exploratory nature permits the collection of rich data which can answer questions about which little is already known. This chapter introduces the reader to qualitative interviewing, the range of traditions within which interviewing is utilized as a method, and highlights the advantages and some of the challenges and misconceptions in its application. The chapter also provides practical guidance on planning and conducting interview studies. Three case examples are presented to highlight the benefits and risks in the use of interviewing with different participants, providing situated insights as well as advice about how to go about learning to interview if you are a novice.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime
• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Interviews in the social sciences

Interviewing in Qualitative Research

Baez B. Confidentiality in qualitative research: reflections on secrets, power and agency. Qual Res. 2002;2(1):35–58. https://doi.org/10.1177/1468794102002001638 .

Article   Google Scholar  

Braun V, Clarke V. Successful qualitative research: a practical guide for beginners. London: Sage Publications; 2013.

Google Scholar  

Braun V, Clarke V, Gray D. Collecting qualitative data: a practical guide to textual, media and virtual techniques. Cambridge: Cambridge University Press; 2017.

Book   Google Scholar  

Bryman A. Social research methods. 5th ed. Oxford: Oxford University Press; 2016.

Crotty M. The foundations of social research: meaning and perspective in the research process. Australia: Allen & Unwin; 1998.

Davies MB. Doing a successful research project: using qualitative or quantitative methods. New York: Palgrave MacMillan; 2007.

Dickson-Swift V, James EL, Liamputtong P. Undertaking sensitive research in the health and social sciences. Cambridge: Cambridge University Press; 2008.

Foster M, Nathan S, Ferry M. The experience of drug-dependent adolescents in a therapeutic community. Drug Alcohol Rev. 2010;29(5):531–9.

Gillham B. The research interview. London: Continuum; 2000.

Glaser B, Strauss A. The discovery of grounded theory: strategies for qualitative research. Chicago: Aldine Publishing Company; 1967.

Hesse-Biber SN, Leavy P. In-depth interview. In: The practice of qualitative research. 2nd ed. Thousand Oaks: Sage Publications; 2011. p. 119–47

Irvine A. Duration, dominance and depth in telephone and face-to-face interviews: a comparative exploration. Int J Qual Methods. 2011;10(3):202–20.

Johnson JM. In-depth interviewing. In: Gubrium JF, Holstein JA, editors. Handbook of interview research: context and method. Thousand Oaks: Sage Publications; 2001.

Kvale S. Interviews: an introduction to qualitative research interviewing. Thousand Oaks: Sage; 1996.

Kvale S. Doing interviews. London: Sage Publications; 2007.

Lancaster K. Confidentiality, anonymity and power relations in elite interviewing: conducting qualitative policy research in a politicised domain. Int J Soc Res Methodol. 2017;20(1):93–103. https://doi.org/10.1080/13645579.2015.1123555 .

Leavy P. Method meets art: arts-based research practice. New York: Guilford Publications; 2015.

Liamputtong P. Researching the vulnerable: a guide to sensitive research methods. Thousand Oaks: Sage Publications; 2007.

Liamputtong P. Qualitative research methods. 4th ed. South Melbourne: Oxford University Press; 2013.

Mays N, Pope C. Quality in qualitative health research. In: Pope C, Mays N, editors. Qualitative research in health care. London: BMJ Books; 2000. p. 89–102.

McLellan E, MacQueen KM, Neidig JL. Beyond the qualitative interview: data preparation and transcription. Field Methods. 2003;15(1):63–84. https://doi.org/10.1177/1525822x02239573 .

Minichiello V, Aroni R, Hays T. In-depth interviewing: principles, techniques, analysis. 3rd ed. Sydney: Pearson Education Australia; 2008.

Morris ZS. The truth about interviewing elites. Politics. 2009;29(3):209–17. https://doi.org/10.1111/j.1467-9256.2009.01357.x .

Nathan S, Foster M, Ferry M. Peer and sexual relationships in the experience of drug-dependent adolescents in a therapeutic community. Drug Alcohol Rev. 2011;30(4):419–27.

National Health and Medical Research Council. National statement on ethical conduct in human research. Canberra: Australian Government; 2007.

Neal S, McLaughlin E. Researching up? Interviews, emotionality and policy-making elites. J Soc Policy. 2009;38(04):689–707. https://doi.org/10.1017/S0047279409990018 .

O’Reilly M, Parker N. ‘Unsatisfactory saturation’: a critical exploration of the notion of saturated sample sizes in qualitative research. Qual Res. 2013;13(2):190–7. https://doi.org/10.1177/1468794112446106 .

Ostrander S. “Surely you're not in this just to be helpful”: access, rapport and interviews in three studies of elites. In: Hertz R, Imber J, editors. Studying elites using qualitative methods. Thousand Oaks: Sage Publications; 1995. p. 133–50.

Chapter   Google Scholar  

Patton M. Qualitative research & evaluation methods: integrating theory and practice. Thousand Oaks: Sage Publications; 2015.

Punch KF. Introduction to social research: quantitative and qualitative approaches. London: Sage; 2005.

Rhodes T, Bernays S, Houmoller K. Parents who use drugs: accounting for damage and its limitation. Soc Sci Med. 2010;71(8):1489–97. https://doi.org/10.1016/j.socscimed.2010.07.028 .

Riessman CK. Narrative analysis. London: Sage; 1993.

Ritchie J. Not everything can be reduced to numbers. In: Berglund C, editor. Health research. Melbourne: Oxford University Press; 2001. p. 149–73.

Rubin H, Rubin I. Qualitative interviewing: the art of hearing data. 2nd ed. Thousand Oaks: Sage Publications; 2012.

Serry T, Liamputtong P. The in-depth interviewing method in health. In: Liamputtong P, editor. Research methods in health: foundations for evidence-based practice. 3rd ed. South Melbourne: Oxford University Press; 2017. p. 67–83.

Silverman D. Doing qualitative research. 5th ed. London: Sage; 2017.

Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (coreq): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57. https://doi.org/10.1093/intqhc/mzm042 .

Download references

Author information

Authors and affiliations.

School of Public Health and Community Medicine, Faculty of Medicine, UNSW, Sydney, NSW, Australia

Sally Nathan

Centre for Social Research in Health, Faculty of Arts and Social Sciences, UNSW, Sydney, NSW, Australia

Christy Newman & Kari Lancaster

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Sally Nathan .

Editor information

Editors and affiliations.

School of Science and Health, Western Sydney University, Penrith, NSW, Australia

Pranee Liamputtong

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Singapore Pte Ltd.

About this entry

Cite this entry.

Nathan, S., Newman, C., Lancaster, K. (2019). Qualitative Interviewing. In: Liamputtong, P. (eds) Handbook of Research Methods in Health Social Sciences. Springer, Singapore. https://doi.org/10.1007/978-981-10-5251-4_77

Download citation

DOI : https://doi.org/10.1007/978-981-10-5251-4_77

Published : 13 January 2019

Publisher Name : Springer, Singapore

Print ISBN : 978-981-10-5250-7

Online ISBN : 978-981-10-5251-4

eBook Packages : Social Sciences Reference Module Humanities and Social Sciences Reference Module Business, Economics and Social Sciences

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• Open access
• Published: 30 August 2024

Medical students in distress: a mixed methods approach to understanding the impact of debt on well-being

• Adrienne Yang 1   na1 ,
• Simone Langness 2   na1 ,
• Lara Chehab 1   na1 ,
• Nikhil Rajapuram 3 ,
• Li Zhang 4 &
• Amanda Sammann 1  

BMC Medical Education volume  24 , Article number:  947 ( 2024 ) Cite this article

Metrics details

Nearly three in four U.S. medical students graduate with debt in six-figure dollar amounts which impairs students emotionally and academically and impacts their career choices and lives long after graduation. Schools have yet to develop systems-level solutions to address the impact of debt on students’ well-being. The objectives of this study were to identify students at highest risk for debt-related stress, define the impact on medical students’ well-being, and to identify opportunities for intervention.

This was a mixed methods, cross-sectional study that used quantitative survey analysis and human-centered design (HCD). We performed a secondary analysis on a national multi-institutional survey on medical student wellbeing, including univariate and multivariate logistic regression, a comparison of logistic regression models with interaction terms, and analysis of free text responses. We also conducted semi-structured interviews with a sample of medical student respondents and non-student stakeholders to develop insights and design opportunities.

Independent risk factors for high debt-related stress included pre-clinical year (OR 1.75), underrepresented minority (OR 1.40), debt $20–100 K (OR 4.85), debt >$100K (OR 13.22), private school (OR 1.45), West Coast region (OR 1.57), and consideration of a leave of absence for wellbeing (OR 1.48). Mental health resource utilization ( p  = 0.968) and counselors ( p  = 0.640) were not protective factors against debt-related stress. HCD analysis produced 6 key insights providing additional context to the quantitative findings, and associated opportunities for intervention.

Conclusions

We used an innovative combination of quantitative survey analysis and in-depth HCD exploration to develop a multi-dimensional understanding of debt-related stress among medical students. This approach allowed us to identify significant risk factors impacting medical students experiencing debt-related stress, while providing context through stakeholder voices to identify opportunities for system-level solutions.

Peer Review reports

Introduction

Over the past few decades, it has become increasingly costly for aspiring physicians to attend medical school and pursue a career in medicine. Most recent data shows that 73% of medical students graduate with debt often amounting to six-Fig [ 1 ]. – an amount that is steadily increasing every year [ 2 ]. In 2020, the median cost of a four-year medical education in the United States (U.S.) was $250,222 for public and $330,180 for private school students [ 1 ] – a price that excludes collateral costs such as living, food, and lifestyle expenses. To meet these varied costs, students typically rely on financial support from their families, personal means, scholarships, or loans. Students are thereby graduating with more debt than ever before and staying indebted for longer, taking 10 to 20 years to repay their student loans regardless of specialty choice or residency length [ 1 ].

Unsurprisingly, higher debt burden has been negatively correlated with generalized severe distress among medical students [ 3 , 4 ], in turn jeopardizing their academic performance and potentially impacting their career choices [ 5 ]. Studies have found that medical students with higher debt relative to their peers were more likely to choose a specialty with a higher average annual income [ 5 ], less likely to plan to practice in underserved locations, and less likely to choose primary care specialties [ 4 ]. However, a survey of 2019 graduating medical students from 142 medical schools found that, when asked to rank factors that influenced their specialty choice, students ranked economic factors, including debt and income, at the bottom of the list. With this inconsistency in the literature, authors Youngclaus and Fresne declare that further studies and analysis are required to better understand this important relationship [ 1 ].

Unfortunately, debt and its negative effects disproportionately impact underrepresented minority (URM) students, including African Americans, Hispanic Americans, American Indian, Native Hawaiian, and Alaska Native [ 6 ], who generally have more debt than students who are White or Asian American [ 1 ]. In 2019, among medical school graduates who identified as Black, 91% reported having education debt, in comparison to the 73% reported by all graduates [ 1 ]. Additionally, Black medical school graduates experience a higher median education debt amount relative to other groups of students, with a median debt of $230,000 [ 1 ]. This inequitable distribution of debt disproportionately places financial-related stress on URM students [ 7 ], discouraging students from pursuing a medical education [ 8 ]. These deterring factors can lead to a physician workforce that lacks diversity and compromises health equity outcomes [ 9 ].

Limited literature exists to identify the impact of moderating variables on the relationship between debt and debt-related stress. Financial knowledge is found to be a strong predictor of self-efficacy and confidence in students’ financial management, leading to financial optimism and potentially alleviating debt stress [ 10 , 11 , 12 ]. Numerous studies list mindfulness practices, exercise, and connecting with loved ones as activities that promote well-being and reduce generalized stress among students [ 13 , 14 , 15 ]. However, to date, no studies have examined whether these types of stress-reducing activities, by alleviating generalized stress, reduce debt-related stress. Studies have not examined whether resources such as physician role models may act as a protective factor against debt-related stress.

Despite the growing recognition that debt burdens medical students emotionally and academically, we have yet to develop systemic solutions that target students’ unmet needs in this space. We performed the first multi-institutional national study on generalized stress among medical students, and found that debt burden was one of several risk factors for generalized stress among medical students [ 3 ]. The goal of this study is to build upon our findings by using a mixed methods approach combining rigorous survey analysis and human-centered design to develop an in-depth understanding of the impact that education debt has on medical students’ emotional and academic well-being and to identify opportunities for intervention.

We conducted a mixed methods, cross-sectional study that explored the impact of debt-related stress on US medical students’ well-being and professional development. This study was conducted at the University of California, San Francisco (UCSF). All activities were approved by the UCSF institutional review board, and informed consent was obtained verbally from participants prior to interviews. We performed a secondary analysis of the quantitative and qualitative results of the Medical Student Wellbeing Survey (MSWS), a national multi-institutional survey on medical student wellbeing administered between 2019 and 2020, to determine risk factors and moderating variables of debt-related stress. To further explore these variables, we used human-centered design (HCD), an approach to problem-solving that places users at the center of the research process in order to determine key pain points and unmet needs, and co-design solutions tailored to their unique context [ 16 ]. In this study, we performed in-depth, semi-structured interviews with a purposefully sampled cohort of medical students and a convenience sample of non-student stakeholders to determine key insights representing students’ unmet needs, and identified opportunities to ameliorate the impact of debt-related stress on medical students.

Quantitative data: the medical student wellbeing survey

The MSWS is a survey to assess medical student wellbeing that was administered from September 2019 to February 2020 to medical students actively enrolled in accredited US or Caribbean medical schools [ 3 ]. Respondents of the MSWS represent a national cohort of > 3,000 medical students from > 100 unique medical school programs. The MSWS utilizes a combination of validated survey questions, such as the Medical Student Wellbeing Index (MS-WBI), and questions based on foundations established from previously validated wellbeing survey methods [ 3 ]. Questions generally focused on student demographics, sources of stress during medical school, specialty consideration, and frequency in activities that promote wellbeing. Some questions ask students to rate physical, emotional, and social domains of wellbeing using a five-point Likert scale. Questions of interest from the MSWS included debt-related stress, generalized stress, intended specialty choice, and utilization of well-being resources and counselors. An additional variable investigated was average school tuition, which was determined by a review of publicly available data for each student’s listed medical school [ 17 ]. All data from the MSWS was de-identified for research purposes.

Stress: debt-related and generalized stress

Debt stress was assessed by the question, “How does financial debt affect your stress level?” Students responded using a five-point Likert scale from − 2 to 2: significant increase in stress (-2), mild increase (-1), no change (0), mild decrease (1), or significant decrease (2). Responses for this question were evaluated as a binary index of ‘high debt stress,’ defined as a response of − 2, versus ‘low debt stress,’ defined as a response of − 1 or 0. In addition, generalized stress from the MSWS was assessed by questions from the embedded MS-WBI, which produced a score. Previous studies have shown that the score can be used to create a binary index of distress: a score ≥ 4 has been associated with severe distress, and a score < 4 has been associated with no severe distress [ 18 ].

Intended specialty

We categorized students’ responses to intended specialty choice by competitiveness, using the 2018 National Resident Match Program data [ 19 ]. ‘High’ and ‘low’ competitiveness were defined as an average United States Medical Licensing Examination (USMLE) Step 1 score of > 240 or ≤ 230, respectively, or if > 18% or < 4% of applicants were unmatched, respectively. ‘Moderate’ competition was defined as any specialty not meeting criteria for either ‘high’ or ‘low’ competitiveness.

Resource utilization

The MSWS assessed the utilization of well-being resources by the question, “At your institution, which of the following well-being resources have you utilized? (Select all that apply)” Students responded by selecting each of the resource(s) they used: Mental Health and Counseling Services, Peer Mentorship, Self-Care Education, Mindfulness/Meditation Classes, Community Building Events, and Other. The number of choices that the student selected was calculated, allowing for placement into a category depending on the amount of resource utilization: 0–20%, 20–40%, 40–60%, 60–80%, 80–100%. Responses for this question were evaluated as a binary index of ‘high resource utilization,’ defined as a response of 80–100% resource utilization, versus ‘low resource utilization,’ defined as a response of < 80% resource utilization. The co-authors collaboratively decided upon this “top-box score approach,” [ 20 ] which is the sum of percentages for the most favorable top one, two or three highest categories on a scale, to assess if the most extreme users (80–100%) of these supportive resources experienced a decrease in debt-related stress. Additionally, use of a counselor for mental health support was assessed by the question, “Which of the following activities do you use to cope with difficult situations (or a difficult day on clinical rotation)? (Select all that apply).” Students responded by selecting the activities that they use from a list (e.g., listen to music, mindfulness practice, meet with a counselor, exercise). Responses for this question were evaluated as a binary index of ‘Meeting with a Counselor,’ defined by selection of that option, versus ‘Not Meeting with a Counselor,’ defined as not selecting that option.

Quantitative data analysis

We performed a secondary analysis of quantitative data from the MSWS to calculate frequencies and odds ratios for the five quantitative variables described above (debt-related stress, generalized stress, intended specialty, resource utilization, and school tuition). Tests performed are summarized in Table  1 (“Secondary Analysis Tests Performed”). Univariate analysis and multivariate logistic regression were performed among students in the high debt stress (-2) and low debt stress (0 or − 1) for select variables, such as clinical phase, URM, debt burden, specialty competitiveness, and average school tuition, to identify risk factors for high debt stress. To determine if ‘high resource utilization’ or ‘meeting with a counselor’ were moderating variables on the relationship between debt burden and debt stress, we applied the logistic regression with the interaction terms of ‘debt’ and ‘resource utilization’ (high vs. low). Then, we performed a similar analysis but replaced the interaction term with ‘debt’ and ‘meeting with a counselor’ (yes vs. no). We also performed Chi-squared tests to determine the degree to which severe distress increases as debt burden increases, if specialty competitiveness varied by debt stress, and if the proportion of students who identified as URM, in comparison to non-URM, differed by debt level. All statistical tests were two-sided and p  < 0.05 was considered significant. Statistical analyses were performed using SAS version 9.4 and R version 4.0.5.

Qualitative data: interviews and MSWS free text responses

Free-text entries.

At the conclusion of the 2019–2020 MSWS, respondents had unlimited text space to provide comments to two prompts. The first prompt read, “What well-being resource(s), if offered at your school, do you feel would be most useful?” The second prompt read “If you have any further comments to share, please write them below.” Answers to either prompt that pertained to debt, cost of medical school, or finances were extracted for the purpose of this study and analyzed with the other qualitative data subsequently described.

Interview selection & purposive sampling

Interview participants were identified from a repository of respondents to the MSWS who had attached their email address and expressed willingness at the time of the survey to be contacted for an interview [ 3 ]. Our recruitment period was between April 19, 2021 to July 2, 2021. The recruitment process involved sending invitations to all of the email addresses in the list to participate in a 45-minute interview on the topic of student debt and wellbeing. The invitation included a brief screening questionnaire asking students to report updates to questions that were previously asked in the MSWS (i.e.: clinical training year, marital status, dependents). Additional novel questions included primary financial support system, estimate of financial support systems’ household income in the last year, estimate of educational financial debt at conclusion of medical school, student’s plan for paying off debt, and degree of stress (using a Likert scale from 0 to 10) over current and future education debt.

Purposeful sampling of medical student stakeholders for interviews allowed us to maximize heterogeneity. We utilized the students’ responses to the brief screening questionnaire with their corresponding responses to demographic questions from the MSWS to select interviewees that varied by gender, race, presence of severe distress, type of medical school (public vs. private), region of school, and tuition level of school. The sampling ensured a diverse representation, in accordance with HCD methodology [ 21 ]. Brief descriptions of participant experiences are listed in Table  2 (“Interviewee Descriptors”). Students who were selected for interviews were sent a confirmation email to participate. Interviews were to be conducted until thematic saturation was reached. In addition, to include representation from the entire ecosystem, we interviewed a financial aid counselor at a medical school and a pre-medical student, chosen through convenience sampling. We directly contacted those two individuals for interviews.

Semi-structured interviews

All interviews were conducted between April 2021 and July 2021 over Zoom. A single researcher conducted interviews over an average of 45 min. Informed consent was obtained verbally from participants prior to interviews; interviews and their recordings only proceeded following verbal consent. The interview guide (S1 File) included open-ended questions about students’ experience of debt-related stress and their reflections on its consequences. The audio recordings were transcribed using Otter.ai, a secure online transcription service that converts audio files to searchable text files. Interview responses were redacted to preserve anonymity of respondent identity.

Qualitative data analysis

Interview data was analyzed using a general inductive approach to thematic analysis. Specifically, two researchers (SL and AY) independently inductively analyzed transcripts from the first three semi-structured interviews to come up with themes relating to the experiences and consequences of debt-related stress. They reconciled discrepancies in themes through discussion to create the codebook (S2 File), which included 18 themes. SL and AY independently coded each subsequent interview transcript as well as the free text responses from the survey, meeting to reach a consensus on representative quotes for applicable themes.

Following the HCD methodology, two researchers met with the core team to discuss the themes from the interviews and translate them into “insight statements”, which reflect key tensions and challenges experienced by stakeholders. Insight statements carefully articulate stakeholders’ unique perspectives and motivations in a way that is actionable for solution development [ 22 ]. As such, these insight statements are reframed into design opportunities, which suggest that multiple solutions are possible [ 23 , 24 ]. For example, discussion about themes 1a and 1b (“Questionable Job Security” and “Disappointing MD salary and Satisfaction Payoff”) revealed that they were related in the way that they led students to wonder whether the investment in medical school would be offset by the salary payoff. This led to the identification of the tension for low-income students in particular, who have to weigh this tradeoff earlier in their medical school journey than other students who are less financially-constrained (insight: “Medical school is a risky investment for low-income students”.) The design opportunity logically translates into a call to action for brainstorming and solution development: “Support low-income students to make values-based tradeoffs when considering a career in medicine.”

MSWS respondents and quantitative analysis

A total of 3,162 students responded to the MSWS and their sociodemographic characteristics have been described previously [ 3 ]. A total of 2,771 respondents (87.6%) responded to our study’s variables of interest, including a response for ‘high debt stress’ (–2) or ‘low debt stress’ (–1 or 0). Table  3 lists the distribution of debt-related stress across different variables for all respondents.

Risk factors for debt-related stress

Factors that were independently associated with higher debt-related stress included being in pre-clinical year (OR 1.75, 95% CI 1.30–2.36, p  < 0.001), identifying as URM (OR 1.40, 95% CI 1.03–1.88), p  = 0.029), having debt $20–100 K (OR 4.85, 95% CI 3.32–7.30, p  < 0.001), debt > 100 K (OR 13.22, 95% CI 9.05–19.90, p  < 0.001), attending a private medical school (OR 1.45, 95% CI 1.06–1.98, p  = 0.019), attending medical school on the West Coast (OR 1.57, 95% CI 1.17–2.13, p  = 0.003), and having considered taking a leave of absence for wellbeing (OR 1.48, 95% CI 1.13–1.93, p  = 0.004) (Table  4 , S1 Table).

Severe distress by debt amount

Levels of generalized severe distress differed across debt burden groups. As debt level increased, the percentage of individuals with “severe” distress increased ( p  < 0.001).

Debt and career decisions

There were significant differences between the high debt stress versus low debt stress groups and plans to pursue highly vs. moderately vs. minimally competitive specialties ( p  = 0.027) (Fig.  1 ) A greater percentage of low debt stress students were pursuing a highly competitive specialty or a minimally competitive specialty. A greater percentage of high debt stress students were pursuing a moderately competitive specialty. As shown in Table  4 , there were no differences in debt-associated stress between students who choose different specialties, such as medical versus surgical versus mixed (medical/surgical).

Debt stress by specialty competitiveness

URM students’ experience of debt

URM identity was an independent risk factor for higher debt-related stress (Table  4 ) In addition, debt levels varied between those who identify as URM versus non-URM ( p  < 0.001). Students identifying as URM tended to have higher debt than those who did not. Although the percentage of non-URM students was higher than that of URM students within the lowest debt burden category (<$20k), among all higher debt burden categories, including $20–100 K, $100–300 K, and >$300K, the percentage of URM students was higher than the percentage of non-URM students.

Moderating factors on the relationship between debt and debt stress

Protective factors such as high degree of mental health resource utilization and meeting with a counselor did not reduce the impact of debt burden on debt stress. Among students who reported a high degree of mental health resource utilization, there was no impact on the relationship between debt and debt stress ( p  = 0.968). Similarly, meeting with a counselor had no impact on the relationship between debt and debt stress ( p  = 0.640).

Interview respondents and qualitative analysis

We conducted in-depth, semi-structured interviews with 11 medical students, who are briefly described in Table  2 . We reached thematic saturation with 11 interviews, a point at which we found recurring themes. Therefore, no further interviews were needed. Among the medical student interviewees, there was representation from all regions, including the Northeast ( n  = 3), West Coast ( n  = 5), Midwest ( n  = 2), and South ( n  = 1). Students were also from all clinical phases, including pre-clinical ( n  = 3), clinical ( n  = 4), gap year/other ( n  = 2), and post-clinical ( n  = 2). Most interviewees were female ( n  = 8) and 5 of the interviewees identified as URM. Financial support systems were diverse, including self ( n  = 3), spouse/partner ( n  = 3), and parents/other ( n  = 5). Most interviewees reported low debt stress ( n  = 8), as opposed to high debt stress ( n  = 3). 55% of interviewees planned to pursue specialties that pay <$300K ( n  = 6), with some pursuing specialties that pay $300–400 K ( n  = 2) and >$400K ( n  = 3).

Among the MSWS free-text responses, to the prompt, “What well-being resource(s), if offered at your school, do you feel would be most useful?” 20 of 118 respondents (16.9%) provided free-text responses that pertained to debt, cost of medical school, or finances. To the prompt “If you have any further comments to share, please write them below” 11 of 342 students (3.2%) provided relevant free-text responses. Analysis of the free-text responses and semi-structured interviews revealed 6 distinct insights (Table  5 ), with each insight translated into an actionable design opportunity.

Medical school is a risky investment for low-income students.

Description

The personal and financial sacrifices required for low-income students to attend medical school and pursue a career in medicine outweigh the benefits of becoming a physician. When considering a career in medicine, students feel discouraged by questionable job security (theme 1a) and reduced financial compensation (theme 1b) – a combination that jeopardizes immediate and long-term job satisfaction. Some students feel hopeful that their decision to pursue medicine will be personally rewarding (1b.6) and their salaries will stabilize (1a.1, 1a.5), but many low-income students experience doubt about whether they made the right career choice (1b.2, 1b.4, 1b.6), and feel stressed that they will be in debt for longer than they expected (1a.3, 1a.4, 1b.1, 1b.5). Support low-income students to make values-based tradeoffs when considering a career in medicine.

Design opportunity

Support low-income students to make values-based tradeoffs when considering a career in medicine.

Medical schools lack the adaptive infrastructure to be welcoming to low-income students.

Students face financial challenges from the moment they apply to medical school (theme 2a), a costly process that limits admissions options for low-income students due to their inability to pay for numerous application fees (2a.1) and expensive test preparation courses (2a.2, 2a.3). Once students begin medical school, they feel unsupported in their varied responsibilities towards their families (theme 2b) and additional financial needs (theme 2c), requiring them to make tradeoffs with their education and personal lives (2b.2, 2c.1).

Design opportunity 2

Develop flexible systems that can recognize and accommodate students’ complex financial needs during medical school.

Students worry about the impact that their medical school debt has on their present and future families, which compounds feelings of guilt and anxiety.

For students who need to take loans, the decision to pursue a career in medicine is a collective investment with their families. Students feel guilty about the sacrifices their families have to make for the sake of their career (theme 3a) and feel pressure to continue to provide financially for their family while having debt (theme 3b). Students are stressed about acquiring more debt throughout their training (3a.1) and the impact that has on loved ones who are dependent on them (3a.4, 3a.5, 3b.2), especially with respect to ensuring their financial security in the future (3b.4).

Design opportunity 3

Create an environment that acknowledges and accounts for the burden of responsibility that students face towards their families.

Without the appropriate education about loans, the stress of debt is exponentially worse.

Students feel the greatest fear around loans when they do not understand them, including the process of securing loans and paying off debt (theme 4a). Students are overwhelmed by their loan amounts (4a.5) and lack the knowledge or resources to manage their debt (4a.1, 4a.2), making them uncertain about how they will become debt-free in the future (4a.3, 4a.4). Students reported that various resources helped to alleviate those burgeoning fears (theme 4b), including financial aid counselors (4b.2, 4b.3) and physician role models (4b.5, 4b.6) that generally increase knowledge and skills related to debt management (4b.1).

Design opportunity 4

Empower students to become experts in managing their debt by making loan-related resources more available and accessible.

The small, daily expenses are the most burdensome and cause the greatest amount of stress.

Students with educational debt are mentally unprepared for the burden of managing their daily living expenses (theme 5a), causing them to make significant lifestyle adjustments in the hopes to ease their resulting anxiety (theme 5b). These costs are immediate and tangible, compared to tuition costs which are more distant and require less frequent management (5a.3) Students learn to temper their expectations for living beyond a bare minimum during medical school (5a.1, 5b.2, 5b.4) and develop strategies to ensure that their necessary expenses are as low as possible (5b.1, 5b.2, 5b.3, 5b.4).

Design opportunity 5

Develop and distribute resources to support both short- and long-term financial costs for medical students.

Students view debt as a dark cloud that constrains their mental health and dictates their career trajectory.

The constant burden of educational debt constrains students’ abilities to control their mental health (theme 6a) and pursue their desired career path in medicine (themes 6b & 6c). Students feel controlled by their debt (6a.3) and concerned that it will impact their [ability] to live a personally fulfilling life (6a.1, 6a.2, 6c.6), especially with respect to pursuing their desired medical specialties (6b.1, 6c.3, 6c.5, 6c.6). Students with scholarships, as opposed to loans, felt more able to choose specialties that prioritized their values rather than their finances (6c.1, 6c.2), an affordance that impacts long-term career growth and satisfaction.

Design opportunity 6

Create a culture of confidence for managing debt and debt-stress among medical students.

This is the first multi-institutional national study to explore the impact of debt-related stress on medical students’ well-being in the United States. We used an innovative, mixed methods approach to better understand the factors that significantly affect debt-related stress, and propose opportunities for improving medical student well-being.

URM students

Analysis of survey results found that students who identify as URM are more likely to experience higher levels of debt-related stress than non-URM students. Our study also found that among all higher debt burden categories, debt levels were higher for URM students, findings consistent with studies that have shown the disproportionate burden of debt among URM students [ 1 ]. Our semi-structured interviews illuminated that students from low-income backgrounds feel unsupported by their medical schools in these varied financial stressors that extend beyond tuition costs (insight 2), leaving their needs unmet and increasing financial stress over time: “We don’t have different socio-economic classes in medicine because there’s constantly a cost that [isn’t] even factored into tuition cost [and] that we can’t take student loans for.” Many URM students feel especially stressed by their financial obligations towards their families (insight 3), and describe the decision to enter into medicine as one that is collective ( “the family’s going to school” ) rather than individual, placing additional pressure on themselves to succeed in their career: “ Being of low SES , the most significant stressor for me is the financing of medical school and the pull of responsibility for my family.” Several other studies from the literature confirm that students who identify as URM and first generation college or medical students are at higher risk for financial stress compared to their counterparts [ 7 ], and report that they feel as though it is their responsibility to honor their families through their educational and career pursuits [ 25 ]. Our study demonstrates and describes how low-income and URM students face numerous financial barriers in medical school, resulting in medical trainees that are less diverse than the patient populations they are serving [ 1 , 8 ].

Debt amount

Our quantitative analysis found that students with debt amounts over $100,000 are at much higher risk for experiencing severe stress than students with debt less than that amount. Although this finding may seem intuitive, it is important to highlight the degree to which this risk differs between these two cohorts. Students with debt amounts between $20,000 and $100,000 are approximately 5 times more likely to experience high stress than students with debt less than $20,000, while students with debt amounts over $100,000 are approximately 13 times more likely to experience severe stress when compared to the same cohort. Interview participants describe that the more debt they have, the less hopeful they feel towards achieving financial security (insight 1): “There are other healthcare professionals that will not accrue the same amount of loans that we will , and then may or may not have the same salary or privileges […] makes me question , did I do the right thing?” Students internalize this rising stress so as not to shift the feelings of guilt onto their families (insight 3), thereby compounding the psychological burden associated with large amounts of debt (insight 6): “As long as you’re in debt , you’re owned by someone or something and the sooner you can get out of it , the better; the sooner I can get started with my life.”

Pre-clinical students

According to our survey analysis, students who are in their pre-clinical years are at higher risk for stress than students in their clinical years. Our interview findings from insight 4 suggest that students feel initially overwhelmed and unsure about what questions to ask ( “One of my fears is that I don’t know what I don’t know”) or how to manage their loans so that it doesn’t have a permanent impact on their lives: “The biggest worry is , what if [the debt] becomes so large that I am never able to pay it off and it ends up ruining me financially.” Pre-clinical students may therefore feel unsure or ill-equipped to manage their loans, making them feel overwhelmed by the initial stimulus of debt. By the time students reach their clinical years, they may have had time to develop strategies for managing stress, acquire more financial knowledge, and/or normalize the idea of having debt.

Medical school characteristics

Our survey analysis found several risk factors related to medical school characteristics. First, we found that students who attended a private school were at higher risk for debt-related stress than students who attended a public school. Not only is the median 4-year cost of attendance in 2023 almost $100,000 higher in private compared to public medical schools [ 26 ], but it is also the case that financial aid packages are more liberally available for public schools due to state government funding [ 27 ]. This not only relieves students from having higher amounts of debt, but it also creates a more inclusive cohort of medical students. Insight 2 from our interviews suggests that private medical schools without the infrastructure to meet students’ varying financial needs force low-income students to make tradeoffs between their education and personal lives.

Another characteristic that was found to be a risk factor for debt stress was attending a medical school on the West Coast (compared to a non-coastal school.) This was a surprising finding given that tuition rates for both private and public schools on the West Coast are no higher than those in other regions [ 17 ]. The distribution of survey respondents did not vary significantly across regional categories, so no bias in sample size is suspected. While these interviews were not designed to address the reasoning behind students’ choice of medical school matriculation, there is a potential explanation for this finding. Historically, students match for residency programs that are in their home state or not far from their home state; [ 28 , 29 ] therefore, we speculate that students may prefer to settle on the West Coast, and may be willing to take on more financial debt in pursuit of their long-term practice and lifestyle goals.

Our quantitative analysis found that students who reported having considered taking a leave of absence for well-being purposes were at higher risk for debt-related stress. This cohort of students likely experience higher levels of stress as they are conscious of the negative impact it has on their life, and have already ruminated on leaving medical school. A study by Fallar et al. found that the period leading up to a leave of absence is particularly stressful for students because they are unfamiliar with the logistics of taking time off, and don’t feel as though leaving medical school is encouraged or normalized for students [ 30 ]. An interview with a student who did a joint MD and PhD program expressed having more time for herself during her PhD program, and described using money for activities that could alleviate stress (“I took figure skating during my PhD”) rather than create more stress by compromising on their lifestyle during medical school (insight 5). More research may be needed to better understand and support students considering taking a leave of absence from medical school.

• Specialty choice

Our study found that students with high debt stress pursue moderately competitive specialties compared to students with low debt stress. This may be explained by the fact that low debt stress gives students the freedom to pursue minimally competitive specialties, which may be more fulfilling to them but typically have lower salaries. Insight 6 further elaborates upon this finding that students with high debt stress deprioritize specialties for which they are passionate in favor of higher paying specialties that might alleviate their debt: “I love working with kids…but being an outpatient pediatrician just wasn’t going to be enough to justify the [private school] price tag.” Students with lower debt stress describe having the freedom to choose specialties that align with their values, regardless of anticipated salary: “Scholarships give me the freedom to do [specialties] that maybe are a little bit less well-paying in medicine.” Interestingly, certain studies examining the relationship between specialty choice and debt stress have found that high debt stress is associated with a higher likelihood of pursuing a more competitive, and presumably higher paying, specialty [ 5 ]. More research investigating the relationship between debt stress and specialty choice could illuminate opportunities for increasing a sense of agency and overall satisfaction among students for their career choices.

In our exploration of potential protective factors against the effects of debt-related stress, our survey analysis found that the two variables measured (high mental health resource utilization and meeting with a counselor) did not have any impact on reducing debt-related stress. This finding is inconsistent with the literature, which considers these activities to promote general well-being among students but has never been studied in the context of debt-related stress [ 13 , 14 , 15 ]. A potential explanation is that the survey questions that assessed these activities were imperfect. For example, the question of meeting with a counselor was not a standalone question, but instead, was at the bottom of a list of other wellbeing activities; therefore, students may have been fatigued by the time they got to the bottom of the list and not selected it. Additionally, our definition of “high” mental health resource utilization may have been perceived as too strict (i.e.: 80–100%) and perhaps we would have seen effects at lower percentages of utilization (i.e.: 40–60%). Despite this finding, students describe in their interviews that having access to certain resources such as financial knowledge and physician role models can help to alleviate stress by helping them feel confident in managing their loans in the immediate and more distant future (insight 4): “I’ve had explicit discussions with physicians who went to med school , had debt , paid it off [.] the debt hasn’t hindered their life in any way. I think that just makes me feel a lot calmer.” This finding aligns with previous studies that suggest that financial knowledge, such as knowledge about loans and a payoff plan, confers confidence in students’ financial management [ 11 , 12 ]. These factors are also aligned with previous studies that suggest financial optimism, such as with a physician role model who successfully paid off loans, is associated with less financial stress [ 10 ].

Our quantitative analysis of risk factors helped us to identify which areas might significantly impact debt-related stress among medical students, while our qualitative analysis provided more in-depth insight into those risk factors for more human-centered intervention design. The HCD process not only provides additional context from the perspective of medical students, but also proposes distinct design opportunities upon which interventions may be designed and tested. Drawing from the six design opportunities outlined in this paper, we propose a solution on a national scale: lowering the cost of the MCAT and medical school applications to reduce the financial barrier to applying to medical school [ 31 ]. We also propose the following solutions that can be implemented at the level of medical schools to better support medical students facing debt-related stress: (1) providing adequate financial aid that prevents low-income students from needing to work while being in medical school [ 32 ], (2) providing targeted financial planning classes and counseling for first-year medical students who have taken loans [ 33 ], and (3) creating mentorship programs that pair medical students with debt with physician role models who had also had debt but successfully paid it off [ 34 ]. We encourage medical schools to consider these suggestions, choosing the ideas from the list that make sense and tailoring them as necessary for their students and their unique needs. Additionally, given that our quantitative portion of the study was a secondary analysis of a survey focused on general medical student well-being, a nationwide study is needed that is specifically designed to explore the topic of debt-related stress among medical students. Furthermore, more research is needed that assesses the impact of activities that promote well-being (e.g., access to therapy, mindfulness practices, exercise) on debt-related stress among medical students.

Limitations

Our study had some notable limitations. One potential limitation is that our data collection occurred between 2019 and 2021 for this publication in 2023. Additionally, as described in the original study [ 3 ], a limitation of the MSWS is the inability to determine a response rate of students due to the survey distribution by medical student liaisons from each medical school; under the reasonable assumption that the survey was distributed to every US allopathic medical student, the response rate was estimated to have been 8.7%. 3 An additional limitation is the potential for response bias [ 3 ]. A limitation of the qualitative interviews is the potential for response bias among the interviewees. Although we purposely sampled, the students who accepted the invitation to interview may have been students with extreme views, either very negative views of debt or very neutral views of debt. Additionally, the interviewees were not representative of all possible financial situations, given that most students were from private schools, which typically have higher tuition rates. Also, all students had debt amounts in the middle and high categories, with none in the low category. Finally, our model of risk factors for debt-related stress suggested the presence of negative confounding factors, which exerted effects on specific variables (i.e.: pre-clinical year, West Coast) for which univariate analysis found no significant associations but multivariate analysis did. We did not perform further analysis to identify which variables served as the negative confounding variables.

In conclusion, our mixed methods, cross-sectional study exploring debt-related stress and its impact on US medical students’ wellbeing and professional development revealed a set of risk factors and design opportunities for intervention. By using a combined quantitative and qualitative HCD approach, we were able to develop a broad, in-depth understanding of the challenges and opportunities facing medical students with education debt. With these efforts to support the well-being and academic success of students at higher risk of debt-related stress, medical education institutions can develop and nurture a more diverse medical field that can best support the needs of future patients.

Data availability

Data is provided within the supplementary information files.

Youngclaus J, Julie Fresne. Physician education debt and the cost to attend medical school: 2020 update. Association of American Medical Colleges; 2020.

Association of American Medical Colleges. Medical school graduation questionnaire: 2020 all schools summary report. 2020 [cited 2023 Sep 7]. https://www.aamc.org/data-reports/students-residents/report/graduation-questionnaire-gq

Rajapuram N, Langness S, Marshall MR, Sammann A. Medical students in distress: the impact of gender, race, debt, and disability. PLoS ONE. 2020;15(12):e0243250.

Article   Google Scholar  

Rohlfing J, Navarro R, Maniya OZ, Hughes BD, Rogalsky DK. Medical student debt and major life choices other than specialty. Med Educ Online. 2014;19. https://doi.org/10.3402/meo.v19.25603

Pisaniello MS, Asahina AT, Bacchi S, Wagner M, Perry SW, Wong ML, et al. Effect of medical student debt on mental health, academic performance and specialty choice: a systematic review. BMJ Open. 2019;9(7):e029980.

AAMC. [cited 2023 Oct 18]. Unique populations. https://www.aamc.org/professional-development/affinity-groups/gfa/unique-populations

McMichael B, Lee IVA, Fallon B, Matusko N, Sandhu G. Racial and socioeconomic inequity in the financial stress of medical school. MedEdPublish (2016). 2022;12:3.

Mclnturff B. E. Frontczak. Medical school applicant survey. 2004.

Morrison E, Grbic D. Dimensions of diversity and perception of having learned from individuals from different backgrounds: the particular importance of racial diversity. Acad Med. 2015;90(7):937.

Heckman S, Lim H, Montalto C. Factors related to financial stress among college students. J Financial Therapy. 2014;5(1):19–39.

Heckman SJ, Grable JE. Testing the role of parental debt attitudes, student income, dependency status, and financial knowledge have in shaping financial self-efficacy among college students. Coll Student J. 2011;45(1):51–64.

Google Scholar  

Gillen M, Loeffler DN. Financial literacy and social work students: knowledge is power. Journal of Financial Therapy. 2012 [cited 2023 Sep 13];3(2). https://newprairiepress.org/jft/vol3/iss2/4

Conley CS, Durlak JA, Dickson DA. An evaluative review of outcome research on universal mental health promotion and prevention programs for higher education students. J Am Coll Health. 2013;61(5):286–301.

Luken M, Sammons A. Systematic review of mindfulness practice for reducing job burnout. Am J Occup Ther. 2016;70(2):p70022500201–10.

Weight CJ, Sellon JL, Lessard-Anderson CR, Shanafelt TD, Olsen KD, Laskowski ER. Physical activity, quality of life, and burnout among physician trainees: the effect of a team-based, incentivized exercise program. Mayo Clin Proc. 2013;88(12):1435–42.

Design. Kit. [cited 2023 Oct 2]. What is human-centered design? https://www.designkit.org/human-centered-design.html

AAMC. Tuition and student, fees reports. 2006–2013 tuition and student fees report. https://www.aamc.org/data-reports/reporting-tools/report/tuition-and-student-fees-reports

Dyrbye LN, Schwartz A, Downing SM, Szydlo DW, Sloan JA, Shanafelt TD. Efficacy of a brief screening tool to identify medical students in distress. Acad Med. 2011;86(7):907–14.

Charting outcomes in the match. U.S. allopathic seniors (Characteristics of U.S. allopathic seniors who matched to their preferred specialty in the 2018 main residency match). [cited 2023 Sep 18]. https://www.nrmp.org/wp-content/uploads/2021/07/Charting-Outcomes-in-the-Match-2018_Seniors-1.pdf

Greg Timpany. Top-box score – deriving a new measure. QuestionPro. https://www.questionpro.com/blog/creating-a-top-box-score/

Design. Kit. [cited 2023 Oct 9]. Extremes and mainstreams. https://www.designkit.org/methods/extremes-and-mainstreams.html

Design Kit. [cited 2023 Sep 18]. Create insight statements. https://www.designkit.org/methods/create-insight-statements.html

Designing an information and communications technology tool with and for victims of violence and their case managers in San Francisco. Human-centered design study. [cited 2024 Aug 12]. https://mhealth.jmir.org/2020/8/e15866

Bridge Innovate ® . 2017 [cited 2024 Aug 12]. Turning insights into opportunities. https://www.bridgeinnovate.com/blog/2017/12/19/turning-insights-into-opportunities

Van T, Bui K. First-generation college students at a four-year university: background characteristics, reasons for pursuing higher education, and first-year experiences. Coll Student J. 2002;36(1):3.

Students. & Residents. [cited 2023 Oct 18]. You can afford medical school. https://students-residents.aamc.org/financial-aid-resources/you-can-afford-medical-school

Medical schools with best financial aid. in 2023 | BeMo ® . [cited 2023 Oct 18]. https://bemoacademicconsulting.com/blog/medical-schools-with-best-financial-aid

Hasnie A, Hasnie U, Nelson B, Aldana I, Estrada C, Williams W. Relationship between residency match distance from medical school and virtual application, school characteristics, and specialty competitiveness. Cureus. 15(5):e38782.

Dorner FH, Burr RM, Tucker SL. The geographic relationships between physicians’ residency sites and the locations of their first practices. Acad Med. 1991;66(9):540.

Fallar R, Leikauf J, Dokun O, Anand S, Gliatto P, Mellman L, et al. Medical students’ experiences of unplanned leaves of absence. Med Sci Educ. 2019;29(4):1003–11.

Millo L, Ho N, Ubel PA. The cost of applying to medical school — a barrier to diversifying the profession. N Engl J Med. 2019;381(16):1505–8.

Should you work during medical school? | Medical School Admissions Doctor | U.S. News. [cited 2024 Jul 27]. https://www.usnews.com/education/blogs/medical-school-admissions-doctor/articles/should-you-work-during-medical-school

First year in medical school?. Here’s your financial checklist | American Medical Association. [cited 2024 Jul 27]. https://www.ama-assn.org/medical-students/medical-student-finance/first-year-medical-school-heres-your-financial-checklist

The White Coat Investor - Investing. & Personal Finance for Doctors. [cited 2024 Jul 27]. Physician Personal Finance | White Coat Investor. https://www.whitecoatinvestor.com/personal-finance-for-doctors/

Download references

Acknowledgements

We thank the members of The Better Lab, including Devika Patel, Christiana Von Hippel, and Marianna Salvatori, for their support. We appreciate Pamela Derish (UCSF) for assistance in manuscript editing and the UCSF Clinical and Translational Science Institute (CTSI) for assistance in statistical analysis. This publication was supported by the National Center for Advancing Translational Sciences, National Institutes of Health, through UCSF-CTSI Grant Number UL1 TR001872. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH.

Funding was not obtained for this project.

Author information

Adrienne Yang, Simone Langness and Lara Chehab contributed equally to this work.

Authors and Affiliations

Department of Surgery, University of California, San Francisco, CA, USA

Adrienne Yang, Lara Chehab & Amanda Sammann

Department of Trauma Surgery, Sharp HealthCare, San Diego, CA, USA

Simone Langness

Department of Pediatrics, Stanford University, Stanford, CA, USA

Nikhil Rajapuram

Department of Epidemiology and Biostatistics, University of California, San Francisco, CA, USA

You can also search for this author in PubMed   Google Scholar

Contributions

A.Y. and L.C. wrote the main manuscript text and prepared the figures. S.L. created the study design. All authors reviewed the manuscript.

Corresponding author

Correspondence to Adrienne Yang .

Ethics declarations

Ethics approval and consent to participate.

All activities conducted as part of this study were approved by the UCSF Institutional Review Board. Informed consent was obtained verbally from all participants prior to conducting interviews.

Consent for publication

We, the authors, consent to the publication of this manuscript.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, supplementary material 3, supplementary material 4, supplementary material 5, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Yang, A., Langness, S., Chehab, L. et al. Medical students in distress: a mixed methods approach to understanding the impact of debt on well-being. BMC Med Educ 24 , 947 (2024). https://doi.org/10.1186/s12909-024-05927-9

Download citation

Received : 12 April 2024

Accepted : 19 August 2024

Published : 30 August 2024

DOI : https://doi.org/10.1186/s12909-024-05927-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

BMC Medical Education

ISSN: 1472-6920

• General enquiries: [email protected]

Vaccine decision-making among pregnant women: a protocol for a cross-sectional mixed-method study in Brazil, Ghana, Kenya and Pakistan

Jessica L Schue Roles: Data Curation, Project Administration, Software, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing Prachi Singh Roles: Data Curation, Project Administration, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing Berhaun Fesshaye Roles: Data Curation, Project Administration, Software, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing Emily S Miller Roles: Data Curation, Project Administration, Software, Writing – Original Draft Preparation, Writing – Review & Editing Shanelle Quinn Roles: Data Curation, Writing – Review & Editing Ruth A Karron Roles: Conceptualization, Funding Acquisition, Writing – Review & Editing Renato T Souza Roles: Data Curation, Methodology, Project Administration, Resources, Software, Supervision, Writing – Review & Editing Maria Laura Costa Roles: Supervision, Writing – Review & Editing Jose Guilherme Cecatti Roles: Supervision, Writing – Review & Editing Kwasi Torpey Roles: Methodology, Project Administration, Resources, Supervision, Writing – Review & Editing Caroline Dinam Badzi Roles: Project Administration, Supervision, Writing – Review & Editing Emefa Modey Roles: Supervision, Writing – Review & Editing Chris Guure Roles: Supervision, Writing – Review & Editing Ferdinand Okwaro Roles: Methodology, Project Administration, Resources, Supervision, Writing – Review & Editing Marleen Temmerman Roles: Resources, Supervision, Writing – Review & Editing Saleem Jessani Roles: Methodology, Project Administration, Resources, Supervision, Writing – Review & Editing Sarah Saleem Roles: Project Administration, Resources, Supervision, Writing – Review & Editing Muhammad Asim Roles: Project Administration, Supervision, Writing – Review & Editing Sidrah Nausheen Roles: Project Administration, Supervision, Writing – Review & Editing Haleema Yasmeen Roles: Project Administration, Supervision, Writing – Review & Editing Grace Belayneh Roles: Project Administration, Writing – Review & Editing Vanessa Brizuela Roles: Conceptualization, Methodology, Project Administration, Resources, Writing – Review & Editing Sami Gottlieb Roles: Conceptualization, Methodology, Project Administration, Resources, Writing – Review & Editing Rupali J Limaye Roles: Conceptualization, Funding Acquisition, Methodology, Project Administration, Resources, Writing – Original Draft Preparation, Writing – Review & Editing

Maternal immunization is a critical strategy to prevent both maternal and infant morbidity and mortality from several infectious diseases. When the first COVID-19 vaccines became available during the pandemic, there was mixed messaging and confusion amongst the broader public and among those associated with health care systems about the recommendations for COVID-19 vaccinations in pregnancy in many countries. A multi-country, mixed-methods study is being undertaken to describe how vaccine decision-making occurs amongst pregnant and postpartum women, with a focus on COVID-19 vaccines. The study is being conducted in Brazil, Ghana, Kenya, and Pakistan. In each country, participants are being recruited from either 2 or 3 maternity hospitals and/or clinics that represent a diverse population in terms of socio-economic and urban/rural status. Data collection includes cross-sectional surveys in pregnant women and semi-structured in-depth interviews with both pregnant and postpartum women. The instruments were designed to identify attitudinal, behavioral, and social correlates of vaccine uptake during and after pregnancy, including the decision-making process related to COVID-19 vaccines, and constructs such as risk perception, self-efficacy, vaccine intentions, and social norms. The aim is to recruit 400 participants for the survey and 50 for the interviews in each country. Qualitative data will be analyzed using a grounded theory approach. Quantitative data will be analyzed using descriptive statistics, latent variable analysis, and prediction modelling. Both the quantitative and qualitative data will be used to explore differences in attitudes and behaviors around maternal immunization across pregnancy trimesters and the postpartum period among and within countries. Each country has planned dissemination activities to share the study findings with relevant stakeholders in the communities from which the data is collected and to conduct country-specific secondary analyses.

COVID-19, pregnancy, maternal immunization, Brazil, Ghana, Kenya, Pakistan

Introduction

Vaccination during pregnancy can be recommended for a variety of reasons: to prevent disease in the pregnant woman, to protect the fetus and prevent pregnancy complications, and to decrease morbidity and mortality in women, newborns and infants. Maternal immunization can compensate for newborns’ inexperienced immune systems, by allowing the mother to transmit protective antibodies to her baby via the placenta or breast milk ( Röbl-Mathieu et al. , 2021 ). Additionally, antibodies transferred from parent to child either during pregnancy or after childbirth play a crucial role in decreasing morbidity and mortality in newborns and infants ( Marchant et al. , 2017 ). In the case of COVID-19, a meta-analysis found that immunization reduces the risk of hypertensive disorders in pregnancy, reduces the likelihood of caesarean section, and reduces a newborn’s risk of being admitted to the neonatal intensive care unit ( Fernández-García et al. , 2024 ). Vaccinating pregnant women is currently recommended for a variety of diseases, including tetanus, pertussis, influenza, hepatitis B and COVID-19, and additional maternal vaccines are expected to be introduced in the coming years ( Geoghegan et al. , 2022 ; Limaye et al. , 2024 ). Maternal vaccines can serve as a crucial prevention tool for common diseases in infancy, such as Group B streptococcus, where currently available screening and/or treatment are complex and may be further challenged by health system constraints, or where births frequently occur outside of health facilities ( Rao & Khanna, 2020 ).

However, despite the congruence of evidence and policies supporting the safety and benefits of several maternal vaccines, there remains considerable disparity in their use and coverage both among and within countries ( Laenen et al. , 2015 ; Sobanjo-ter Meulen et al. , 2019 ). Attitudes and decision-making regarding maternal immunizations are complex; pregnant women must weigh the risk-benefit ratio for both themselves and their fetus ( Cox et al. , 2023 ). There are a multitude of factors that influence maternal immunization decision-making, among these are the opinions and recommendations of family and healthcare providers ( Cox et al. , 2023 ; Kilich et al. , 2020 ; Limaye et al. , 2022 ). Immunization decision-making while pregnant and in the postpartum period is also influenced by other factors, such as risk perception, knowledge of the disease and vaccine, social norms, and self-efficacy, to name a few ( Cox et al. , 2023 ; Kilich et al. , 2020 ).

During the height of the pandemic, pregnant women with COVID-19 were shown to be at greater risk of severe disease, hospital admission, and pre-term birth ( Allotey et al. , 2020 ; Smith et al. , 2023 ). But with the exclusion of pregnant individuals from the vast majority of COVID-19 vaccine trials, there was limited early vaccine safety data for this population and large variation in countries’ initial policy recommendations for COVID-19 vaccine use in pregnancy ( Hameed et al. , 2023 ; Zavala et al. , 2022 ). Over time, the availability of additional vaccine safety and effectiveness data for pregnant women led to more countries recommending or permitting the use of COVID-19 vaccines during pregnancy ( Hameed et al. , 2023 ; Prasad et al. , 2022 ; Wang et al. , 2022 ; Zavala et al. , 2022 ). But these varying and changing policies gave considerable latitude in the way local advisory groups and managers interpreted vaccine recommendations. The World Health Organization (WHO) now recommends a dose of COVID-19 vaccine to be given during each pregnancy ( World Health Organization, 2023 ). However, even where COVID-19 vaccination during pregnancy has been strongly encouraged, uptake has been sluggish ( Blakeway et al. , 2022 ; Goncu Ayhan et al. , 2021 ; Razzaghi et al. , 2021 ; Shamshirsaz et al. , 2022 ).

To better inform demand generation and communication strategies for vaccines in pregnancy, it is crucial to address several knowledge gaps and gather information from pregnant and postpartum women to understand factors that influence their vaccine decision-making process. This paper describes the protocol and early implementation for a mixed methods study to better understand how COVID-19 vaccine decision-making occurs, including attitudes about maternal immunization more broadly, among pregnant and postpartum women in Brazil, Ghana, Kenya, and Pakistan. The study includes five objectives ( Figure 1 ) with an aim to strengthen guidance, policy, and programs related to COVID-19 vaccination of pregnant women, especially in low- and middle-income countries.

Figure 1. Objectives for a multi-country, mixed methods, cross-sectional study.

Study design.

This descriptive study aims to understand COVID-19 vaccine decision-making amongst pregnant women. The study objectives are being addressed using mixed methods across four countries consisting of cross-sectional quantitative surveys among pregnant women and qualitative semi-structured in-depth interviews with both pregnant and postpartum women. The multi-country study team consists of an interdisciplinary group of researchers and policy makers with expertise in vaccine and behavioral science, obstetrics and nursing, maternal and child health, epidemiology, and biostatistics, as well as mixed method study design and data collection expertise in both quantitative and qualitative methods.

This document uses the term ‘pregnant women’. Although most people who are or can get pregnant are cisgender women who were born and identify as female, these topics are also relevant to the experiences of transgender men and other gender diverse people who may have the capacity to become pregnant.

Study locations

Each of the four countries included in this study was chosen based on participation in a WHO-led multi-country cohort study of COVID-19 in pregnancy ( Broutet & Thorson, 2022 ) and various other factors when it was conceptualized in 2021, including geographic diversity, varying COVID-19 vaccine policies related to pregnant women, diversity of COVID-19 vaccine products available, phase of the COVID-19 epidemic, and country interest. Study locations within each country vary by clinic type, clinic level, and the population served. Within each country, sites were selected to ensure inclusion of perspectives from people living in urban and rural settings, from high and low socio-economic status, and/or seeking care at private or public clinics. ( Figure 2 )

Figure 2. Study locations and clinic names in Brazil, Ghana, Kenya, and Pakistan.

Brazil first introduced the COVID-19 vaccine in January 2021, and vaccination was only recommended for pregnant and lactating women with comorbidities who underwent a risk-benefit assessment by their physicians starting in March 2021 ( Covas et al. , 2023 ; Secretaria Extraordinária de Enfrentamento à COVID-19 Gabinete, 2021 ). Following the death of a pregnant Brazilian woman after receiving a dose of the AstraZeneca/Oxford (AZO) vaccine, the AZO vaccine was prohibited for use for pregnant women in May 2021 ( Covas et al. , 2023 ; Fonseca & Brito, 2021 ; Kobayashi et al. , 2022 ). Starting in September 2021, the Brazilian Ministry of Health (MoH) recommended Pfizer/BioNTech and Sinovac for all pregnant and lactating individuals, and the MoH continues to include pregnant and lactating women in their recommended COVID-19 vaccination schedules.

Study sites in Brazil include two maternity hospitals in São Paulo, CAISM/Unicamp Hospital in Campinas, and Hospital Universitario de Jundiaí in Jundiaí, both public hospitals caring for pregnant women from urban and semirural areas and covered by the National Health Systems (SUS) and also private insurances.

Ghana was the first country to receive vaccines from the COVAX Facility in February 2021 ( UNICEF & World Health Organization, 2024 ). However, pregnant women were not included in the initial vaccine rollout, which focused on health workers and those with comorbidities, nor in the next two phases which expanded recommendations to include all adults over 18 years throughout 2021 ( The World Bank, 2021 ). Ghana only recommended COVID-19 vaccination for pregnant and lactating individuals after January 20, 2022, following updated guidance from WHO ( Berman Institute of Bioethics & Center for Immunization Research, 2022 ) .

Study sites in Ghana are in the Greater Accra region. Three hospitals are included that represent regional, district, and secondary levels of care. These include Tema General Hospital, Ga West Municipal Hospital, and Shai-Osudoku District Hospital, each serving urban, mix of urban and rural, and rural populations, respectively.

Similarly to Ghana, Kenya introduced the COVID-19 vaccine with 1.02 million doses of COVAX-provided AZO vaccines in March 2021 ( World Health Organization, 2021a ). Pregnant and lactating individuals were explicitly excluded from vaccination campaigns from February 2021 to January 2022, when the MoH revised its directives and recommended all COVID-19 vaccine types and brands for pregnant and lactating women ( Berman Institute of Bioethics & Center for Immunization Research, 2022 ; National Vaccine & Immunization Program, 2021 ).

In Kenya, two antenatal and postnatal clinics in Nairobi were chosen: Aga Khan University Hospital Nairobi, a private referral hospital serving middle and higher socio-economic classes, and Pumwani Maternity Hospital, a public referral hospital that serves largely lower socio-economic status classes.

The first COVID-19 vaccines introduced to Pakistan were half a million doses of the Sinopharm vaccine donated by China in February 2021 ( Siddiqui et al. , 2021 ). Other COVID-19 vaccine brands, such as AZO were introduced in Pakistan via the COVAX Facility starting in May 2021 ( World Health Organization, 2021b ). Unlike Brazil, Ghana, and Kenya, pregnant and lactating women were recommended for vaccination against COVID-19 from the beginning of the vaccine rollout, with the Special Minister to the Prime Minister on Health strongly urging all pregnant and lactating women to receive the vaccine following the deaths of two unvaccinated pregnant women from COVID-19 in August 2021 ( Berman Institute of Bioethics & Center for Immunization Research, 2022 ; Jajja, 2021 ).

Pakistan study sites include two hospitals in Karachi, a community private hospital, The Aga Khan Hospital for Women and Children, Kharadar, serving mostly people of lower and higher middle socio-economic community and Jinnah Postgraduate Medical Center, a tertiary care public hospital serving mostly low and lower-middle socio-economic community.

Sample size

For the qualitative component of the study, for each country, we aim to interview 25 pregnant and 25 postpartum women, for a total of 50 in-depth interviews per country, taking into consideration when data saturation might be reached. For those pregnant, we aim to interview approximately equal samples by trimester (1st, 2nd, and 3rd). For the quantitative component of the study, we aim to administer a survey to 400 pregnant women in each country to evaluate the proportion of participants with a given attitude and the comparison of attitude proportions by vaccination status. The sample size was determined with the following objectives and assumptions: 1) to evaluate the proportion of pregnant women with an attitude with 95% confidence intervals and 5% margin of error, assuming 50% of the population has the attitude (to provide maximum variability), and an unknown population size; 2) to compare two proportions with 95% confidence interval and 80% power, assuming 50% of the group 1 has the attitude and 40% of group 2 has the attitude. An unknown population size was assumed to facilitate evaluation across countries and the uncertainty in patient volume across facilities that sampling is occurring in.

In three of the four countries, Brazil, Kenya, and Pakistan, the goal is to sample approximately equal numbers of pregnant women across the three trimesters of pregnancy. In Ghana, due to cultural beliefs about seeking care in the 1st trimester, the target for the 1st trimester was decreased. In Brazil, Kenya, and Pakistan, an equal representation is being sought from each participating study clinic for both components of the study overall, but not necessarily for the trimester subgroup targets. All countries’ sample size and subgroup targets are listed in Table 1 .

Table 1. Matrix of protocol components.

IDI: in-depth interview, Tri: pregnancy trimester, Post: post-partum, GHS: Ghana Cedi, USD: US Dollar, KES: Kenyan Shilling

Recruitment

The recruitment strategy varies by country. Most sites are using a consecutive sampling method, approaching every eligible participant until they reach subgroup targets, alternating between the survey and the interview. In Brazil, both study sites are using systematic sampling, or sampling every n th person at the antenatal or postnatal clinics. The value of n is based on the patient volume of the clinic. At all three sites in Ghana, both sites in Kenya, and both sites in Pakistan, a consecutive sampling method of women in the clinic’s waiting area is used. In Brazil, participants can join both components of the study (quantitative and qualitative). In Ghana, Kenya, and Pakistan, participants can join only one component of the study. Three of the countries, Ghana, Kenya, and Pakistan, are providing some type of remuneration to reimburse participants’ travel cost or thank you gift to participants. Brazil is not providing any renumeration due to ethical constraints. In Brazil, reimbursement is only accepted if extra costs are incurred by participating in the study, which do not apply to this study.

Recruitment starts with study staff approaching potentially eligible persons in the waiting or reception area of the health care facility. The study staff reads a study recruitment script to the potential participant in a semi-private area. At the end of the recruitment script is an eligibility screen. Eligibility in this study is broad and include five criteria: 1) pregnant or up to six weeks postpartum (interview only), 2) study interest, 3) age of 18 or older (or an emancipated minor in Brazil only), 4) fluent in the local language (or English if applicable), and 5) knowledge of the COVID-19 vaccine. After passing the eligibility questions, the script also asks for their trimester of pregnancy and COVID-19 vaccination status. The trimester of pregnancy question is used to fill the trimester quotas defined in the sample size targets. While there are no sample size targets for vaccinated and unvaccinated within any of the countries, the study aspires to obtain a representation of vaccinated and unvaccinated participants across all four countries. If eligibility is met and the sub-group is needed, the study staff member invites the participant to join. If the prospective participant agrees, informed consent occurs in a private location followed by data collection. Study staff ensure that the participant’s clinic appointment is not missed due to study participation and pauses any study activities if the participant is called to see a provider. Study participation only restarts after the visit is complete.

Data collection

Data collection instruments, surveys and interview guides, were developed through an iterative process that started with a review of the literature, including a review of relevant instruments ( Alagarsamy et al. , 2022 ; Betsch et al. , 2018 ; Bronfenbrenner, 1979 ; Larson et al. , 2015 ; Rosenstock et al. , 1988 ). They were then reviewed by country teams, and pre-tested in each country among data collectors before finalization. Each of the four country teams was able to amend the questionnaire and interview guides to better align to local contexts while efforts were made to ensure sufficient data would be available for pooled, cross-country analyses.

The questionnaire was developed to identify attitudinal, behavioral, and social correlates of vaccine uptake and we sought to use validated items or adapt validated items ( Alagarsamy et al. , 2022 ; Betsch et al. , 2018 ; Bronfenbrenner, 1979 ; Larson et al. , 2015 ; Rosenstock et al. , 1988 ). The questionnaire contains questions on socio-demographics, attitudes toward COVID-19 vaccines, COVID-19 vaccine knowledge and information sources, COVID-19 vaccine behaviors and intentions, and general attitudes towards vaccination in pregnancy, including receipt of other maternal vaccines that might become available in the future.

The in-depth interview guide includes topics on the decision-making process related to COVID-19 vaccines, including risk perception, self-efficacy, vaccine intentions, and social norms, etc. Two interview guides were developed, one for pregnant and one for postpartum participants.

Questions related to the following constructs are included: influences of decision-making, self-efficacy, norms, risk perception, knowledge of disease, knowledge of vaccines, information sources, and vaccine hesitancy.

In Brazil and Pakistan, questionnaires and interviews are done by two separate data collection teams. In Ghana and Kenya, both components of the study are done by one data collection team. All countries are digitally audio recording the qualitative interviews; Pakistan is also including a note-taker in each of the interviews. Brazil and Kenya are using paper-based data collection and double data entry for all questionnaires. Ghana and Pakistan are using tablet-based data collection using either the REDCap Mobile Application or REDCap’s web-based data entry interface. All study data, including in-depth interview audio files, are managed and stored using REDCap electronic data capture tools hosted at JHU ( Harris et al. , 2009 ; Harris et al. , 2019 ). REDCap (Research Electronic Data Capture, Nashville, TN, USA: https://projectredcap.org ) is a secure, web-based software platform designed to support data capture for research studies that is available to non-profit groups who join the consortium. Alternatives that are also free for non-profit groups include Kobo Toolbox (Cambridge, MA. USA: https://www.kobotoolbox.org ) and a self-managed version of Open Data Kit (Seattle, WA, USA: https://getodk.org ). Data collection is done in Brazilian Portuguese in Brazil; Ga, Twi, or English in Ghana; Kiswahili or English in Kenya; and Urdu or English in Pakistan. Both components of the study (questionnaires and interviews) are estimated to take 30-60 minutes to complete, inclusive of the time needed for the consent process.

Data analysis and statistical plan

In Brazil, Kenya, and Pakistan, audio files from qualitative interviews are transcribed in the language they were completed in and then translated to English. In Ghana, transcripts are typed directly into English given the colloquial nature of Ga and Twi languages. Any notes that are taken during the interview are incorporated during transcription. All transcriptions and translations undergo review by an independent study team member as part of standard practice. Questionnaire data are reviewed and cleaned following a standardized data cleaning procedure. No personally identifying information (PII) is captured during the questionnaire and while no PII is intentionally captured during interviews, an anonymization procedure is being followed during transcription to ensure no PII is included in the final transcripts.

For the qualitative aim of the study, a grounded theory approach is followed for data processing and analyses. Each country undergoes an independent and iterative open coding process with representatives from the country team, JHU, and WHO. A minimum of two open coding sessions are conducted to develop and refine a codebook for analysis. Participants for each open coding session review the same random selection of transcripts and through an inductive coding approach, a final codebook is generated. After all countries complete their codebooks, a final code structure and thematic categories will be selected; these will be applied to each transcript in the final coding process. All transcript coding is done with ATLAS.ti ( Smit, 2002 ). An alternative open access qualitative coding platform is Taguette, https://www.taguette.org ( Rampin & Rampin, 2021 ).

A-priori analyses for pooled cross-country qualitative data fall into three topic areas, outlined in Table 2 . For the quantitative aim of the study, there are four main topic areas for planned analyses and the questionnaire was structured around these four themes ( Table 2 ). The primary aims of this study are descriptive and are covered by the planned analyses of both the qualitative and quantitative components of the study. Country specific analyses will be defined and led by each of the country teams.

Table 2. Planned analyses for pooled, cross-country data.

Ethical review

Ethical review and approval for the 4-country study was sought from the Johns Hopkins Bloomberg School of Public Health Institutional Review Board (Ref. IRB00020864, approved 2023-07-06; Ref. IRB00020850, approved 2023-09-12; Ref. IRB00020861, approved 2023-09-27; Ref. IRB00020866, approved 2024-02-01), and the World Health Organization’s Research Ethics Review Committee (Ghana: Ref. CERC.0193A, approved 2023-06-05; Kenya: Ref. CERC.0193B, approved 2023-06-19; Pakistan: Ref. CERC.0193C, approved 2023-09-19) or the Pan American Health Organization (Brazil: Ref. PAHOERC.0633.01, approved 2023-03-24),. Each country protocol also underwent scientific review through the WHO/HRP Research review research panel (Switzerland). Individual country teams sought and obtained approvals for each country-level research plan with the following entities: Committee of Research Ethics from the University of Campinas (Brazil: Ref. 63968222.1.1001.5404, approved 2023-04-10), Jundiaí University Institutional Review Board (Brazil: Ref. CAAE 63968222.1.2001.5412, approved 2023-07-07), Ghana Health Service Ethics Review Committee (Ghana: Ref. 028/03/23, approved 2023-05-23), The Aga Khan University’s Institutional Scientific and Ethics Committee (Kenya: Ref. 2023/ISERC-17, approved 2023-06-19), Pumwani maternity hospital ethics review committee (Kenya: Ref. PMH/CEO/76/0785/2023, approved: 2023-12-13), the National Council for Science Technology and Innovation (Kenya: Ref. NACOSTI/P/23/29152, approved 2023-09-27), Nairobi County Research and Development Committee (Kenya: Ref. NCC/CS/RPD/84/2023, approved 2023-11-27), National Bioethics Committee (Pakistan: Ref. No.4-87/NBCR-1029/23, approved 2024-01-03), and The Aga Khan University Institutional Ethics Review Committee (Pakistan: Ref. 2023-8633-25854, approved 2023-07-27), and the Institutional Review Board at Jinnah Postgraduate Medical Center (Pakistan: Ref. F.2-81/2023-GENL/182/JPMC, approved 2023-12-14).

All study staff were trained in human subjects’ research ethics as well as qualitative and/or quantitative data collection during a three-day country-specific training session. Qualitative training included interviewing techniques to reduce bias, transcription, and translation. Participants in all four countries were recruited in semi-private areas of the clinic and underwent an informed consent process with trained study team members in private areas. All four countries used written informed consent, using alternatives for illiterate participants as allowed by each country. A transcription standardized operating procedure was developed and will be used by all country teams to ensure that all personally identifying information is removed from final transcripts. Standard data cleaning procedures will also be used by all countries.

Dissemination plans

The results of the research will be submitted to peer-reviewed publications in specialized journals and to scientific dissemination meetings and congresses.

In Brazil, at the national and regional level, dissemination will be done through conferences and reports to policy makers to inform strategies and gaps related to the topic. The investigators involved in the study in Brazil are part of National and Regional policy-making committees in maternal and perinatal health and they will work with local partners and stakeholders to develop local dissemination plans. In Ghana, prior to publication, preliminary findings will be disseminated to study facilities. The data and findings from the study will also be disseminated to the Ghana Health Service and other key stakeholders to inform context-specific guidelines for vaccine decision-making and uptake among pregnant and postpartum women in Ghana.

In Kenya, the results from this project will be used for advocacy with health managers and policy makers focusing on the best demand generation and communication strategies to improve the uptake of COVID-19 vaccines for pregnant women. The research team will disseminate the findings in an organized forum comprising different cadres of ministry of health personnel at policy and practice level as well as other relevant stakeholders involved with health care service provision in Kenya. Furthermore, the research team will develop policy briefs for the policy makers and peer reviewed publications in international journals for wider dissemination. In Pakistan, the research findings will be shared with relevant stakeholders, including policymakers and leading obstetricians, through peer-reviewed journals, provincial/national-level seminars, and the meetings of the Society of Obstetricians and Gynecologists of Pakistan (SOGP). The aim is to maximize the visibility and impact of the research findings and contribute to informed decision-making and improved healthcare practices in Pakistan.

Conclusion/discussion

With COVID-19 vaccine uptake during pregnancy lagging behind its recommendations for use, the lessons learned from this study can help inform future COVID-19 vaccine delivery and communications strategies. In addition, as several new maternal vaccines are in the late stages of development or the earliest stages of implementation, this study can also help to inform future vaccine introductions. These findings will also be useful for global policy makers to understand how important factors related to maternal vaccine uptake vary by location, as well as contextual factors that should be considered in program implementation. For policy makers at the national and local level, these data can inform strategies to improve maternal vaccination acceptance and coverage and encourage execution of similar studies in other settings to learn about specific local contexts.

Study status

Data collection was completed in all countries as of 25 May 2024. Data processing, cleaning, and analysis are underway. No data has been published from this study at the time of writing. Results will be presented in subsequent publications.

Ethics and consent

Ethical review and approval for the 4-country study was sought from the Johns Hopkins Bloomberg School of Public Health Institutional Review Board (Ref. IRB00020864, approved 2023-07-06; Ref. IRB00020850, approved 2023-09-12; Ref. IRB00020861, approved 2023-09-27; Ref. IRB00020866, approved 2024-02-01), and the World Health Organization’s Research Ethics Review Committee (Ghana: Ref. CERC.0193A, approved 2023-06-05; Kenya: Ref. CERC.0193B, approved 2023-06-19; Pakistan: Ref. CERC.0193C, approved 2023-09-19) or the Pan American Health Organization (Brazil: Ref. PAHOERC.0633.01, approved 2023-03-24),. Each country protocol also underwent scientific review through the WHO/HRP Research review research panel (Switzerland).Individual country teams sought and obtained approvals for each country-level research plan with the following entities: Committee of Research Ethics from the University of Campinas (Brazil: Ref. 63968222.1.1001.5404, approved 2023-04-10), Jundiaí University Institutional Review Board (Brazil: Ref. CAAE 63968222.1.2001.5412, approved 2023-07-07), Ghana Health Service Ethics Review Committee (Ghana: Ref. 028/03/23, approved 2023-05-23), The Aga Khan University’s Institutional Scientific and Ethics Committee (Kenya: Ref. 2023/ISERC-17, approved 2023-06-19), Pumwani maternity hospital ethics review committee (Kenya: Ref. PMH/CEO/76/0785/2023, approved: 2023-12-13), the National Council for Science Technology and Innovation (Kenya: Ref. NACOSTI/P/23/29152, approved 2023-09-27), Nairobi County Research and Development Committee (Kenya: Ref. NCC/CS/RPD/84/2023, approved 2023-11-27), National Bioethics Committee (Pakistan: Ref. No.4-87/NBCR-1029/23, approved 2024-01-03), and The Aga Khan University Institutional Ethics Review Committee (Pakistan: Ref. 2023-8633-25854, approved 2023-07-27), and the Institutional Review Board at Jinnah Postgraduate Medical Center (Pakistan: Ref. F.2-81/2023-GENL/182/JPMC, approved 2023-12-14).

All four countries used written informed consent, using alternatives for illiterate participants as allowed by each country

Data availability

Underlying data.

No data are associated with this article. Data collected during this study will be made available when results are published as allowed by the data sharing policies of the individual institutions that led data collection in each of the four participating countries.

Extended data

Open Science Framework: Exploring Knowledge, Attitudes, and Practices Related to Vaccine Decision-Making among Pregnant People, DOI 10.17605/OSF.IO/G3YD2 ( Schue, 2024 ).

This project contains the following extended data:

Consent Form Interview: Written consent form for interview participants.

Consent Form Survey: Written consent form for survey participants.

Master Post-Pregnancy IDI Guide: Semi-structured interview guide for post-pregnant women

Master Pregnancy IDI Guide: Semi-structured interview guide for pregnant women

Survey Master: Survey instrument for pregnant women

License: CC-By Attribution 4.0 International

Acknowledgments

We would like to thank Anna Thorson (UNDP-UNFPA-UNICEF-WHO-World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP), Department of Sexual and Reproductive Health and Research, WHO) and Ibukun Abejirinde (Dalla Lana School of Public Health, University of Toronto) for their support in the initial design of the study and ongoing administrative support of this project. We would like to thank the staff at each of the clinics where this study is conducted and the participants that contribute their time and thoughts. The named authors alone are responsible for the views expressed in this publication and do not necessarily represent the decisions or the policies of the UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP) or the World Health Organization (WHO) or any of their affiliated institutions.

•   Alagarsamy S, Mehrolia S, Pushparaj U, et al. : Explaining the intention to uptake COVID-19 vaccination using the Behavioral and Social Drivers of vaccination (BeSD) model. Vaccine X. 2022; 10 : 100140. PubMed Abstract | Publisher Full Text | Free Full Text
•   Allotey J, Stallings E, Bonet M, et al. : Clinical manifestations, risk factors, and maternal and perinatal outcomes of coronavirus disease 2019 in pregnancy: living systematic review and meta-analysis. BMJ. 2020; 370 : m3320. PubMed Abstract | Publisher Full Text | Free Full Text
•   Berman Institute of Bioethics & Center for Immunization Research: COVID-19 Maternal Immunization Tracker (COMIT). Johns Hopkins University, 2022. Reference Source
•   Betsch C, Schmid P, Heinemeier D, et al. : Beyond confidence: development of a measure assessing the 5C psychological antecedents of vaccination. PLoS One. 2018; 13 (12): e0208601. PubMed Abstract | Publisher Full Text | Free Full Text
•   Blakeway H, Prasad S, Kalafat E, et al. : COVID-19 vaccination during pregnancy: coverage and safety. Am J Obstet Gynecol. 2022; 226 (2): 236.e1–236.e14. PubMed Abstract | Publisher Full Text | Free Full Text
•   Bronfenbrenner U: The ecology of human development: experiments by nature and design. Harvard University Press, 1979. Publisher Full Text
•   Broutet N, Thorson A: Generic protocol: a prospective cohort study investigating maternal, pregnancy and neonatal outcomes for women and neonates infected with SARS-CoV-2, 1 November 2022. World Health Organization, 2022. Reference Source
•   Covas DT, De Jesus Lopes De Abreu A, Zampirolli Dias C, et al. : Adverse events of COVID-19 vaccines in pregnant and postpartum women in Brazil: a cross-sectional study. PLoS One. 2023; 18 (1): e0280284. PubMed Abstract | Publisher Full Text | Free Full Text
•   Cox E, Sanchez M, Taylor K, et al. : A mother’s dilemma: the 5-P model for vaccine decision-making in pregnancy. Vaccines (Basel). 2023; 11 (7): 1248. PubMed Abstract | Publisher Full Text | Free Full Text
•   Fernández-García S, del Campo-Albendea L, Sambamoorthi D, et al. : Effectiveness and safety of COVID-19 vaccines on maternal and perinatal outcomes: a systematic review and meta-analysis. BMJ Glob Health. 2024; 9 (4): e014247. PubMed Abstract | Publisher Full Text | Free Full Text
•   Fonseca P, Brito R: Brazil suspends use of AstraZeneca vaccine in pregnant women nationally after death. Reuters , May 11, 2021. Reference Source
•   Geoghegan S, Shuster S, Butler KM, et al. : Understanding barriers and facilitators to maternal immunization: a systematic narrative synthesis of the published literature. Matern Child Health J. 2022; 26 (11): 2198–2209. PubMed Abstract | Publisher Full Text | Free Full Text
•   Goncu Ayhan S, Oluklu D, Atalay A, et al. : COVID-19 vaccine acceptance in pregnant women. Int J Gynaecol Obstet. 2021; 154 (2): 291–296. PubMed Abstract | Publisher Full Text | Free Full Text
•   Hameed I, Khan MO, Nusrat K, et al. : Is it safe and effective to administer COVID-19 vaccines during pregnancy? A systematic review and meta-analysis. Am J Infect Control. 2023; 51 (5): 582–593. PubMed Abstract | Publisher Full Text | Free Full Text
•   Harris PA, Taylor R, Minor BL, et al. : The REDCap consortium: building an international community of software platform partners. J Biomed Inform. 2019; 95 : 103208. PubMed Abstract | Publisher Full Text | Free Full Text
•   Harris PA, Taylor R, Thielke R, et al. : Research Electronic Data Capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009; 42 (2): 377–381. PubMed Abstract | Publisher Full Text | Free Full Text
•   Jajja S: COVID-19 vaccine strongly recommended for pregnant, lactating women. Dawn , August 24, 2021. Reference Source
•   Kilich E, Dada S, Francis MR, et al. : Factors that influence vaccination decision-making among pregnant women: a systematic review and meta-analysis. PLoS One. 2020; 15 (7): e0234827. PubMed Abstract | Publisher Full Text | Free Full Text
•   Kobayashi CD, Porto VBG, Da Nóbrega MEB, et al. : Adverse events related to COVID-19 vaccines reported in pregnant women in Brazil. Rev Bras Ginecol Obstet. 2022; 44 (9): 821–829. PubMed Abstract | Publisher Full Text | Free Full Text
•   Laenen J, Roelants M, Devlieger R, et al. : Influenza and pertussis vaccination coverage in pregnant women. Vaccine. 2015; 33 (18): 2125–2131. PubMed Abstract | Publisher Full Text
•   Larson HJ, Jarrett C, Schulz WS, et al. : Measuring vaccine hesitancy: the development of a survey tool. Vaccine. 2015; 33 (34): 4165–4175. PubMed Abstract | Publisher Full Text
•   Limaye RJ, Paul A, Gur-Arie R, et al. : A socio-ecological exploration to identify factors influencing the COVID-19 vaccine decision-making process among pregnant and lactating women: findings from Kenya. Vaccine. 2022; 40 (50): 7305–7311. PubMed Abstract | Publisher Full Text | Free Full Text
•   Limaye RJ, Singh P, Fesshaye B, et al. : Lessons learned from COVID-19 vaccine acceptance among pregnant and lactating women from two districts in Kenya to inform demand generation efforts for future maternal RSV vaccines. BMC Pregnancy Childbirth. 2024; 24 (1): 221. PubMed Abstract | Publisher Full Text | Free Full Text
•   Marchant A, Sadarangani M, Garand M, et al. : Maternal immunisation: collaborating with mother nature. Lancet Infect Dis. 2017; 17 (7): e197–e208. PubMed Abstract | Publisher Full Text
•   National Vaccine & Immunization Program: National COVID-19 vaccines deployment and vaccination plan. Republic of Kenya Ministry of Health, 2021. Reference Source
•   Prasad S, Kalafat E, Blakeway H, et al. : Systematic review and meta-analysis of the effectiveness and perinatal outcomes of COVID-19 vaccination in pregnancy. Nat Commun. 2022; 13 (1): 2414. PubMed Abstract | Publisher Full Text | Free Full Text
•   Rampin R, Rampin V: Taguette: open-source qualitative data analysis. Journal of Open Source Software. 2021; 6 (68): 3522. Publisher Full Text
•   Rao GG, Khanna P: To screen or not to screen women for Group B Streptococcus ( Streptococcus agalactiae ) to prevent early onset sepsis in newborns: recent advances in the unresolved debate. Ther Adv Infect Dis. 2020; 7 : 204993612094242. PubMed Abstract | Publisher Full Text | Free Full Text
•   Razzaghi H, Meghani M, Pingali C, et al. : COVID-19 vaccination coverage among pregnant women during pregnancy – eight integrated health care organizations, United States, December 14, 2020–May 8, 2021. MMWR Morb Mortal Wkly Rep. 2021; 70 (24): 895–899. PubMed Abstract | Publisher Full Text | Free Full Text
•   Röbl-Mathieu M, Kunstein A, Liese J, et al. : Vaccination in Pregnancy. Dtsch Arztebl Int. 2021; 118 (15): 262–268. PubMed Abstract | Publisher Full Text | Free Full Text
•   Rosenstock IM, Strecher VJ, Becker MH: Social learning theory and the Health Belief Model. Health Educ Q. 1988; 15 (2): 175–183. PubMed Abstract | Publisher Full Text
•   Schue JL: Exploring knowledge, attitudes, and practices related to vaccine decision-making among pregnant people. [Dataset]. OSF. 2024. http://www.doi.org/10.17605/OSF.IO/G3YD2
•   Secretaria Extraordinária de Enfrentamento à COVID-19 Gabinete: NOTA TÉCNICA No 2/2021-SECOVID/GAB/SECOVID/MS. (0021464579). Brasil Ministério da Saúde, 2021. Reference Source
•   Shamshirsaz AA, Hessami K, Morain S, et al. : Intention to receive COVID-19 vaccine during pregnancy: a systematic review and meta-analysis. Am J Perinatol. 2022; 39 (5): 492–500. PubMed Abstract | Publisher Full Text
•   Siddiqui A, Ahmed A, Tanveer M, et al. : An overview of procurement, pricing, and uptake of COVID-19 vaccines in Pakistan. Vaccine. 2021; 39 (37): 5251–5253. PubMed Abstract | Publisher Full Text | Free Full Text
•   Smit B: Atlas.ti for qualitative data analysis. Research paper. Perspectives in Education. 2002; 20 (3): 65–75. Reference Source
•   Smith ER, Oakley E, Grandner GW, et al. : Clinical risk factors of adverse outcomes among women with COVID-19 in the pregnancy and postpartum period: a sequential, prospective meta-analysis. Am J Obstet Gynecol. 2023; 228 (2): 161–177. PubMed Abstract | Publisher Full Text | Free Full Text
•   Sobanjo-Ter Meulen A, Munoz FM, Kaslow DC, et al. : Maternal interventions vigilance harmonization in low- and middle-income countries: stakeholder meeting report; Amsterdam, May 1–2, 2018. Vaccine. 2019; 37 (20): 2643–2650. PubMed Abstract | Publisher Full Text | Free Full Text
•   The World Bank: 13 million People to Receive COVID-19 vaccination in Ghana. The World Bank Press Release, 2021. Reference Source
•   UNICEF & World Health Organization: Ghana becomes recipient of historic first shipment of COVAX vaccine. April 19, 2024. Reference Source
•   Wang H, Li N, Sun C, et al. : The association between pregnancy and COVID-19: a systematic review and meta-analysis. Am J Emerg Med. 2022; 56 : 188–195. PubMed Abstract | Publisher Full Text | Free Full Text
•   World Health Organization: Kenya receives COVID-19 vaccines and launches landmark national campaign. 2021a. Reference Source
•   World Health Organization: Pakistan receives first consignment of COVID-19 vaccines via COVAX Facility. Eastern Mediterranean Region, 2021b. Reference Source
•   World Health Organization: WHO SAGE Roadmap for prioritizing uses of COVID-19 vaccines. (WHO/2019-nCoV/Vaccines/SAGE/Prioritization/2023.1). Immunization, Vaccines and Biologicals (IVB), Strategic Advisory Group of Experts on Immunization, 2023. Reference Source
•   Zavala E, Krubiner CB, Jaffe EF, et al. : Global disparities in public health guidance for the use of COVID-19 vaccines in pregnancy. BMJ Glob Health. 2022; 7 (2): e007730. PubMed Abstract | Publisher Full Text | Free Full Text

Comments on this article Comments (0)

Open Peer Review

Reviewer status, comments on this article.

All Comments (0)

Competing Interests Policy

Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list:

• Within the past 4 years, you have held joint grants, published or collaborated with any of the authors of the selected paper.
• You have a close personal relationship (e.g. parent, spouse, sibling, or domestic partner) with any of the authors.
• You are a close professional associate of any of the authors (e.g. scientific mentor, recent student).
• You work at the same institute as any of the authors.
• You hope/expect to benefit (e.g. favour or employment) as a result of your submission.
• You are an Editor for the journal in which the article is published.
• You expect to receive, or in the past 4 years have received, any of the following from any commercial organisation that may gain financially from your submission: a salary, fees, funding, reimbursements.
• You expect to receive, or in the past 4 years have received, shared grant support or other funding with any of the authors.
• You hold, or are currently applying for, any patents or significant stocks/shares relating to the subject matter of the paper you are commenting on.

Stay Updated

Sign up for content alerts and receive a weekly or monthly email with all newly published articles

Register with Gates Open Research

Already registered? Sign in

Not now, thanks

Are you a Gates-funded researcher?

If you are a previous or current Gates grant holder, sign up for information about developments, publishing and publications from Gates Open Research.

We'll keep you updated on any major new updates to Gates Open Research

The email address should be the one you originally registered with F1000.

You registered with F1000 via Google, so we cannot reset your password.

To sign in, please click here .

If you still need help with your Google account password, please click here .

You registered with F1000 via Facebook, so we cannot reset your password.

If you still need help with your Facebook account password, please click here .

If your email address is registered with us, we will email you instructions to reset your password.

If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance.

• Open access
• Published: 28 August 2024

Facilitators and barriers of midwife-led model of care at public health institutions of dire Dawa city, Eastern Ethiopia, 2022: a qualitative study

• Mickiale Hailu 1 ,
• Aminu Mohammed 1 ,
• Daniel Tadesse 1 ,
• Neil Abdurashid 1 ,
• Legesse Abera 1 ,
• Samrawit Ali 2 ,
• Yesuneh Dejene 2 ,
• Tadesse Weldeamaniel 1 ,
• Meklit Girma 3 ,
• Tekleberhan Hailemariam 1 ,
• Netsanet Melkamu 1 ,
• Tewodros Getnet 1 ,
• Yibekal Manaye 1 ,
• Tariku Derese 1 ,
• Muluken Yigezu 1 ,
• Natnael Dechasa 1 &
• Anteneh Atle 1  

BMC Health Services Research volume  24 , Article number:  998 ( 2024 ) Cite this article

1 Altmetric

Metrics details

The midwife-led model of care is woman-centered and based on the premise that pregnancy and childbirth are normal life events, and the midwife plays a fundamental role in coordinating care for women and linking with other health care professionals as required. Worldwide, this model of care has made a great contribution to the reduction of maternal and child mortality. For example, the global under-5 mortality rate fell from 42 deaths per 1,000 live births in 2015 to 39 in 2018. The neonatal mortality rate fell from 31 deaths per 1,000 live births in 2000 to 18 deaths per 1,000 in 2018. Even if this model of care has a pivotal role in the reduction of maternal and newborn mortality, in recent years it has faced many challenges.

To explore facilitators and barriers to a midwife-led model of care at a public health institution in Dire Dawa, Eastern Ethiopia, in 2021.

Methodology

: A qualitative approach was conducted at Dire Dawa public health institution from March 1–April 30, 2022. Data was collected using a semi-structured, in-depth interview tool guide, focused group discussions, and key informant interviews. A convenience sampling method was implemented to select study participants, and the data were analyzed thematically using computer-assisted qualitative data analysis software Atlas.ti7. The thematic analysis with an inductive approach goes through six steps: familiarization, coding, generating themes, reviewing themes, defining and naming themes, and writing up.

Two major themes were driven from facilitators of the midwife-led model of care (professional pride and good team spirit), and seven major themes were driven from barriers to the midwife-led model of care (lack of professional development, shortage of resources, unfair risk or hazard payment, limited organizational power of midwives, feeling of demoralization absence of recognition from superiors, lack of work-related security).

The midwifery-led model of care is facing considerable challenges, both pertaining to the management of the healthcare service locally and nationally. A multidisciplinary and collaborative effort is needed to solve those challenges.

Peer Review reports

Introduction

A midwife-led model of care is defined as care where “the midwife is the lead professional in the planning, organization, and delivery of care given to a woman from the initial booking to the postnatal period“ [ 1 ]. Within these models, midwives are, however, in partnership with the woman, the lead professional with responsibility for the assessment of her needs, planning her care, referring her to other professionals as appropriate, and ensuring the provision of maternity services. Most industrialized countries with the lowest mortality and morbidity rates of mothers and infants are those in which midwifery is a valued and integral pillar of the maternity care system [ 2 , 3 , 4 , 5 ].

Over the past 20 years, midwife-led model of care (MLC) has significantly lowered mother and infant mortality across the globe. In 2018, there were 39 deaths for every 1,000 live births worldwide, down from 42 in 2015. From 31 deaths per 1,000 live births in 2000 to 18 deaths per 1,000 in 2018, the neonatal mortality rate (NMR) decreased. The midwifery-led care approach is regarded as the gold standard of care for expectant women in many industrialized nations, including Canada, Australia, the United Kingdom, Sweden, the Netherlands, Norway, and Denmark. Evidence from those nations demonstrates that women and babies who get midwife-led care, as opposed to alternative types of care, experience favorable maternal outcomes, fewer interventions, and lower rates of fetal loss or neonatal death [ 6 , 7 , 8 ].

In Pakistan, the MLC was accompanied by many challenges. Some of the challenges were political threats, a lack of diversity (midwives had no opportunities for collaborating with other midwives outside their institutions), long duty hours and low remuneration, a lack of a career ladder, and a lack of socialization (the health centers are isolated from other parts of the country due to relative geographical inaccessibility, transportation issues, and a lack of infrastructure). Currently, in Pakistan, 276 women die for every 100,000 live births, and the infant mortality rate is 74/1000. But the majority of these deaths are preventable through the midwife-led care model [ 7 ].

The MLC in African countries has faced many challenges. Shortages of resources, work overload, low inter-professional collaboration between health facilities, lack of personal development, lack of a well-functioning referral system, societal challenges, family life troubles, low professional autonomy, and unmanageable workloads are the main challenges [ 8 ].

Due to the aforementioned challenges, Sub Saharan Africa (SSA) is currently experiencing the highest rate of infant mortality (1 in 13) and is responsible for 86% of all maternal fatalities worldwide. As a result, it is imperative to look at the MLC issues in low-income countries, which continue to be responsible for 99% of all maternal and newborn deaths worldwide [ 8 , 9 ].

Ethiopia’s has a Maternal mortality rate (MMR) and NMR of 412 per 100,000 live births and 33 per 1000 live births, respectively, remain high, making Ethiopia one of the largest contributors to the global burden of maternal and newborn deaths, placed 4th and 6th, although MLC could prevent a total of 83% of all neonatal and maternal fatalities in an environment that supports it. The MMR & infant mortality rate (IMR) in the research area were indistinguishable from that, at 150 per 100,000 live births and 67 fatalities per 1,000 live births, respectively [ 10 , 11 , 12 , 13 ].

Since the Federal Ministry of Health is currently viewing midwifery-led care as an essential tool in reducing the maternal mortality ratio and ending preventable deaths of newborns, exploring the facilitators and barriers of MLC may have a great contribution to make in reducing maternal and newborn mortality [ 14 ]. Since there has been no study done in Ethiopia or the study area regarding the facilitators and barriers of MLC, the aim of this research was to explore the facilitators and barriers of MLC in Dire Dawa City public health institutions.

In so doing, the research attempted to address the following research questions:

What were the facilitators for a midwife-led model of care at the Dire Dawa city public health institution?

What were the barriers to a midwife-led model of care at the Dire Dawa city public health institution?

Study setting and design

Institutional based qualitative study was conducted from March 01-April 30, 2022 in Dire Dawa city. Dire Dawa city is one of the federal city administrations in Ethiopia which is located at the distance of 515killo meters away from Addis Ababa (the capital city) to the east. The city administration has 9 urban and 38 rural kebeles (kebeles are the smallest administrative unit in Ethiopia). There are 2 government hospitals, 5 private hospitals, 15 health centers, and 33 health posts. The current metro area population of Dire Dawa city is 426,129.Of which 49.8% of them are males and 50.2% females. The total number of women in reproductive age group (15–49 years) is 52,673 which account 15.4% of the total population. It has hot temperature with a mean of 25 degree centigrade [ 15 ].

Study population and sampling procedure

The source population for this study included all midwives who worked at Dire Dawa City public health facilities as well as key informants from appropriate organizations (the focal person for the Ethiopian Midwives Association and maternal and child health (MCH) team leaders). The study encompassed basically 41 healthcare professionals who worked in Dire Dawa public health institutions in total, and the final sample size was decided based on the saturation of the data or information.

From the total 15 Health centers and 2 Governmental Hospitals found in Dire Dawa city administration, 8 Health centers and 2 Governmental Hospitals were selected by non-probability purposive sampling method. In addition to that a non-probability convenience sampling method was used to select midwives who were working in Dire Dawa city public health institutions and key informants from the relevant organization such as Ethiopian midwives association focal person and MCH team leaders. Midwives who were working for at least six months in the institution were taken as inclusion criteria while those who were working as a free service were excluded from the study.

Data collection tool and procedures

Focus groups, in-depth interviews, and key informant interviews were used in collecting data. A voice recorder, a keynote-keeping, and a semi-structured interview tool were all used to conduct the interviews. Voluntary informed written consent was obtained from the study participant’s before they participated in the study. Then an in-depth interview and focus group discussion were held with midwives chosen from various healthcare organizations. The MCH department heads and the Dire Dawa branch of the Ethiopian Midwife Association served as the key informants. In-depth interview (IDI) and key informant interviews (KII) with participants took place only once and lasted for roughly 50–60 min. In the midwives’ duty room, the interview was held. Six to eight people participated in focus group discussions (FGD), which lasted 90 to 100 min. Two midwives with experience in gathering qualitative data gathered the information.

Data quality control

The qualitative design is prone for bias but open-ended questions were used to avoid acquiescence and 2 day proper training was given for the data collector regarding taking keynotes and recording using a tape recorder. For consistency and possible modification, a pretest was done in one FGD and In-depth interviews at non selected health institutions of Dire Dawa city administrations. A detailed explanation was given for the study participants about the objectives of the study prior to the actual data collections. All (FGDs, key informant interview and In-depth interviews) were taken in a silent place.

Data analysis

Atlas.ti7, a qualitative data analysis program, was used for analyzing the data thematically. An inductive approach to thematic analysis involves six steps: familiarization, coding, generation of themes, review of themes, defining and naming of themes, and writing up. By listening to the taped interview again, the data was transcribed. The participants’ well-spoken verbatim was used to extract and describe the inductive meanings of the statements. The data was then coded after that. Each code describes the concept or emotion made clear in that passage of text. Then we look at the codes we’ve made, search for commonalities, and begin to develop themes. To ensure the data’s accuracy and representation, the generated themes were reviewed. Themes were defined and named, and then the analysis of the data was written up.

Trustworthiness of data

Meeting standards of trustworthiness by addressing credibility, conformability, and transferability ensures the quality of qualitative research. Data triangulation, data collection from various sites and study participants, the use of multiple data collection techniques (IDI, KII, and FGD), multiple peer reviews of the proposal, and the involvement of more than two researchers in the coding, analysis, and interpretation decisions are all instances of the methods that were used in order to fulfill the criteria for credibility. To increase its transferability to various contexts, the study gave details of the context, sample size and sampling method, eligibility criteria, and interview processes. To ensure conformability, the research paths were maintained throughout the study in accordance with the work plan [ 16 , 17 ].

Background characteristics of the study participants

In this study, a total of 41 health care providers who are working in Dire Dawa public health facilities participated in the three FGDs, six KIIs, and fifteen IDIs. The years of experience of study participants range from one year to 12 years. The participants represented a wide age range (30–39 years), and the educational status of the respondents ranged from diploma to master’s degree. (Table  1 )

As shown in Table  2 , from the qualitative analysis of the data, two major themes were driven from facilitators of MLC, and seven major themes were driven from barriers to MLC. (Table  2 ).

Facilitators of midwife-led model of care at a public health institution of Dire Dawa city, Eastern Ethiopia, in 2021

Professional pride.

This study found that saving the lives of mothers and newborns was a strong facilitator. Specifically, it was motivational to have skills within the midwifery domain, such as managing the full continuum of care during pregnancy and labour, supporting women in having normal physiologic births, being able to handle complications, and building relationships with the women and the community, as mentioned below by one of the IDI participants.

“I am so proud since I am a midwife; nothing is more satisfying than seeing a pregnant mother give birth almost without complications. I always see their smile and happiness on their faces , especially in the postpartum period , and they warmly thank me and say , “Here is your child; he or she is yours.” They bless me a lot. Even sometimes , when they sew me in the transport area , cafeteria , or other area , they thank me warmly , and some of them also want to invite me to something else. The sum total of those things motivates me to be in this profession or to provide midwifery care.“ IDI participants.

This finding is also supported by other participants in FGD.

“We have learned and promised to work as midwives. We are proud of our profession , to help women and children’s health. The greatest motivation is that we are midwives , we love the profession , and we are contributing a great role in decreasing maternal and child mortality….” FGD discussant.

Good teamwork

The research revealed that good midwifery teamwork and good social interaction within the staff have become facilitators of MLC. FGD participants share their experiences of working in a team.

“In our facility , all the midwives have good teamwork; we have good communication , and we share client information accurately and timely. In case a severe complication happens , we manage it as a team , and we try to cover the gap if some of our staff are absent. Further from that , we do have good social interactions in the case of weeding , funeral ceremonies , and other social activities. We do have good team spirit; we work as a team in the clinical area , and we also have good social relationships. “If some of our staff gets sick or if she or he has other social issues , the other free staff will cover her or his task.” FGD discussant.

Another participant from IDI also shared the same experience regarding their good teamwork and their social interactions.

“As a maternal and child health team , we do have a good team spirit , not only with midwives but also with other professions. We are not restricted by the ward that we assign. If there is a caseload in any unit , some midwives will volunteer to help the other team. Most of the time in the night , we admit more than 3 or 4 labouring mothers at the same time. Since in our health center only one midwife is assigned in the night , we always call nurses to help us. This is our routine experience.” IDI participants.

Barriers of midwife-led model of care at a public health institution of Dire Dawa city, Eastern Ethiopia, in 2021

Lack of professional development.

This study revealed that insufficient opportunities for further education and updated training were the main barriers for MLC. Even the few trainings and update courses that were actually arranged were unavailable to them, either because they did not meet the criteria seated or because the people who work in administration were selected. Even though opportunities are not arranged for them to upgrade themselves through self-sponsored. One of the participants from IDI narrates her opinion about opportunities for further education as follows:

“Training and updates are not sufficient; currently we are almost working with almost old science. For example , the new obstetrics management protocol for 2021 has been released from the ministry of health , and many things have changed there. But we did not receive any training or even announcements. Even the few trainings and update courses that were truly organized and turned in to us are unavailable since the selection criteria are not fair. As a result , we miss those trainings either because we did not meet the selection criteria or because those who work in administration are prioritized.” IDI participant.

FGD discussants also support this idea. She mentioned that even though opportunities are not arranged for them to upgrade themselves through self-sponsorship,

“There is almost no educational opportunity in our institution. Every year , one or two midwives may get institutional sponsorship. Midwives that will be selected for this opportunity are those who have served for more than five to ten years. Imagine that to get this chance , every midwife is expected to serve five or more years. Not only this , even if staff want to learn or upgrade at governmental or private colleges through self-sponsored programmes , whether at night or in an extension programme , they are not cooperative. Let me share with you my personal experience. Before two years , I personally started my MSc degree at Dire Dawa University in a weekend programme , and I have repeatedly asked the management bodies to let me free on weekends and to compensate me at night or any time from Monday to Friday. Since they refuse to accept my concern , I withdraw from the programme.“ FGD discussant.

Shortage of resource

The finding indicates that a shortage of equipment, staff, and rooms or wards was a challenge for MLC. Midwives claimed they were working with few staff, insufficient essential supplies, and advanced materials. This lack of equipment endangers both the midwives and their patients. One of the participants from IDI narrates her opinion about the shortage of resources as follows:

“Of course there is a shortage of resources in our hospital , like gloves and personal protective devices. Even the few types of medical equipment available , like the autoclave , forceps , vacuum delivery couch , and BP apparatus , are outdated , and some of them are unfunctional. If you see the Bp apparatus we used in ANC , it is digital but full of false positives. When I worked in the ANC , I did not trust it and always brought the analogue one from other wards. This is the routine experience of every staff member.“ IDI participants.

Another participant from IDI also shared the same experience regarding the crowdedness of rooms or wards.

“In our health center , there are no adequate wards or rooms. For example , the delivery ward and postnatal ward are almost in one room. Postnatal mothers and neonates did not get enough rest and sleep because of the sound of laboring mothers. Not only is this , but even the antenatal care and midwifery duty rooms are also very narrow.“ IDI participants.

The study also revealed midwifery staff were pressured to work long hours because they were understaffed, which in turn affected the quality of midwifery care. The experience of a certain midwife is shared as follows:

“I did not think that the management bodies understood the risk and stress that we midwives face. They did not want to consider the risk of midwives even equal to that of other disciplines but lower than the others. For example , in our health centre , during the night , only one midwife is assigned for the next 12 hours , but if you see in the nurse department , two or more nurses are assigned at night in the emergency ward.” IDI participants.

The discussion affirms the fact that being understaffed and not having an adequate allocation of midwife professionals on night shifts are affecting labouring mothers’ ability to get sufficient health midwifery care. The above narration is also supported by the FGD discussant.

“In our case , only one midwife is assigned to the labour ward during the night shift. I think this is the main challenge for midwives that needs attention. Let me share with you my experience that happened months before. While I was on night shift , two labouring mothers were fully dilated within three or four minutes. It was very difficult for me , to manage two labouring mothers at the same time. Immediately , I call one of my nurse friends from the emergency department to help me. If my friend was so busy , what could happen to the labouring mother and also to me? This is not only my experience but also the routine experience of other midwives.” FGD discussant.

Unfair risk or hazard payments

It is reported that the compensation amount paid for risk is lower than in other health professions. The health risks are not any less, but the remuneration system failed to capture the need to fairly compensate midwifery professionals. The narration from the FGD discussant regarding unfair payment is mentioned below.

“Only 470 ETB is paid for midwives as risk payments , which is incomparable with the risks that midwives are facing. But contrary to that , the risk payments for nurses (in emergencies) are about 1200 Ethiopian birr (ETB) , and Anesthesia is 1000 ETB. I did not want to compare my profession with other disciplines , but with the lowest cost , how the risk of midwifery cannot be equal to that of nursing and other professions. I did not know whose professionals made such types of unfair decisions and with what scientific background or base this calculation was done . ” FGD discussant.

The above finding is also supported by an IDI participant.

“………………………….Even though the midwifery profession is full of risks , with the current Ethiopian health care system , midwives are being paid the lowest risk payments compared to other disciplines…………….” IDI participants.

Limited organizational power of midwives

Midwives’ interviews reported that limited senior midwifery positions in the health system have become the challenge of midwifery care. This constrains the decision-making power and capability of midwives. This was compounded by limited opportunities for midwifery personnel to address their concerns to the responsible bodies, as stated by one of the key informants.

“Our staff has many concerns , especially professional-related concerns , which can contribute to the quality of midwifery care. Personally , as department head , I have tried to address those concerns in different management meetings at different times. But since the leadership positions are dominated by other disciplines , many of our staff concerns have not been solved yet. But let me tell you my personal prediction… If those concerns are not solved early and if this trend continues , the quality of midwifery care will be in danger.“ Participant from Key Informant.

The above finding is also supported by another IDI participant.

“In our hospital , at every hierarchal and structural level , midwives are not well represented. That is why all of our challenges or concerns have not been solved yet. For example , as a structure in the Dire Dawa Health Office (DDHO) , there is a team of management related to maternal and child health. But unfortunately , those professionals working there are not midwives. I was one of three midwives chosen to meet with Dr. X (former DDHO leader) to discuss this issue. At the time , we were reaching an agreement that two or three midwives would be represented on that team. But since a few months later the leader resigned , the issue has not gotten a solution yet.“ IDI participant.

Feeling of demoralization

One of the main concerns reported by the participants during the interviews was a feeling of demoralization induced by both their clients and their supervisors about barriers to midwifery care. They reported having been verbally abused by their patients, something that made them feel that their hard work was being undermined, as stated by an FGD participant.

“I don’t think there is any midwife who would be happy for anybody to lose their baby , or that there is any midwife who would want a woman to die. These things are accidents , but the patient and leaders will always blame the midwife.” FDG discussant.

A narration from an IDI participant also mentioned the following:

“……….If something happens , like a conflict with the patients or clients , the management is on the patient side. Not only that , the way in which they communicate with us is in an aggressive or disrespectful manner . ” IDI participant.

Absence of recognition or /motivation from superiors

This study revealed that midwives experience a loss of motivation at work due to limited support from their superiors. Their effort is used only for reporting purposes. A midwife from FGD shared her experience as follows.

“In our scenario , till the nearest time , the maternal and child health services are provided in a good way. But this was not easy; it is the cumulative effort of midwives. But unfortunately , only those in managerial positions are recognized. Nothing was done for us despite our efforts. To me , our efforts are used only for reporting purposes.” FGD discussant.

This finding was also supported by IDI participants.

“Even though we have good achievements in the MCH services , there is no motivation mechanism done to motivate midwives.” But if something or a minor mistake happens , they are on the front lines to intimidate us or write a warning letter. Generally , their concern is a report or a number issue. We are tired of such types of scenarios.” IDI participant.

Insufficient of work-related security

One of the main concerns reported by the participants during the interviews was the work related security, which has become a challenge for MLC. The midwives’ work environment was surrounded by insecurity, especially during night shifts, when midwives were facing verbal and even physical attack, as mentioned by participants.

“In the labour ward , especially at night , we face many security-related issues. The families of labouring mothers , especially those who are young , are very aggressive. Sometimes they even want to enter the delivery room. They did not hear what we told them to do , but if they hear any labour sounds from their family , they disturb the whole ward. This leads to verbal abuse , and sometimes we face physical abuse. There may be one or two security personnel at the main gate , but since the delivery ward is far from the main gate , they do not know what is happening in the delivery ward. When things become beyond our scope , we call security guards. Immediately after the security guards go back , similar things will continue. What makes it difficult to manage such situations is that only one midwife is assigned at night , and labouring mothers will not get quality midwifery care.” IDI participant.

FGD discussants also shared their experience that their working environment is full of insecurity.

“In case any complications occur , especially at night , it is very difficult to tell the labouring mother’s family or husband unless we call security personnel. It is not only swearing that we face but also that they intimidate us.” FDG discussant.

Discussions

The aim of this study was to explore facilitators’ and barriers to a midwifery-led model of care at Dire Dawa public health facilities. In this study, professional pride was the main facilitator of the midwifery-led model of care. Another qualitative study that examined the midwifery care challenges and factors that motivate them to remain in their workplace lends confirmation to this conclusion. It was found that a strong feeling of love for their work was the main facilitator’s midwifery-led model of care [ 9 ]. Having a good team spirit was also another facilitator’s midwifery-led model of care in our study. Another study’s findings confirmed this one, which emphasizes that building relationships with the midwives, women, and community was the driving force behind providing midwifery care [ 7 , 18 ].

The midwives in this study expressed a need for additional professional training, updates, and competence as part of their continuing professional development. Similar findings have been reported in the worldwide literature that midwives were struggling for survival due to a lack of limited in-service training opportunities to improve their knowledge and skills [ 19 ]. This phenomenon does not seem to differ between settings in high-, middle-, and low-income countries [ 7 , 9 , 18 ], in which midwives experienced difficult work situations due to a lack of professional development to autonomously manage work tasks, which made them feel frustrated, guilty, and inadequate. As such, this can contribute to distress and burnout, which in turn prevent midwives from being able to provide quality care and can eventually cause them to leave the profession [ 19 ].

Shortages of resources (shortage of staff, lack of physical space, and equipment) were the other reported barriers to midwifery care explored in this study. They reported that they are working in an environment with a shortage of resources, which leads to poor patient outcomes. This finding is supported by many other studies conducted around the globe [ 20 , 21 , 22 , 23 ]. Another qualitative finding, which likewise supports the aforementioned finding, which emphasizes that a shortage of resources was reported as a barrier to providing adequate midwifery care [ 19 ]. Delivery attended by skilled personnel with appropriate supplies and equipment has been found to be strongly associated with a reduction in child and maternal mortality [ 24 ].

The feeling of demoralization and lack of motivation from their superiors were other barriers to midwifery care explored in this study. This finding is concurrent with other studies conducted around the globe [ 19 , 25 , 26 , 28 ]. The above finding is also is in accord with another qualitative narration, which emphasizes that feelings of demoralization and a lack of motivation were the main challenges of midwifery care [ 22 ]. Positive support from supervisors has been demonstrated to be important for the quality of services that health workers are able to deliver. In the World Health Organization’s report on improving performance in healthcare, the WHO stresses that supportive supervision can contribute to the improved performance of health workers [ 27 ].

Unfair risk payment was the other challenge identified by the current study. Even though there is no difference in the risk they face among health professionals, the risk payment for midwives is very low compared to others. This finding was in conformity with another qualitative narration, which emphasizes that the lack of an equitable remuneration system was experienced by the DRC midwives, and it has also been confirmed to be highly problematic in other studies in low- and middle-income settings [ 7 , 8 , 22 , 28 ], leading to serious challenges. In settings where salaries are extremely low or unpredictable, proper remuneration is seen as crucial to worker motivation and the quality of midwifery care [ 29 , 30 ].

The limited organizational power of midwives was another identified challenge of MLC. This finding was in step with other studies that emphasize that limited senior midwifery positions in the health system constrain the decision-making power and capability of midwives. This was compounded by limited opportunities for midwifery personnel to address their concerns to the responsible bodies. Hence, midwives need to take control of their own situations. When midwives are included in customizing their work environments, it has proven to result in improved quality of care for women and newborns around the globe [ 8 , 15 ].

Lack of work-related security was another barrier to MLC explored in this study, in which the midwives’ work environment was surrounded by insecurity, especially during night shifts, when midwives are facing verbal and even physical attack, as mentioned by participants. This finding is supported by many other studies conducted around the globe [ 22 , 23 , 25 , 31 ]. The above finding is also in agreement with another qualitative narration, which emphasizes that the midwives’ work environment was surrounded by insecurity, especially during night shifts due to a lack of available security personnel; they often felt frightened on their way to and from work [ 7 ]. In order for midwives to provide quality care, it is crucial to create supportive work environments by ensuring sufficient pre-conditions, primarily security issues [ 31 ].

Conclusions

The study findings contribute to a better understanding of the facilitators’ and barriers of a midwifery-led model of care in the case of Dire Dawa public health facilities. Professional pride and having good team spirit were the main facilitators of midwifery-led model care. Contrary to that, insufficient professional development, shortage of resources, feeling of demoralization, lack of motivation, limited organizational power of midwives, unfair risk payment, and lack of work-related security were the main barriers to a midwifery-led model of care in the case of Dire Dawa public health facilities. Generally, midwifery care is facing considerable challenges, both pertaining to the management of the healthcare service locally and nationally.

Study implications

The findings of the study have implications for midwifery care practices in Eastern Ethiopia. Addressing these areas could potentially contribute to the reduction of IMR and MMR.

Strengths and limitations

The first strength of the study is that the participants represented different healthcare facilities, both urban and rural, thereby offering deeper and more varied experiences and reflections. A second strength is using a midwife as a moderator. She or he understood the midwives’ situation, thereby making the participants feel more comfortable and willing to share their stories. However, focusing solely on the perspective of the midwives is a limitation.

Recommendations

To overcome the barriers of midwifery care, based on the result of this study and in accordance with the 2020 Triad Statement made by the International Council of Nurses, the International Confederation of Midwives, and the World Health Organization, it is suggested that policymakers, Ethiopian federal ministry of health, Dire dawa health office, and regulators in Dire Dawa city and settings with similar conditions coordinate actions in the following:

To the Ethiopian federal ministry of health (FMOH)

Should strengthen regular and continuous educational opportunities, trainings, and updates for midwives, prioritizing and enforcing policies to include adequate and reasonable remuneration and hazard payment for midwives. Support midwifery leadership at all levels of the health system to contribute to health policy development and decision-making.

To dire Dawa health Bureau

Ensure decent working conditions and an enabling environment for midwives. This includes reasonable working hours, occupational safety, safe staffing levels, and merit-based opportunities for career progression. Special efforts must be made to ensure safe, respectful, and enabling workplaces for midwives operating on the night shift. Midwifery leaders should be involved in management bodies within an appropriate legal framework. Made regular mentorships on the functionality of different diagnostic instruments in respective health facilities.

To Dire Dawa public health facility’s

Create an arena for dialogue and implement a more supportive leadership style at the respective health facilities. Should address professional-related concerns of midwives early. Ensure midwives’ representation at the management bodies. Ensure the selection criteria for educational opportunities and different trainings are fair and inclusive. Ensure the safety and security of midwives, especially those who work night shifts. Should assign adequate staff (midwives and security guards) to the night shifts.

Ethiopian midwifery association

Should influence different stakeholders to solve midwife’s concerns like hazards payment and educational opportunity.

Data availability

All the datasets for this study are available from the corresponding author upon request.

Abbreviations

Focused group discussion

In-depth interview

Infant mortality rate

Key informant interview

Maternal and child health

Midwives led model of care

Neonatal mortality rate

The midwives model of care. Midwives alliance North America, the MANA core documents, 2020.

WHO. Midwife-led care delivers positive pregnancy and birth outcomes. The global health work force alliance,2020.

ICM, Midwifery Led Care, the First Choice for All Women, Netherlands, 2017.

Alba R, Franco R, Patrizia B, Maria CB, Giovanna A, Chiara F, Isabella N. The midwifery-led care model: a continuity of care model in the birth path. Acta Bio Medica: Atenei Parmensis. 2019;90(Suppl 6):41.

Google Scholar  

Dahl B, Heinonen K, Bondas TE. From midwife-dominated to midwifery-led antenatal care: a meta-ethnography. Int J Environ Res Public Health. 2020;17(23):8946.

Article   PubMed   PubMed Central   Google Scholar  

McConville F, Lavender DT. Quality of care and midwifery services to meet the needs of women and newborns. BJOG: Int J Obstet Gynecol. 2014;121.

Shahnaz S, Jan R, Lakhani A, Sikandar R. Factors affecting the midwifery-led service provider model in Pakistan. J Asian Midwives (JAM). 2015;1(2):33–45.

Bogren M, Grahn M, Kaboru BB, Berg M. Midwives’ challenges and factors that motivate them to remain in their workplace in the Democratic Republic of Congo—an interview study. Hum Resour Health. 2020;18:1–0.

Article   Google Scholar  

Bremnes HS, Wiig ÅK, Abeid M, Darj E. Challenges in day-to-day midwifery practice; a qualitative study from a regional referral hospital in Dar Es Salaam. Tanzan Global Health Action. 2018;11(1):1453333.

Yigzaw T, Abebe F, Belay L, Assaye Y, Misganaw E, Kidane A, Ademie D, van Roosmalen J, Stekelenburg J, Kim YM. Quality of midwife-provided intrapartum care in Amhara regional state, Ethiopia. BMC Pregnancy Childbirth. 2017;17:1–2.

Federal Democratic Republic of Ethiopia Mini Demographic and Health Survey. 2019 Ethiopian Public Health Institution, Addis Ababa The DHS Program ICF Rockville, Maryland, USA May 2021.

Federal Democratic Republic of Ethiopia. Demographic and Health Survey 2016 Central Statistical Agency Addis Ababa, Ethiopia The DHS Program ICF Rockville, Maryland, USA July 2017.

UNICEF for every child. Situation Analysis of children and women. Dire Dawa Administration; 2020.

Federal Ministry of. Health, Midwifery care process,2021.

Dire Dawa administration Regional Health Bureau. 2017 six months report [unpublished].

Shenton AK. Strategies for ensuring trustworthiness in qualitative research projects. Educ Inform. 2004;22(2):63–75.

Irene K, Albine M, Series. Practical guidance to qualitative research. Trustworthiness and publishing. Eur J Gen Pract. 2018;24(1):120–4.

Behruzi R, Hatem M, Fraser W, Goulet L, Ii M, Misago C. Facilitators and barriers in the humanization of childbirth practice in Japan. BMC Pregnancy Childbirth. 2010;10:1–8.

Adatara P, Amooba PA, Afaya A, Salia SM, Avane MA, Kuug A, Maalman RS, Atakro CA, Attachie IT, Atachie C. Challenges experienced by midwives working in rural communities in the Upper East Region of Ghana: a qualitative study. BMC Pregnancy Childbirth. 2021;21:1–8.

Roets L. Independent midwifery practice: opportunities and challenges. Afr J Phys Health Educ Recreation Dance. 2014;20(3):1209–24.

Mselle LT, Moland KM, Mvungi A, Evjen-Olsen B, Kohi TW. Why give birth in health facility? Users’ and providers’ accounts of poor quality of birth care in Tanzania. BMC Health Serv Res. 2013;13:1–2.

Bogren M, Erlandsson K, Byrskog U. What prevents midwifery quality care in Bangladesh? A focus group enquiry with midwifery students. BMC Health Serv Res. 2018;18(1):639.

Mtegha MB, Chodzaza E, Chirwa E, Kalembo FW, Zgambo M. Challenges experienced by newly qualified nurse-midwives transitioning to practice in selected midwifery settings in northern Malawi. BMC Nurs. 2022;21(1):236.

Floyd L. Helping midwives in Ghana to reduce maternal mortality. Afr J Midwifery Women’s Health. 2013;7(1):34–8.

Filby A, McConville F, Portela A. What prevents quality midwifery care? A systematic mapping of barriers in low and middle income countries from the provider perspective. PLoS ONE. 2016;11(5):e0153391.

Prytherch H, Kagoné M, Aninanya GA, Williams JE, Kakoko DC, Leshabari MT, Yé M, Marx M, Sauerborn R. Motivation and incentives of rural maternal and neonatal health care providers: a comparison of qualitative findings from Burkina Faso, Ghana and Tanzania. BMC Health Serv Res. 2013;13:1–5.

World Health Organization. The world health report 2000: health systems: improving performance. World Health Organization; 2000.

Oyetunde MO, Nkwonta CA. Quality issues in midwifery: a critical analysis of midwifery in Nigeria within the context of the International Confederation of Midwives (ICM) global standards. Int J Nurs Midwifery. 2014;6(3):40–8.

Kruk ME, Gage AD, Arsenault C, Jordan K, Leslie HH, Roder-DeWan S, Adeyi O, Barker P, Daelmans B, Doubova SV, English M. High-quality health systems in the Sustainable Development goals era: time for a revolution. Lancet Global Health. 2018;6(11):e1196–252.

Article   PubMed   Google Scholar  

Mathauer I, Imhoff I. Health worker motivation in Africa: the role of non-financial incentives and human resource management tools. Hum Resour Health. 2006;4:1–7.

World Health Organization. Global strategy on human resources for health: workforce 2030.

Download references

Acknowledgements

We are very grateful to Dire Dawa University for the financial support for this study and to the College of Medicine and Health for its monitoring ship. All study participants for their willingness to respond to our questionnaire.

this work has been funded by Dire Dawa University for data collection purposes. The Dire Dawa University College of Medicine and Health Sciences was involved in the project through monitoring and evaluation of the work from the beginning to the result submission. However, this organization was not involved in the design, analysis, critical review of its intellectual content, or manuscript preparation, and its budget did not include publication.

Author information

Authors and affiliations.

College of Medicine and Health Sciences, Dire Dawa University, Dire Dawa, Ethiopia

Mickiale Hailu, Aminu Mohammed, Daniel Tadesse, Neil Abdurashid, Legesse Abera, Tadesse Weldeamaniel, Tekleberhan Hailemariam, Netsanet Melkamu, Tewodros Getnet, Yibekal Manaye, Tariku Derese, Muluken Yigezu, Natnael Dechasa & Anteneh Atle

College of Health Sciences, Wachemo University, Hossana, Ethiopia

Samrawit Ali & Yesuneh Dejene

College of Health Sciences, Mekelle University, Mekelle, Ethiopia

Meklit Girma

You can also search for this author in PubMed   Google Scholar

Contributions

MH developed the study proposal, served as the primary lead for study implementation and data analysis/interpretation, and was a major contributor in writing and revising all drafts of the paper. AM, DT, NA, LA, and SA supported study implementation and data analysis, and contributed to writing the initial draft of the paper. YD, TW, MG, TH and, NM supported study recruitment and contributed to writing the final draft of the paper. TG, YM, TD, MY, ND and, AA conceptualized, acquired funding, and led protocol development for the study, co-led study implementation and data analysis/interpretation, and was a major contributor in writing and revising all drafts of the paper. All authors contributed to its content. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Mickiale Hailu .

Ethics declarations

Ethics approval and consent to participate.

All methods were followed in accordance with relevant guidelines and regulations. The institutional review board of Dire Dawa University has also examined and evaluated it for its methodological approach and ethical concerns. Ethical clearance was obtained from Dire Dawa University Institutional Review Board and an official letter from research affairs directorate office of Dire Dawa University was submitted to Dire Dawa health office and it was distributed to selected health institutions. Voluntary informed written consent was obtained from the study participant’s right after the objectives of the study were explained to the study participants and confidentiality of the study participants was assured throughout the study period. Participants were informed that they have the right to terminate the discussion (interview) or they can’t answer any questions they didn’t want to answer.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Hailu, M., Mohammed, A., Tadesse, D. et al. Facilitators and barriers of midwife-led model of care at public health institutions of dire Dawa city, Eastern Ethiopia, 2022: a qualitative study. BMC Health Serv Res 24 , 998 (2024). https://doi.org/10.1186/s12913-024-11417-x

Download citation

Received : 03 September 2023

Accepted : 09 August 2024

Published : 28 August 2024

DOI : https://doi.org/10.1186/s12913-024-11417-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Continuous midwifery care model
• Obstetric care by midwives
• Barriers to obstetric care
• Facilitators to obstetric care

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 31 August 2024

Development and validation of a higher-order thinking skills (HOTS) scale for major students in the interior design discipline for blended learning

• Dandan Li 1 ,
• Xiaolei Fan 2 &
• Lingchao Meng 3  

Scientific Reports volume  14 , Article number:  20287 ( 2024 ) Cite this article

Metrics details

• Environmental social sciences

Assessing and cultivating students’ HOTS are crucial for interior design education in a blended learning environment. However, current research has focused primarily on the impact of blended learning instructional strategies, learning tasks, and activities on the development of HOTS, whereas few studies have specifically addressed the assessment of these skills through dedicated scales in the context of blended learning. This study aimed to develop a comprehensive scale for assessing HOTS in interior design major students within the context of blended learning. Employing a mixed methods design, the research involved in-depth interviews with 10 education stakeholders to gather qualitative data, which informed the development of a 66-item soft skills assessment scale. The scale was administered to a purposive sample of 359 undergraduate students enrolled in an interior design program at a university in China. Exploratory and confirmatory factor analyses were also conducted to evaluate the underlying factor structure of the scale. The findings revealed a robust four-factor model encompassing critical thinking skills, problem-solving skills, teamwork skills, and practical innovation skills. The scale demonstrated high internal consistency (Cronbach's alpha = 0.948–0.966) and satisfactory convergent and discriminant validity. This scale provides a valuable instrument for assessing and cultivating HOTS among interior design major students in blended learning environments. Future research can utilize a scale to examine the factors influencing the development of these skills and inform instructional practices in the field.

Similar content being viewed by others

A meta-analysis of the effects of design thinking on student learning

Blended knowledge sharing model in design professional

Using design thinking for interdisciplinary curriculum design and teaching: a case study in higher education

Introduction.

In the contemporary landscape of the twenty-first century, students face numerous challenges that necessitate the development of competitive skills, with a particular emphasis on the cultivation of HOTS 1 , 2 , 3 , this has become a crucial objective in educational reform. Notably, it is worth noting that the National Education Association (NEA, 2012) has clearly identified critical thinking and problem-solving, communication, collaboration, creativity, and innovation as key competencies that students must possess in the current era, which are considered important components of twenty-first century skills 4 , 5 , 6 , 7 . As learners in the fields of creativity and design, students in the interior design profession also need to possess HOTS to address complex design problems and the evolving demands of the industry 8 , 9 .

Currently, blended learning has become an important instructional model in interior design education 10 , 11 . It serves as a teaching approach that combines traditional face-to-face instruction with online learning, providing students with a more flexible and personalized learning experience 12 , 13 . Indeed, several scholars have recognized the benefits of blended learning in providing students with diverse learning resources, activities, and opportunities for interaction, thereby fostering HOTS 14 , 15 , 16 , 17 . For example, blended learning, as evidenced by studies conducted by Anthony et al. 10 and Castro 11 , has demonstrated its efficacy in enhancing students' HOTS. The integration of online resources, virtual practices, and online discussions in blended learning fosters active student engagement and improves critical thinking, problem solving, and creative thinking skills. Therefore, teachers need to determine appropriate assessment methods and construct corresponding assessment tasks to assess students' expected learning outcomes. This decision requires teachers to have a clear understanding of students' learning progress and the development of various skills, whereas students have knowledge of only their scores and lack awareness of their individual skill development 18 , 19 .

Nevertheless, the precise assessment of students' HOTS in the blended learning milieu poses a formidable challenge. The dearth of empirically validated assessment tools impedes researchers from effectively discerning students' levels of cognitive aptitude and developmental growth within the blended learning realm 20 , 21 , 22 . In addition, from the perspective of actual research topics, current studies on blended learning focus mainly on the "concept, characteristics, mechanisms, models, and supporting technologies of blended learning 23 . " Research on "measuring students' HOTS in blended learning" is relatively limited, with most of the focus being on elementary, middle, and high school students 24 , 25 . Few studies have specifically examined HOTS measurement in the context of university students 26 , 27 , particularly in practical disciplines such as interior design. For example, Bervell et al. 28 suggested that the lack of high-quality assessment scales inevitably impacts the quality of research. Additionally, Schmitt 29 proposed the “Three Cs” principle for measurement, which includes clarity, coherence, and consistency. He highlighted that high-quality assessment scales should possess clear and specific measurement objectives, logically coherent items, and consistent measurement results to ensure the reliability and validity of the data. This reflects the importance of ensuring the alignment of the measurement goals of assessment scales with the research questions and the content of the discipline in the design of assessments.

The development of an assessment scale within the blended learning environment is expected to address the existing gap in measuring and assessing HOTS scores in interior design education. This scale not only facilitates the assessment of students' HOTS but also serves as a guide for curriculum design, instructional interventions, and student support initiatives. Ultimately, the integration of this assessment scale within the blended learning environment has the potential to optimize the development of HOTS among interior design students, empowering them to become adept critical thinkers, creative problem solvers, and competent professionals in the field.

Therefore, this study follows a scientific scale development procedure to develop an assessment scale specifically designed to measure the HOTS of interior design students in blended learning environments. This endeavor aims to provide educators with a reliable instrument for assessing students' progress in cultivating and applying HOTS, thus enabling the implementation of more effective teaching strategies and enhancing the overall quality of interior design education. The research questions are as follows:

What key dimensions should be considered when developing a HOTS assessment scale to accurately capture students' HOTS in an interior design major blended learning environment?

How can an advanced thinking skills assessment scale for blended learning in interior design be developed?

How can the reliability and validity of the HOTS assessment scale be verified and ensured, and is it reliable and effective in the interior design of major blended learning environments?

Key dimensions of HOTS assessment scale in an interior design major blended learning environment

The research results indicate that in the blended learning environment of interior design, this study identified 16 initial codes representing key dimensions for enhancing students' HOTS. These codes were further categorized into 8 main categories and 4 overarching themes: critical thinking, problem-solving, teamwork skills and practical innovation skills. They provide valuable insights for data comprehension and analysis, serving as a comprehensive framework for the HOTS scale. Analyzing category frequency and assessing its significance and universality in a qualitative dataset hold significant analytical value 30 , 31 . High-frequency terms indicate the central position of specific categories in participants' narratives, texts, and other data forms 32 . Through interviews with interior design experts and teachers, all core categories were mentioned more than 20 times, providing compelling evidence of their universality and importance within the field of interior design's HOTS dimensions. As shown in Table 1 .

Themes 1: critical thinking skills

Critical thinking skills constitute a key core category in blended learning environments for interior design and are crucial for cultivating students' HOTS. This discovery emphasizes the importance of critical thinking in interior design learning. This mainly includes the categories of logical reasoning and judgment, doubt and reflection, with a frequency of more than 8, highlighting the importance of critical thinking skills. Therefore, a detailed discussion of each feature is warranted. As shown in Table 2 .

Category 1: logical reasoning and judgment

The research results indicate that in a blended learning environment for interior design, logical reasoning and judgment play a key role in cultivating critical thinking skills. Logical reasoning refers to inferring reasonable conclusions from information through analysis and evaluation 33 . Judgment is based on logic and evidence for decision-making and evaluation. The importance of these concepts lies in their impact on the development and enhancement of students' HOTS. According to the research results, interior design experts and teachers unanimously believe that logical reasoning and judgment are very important. For example, as noted by Interviewee 1, “For students, logical reasoning skills are still very important. Especially in indoor space planning, students use logical reasoning to determine whether the layout of different functional areas is reasonable”. Similarly, Interviewee 2 also stated that “logical reasoning can help students conduct rational analysis of various design element combinations during the conceptual design stage, such as color matching, material selection, and lighting application”.

As emphasized by interviewees 1 and 2, logical reasoning and judgment are among the core competencies of interior designers in practical applications. These abilities enable designers to analyze and evaluate design problems and derive reasonable solutions from them. In the interior design industry, being able to conduct accurate logical reasoning and judgment is one of the key factors for success. Therefore, through targeted training and practice, students can enhance their logical thinking and judgment, thereby better addressing design challenges and providing innovative solutions.

Category 2: skepticism and reflection

Skepticism and reflection play crucial roles in cultivating students' critical thinking skills in a blended learning environment for interior design. Doubt can prompt students to question and explore information and viewpoints, whereas reflection helps students think deeply and evaluate their own thinking process 34 . These abilities are crucial for cultivating students' higher-order thinking skills. According to the research findings, most interior design experts and teachers agree that skepticism and reflection are crucial. For example, as noted by interviewees 3, “Sometimes, when facing learning tasks, students will think about how to better meet the needs of users”. Meanwhile, Interviewee 4 also agreed with this viewpoint. As emphasized by interviewees 3 and 4, skepticism and reflection are among the core competencies of interior designers in practical applications. These abilities enable designers to question existing perspectives and practices and propose innovative design solutions through in-depth thinking and evaluation. Therefore, in the interior design industry, designers with the ability to doubt and reflect are better able to respond to complex design needs and provide clients with unique and valuable design solutions.

Themes 2: problem-solving skills

The research findings indicate that problem-solving skills constitute a key core category in blended learning environments for interior design and are crucial for cultivating students' HOTS. This discovery emphasizes the importance of problem-solving skills in interior design learning. Specifically, categories such as identifying and defining problems, as well as developing and implementing plans, have been studied more than 8 times, highlighting the importance of problem-solving skills. Therefore, it is necessary to discuss each function in detail to better understand and cultivate students' problem-solving skills. As shown in Table 3 .

Category 1: identifying and defining issues

The research findings indicate that in a blended learning environment for interior design, identifying and defining problems play a crucial role in fostering students' problem-solving skills. Identifying and defining problems require students to possess the ability to analyze and evaluate problems, enabling them to accurately determine the essence of the problems and develop effective strategies and approaches to solve them 35 . Interior design experts and teachers widely recognize the importance of identifying and defining problems as core competencies in interior design practice. For example, Interviewee 5 emphasized the importance of identifying and defining problems, stating, "In interior design, identifying and defining problems is the first step in addressing design challenges. Students need to be able to clearly identify the scope, constraints, and objectives of the problems to engage in targeted thinking and decision-making in the subsequent design process." Interviewee 6 also supported this viewpoint. As stressed by Interviewees 5 and 6, identifying and defining problems not only require students to possess critical thinking abilities but also necessitate broad professional knowledge and understanding. Students need to comprehend principles of interior design, spatial planning, human behavior, and other relevant aspects to accurately identify and define problems associated with design tasks.

Category 2: developing and implementing a plan

The research results indicate that in a blended learning environment for interior design, developing and implementing plans plays a crucial role in cultivating students' problem-solving abilities. The development and implementation of a plan refers to students identifying and defining problems, devising specific solutions, and translating them into concrete implementation plans. Specifically, after determining the design strategy, students refine it into specific implementation steps and timelines, including drawing design drawings, organizing PPT reports, and presenting design proposals. For example, Interviewee 6 noted, “Students usually break down design strategies into specific tasks and steps by refining them.” Other interviewees also unanimously support this viewpoint. As emphasized by respondent 6, developing and implementing plans can help students maintain organizational, systematic, and goal-oriented problem-solving skills, thereby enhancing their problem-solving skills.

Themes 3: teamwork skills

The research results indicate that teamwork skills constitute a key core category in blended learning environments for interior design and are crucial for cultivating students' HOTS. This discovery emphasizes the importance of teamwork skills in interior design learning. This mainly includes communication and coordination and division of labor and collaboration, which are mentioned frequently in the interview documents. Therefore, it is necessary to discuss each function in detail to better understand and cultivate students' teamwork skills. As shown in Table 4 .

Category 1: communication and coordination

The research results indicate that communication and collaboration play crucial roles in cultivating students' teamwork abilities in a blended learning environment for interior design. Communication and collaboration refer to the ability of students to effectively share information, understand each other's perspectives, and work together to solve problems 36 . Specifically, team members need to understand each other's resource advantages integrate and share these resources to improve work efficiency and project quality. For example, Interviewee 7 noted, “In interior design, one member may be skilled in spatial planning, while another member may be skilled in color matching. Through communication and collaboration, team members can collectively utilize this expertise to improve work efficiency and project quality.” Other interviewees also unanimously believe that this viewpoint can promote students' teamwork skills, thereby promoting the development of their HOTS. As emphasized by the viewpoints of these interviewees, communication and collaboration enable team members to collectively solve problems and overcome challenges. Through effective communication, team members can exchange opinions and suggestions with each other, provide different solutions, and make joint decisions. Collaboration and cooperation among team members contribute to brainstorming and finding the best solution.

Category 2: division of labor and collaboration

The research results indicate that in the blended learning environment of interior design, the division of labor and collaboration play crucial roles in cultivating students' teamwork ability. The division of labor and collaboration refer to the ability of team members to assign different tasks and roles in a project based on their respective expertise and responsibilities and work together to complete the project 37 . For example, Interviewee 8 noted, “In an internal design project, some students are responsible for space planning, some students are responsible for color matching, and some students are responsible for rendering production.” Other interviewees also support this viewpoint. As emphasized by interviewee 8, the division of labor and collaboration help team members fully utilize their respective expertise and abilities, promote resource integration and complementarity, cultivate a spirit of teamwork, and enable team members to collaborate, support, and trust each other to achieve project goals together.

Themes 4: practical innovation skills

The research results indicate that practical innovation skills constitute a key core category in blended learning environments for interior design and are crucial for cultivating students' HOTS. This discovery emphasizes the importance of practical innovation skills in interior design learning. This mainly includes creative conception and design expression, as well as innovative application of materials and technology, which are often mentioned in interview documents. Therefore, it is necessary to discuss each function in detail to better understand and cultivate students' practical innovation skills. As shown in Table 5 .

Category 1: creative conception and design expression

The research results indicate that in the blended learning environment of interior design, creative ideation and design expression play crucial roles in cultivating students' practical and innovative skills. Creative ideation and design expression refer to the ability of students to break free from traditional thinking frameworks and try different design ideas and methods through creative ideation, which helps stimulate their creativity and cultivate their ability to think independently and solve problems. For example, interviewee 10 noted that "blended learning environments combine online and offline teaching modes, allowing students to acquire knowledge and skills more flexibly. Through learning and practice, students can master various expression tools and techniques, such as hand-drawn sketches, computer-aided design software, model making, etc., thereby more accurately conveying their design concepts." Other interviewees also expressed the importance of this viewpoint, emphasizing the importance of creative ideas and design expression in blended learning environments that cannot be ignored. As emphasized by interviewee 10, creative ideation and design expression in the blended learning environment of interior design can not only enhance students' creative thinking skills and problem-solving abilities but also strengthen their application skills in practical projects through diverse expression tools and techniques. The cultivation of these skills is crucial for students' success in their future careers.

Category 2: innovative application of materials and technology

Research findings indicate that the innovative application of materials and technology plays a crucial role in developing students' practical and creative skills within a blended learning environment for interior design. The innovative application of materials and technology refers to students' exploration and utilization of new materials and advanced technologies, enabling them to overcome the limitations of traditional design thinking and experiments with diverse design methods and approaches. This process not only stimulates their creativity but also significantly enhances their problem-solving skills. Specifically, the innovative application of materials and technology involves students gaining a deep understanding of the properties of new materials and their application methods in design, as well as becoming proficient in various advanced technological tools and equipment, such as 3D printing, virtual reality (VR), and augmented reality (AR). These skills enable students to more accurately realize their design concepts and effectively apply them in real-world projects.

For example, Interviewee 1 stated, "The blended learning environment combines online and offline teaching modes, allowing students to flexibly acquire the latest knowledge on materials and technology and apply these innovations in real projects." Other interviewees also emphasized the importance of this view. Therefore, the importance of the innovative application of materials and technology in a blended learning environment cannot be underestimated. As emphasized by interviewee 1, the innovative application of materials and technologies is crucial in the blended learning environment of interior design. This process not only enables students to flexibly acquire the latest materials and technical knowledge but also enables them to apply these innovations to practice in practical projects, thereby improving their practical abilities and professional ethics.

In summary, through research question 1 research, the dimensions of the HOTS assessment scale in blended learning for interior design include four main aspects: critical thinking skills, problem-solving skills, teamwork skills, and practical innovation skills. Based on the assessment scales developed by previous scholars in various dimensions, the researcher developed a HOTS assessment scale suitable for blended learning environments in interior design and collected feedback from interior design experts through interviews.

Development of the HOTS assessment scale

The above research results indicate that the dimensions of the HOTS scale mainly include critical thinking, problem-solving, teamwork skills and practical innovation skills. The dimensions of a scale represent the abstract characteristics and structure of the concept being measured. Since these dimensions are often abstract and difficult to measure directly, they need to be converted into several concrete indicators that can be directly observed or self-reported 38 . These concrete indicators, known as dimension items, operationalize the abstract dimensions, allowing for the measurement and evaluation of various aspects of the concept. This process transforms the abstract dimensions into specific, measurable components. The following content is based on the results of research question 1 to develop an advanced thinking skills assessment scale for mixed learning in interior design.

Dimension 1: critical thinking skills

The research results indicate that critical thinking skills constitute a key core category in blended learning environments for interior design and are crucial for cultivating students' HOTS. Critical thinking skills refer to the ability to analyze information objectively and make a reasoned judgment 39 . Scholars tend to emphasize this concept as a method of general skepticism, rational thinking, and self-reflection 7 , 40 . For example, Goodsett 26 suggested that it should be based on rational skepticism and careful thought about external matters as well as open self-reflection about internal thoughts and actions. Moreover, the California Critical Thinking Disposition Inventory (CCTDI) is widely used to measure critical thinking skills, including dimensions such as seeking truth, confidence, questioning and courage to seek truth, curiosity and openness, as well as analytical and systematic methods 41 . In addition, maturity means continuous adjustment and improvement of a person's cognitive system and learning activities through continuous awareness, reflection, and self-awareness 42 . Moreover, Nguyen 43 confirmed that critical thinking and cognitive maturity can be achieved through these activities, emphasizing that critical thinking includes cognitive skills such as analysis, synthesis, and evaluation, as well as emotional tendencies such as curiosity and openness.

In addition, in a blended learning environment for interior design, critical thinking skills help students better understand, evaluate, and apply design knowledge and skills, cultivating independent thinking and innovation abilities 44 . If students lack these skills, they may accept superficial information and solutions without sufficient thinking and evaluation, resulting in the overlooking of important details or the selection of inappropriate solutions in the design process. Therefore, for the measurement of critical thinking skills, the focus should be on cognitive skills such as analysis, synthesis, and evaluation, as well as curiosity and open mindedness. The specific items for critical thinking skills are shown in Table 6 .

Dimension 2: problem-solving skills

Problem-solving skills constitute a key core category in blended learning environments for interior design and are crucial for cultivating students' HOTS. Problem-solving skills involve the ability to analyze and solve problems by understanding them, identifying their root causes, and developing appropriate solutions 45 . According to the 5E-based STEM education approach, problem-solving skills encompass the following abilities: problem identification and definition, formulation of problem-solving strategies, problem representation, resource allocation, and monitoring and evaluation of solution effectiveness 7 , 46 . Moreover, D'zurilla and Nezu 47 and Tan 48 indicated that attitudes, beliefs, and knowledge skills during problem solving, as well as the quality of proposed solutions and observable outcomes, are demonstrated. In addition, D'Zurilla and Nezu devised the Social Problem-Solving Inventory (SPSI), which comprises seven subscales: cognitive response, emotional response, behavioral response, problem identification, generation of alternative solutions, decision-making, and solution implementation. Based on these research results, the problem-solving skills dimension questions designed in this study are shown in Table 7 .

Dimension 3: teamwork skills

The research results indicate that teamwork skills constitute a key core category in blended learning environments for interior design and are crucial for cultivating students' HOTS. Teamwork skills refer to the ability to effectively collaborate, coordinate, and communicate with others in a team environment 49 . For example, the Teamwork Skills Assessment Tool (TWKSAT) developed by Stevens and Campion 50 identifies five core dimensions of teamwork: conflict management; collaborative problem-solving; communication; goal setting; performance management; decision-making; and task coordination. The design of this tool highlights the essential skills in teamwork and provides a structured approach for evaluating these skills. In addition, he indicated that successful teams need to have a range of skills for problem solving, including situational control, conflict management, decision-making and coordination, monitoring and feedback, and an open mindset. These skills help team members effectively address complex challenges and demonstrate the team’s collaboration and flexibility. Therefore, the assessment of learners' teamwork skills needs to cover the above aspects. As shown in Table 8 .

Dimension 4: practice innovative skills

The research results indicate that practical innovation skills constitute a key core category in blended learning environments for interior design, which is crucial for cultivating students' HOTS. The practice of innovative skills encompasses the utilization of creative cognitive processes and problem-solving strategies to facilitate the generation of original ideas, solutions, and approaches 51 . This practice places significant emphasis on two critical aspects: creative conception and design expression, as well as the innovative application of materials and technology. Tang et al. 52 indicated that creative conception and design expression involve the generation and articulation of imaginative and inventive ideas within a given context. With the introduction of concepts such as 21st-century learning skills, the "5C" competency framework, and core student competencies, blended learning has emerged as the goal and direction of educational reform. It aims to promote the development of students' HOTS, equipping them with the essential qualities and key abilities needed for lifelong development and societal advancement. Blended learning not only emphasizes the mastery of core learning content but also requires students to develop critical thinking, complex problem-solving, creative thinking, and practical innovation skills. To adapt to the changes and developments in the blended learning environment, this study designed 13 preliminary test items based on 21st-century learning skills, the "5C" competency framework, core student competencies, and the TTCT assessment scale developed by Torrance 53 . These items aim to assess students' practice of innovative skills within a blended learning environment, as shown in Table 9 .

The researchers' results indicate that the consensus among the interviewed expert participants is that the structural integrity of the scale is satisfactory and does not require modification. However, certain measurement items have been identified as problematic and require revision. The primary recommendations are as follows: Within the domain of problem-solving skills, the item "I usually conduct classroom and online learning with questions and clear goals" was deemed biased because of its emphasis on the "online" environment. Consequently, the evaluation panel advised splitting this item into two separate components: (1) "I am adept at frequently adjusting and reversing a negative team atmosphere" and (2) "I consistently engage in praising and encouraging others, fostering harmonious relationships. “The assessment process requires revisions and adjustments to specific projects, forming a pilot test scale consisting of 66 observable results from the original 65 items. In addition, there were other suggestions about linguistic formulation and phraseology, which are not expounded upon herein.

Verify the effectiveness of the HOTS assessment scale

The research results indicate that there are significant differences in the average scores of the four dimensions of the HOTS, including critical thinking skills (A1–A24 items), problem-solving skills (B1–B13 items), teamwork skills (C1–C16 items), and practical innovation skills (D1–D13 items). Moreover, this also suggests that each item has discriminative power. Specifically, this will be explained through the following aspects.

Project analysis based on the CR value

The critical ratio (CR) method, which uses the CR value (decision value) to remove measurement items with poor discrimination, is the most used method in project analysis. The specific process involves the use of the CR value (critical value) to identify and remove such items. First, the modified pilot test scale data are aggregated and sorted. Individuals representing the top and bottom 27% of the distribution were subsequently selected, constituting 66 respondents in each group. The high-score group comprises individuals with a total score of 127 or above (including 127), whereas the low-score group comprises individuals with a total score of 99 or below (including 99). Finally, an independent sample t test was conducted to determine the significant differences in the mean scores for each item between the high-score and low-score groups. The statistical results are presented in Table 10 .

The above table shows that independent sample t tests were conducted for all the items; their t values were greater than 3, and their p values were less than 0.001, indicating that the difference between the highest and lowest 27% of the samples was significant and that each item had discriminative power.

In summary, based on previous research and relevant theories, the HOTS scale for interior design was revised. This revision process involved interviews with interior design experts, teachers, and students, followed by item examination and homogeneity testing via the critical ratio (CR) method. The results revealed significant correlations ( p  < 0.01) between all the items and the total score, with correlation coefficients (R) above 0.4. Therefore, the scale exhibits good accuracy and internal consistency in capturing measured HOTS. These findings provide a reliable foundation for further research and practical applications.

Pilot study exploratory factor analysis

This study used SPSS (version 28) to conduct the KMO and Bartlett tests on the scale. The total HOTS test scale as well as the KMO and Bartlett sphericities were first calculated for the four subscales to ensure that the sample data were suitable for factor analysis 7 . The overall KMO value is 0.946, indicating that the data are highly suitable for factor analysis. Additionally, Bartlett's test of sphericity was significant, further supporting the appropriateness of conducting factor analysis ( p  < 0.05). All the values are above 0.7, indicating that the data for these subscales are also suitable for factor analysis. According to Javadi et al. 54 , these results suggest the presence of shared factors among the items within the subscales, as shown in Table 11 .

For each subscale, exploratory factor analysis was conducted to extract factors with eigenvalues greater than 1 while eliminating items with communalities less than 0.30, loadings less than 0.50, and items that cross multiple (more than one) common factors 55 , 56 . Additionally, items that were inconsistent with the assumed structure of the measure were identified and eliminated to ensure the best structural validity. These principles were applied to the factor analysis of each subscale, ensuring that the extracted factor structure and observed items are consistent with the hypothesized measurement structure and analysis results, as shown in the table 55 , 58 . In the exploratory factor analysis (EFA), the latent variables were effectively interpreted and demonstrated a significant response, with cumulative explained variances of the common factors exceeding 60%. This finding confirms the alignment between the scale structure, comprising the remaining items, and the initial theoretical framework proposed in this study. Additionally, the items were systematically reorganized to construct the final questionnaire. Consequently, items A1 to A24 were associated with the critical thinking skills dimension, items B25 to B37 were linked to problem-solving skills, items C38 to C53 were indicative of teamwork skills, and items D54 to D66 were reflective of practical innovation skills. As shown in Table 12 below.

In addition, the criterion for extracting principal components in factor analysis is typically based on eigenvalues, with values greater than 1 indicating greater explanatory power than individual variables. The variance contribution ratio reflects the proportion of variance explained by each principal component relative to the total variance and signifies the ability of the principal component to capture comprehensive information. The cumulative variance contribution ratio measures the accumulated proportion of variance explained by the selected principal components, aiding in determining the optimal number of components to retain while minimizing information loss. The above table shows that four principal components can be extracted from the data, and their cumulative variance contribution rate reaches 59.748%.

However, from the scree plot (as shown in Fig.  1 ), the slope flattens starting from the fifth factor, indicating that no distinct factors can be extracted beyond that point. Therefore, retaining four factors seems more appropriate. The factor loading matrix is the core of factor analysis, and the values in the matrix represent the factor loading of each item on the common factors. Larger values indicate a stronger correlation between the item variable and the common factor. For ease of analysis, this study used the maximum variance method to rotate the initial factor loading matrix, redistributing the relationships between the factors and original variables and making the correlation coefficients range from 0 to 1, which facilitates interpretation. In this study, factor loadings with absolute values less than 0.4 were filtered out. According to the analysis results, the items of the HOTS assessment scale can be divided into four dimensions, which is consistent with theoretical expectations.

Gravel plot of factors.

Through the pretest of the scale and selection of measurement items, 66 measurement items were ultimately determined. On this basis, a formal scale for assessing HOTS in a blended learning environment was developed, and the reliability and validity of the scale were tested to ultimately confirm its usability.

Confirmatory factor analysis of final testing

Final test employed that AMOS (version 26.0), a confirmatory factor analysis (CFA) was conducted on the retested sample data to validate the stability of the HOTS structural model obtained through exploratory factor analysis. This analysis aimed to assess the fit between the measurement results and the actual data, confirming the robustness of the derived HOTS structure and its alignment with the empirical data. The relevant model was constructed based on the factor structure of each component obtained through EFA and the observed variables, as shown in the diagram. The model fit indices are presented in Fig.  2 (among them, A represents critical thinking skills, B represents problem-solving skills, C represents teamwork skills, and D represents practical innovation skills). The models strongly support the "4-dimensional" structure of the HOTS, which includes four first-order factors: critical thinking skills, problem-solving skills, teamwork skills, and practical innovation skills. Critical thinking skills play a pivotal role in the blended learning environment of interior design, connecting problem-solving skills, teamwork skills, and innovative practices. These four dimensions form the assessment structure of HOTS, with critical thinking skills serving as the core element, inspiring individuals to assess problems and propose innovative solutions. By providing appropriate learning resources, diverse learning activities, and learning tasks, as well as designing items for assessment scales, it is possible to delve into the measurement and development of HOTS in the field of interior design, providing guidance for educational and organizational practices. This comprehensive approach to learning and assessment helps cultivate students' HOTS and lays a solid foundation for their comprehensive abilities in the field of interior design. Thus, the CFA structural models provide strong support for the initial hypothesis of the proposed HOTS assessment structure in this study. As shown in Fig.  2 .

Confirmatory factor analysis based on 4 dimensions. *A represents the dimension of critical thinking. B represents the dimension of problem-solving skills. C represents the dimension of teamwork skills. D represents the dimension of practical innovation skills.

Additionally, χ2. The fitting values of RMSEA and SRMR are both below the threshold, whereas the fitting values of the other indicators are all above the threshold, indicating that the model fits well. As shown in Table 13 .

Reliability and validity analysis

The reliability and validity of the scale need to be assessed after the model fit has been determined through validation factor analysis 57 . Based on the findings of Marsh et al. 57 , the following conclusions can be drawn. In terms of hierarchical and correlational model fit, the standardized factor loadings of each item range from 0.700 to 0.802, all of which are greater than or equal to 0.7. This indicates a strong correspondence between the observed items and each latent variable. Furthermore, the Cronbach's α coefficients, which are used to assess the internal consistency or reliability of the scale, ranged from 0.948 to 0.966 for each dimension, indicating a high level of data reliability and internal consistency. The composite reliabilities ranged from 0.948 to 0.967, exceeding the threshold of 0.6 and demonstrating a substantial level of consistency (as shown in Table 14 ).

Additionally, the diagonal bold font represents the square root of the AVE for each dimension. All the dimensions have average variance extracted (AVE) values ranging from 0.551 to 0.589, all of which are greater than 0.5, indicating that the latent variables have strong explanatory power for their corresponding items. These results suggest that the scale structure constructed in this study is reliable and effective. Furthermore, according to the results presented in Table 15 , the square roots of the AVE values for each dimension are greater than the absolute values of the correlations with other dimensions, indicating discriminant validity of the data. Therefore, these four subscales demonstrate good convergent and discriminant validity, indicating that they are both interrelated and independent. This implies that they can effectively capture the content required to complete the HOTS test scale.

Discussion and conclusion

The assessment scale for HOTS in interior design blended learning encompasses four dimensions: critical thinking skills, problem-solving skills, teamwork skills, and practical innovation skills. The selection of these dimensions is based on the characteristics and requirements of the interior design discipline, which aims to comprehensively evaluate students' HOTS demonstrated in blended learning environments to better cultivate their ability to successfully address complex design projects in practice. Notably, multiple studies have shown that HOTSs include critical thinking, problem-solving skills, creative thinking, and decision-making skills, which are considered crucial in various fields, such as education, business, and engineering 20 , 59 , 60 , 61 . Compared with prior studies, these dimensions largely mirror previous research outcomes, with notable distinctions in the emphasis on teamwork skills and practical innovation skills 62 , 63 . Teamwork skills underscore the critical importance of collaboration in contemporary design endeavors, particularly within the realm of interior design 64 . Effective communication and coordination among team members are imperative for achieving collective design objectives.

Moreover, practical innovation skills aim to increase students' capacity for creatively applying theoretical knowledge in practical design settings. Innovation serves as a key driver of advancement in interior design, necessitating students to possess innovative acumen and adaptability to evolving design trends for industry success. Evaluating practical innovation skills aims to motivate students toward innovative thinking, exploration of novel concepts, and development of unique design solutions, which is consistent with the dynamic and evolving nature of the interior design sector. Prior research suggests a close interplay between critical thinking, problem-solving abilities, teamwork competencies, and creative thinking, with teamwork skills acting as a regulatory factor for critical and creative thought processes 7 , 65 . This interconnected nature of HOTS provides theoretical support for the construction and validation of a holistic assessment framework for HOTS.

After the examination by interior design expert members, one item needed to be split into two items. The results of the CR (construct validity) analysis of the scale items indicate that independent sample t tests were subsequently conducted on all the items. The t values were greater than 3, with p values less than 0.001, indicating significant differences between the top and bottom 27% of the samples and demonstrating the discriminant validity of each item. This discovery highlights the diversity and effectiveness of the scale's internal items, revealing the discriminatory power of the scale in assessing the study subjects. The high t values and significant p values reflect the substantiality of the internal items in distinguishing between different sample groups, further confirming the efficacy of these items in evaluating the target characteristics. These results provide a robust basis for further refinement and optimization of the scale and offer guidance for future research, emphasizing the importance of scale design in research and providing strong support for data interpretation and analysis.

This process involves evaluating measurement scales through EFA, and it was found that the explanatory variance of each subscale reached 59.748%, and the CR, AVE, Cronbach's alpha, and Pearson correlation coefficient values of the total scale and subscales were in a better state, which strongly demonstrates the structure, discrimination, and convergence effectiveness of the scale 57 .

The scale structure and items of this study are reliable and effective, which means that students in the field of interior design can use them to test their HOTS level and assess their qualities and abilities. In addition, scholars can use this scale to explore the relationships between students' HOTS and external factors, personal personalities, etc., to determine different methods and strategies for developing and improving HOTS.

Limitations and future research

The developed mixed learning HOTS assessment scale for interior design also has certain limitations that need to be addressed in future research. The first issue is that, owing to the requirement of practical innovation skills, students need to have certain practical experience and innovative abilities. First-grade students usually have not yet had sufficient opportunities for learning and practical experience, so it may not be possible to evaluate their abilities effectively in this dimension. Therefore, when this scale is used for assessment, it is necessary to consider students' grade level and learning experience to ensure the applicability and accuracy of the assessment tool. For first-grade students, it may be necessary to use other assessment tools that are suitable for their developmental stage and learning experience to evaluate other aspects of their HOTS 7 . Future research should focus on expanding the scope of this dimension to ensure greater applicability.

The second issue is that the sample comes from ordinary private undergraduate universities in central China and does not come from national public universities or key universities. Therefore, there may be regional characteristics in the obtained data. These findings suggest that the improved model should be validated with a wider range of regional origins, a more comprehensive school hierarchy, and a larger sample size. The thirdly issue is the findings of this study are derived from self-reported data collected from participants through surveys. However, it is important to note that the literature suggests caution in heavily relying on such self-reported data, as perception does not always equate to actions 66 . In addition, future research can draw on this scale to evaluate the HOTS of interior design students, explore the factors that affect their development, determine their training and improvement paths, and cultivate skilled talent for the twenty-first century.

This study adopts a mixed method research approach, combining qualitative and quantitative methods to achieve a comprehensive understanding of the phenomenon 67 . By integrating qualitative and quantitative research methods, mixed methods research provides a comprehensive and detailed exploration of research questions, using multiple data sources and analytical methods to obtain accurate and meaningful answers 68 . To increase the quality of the research, the entire study followed the guidelines for scale development procedures outlined by Professor Li after the data were obtained. As shown in Fig.  3

Scale development program.

Basis of theory

This study is guided by educational objectives such as 21st-century learning skills, the "5C" competency framework, and students' core abilities 4 . The construction process of the scale is based on theoretical foundations, including Bloom's taxonomy. Drawing from existing research, such as the CCTDI 41 , SPSI 69 , and TWKSAT scales, the dimensions and preliminary items of the scale were developed. Additionally, to enhance the validity and reliability of the scale, dimensions related to HOTS in interior design were obtained through semi-structured interviews, and the preliminary project adapted or directly cited existing research results. The preliminary items were primarily adapted or directly referenced from existing research findings. Based on existing research, such as the CCTDI, SPSI, TWKSAT, and twenty-first century skills frameworks, this study takes "critical thinking skills, problem-solving skills, teamwork skills, and practical innovative skills" as the four basic dimensions of the scale.

Participants and procedures

This study is based on previous research and develops a HOTS assessment scale to measure the thinking levels of interior design students in blended learning. By investigating the challenges and opportunities students encounter in blended learning environments and exploring the complexity and diversity of their HOTS, this study aims to obtain comprehensive insights. For research question 1, via the purposive sampling method, 10 interior design experts are selected to investigate the dimensions and evaluation indicators of HOTS in blended learning of interior design. The researcher employed a semi structured interview method, and a random sampling technique was used to select 10 senior experts and teachers in the field of interior design, holding the rank of associate professor or above. This included 5 males and 5 females. As shown in Table 16 .

For research question 2 and 3, the research was conducted at an undergraduate university in China, in the field of interior design and within a blended learning environment. In addition, a statement confirms that all experimental plans have been approved by the authorized committee of Zhengzhou University of Finance and Economics. In the process of practice, the methods used were all in accordance with relevant guidelines and regulations, and informed consent was obtained from all participants. The Interior Design Blended Learning HOTS assessment scale was developed based on sample data from 350 students who underwent one pre-test and retest. The participants in the study consisted of second-, third-, and fourth-grade students who had participated in at least one blended learning course. The sample sizes were 115, 118, and 117 for the respective grade levels, totaling 350 individuals. Among the participants, there were 218 male students and 132 female students, all of whom were within the age range of 19–22 years. Through purposeful sampling, this study ensured the involvement of relevant participants and focused on a specific university environment with diverse demographic characteristics and rich educational resources.

This approach enhances the reliability and generalizability of the research and contributes to a deeper understanding of the research question (as shown in Table 17 ).

Instruments

The tools used in this study include semi structured interview guidelines and the HOTS assessment scale developed by the researchers. For research question 1, the semi structured interview guidelines were reviewed by interior design experts to ensure the accuracy and appropriateness of their content and questions. In addition, for research question 2 and 3, the HOTS assessment scale developed by the researchers will be checked via the consistency ratio (CR) method to assess the consistency and reliability of the scale items and validate their effectiveness.

Data analysis

For research question 1, the researcher will utilize the NVivo version 14 software tool to conduct thematic analysis on the data obtained through semi structured interviews. Thematic analysis is a commonly used qualitative research method that aims to identify and categorize themes, concepts, and perspectives that emerge within a dataset 70 . By employing NVivo software, researchers can effectively organize and manage large amounts of textual data and extract themes and patterns from them.

For research question 2, the critical ratio (CR) method was employed to conduct item analysis and homogeneity testing on the items of the pilot test questionnaire. The CR method allows for the assessment of each item's contribution to the total score and the evaluation of the interrelationships among the items within the questionnaire. These analytical techniques served to facilitate the evaluation and validation of the scale's reliability and validity.

For research question 3, this study used SPSS (version 26), in which confirmatory factor analysis (CFA) was conducted on the confirmatory sample data via maximum likelihood estimation. The purpose of this analysis was to verify whether the hypothesized factor structure model of the questionnaire aligned with the actual survey data. Finally, several indices, including composite reliability (CR), average variance extracted (CR), average variance extracted (AVE), Cronbach's alpha coefficient, and the Pearson correlation coefficient, were computed to assess the reliability and validity of the developed scale and assess its reliability and validity.

In addition, exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) are commonly utilized techniques in questionnaire development and adaptation research 31 , 70 . The statistical software packages SPSS and AMOS are frequently employed for implementing these analytical techniques 71 , 72 , 73 . CFA is a data-driven approach to factor generation that does not require a predetermined number of factors or specific relationships with observed variables. Its focus lies in the numerical characteristics of the data. Therefore, prior to conducting CFA, survey questionnaires are typically constructed through EFA to reveal the underlying structure and relationships between observed variables and the latent structure.

In contrast, CFA tests the hypothesized model structure under specific theoretical assumptions or structural hypotheses, including the interrelationships among factors and the known number of factors. Its purpose is to validate the hypothesized model structure. Thus, the initial validity of the questionnaire structure, established through EFA, necessitates further confirmation through CFA 57 , 70 . Additionally, a sample size of at least 200 is recommended for conducting the validation factor analysis. In this study, confirmatory factor analysis was performed on a sample size of 317.

Data availability

All data generated or analyzed during this study are included in this published article. All the experimental protocols were approved by the Zhengzhou College of Finance and Economics licensing committee.

Hariadi, B. et al. Higher order thinking skills based learning outcomes improvement with blended web mobile learning Model. Int. J. Instr. 15 (2), 565–578 (2022).

Google Scholar  

Sagala, P. N. & Andriani, A. Development of higher-order thinking skills (HOTS) questions of probability theory subject based on bloom’s taxonomy. J. Phys. Conf. Ser. https://doi.org/10.1088/1742-6596/1188/1/012025 (2019).

Article   Google Scholar  

Yudha, R. P. Higher order thinking skills (HOTS) test instrument: Validity and reliability analysis with the rasch model. Eduma Math. Educ. Learn. Teach. https://doi.org/10.24235/eduma.v12i1.9468 (2023).

Leach, S. M., Immekus, J. C., French, B. F. & Hand, B. The factorial validity of the Cornell critical thinking tests: A multi-analytic approach. Think. Skills Creat. https://doi.org/10.1016/j.tsc.2020.100676 (2020).

Noroozi, O., Dehghanzadeh, H. & Talaee, E. A systematic review on the impacts of game-based learning on argumentation skills. Entertain. Comput. https://doi.org/10.1016/j.entcom.2020.100369 (2020).

Supena, I., Darmuki, A. & Hariyadi, A. The influence of 4C (constructive, critical, creativity, collaborative) learning model on students’ learning outcomes. Int. J. Instr. 14 (3), 873–892. https://doi.org/10.29333/iji.2021.14351a (2021).

Zhou, Y., Gan, L., Chen, J., Wijaya, T. T. & Li, Y. Development and validation of a higher-order thinking skills assessment scale for pre-service teachers. Think. Skills Creat. https://doi.org/10.1016/j.tsc.2023.101272 (2023).

Musfy, K., Sosa, M. & Ahmad, L. Interior design teaching methodology during the global COVID-19 pandemic. Interiority 3 (2), 163–184. https://doi.org/10.7454/in.v3i2.100 (2020).

Yong, S. D., Kusumarini, Y. & Tedjokoesoemo, P. E. D. Interior design students’ perception for AutoCAD SketchUp and Rhinoceros software usability. IOP Conf. Ser. Earth Environ. Sci. https://doi.org/10.1088/1755-1315/490/1/012015 (2020).

Anthony, B. et al. Blended learning adoption and implementation in higher education: A theoretical and systematic review. Technol. Knowl. Learn. 27 (2), 531–578. https://doi.org/10.1007/s10758-020-09477-z (2020).

Castro, R. Blended learning in higher education: Trends and capabilities. Edu. Inf. Technol. 24 (4), 2523–2546. https://doi.org/10.1007/s10639-019-09886-3 (2019).

Alismaiel, O. Develop a new model to measure the blended learning environments through students’ cognitive presence and critical thinking skills. Int. J. Emerg. Technol. Learn. 17 (12), 150–169. https://doi.org/10.3991/ijet.v17i12.30141 (2022).

Gao, Y. Blended teaching strategies for art design major courses in colleges. Int. J. Emerg. Technol. Learn. https://doi.org/10.3991/ijet.v15i24.19033 (2020).

Banihashem, S. K., Kerman, N. T., Noroozi, O., Moon, J. & Drachsler, H. Feedback sources in essay writing: peer-generated or AI-generated feedback?. Int. J. Edu. Technol. Higher Edu. 21 (1), 23 (2024).

Ji, J. A Design on Blended Learning to Improve College English Students’ Higher-Order Thinking Skills. https://doi.org/10.18282/l-e.v10i4.2553 (2021).

Noroozi, O. The role of students’ epistemic beliefs for their argumentation performance in higher education. Innov. Edu. Teach. Int. 60 (4), 501–512 (2023).

Valero Haro, A., Noroozi, O., Biemans, H. & Mulder, M. First- and second-order scaffolding of argumentation competence and domain-specific knowledge acquisition: A systematic review. Technol. Pedag. Edu. 28 (3), 329–345. https://doi.org/10.1080/1475939x.2019.1612772 (2019).

Narasuman, S. & Wilson, D. M. Investigating teachers’ implementation and strategies on higher order thinking skills in school based assessment instruments. Asian J. Univ. Edu. https://doi.org/10.24191/ajue.v16i1.8991 (2020).

Valero Haro, A., Noroozi, O., Biemans, H. & Mulder, M. Argumentation competence: Students’ argumentation knowledge, behavior and attitude and their relationships with domain-specific knowledge acquisition. J. Constr. Psychol. 35 (1), 123–145 (2022).

Johansson, E. The Assessment of Higher-order Thinking Skills in Online EFL Courses: A Quantitative Content Analysis (2020).

Noroozi, O., Kirschner, P. A., Biemans, H. J. A. & Mulder, M. Promoting argumentation competence: Extending from first- to second-order scaffolding through adaptive fading. Educ. Psychol. Rev. 30 (1), 153–176. https://doi.org/10.1007/s10648-017-9400-z (2017).

Noroozi, O., Weinberger, A., Biemans, H. J. A., Mulder, M. & Chizari, M. Facilitating argumentative knowledge construction through a transactive discussion script in CSCL. Comput. Educ. 61 , 59–76. https://doi.org/10.1016/j.compedu.2012.08.013 (2013).

Noroozi, O., Weinberger, A., Biemans, H. J. A., Mulder, M. & Chizari, M. Argumentation-based computer supported collaborative learning (ABCSCL): A synthesis of 15 years of research. Educ. Res. Rev. 7 (2), 79–106. https://doi.org/10.1016/j.edurev.2011.11.006 (2012).

Setiawan, Baiq Niswatul Khair, Ratnadi Ratnadi, Mansur Hakim, & Istiningsih, S. Developing HOTS-Based Assessment Instrument for Primary Schools (2019).

Suparman, S., Juandi, D., & Tamur, M. Does Problem-Based Learning Enhance Students’ Higher Order Thinking Skills in Mathematics Learning? A Systematic Review and Meta-Analysis 2021 4th International Conference on Big Data and Education (2021).

Goodsett, M. Best practices for teaching and assessing critical thinking in information literacy online learning objects. J. Acad. Lib. https://doi.org/10.1016/j.acalib.2020.102163 (2020).

Putra, I. N. A. J., Budiarta, L. G. R., & Adnyayanti, N. L. P. E. Developing Authentic Assessment Rubric Based on HOTS Learning Activities for EFL Teachers. In Proceedings of the 2nd International Conference on Languages and Arts across Cultures (ICLAAC 2022) (pp. 155–164). https://doi.org/10.2991/978-2-494069-29-9_17 .

Bervell, B., Umar, I. N., Kumar, J. A., Asante Somuah, B. & Arkorful, V. Blended learning acceptance scale (BLAS) in distance higher education: Toward an initial development and validation. SAGE Open https://doi.org/10.1177/21582440211040073 (2021).

Byrne, D. A worked example of Braun and Clarke’s approach to reflexive thematic analysis. Qual. Quant. 56 (3), 1391–1412 (2022).

Xu, W. & Zammit, K. Applying thematic analysis to education: A hybrid approach to interpreting data in practitioner research. Int. J. Qual. Methods 19 , 1609406920918810 (2020).

Braun, V. & Clarke, V. Conceptual and design thinking for thematic analysis. Qual. Psychol. 9 (1), 3 (2022).

Creswell, A., Shanahan, M., & Higgins, I. Selection-inference: Exploiting large language models for interpretable logical reasoning. arXiv:2205.09712 (2022).

Baron, J. Thinking and Deciding 155–156 (Cambridge University Press, 2023).

Book   Google Scholar  

Silver, N., Kaplan, M., LaVaque-Manty, D. & Meizlish, D. Using Reflection and Metacognition to Improve Student Learning: Across the Disciplines, Across the Academy (Taylor & Francis, 2023).

Oksuz, K., Cam, B. C., Kalkan, S. & Akbas, E. Imbalance problems in object detection: A review. IEEE Trans. Pattern Anal. Mach. Intell. 43 (10), 3388–3415 (2020).

Saputra, M. D., Joyoatmojo, S., Wardani, D. K. & Sangka, K. B. Developing critical-thinking skills through the collaboration of jigsaw model with problem-based learning model. Int. J. Instr. 12 (1), 1077–1094 (2019).

Imam, H. & Zaheer, M. K. Shared leadership and project success: The roles of knowledge sharing, cohesion and trust in the team. Int. J. Project Manag. 39 (5), 463–473 (2021).

DeCastellarnau, A. A classification of response scale characteristics that affect data quality: A literature review. Qual. Quant. 52 (4), 1523–1559 (2018).

Article   PubMed   Google Scholar  

Haber, J. Critical Thinking 145–146 (MIT Press, 2020).

Hanscomb, S. Critical Thinking: The Basics 180–181 (Routledge, 2023).

Sulaiman, W. S. W., Rahman, W. R. A. & Dzulkifli, M. A. Examining the construct validity of the adapted California critical thinking dispositions (CCTDI) among university students in Malaysia. Proc. Social Behav. Sci. 7 , 282–288 (2010).

Jaakkola, N. et al. Becoming self-aware—How do self-awareness and transformative learning fit in the sustainability competency discourse?. Front. Educ. https://doi.org/10.3389/feduc.2022.855583 (2022).

Nguyen, T. T. B. Critical thinking: What it means in a Vietnamese tertiary EFL context. English For. Language Int. J. 2 (3), 4–23 (2022).

Henriksen, D., Gretter, S. & Richardson, C. Design thinking and the practicing teacher: Addressing problems of practice in teacher education. Teach. Educ. 31 (2), 209–229 (2020).

Okes, D. Root cause analysis: The core of problem solving and corrective action 179–180 (Quality Press, 2019).

Eroğlu, S. & Bektaş, O. The effect of 5E-based STEM education on academic achievement, scientific creativity, and views on the nature of science. Learn. Individual Differ. 98 , 102181 (2022).

Dzurilla, T. J. & Nezu, A. M. Development and preliminary evaluation of the social problem-solving inventory. Psychol. Assess. J. Consult. Clin. Psychol. 2 (2), 156 (1990).

Tan, O.-S. Problem-based learning innovation: Using problems to power learning in the 21st century. Gale Cengage Learning (2021).

Driskell, J. E., Salas, E. & Driskell, T. Foundations of teamwork and collaboration. Am. Psychol. 73 (4), 334 (2018).

Lower, L. M., Newman, T. J. & Anderson-Butcher, D. Validity and reliability of the teamwork scale for youth. Res. Social Work Pract. 27 (6), 716–725 (2017).

Landa, R. Advertising by design: generating and designing creative ideas across media (Wiley, 2021).

Tang, T., Vezzani, V. & Eriksson, V. Developing critical thinking, collective creativity skills and problem solving through playful design jams. Think. Skills Creat. 37 , 100696 (2020).

Torrance, E. P. Torrance tests of creative thinking. Educational and psychological measurement (1966).

Javadi, M. H., Khoshnami, M. S., Noruzi, S. & Rahmani, R. Health anxiety and social health among health care workers and health volunteers exposed to coronavirus disease in Iran: A structural equation modeling. J. Affect. Disord. Rep. https://doi.org/10.1016/j.jadr.2022.100321 (2022).

Article   PubMed   PubMed Central   Google Scholar  

Hu, L. & Bentler, P. M. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct. Equ. Model. Multidiscip. J. 6 (1), 1–55. https://doi.org/10.1080/10705519909540118 (1999).

Matsunaga, M. Item parceling in structural equation modeling: A primer. Commun. Methods Measures 2 (4), 260–293. https://doi.org/10.1080/19312450802458935 (2008).

Marsh, H. W., Morin, A. J., Parker, P. D. & Kaur, G. Exploratory structural equation modeling: An integration of the best features of exploratory and confirmatory factor analysis. Ann. Rev. Clin. Psychol. 10 (1), 85–110 (2014).

Song, Y., Lee, Y. & Lee, J. Mediating effects of self-directed learning on the relationship between critical thinking and problem-solving in student nurses attending online classes: A cross-sectional descriptive study. Nurse Educ. Today https://doi.org/10.1016/j.nedt.2021.105227 (2022).

Chu, S. K. W., Reynolds, R. B., Tavares, N. J., Notari, M., & Lee, C. W. Y. 21st century skills development through inquiry-based learning from theory to practice . Springer (2021).

Eliyasni, R., Kenedi, A. K. & Sayer, I. M. Blended learning and project based learning: the method to improve students’ higher order thinking skill (HOTS). Jurnal Iqra’: Kajian Ilmu Pendidikan 4 (2), 231–248 (2019).

Yusuf, P. & Istiyono,. Blended learning: Its effect towards higher order thinking skills (HOTS). J. Phys. Conf. Ser. https://doi.org/10.1088/1742-6596/1832/1/012039 (2021).

Byron, K., Keem, S., Darden, T., Shalley, C. E. & Zhou, J. Building blocks of idea generation and implementation in teams: A meta-analysis of team design and team creativity and innovation. Personn. Psychol. 76 (1), 249–278 (2023).

Walid, A., Sajidan, S., Ramli, M. & Kusumah, R. G. T. Construction of the assessment concept to measure students’ high order thinking skills. J. Edu. Gift. Young Sci. 7 (2), 237–251 (2019).

Alawad, A. Evaluating online learning practice in the interior design studio. Int. J. Art Des. Edu. 40 (3), 526–542. https://doi.org/10.1111/jade.12365 (2021).

Awuor, N. O., Weng, C. & Militar, R. Teamwork competency and satisfaction in online group project-based engineering course: The cross-level moderating effect of collective efficacy and flipped instruction. Comput. Educ. 176 , 104357 (2022).

Noroozi, O., Alqassab, M., Taghizadeh Kerman, N., Banihashem, S. K. & Panadero, E. Does perception mean learning? Insights from an online peer feedback setting. Assess. Eval. Higher Edu. https://doi.org/10.1080/02602938.2024.2345669 (2024).

Creswell, J. W. A concise introduction to mixed methods research. SAGE publications124–125 (2021) .

Tashakkori, A., Johnson, R. B., & Teddlie, C. Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. Sage Publications 180–181(2020).

Jiang, X., Lyons, M. D. & Huebner, E. S. An examination of the reciprocal relations between life satisfaction and social problem solving in early adolescents. J. Adolescence 53 (1), 141–151. https://doi.org/10.1016/j.adolescence.2016.09.004 (2016).

Orcan, F. Exploratory and confirmatory factor analysis: Which one to use first. Egitimde ve Psikolojide Olçme ve Degerlendirme Dergisi https://doi.org/10.21031/epod.394323 (2018).

Asparouhov, T. & Muthén, B. Exploratory structural equation modeling. Struct. Eq. Model. Multidiscip. J. 16 (3), 397–438 (2009).

Article   MathSciNet   Google Scholar  

Finch, H., French, B. F., & Immekus, J. C. Applied psychometrics using spss and amos. IAP (2016).

Marsh, H. W., Guo, J., Dicke, T., Parker, P. D. & Craven, R. G. Confirmatory factor analysis (CFA), exploratory structural equation modeling (ESEM), and Set-ESEM: Optimal balance between goodness of fit and parsimony. Multivar. Behav. Res. 55 (1), 102–119. https://doi.org/10.1080/00273171.2019.1602503 (2020).

Download references

Acknowledgements

Thanks to the editorial team and reviewers of Scientific Reports for their valuable comments.

Author information

Authors and affiliations.

Faculty of Education, SEGI University, 47810 Petaling Jaya, Selangor, Malaysia

Department of Art and Design, Zhengzhou College of Finance and Economics, Zhengzhou, 450000, Henan, China

Xiaolei Fan

Faculty of Humanities and Arts, Macau University of Science and Technology, Avenida Wai Long, 999078, Taipa, Macao, Special Administrative Region of China

Lingchao Meng

You can also search for this author in PubMed   Google Scholar

Contributions

D.L. Conceptualized a text experiment, and wrote the main manuscript text. D.L. and X.F. conducted experiments, D.L., X.F. and L.M. analyzed the results. L.M. contributed to the conceptualization, methodology and editing, and critically reviewed the manuscript. All authors have reviewed the manuscript.

Corresponding author

Correspondence to Lingchao Meng .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Li, D., Fan, X. & Meng, L. Development and validation of a higher-order thinking skills (HOTS) scale for major students in the interior design discipline for blended learning. Sci Rep 14 , 20287 (2024). https://doi.org/10.1038/s41598-024-70908-3

Download citation

Received : 28 February 2024

Accepted : 22 August 2024

Published : 31 August 2024

DOI : https://doi.org/10.1038/s41598-024-70908-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Assessment scale
• Higher-order thinking skills
• Interior design
• Blended learning

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: Anthropocene newsletter — what matters in anthropocene research, free to your inbox weekly.

• Open access
• Published: 29 August 2024

Facilitators and barriers to initiating and completing tuberculosis preventive treatment among children and adolescents living with HIV in Uganda: a qualitative study of adolescents, caretakers and health workers

• Pauline Mary Amuge 1 ,
• Denis Ndekezi 2 ,
• Moses Mugerwa 1 ,
• Dickson Bbuye 1 ,
• Diana Antonia Rutebarika 3 ,
• Lubega Kizza 4 ,
• Christine Namugwanya 1 ,
• Angella Baita 1 ,
• Peter James Elyanu 1 ,
• Patricia Nahirya Ntege 1 ,
• Dithan Kiragga 1 ,
• Carol Birungi 4 ,
• Adeodata Rukyalekere Kekitiinwa 1 ,
• Agnes Kiragga 5 ,
• Moorine Peninah Sekadde 6 ,
• Nicole-Austin Salazar 7 ,
• Anna Maria Mandalakas 8 &
• Philippa Musoke 9  

AIDS Research and Therapy volume  21 , Article number:  59 ( 2024 ) Cite this article

Metrics details

Introduction

People living with HIV (PLHIV) have a 20-fold risk of tuberculosis (TB) disease compared to HIV-negative people. In 2021, the uptake of TB preventive treatment among the children and adolescents living with HIV at the Baylor-Uganda HIV clinic was 45%, which was below the national target of 90%. Minimal evidence documents the enablers and barriers to TB preventive treatment (TPT) initiation and completion among children and adolescents living with HIV(CALHIV). We explored the facilitators and barriers to TPT initiation and completion among CALHIV among adolescents aged 10-19years and caretakers of children below 18years.

We conducted a qualitative study from February 2022 to March 2023, at three paediatric and adolescent HIV treatment centers in Uganda. In-depth interviews were conducted at TPT initiation and after completion for purposively selected health workers, adolescents aged 10–19 years living with HIV, and caretakers of children aged below 18years.

The desire to avoid TB disease, previous TB treatment, encouragement from family members, and ministry of health policies emerged as key facilitators for the children and adolescents to initiate TPT. Barriers to TPT initiation included; TB and HIV-related stigma, busy carer and adolescent work schedules, reduced social support from parents and family, history of drug side effects, high pill burden and fatigue, and perception of not being ill. TPT completion was enabled by combined TPT and ART refill visits, delivery of ART and TPT within the community, and continuous education and counseling from health workers. Reported barriers to TPT completion included TB and HIV-related stigma, long waiting time. Non-disclosure of HIV status by caretakers to CALHIV and fear of side effects was cited by health workers as a barrier to starting TPT. Facilitators of TPT initiation and completion reported by healthcare workers included patient and caretaker health education, counselling about benefits of TPT and risk of TB disease, having same appointment for TPT and ART refill to reduce patient waiting time, adolescent-friendly services, and appointment reminder phone calls.

The facilitators and barriers of TPT initiation and completion among CALHIV span from individual, to health system and structural factors. Health education about benefits of TPT and risk of TB, social support, adolescent-friendly services, and joint appointments for TPT and ART refill are major facilitators of TPT initiation and completion among CALHIV in Uganda.

Globally, 10.6 million people fell ill with tuberculosis (TB) in 2022, of which 12% were children below 15 years of age, and 23% reported in Africa [ 1 ]. People living with HIV (PLHIV) accounted for a disproportionate 6.7% of the TB cases and TB-HIV co-infection rates greater than 50% persist in numerous countries [ 1 ]. Out of the 1.6 million TB related deaths that occurred in 2021, 187,000 were among PLHIV, with 11% among children living with HIV [ 1 ].

Following TB exposure, PLHIV have a 20-fold increased life-time risk of progressing to TB disease, and up to 15% annual risk of TB disease, compared to the general population [ 2 ]. There is evidence that TB preventive treatment (TPT) in combination with anti-retroviral therapy (ART), reduces the risk of TB disease by up to 90% [ 3 , 4 ]. During the period 2018–2021, 10.6 million PLHIV received TPT globally, which was more than the targeted 6 million PLHIV. Nevertheless, there is minimal global data reporting TPT completion rates.

Uganda is one of the 30 countries categorized as high TB and TB/HIV burden by the World Health Organization (WHO) [ 1 ], with 74,799 TB patients reported in 2022, of which 32% were HIV-co-infected, and 12% were children below 15years of age [ 1 ]. Following three nation-wide TPT uptake campaigns led by the Ugandan ministry of health, 88.8% of the eligible PLHIV received TPT [ 5 ]. In Ugandan public health facilities, only 17% PLHIV initiated TPT out of the 93% who were eligible for TPT, with only 58% completing the full TPT course [ 6 ]. Some of the documented challenges contributing to such gaps in TPT uptake among PLHIV include; hesitancy of health workers to prescribe TPT for fear of promoting drug resistance, interrupted TPT supply, patients’ fear of additional pill burden and side-effects [ 6 ]. Non-completion of TPT was also associated with ART non-adherence, ART regime switch, and patient representation among adult PLHIV in rural Uganda [ 7 ]. Effective implementation of TPT, through addressing identified barriers and enhancing the facilitators of TPT [ 8 ], is key in reducing the burden of TB disease among PLHIV and bridging the TPT uptake and completion gaps [ 9 , 10 , 11 ]. However, there is limited data on TPT completion especially among PLHIV who are concurrently on ART. Therefore, it is important to understand the multi-faceted barriers and facilitators of initiating and completing TPT among the PLHIV. These may be related to the different healthcare system components such as; the clients or community, health policies, leadership and governance, drugs and logistics management, clinical information systems, service delivery, health workforce and financing [ 12 ]. Individual factors reported to facilitate TPT uptake and delivery among PLHIV in Tanzania include; alignment of ART and TPT visits, and TPT-related education and counseling. In South Africa, individual facilitators of TPT completion among PLHIV included; knowledge about TB and TPT, acceptance of one’s HIV status, having social support in the community and at the health facility, and desire for health preservation [ 13 ]. Individual barriers to TPT uptake and delivery included; perceived or previous experience of side effects, HIV stigma, pill burden, negative cultural and religious values, misunderstanding of TPT’s preventive role, financial burden of transport to the clinic and lost wages, and ineffective communication with the health workers [ 13 , 14 , 15 ].

Health care worker facilitators of TPT initiation among PLHIV include; comprehensive and collective planning, and supervision, presence of guidelines, TB-HIV training, positive attitude and being knowledgeable about TPT, known benefit of TPT, and effective health worker communication [ 8 , 13 , 16 ]. Health care worker and health system barriers to TPT delivery and uptake include; fear for isoniazid resistance due to interrupted drug supply, poor knowledge and attitude, misunderstanding about timing of TPT initiation, shortage of skilled health workers, variable TB screening practices and responsibilities, drug shortage [ 10 ], and contradicting guidelines from TB programs and HIV care programs [ 14 , 17 , 18 , 19 ]. In South Africa, lack of fidelity to national TPT guidelines was a barrier among health workers to initiation of TPT for PLHIV [ 20 ]. Absence of parental risk perception was reported as a barrier to TPT uptake among children in TB endemic areas [ 21 ]. Most of the documented facilitators and barriers to TPT initiation and completion are among adults, with limited reports for children, adolescents and their care takers.

Therefore, we conducted a qualitative study to explore the perceived and experienced barriers and facilitators to TPT initiation and completion among Ugandan children and adolescents living with HIV (CALHIV).

Theoretical orientation

A growing body of literature illustrates that health outcomes are progressively influenced by the environments within which individuals thrive and less by individual behaviors [ 22 ]. We therefore adopted the social ecological model (SEM) as a theoretical framework for analysis (see Fig.  1 below). The social-ecological model (SEM) of health promotion by McLeroy and colleagues states that health behaviour and promotion are interrelated and occur around multiple levels in the individual, interpersonal, institutional, community, and policy levels [ 23 ] This multifaceted perspective is important to understand and explicate barriers and facilitators of TPT initiation and completion among children and adolescents living with HIV, caregivers, and health care workers. The first level refers to individual factors that facilitate or inhibit a person’s choices, including personal stigma, limited knowledge about the prevention treatment, financial constraints and drug characteristics. The second level is interpersonal or network influences. An individual’s relationship with their closest caretakers, and family members influences their uptake and completion of preventative treatments. The third level is community perspectives, as children, caregivers and health care workers are influenced by community-held mass awareness campaigns community drug delivery services and community misconception about prevention treatments. The fourth level refers to health system (institutional) influences, including busy, unapproachable health care workers, limited access to the right treatment and the long waits. The final level refers to structural influences including the accessibility of the information and services related to TB.

Illustration of the SEM framework showing the interrelations at various levels

Study design and data collection methods

This qualitative study was part of a prospective cohort study conducted from February 2022 to March 2023; where CALHIV and their care takers were offered to choose either facility-based or community-based initiation and delivery of TPT. This was part of the differentiated TPT delivery among CALHIV in Uganda (COMBAT TB study).

Study setting

The study was conducted at three high-volume paediatric and adolescent HIV treatment clinics; Baylor College of Medicine Children’s Foundation-Uganda (Baylor-Uganda) center of excellence (COE) HIV clinic located in Mulago Hospital Kampala, Joint Clinical Research Center (JCRC) located in Lubowa, and the Makerere Joint AIDS Program (MJAP) ISS Clinic located on Mulago Hill in Kampala. The Baylor-Uganda clinic located about 4 km from the Kampala city center, provides comprehensive HIV care services for more than 4000 CALHIV out of more than 8000 PLHIV in care at the clinic. The JCRC Lubowa HIV clinic located in Wakiso district, 11 km from Kampala, and it provides comprehensive HIV care services to 1300 CALHIV out of 15,000 PLHIV in care. The MJAP ISS clinic located on Mulago Hill in Kampala, provides comprehensive HIV care services to 612 adolescents out of over 17,000 PLHIV in care. The three clinics run from Monday to Friday as one-stop-centers for care and research on HIV, TB and other HIV-related conditions. The HIV and TB care is provided by multi-disciplinary teams which include counselors, community health workers, peer educators, nurses, pharmacy staff, doctors and laboratory staff. The clients receive HIV prevention services, ART, TB preventive treatment and TB treatment. There is also screening and treatment of other opportunistic infections and non-communicable conditions like mental health issues, hypertension, and diabetes. The services are provided at the health facilities or within the community, based on the national HIV and TB treatment and prevention guidelines.

The CALHIV were screened for TB using the WHO-recommended TB symptom screening tool at every clinic visit. Individuals with TB symptoms completed a clinical evaluation, and TB diagnostic tests, such as Xpert MTB/RIF ultra, urine TB lipoarabinomannan (TB-LAM) for those with CD4 count < 200cells/ul, and chest X-ray. Patients diagnosed with TB then start TB treatment.

Individuals who were assessed as not having TB were considered eligible for TPT, such as; PLHIV above one year of age with no evidence of TB disease, PLHIV who are close contacts of TB patients, and PLHIV who have recently completed a full course of TB treatment. The ministry of health supplied the study sites with TPT drugs; initially isoniazid taken daily for six (6) months, and later rolled-out once weekly isoniazid and rifapentine for three months. The TPT is dispensed with pyridoxine, to prevent peripheral neuropathy, a common side-effect of isoniazid. Individuals who developed mild or moderate side effects, were usually advised to continue with the TPT while the side-effects were managed. If any individuals developed severe side effects, the TPT was withheld to first manage the side effects.

Individuals who initiated TPT within the differentiated delivery approach, had follow-up done via phone calls at two weeks and four weeks after TPT initiation. Follow-up was done at 3months after TPT initiation, and thereafter every three-months at the clinic or within the community to identify and manage side-effects, screen for TB symptoms, and assess adherence to the TPT and ART.

TB screening and diagnostic tests were done for participants with TB symptoms after starting TPT. Participants diagnosed with TB disease before completion of their full TPT course had their TPT stopped and TB treatment started. Adolescents living with HIV were eligible for the study if they were aged 10–19 years, initiating TPT, and completed or did not complete the full dose of TPT. Care takers were eligible for the study if their children aged < 18years living with HIV were initiating TPT, completed or did not complete the full dose of TPT and were willing to provide written informed consent. Health care workers were eligible if they were actively involved in providing TPT and willing to provide written informed consent.

Purposive sampling was done to select eligible health workers, adolescents aged 10-19years and parents or care takers of children who were eligible to start TPT.

During selection of adolescents and care takers, selection was done to try and achieve representation from; the three clinics, with almost equal numbers of; males and females, and age categories (10-14years, 15-19years), TPT completion status (completed, did not complete, missed doses or lost to follow-up), facility-based or community-based delivery models, and ART status (initiating ART or ART-experienced).

The health care workers in this study were involved in screening the children and adolescents for TB, assessing TPT eligibility, prescribing TPT, monitoring individuals on TPT, and providing TB-HIV counseling and guidance according to the national TB and leprosy control guidelines (24). Among the health workers, efforts were made to select equal numbers of males and females, and fair representation by different cadres (nurses, clinical officers, doctors, pharmacists).

Data collection procedure

A semi-structured interview guide was used for each category to obtain in-depth descriptions and valuable insights about the barriers and facilitators to TPT initiation and completion from the three categories of participants.

During the TPT initiation visits, qualitative in-depth interviews (IDIs) were conducted face to-face by an experienced male social scientist (DN), using the piloted interview guide for the data collection process. Interviews lasted between 30 and 45 min. Field notes were also made after each data collection session. Participants were recruited through purposive sampling with the help of the study nurse (CN) at three HIV clinics between June 2022 and August 2023. The IDIs were carried out with the CALHIV, Caretakers/parents and health workers. All the IDIs were held in a conducive place that was safe, neutral and with minimal distractions for the participants and the researcher. This place was either suggested by the interviewee or preset by the interviewer at the participating HIV clinics. Data collection was conducted in a language preferred by the participant, either English or Luganda. The interviewer (DN) took time at the outset of the discussions to develop a rapport with participants, acknowledging the sensitivity of the topic and creating a safe space for them to share their thoughts and experiences. Participants were fully informed about the purpose and objectives of the study, and they provided their informed consent to participate, indicating their understanding and agreement with the research goals and procedures. Approximately four months into the TPT study, participants were approached to participate in the second phase of IDIs for TPT completion.

Sample size

During TPT initiation, thirty (30) IDIs were carried out with the caretakers/parents and children ( N  = 30; 10 health workers, 10 CALHIV, and 10 Caretakers/parents). After TPT completion, interviews were conducted with 10 care takers, and 10 CALHIV. Participants were purposively sampled to represent those CALHIV who completed and those who did not complete or defaulted their TPT dose. The interview guide explored both the facilitators and the barriers for the TPT initiation and completion.

Data management and analysis

In-depth interviews were audio recorded, transcribed verbatim, and then translated into English for a hybrid approach of inductive and deductive thematic analysis [ 22 ] by two researchers (DN and PMA) experienced in qualitative methodology. The initial deductive coding was based on the five levels of the Social Ecological Model (SEM) in Fig.  1 above, and inductive coding was used to explore other themes that were not covered by the SEM. Three transcripts were initially selected and read through for familiarization and coded manually by DN. To ensure coding consistency, the developed codes were shared with the study principal investigator PMA to facilitate collaborative thematic analyses throughout [ 23 ]. All transcripts were imported into NVivo 14 and coded using the refined codebook by DN and PMA. The transcripts were not returned to the participants. The data was organized into pre-defined key themes outlined by the levels of the SEM. A framework approach using SEM was used for data analysis [ 25 ]. Themes and sub-themes were continually reviewed and refined to capture emerging new codes. Quotes were captured to highlight thematic areas and increase our understanding of the context. The methods and results were aligned to the consolidated criteria for reporting qualitative research (CORE-Q) [ 26 ].

A total of 50 IDIs were conducted for the selected participants (health workers ( N  = 10), adolescents ( N  = 10), care takers ( n  = 10) until saturation of content was achieved. Table  1 below summarises the demographic characteristics of the study participants.

Facilitators to initiation and completion of TPT among adolescents and children

From the IDIs, we found the following facilitators at individual level. Participants perceiving themselves as being at risk of contracting TB was a key facilitator to initiate and complete TPT. In addition, some care takers highlighted that the TPT will also help the child to have a good life without TB, but if she acquires TB and yet is already HIV positive, the child may be severally affected.

“Apart from the fact that it will help me to prevent TB, it will help me not to get TB and am assured that I will not get TB because TB is very risky, inconvenient and I will protect others because I know I am at a very high risk. So by taking the drugs, at least I know am protecting someone in case I get it, am protecting a family member, a sibling, a sister”. Male Adolescent 15 years.

Further analysis revealed that care takers and participants who were once diagnosed with TB and recovered narrated their agony and the experience of treating TB which they noted that they would not want to experience again. The experience they had with TB disease compelled them to initiate and complete their TPT dose.

“Another reason why I accepted my child to start on TPT is because my child has ever suffered from TB, and given that now we have the drugs for preventing it, I had no reason to resist it. I was afraid the child might acquire it again”. Female carer of 10-year-old adolescent.

The desire to remain free from TB emerged as a facilitator to initiating and completing TPT. The TPT was perceived as a breakthrough strategy to prevent acquisition of TB.

“Since I had an experience of a person with TB that I told you about, I didn’t want to wait until he is affected as it did to the other one I saw. So that forced me to ensure that the dose is completed”. Female caretaker of 14-year-old adolescent.

At the interpersonal level, support, care and encouragement from family, supervision from the caretakers also emerged as important facilitators to initiate and complete TPT. The participants remarked that receiving care and support (reminders) from immediate family encouraged them to complete their treatment.

“Like at home, there is my mother who always reminds me to take my drugs. That helped me to always take my drugs in time”. Female Adolescent, 18 years.

Community level facilitators included guidance and counseling, comprehensive information, mass awareness and sensitization about TPT. Participants mentioned that receiving adequate information and sensitization was helpful for their decision to initiate TPT. Participants reported that they received information from the health workers on how the child should take the medicine and how the treatment works to prevent the disease, something that encouraged most of them to start their children on treatment.

“The encouragement I got from doctors helped me to give treatment to my child for TB treatment which also made it easy for me to start him on TPT. I believe by the time the dose is completed the child will be okay. Doctors also sensitized us about the possible side effects of the drugs and they follow up with phone calls”.  Female care taker for a 7-year old child.

It emerged that information about the TPT made available by the health workers, with opportunities to discuss the treatment with the doctors, and making it known in the community, enabled the care givers to allow TPT to be given to their children and adolescents.

“When people are aware, it makes the services easy to access. Many people talk about other things on TVs and radios but they don’t take about TB. We have to tell people TB is real and a killer disease. You can also inform them in case someone sees the symptoms they should be screened for TB”. Medical doctor 01.

At the institutional and organizational level, participants preferred to have convenient services as a facilitator for the initiation and completion of the treatment. This was in terms of having TPT appointments scheduled on the same days of ART refill so that they can have all the drugs on the same appointment as this will reduce the time spent at the clinic and cost of repeat visits.

“The other issue is integrating those TPT refills with their usual clinic visits and community services so that they can readily receive the drugs at times without even wasting much time and transport to come to the clinic”. Medical doctor 02.

Among the healthcare providers, it emerged that many young people preferred to have the drugs taken to them so that they don’t have any excuses of not coming to the clinic for treatment.

“Also initiating TPT delivery models that reduce the transport costs and avoid missing clinical appointments and doses. Also to make sure their drugs are delivered before they are out of stock”. Nursing officer 01.

Besides the convenient services, health workers recognized mechanisms of following up the patients initiated on TPT or reminding them when to take their treatment as facilitator for the completion of TPT.

“We need to make mechanisms of follow ups when you put someone on TPT, you have to check on them to see how they are doing sometimes when you tell them to take the drug on Sunday it means they will even shift the ARVs to the same date”. Epidemiologist 01.

Health workers also cited frequent and friendly communication with children and caretakers in terms of the health talks at the clinic, calling the patients through the mobile phones and receive their feedback.

“Another thing is when you relate with children they bring out their challenges where you share and help them out. Smoothly they can cooperate and complete the six months’ TB preventive treatment". Study counsellor 01. “With the care takers, it is just a matter of explaining to them. It will not be hard for them if they have understood the importance of TPT and even the challenges will be less. The information should be explained in a way which is understood.” TB community linkage facilitator 01.

At the structural level, what emerged was having national policies and good performance indicators at the health facilities that are developed to create demand for the TPT among CALHIV has a great advantage and facilitates TPT uptake.

“Demand creation, tasking health workers. We have our weekly performance review and TPT is among the many indicators we track. Ministry of health asks us how many people are on TPT which helps the health worker to improve on performance and this will facilitate the uptake of TPT”. Medical officer 01.

Regular auditing and identifying the challenges and weaknesses at the facilitate level in relation to the prescription of the treatment emerged as a key facilitator for the uptake of TPT among CALHIV.

“We have reached that level where we appreciate if you find your health workers are not performing well, sit down as a unit and ask yourself on the weaknesses. If you planned to start 56 participants on TPT this week what happened, open the file and do file audits. You will discover interesting things other than patients missed to come or ask the pharmacist why were you not prescribing the drugs when there was even an alert”. Epidemiologist 02.

The following themes emerged as barriers to TPT initiation and completion at patient-level, structural, community and interpersonal levels.

We found the following individual-level barriers to TPT initiation and completion. One of the emerging barriers to initiate or complete their TPT was the stigma associated with taking TB or HIV drugs. The fear of being seen taking many pills on a daily basis was cited as affecting their emotional well-being and mental health.

“Stigma will always be there and I think it’s a reason why so many kids out there fear. Personally before, I didn’t have any problem taking my medicine. So when the stigma started I stopped taking medicine, I stopped caring, it really caused me a lot of mental damage and trauma”. Male Adolescent 18 years.

Where there is limited privacy, taking the treatment would be difficult. Participants also mentioned that they would fail to come for their HIV clinic appointments, for fear of being identified as HIV patients or TB patients.

“…the main challenge is the stigma of HIV which is a leading factor in the community. Some of them fail to come for their appointments because of stigma. They don’t want to be identified as HIV or TB-positive”. Medical officer 03.

The fear of drug-related side effects was reported as a key barrier to starting TPT. Participants expressed their fear of taking TPT treatment for fear of side effects based on their past experiences with different drugs. At TPT completion, experience of side-effects like dizziness and nausea emerged as barriers to TPT completion.

“It would make me feel nausea or feel like vomiting, headache and dizziness. Me I decided not to take them anymore… I even didn’t tell anyone”. Male adolescent, 12 years old.

High pill burden coupled with poor drug adherence also emerged as key barriers reported by the participants, especially if the child was also on ART regimens.

“Another issue is about the pill burden because these are people who are already on ARVs and then they are added more pills for TB so it becomes a lot for them”. Nursing officer 3. “The biggest barrier is adherence because it’s still a challenge to even those that are HIV negative. There are clients who are not used to taking treatment and if the treatment is for six months there will be a challenge of commitment to take the drugs every day.” Medical officer 03.

Among the caretakers, it emerged that pill fatigue created by taking tablets when a person is not sick with TB, caused many adolescents to miss their doses and some did not complete, even though they reported taking the drugs when it is not true.

“Some children fear taking drugs and time comes when the child is tired and no longer wants to take the medicine. … the child can pretend to be taking the medicine when it is not true because the child got tired of taking the drugs”. Female Caretaker of 8-year-old child. “That the medicine was a lot, and the child got tired of it, so she didn’t complete. “Sometimes she could say, “it is just for prevention, I will not take it”. The fact that the child didn’t have TB, she could not care at all”. Female caretaker of 15 years adolescent.

Caretakers expressed the discomfort of children taking pills with a bad smell, big size, unpleasant color and poorly packaged. Participants said that a pill with no smell, small size and attractive packaging would be easier to swallow.

“One, the smell of the medication might not be really good to the child, the pill size can be too big, you even see and say ooh! Female caretaker to 13-year-old adolescent.

It emerged that some adolescents and their caretakers are “ engaged in demanding jobs that may not allow time to collect their medication or they may forget to take it ”. Community Health linkage officer 01.

Forgetting to take the additional drugs also emerged as hinderance to complete the TPT.

“…when you work a lot and do not get time, because you are not used to it like ARVs, the busy schedule can also cause you from not taking the drugs. Male adolescent-18 years. “She is so forgetful. You always have to ask her whether she has taken the medicine. If you are not around, I just know she has not taken and that’s why she didn’t complete”. Female caretaker to a 16year-old adolescent.

At the interpersonal level, the change of caretakers and lack of support mainly from parents also emerged as key barriers to the completion of TPT.

“Some of them like children depend on their caretakers and sometimes we experience changes of the caretakers”. Nursing officer 04.

Among female caregivers, denial or restrictions by the husbands to come to the clinic for refills, also emerged as a barrier for TPT completion among their children

“For those that are married, their husbands don’t allow them to come to the clinic since it was not on the program”. Female caretaker 14 years child.

Financial constraints and lack of food contributed to delay in TPT initiation and failure to complete the treatment. Caretakers expressed concerns that certain medications require a specific diet to be effective, but they struggled to provide the necessary nutritional support, particularly for their school-aged children, which in turn impacted their ability to adhere to treatment regimens, as highlighted by one adolescent’s experience

“Ok the major challenge I faced at school is sometimes I don’t take medicine because I have not eaten. I know the medicine is very strong and I know it will affect my stomach. It will affect me so if am to take it on an empty stomach it wouldn’t be possible. So sometimes I just don’t take it because I know it will cause me effects”. Female Adolescent 18 years.

Failure of the caretakers to disclose HIV status to the children was cited as a barrier of children to initiate and take TPT treatment. One health worker noted that most mothers at home have never disclosed the reason why their children take these drugs daily, and when the husband is around they cannot take their drugs.

“There is also no disclosure especially to the children. So you find when the child doesn’t take the drugs because they do not understand why they are taking the drugs”. Medical doctor 04.

This has also been a challenge to trace TB contacts in families where the patient has never disclosed to the family members and as a result, children in these families miss the opportunity to take the TPT treatment.

“Disclosure is the problem when families have not yet disclosed, and someone comes down with TB. It is difficult to conduct contact tracing, for example on what ground are you asking the family about TB. So it is hard”. Epidemiologist 02.

At the community level, misconception about TPT and Community stigma associated to TB were some of the barriers identified. Further analysis revealed that some adolescents are so inquisitive about drugs and the intended benefit of taking the drugs. However, many are confused with the different sources of information about the benefits of the drugs. In addition, they did not understand how it could work to prevent infection. For example, there was a misconception about the dangers of taking medication when you are well. Some perceived that the government would introduce these treatments as a gateway to reducing their life span.

“Adolescents are very inquisitive. They keep questioning depending on the different sources of information they receive. So some of the questions are like, “don’t you think these are the drugs that stimulate our TB?” Most of them have those questions and I don’t know whether it’s propaganda now they keep saying “the government or the health facilities are trying to make us fall sick quickly and we even google some of these drugs kill the cells that could have protected our bodies”. This affects their TPT drug adherence”. Medical officer 02.

Participants also reported that there was stigma related to TB disease at health facilities and in the communities where patients reside. The situation worsens especially for adolescents in schools where students fail to take their medication until their next appointment because of the stigma from their fellow peers.

“Students may stigmatize you, which at times makes you not to take the drugs or hide it from them that you are not taking the drugs”. Female adolescent 18 years. “Yes, because they disturb you, they say that one is a TB patient, and they talk a lot. This caused me to miss the refill days”. Female adolescent 14 years.

At the institution level, the long waiting-time at the clinic emerged as a barrier to completing TPT. Participants revealed that they preferred quick access to services without having to spend long hours in queues waiting to receive the treatment.

“It’s just embarrassing, it’s just too much. The long waiting really makes me feel like opting out. That’s the truth I can tell you”. Female care takers to a 13-year-old adolescent. “I come early and leave late. That issue made it hard for me. Sometimes I tell her to go by herself but then I remember that she will not give in her complaints. Sometimes we missed coming”. Female caretaker to a 12-year-old adolescent.

Participants were concerned about the attitude of health workers when they are seeking services. This was viewed as a major barrier because they thought if the health workers are rude to the clients, they might not find it conducive to collect their treatment. This was echoed by some health workers who shared the experience that when patients are mistreated, they fail to come back until they are followed up.

“You may find when the person has failed to come on a clinic visit because he was mistreated by a nurse and has not been listened to. Then the person concludes by saying I will not come back”. When it comes to the next appointment, they don’t come back”. Medical officer 05.

Health care workers forgetting to prescribe the drugs at refill visits emerged as one of the barriers to TPT completion.

“Also to the prescribers, someone might have taken TPT like for three months and when they report back, the prescriber forgets to give the refill to add up the six months. So, a patient ends up missing the three months and restart the treatment again”. Medical officer 01.

Health care workers also commented that health facilities may lack essential medicines, and clients are advised to buy from private pharmacies which hinders completion.

At the structural level, participants reported that if the clinic was not within easy reach, they found it a problem to pick their drug refills. This required them to travel long distances with costly transport.

“Transport also affects us, there is a time when you have to come and get treatment but when you don’t have money and that’s why some people fail to come”. Female care giver to 12-year-old adolescent.

This qualitative study explored the perceived, and experienced facilitators, and barriers to TPT initiation and completion among children and adolescents living with HIV, as reported by the Ugandan health workers, adolescents, and care takers of children.

Parental support and supervision, perceived risk of TB disease, and previous experiences of TB treatment were reported by adolescents and care takers of children as the major facilitators of TPT initiation and completion. Similar to a Kenyan study by S. Ngugi et al. [ 15 ], this study found that provision of adequate information about TPT benefits and dosing by health workers, family and community support, and experience of treating children with TB were highlighted by care takers as facilitators that enabled their children to initiate and complete TPT. Social support is very key in determining TPT initiation and completion among CALHIV, calling for integration of psychosocial support in TPT programs.

Facilitators of TPT initiation and completion highlight the need to provide adolescent friendly services and integrated TB and HIV services to facilitate initiation and completion of TPT among adolescents living with HIV [ 8 ]. Adolescent friendly services should be accessible, acceptable, appropriate and delivered in safe and respectful environment by supportive healthcare providers (27, 28). These include promotive, preventive, curative, and referral health services (28).

The barriers to TPT initiation and completion reported by adolescents included; TB or HIV-related stigma, busy work schedules of the adolescents and care takers, reduced social support from parents and family, previous experience of side effects from other drugs, pill burden and fatigue when that are not sick, financial constraints to travel to the clinic, and lack of food to take with the medicines. The roll-out of shorter TPT regimens is very timely [ 9 ], and will most likely address concerns of pill burden and fatigue among CALHIV who are already receiving daily ART.

Although care takers identified barriers to TPT initiation and completion that were similar to those reported by the adolescents, care takers additionally reported barriers such as; pill size, burden and odour, misconception and misinformation about the benefits and duration of the TPT, long distances to the health facilities, and rude health workers. It is important to provide regular adherence support from TPT initiation to facilitate completion, and therefore the efficacious benefits of TPT.

In contrast to the study by Teklay G et al. [ 18 ], health workers did not report fear of creating isoniazid resistance as a barrier to TPT initiation among CALHIV. Barriers cited by health workers included; TB and HIV-related stigma, undisclosed HIV status to the CALHIV, misconceptions that TPT puts their life at risk, fear of side effects, missed opportunities due to forgetting by health workers, poor attitude of health workers towards the adolescents, long waiting hours, change of care takers, and lack of parental or social support. These are closely related to the contextual barriers reported by Nyarubamba R. F et al. in Tanzania [ 14 ], and Lai J et al. in Ethiopia [ 16 ]. Drug stock outs in some facilities were reported as barriers, similar to a study among health workers in Ethiopia [ 18 ].

Limitations

The purposively selected sample is not widely representative of the CALHIV and their care takers in high TB burden countries. Therefore, transferability of these results in other settings may vary based on; the social-ecological models used to assess patient perceptions, TB disease burden, patient/family education and support initiatives within the healthcare system. There were limited numbers of participants who did not complete TPT, limiting the depth of lived experiences about barriers to TPT completion among CALHIV. This study did not explore the perspectives of policy makers in TB care, as these are also important to guide concerted efforts to improve TPT uptake and completion among CALHIV. There was no quantitative data for triangulation with the qualitative results.

The in-depth interviews were conducted at TPT initiation and after TPT completion. This minimised recall bias. This enabled deeper understanding of both perceived and experienced facilitators and barriers to TPT initiation and completion among CALHIV.

The facilitators and barriers of TPT initiation and completion among CALHIV are diverse, spanning from individual factors to healthcare system and structural factors. Educating patients about the benefits of TPT and the need to reduce the risk of TB, facilitates TPT initiation and completion among CALHIV. Availability of social support, adolescent-friendly services, and integration of TPT refills into ART refill visits are also major facilitators of TPT initiation and completion among CALHIV.

TB and HIV-related stigma, high pill burden of TPT in addition to ART, non-disclosure of HIV status of the children and adolescents, lack of parental support, transport difficulties, and misconceptions about TPT side effects, were the major barriers to initiation and completion among these CALHIV. Therefore, it is important to implement patient-centered TB and TPT services for CALHIV and their caretakers, so as to improve TPT initiation and completion, ultimately, reducing TB burden in this high-risk population.

Recommendations

Provision of clear information about TPT and TB, psychosocial and adherence support, adolescent-friendly TB-HIV services, and integration of TPT delivery into ART delivery models, are promising strategies to improve the uptake and completion of TPT among children and adolescents living with HIV in high TB-HIV burden settings like Uganda. TPT completion is likely where services are offered within a family-centered approaches to enhance psychosocial support for adherence. We recommend integrating TPT delivery into existing ART delivery approaches, at health facility and community level, to enhance uptake and completion of TPT among CALHIV.

Data availability

The data that support the findings of this study are available on request from the corresponding author Dr Pauline Mary Amuge (PMA) [email protected], and the institutional representative [email protected] This is to ensure that the data is shared within the provisions of the protocol approved by the Makerere University School of Medicine research and ethics committee, as it was aimed to accomplish specified study objectives.

Abbreviations

Assisted Partner Notification

Anti-retroviral therapy

Anti-retroviral drugs

Children and Adolescents Living with HIV

Severe Acute Respiratory Syndrome due to Corona Virus-19

Differentiated Service Delivery

Differentiated Service Delivery Models

Human Immune-deficiency Virus

3months course of Isoniazid and Rifapentine

3months course of Isoniazid and Rifampicin

Integrated community case management

Isoniazid (isonicotinylhydrazide)

Isoniazid Preventive Therapy

Interrupted time series

Latent Tuberculosis Infection

Ministry of Health

National Drug Authority

National Tuberculosis and Leprosy control Program

Bacteriologically Confirmed Pulmonary Tuberculosis

Clinically Diagnosed Pulmonary Tuberculosis

People Living with HIV

Pulmonary Tuberculosis

• Tuberculosis

Tuberculosis Preventive Treatment

Village Health Team

World Health Organisation

World Health Organisation. Global tuberculosis report 2023. World Health Organisation: Geneva; 2023.

Selwyn PA, et al. A prospective study of the risk of tuberculosis among intravenous drug users with human immunodeficiency virus infection. N Engl J Med. 1989;320(9):545–50.

Article   CAS   PubMed   Google Scholar  

World Health Organization. Latent TB infection: updated and consolidated guidelines for programatic management. Geneva; 2018.

Ayieko J, et al. Efficacy of isoniazid prophylactic therapy in prevention of tuberculosis in children: a meta–analysis. BMC Infect Dis. 2014;14(1):91.

Article   PubMed   PubMed Central   Google Scholar  

Lukoye D et al. Tuberculosis preventive therapy among persons living with HIV, Uganda, 2016–2022. Emerg Infect Dis, 2023. 29(3).

Kalema N, Semeere A, Banturaki G, Kyamugabwa A, Ssozi S, Ggita J, et al. Gaps in TB preventive therapy for persons initiating antiretroviral therapy in Uganda: an explanatory sequential cascade analysis. Int J Tuberc Lung Dis. 2021;25(5):388–94.

Lwevola P, Izudi J, Kimuli D, Komuhangi A, Okoboi S. Low level of tuberculosis preventive therapy incompletion among people living with human immunodeficiency virus in eastern Uganda: a retrospective data review. J Clin Tuberculosis Other Mycobact Dis. 2021;25:100269.

Article   CAS   Google Scholar  

Masini E, Mungai B, Wandwalo E. Tuberculosis preventive therapy uptake barriers: what are the low-lying fruits to surmount this? Public Health Action. 2020;10(1):3.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Vasiliu A, et al. Landscape of TB Infection and Prevention among people living with HIV. Pathogens. 2022;11(12):1552.

Surie D, et al. Policies, practices and barriers to implementing tuberculosis preventive treatment—35 countries, 2017. Int J Tuberc Lung Dis. 2019;23(12):1308–13.

Nyathi S, et al. Isoniazid preventive therapy: uptake, incidence of tuberculosis and survival among people living with HIV in Bulawayo. Zimbabwe PloS One. 2019;14(10):e0223076.

Müller P, Velez L, Lapão. Mixed methods systematic review and metasummary about barriers and facilitators for the implementation of cotrimoxazole and isoniazid—preventive therapies for people living with HIV. PLoS ONE. 2022;17(3):e0251612.

Jacobson KB, et al. It’s about my life: facilitators of and barriers to isoniazid preventive therapy completion among people living with HIV in rural South Africa. AIDS Care. 2017;29(7):936–42.

Nyarubamba RF, et al. Assessment of contextual factors shaping delivery and uptake of isoniazid preventive therapy among people living with HIV in Dar Es Salaam, Tanzania. BMC Infect Dis. 2022;22(1):1–9.

Article   Google Scholar  

Ngugi SK, et al. Factors affecting uptake and completion of isoniazid preventive therapy among HIV-infected children at a national referral hospital, Kenya: a mixed quantitative and qualitative study. BMC Infect Dis. 2020;20:1–11.

Lai J, et al. Provider barriers to the uptake of isoniazid preventive therapy among people living with HIV in Ethiopia. Int J Tuberc Lung Dis. 2019;23(3):371–7.

Roscoe C, et al. Evaluation of the uptake of Tuberculosis preventative therapy for people living with HIV in Namibia: a multiple methods analysis. BMC Public Health. 2020;20:1–12.

Teklay G, et al. Barriers in the implementation of isoniazid preventive therapy for people living with HIV in Northern Ethiopia: a mixed quantitative and qualitative study. BMC Public Health. 2016;16(1):1–9.

Kagujje M, et al. Implementation of isoniazid preventive therapy in people living with HIV in Zambia: challenges and lessons. BMC Public Health. 2019;19:1–4.

Van Ginderdeuren E, et al. Health system barriers to implementation of TB preventive strategies in South African primary care facilities. PLoS ONE. 2019;14(2):e0212035.

Grace SG. Barriers to the implementation of isoniazid preventive therapy for tuberculosis in children in endemic settings: a review. J Paediatr Child Health. 2019;55(3):278–84.

Article   PubMed   Google Scholar  

Busza J, et al. Community-based approaches for prevention of mother to child transmission in resource‐poor settings: a social ecological review. J Int AIDS Soc. 2012;15:17373.

McLeroy KR, et al. An ecological perspective on health promotion programs. Health Educ Q. 1988;15(4):351–77.

Uganda National Tuberculosis and Leprosy Control Programme. Manual for management and control of Tuberculosis and Leprosy in Uganda. 2017;(3rd edition):1–177.

Fereday J, Muir-Cochrane E. Demonstrating rigor using thematic analysis: a hybrid approach of inductive and deductive coding and theme development. Int J Qualitative Methods. 2006;5(1):80–92.

Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57.

World Health Organization. Adolescent friendly health services for adolescents living with HIV: from theory to practice, December 2019: technical brief. World Health Organization; 2019.

World Health Organization. Making health services adolescent friendly: developing national quality standards for adolescent-friendly health services. Geneva, Switzerland. 2012. Report No.: 978 92 4 150359 4.1.

Download references

Acknowledgements

Baylor College of Medicine Children’s Foundation-Uganda: Henry Balwa, Susan Tukamuhebwa, Rachel Namuddu Kikabi, Florence Namuli, Kizito David, Wasswa George, Rogers Nizeyimana, Geofrey Musoba, Alex Tekakwo, Brenda Nakabuye, David Mpagi. Joint Clinical Research Center (JCRC) Lubowa: Flavia Nakato, Joan Nangiya, Henry Mugerwa, Drollah Ssebagala. Makerere Joint AIDS Program (MJAP) Mulago ISS Clinic Kampala Uganda: Douglas Musimbago, Fred Semitala.

This work was supported by the Collaborative Initiative for Paediatric HIV Education and Research (CIPHER) grant programme at the International AIDS Society (IAS), through the CIPHER Research grant awarded to PA for the period 1st November 2021 to 31st October 2023, for a project titled “Differentiated delivery of tuberculosis preventive treatment (TPT) within existing health facility and community HIV care models to improve TPT uptake and completion among children and adolescents living with HIV in Uganda following the COVID-19 pandemic.”

Author information

Authors and affiliations.

Mulago Hospital, Baylor College of Medicine Children’s Foundation-Uganda, Block 5, P.O. Box 72052, Kampala, Uganda

Pauline Mary Amuge, Moses Mugerwa, Dickson Bbuye, Christine Namugwanya, Angella Baita, Peter James Elyanu, Patricia Nahirya Ntege, Dithan Kiragga & Adeodata Rukyalekere Kekitiinwa

Department of Social Aspect of Health, Medical Research Council, Uganda Virus Research Institute and London School of Hygiene and Tropical Medicine, Kampala, Uganda

Denis Ndekezi

Department of Research and Clinical Care, Joint Clinical Research Center, Lubowa, Wakiso, Uganda

Diana Antonia Rutebarika

Medical Department, Makerere Joint AIDS Program-Mulago ISS Clinic, Kampala, Uganda

Lubega Kizza & Carol Birungi

African Population and Health Research Center, Nairobi, Kenya

Agnes Kiragga

National Tuberculosis and Leprosy Program, Ministry of Health, Kampala, Uganda

Moorine Peninah Sekadde

School of Medicine, Johns Hopkins University, Baltimore, USA

Nicole-Austin Salazar

Global TB Program, Baylor College of Medicine, Houston, USA

Anna Maria Mandalakas

Department of Paediatrics and Child Health, Makerere University Johns Hopkins University (MUJHU) Care Limited, Kampala, Uganda

Philippa Musoke

You can also search for this author in PubMed   Google Scholar

Contributions

PMA conceived the original concept. The funding was secured by PMA, PJE, PNN, ARK, AK, AMM, PM. The study was designed by PMA, PJE, MSP, AG, NAS, AMM, PM. Data was curated by PMA, DN, AB, DB, MM, CB, LK and CN. The data was analysed by DN and PMA. The project was co-ordinated by PMA, DN, MM, DB, DAR, and CB. The project technical advisors and mentors were; PJE, AK, ARK, AMM, NAS, MSP, AMM, PM. The original manuscript draft and responses to all author comments were written by PMA and DN. All authors reviewed and edited the original manuscript draft before submission. PMA and DN addressed all comments, and revised the manuscript. All authors reviewed and approved the final manuscript for publication.

Corresponding author

Correspondence to Pauline Mary Amuge .

Ethics declarations

Ethics approval and consent to participate.

Written informed consent was obtained before data collection from participants aged ≥ 18 years, and parents/carers of children under 18years. Written informed assent was obtained from children aged 8years to under 18 years. All data were stored on encrypted computers. Filed notes and signed participant-informed consent forms were kept in a locked drawer at the study site. Participants’ names were not recorded anywhere during data collection. Each participant was given a unique identifying number to ensure confidentiality. The research teams did not include any identifying information that could have harmful consequences for the participants. Ethical approval was granted by the Makerere University school of medicine Research and Ethics Committee (17th June 2020, REF 2020 − 127), and the Uganda National Council for Science and Technology (12th November 2020; HS768ES).

Consent for publication

All authors reviewed and the final manuscript and consented to submission for publication.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Amuge, P.M., Ndekezi, D., Mugerwa, M. et al. Facilitators and barriers to initiating and completing tuberculosis preventive treatment among children and adolescents living with HIV in Uganda: a qualitative study of adolescents, caretakers and health workers. AIDS Res Ther 21 , 59 (2024). https://doi.org/10.1186/s12981-024-00643-2

Download citation

Received : 10 June 2024

Accepted : 05 August 2024

Published : 29 August 2024

DOI : https://doi.org/10.1186/s12981-024-00643-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Adolescents
• Preventive treatment
• Facilitators

AIDS Research and Therapy

ISSN: 1742-6405

• Submission enquiries: [email protected]