the research problem and research question

Research Problem vs. Research Question

What's the difference.

Research problem and research question are two essential components of any research study. The research problem refers to the issue or gap in knowledge that the researcher aims to address through their study. It identifies the area of research that requires further investigation and highlights the significance of the study. On the other hand, the research question is a specific inquiry that the researcher formulates to guide their investigation. It is a concise and focused query that helps to narrow down the research problem and provides a clear direction for the study. While the research problem sets the broader context, the research question provides a specific and measurable objective for the research study.

Further Detail

Introduction.

Research is a systematic process that involves the exploration and investigation of a particular topic or issue. It aims to generate new knowledge, solve problems, or answer specific questions. In any research endeavor, it is crucial to clearly define the research problem and research question. While they are closely related, they have distinct attributes that shape the research process. This article will delve into the characteristics of research problems and research questions, highlighting their similarities and differences.

Research Problem

A research problem is the foundation of any research study. It refers to an area of concern or a gap in knowledge that requires investigation. Identifying a research problem is the initial step in the research process, as it sets the direction and purpose of the study. A research problem should be specific, clear, and well-defined to guide the research process effectively.

One of the key attributes of a research problem is that it should be significant. It should address an issue that has practical or theoretical implications and contributes to the existing body of knowledge. A significant research problem has the potential to make a positive impact on society, industry, or academia.

Furthermore, a research problem should be researchable. This means that it should be feasible to investigate and gather relevant data to address the problem. It should be within the researcher's capabilities and resources to conduct the study. A research problem that is too broad or vague may hinder the research process and lead to inconclusive results.

Additionally, a research problem should be specific and well-defined. It should clearly state the variables or concepts under investigation and provide a clear focus for the study. A well-defined research problem helps in formulating research questions and hypotheses, as it narrows down the scope of the study.

Lastly, a research problem should be original. It should contribute to the existing body of knowledge by addressing a gap or extending previous research. Originality ensures that the research study adds value and novelty to the field, making it relevant and interesting to researchers and practitioners.

Research Question

A research question is a specific inquiry that guides the research process and aims to provide an answer or solution to the research problem. It is derived from the research problem and helps in focusing the study, collecting relevant data, and analyzing the findings. A well-formulated research question is crucial for conducting a successful research study.

Similar to a research problem, a research question should be clear and specific. It should be concise and focused on a particular aspect of the research problem. A clear research question helps in determining the appropriate research design, methodology, and data collection techniques.

Furthermore, a research question should be answerable. It should be feasible to gather data and evidence to address the research question. An answerable research question ensures that the research study is practical and achievable within the given constraints.

A research question should also be relevant. It should directly relate to the research problem and contribute to the existing body of knowledge. A relevant research question ensures that the study has significance and value in the field, making it meaningful to researchers and stakeholders.

Lastly, a research question should be specific to the research context. It should consider the scope, objectives, and limitations of the study. A specific research question helps in avoiding ambiguity and ensures that the research study remains focused and coherent.

While research problems and research questions share some similarities, they also have distinct attributes that differentiate them. Both research problems and research questions should be clear, specific, and relevant to the research study. They should address a gap in knowledge and contribute to the existing body of knowledge.

However, a research problem is broader in scope compared to a research question. It sets the overall direction and purpose of the study, while a research question focuses on a specific aspect or inquiry within the research problem. A research problem provides a broader context for the study, while a research question narrows down the focus and guides the investigation.

Another difference lies in their formulation. A research problem is typically formulated as a statement or a declarative sentence, highlighting the area of concern or gap in knowledge. On the other hand, a research question is formulated as an interrogative sentence, posing a specific inquiry that needs to be answered or explored.

Furthermore, a research problem is often derived from a literature review or an analysis of existing research. It identifies the gap or area of concern based on the current state of knowledge. On the contrary, a research question is derived from the research problem itself. It is formulated to address the specific aspect or inquiry identified in the research problem.

Lastly, a research problem is usually stated at the beginning of a research study, while research questions are developed during the research design phase. The research problem sets the foundation for the study, while research questions are refined and finalized based on the research problem and objectives.

In conclusion, research problems and research questions are essential components of any research study. While they share similarities in terms of being clear, specific, and relevant, they also have distinct attributes that shape the research process. A research problem sets the overall direction and purpose of the study, while research questions focus on specific inquiries within the research problem. Both are crucial in guiding the research process, collecting relevant data, and generating new knowledge. By understanding the attributes of research problems and research questions, researchers can effectively design and conduct their studies, contributing to the advancement of knowledge in their respective fields.

Comparisons may contain inaccurate information about people, places, or facts. Please report any issues.

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• The Research Problem/Question
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A research problem is a definite or clear expression [statement] about an area of concern, a condition to be improved upon, a difficulty to be eliminated, or a troubling question that exists in scholarly literature, in theory, or within existing practice that points to a need for meaningful understanding and deliberate investigation. A research problem does not state how to do something, offer a vague or broad proposition, or present a value question. In the social and behavioral sciences, studies are most often framed around examining a problem that needs to be understood and resolved in order to improve society and the human condition.

Bryman, Alan. “The Research Question in Social Research: What is its Role?” International Journal of Social Research Methodology 10 (2007): 5-20; Guba, Egon G., and Yvonna S. Lincoln. “Competing Paradigms in Qualitative Research.” In Handbook of Qualitative Research . Norman K. Denzin and Yvonna S. Lincoln, editors. (Thousand Oaks, CA: Sage, 1994), pp. 105-117; Pardede, Parlindungan. “Identifying and Formulating the Research Problem." Research in ELT: Module 4 (October 2018): 1-13; Li, Yanmei, and Sumei Zhang. "Identifying the Research Problem." In Applied Research Methods in Urban and Regional Planning . (Cham, Switzerland: Springer International Publishing, 2022), pp. 13-21.

Importance of...

The purpose of a problem statement is to:

• Introduce the reader to the importance of the topic being studied . The reader is oriented to the significance of the study.
• Anchors the research questions, hypotheses, or assumptions to follow . It offers a concise statement about the purpose of your paper.
• Place the topic into a particular context that defines the parameters of what is to be investigated.
• Provide the framework for reporting the results and indicates what is probably necessary to conduct the study and explain how the findings will present this information.

In the social sciences, the research problem establishes the means by which you must answer the "So What?" question. This declarative question refers to a research problem surviving the relevancy test [the quality of a measurement procedure that provides repeatability and accuracy]. Note that answering the "So What?" question requires a commitment on your part to not only show that you have reviewed the literature, but that you have thoroughly considered the significance of the research problem and its implications applied to creating new knowledge and understanding or informing practice.

To survive the "So What" question, problem statements should possess the following attributes:

• Clarity and precision [a well-written statement does not make sweeping generalizations and irresponsible pronouncements; it also does include unspecific determinates like "very" or "giant"],
• Demonstrate a researchable topic or issue [i.e., feasibility of conducting the study is based upon access to information that can be effectively acquired, gathered, interpreted, synthesized, and understood],
• Identification of what would be studied, while avoiding the use of value-laden words and terms,
• Identification of an overarching question or small set of questions accompanied by key factors or variables,
• Identification of key concepts and terms,
• Articulation of the study's conceptual boundaries or parameters or limitations,
• Some generalizability in regards to applicability and bringing results into general use,
• Conveyance of the study's importance, benefits, and justification [i.e., regardless of the type of research, it is important to demonstrate that the research is not trivial],
• Does not have unnecessary jargon or overly complex sentence constructions; and,
• Conveyance of more than the mere gathering of descriptive data providing only a snapshot of the issue or phenomenon under investigation.

Bryman, Alan. “The Research Question in Social Research: What is its Role?” International Journal of Social Research Methodology 10 (2007): 5-20; Brown, Perry J., Allen Dyer, and Ross S. Whaley. "Recreation Research—So What?" Journal of Leisure Research 5 (1973): 16-24; Castellanos, Susie. Critical Writing and Thinking. The Writing Center. Dean of the College. Brown University; Ellis, Timothy J. and Yair Levy Nova. "Framework of Problem-Based Research: A Guide for Novice Researchers on the Development of a Research-Worthy Problem." Informing Science: the International Journal of an Emerging Transdiscipline 11 (2008); Thesis and Purpose Statements. The Writer’s Handbook. Writing Center. University of Wisconsin, Madison; Thesis Statements. The Writing Center. University of North Carolina; Tips and Examples for Writing Thesis Statements. The Writing Lab and The OWL. Purdue University; Selwyn, Neil. "‘So What?’…A Question that Every Journal Article Needs to Answer." Learning, Media, and Technology 39 (2014): 1-5; Shoket, Mohd. "Research Problem: Identification and Formulation." International Journal of Research 1 (May 2014): 512-518.

Structure and Writing Style

I.  Types and Content

There are four general conceptualizations of a research problem in the social sciences:

• Casuist Research Problem -- this type of problem relates to the determination of right and wrong in questions of conduct or conscience by analyzing moral dilemmas through the application of general rules and the careful distinction of special cases.
• Difference Research Problem -- typically asks the question, “Is there a difference between two or more groups or treatments?” This type of problem statement is used when the researcher compares or contrasts two or more phenomena. This a common approach to defining a problem in the clinical social sciences or behavioral sciences.
• Descriptive Research Problem -- typically asks the question, "what is...?" with the underlying purpose to describe the significance of a situation, state, or existence of a specific phenomenon. This problem is often associated with revealing hidden or understudied issues.
• Relational Research Problem -- suggests a relationship of some sort between two or more variables to be investigated. The underlying purpose is to investigate specific qualities or characteristics that may be connected in some way.

A problem statement in the social sciences should contain :

• A lead-in that helps ensure the reader will maintain interest over the study,
• A declaration of originality [e.g., mentioning a knowledge void or a lack of clarity about a topic that will be revealed in the literature review of prior research],
• An indication of the central focus of the study [establishing the boundaries of analysis], and
• An explanation of the study's significance or the benefits to be derived from investigating the research problem.

NOTE:   A statement describing the research problem of your paper should not be viewed as a thesis statement that you may be familiar with from high school. Given the content listed above, a description of the research problem is usually a short paragraph in length.

II.  Sources of Problems for Investigation

The identification of a problem to study can be challenging, not because there's a lack of issues that could be investigated, but due to the challenge of formulating an academically relevant and researchable problem which is unique and does not simply duplicate the work of others. To facilitate how you might select a problem from which to build a research study, consider these sources of inspiration:

Deductions from Theory This relates to deductions made from social philosophy or generalizations embodied in life and in society that the researcher is familiar with. These deductions from human behavior are then placed within an empirical frame of reference through research. From a theory, the researcher can formulate a research problem or hypothesis stating the expected findings in certain empirical situations. The research asks the question: “What relationship between variables will be observed if theory aptly summarizes the state of affairs?” One can then design and carry out a systematic investigation to assess whether empirical data confirm or reject the hypothesis, and hence, the theory.

Interdisciplinary Perspectives Identifying a problem that forms the basis for a research study can come from academic movements and scholarship originating in disciplines outside of your primary area of study. This can be an intellectually stimulating exercise. A review of pertinent literature should include examining research from related disciplines that can reveal new avenues of exploration and analysis. An interdisciplinary approach to selecting a research problem offers an opportunity to construct a more comprehensive understanding of a very complex issue that any single discipline may be able to provide.

Interviewing Practitioners The identification of research problems about particular topics can arise from formal interviews or informal discussions with practitioners who provide insight into new directions for future research and how to make research findings more relevant to practice. Discussions with experts in the field, such as, teachers, social workers, health care providers, lawyers, business leaders, etc., offers the chance to identify practical, “real world” problems that may be understudied or ignored within academic circles. This approach also provides some practical knowledge which may help in the process of designing and conducting your study.

Personal Experience Don't undervalue your everyday experiences or encounters as worthwhile problems for investigation. Think critically about your own experiences and/or frustrations with an issue facing society or related to your community, your neighborhood, your family, or your personal life. This can be derived, for example, from deliberate observations of certain relationships for which there is no clear explanation or witnessing an event that appears harmful to a person or group or that is out of the ordinary.

Relevant Literature The selection of a research problem can be derived from a thorough review of pertinent research associated with your overall area of interest. This may reveal where gaps exist in understanding a topic or where an issue has been understudied. Research may be conducted to: 1) fill such gaps in knowledge; 2) evaluate if the methodologies employed in prior studies can be adapted to solve other problems; or, 3) determine if a similar study could be conducted in a different subject area or applied in a different context or to different study sample [i.e., different setting or different group of people]. Also, authors frequently conclude their studies by noting implications for further research; read the conclusion of pertinent studies because statements about further research can be a valuable source for identifying new problems to investigate. The fact that a researcher has identified a topic worthy of further exploration validates the fact it is worth pursuing.

III.  What Makes a Good Research Statement?

A good problem statement begins by introducing the broad area in which your research is centered, gradually leading the reader to the more specific issues you are investigating. The statement need not be lengthy, but a good research problem should incorporate the following features:

1.  Compelling Topic The problem chosen should be one that motivates you to address it but simple curiosity is not a good enough reason to pursue a research study because this does not indicate significance. The problem that you choose to explore must be important to you, but it must also be viewed as important by your readers and to a the larger academic and/or social community that could be impacted by the results of your study. 2.  Supports Multiple Perspectives The problem must be phrased in a way that avoids dichotomies and instead supports the generation and exploration of multiple perspectives. A general rule of thumb in the social sciences is that a good research problem is one that would generate a variety of viewpoints from a composite audience made up of reasonable people. 3.  Researchability This isn't a real word but it represents an important aspect of creating a good research statement. It seems a bit obvious, but you don't want to find yourself in the midst of investigating a complex research project and realize that you don't have enough prior research to draw from for your analysis. There's nothing inherently wrong with original research, but you must choose research problems that can be supported, in some way, by the resources available to you. If you are not sure if something is researchable, don't assume that it isn't if you don't find information right away--seek help from a librarian !

NOTE:   Do not confuse a research problem with a research topic. A topic is something to read and obtain information about, whereas a problem is something to be solved or framed as a question raised for inquiry, consideration, or solution, or explained as a source of perplexity, distress, or vexation. In short, a research topic is something to be understood; a research problem is something that needs to be investigated.

IV.  Asking Analytical Questions about the Research Problem

Research problems in the social and behavioral sciences are often analyzed around critical questions that must be investigated. These questions can be explicitly listed in the introduction [i.e., "This study addresses three research questions about women's psychological recovery from domestic abuse in multi-generational home settings..."], or, the questions are implied in the text as specific areas of study related to the research problem. Explicitly listing your research questions at the end of your introduction can help in designing a clear roadmap of what you plan to address in your study, whereas, implicitly integrating them into the text of the introduction allows you to create a more compelling narrative around the key issues under investigation. Either approach is appropriate.

The number of questions you attempt to address should be based on the complexity of the problem you are investigating and what areas of inquiry you find most critical to study. Practical considerations, such as, the length of the paper you are writing or the availability of resources to analyze the issue can also factor in how many questions to ask. In general, however, there should be no more than four research questions underpinning a single research problem.

Given this, well-developed analytical questions can focus on any of the following:

• Highlights a genuine dilemma, area of ambiguity, or point of confusion about a topic open to interpretation by your readers;
• Yields an answer that is unexpected and not obvious rather than inevitable and self-evident;
• Provokes meaningful thought or discussion;
• Raises the visibility of the key ideas or concepts that may be understudied or hidden;
• Suggests the need for complex analysis or argument rather than a basic description or summary; and,
• Offers a specific path of inquiry that avoids eliciting generalizations about the problem.

NOTE:   Questions of how and why concerning a research problem often require more analysis than questions about who, what, where, and when. You should still ask yourself these latter questions, however. Thinking introspectively about the who, what, where, and when of a research problem can help ensure that you have thoroughly considered all aspects of the problem under investigation and helps define the scope of the study in relation to the problem.

V.  Mistakes to Avoid

Beware of circular reasoning! Do not state the research problem as simply the absence of the thing you are suggesting. For example, if you propose the following, "The problem in this community is that there is no hospital," this only leads to a research problem where:

• The need is for a hospital
• The objective is to create a hospital
• The method is to plan for building a hospital, and
• The evaluation is to measure if there is a hospital or not.

This is an example of a research problem that fails the "So What?" test . In this example, the problem does not reveal the relevance of why you are investigating the fact there is no hospital in the community [e.g., perhaps there's a hospital in the community ten miles away]; it does not elucidate the significance of why one should study the fact there is no hospital in the community [e.g., that hospital in the community ten miles away has no emergency room]; the research problem does not offer an intellectual pathway towards adding new knowledge or clarifying prior knowledge [e.g., the county in which there is no hospital already conducted a study about the need for a hospital, but it was conducted ten years ago]; and, the problem does not offer meaningful outcomes that lead to recommendations that can be generalized for other situations or that could suggest areas for further research [e.g., the challenges of building a new hospital serves as a case study for other communities].

Alvesson, Mats and Jörgen Sandberg. “Generating Research Questions Through Problematization.” Academy of Management Review 36 (April 2011): 247-271 ; Choosing and Refining Topics. Writing@CSU. Colorado State University; D'Souza, Victor S. "Use of Induction and Deduction in Research in Social Sciences: An Illustration." Journal of the Indian Law Institute 24 (1982): 655-661; Ellis, Timothy J. and Yair Levy Nova. "Framework of Problem-Based Research: A Guide for Novice Researchers on the Development of a Research-Worthy Problem." Informing Science: the International Journal of an Emerging Transdiscipline 11 (2008); How to Write a Research Question. The Writing Center. George Mason University; Invention: Developing a Thesis Statement. The Reading/Writing Center. Hunter College; Problem Statements PowerPoint Presentation. The Writing Lab and The OWL. Purdue University; Procter, Margaret. Using Thesis Statements. University College Writing Centre. University of Toronto; Shoket, Mohd. "Research Problem: Identification and Formulation." International Journal of Research 1 (May 2014): 512-518; Trochim, William M.K. Problem Formulation. Research Methods Knowledge Base. 2006; Thesis and Purpose Statements. The Writer’s Handbook. Writing Center. University of Wisconsin, Madison; Thesis Statements. The Writing Center. University of North Carolina; Tips and Examples for Writing Thesis Statements. The Writing Lab and The OWL. Purdue University; Pardede, Parlindungan. “Identifying and Formulating the Research Problem." Research in ELT: Module 4 (October 2018): 1-13; Walk, Kerry. Asking an Analytical Question. [Class handout or worksheet]. Princeton University; White, Patrick. Developing Research Questions: A Guide for Social Scientists . New York: Palgrave McMillan, 2009; Li, Yanmei, and Sumei Zhang. "Identifying the Research Problem." In Applied Research Methods in Urban and Regional Planning . (Cham, Switzerland: Springer International Publishing, 2022), pp. 13-21.

• << Previous: Background Information
• Next: Theoretical Framework >>
• Last Updated: Sep 4, 2024 9:40 AM
• URL: https://libguides.usc.edu/writingguide

Research Problem vs. Research Question: What's the Difference?

Key Differences

Comparison chart, formulation, role in research, research problem and research question definitions, research problem, research question, what makes a good research question, what is a research problem, can a research problem lead to multiple research questions, why is identifying a research problem important, should a research problem be solvable, is the research problem part of the research proposal, how is a research question different from a research problem, what role does literature review play in defining a research problem, how specific should a research question be, can a research problem be too broad, can a research problem be theoretical, how does one validate a research problem, how does one identify a research problem, why is clarity important in a research problem, do all research questions require empirical data, what impact does the research problem have on methodology, is the research problem always stated explicitly in a study, can research questions evolve during a study, how is a research question formulated, what is the relationship between a research problem and hypothesis.

Trending Comparisons

Popular Comparisons

New Comparisons

Educational resources and simple solutions for your research journey

What is a Research Problem? Characteristics, Types, and Examples

A research problem is a gap in existing knowledge, a contradiction in an established theory, or a real-world challenge that a researcher aims to address in their research. It is at the heart of any scientific inquiry, directing the trajectory of an investigation. The statement of a problem orients the reader to the importance of the topic, sets the problem into a particular context, and defines the relevant parameters, providing the framework for reporting the findings. Therein lies the importance of research problem s.  

The formulation of well-defined research questions is central to addressing a research problem . A research question is a statement made in a question form to provide focus, clarity, and structure to the research endeavor. This helps the researcher design methodologies, collect data, and analyze results in a systematic and coherent manner. A study may have one or more research questions depending on the nature of the study.   

Identifying and addressing a research problem is very important. By starting with a pertinent problem , a scholar can contribute to the accumulation of evidence-based insights, solutions, and scientific progress, thereby advancing the frontier of research. Moreover, the process of formulating research problems and posing pertinent research questions cultivates critical thinking and hones problem-solving skills.   

Table of Contents

What is a Research Problem ?  

Before you conceive of your project, you need to ask yourself “ What is a research problem ?” A research problem definition can be broadly put forward as the primary statement of a knowledge gap or a fundamental challenge in a field, which forms the foundation for research. Conversely, the findings from a research investigation provide solutions to the problem .  

A research problem guides the selection of approaches and methodologies, data collection, and interpretation of results to find answers or solutions. A well-defined problem determines the generation of valuable insights and contributions to the broader intellectual discourse.  

Characteristics of a Research Problem  

Knowing the characteristics of a research problem is instrumental in formulating a research inquiry; take a look at the five key characteristics below:  

Novel : An ideal research problem introduces a fresh perspective, offering something new to the existing body of knowledge. It should contribute original insights and address unresolved matters or essential knowledge.   

Significant : A problem should hold significance in terms of its potential impact on theory, practice, policy, or the understanding of a particular phenomenon. It should be relevant to the field of study, addressing a gap in knowledge, a practical concern, or a theoretical dilemma that holds significance.  

Feasible: A practical research problem allows for the formulation of hypotheses and the design of research methodologies. A feasible research problem is one that can realistically be investigated given the available resources, time, and expertise. It should not be too broad or too narrow to explore effectively, and should be measurable in terms of its variables and outcomes. It should be amenable to investigation through empirical research methods, such as data collection and analysis, to arrive at meaningful conclusions A practical research problem considers budgetary and time constraints, as well as limitations of the problem . These limitations may arise due to constraints in methodology, resources, or the complexity of the problem.  

Clear and specific : A well-defined research problem is clear and specific, leaving no room for ambiguity; it should be easily understandable and precisely articulated. Ensuring specificity in the problem ensures that it is focused, addresses a distinct aspect of the broader topic and is not vague.  

Rooted in evidence: A good research problem leans on trustworthy evidence and data, while dismissing unverifiable information. It must also consider ethical guidelines, ensuring the well-being and rights of any individuals or groups involved in the study.

Types of Research Problems  

Across fields and disciplines, there are different types of research problems . We can broadly categorize them into three types.  

• Theoretical research problems

Theoretical research problems deal with conceptual and intellectual inquiries that may not involve empirical data collection but instead seek to advance our understanding of complex concepts, theories, and phenomena within their respective disciplines. For example, in the social sciences, research problem s may be casuist (relating to the determination of right and wrong in questions of conduct or conscience), difference (comparing or contrasting two or more phenomena), descriptive (aims to describe a situation or state), or relational (investigating characteristics that are related in some way).  

Here are some theoretical research problem examples :   

• Ethical frameworks that can provide coherent justifications for artificial intelligence and machine learning algorithms, especially in contexts involving autonomous decision-making and moral agency.  
• Determining how mathematical models can elucidate the gradual development of complex traits, such as intricate anatomical structures or elaborate behaviors, through successive generations.  
• Applied research problems

Applied or practical research problems focus on addressing real-world challenges and generating practical solutions to improve various aspects of society, technology, health, and the environment.  

Here are some applied research problem examples :   

• Studying the use of precision agriculture techniques to optimize crop yield and minimize resource waste.  
• Designing a more energy-efficient and sustainable transportation system for a city to reduce carbon emissions.  
• Action research problems

Action research problems aim to create positive change within specific contexts by involving stakeholders, implementing interventions, and evaluating outcomes in a collaborative manner.  

Here are some action research problem examples :   

• Partnering with healthcare professionals to identify barriers to patient adherence to medication regimens and devising interventions to address them.  
• Collaborating with a nonprofit organization to evaluate the effectiveness of their programs aimed at providing job training for underserved populations.  

These different types of research problems may give you some ideas when you plan on developing your own.  

How to Define a Research Problem  

You might now ask “ How to define a research problem ?” These are the general steps to follow:   

• Look for a broad problem area: Identify under-explored aspects or areas of concern, or a controversy in your topic of interest. Evaluate the significance of addressing the problem in terms of its potential contribution to the field, practical applications, or theoretical insights.
• Learn more about the problem: Read the literature, starting from historical aspects to the current status and latest updates. Rely on reputable evidence and data. Be sure to consult researchers who work in the relevant field, mentors, and peers. Do not ignore the gray literature on the subject.
• Identify the relevant variables and how they are related: Consider which variables are most important to the study and will help answer the research question. Once this is done, you will need to determine the relationships between these variables and how these relationships affect the research problem . 
• Think of practical aspects : Deliberate on ways that your study can be practical and feasible in terms of time and resources. Discuss practical aspects with researchers in the field and be open to revising the problem based on feedback. Refine the scope of the research problem to make it manageable and specific; consider the resources available, time constraints, and feasibility.
• Formulate the problem statement: Craft a concise problem statement that outlines the specific issue, its relevance, and why it needs further investigation.
• Stick to plans, but be flexible: When defining the problem , plan ahead but adhere to your budget and timeline. At the same time, consider all possibilities and ensure that the problem and question can be modified if needed.

Key Takeaways  

• A research problem concerns an area of interest, a situation necessitating improvement, an obstacle requiring eradication, or a challenge in theory or practical applications.   
• The importance of research problem is that it guides the research and helps advance human understanding and the development of practical solutions.  
• Research problem definition begins with identifying a broad problem area, followed by learning more about the problem, identifying the variables and how they are related, considering practical aspects, and finally developing the problem statement.  
• Different types of research problems include theoretical, applied, and action research problems , and these depend on the discipline and nature of the study.  
• An ideal problem is original, important, feasible, specific, and based on evidence.  

Frequently Asked Questions  

Why is it important to define a research problem?  

Identifying potential issues and gaps as research problems is important for choosing a relevant topic and for determining a well-defined course of one’s research. Pinpointing a problem and formulating research questions can help researchers build their critical thinking, curiosity, and problem-solving abilities.   

How do I identify a research problem?  

Identifying a research problem involves recognizing gaps in existing knowledge, exploring areas of uncertainty, and assessing the significance of addressing these gaps within a specific field of study. This process often involves thorough literature review, discussions with experts, and considering practical implications.  

Can a research problem change during the research process?  

Yes, a research problem can change during the research process. During the course of an investigation a researcher might discover new perspectives, complexities, or insights that prompt a reevaluation of the initial problem. The scope of the problem, unforeseen or unexpected issues, or other limitations might prompt some tweaks. You should be able to adjust the problem to ensure that the study remains relevant and aligned with the evolving understanding of the subject matter.

How does a research problem relate to research questions or hypotheses?  

A research problem sets the stage for the study. Next, research questions refine the direction of investigation by breaking down the broader research problem into manageable components. Research questions are formulated based on the problem , guiding the investigation’s scope and objectives. The hypothesis provides a testable statement to validate or refute within the research process. All three elements are interconnected and work together to guide the research.  

R Discovery is a literature search and research reading platform that accelerates your research discovery journey by keeping you updated on the latest, most relevant scholarly content. With 250M+ research articles sourced from trusted aggregators like CrossRef, Unpaywall, PubMed, PubMed Central, Open Alex and top publishing houses like Springer Nature, JAMA, IOP, Taylor & Francis, NEJM, BMJ, Karger, SAGE, Emerald Publishing and more, R Discovery puts a world of research at your fingertips.  

Try R Discovery Prime FREE for 1 week or upgrade at just US$72 a year to access premium features that let you listen to research on the go, read in your language, collaborate with peers, auto sync with reference managers, and much more. Choose a simpler, smarter way to find and read research – Download the app and start your free 7-day trial today !  

Related Posts

What is Research Impact: Types and Tips for Academics

Research in Shorts: R Discovery’s New Feature Helps Academics Assess Relevant Papers in 2mins 

Organizing Academic Research Papers: The Research Problem/Question

• Purpose of Guide
• Design Flaws to Avoid
• Glossary of Research Terms
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Choosing a Title
• Making an Outline
• Paragraph Development
• Executive Summary
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tertiary Sources
• What Is Scholarly vs. Popular?
• Qualitative Methods
• Quantitative Methods
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Annotated Bibliography
• Dealing with Nervousness
• Using Visual Aids
• Grading Someone Else's Paper
• How to Manage Group Projects
• Multiple Book Review Essay
• Reviewing Collected Essays
• About Informed Consent
• Writing Field Notes
• Writing a Policy Memo
• Writing a Research Proposal
• Acknowledgements

A research problem is a statement about an area of concern, a condition to be improved, a difficulty to be eliminated, or a troubling question that exists in scholarly literature, in theory, or in practice that points to the need for meaningful understanding and deliberate investigation. In some social science disciplines the research problem is typically posed in the form of a question. A research problem does not state how to do something, offer a vague or broad proposition, or present a value question.

Importance of...

The purpose of a problem statement is to:

• Introduce the reader to the importance of the topic being studied . The reader is oriented to the significance of the study and the research questions or hypotheses to follow.
• Places the problem into a particular context that defines the parameters of what is to be investigated.
• Provides the framework for reporting the results and indicates what is probably necessary to conduct the study and explain how the findings will present this information.

In the social sciences, the research problem establishes the means by which you must answer the "So What?" question. The "So What?" question refers to a research problem surviving the relevancy test [the quality of a measurement procedure that provides repeatability and accuracy]. Note that answering the "So What" question requires a commitment on your part to not only show that you have researched the material, but that you have thought about its significance.

To survive the "So What" question, problem statements should possess the following attributes:

• Clarity and precision [a well-written statement does not make sweeping generalizations and irresponsible statements],
• Identification of what would be studied, while avoiding the use of value-laden words and terms,
• Identification of an overarching question and key factors or variables,
• Identification of key concepts and terms,
• Articulation of the study's boundaries or parameters,
• Some generalizability in regards to applicability and bringing results into general use,
• Conveyance of the study's importance, benefits, and justification [regardless of the type of research, it is important to address the “so what” question by demonstrating that the research is not trivial],
• Does not have unnecessary jargon; and,
• Conveyance of more than the mere gathering of descriptive data providing only a snapshot of the issue or phenomenon under investigation.

Castellanos, Susie. Critical Writing and Thinking . The Writing Center. Dean of the College. Brown University; Ellis, Timothy J. and Yair Levy Nova Framework of Problem-Based Research: A Guide for Novice Researchers on the Development of a Research-Worthy Problem. Informing Science: the International Journal of an Emerging Transdiscipline 11 (2008); Thesis and Purpose Statements . The Writer’s Handbook. Writing Center. University of Wisconsin, Madison; Thesis Statements . The Writing Center. University of North Carolina; Tips and Examples for Writing Thesis Statements . The Writing Lab and The OWL. Purdue University.  

Structure and Writing Style

I.  Types and Content

There are four general conceptualizations of a research problem in the social sciences:

• Casuist Research Problem -- this type of problem relates to the determination of right and wrong in questions of conduct or conscience by analyzing moral dilemmas through the application of general rules and the careful distinction of special cases.
• Difference Research Problem -- typically asks the question, “Is there a difference between two or more groups or treatments?” This type of problem statement is used when the researcher compares or contrasts two or more phenomena.
• Descriptive Research Problem -- typically asks the question, "what is...?" with the underlying purpose to describe a situation, state, or existence of a specific phenomenon.
• Relational Research Problem -- suggests a relationship of some sort between two or more variables to be investigated. The underlying purpose is to investigate qualities/characteristics that are connected in some way.

A problem statement in the social sciences should contain :

• A lead-in that helps ensure the reader will maintain interest over the study
• A declaration of originality [e.g., mentioning a knowledge void, which would be supported by the literature review]
• An indication of the central focus of the study, and
• An explanation of the study's significance or the benefits to be derived from an investigating the problem.

II.  Sources of Problems for Investigation

Identifying a problem to study can be challenging, not because there is a lack of issues that could be investigated, but due to pursuing a goal of formulating a socially relevant and researchable problem statement that is unique and does not simply duplicate the work of others. To facilitate how you might select a problem from which to build a research study, consider these three broad sources of inspiration:

Deductions from Theory This relates to deductions made from social philosophy or generalizations embodied in life in society that the researcher is familiar with. These deductions from human behavior are then fitted within an empirical frame of reference through research. From a theory, the research can formulate a research problem or hypothesis stating the expected findings in certain empirical situations. The research asks the question: “What relationship between variables will be observed if theory aptly summarizes the state of affairs?” One can then design and carry out a systematic investigation to assess whether empirical data confirm or reject the hypothesis and hence the theory.

Interdisciplinary Perspectives Identifying a problem that forms the basis for a research study can come from academic movements and scholarship originating in disciplines outside of your primary area of study. A review of pertinent literature should include examining research from related disciplines, which can expose you to new avenues of exploration and analysis. An interdisciplinary approach to selecting a research problem offers an opportunity to construct a more comprehensive understanding of a very complex issue than any single discipline might provide.

Interviewing Practitioners The identification of research problems about particular topics can arise from formal or informal discussions with practitioners who provide insight into new directions for future research and how to make research findings increasingly relevant to practice. Discussions with experts in the field, such as, teachers, social workers, health care providers, etc., offers the chance to identify practical, “real worl” problems that may be understudied or ignored within academic circles. This approach also provides some practical knowledge which may help in the process of designing and conducting your study.

Personal Experience Your everyday experiences can give rise to worthwhile problems for investigation. Think critically about your own experiences and/or frustrations with an issue facing society, your community, or in your neighborhood. This can be derived, for example, from deliberate observations of certain relationships for which there is no clear explanation or witnessing an event that appears harmful to a person or group or that is out of the ordinary.

Relevant Literature The selection of a research problem can often be derived from an extensive and thorough review of pertinent research associated with your overall area of interest. This may reveal where gaps remain in our understanding of a topic. Research may be conducted to: 1) fill such gaps in knowledge; 2) evaluate if the methodologies employed in prior studies can be adapted to solve other problems; or, 3) determine if a similar study could be conducted in a different subject area or applied to different study sample [i.e., different groups of people]. Also, authors frequently conclude their studies by noting implications for further research; this can also be a valuable source of problems to investigate.

III.  What Makes a Good Research Statement?

A good problem statement begins by introducing the broad area in which your research is centered and then gradually leads the reader to the more narrow questions you are posing. The statement need not be lengthy but a good research problem should incorporate the following features:

Compelling topic Simple curiosity is not a good enough reason to pursue a research study. The problem that you choose to explore must be important to you and to a larger community you share. The problem chosen must be one that motivates you to address it. Supports multiple perspectives The problem most be phrased in a way that avoids dichotomies and instead supports the generation and exploration of multiple perspectives. A general rule of thumb is that a good research problem is one that would generate a variety of viewpoints from a composite audience made up of reasonable people. Researchable It seems a bit obvious, but you don't want to find yourself in the midst of investigating a complex  research project and realize that you don't have much to draw on for your research. Choose research problems that can be supported by the resources available to you. Not sure? Seek out help  from a librarian!

NOTE:   Do not confuse a research problem with a research topic. A topic is something to read and obtain information about whereas a problem is something to solve or framed as a question that must be answered.

IV.  Mistakes to Avoid

Beware of circular reasoning . Don’t state that the research problem as simply the absence of the thing you are suggesting. For example, if you propose, "The problem in this community is that it has no hospital."

This only leads to a research problem where:

• The need is for a hospital
• The objective is to create a hospital
• The method is to plan for building a hospital, and
• The evaluation is to measure if there is a hospital or not.

This is an example of a research problem that fails the "so what?" test because it does not reveal the relevance of why you are investigating the problem of having no hospital in the community [e.g., there's a hospital in the community ten miles away] and because the research problem does not elucidate the significance of why one should study the fact that no hospital exists in the community [e.g., that hospital in the community ten miles away has no emergency room].

Choosing and Refining Topics . Writing@CSU. Colorado State University; Ellis, Timothy J. and Yair Levy Nova Framework of Problem-Based Research: A Guide for Novice Researchers on the Development of a Research-Worthy Problem. Informing Science: the International Journal of an Emerging Transdiscipline 11 (2008); How to Write a Research Question . The Writing Center. George Mason University; Invention: Developing a Thesis Statement . The Reading/Writing Center. Hunter College; Problem Statements PowerPoint Presentation . The Writing Lab and The OWL. Purdue University; Procter, Margaret. Using Thesis Statements . University College Writing Centre. University of Toronto; Trochim, William M.K. Problem Formulation . Research Methods Knowledge Base. 2006; Thesis and Purpose Statements . The Writer’s Handbook. Writing Center. University of Wisconsin, Madison; Thesis Statements . The Writing Center. University of North Carolina; Tips and Examples for Writing Thesis Statements . The Writing Lab and The OWL. Purdue University.

• << Previous: Background Information
• Next: Theoretical Framework >>
• Last Updated: Jul 18, 2023 11:58 AM
• URL: https://library.sacredheart.edu/c.php?g=29803
• QuickSearch
• Library Catalog
• Databases A-Z
• Publication Finder
• Course Reserves
• Citation Linker
• Digital Commons
• Our Website

Research Support

• Ask a Librarian
• Appointments
• Interlibrary Loan (ILL)
• Research Guides
• Databases by Subject
• Citation Help

Using the Library

• Reserve a Group Study Room
• Renew Books
• Honors Study Rooms
• Off-Campus Access
• Library Policies
• Library Technology

User Information

• Grad Students
• Online Students
• COVID-19 Updates
• Staff Directory
• News & Announcements
• Library Newsletter

My Accounts

• Interlibrary Loan
• Staff Site Login

FIND US ON  

• Privacy Policy

Home » Research Problem – Examples, Types and Guide

Research Problem – Examples, Types and Guide

Table of Contents

Research Problem

Definition:

Research problem is a specific and well-defined issue or question that a researcher seeks to investigate through research. It is the starting point of any research project, as it sets the direction, scope, and purpose of the study.

Types of Research Problems

Types of Research Problems are as follows:

Descriptive problems

These problems involve describing or documenting a particular phenomenon, event, or situation. For example, a researcher might investigate the demographics of a particular population, such as their age, gender, income, and education.

Exploratory problems

These problems are designed to explore a particular topic or issue in depth, often with the goal of generating new ideas or hypotheses. For example, a researcher might explore the factors that contribute to job satisfaction among employees in a particular industry.

Explanatory Problems

These problems seek to explain why a particular phenomenon or event occurs, and they typically involve testing hypotheses or theories. For example, a researcher might investigate the relationship between exercise and mental health, with the goal of determining whether exercise has a causal effect on mental health.

Predictive Problems

These problems involve making predictions or forecasts about future events or trends. For example, a researcher might investigate the factors that predict future success in a particular field or industry.

Evaluative Problems

These problems involve assessing the effectiveness of a particular intervention, program, or policy. For example, a researcher might evaluate the impact of a new teaching method on student learning outcomes.

How to Define a Research Problem

Defining a research problem involves identifying a specific question or issue that a researcher seeks to address through a research study. Here are the steps to follow when defining a research problem:

• Identify a broad research topic : Start by identifying a broad topic that you are interested in researching. This could be based on your personal interests, observations, or gaps in the existing literature.
• Conduct a literature review : Once you have identified a broad topic, conduct a thorough literature review to identify the current state of knowledge in the field. This will help you identify gaps or inconsistencies in the existing research that can be addressed through your study.
• Refine the research question: Based on the gaps or inconsistencies identified in the literature review, refine your research question to a specific, clear, and well-defined problem statement. Your research question should be feasible, relevant, and important to the field of study.
• Develop a hypothesis: Based on the research question, develop a hypothesis that states the expected relationship between variables.
• Define the scope and limitations: Clearly define the scope and limitations of your research problem. This will help you focus your study and ensure that your research objectives are achievable.
• Get feedback: Get feedback from your advisor or colleagues to ensure that your research problem is clear, feasible, and relevant to the field of study.

Components of a Research Problem

The components of a research problem typically include the following:

• Topic : The general subject or area of interest that the research will explore.
• Research Question : A clear and specific question that the research seeks to answer or investigate.
• Objective : A statement that describes the purpose of the research, what it aims to achieve, and the expected outcomes.
• Hypothesis : An educated guess or prediction about the relationship between variables, which is tested during the research.
• Variables : The factors or elements that are being studied, measured, or manipulated in the research.
• Methodology : The overall approach and methods that will be used to conduct the research.
• Scope and Limitations : A description of the boundaries and parameters of the research, including what will be included and excluded, and any potential constraints or limitations.
• Significance: A statement that explains the potential value or impact of the research, its contribution to the field of study, and how it will add to the existing knowledge.

Research Problem Examples

Following are some Research Problem Examples:

Research Problem Examples in Psychology are as follows:

• Exploring the impact of social media on adolescent mental health.
• Investigating the effectiveness of cognitive-behavioral therapy for treating anxiety disorders.
• Studying the impact of prenatal stress on child development outcomes.
• Analyzing the factors that contribute to addiction and relapse in substance abuse treatment.
• Examining the impact of personality traits on romantic relationships.

Research Problem Examples in Sociology are as follows:

• Investigating the relationship between social support and mental health outcomes in marginalized communities.
• Studying the impact of globalization on labor markets and employment opportunities.
• Analyzing the causes and consequences of gentrification in urban neighborhoods.
• Investigating the impact of family structure on social mobility and economic outcomes.
• Examining the effects of social capital on community development and resilience.

Research Problem Examples in Economics are as follows:

• Studying the effects of trade policies on economic growth and development.
• Analyzing the impact of automation and artificial intelligence on labor markets and employment opportunities.
• Investigating the factors that contribute to economic inequality and poverty.
• Examining the impact of fiscal and monetary policies on inflation and economic stability.
• Studying the relationship between education and economic outcomes, such as income and employment.

Political Science

Research Problem Examples in Political Science are as follows:

• Analyzing the causes and consequences of political polarization and partisan behavior.
• Investigating the impact of social movements on political change and policymaking.
• Studying the role of media and communication in shaping public opinion and political discourse.
• Examining the effectiveness of electoral systems in promoting democratic governance and representation.
• Investigating the impact of international organizations and agreements on global governance and security.

Environmental Science

Research Problem Examples in Environmental Science are as follows:

• Studying the impact of air pollution on human health and well-being.
• Investigating the effects of deforestation on climate change and biodiversity loss.
• Analyzing the impact of ocean acidification on marine ecosystems and food webs.
• Studying the relationship between urban development and ecological resilience.
• Examining the effectiveness of environmental policies and regulations in promoting sustainability and conservation.

Research Problem Examples in Education are as follows:

• Investigating the impact of teacher training and professional development on student learning outcomes.
• Studying the effectiveness of technology-enhanced learning in promoting student engagement and achievement.
• Analyzing the factors that contribute to achievement gaps and educational inequality.
• Examining the impact of parental involvement on student motivation and achievement.
• Studying the effectiveness of alternative educational models, such as homeschooling and online learning.

Research Problem Examples in History are as follows:

• Analyzing the social and economic factors that contributed to the rise and fall of ancient civilizations.
• Investigating the impact of colonialism on indigenous societies and cultures.
• Studying the role of religion in shaping political and social movements throughout history.
• Analyzing the impact of the Industrial Revolution on economic and social structures.
• Examining the causes and consequences of global conflicts, such as World War I and II.

Research Problem Examples in Business are as follows:

• Studying the impact of corporate social responsibility on brand reputation and consumer behavior.
• Investigating the effectiveness of leadership development programs in improving organizational performance and employee satisfaction.
• Analyzing the factors that contribute to successful entrepreneurship and small business development.
• Examining the impact of mergers and acquisitions on market competition and consumer welfare.
• Studying the effectiveness of marketing strategies and advertising campaigns in promoting brand awareness and sales.

Research Problem Example for Students

An Example of a Research Problem for Students could be:

“How does social media usage affect the academic performance of high school students?”

This research problem is specific, measurable, and relevant. It is specific because it focuses on a particular area of interest, which is the impact of social media on academic performance. It is measurable because the researcher can collect data on social media usage and academic performance to evaluate the relationship between the two variables. It is relevant because it addresses a current and important issue that affects high school students.

To conduct research on this problem, the researcher could use various methods, such as surveys, interviews, and statistical analysis of academic records. The results of the study could provide insights into the relationship between social media usage and academic performance, which could help educators and parents develop effective strategies for managing social media use among students.

Another example of a research problem for students:

“Does participation in extracurricular activities impact the academic performance of middle school students?”

This research problem is also specific, measurable, and relevant. It is specific because it focuses on a particular type of activity, extracurricular activities, and its impact on academic performance. It is measurable because the researcher can collect data on students’ participation in extracurricular activities and their academic performance to evaluate the relationship between the two variables. It is relevant because extracurricular activities are an essential part of the middle school experience, and their impact on academic performance is a topic of interest to educators and parents.

To conduct research on this problem, the researcher could use surveys, interviews, and academic records analysis. The results of the study could provide insights into the relationship between extracurricular activities and academic performance, which could help educators and parents make informed decisions about the types of activities that are most beneficial for middle school students.

Applications of Research Problem

Applications of Research Problem are as follows:

• Academic research: Research problems are used to guide academic research in various fields, including social sciences, natural sciences, humanities, and engineering. Researchers use research problems to identify gaps in knowledge, address theoretical or practical problems, and explore new areas of study.
• Business research : Research problems are used to guide business research, including market research, consumer behavior research, and organizational research. Researchers use research problems to identify business challenges, explore opportunities, and develop strategies for business growth and success.
• Healthcare research : Research problems are used to guide healthcare research, including medical research, clinical research, and health services research. Researchers use research problems to identify healthcare challenges, develop new treatments and interventions, and improve healthcare delivery and outcomes.
• Public policy research : Research problems are used to guide public policy research, including policy analysis, program evaluation, and policy development. Researchers use research problems to identify social issues, assess the effectiveness of existing policies and programs, and develop new policies and programs to address societal challenges.
• Environmental research : Research problems are used to guide environmental research, including environmental science, ecology, and environmental management. Researchers use research problems to identify environmental challenges, assess the impact of human activities on the environment, and develop sustainable solutions to protect the environment.

Purpose of Research Problems

The purpose of research problems is to identify an area of study that requires further investigation and to formulate a clear, concise and specific research question. A research problem defines the specific issue or problem that needs to be addressed and serves as the foundation for the research project.

Identifying a research problem is important because it helps to establish the direction of the research and sets the stage for the research design, methods, and analysis. It also ensures that the research is relevant and contributes to the existing body of knowledge in the field.

A well-formulated research problem should:

• Clearly define the specific issue or problem that needs to be investigated
• Be specific and narrow enough to be manageable in terms of time, resources, and scope
• Be relevant to the field of study and contribute to the existing body of knowledge
• Be feasible and realistic in terms of available data, resources, and research methods
• Be interesting and intellectually stimulating for the researcher and potential readers or audiences.

Characteristics of Research Problem

The characteristics of a research problem refer to the specific features that a problem must possess to qualify as a suitable research topic. Some of the key characteristics of a research problem are:

• Clarity : A research problem should be clearly defined and stated in a way that it is easily understood by the researcher and other readers. The problem should be specific, unambiguous, and easy to comprehend.
• Relevance : A research problem should be relevant to the field of study, and it should contribute to the existing body of knowledge. The problem should address a gap in knowledge, a theoretical or practical problem, or a real-world issue that requires further investigation.
• Feasibility : A research problem should be feasible in terms of the availability of data, resources, and research methods. It should be realistic and practical to conduct the study within the available time, budget, and resources.
• Novelty : A research problem should be novel or original in some way. It should represent a new or innovative perspective on an existing problem, or it should explore a new area of study or apply an existing theory to a new context.
• Importance : A research problem should be important or significant in terms of its potential impact on the field or society. It should have the potential to produce new knowledge, advance existing theories, or address a pressing societal issue.
• Manageability : A research problem should be manageable in terms of its scope and complexity. It should be specific enough to be investigated within the available time and resources, and it should be broad enough to provide meaningful results.

Advantages of Research Problem

The advantages of a well-defined research problem are as follows:

• Focus : A research problem provides a clear and focused direction for the research study. It ensures that the study stays on track and does not deviate from the research question.
• Clarity : A research problem provides clarity and specificity to the research question. It ensures that the research is not too broad or too narrow and that the research objectives are clearly defined.
• Relevance : A research problem ensures that the research study is relevant to the field of study and contributes to the existing body of knowledge. It addresses gaps in knowledge, theoretical or practical problems, or real-world issues that require further investigation.
• Feasibility : A research problem ensures that the research study is feasible in terms of the availability of data, resources, and research methods. It ensures that the research is realistic and practical to conduct within the available time, budget, and resources.
• Novelty : A research problem ensures that the research study is original and innovative. It represents a new or unique perspective on an existing problem, explores a new area of study, or applies an existing theory to a new context.
• Importance : A research problem ensures that the research study is important and significant in terms of its potential impact on the field or society. It has the potential to produce new knowledge, advance existing theories, or address a pressing societal issue.
• Rigor : A research problem ensures that the research study is rigorous and follows established research methods and practices. It ensures that the research is conducted in a systematic, objective, and unbiased manner.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Future Research – Thesis Guide

Research Methods – Types, Examples and Guide

Chapter Summary & Overview – Writing Guide...

Research Approach – Types Methods and Examples

Research Report – Example, Writing Guide and...

Research Contribution – Thesis Guide

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base
• Starting the research process
• Writing Strong Research Questions | Criteria & Examples

Writing Strong Research Questions | Criteria & Examples

Published on October 26, 2022 by Shona McCombes . Revised on November 21, 2023.

A research question pinpoints exactly what you want to find out in your work. A good research question is essential to guide your research paper , dissertation , or thesis .

All research questions should be:

• Focused on a single problem or issue
• Researchable using primary and/or secondary sources
• Feasible to answer within the timeframe and practical constraints
• Specific enough to answer thoroughly
• Complex enough to develop the answer over the space of a paper or thesis
• Relevant to your field of study and/or society more broadly

Table of contents

How to write a research question, what makes a strong research question, using sub-questions to strengthen your main research question, research questions quiz, other interesting articles, frequently asked questions about research questions.

You can follow these steps to develop a strong research question:

• Choose your topic
• Do some preliminary reading about the current state of the field
• Narrow your focus to a specific niche
• Identify the research problem that you will address

The way you frame your question depends on what your research aims to achieve. The table below shows some examples of how you might formulate questions for different purposes.

Using your research problem to develop your research question

Note that while most research questions can be answered with various types of research , the way you frame your question should help determine your choices.

Prevent plagiarism. Run a free check.

Research questions anchor your whole project, so it’s important to spend some time refining them. The criteria below can help you evaluate the strength of your research question.

Focused and researchable

Feasible and specific

Complex and arguable

Relevant and original

Chances are that your main research question likely can’t be answered all at once. That’s why sub-questions are important: they allow you to answer your main question in a step-by-step manner.

Good sub-questions should be:

• Less complex than the main question
• Focused only on 1 type of research
• Presented in a logical order

Here are a few examples of descriptive and framing questions:

• Descriptive: According to current government arguments, how should a European bank tax be implemented?
• Descriptive: Which countries have a bank tax/levy on financial transactions?
• Framing: How should a bank tax/levy on financial transactions look at a European level?

Keep in mind that sub-questions are by no means mandatory. They should only be asked if you need the findings to answer your main question. If your main question is simple enough to stand on its own, it’s okay to skip the sub-question part. As a rule of thumb, the more complex your subject, the more sub-questions you’ll need.

Try to limit yourself to 4 or 5 sub-questions, maximum. If you feel you need more than this, it may be indication that your main research question is not sufficiently specific. In this case, it’s is better to revisit your problem statement and try to tighten your main question up.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

Methodology

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

The way you present your research problem in your introduction varies depending on the nature of your research paper . A research paper that presents a sustained argument will usually encapsulate this argument in a thesis statement .

A research paper designed to present the results of empirical research tends to present a research question that it seeks to answer. It may also include a hypothesis —a prediction that will be confirmed or disproved by your research.

As you cannot possibly read every source related to your topic, it’s important to evaluate sources to assess their relevance. Use preliminary evaluation to determine whether a source is worth examining in more depth.

This involves:

• Reading abstracts , prefaces, introductions , and conclusions
• Looking at the table of contents to determine the scope of the work
• Consulting the index for key terms or the names of important scholars

A research hypothesis is your proposed answer to your research question. The research hypothesis usually includes an explanation (“ x affects y because …”).

A statistical hypothesis, on the other hand, is a mathematical statement about a population parameter. Statistical hypotheses always come in pairs: the null and alternative hypotheses . In a well-designed study , the statistical hypotheses correspond logically to the research hypothesis.

Formulating a main research question can be a difficult task. Overall, your question should contribute to solving the problem that you have defined in your problem statement .

However, it should also fulfill criteria in three main areas:

• Researchability
• Feasibility and specificity
• Relevance and originality

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, November 21). Writing Strong Research Questions | Criteria & Examples. Scribbr. Retrieved September 3, 2024, from https://www.scribbr.com/research-process/research-questions/

Is this article helpful?

Shona McCombes

Other students also liked, how to define a research problem | ideas & examples, how to write a problem statement | guide & examples, 10 research question examples to guide your research project, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

How to Write a Research Question: Types and Examples 

The first step in any research project is framing the research question. It can be considered the core of any systematic investigation as the research outcomes are tied to asking the right questions. Thus, this primary interrogation point sets the pace for your research as it helps collect relevant and insightful information that ultimately influences your work.   

Typically, the research question guides the stages of inquiry, analysis, and reporting. Depending on the use of quantifiable or quantitative data, research questions are broadly categorized into quantitative or qualitative research questions. Both types of research questions can be used independently or together, considering the overall focus and objectives of your research.  

What is a research question?

A research question is a clear, focused, concise, and arguable question on which your research and writing are centered. 1 It states various aspects of the study, including the population and variables to be studied and the problem the study addresses. These questions also set the boundaries of the study, ensuring cohesion. 

Designing the research question is a dynamic process where the researcher can change or refine the research question as they review related literature and develop a framework for the study. Depending on the scale of your research, the study can include single or multiple research questions. 

A good research question has the following features: 

• It is relevant to the chosen field of study. 
• The question posed is arguable and open for debate, requiring synthesizing and analysis of ideas. 
• It is focused and concisely framed. 
• A feasible solution is possible within the given practical constraint and timeframe. 

A poorly formulated research question poses several risks. 1   

• Researchers can adopt an erroneous design. 
• It can create confusion and hinder the thought process, including developing a clear protocol.  
• It can jeopardize publication efforts.  
• It causes difficulty in determining the relevance of the study findings.  
• It causes difficulty in whether the study fulfils the inclusion criteria for systematic review and meta-analysis. This creates challenges in determining whether additional studies or data collection is needed to answer the question.  
• Readers may fail to understand the objective of the study. This reduces the likelihood of the study being cited by others. 

Now that you know “What is a research question?”, let’s look at the different types of research questions. 

Types of research questions

Depending on the type of research to be done, research questions can be classified broadly into quantitative, qualitative, or mixed-methods studies. Knowing the type of research helps determine the best type of research question that reflects the direction and epistemological underpinnings of your research. 

The structure and wording of quantitative 2 and qualitative research 3 questions differ significantly. The quantitative study looks at causal relationships, whereas the qualitative study aims at exploring a phenomenon. 

• Quantitative research questions:  
• Seeks to investigate social, familial, or educational experiences or processes in a particular context and/or location.  
• Answers ‘how,’ ‘what,’ or ‘why’ questions. 
• Investigates connections, relations, or comparisons between independent and dependent variables. 

Quantitative research questions can be further categorized into descriptive, comparative, and relationship, as explained in the Table below. 

• Qualitative research questions  

Qualitative research questions are adaptable, non-directional, and more flexible. It concerns broad areas of research or more specific areas of study to discover, explain, or explore a phenomenon. These are further classified as follows: 

• Mixed-methods studies  

Mixed-methods studies use both quantitative and qualitative research questions to answer your research question. Mixed methods provide a complete picture than standalone quantitative or qualitative research, as it integrates the benefits of both methods. Mixed methods research is often used in multidisciplinary settings and complex situational or societal research, especially in the behavioral, health, and social science fields. 

What makes a good research question

A good research question should be clear and focused to guide your research. It should synthesize multiple sources to present your unique argument, and should ideally be something that you are interested in. But avoid questions that can be answered in a few factual statements. The following are the main attributes of a good research question. 

• Specific: The research question should not be a fishing expedition performed in the hopes that some new information will be found that will benefit the researcher. The central research question should work with your research problem to keep your work focused. If using multiple questions, they should all tie back to the central aim. 
• Measurable: The research question must be answerable using quantitative and/or qualitative data or from scholarly sources to develop your research question. If such data is impossible to access, it is better to rethink your question. 
• Attainable: Ensure you have enough time and resources to do all research required to answer your question. If it seems you will not be able to gain access to the data you need, consider narrowing down your question to be more specific. 
• You have the expertise 
• You have the equipment and resources 
• Realistic: Developing your research question should be based on initial reading about your topic. It should focus on addressing a problem or gap in the existing knowledge in your field or discipline. 
• Based on some sort of rational physics 
• Can be done in a reasonable time frame 
• Timely: The research question should contribute to an existing and current debate in your field or in society at large. It should produce knowledge that future researchers or practitioners can later build on. 
• Novel 
• Based on current technologies. 
• Important to answer current problems or concerns. 
• Lead to new directions. 
• Important: Your question should have some aspect of originality. Incremental research is as important as exploring disruptive technologies. For example, you can focus on a specific location or explore a new angle. 
• Meaningful whether the answer is “Yes” or “No.” Closed-ended, yes/no questions are too simple to work as good research questions. Such questions do not provide enough scope for robust investigation and discussion. A good research question requires original data, synthesis of multiple sources, and original interpretation and argumentation before providing an answer. 

Steps for developing a good research question

The importance of research questions cannot be understated. When drafting a research question, use the following frameworks to guide the components of your question to ease the process. 4  

• Determine the requirements: Before constructing a good research question, set your research requirements. What is the purpose? Is it descriptive, comparative, or explorative research? Determining the research aim will help you choose the most appropriate topic and word your question appropriately. 
• Select a broad research topic: Identify a broader subject area of interest that requires investigation. Techniques such as brainstorming or concept mapping can help identify relevant connections and themes within a broad research topic. For example, how to learn and help students learn. 
• Perform preliminary investigation: Preliminary research is needed to obtain up-to-date and relevant knowledge on your topic. It also helps identify issues currently being discussed from which information gaps can be identified. 
• Narrow your focus: Narrow the scope and focus of your research to a specific niche. This involves focusing on gaps in existing knowledge or recent literature or extending or complementing the findings of existing literature. Another approach involves constructing strong research questions that challenge your views or knowledge of the area of study (Example: Is learning consistent with the existing learning theory and research). 
• Identify the research problem: Once the research question has been framed, one should evaluate it. This is to realize the importance of the research questions and if there is a need for more revising (Example: How do your beliefs on learning theory and research impact your instructional practices). 

How to write a research question

Those struggling to understand how to write a research question, these simple steps can help you simplify the process of writing a research question. 

Sample Research Questions

The following are some bad and good research question examples 

• Example 1 

• Example 2 

References:  

• Thabane, L., Thomas, T., Ye, C., & Paul, J. (2009). Posing the research question: not so simple.  Canadian Journal of Anesthesia/Journal canadien d’anesthésie ,  56 (1), 71-79. 
• Rutberg, S., & Bouikidis, C. D. (2018). Focusing on the fundamentals: A simplistic differentiation between qualitative and quantitative research.  Nephrology Nursing Journal ,  45 (2), 209-213. 
• Kyngäs, H. (2020). Qualitative research and content analysis.  The application of content analysis in nursing science research , 3-11. 
• Mattick, K., Johnston, J., & de la Croix, A. (2018). How to… write a good research question.  The clinical teacher ,  15 (2), 104-108. 
• Fandino, W. (2019). Formulating a good research question: Pearls and pitfalls.  Indian Journal of Anaesthesia ,  63 (8), 611. 
• Richardson, W. S., Wilson, M. C., Nishikawa, J., & Hayward, R. S. (1995). The well-built clinical question: a key to evidence-based decisions.  ACP journal club ,  123 (3), A12-A13 

Paperpal is a comprehensive AI writing toolkit that helps students and researchers achieve 2x the writing in half the time. It leverages 21+ years of STM experience and insights from millions of research articles to provide in-depth academic writing, language editing, and submission readiness support to help you write better, faster.  

Get accurate academic translations, rewriting support, grammar checks, vocabulary suggestions, and generative AI assistance that delivers human precision at machine speed. Try for free or upgrade to Paperpal Prime starting at US$19 a month to access premium features, including consistency, plagiarism, and 30+ submission readiness checks to help you succeed.  

Experience the future of academic writing – Sign up to Paperpal and start writing for free!  

Related Reads:

• Scientific Writing Style Guides Explained
• Ethical Research Practices For Research with Human Subjects
• 8 Most Effective Ways to Increase Motivation for Thesis Writing 
• 6 Tips for Post-Doc Researchers to Take Their Career to the Next Level

Transitive and Intransitive Verbs in the World of Research

Language and grammar rules for academic writing, you may also like, how to cite in apa format (7th edition):..., how to write your research paper in apa..., how to choose a dissertation topic, how to write a phd research proposal, how to write an academic paragraph (step-by-step guide), research funding basics: what should a grant proposal..., how to write the first draft of a..., mla works cited page: format, template & examples, academic editing: how to self-edit academic text with..., measuring academic success: definition & strategies for excellence.

• Thesis Action Plan New
• Academic Project Planner

Literature Navigator

Thesis dialogue blueprint, writing wizard's template, research proposal compass.

• Why students love us
• Rebels Blog
• Why we are different
• All Products
• Coming Soon

Identifying a Research Problem: A Step-by-Step Guide

The first and perhaps most important step in the research process is identifying a research problem. This step sets the foundation for all subsequent research activities and largely determines the success of your scholarly work.

This guide provides a comprehensive overview of the steps involved in identifying a research problem, from understanding its essence to employing advanced strategies for refinement.

Key Takeaways

• Remember: Grasping the definition and importance of a research problem isn't just a step—it's crucial for your academic success.
• Exploring various sources, like literature reviews and expert consultations, can guide you in formulating a solid research problem.
• A clear problem statement, aligned research objectives, and well-defined questions are crucial for a focused study.
• Evaluating the feasibility and potential impact of a research problem ensures its relevance and scope.
• Advanced strategies, including interdisciplinary approaches and technology utilization, can enhance the identification and refinement of research problems.

Understanding the Essence of Identifying a Research Problem

Defining the research problem.

A research problem is the focal point of any academic inquiry. It is a concise and well-defined statement that outlines the specific issue or question that the research aims to address. This research problem usually sets the tone for the entire study and provides you, the researcher, with a clear purpose and a clear direction on how to go about conducting your research.

Importance in Academic Research

It also demonstrates the significance of your research and its potential to contribute new knowledge to the existing body of literature in the world. A compelling research problem not only captivates the attention of your peers but also lays the foundation for impactful and meaningful research outcomes.

Initial Steps to Identification

To identify a research problem, you need a systematic approach and a deep understanding of the subject area. Below are some steps to guide you in this process:

• Conduct a thorough literature review to understand what has been studied before.
• Identify gaps in the existing research that could form the basis of your study.
• Consult with academic mentors to refine your ideas and approach.

Exploring Sources for Research Problem Identification

Literature review.

When you embark on the journey of identifying a research problem, a thorough literature review is indispensable. This process involves scrutinizing existing research to find literature gaps and unexplored areas that could form the basis of your research. It's crucial to analyze recent studies, seminal works, and review articles to ensure a comprehensive understanding of the topic.

Existing Theories and Frameworks

The exploration of existing theories and frameworks provides a solid foundation for developing a research problem. By understanding the established models and theories, you can identify inconsistencies or areas lacking in depth which might offer fruitful avenues for research.

Consultation with Academic Mentors

Engaging with academic mentors is vital in shaping a well-defined research problem. Their expertise can guide you through the complexities of your field, offering insights into feasible research questions and helping you refine your focus. This interaction often leads to the identification of unique and significant research opportunities that align with current academic and industry trends.

Formulating the Research Problem

Crafting a clear problem statement.

To effectively address your research problem, start by crafting a clear problem statement . This involves succinctly describing who is affected by the problem, why it is important, and how your research will contribute to solving it. Ensure your problem statement is concise and specific to guide the entire research process.

Setting Research Objectives

Setting clear research objectives is crucial for maintaining focus throughout your study. These objectives should directly align with the problem statement and guide your research activities. Consider using a bulleted list to outline your main objectives:

• Understand the underlying factors contributing to the problem
• Explore potential solutions
• Evaluate the effectiveness of proposed solutions

Determining Research Questions

The formulation of precise research questions is a pivotal step in defining the scope and direction of your study. These questions should be directly derived from your research objectives and designed to be answerable through your chosen research methods. Crafting well-defined research questions will help you maintain a clear focus and avoid common pitfalls in the research process.

How to Evaluate the Scope and Relevance of Your Research Problem

Feasibility assessment.

Before you finalize a research problem, it is crucial to assess its feasibility. Consider the availability of resources, time, and expertise required to conduct the research. Evaluate potential constraints and determine if the research problem can be realistically tackled within the given limitations.

Significance to the Field

Ask yourself: Does my research problem have a clear and direct impact on my field? How will it contribute to advancing knowledge? It should aim to contribute to existing knowledge and address a real-world issue that is relevant to your academic discipline.

Potential Impact on Existing Knowledge

The potential impact of your research problem on existing knowledge cannot be understated. It should challenge, extend, or refine current understanding in a meaningful way. Consider how your research can add value to the existing body of work and potentially lead to significant advancements in your field.

Techniques for Refining the Research Problem

Narrowing down the focus.

To effectively refine your research problem, start by narrowing down the focus . This involves pinpointing the specific aspects of your topic that are most significant and ensuring that your research problem is not too broad. This targeted approach helps in identifying knowledge gaps and formulating more precise research questions.

Incorporating Feedback

Feedback is crucial in the refinement process. Engage with academic mentors, peers, and experts in your field to gather insights and suggestions. This collaborative feedback can lead to significant improvements in your research problem, making it more robust and relevant.

Iterative Refinement Process

Refinement should be seen as an iterative process, where you continuously refine and revise your research problem based on new information and feedback. This approach ensures that your research problem remains aligned with current trends and academic standards, ultimately enhancing its feasibility and relevance.

Challenges in Identifying a Research Problem

Common pitfalls and how to avoid them.

Identifying a research problem can be fraught with common pitfalls such as selecting a topic that is too broad or too narrow. To avoid these, you should conduct a thorough literature review and seek feedback from peers and mentors. This proactive approach ensures that your research question is both relevant and manageable.

Dealing with Ambiguity

Ambiguity in defining the research problem can lead to significant challenges down the line. Ensure clarity by operationalizing variables and explicitly stating the research objectives. This clarity will guide your entire research process, making it more structured and focused.

Balancing Novelty and Practicality

While it's important to address a novel issue in your research, practicality should not be overlooked. A research problem should not only contribute new knowledge but also be feasible and have clear implications. Balancing these aspects often requires iterative refinement and consultation with academic mentors to align your research with real-world applications.

Advanced Strategies for Identifying a Research Problem

Interdisciplinary approaches.

Embrace the power of interdisciplinary approaches to uncover unique and comprehensive research problems. By integrating knowledge from various disciplines, you can address complex issues that single-field studies might overlook. This method not only broadens the scope of your research but also enhances its applicability and depth.

Utilizing Technology and Data Analytics

Leverage technology and data analytics to refine and identify research problems with precision. Advanced tools like machine learning and big data analysis can reveal patterns and insights that traditional methods might miss. This approach is particularly useful in fields where large datasets are involved, or where real-time data integration can lead to more dynamic research outcomes.

Engaging with Industry and Community Needs

Focus on the needs of industry and community to ensure your research is not only academically sound but also practically relevant. Engaging with real-world problems can provide a rich source of research questions that are directly applicable and beneficial to society. This strategy not only enhances the relevance of your research but also increases its potential for impact.

Dive into the world of academic success with our 'Advanced Strategies for Identifying a Research Problem' at Research Rebels. Our expertly crafted guides and action plans are designed to simplify your thesis journey, transforming complex academic challenges into manageable tasks. Don't wait to take control of your academic future. Visit our website now to learn more and claim your special offer! 

Struggling to Navigate the Complexities of Identifying a Research Problem?

• Quickly identify valuable research gaps.
• Develop clear and impactful problem statements.
• Align your research objectives with precise, answerable questions.
• Streamline your research planning process, saving time and reducing stress.

Ready to take control of your research journey? Get the Research Proposal Compass now and lay the groundwork for academic success. 

In conclusion, identifying a research problem is a foundational step in the academic research process that requires careful consideration and systematic approach. This guide has outlined the essential steps involved, from understanding the context and reviewing existing literature to formulating clear research questions. By adhering to these guidelines, researchers can ensure that their studies are grounded in a well-defined problem, enhancing the relevance and impact of their findings. It is crucial for scholars to approach this task with rigor and critical thinking to contribute meaningfully to the body of knowledge in their respective fields. 

Frequently Asked Questions

What is a research problem.

A research problem is a specific issue, inconsistency, or gap in knowledge that needs to be addressed through scientific inquiry. It forms the foundation of a research study, guiding the research questions, methodology, and analysis.

Why is identifying a research problem important?

Identifying a research problem is crucial as it determines the direction and scope of the study. It helps researchers focus their inquiry, formulate hypotheses, and contribute to the existing body of knowledge.

How do I identify a suitable research problem?

To identify a suitable research problem, start with a thorough literature review to understand existing research and identify gaps. Consult with academic mentors, and consider relevance, feasibility, and your own interests.

What are some common pitfalls in identifying a research problem?

Common pitfalls include choosing a problem that is too broad or too narrow, not aligning with existing literature, lack of originality, and failing to consider the practical implications and feasibility of the study.

Can technology help in identifying a research problem?

Yes, technology and data analytics can aid in identifying research problems by providing access to a vast amount of data, revealing patterns and trends that might not be visible otherwise. Tools like digital libraries and research databases are particularly useful.

How can I refine my research problem?

Refine your research problem by narrowing its focus, seeking feedback from peers and mentors, and continually reviewing and adjusting the problem statement based on new information and insights gained during preliminary research.

Discovering Statistics Using IBM SPSS Statistics: A Fun and Informative Guide

Unlocking the Power of Data: A Review of 'Essentials of Modern Business Statistics with Microsoft Excel'

Discovering Statistics Using SAS: A Comprehensive Review

Trending Topics for Your Thesis: What's Hot in 2024

How to Deal with a Total Lack of Motivation, Stress, and Anxiety When Finishing Your Master's Thesis

Mastering the First Step: How to Start Your Thesis with Confidence

Thesis Action Plan

• Blog Articles
• Affiliate Program
• Terms and Conditions
• Payment and Shipping Terms
• Privacy Policy
• Return Policy

© 2024 Research Rebels, All rights reserved.

Your cart is currently empty.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Indian Assoc Pediatr Surg
• v.24(1); Jan-Mar 2019

Formulation of Research Question – Stepwise Approach

Simmi k. ratan.

Department of Pediatric Surgery, Maulana Azad Medical College, New Delhi, India

1 Department of Community Medicine, North Delhi Municipal Corporation Medical College, New Delhi, India

2 Department of Pediatric Surgery, Batra Hospital and Research Centre, New Delhi, India

Formulation of research question (RQ) is an essentiality before starting any research. It aims to explore an existing uncertainty in an area of concern and points to a need for deliberate investigation. It is, therefore, pertinent to formulate a good RQ. The present paper aims to discuss the process of formulation of RQ with stepwise approach. The characteristics of good RQ are expressed by acronym “FINERMAPS” expanded as feasible, interesting, novel, ethical, relevant, manageable, appropriate, potential value, publishability, and systematic. A RQ can address different formats depending on the aspect to be evaluated. Based on this, there can be different types of RQ such as based on the existence of the phenomenon, description and classification, composition, relationship, comparative, and causality. To develop a RQ, one needs to begin by identifying the subject of interest and then do preliminary research on that subject. The researcher then defines what still needs to be known in that particular subject and assesses the implied questions. After narrowing the focus and scope of the research subject, researcher frames a RQ and then evaluates it. Thus, conception to formulation of RQ is very systematic process and has to be performed meticulously as research guided by such question can have wider impact in the field of social and health research by leading to formulation of policies for the benefit of larger population.

I NTRODUCTION

A good research question (RQ) forms backbone of a good research, which in turn is vital in unraveling mysteries of nature and giving insight into a problem.[ 1 , 2 , 3 , 4 ] RQ identifies the problem to be studied and guides to the methodology. It leads to building up of an appropriate hypothesis (Hs). Hence, RQ aims to explore an existing uncertainty in an area of concern and points to a need for deliberate investigation. A good RQ helps support a focused arguable thesis and construction of a logical argument. Hence, formulation of a good RQ is undoubtedly one of the first critical steps in the research process, especially in the field of social and health research, where the systematic generation of knowledge that can be used to promote, restore, maintain, and/or protect health of individuals and populations.[ 1 , 3 , 4 ] Basically, the research can be classified as action, applied, basic, clinical, empirical, administrative, theoretical, or qualitative or quantitative research, depending on its purpose.[ 2 ]

Research plays an important role in developing clinical practices and instituting new health policies. Hence, there is a need for a logical scientific approach as research has an important goal of generating new claims.[ 1 ]

C HARACTERISTICS OF G OOD R ESEARCH Q UESTION

“The most successful research topics are narrowly focused and carefully defined but are important parts of a broad-ranging, complex problem.”

A good RQ is an asset as it:

• Details the problem statement
• Further describes and refines the issue under study
• Adds focus to the problem statement
• Guides data collection and analysis
• Sets context of research.

Hence, while writing RQ, it is important to see if it is relevant to the existing time frame and conditions. For example, the impact of “odd-even” vehicle formula in decreasing the level of air particulate pollution in various districts of Delhi.

A good research is represented by acronym FINERMAPS[ 5 ]

Interesting.

• Appropriate
• Potential value and publishability
• Systematic.

Feasibility means that it is within the ability of the investigator to carry out. It should be backed by an appropriate number of subjects and methodology as well as time and funds to reach the conclusions. One needs to be realistic about the scope and scale of the project. One has to have access to the people, gadgets, documents, statistics, etc. One should be able to relate the concepts of the RQ to the observations, phenomena, indicators, or variables that one can access. One should be clear that the collection of data and the proceedings of project can be completed within the limited time and resources available to the investigator. Sometimes, a RQ appears feasible, but when fieldwork or study gets started, it proves otherwise. In this situation, it is important to write up the problems honestly and to reflect on what has been learned. One should try to discuss with more experienced colleagues or the supervisor so as to develop a contingency plan to anticipate possible problems while working on a RQ and find possible solutions in such situations.

This is essential that one has a real grounded interest in one's RQ and one can explore this and back it up with academic and intellectual debate. This interest will motivate one to keep going with RQ.

The question should not simply copy questions investigated by other workers but should have scope to be investigated. It may aim at confirming or refuting the already established findings, establish new facts, or find new aspects of the established facts. It should show imagination of the researcher. Above all, the question has to be simple and clear. The complexity of a question can frequently hide unclear thoughts and lead to a confused research process. A very elaborate RQ, or a question which is not differentiated into different parts, may hide concepts that are contradictory or not relevant. This needs to be clear and thought-through. Having one key question with several subcomponents will guide your research.

This is the foremost requirement of any RQ and is mandatory to get clearance from appropriate authorities before stating research on the question. Further, the RQ should be such that it minimizes the risk of harm to the participants in the research, protect the privacy and maintain their confidentiality, and provide the participants right to withdraw from research. It should also guide in avoiding deceptive practices in research.

The question should of academic and intellectual interest to people in the field you have chosen to study. The question preferably should arise from issues raised in the current situation, literature, or in practice. It should establish a clear purpose for the research in relation to the chosen field. For example, filling a gap in knowledge, analyzing academic assumptions or professional practice, monitoring a development in practice, comparing different approaches, or testing theories within a specific population are some of the relevant RQs.

Manageable (M): It has the similar essence as of feasibility but mainly means that the following research can be managed by the researcher.

Appropriate (A): RQ should be appropriate logically and scientifically for the community and institution.

Potential value and publishability (P): The study can make significant health impact in clinical and community practices. Therefore, research should aim for significant economic impact to reduce unnecessary or excessive costs. Furthermore, the proposed study should exist within a clinical, consumer, or policy-making context that is amenable to evidence-based change. Above all, a good RQ must address a topic that has clear implications for resolving important dilemmas in health and health-care decisions made by one or more stakeholder groups.

Systematic (S): Research is structured with specified steps to be taken in a specified sequence in accordance with the well-defined set of rules though it does not rule out creative thinking.

Example of RQ: Would the topical skin application of oil as a skin barrier reduces hypothermia in preterm infants? This question fulfills the criteria of a good RQ, that is, feasible, interesting, novel, ethical, and relevant.

Types of research question

A RQ can address different formats depending on the aspect to be evaluated.[ 6 ] For example:

• Existence: This is designed to uphold the existence of a particular phenomenon or to rule out rival explanation, for example, can neonates perceive pain?
• Description and classification: This type of question encompasses statement of uniqueness, for example, what are characteristics and types of neuropathic bladders?
• Composition: It calls for breakdown of whole into components, for example, what are stages of reflux nephropathy?
• Relationship: Evaluate relation between variables, for example, association between tumor rupture and recurrence rates in Wilm's tumor
• Descriptive—comparative: Expected that researcher will ensure that all is same between groups except issue in question, for example, Are germ cell tumors occurring in gonads more aggressive than those occurring in extragonadal sites?
• Causality: Does deletion of p53 leads to worse outcome in patients with neuroblastoma?
• Causality—comparative: Such questions frequently aim to see effect of two rival treatments, for example, does adding surgical resection improves survival rate outcome in children with neuroblastoma than with chemotherapy alone?
• Causality–Comparative interactions: Does immunotherapy leads to better survival outcome in neuroblastoma Stage IV S than with chemotherapy in the setting of adverse genetic profile than without it? (Does X cause more changes in Y than those caused by Z under certain condition and not under other conditions).

How to develop a research question

• Begin by identifying a broader subject of interest that lends itself to investigate, for example, hormone levels among hypospadias
• Do preliminary research on the general topic to find out what research has already been done and what literature already exists.[ 7 ] Therefore, one should begin with “information gaps” (What do you already know about the problem? For example, studies with results on testosterone levels among hypospadias
• What do you still need to know? (e.g., levels of other reproductive hormones among hypospadias)
• What are the implied questions: The need to know about a problem will lead to few implied questions. Each general question should lead to more specific questions (e.g., how hormone levels differ among isolated hypospadias with respect to that in normal population)
• Narrow the scope and focus of research (e.g., assessment of reproductive hormone levels among isolated hypospadias and hypospadias those with associated anomalies)
• Is RQ clear? With so much research available on any given topic, RQs must be as clear as possible in order to be effective in helping the writer direct his or her research
• Is the RQ focused? RQs must be specific enough to be well covered in the space available
• Is the RQ complex? RQs should not be answerable with a simple “yes” or “no” or by easily found facts. They should, instead, require both research and analysis on the part of the writer
• Is the RQ one that is of interest to the researcher and potentially useful to others? Is it a new issue or problem that needs to be solved or is it attempting to shed light on previously researched topic
• Is the RQ researchable? Consider the available time frame and the required resources. Is the methodology to conduct the research feasible?
• Is the RQ measurable and will the process produce data that can be supported or contradicted?
• Is the RQ too broad or too narrow?
• Create Hs: After formulating RQ, think where research is likely to be progressing? What kind of argument is likely to be made/supported? What would it mean if the research disputed the planned argument? At this step, one can well be on the way to have a focus for the research and construction of a thesis. Hs consists of more specific predictions about the nature and direction of the relationship between two variables. It is a predictive statement about the outcome of the research, dictate the method, and design of the research[ 1 ]
• Understand implications of your research: This is important for application: whether one achieves to fill gap in knowledge and how the results of the research have practical implications, for example, to develop health policies or improve educational policies.[ 1 , 8 ]

Brainstorm/Concept map for formulating research question

• First, identify what types of studies have been done in the past?
• Is there a unique area that is yet to be investigated or is there a particular question that may be worth replicating?
• Begin to narrow the topic by asking open-ended “how” and “why” questions
• Evaluate the question
• Develop a Hypothesis (Hs)
• Write down the RQ.

Writing down the research question

• State the question in your own words
• Write down the RQ as completely as possible.

For example, Evaluation of reproductive hormonal profile in children presenting with isolated hypospadias)

• Divide your question into concepts. Narrow to two or three concepts (reproductive hormonal profile, isolated hypospadias, compare with normal/not isolated hypospadias–implied)
• Specify the population to be studied (children with isolated hypospadias)
• Refer to the exposure or intervention to be investigated, if any
• Reflect the outcome of interest (hormonal profile).

Another example of a research question

Would the topical skin application of oil as a skin barrier reduces hypothermia in preterm infants? Apart from fulfilling the criteria of a good RQ, that is, feasible, interesting, novel, ethical, and relevant, it also details about the intervention done (topical skin application of oil), rationale of intervention (as a skin barrier), population to be studied (preterm infants), and outcome (reduces hypothermia).

Other important points to be heeded to while framing research question

• Make reference to a population when a relationship is expected among a certain type of subjects
• RQs and Hs should be made as specific as possible
• Avoid words or terms that do not add to the meaning of RQs and Hs
• Stick to what will be studied, not implications
• Name the variables in the order in which they occur/will be measured
• Avoid the words significant/”prove”
• Avoid using two different terms to refer to the same variable.

Some of the other problems and their possible solutions have been discussed in Table 1 .

Potential problems and solutions while making research question

G OING B EYOND F ORMULATION OF R ESEARCH Q UESTION–THE P ATH A HEAD

Once RQ is formulated, a Hs can be developed. Hs means transformation of a RQ into an operational analog.[ 1 ] It means a statement as to what prediction one makes about the phenomenon to be examined.[ 4 ] More often, for case–control trial, null Hs is generated which is later accepted or refuted.

A strong Hs should have following characteristics:

• Give insight into a RQ
• Are testable and measurable by the proposed experiments
• Have logical basis
• Follows the most likely outcome, not the exceptional outcome.

E XAMPLES OF R ESEARCH Q UESTION AND H YPOTHESIS

Research question-1.

• Does reduced gap between the two segments of the esophagus in patients of esophageal atresia reduces the mortality and morbidity of such patients?

Hypothesis-1

• Reduced gap between the two segments of the esophagus in patients of esophageal atresia reduces the mortality and morbidity of such patients
• In pediatric patients with esophageal atresia, gap of <2 cm between two segments of the esophagus and proper mobilization of proximal pouch reduces the morbidity and mortality among such patients.

Research question-2

• Does application of mitomycin C improves the outcome in patient of corrosive esophageal strictures?

Hypothesis-2

In patients aged 2–9 years with corrosive esophageal strictures, 34 applications of mitomycin C in dosage of 0.4 mg/ml for 5 min over a period of 6 months improve the outcome in terms of symptomatic and radiological relief. Some other examples of good and bad RQs have been shown in Table 2 .

Examples of few bad (left-hand side column) and few good (right-hand side) research questions

R ESEARCH Q UESTION AND S TUDY D ESIGN

RQ determines study design, for example, the question aimed to find the incidence of a disease in population will lead to conducting a survey; to find risk factors for a disease will need case–control study or a cohort study. RQ may also culminate into clinical trial.[ 9 , 10 ] For example, effect of administration of folic acid tablet in the perinatal period in decreasing incidence of neural tube defect. Accordingly, Hs is framed.

Appropriate statistical calculations are instituted to generate sample size. The subject inclusion, exclusion criteria and time frame of research are carefully defined. The detailed subject information sheet and pro forma are carefully defined. Moreover, research is set off few examples of research methodology guided by RQ:

• Incidence of anorectal malformations among adolescent females (hospital-based survey)
• Risk factors for the development of spontaneous pneumoperitoneum in pediatric patients (case–control design and cohort study)
• Effect of technique of extramucosal ureteric reimplantation without the creation of submucosal tunnel for the preservation of upper tract in bladder exstrophy (clinical trial).

The results of the research are then be available for wider applications for health and social life

C ONCLUSION

A good RQ needs thorough literature search and deep insight into the specific area/problem to be investigated. A RQ has to be focused yet simple. Research guided by such question can have wider impact in the field of social and health research by leading to formulation of policies for the benefit of larger population.

Financial support and sponsorship

Conflicts of interest.

There are no conflicts of interest.

R EFERENCES

Research Aims, Objectives & Questions

The “Golden Thread” Explained Simply (+ Examples)

By: David Phair (PhD) and Alexandra Shaeffer (PhD) | June 2022

The research aims , objectives and research questions (collectively called the “golden thread”) are arguably the most important thing you need to get right when you’re crafting a research proposal , dissertation or thesis . We receive questions almost every day about this “holy trinity” of research and there’s certainly a lot of confusion out there, so we’ve crafted this post to help you navigate your way through the fog.

Overview: The Golden Thread

• What is the golden thread
• What are research aims ( examples )
• What are research objectives ( examples )
• What are research questions ( examples )
• The importance of alignment in the golden thread

What is the “golden thread”?  

The golden thread simply refers to the collective research aims , research objectives , and research questions for any given project (i.e., a dissertation, thesis, or research paper ). These three elements are bundled together because it’s extremely important that they align with each other, and that the entire research project aligns with them.

Importantly, the golden thread needs to weave its way through the entirety of any research project , from start to end. In other words, it needs to be very clearly defined right at the beginning of the project (the topic ideation and proposal stage) and it needs to inform almost every decision throughout the rest of the project. For example, your research design and methodology will be heavily influenced by the golden thread (we’ll explain this in more detail later), as well as your literature review.

The research aims, objectives and research questions (the golden thread) define the focus and scope ( the delimitations ) of your research project. In other words, they help ringfence your dissertation or thesis to a relatively narrow domain, so that you can “go deep” and really dig into a specific problem or opportunity. They also help keep you on track , as they act as a litmus test for relevance. In other words, if you’re ever unsure whether to include something in your document, simply ask yourself the question, “does this contribute toward my research aims, objectives or questions?”. If it doesn’t, chances are you can drop it.

Alright, enough of the fluffy, conceptual stuff. Let’s get down to business and look at what exactly the research aims, objectives and questions are and outline a few examples to bring these concepts to life.

Research Aims: What are they?

Simply put, the research aim(s) is a statement that reflects the broad overarching goal (s) of the research project. Research aims are fairly high-level (low resolution) as they outline the general direction of the research and what it’s trying to achieve .

Research Aims: Examples  

True to the name, research aims usually start with the wording “this research aims to…”, “this research seeks to…”, and so on. For example:

“This research aims to explore employee experiences of digital transformation in retail HR.”   “This study sets out to assess the interaction between student support and self-care on well-being in engineering graduate students”  

As you can see, these research aims provide a high-level description of what the study is about and what it seeks to achieve. They’re not hyper-specific or action-oriented, but they’re clear about what the study’s focus is and what is being investigated.

Need a helping hand?

Research Objectives: What are they?

The research objectives take the research aims and make them more practical and actionable . In other words, the research objectives showcase the steps that the researcher will take to achieve the research aims.

The research objectives need to be far more specific (higher resolution) and actionable than the research aims. In fact, it’s always a good idea to craft your research objectives using the “SMART” criteria. In other words, they should be specific, measurable, achievable, relevant and time-bound”.

Research Objectives: Examples  

Let’s look at two examples of research objectives. We’ll stick with the topic and research aims we mentioned previously.  

For the digital transformation topic:

To observe the retail HR employees throughout the digital transformation. To assess employee perceptions of digital transformation in retail HR. To identify the barriers and facilitators of digital transformation in retail HR.

And for the student wellness topic:

To determine whether student self-care predicts the well-being score of engineering graduate students. To determine whether student support predicts the well-being score of engineering students. To assess the interaction between student self-care and student support when predicting well-being in engineering graduate students.

  As you can see, these research objectives clearly align with the previously mentioned research aims and effectively translate the low-resolution aims into (comparatively) higher-resolution objectives and action points . They give the research project a clear focus and present something that resembles a research-based “to-do” list.

Research Questions: What are they?

Finally, we arrive at the all-important research questions. The research questions are, as the name suggests, the key questions that your study will seek to answer . Simply put, they are the core purpose of your dissertation, thesis, or research project. You’ll present them at the beginning of your document (either in the introduction chapter or literature review chapter) and you’ll answer them at the end of your document (typically in the discussion and conclusion chapters).  

The research questions will be the driving force throughout the research process. For example, in the literature review chapter, you’ll assess the relevance of any given resource based on whether it helps you move towards answering your research questions. Similarly, your methodology and research design will be heavily influenced by the nature of your research questions. For instance, research questions that are exploratory in nature will usually make use of a qualitative approach, whereas questions that relate to measurement or relationship testing will make use of a quantitative approach.  

Let’s look at some examples of research questions to make this more tangible.

Research Questions: Examples  

Again, we’ll stick with the research aims and research objectives we mentioned previously.  

For the digital transformation topic (which would be qualitative in nature):

How do employees perceive digital transformation in retail HR? What are the barriers and facilitators of digital transformation in retail HR?  

And for the student wellness topic (which would be quantitative in nature):

Does student self-care predict the well-being scores of engineering graduate students? Does student support predict the well-being scores of engineering students? Do student self-care and student support interact when predicting well-being in engineering graduate students?  

You’ll probably notice that there’s quite a formulaic approach to this. In other words, the research questions are basically the research objectives “converted” into question format. While that is true most of the time, it’s not always the case. For example, the first research objective for the digital transformation topic was more or less a step on the path toward the other objectives, and as such, it didn’t warrant its own research question.  

So, don’t rush your research questions and sloppily reword your objectives as questions. Carefully think about what exactly you’re trying to achieve (i.e. your research aim) and the objectives you’ve set out, then craft a set of well-aligned research questions . Also, keep in mind that this can be a somewhat iterative process , where you go back and tweak research objectives and aims to ensure tight alignment throughout the golden thread.

The importance of strong alignment 

Alignment is the keyword here and we have to stress its importance . Simply put, you need to make sure that there is a very tight alignment between all three pieces of the golden thread. If your research aims and research questions don’t align, for example, your project will be pulling in different directions and will lack focus . This is a common problem students face and can cause many headaches (and tears), so be warned.

Take the time to carefully craft your research aims, objectives and research questions before you run off down the research path. Ideally, get your research supervisor/advisor to review and comment on your golden thread before you invest significant time into your project, and certainly before you start collecting data .  

Recap: The golden thread

In this post, we unpacked the golden thread of research, consisting of the research aims , research objectives and research questions . You can jump back to any section using the links below.

As always, feel free to leave a comment below – we always love to hear from you. Also, if you’re interested in 1-on-1 support, take a look at our private coaching service here.

Psst... there’s more!

This post was based on one of our popular Research Bootcamps . If you're working on a research project, you'll definitely want to check this out ...

41 Comments

Thank you very much for your great effort put. As an Undergraduate taking Demographic Research & Methodology, I’ve been trying so hard to understand clearly what is a Research Question, Research Aim and the Objectives in a research and the relationship between them etc. But as for now I’m thankful that you’ve solved my problem.

Well appreciated. This has helped me greatly in doing my dissertation.

An so delighted with this wonderful information thank you a lot.

so impressive i have benefited a lot looking forward to learn more on research.

I am very happy to have carefully gone through this well researched article.

Infact,I used to be phobia about anything research, because of my poor understanding of the concepts.

Now,I get to know that my research question is the same as my research objective(s) rephrased in question format.

I please I would need a follow up on the subject,as I intends to join the team of researchers. Thanks once again.

Thanks so much. This was really helpful.

I know you pepole have tried to break things into more understandable and easy format. And God bless you. Keep it up

i found this document so useful towards my study in research methods. thanks so much.

This is my 2nd read topic in your course and I should commend the simplified explanations of each part. I’m beginning to understand and absorb the use of each part of a dissertation/thesis. I’ll keep on reading your free course and might be able to avail the training course! Kudos!

Thank you! Better put that my lecture and helped to easily understand the basics which I feel often get brushed over when beginning dissertation work.

This is quite helpful. I like how the Golden thread has been explained and the needed alignment.

This is quite helpful. I really appreciate!

The article made it simple for researcher students to differentiate between three concepts.

Very innovative and educational in approach to conducting research.

I am very impressed with all these terminology, as I am a fresh student for post graduate, I am highly guided and I promised to continue making consultation when the need arise. Thanks a lot.

A very helpful piece. thanks, I really appreciate it .

Very well explained, and it might be helpful to many people like me.

Wish i had found this (and other) resource(s) at the beginning of my PhD journey… not in my writing up year… 😩 Anyways… just a quick question as i’m having some issues ordering my “golden thread”…. does it matter in what order you mention them? i.e., is it always first aims, then objectives, and finally the questions? or can you first mention the research questions and then the aims and objectives?

Thank you for a very simple explanation that builds upon the concepts in a very logical manner. Just prior to this, I read the research hypothesis article, which was equally very good. This met my primary objective.

My secondary objective was to understand the difference between research questions and research hypothesis, and in which context to use which one. However, I am still not clear on this. Can you kindly please guide?

In research, a research question is a clear and specific inquiry that the researcher wants to answer, while a research hypothesis is a tentative statement or prediction about the relationship between variables or the expected outcome of the study. Research questions are broader and guide the overall study, while hypotheses are specific and testable statements used in quantitative research. Research questions identify the problem, while hypotheses provide a focus for testing in the study.

Exactly what I need in this research journey, I look forward to more of your coaching videos.

This helped a lot. Thanks so much for the effort put into explaining it.

What data source in writing dissertation/Thesis requires?

What is data source covers when writing dessertation/thesis

This is quite useful thanks

I’m excited and thankful. I got so much value which will help me progress in my thesis.

where are the locations of the reserch statement, research objective and research question in a reserach paper? Can you write an ouline that defines their places in the researh paper?

Very helpful and important tips on Aims, Objectives and Questions.

Thank you so much for making research aim, research objectives and research question so clear. This will be helpful to me as i continue with my thesis.

Thanks much for this content. I learned a lot. And I am inspired to learn more. I am still struggling with my preparation for dissertation outline/proposal. But I consistently follow contents and tutorials and the new FB of GRAD Coach. Hope to really become confident in writing my dissertation and successfully defend it.

As a researcher and lecturer, I find splitting research goals into research aims, objectives, and questions is unnecessarily bureaucratic and confusing for students. For most biomedical research projects, including ‘real research’, 1-3 research questions will suffice (numbers may differ by discipline).

Awesome! Very important resources and presented in an informative way to easily understand the golden thread. Indeed, thank you so much.

Well explained

The blog article on research aims, objectives, and questions by Grad Coach is a clear and insightful guide that aligns with my experiences in academic research. The article effectively breaks down the often complex concepts of research aims and objectives, providing a straightforward and accessible explanation. Drawing from my own research endeavors, I appreciate the practical tips offered, such as the need for specificity and clarity when formulating research questions. The article serves as a valuable resource for students and researchers, offering a concise roadmap for crafting well-defined research goals and objectives. Whether you’re a novice or an experienced researcher, this article provides practical insights that contribute to the foundational aspects of a successful research endeavor.

A great thanks for you. it is really amazing explanation. I grasp a lot and one step up to research knowledge.

I really found these tips helpful. Thank you very much Grad Coach.

I found this article helpful. Thanks for sharing this.

thank you so much, the explanation and examples are really helpful

This is a well researched and superbly written article for learners of research methods at all levels in the research topic from conceptualization to research findings and conclusions. I highly recommend this material to university graduate students. As an instructor of advanced research methods for PhD students, I have confirmed that I was giving the right guidelines for the degree they are undertaking.

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

We Trust in Human Precision

20,000+ Professional Language Experts Ready to Help. Expertise in a variety of Niches.

API Solutions

• API Pricing
• Cost estimate
• Customer loyalty program
• Educational Discount
• Non-Profit Discount
• Green Initiative Discount1

Value-Driven Pricing

Unmatched expertise at affordable rates tailored for your needs. Our services empower you to boost your productivity.

• Special Discounts
• Enterprise transcription solutions
• Enterprise translation solutions
• Transcription/Caption API
• AI Transcription Proofreading API

Trusted by Global Leaders

GoTranscript is the chosen service for top media organizations, universities, and Fortune 50 companies.

GoTranscript

One of the Largest Online Transcription and Translation Agencies in the World. Founded in 2005.

Speaker 1: Hello everyone, this is Dr. Wallace. Have you ever wondered why so much time and emphasis is placed on understanding research questions, hypothesis statements, and variables in a research methodology course? Let's face it, when you look at these three items in a research proposal, they do not take up a lot of space. We spend more time discussing our research methodology, data collection and analysis, and prior research on our topic than we do on these three components in a research proposal. However, these three components are the foundation upon which any good research study is conducted. I want to take a few minutes and explain, at a high level, the role of each in a research proposal and study, and then show you how they are connected to each other. I think the best way to examine this subject is to begin with an example research topic. I can then demonstrate how these three components are connected by using this research topic as an example. Let's keep things easy and formulate a very simple topic. For the purposes of our lesson, I am going to say that I want to examine the effects of a thermostat on room temperatures. Now, with this research topic established, let's begin by looking at our three components. Okay, let's kick things off by looking at research questions. What is a research question? The research question is the way we succinctly define what we hope the research study will show us. We have identified a problem or issue that we think is important to study. Now, the research question specifically identifies what part of that issue or problem we hope to answer or address. As implied by its name, our research question is presented in the form of a question. The type of question we are asking is often driven by the type of study we are conducting in our research methodology. Using our example topic, a potential research question that we might ask is, how does changing the thermostat setting impact a room's temperature? Now that we have a research question, let's go take a look at the role of the hypothesis. So, what is a hypothesis? A hypothesis is a prediction of what we believe the study will find, or put another way, the answer to the research question. A hypothesis is an empirical statement that can be verified based upon observation or experience. It is testable to be true or false through the research study findings. You will also sometimes hear reference to a null hypothesis, but let's save that for another lesson. Now, let's return to our research topic example and create a hypothesis. The hypothesis for our example study might be, changing the temperature setting on a thermostat, up or down, will cause the room temperature, where the thermostat is located, to change in the same way. We have made a prediction of how we think the research study will answer our question. Okay, let's move to the final piece of the puzzle, variables. Variables, my Achilles heel during my doctoral study days. That's right, I'll be the first to admit that this simple concept gave me a hard time when I was a student. Let me take a shot at breaking down variables as simply as possible in one slide. There are two types of variables, dependent and independent, also referred to as DV or IV. Okay, so what's the difference? Let's see if I can make this as simple as possible. The dependent variable is what we watch to see if a change is occurring, while the independent variable is the thing we manipulate to influence a change in the dependent variable. I understand this can get confusing. Perhaps if we go back to our example research topic, things will become a bit clearer. In this research study, the thermostat setting is my independent variable, because it is the thing I have the power to change or alter. I am going to manipulate the independent variable to see if it creates a change in my dependent variable. The thermostat setting influences the room temperature, which is my dependent variable. The room temperature is my dependent variable because it is the thing I am watching to see if a change occurs when I alter the independent variable. Now, I manipulate my thermostat setting by moving it to a different temperature. I have just made a change that I predicted would influence my dependent variable. I will now watch my dependent variable, the room temperature, to see if a change occurs that matches the change I made to the thermostat setting, my independent variable. This is what I predicted with my hypothesis, so I want to watch how the room temperature, my dependent variable, is impacted when I make changes to the thermostat setting, my independent variable. Okay, now that we have a high-level understanding of these three components of a research proposal and study, let's see how they all work together. Let's connect those dots. The research question or questions that we create for the study specified what we want to answer or find out in the study. In other words, they put meaning to why our study is important. The hypothesis is our prediction of what we believe the study will find, or put another way, our predicted answer to the research questions. With the hypothesis, you are looking into your magic ball and telling everyone what you think the research study will find when all is said and done. In many cases, the research questions and hypothesis will be closely related, with the primary difference being that the hypothesis is presented as a statement, while the research questions are, well to put this plainly, formatted as a question. Now let's see how the variables complete our puzzle. The independent and dependent variables are the components of our study that we manipulate and watch for outcomes. Since our hypothesis made a prediction, it is only logical that our variables have to be included in the hypothesis. Our hypothesis is where we show the relationship between the independent and dependent variables. While this is the logical flow for building many research studies, I cannot overstress the fact that a research proposal is a constantly evolving process. Until we have locked down all components of the research proposal and are ready to conduct the study, it is entirely possible that you may go back and tweak or change any one of these components based upon new information you uncover or updates that you make to one of the three components. Don't let this worry you though, because this is a natural occurrence when working the kinks out of your research strategy. Let's conclude by summarizing what we have covered. The research question is our way of stating what we hope the study will find or help us learn. Our hypothesis is our prediction statement of what we think the research study findings will show. Finally, our variables are the aspects of the study that we manipulate and observe to determine whether our hypothesis was correct. Thank you for taking the time to view this lesson. I hope the information that I have presented helps clarify why understanding research questions, hypothesis, and variables is so important in research methodology and how these three work together to establish a strong foundation for a research study.

The Role of Research: To Learn, to Solve, to Inspire

• Click to share on LinkedIn (Opens in new window)
• Click to share on Facebook (Opens in new window)
• Click to share on X (Opens in new window)
• Click to email a link to a friend (Opens in new window)
• Click to print (Opens in new window)

Taking a peek into the unknown is the job of professors. With their research, they ask the big, knotty questions, the questions at the limits of human understanding for which answers are not easily found.

“It is challenging,” says Joanna Carey , associate professor of environmental science and the Debi and Andy Butler Term Chair. “Basically, our job is to figure out knowledge that nobody knows yet.”

Babson College may not be a large research institution, but its professors still produce a sizable amount of research in a wide range of fields, from medicine and the environment, to history and culture, to technology and innovation, to business and entrepreneurship. In this article, five Babson professors discuss their diverse work, giving a greater appreciation of the breadth and significance of research at the College.

BABSON MAGAZINE : Read the complete Summer 2024 issue .

Their research, along with that of their colleagues, flourishes in a supportive environment. The College helps fund research endeavors and trips to academic conferences, while giving professors the freedom to pursue their scholarly interests.

Alumni donors, meanwhile, have funded numerous term chairs, which allow professors to spend more time on their research, and professors at the tight-knit school frequently collaborate across disciplines. “We challenge each other to solve problems in different ways,” says Dessislava Pachamanova , professor of analytics and computational finance and the Zwerling Family Endowed Term Chair.

The end result is research that influences and inspires, that makes an impact around the globe, that helps us understand the world and our place in it.

Professors’ research also makes its way into the classroom. “The research they’re doing outside of the classroom informs and strengthens their approach within it,” says Babson President Stephen Spinelli Jr. MBA’92, PhD . “That value proposition enhances our academic rigor and ensures Babson remains at the forefront of emerging trends in entrepreneurship and beyond.”

‘I Have Always Been Fascinated by Nature.’

Imagine a stream trickling down a mountain, as it makes its way to a river, which widens as it reaches the sea. That water is majestic and immense, and it carries with it many things on its journey. “Every time I see a river, I think, ‘That’s a lot of material being moved,’ ” says Joanna Carey, associate professor of environmental science and the Debi and Andy Butler Term Chair.

One of those things in the water is silicon, the second most abundant element in Earth’s crust and a frequent subject of Carey’s research. Silicon moves from the crust, into plants, into rivers, and finally, into the ocean.

Following that path and examining watery places such as rivers, marshes, and estuaries, Carey’s research demonstrates how human activity is causing drastic changes in the amount of silicon as it cycles through the world. Those changes have a story to tell about land use, about the food chain, about carbon dioxide levels, and about our planet as it warms.

Consider the microscopic but mighty diatoms, for instance, an abundant alga that requires silicon to grow. What will the changing levels of silicon mean for these organisms responsible for more than 20% of the oxygen produced every year on Earth?

“They are really important,” Carey says. “Their importance on a global scale can’t be overestimated.”

For the last few years, Carey has led a team that created and examined the largest data set in the world on river silicon chemistry with funding from the National Center for Ecological Analysis and Synthesis. She’ll soon be looking at data from all seven continents for a project funded by the United States Geological Survey.

To be a scientist now, trying to bring a clearer understanding of climate change’s formidable and far-reaching impact, is to perform critical work.

“It is very fundamental science,” Carey says. “There is an urgency in what we are doing.”

Much is at stake, including the future of the students she teaches in the classroom. “This is an issue they will deal with their whole lives,” Carey says.

‘Research Has Always Felt Natural to Me.’

Technology evolves fast. For those in the workplace, that speed can feel overwhelming. “There is a lot of risk of people falling behind,” says Ruben Mancha , associate professor of information systems.

Those workers falling behind are a major concern of Mancha’s research. He looks at how organizations can adopt technology and transform how they operate in a responsible way, by considering the many human implications of the changes they’re implementing. “What is different about my framing is that responsibility,” he says. “It’s focused on the human side. It’s a people-first approach.”

New tech, such as an artificial intelligence tool like ChatGPT, actually can have a positive effect on employees’ workdays, taking on tedious tasks and freeing them to focus on more essential matters.

This benefit only works, though, if workers are confident using these tools. The line between those who are tech literate and those who are not is a stark one. “Those who can work with the technology will use it,” Mancha says. “Those who can’t will be replaced.”

In his research, Mancha examines two ways that organizations can not only guide employees through technological changes but also empower them. One way is by introducing them to low-code development platforms, which offer a much easier way to code, thus enabling many more employees to become developers. The second way is to launch a sustained and effective program for upskilling, creating a workforce that is competent and confident with tech.

Such measures do more than train an employee in the latest and greatest. They also change a workforce’s mindset. Give employees a new program they know how to use, and suddenly they have greater power to transform and innovate. “It changes how people see themselves and how they use technology,” Mancha says. “It’s about bringing the innovation culture to the enterprise.”

Mancha hopes decision makers in business will take his research to heart, and he’s excited to share it with students in the classroom. Before becoming a professor, he worked as a lab scientist in biotech. Research is something that comes naturally to him.

“It is my way of thinking,” he says.

‘I Like to Solve Problems.’

Unfortunately, the work of the International Committee of the Red Cross is seemingly never-ending. The essential organization operates in war zones—in Ukraine, in the Gaza Strip, and in conflict areas far removed from the world’s spotlight. “There are fires everywhere,” says Dessislava Pachamanova, professor of analytics and computational finance and the Zwerling Family Endowed Term Chair.

The work is not only relentless, but it is also costly and logistically challenging. Because of the alarming number of conflicts around the world in recent years, the organization faced a substantial funding shortfall. As a result, a team composed of Pachamanova, other researchers, and supply chain coordinators within the organization sought to determine how to best allocate medical supplies for where they need to go.

This was a tricky thing to figure out. Ship too many supplies, and costly medications may sit unused and expire. Ship too little, and people may not receive the critical, lifesaving supplies they need. For more than a year, Pachamanova and the group looked at the issue.

Ultimately, they developed an inventory management decision support system that was rolled out across a dozen medical distribution centers in Africa, the Middle East, and Ukraine in 2023. By reducing the inventory levels of medical supplies by nearly a quarter with virtually no negative effect on service, the system saved the Red Cross a significant amount of money while facilitating a collaborative planning process across the organization. “The Red Cross considers it a great success,” Pachamanova says.

This is exactly the type of result she is seeking with her research. “I am looking for impact. That is the main thing that drives me,” says Pachamanova, who has applied her expertise in optimization, analytics, machine learning, and simulation to fields as diverse as finance, logistics, and health care.

Pachamanova wants to incorporate her experience working with the Red Cross in a new Babson class she is designing. “I want to introduce students to this kind of experience,” she says, “where you go in, you understand the big problem, but identifying how to start a solution is very hard, and you won’t know where you’ll end up.”

‘I Like Asking Questions and Looking for Answers.’

A company is not an island. Its actions are not secluded. Rather, they ripple outward. A company’s supply chain, its partners, its manufacturing, its customers—all of these relationships, all of these connections, operate within one intertwined system that has an impact on communities and the environment.

The research of Sinan Erzurumlu , professor of innovation and operations management, concerns itself with these systems in which organizations operate. Lying at the intersection of business, society, and the environment, his work focuses on how companies can make decisions that are both sustainable and innovative.

When looking at a company’s actions in his research, Erzurumlu typically asks a direct question: Who does this benefit? “It could benefit a community. It could benefit the planet,” he says. “That perspective drives me a lot.”

The goal is to build a more sustainable future, but talking and researching about sustainability, such an immense, complex, and daunting challenge, is not easy. “I think sustainability is a human mindset problem,” Erzurumlu says. “It’s not just reducing carbon emissions. It’s about changing that mindset. We need to make that transition to a sustainable future. Convincing people to do that is a hard job.”

Large organizations also can’t simply transition into sustainable businesses overnight. Making integral changes is like trying to turn a cargo ship. “It’s a process,” Erzurumlu says. “They can’t make the turn immediately.”

In one recent research article he co-authored, Erzurumlu looked at the systems-thinking approach that three companies—retailers of household products, fashion, and beverages, respectively—took to sustainability. The companies, among other measures, sought to limit the water they used in their operations, reduce the use of hazardous chemicals, and collect waste to recycle and remake into new products.

He hopes other companies can learn from their efforts. He also hopes such research will give his students real-world insights about sustainability. “I think teaching is as important as being a researcher,” he says. “I see the classroom as an outlet for my research. Teaching about my research is an extension of my scholarly identity.”

‘Research Is a Great Opportunity to Better Understand Hard Problems.’

Hospitals are full of caring, smart people striving to deliver the best treatment possible. Helping them with that mission is what Wiljeana Glover tries to achieve in her research. 

To conduct her research, Glover likes to leave her desk and work on site, embedded with those on the front lines of health care. “When I can, I am physically going into hospitals, observing, getting to know clinicians,” says the Stephen C. and Carmella R. Kletjian Foundation Distinguished Professor of Global Healthcare Entrepreneurship. 

“That is fun for me and helps me understand how they do the work they do.”  

Glover often works with hospitals and clinics to understand how they identify and implement improvements, whether a new procedure or innovation. The goal is to make sure that these improvements support patients equitably. Equity in care, for people of color, for women, can remain elusive.  

Clinicians typically see her as a partner. “In some cases, they see me as part of their innovation or improvement team,” Glover says. “They appreciate the insights they are receiving along the way.” 

In a recent research article she co-authored, Glover looked at quality improvement efforts at medical centers and how those organizations can make a sustained commitment to addressing equity. Data measurement, team composition, and the need for ongoing actions were all examined. “How do we not think of equity as a one off?” Glover says. “How do we build in equitable practice? How does it become part of the way we do things?” 

Glover also serves as the founding faculty director of Babson’s Kerry Murphy Healey Center for Health Innovation and Entrepreneurship. In addition to studying equity, the center’s affiliated faculty conducts research on healthcare startups, the impact of artificial intelligence and analytics, and entrepreneurial training for clinicians and scientists. 

Glover praises the spirit of collaboration she sees in her fellow faculty members, who share ideas with one another and work together on research. “It is one of my favorite things about doing research at Babson,” she says. “It really is a part of the secret sauce of research here.”

Posted in Insights , Outcomes

Tagged Entrepreneurship Education , Faculty , Research , Babson Magazine

More from Babson Magazine »

Related Stories

Latest stories.

From Babson to Nasdaq: Karen Snow ’93 on Her Journey to the Top of Global Finance

Young Entrepreneurs Drive ‘Bright Future,’ GEM U.S. Report Shows

Babson College Climbs to No. 2 in Wall Street Journal’s Rankings of the Best Colleges in the U.S.

More babson.

Organizing Your Social Sciences Research Paper: The Research Problem/Question

• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Choosing a Title
• Making an Outline
• Paragraph Development
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Reading Research Effectively
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• What Is Scholarly vs. Popular?
• Qualitative Methods
• Quantitative Methods
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Annotated Bibliography
• Dealing with Nervousness
• Using Visual Aids
• Grading Someone Else's Paper
• Types of Structured Group Activities
• Group Project Survival Skills
• Multiple Book Review Essay
• Reviewing Collected Essays
• Writing a Case Study
• About Informed Consent
• Writing Field Notes
• Writing a Policy Memo
• Writing a Research Proposal
• Bibliography

A research problem is a definite or clear expression [statement] about an area of concern, a condition to be improved upon, a difficulty to be eliminated, or a troubling question that exists in scholarly literature, in theory, or within existing practice that points to a need for meaningful understanding and deliberate investigation. A research problem does not state how to do something, offer a vague or broad proposition, or present a value question.

Bryman, Alan. “The Research Question in Social Research: What is its Role?” International Journal of Social Research Methodology 10 (2007): 5-20; Guba, Egon G., and Yvonna S. Lincoln. “Competing Paradigms in Qualitative Research.” In Handbook of Qualitative Research . Norman K. Denzin and Yvonna S. Lincoln, editors. (Thousand Oaks, CA: Sage, 1994), pp. 105-117.

Importance of...

The purpose of a problem statement is to:

• Introduce the reader to the importance of the topic being studied . The reader is oriented to the significance of the study.
• Anchors the research questions, hypotheses, or assumptions to follow . It offers a concise statement about the purpose of your paper.
• Place the topic into a particular context that defines the parameters of what is to be investigated.
• Provide the framework for reporting the results and indicates what is probably necessary to conduct the study and explain how the findings will present this information.

In the social sciences, the research problem establishes the means by which you must answer the "So What?" question. This question refers to a research problem surviving the relevancy test [the quality of a measurement procedure that provides repeatability and accuracy]. Note that answering the "So What?" question requires a commitment on your part to not only show that you have reviewed the literature, but that you have thoroughly considered its significance and its implications applied to obtaining new knowledge or understanding.

To survive the "So What" question, problem statements should possess the following attributes:

• Clarity and precision [a well-written statement does not make sweeping generalizations and irresponsible pronouncements; it also does include unspecific determinates like "very" or "giant"],
• Demonstrate a researchable topic or issue [i.e., feasibility of conducting the study is based upon access to information that can be effectively acquired, gathered, interpreted, synthesized, and understood],
• Identification of what would be studied, while avoiding the use of value-laden words and terms,
• Identification of an overarching question or small set of questions accompanied by key factors or variables,
• Identification of key concepts and terms,
• Articulation of the study's boundaries or parameters or limitations,
• Some generalizability in regards to applicability and bringing results into general use,
• Conveyance of the study's importance, benefits, and justification [i.e., regardless of the type of research, it is important to demonstrate that the research is not trivial],
• Does not have unnecessary jargon or overly complex sentence constructions; and,
• Conveyance of more than the mere gathering of descriptive data providing only a snapshot of the issue or phenomenon under investigation.

Bryman, Alan. “The Research Question in Social Research: What is its Role?” International Journal of Social Research Methodology 10 (2007): 5-20; Castellanos, Susie. Critical Writing and Thinking . The Writing Center. Dean of the College. Brown University; Ellis, Timothy J. and Yair Levy Nova Framework of Problem-Based Research: A Guide for Novice Researchers on the Development of a Research-Worthy Problem. Informing Science: the International Journal of an Emerging Transdiscipline 11 (2008); Thesis and Purpose Statements . The Writer’s Handbook. Writing Center. University of Wisconsin, Madison; Thesis Statements . The Writing Center. University of North Carolina; Tips and Examples for Writing Thesis Statements . The Writing Lab and The OWL. Purdue University.  

Structure and Writing Style

I.  Types and Content

There are four general conceptualizations of a research problem in the social sciences:

• Casuist Research Problem -- this type of problem relates to the determination of right and wrong in questions of conduct or conscience by analyzing moral dilemmas through the application of general rules and the careful distinction of special cases.
• Difference Research Problem -- typically asks the question, “Is there a difference between two or more groups or treatments?” This type of problem statement is used when the researcher compares or contrasts two or more phenomena. This a common approach to defining a problem in the clinical social sciences or behavioral sciences.
• Descriptive Research Problem -- typically asks the question, "what is...?" with the underlying purpose to describe the significance of a situation, state, or existence of a specific phenomenon. This problem is often associated with revealing hidden or understudied issues.
• Relational Research Problem -- suggests a relationship of some sort between two or more variables to be investigated. The underlying purpose is to investigate specific qualities or characteristics that may be connected in some way.

A problem statement in the social sciences should contain :

• A lead-in that helps ensure the reader will maintain interest over the study,
• A declaration of originality [e.g., mentioning a knowledge void or a lack of clarity about a topic that will be revealed in the literature review],
• An indication of the central focus of the study [establishing the boundaries of analysis], and
• An explanation of the study's significance or the benefits to be derived from investigating the research problem.

II.  Sources of Problems for Investigation

The identification of a problem to study can be challenging, not because there's a lack of issues that could be investigated, but due to the challenge of formulating an academically relevant and researchable problem which is unique and does not simply duplicate the work of others. To facilitate how you might select a problem from which to build a research study, consider these sources of inspiration:

Deductions from Theory This relates to deductions made from social philosophy or generalizations embodied in life and in society that the researcher is familiar with. These deductions from human behavior are then placed within an empirical frame of reference through research. From a theory, the researcher can formulate a research problem or hypothesis stating the expected findings in certain empirical situations. The research asks the question: “What relationship between variables will be observed if theory aptly summarizes the state of affairs?” One can then design and carry out a systematic investigation to assess whether empirical data confirm or reject the hypothesis, and hence, the theory.

Interdisciplinary Perspectives Identifying a problem that forms the basis for a research study can come from academic movements and scholarship originating in disciplines outside of your primary area of study. This can be an intellectually stimulating exercise. A review of pertinent literature should include examining research from related disciplines that can reveal new avenues of exploration and analysis. An interdisciplinary approach to selecting a research problem offers an opportunity to construct a more comprehensive understanding of a very complex issue that any single discipline may be able to provide.

Interviewing Practitioners The identification of research problems about particular topics can arise from formal interviews or informal discussions with practitioners who provide insight into new directions for future research and how to make research findings more relevant to practice. Discussions with experts in the field, such as, teachers, social workers, health care providers, lawyers, business leaders, etc., offers the chance to identify practical, “real world” problems that may be understudied or ignored within academic circles. This approach also provides some practical knowledge which may help in the process of designing and conducting your study.

Personal Experience Don't undervalue your everyday experiences or encounters as worthwhile problems for investigation. Think critically about your own experiences and/or frustrations with an issue facing society, your community, your neighborhood, your family, or your personal life. This can be derived, for example, from deliberate observations of certain relationships for which there is no clear explanation or witnessing an event that appears harmful to a person or group or that is out of the ordinary.

Relevant Literature The selection of a research problem can be derived from a thorough review of pertinent research associated with your overall area of interest. This may reveal where gaps exist in understanding a topic or where an issue has been understudied. Research may be conducted to: 1) fill such gaps in knowledge; 2) evaluate if the methodologies employed in prior studies can be adapted to solve other problems; or, 3) determine if a similar study could be conducted in a different subject area or applied in a different context or to different study sample [i.e., different setting or different group of people].Also, authors frequently conclude their studies by noting implications for further research; read the conclusion of pertinent studies because statements about further research can be a valuable source for identifying new problems to investigate. The fact that a researcher has identified a topic worthy of further exploration validates the fact it is worth pursuing.

III.  What Makes a Good Research Statement?

A good problem statement begins by introducing the broad area in which your research is centered, gradually leading the reader to the more specific issues you are investigating. The statement need not be lengthy, but a good research problem should incorporate the following features:

1.  Compelling Topic The problem chosen should be one that motivates you to address it but simple curiosity is not a good enough reason to pursue a research study because this does not indicate significance. The problem that you choose to explore must be important to you, but it must also be viewed as important by your readers and to a the larger academic and/or social community that could be impacted by the results of your study. 2.  Supports Multiple Perspectives The problem must be phrased in a way that avoids dichotomies and instead supports the generation and exploration of multiple perspectives. A general rule of thumb in the social sciences is that a good research problem is one that would generate a variety of viewpoints from a composite audience made up of reasonable people. 3.  Researchability This isn't a real word but it represents an important aspect of creating a good research statement. It seems a bit obvious, but you don't want to find yourself in the midst of investigating a complex research project and realize that you don't have enough prior research to draw from for your analysis. There's nothing inherently wrong with original research, but you must choose research problems that can be supported, in some way, by the resources available to you. If you are not sure if something is researchable, don't assume that it isn't if you don't find information right away--seek help from a librarian !

NOTE:   Do not confuse a research problem with a research topic. A topic is something to read and obtain information about, whereas a problem is something to be solved or framed as a question raised for inquiry, consideration, or solution , or explained as a source of perplexity, distress, or vexation . In short, a research topic is something to be understood; a research problem is something that needs to be investigated.

IV.  Asking Analytical Questions about the Research Problem

Research problems in the social and behavioral sciences are often analyzed around critical questions that must be investigated. These questions can be explicitly listed in the introduction [i.e., "This study addresses three research questions about women's psychological recovery from domestic abuse in multi-generational home settings..."], or, the questions are implied in the text as specific areas of study related to the research problem. Explicitly listing your research questions at the end of your introduction can help in designing a clear roadmap of what you plan to address in your study, whereas, implicitly integrating them into the text of the introduction allows you to create a more compelling narrative around the key issues under investigation. Either approach is appropriate.

The number of questions you attempt to address should be based on the complexity of the problem you are investigating and what areas of inquiry you find most critical to study. Practical considerations, such as, the length of the paper you are writing or the availability of resources to analyze the issue can also factor in how many questions to ask. In general, however, there should be no more than four research questions underpinning a single research problem.

Given this, well-developed analytical questions can focus on any of the following:

• Highlights a genuine dilemma, area of ambiguity, or point of confusion about a topic open to interpretation by your readers;
• Yields an answer that is unexpected and not obvious rather than inevitable and self-evident;
• Provokes meaningful thought or discussion;
• Raises the visibility of the key ideas or concepts that may be understudied or hidden;
• Suggests the need for complex analysis or argument rather than a basic description or summary; and,
• Offers a specific path of inquiry that avoids eliciting generalizations about the problem.

NOTE:   Questions of how and why concerning a research problem often require more analysis than questions about who, what, where, and when. You should still ask yourself these latter questions, however. Thinking introspectively about the who, what, where, and when of a research problem can help ensure that you have thoroughly considered all aspects of the problem under investigation.

V.  Mistakes to Avoid

Beware of circular reasoning! Do not state that the research problem as simply the absence of the thing you are suggesting. For example, if you propose the following, "The problem in this community is that there is no hospital," this only leads to a research problem where:

• The need is for a hospital
• The objective is to create a hospital
• The method is to plan for building a hospital, and
• The evaluation is to measure if there is a hospital or not.

This is an example of a research problem that fails the "So What?" test . In this example, the problem does not reveal the relevance of why you are investigating the fact there is no hospital in the community [e.g., there's a hospital in the community ten miles away]; it does not elucidate the significance of why one should study the fact there is no hospital in the community [e.g., that hospital in the community ten miles away has no emergency room]; the research problem does not offer an intellectual pathway towards adding new knowledge or clarifying prior knowledge [e.g., the county in which there is no hospital already conducted a study about the need for a hospital]; and, the problem does not offer meaningful outcomes that lead to recommendations that can be generalized for other situations or that could suggest areas for further research [e.g., the challenges of building a new hospital serves as a case study for other communities].

Alvesson, Mats and Jörgen Sandberg. “Generating Research Questions Through Problematization.” Academy of Management Review 36 (April 2011): 247-271 ; Choosing and Refining Topics . Writing@CSU. Colorado State University; Ellis, Timothy J. and Yair Levy Nova. "Framework of Problem-Based Research: A Guide for Novice Researchers on the Development of a Research-Worthy Problem." Informing Science: the International Journal of an Emerging Transdiscipline 11 (2008); How to Write a Research Question . The Writing Center. George Mason University; Invention: Developing a Thesis Statement . The Reading/Writing Center. Hunter College; Problem Statements PowerPoint Presentation . The Writing Lab and The OWL. Purdue University; Procter, Margaret. Using Thesis Statements . University College Writing Centre. University of Toronto; Trochim, William M.K. Problem Formulation . Research Methods Knowledge Base. 2006; Thesis and Purpose Statements . The Writer’s Handbook. Writing Center. University of Wisconsin, Madison; Thesis Statements . The Writing Center. University of North Carolina; Tips and Examples for Writing Thesis Statements . The Writing Lab and The OWL. Purdue University; Walk, Kerry. Asking an Analytical Question . [Class handout or worksheet]. Princeton University; White, Patrick. Developing Research Questions: A Guide for Social Scientists . New York: Palgrave McMillan, 2009.

• << Previous: Background Information
• Next: Theoretical Framework >>
• Last Updated: Jan 17, 2023 10:50 AM
• URL: https://libguides.pointloma.edu/ResearchPaper

Numbers, Facts and Trends Shaping Your World

Read our research on:

Full Topic List

Regions & Countries

• Publications
• Our Methods
• Short Reads
• Tools & Resources

Read Our Research On:

Key things to know about U.S. election polling in 2024

Confidence in U.S. public opinion polling was shaken by errors in 2016 and 2020. In both years’ general elections, many polls underestimated the strength of Republican candidates, including Donald Trump. These errors laid bare some real limitations of polling.

In the midterms that followed those elections, polling performed better . But many Americans remain skeptical that it can paint an accurate portrait of the public’s political preferences.

Restoring people’s confidence in polling is an important goal, because robust and independent public polling has a critical role to play in a democratic society. It gathers and publishes information about the well-being of the public and about citizens’ views on major issues. And it provides an important counterweight to people in power, or those seeking power, when they make claims about “what the people want.”

The challenges facing polling are undeniable. In addition to the longstanding issues of rising nonresponse and cost, summer 2024 brought extraordinary events that transformed the presidential race . The good news is that people with deep knowledge of polling are working hard to fix the problems exposed in 2016 and 2020, experimenting with more data sources and interview approaches than ever before. Still, polls are more useful to the public if people have realistic expectations about what surveys can do well – and what they cannot.

With that in mind, here are some key points to know about polling heading into this year’s presidential election.

Probability sampling (or “random sampling”). This refers to a polling method in which survey participants are recruited using random sampling from a database or list that includes nearly everyone in the population. The pollster selects the sample. The survey is not open for anyone who wants to sign up.

Online opt-in polling (or “nonprobability sampling”). These polls are recruited using a variety of methods that are sometimes referred to as “convenience sampling.” Respondents come from a variety of online sources such as ads on social media or search engines, websites offering rewards in exchange for survey participation, or self-enrollment. Unlike surveys with probability samples, people can volunteer to participate in opt-in surveys.

Nonresponse and nonresponse bias. Nonresponse is when someone sampled for a survey does not participate. Nonresponse bias occurs when the pattern of nonresponse leads to error in a poll estimate. For example, college graduates are more likely than those without a degree to participate in surveys, leading to the potential that the share of college graduates in the resulting sample will be too high.

Mode of interview. This refers to the format in which respondents are presented with and respond to survey questions. The most common modes are online, live telephone, text message and paper. Some polls use more than one mode.

Weighting. This is a statistical procedure pollsters perform to make their survey align with the broader population on key characteristics like age, race, etc. For example, if a survey has too many college graduates compared with their share in the population, people without a college degree are “weighted up” to match the proper share.

How are election polls being conducted?

Pollsters are making changes in response to the problems in previous elections. As a result, polling is different today than in 2016. Most U.S. polling organizations that conducted and publicly released national surveys in both 2016 and 2022 (61%) used methods in 2022 that differed from what they used in 2016 . And change has continued since 2022.

One change is that the number of active polling organizations has grown significantly, indicating that there are fewer barriers to entry into the polling field. The number of organizations that conduct national election polls more than doubled between 2000 and 2022.

This growth has been driven largely by pollsters using inexpensive opt-in sampling methods. But previous Pew Research Center analyses have demonstrated how surveys that use nonprobability sampling may have errors twice as large , on average, as those that use probability sampling.

The second change is that many of the more prominent polling organizations that use probability sampling – including Pew Research Center – have shifted from conducting polls primarily by telephone to using online methods, or some combination of online, mail and telephone. The result is that polling methodologies are far more diverse now than in the past.

(For more about how public opinion polling works, including a chapter on election polls, read our short online course on public opinion polling basics .)

All good polling relies on statistical adjustment called “weighting,” which makes sure that the survey sample aligns with the broader population on key characteristics. Historically, public opinion researchers have adjusted their data using a core set of demographic variables to correct imbalances between the survey sample and the population.

But there is a growing realization among survey researchers that weighting a poll on just a few variables like age, race and gender is insufficient for getting accurate results. Some groups of people – such as older adults and college graduates – are more likely to take surveys, which can lead to errors that are too sizable for a simple three- or four-variable adjustment to work well. Adjusting on more variables produces more accurate results, according to Center studies in 2016 and 2018 .

A number of pollsters have taken this lesson to heart. For example, recent high-quality polls by Gallup and The New York Times/Siena College adjusted on eight and 12 variables, respectively. Our own polls typically adjust on 12 variables . In a perfect world, it wouldn’t be necessary to have that much intervention by the pollster. But the real world of survey research is not perfect.

Predicting who will vote is critical – and difficult. Preelection polls face one crucial challenge that routine opinion polls do not: determining who of the people surveyed will actually cast a ballot.

Roughly a third of eligible Americans do not vote in presidential elections , despite the enormous attention paid to these contests. Determining who will abstain is difficult because people can’t perfectly predict their future behavior – and because many people feel social pressure to say they’ll vote even if it’s unlikely.

No one knows the profile of voters ahead of Election Day. We can’t know for sure whether young people will turn out in greater numbers than usual, or whether key racial or ethnic groups will do so. This means pollsters are left to make educated guesses about turnout, often using a mix of historical data and current measures of voting enthusiasm. This is very different from routine opinion polls, which mostly do not ask about people’s future intentions.

When major news breaks, a poll’s timing can matter. Public opinion on most issues is remarkably stable, so you don’t necessarily need a recent poll about an issue to get a sense of what people think about it. But dramatic events can and do change public opinion , especially when people are first learning about a new topic. For example, polls this summer saw notable changes in voter attitudes following Joe Biden’s withdrawal from the presidential race. Polls taken immediately after a major event may pick up a shift in public opinion, but those shifts are sometimes short-lived. Polls fielded weeks or months later are what allow us to see whether an event has had a long-term impact on the public’s psyche.

How accurate are polls?

The answer to this question depends on what you want polls to do. Polls are used for all kinds of purposes in addition to showing who’s ahead and who’s behind in a campaign. Fair or not, however, the accuracy of election polling is usually judged by how closely the polls matched the outcome of the election.

By this standard, polling in 2016 and 2020 performed poorly. In both years, state polling was characterized by serious errors. National polling did reasonably well in 2016 but faltered in 2020.

In 2020, a post-election review of polling by the American Association for Public Opinion Research (AAPOR) found that “the 2020 polls featured polling error of an unusual magnitude: It was the highest in 40 years for the national popular vote and the highest in at least 20 years for state-level estimates of the vote in presidential, senatorial, and gubernatorial contests.”

How big were the errors? Polls conducted in the last two weeks before the election suggested that Biden’s margin over Trump was nearly twice as large as it ended up being in the final national vote tally.

Errors of this size make it difficult to be confident about who is leading if the election is closely contested, as many U.S. elections are .

Pollsters are rightly working to improve the accuracy of their polls. But even an error of 4 or 5 percentage points isn’t too concerning if the purpose of the poll is to describe whether the public has favorable or unfavorable opinions about candidates , or to show which issues matter to which voters. And on questions that gauge where people stand on issues, we usually want to know broadly where the public stands. We don’t necessarily need to know the precise share of Americans who say, for example, that climate change is mostly caused by human activity. Even judged by its performance in recent elections, polling can still provide a faithful picture of public sentiment on the important issues of the day.

The 2022 midterms saw generally accurate polling, despite a wave of partisan polls predicting a broad Republican victory. In fact, FiveThirtyEight found that “polls were more accurate in 2022 than in any cycle since at least 1998, with almost no bias toward either party.” Moreover, a handful of contrarian polls that predicted a 2022 “red wave” largely washed out when the votes were tallied. In sum, if we focus on polling in the most recent national election, there’s plenty of reason to be encouraged.

Compared with other elections in the past 20 years, polls have been less accurate when Donald Trump is on the ballot. Preelection surveys suffered from large errors – especially at the state level – in 2016 and 2020, when Trump was standing for election. But they performed reasonably well in the 2018 and 2022 midterms, when he was not.

During the 2016 campaign, observers speculated about the possibility that Trump supporters might be less willing to express their support to a pollster – a phenomenon sometimes described as the “shy Trump effect.” But a committee of polling experts evaluated five different tests of the “shy Trump” theory and turned up little to no evidence for each one . Later, Pew Research Center and, in a separate test, a researcher from Yale also found little to no evidence in support of the claim.

Instead, two other explanations are more likely. One is about the difficulty of estimating who will turn out to vote. Research has found that Trump is popular among people who tend to sit out midterms but turn out for him in presidential election years. Since pollsters often use past turnout to predict who will vote, it can be difficult to anticipate when irregular voters will actually show up.

The other explanation is that Republicans in the Trump era have become a little less likely than Democrats to participate in polls . Pollsters call this “partisan nonresponse bias.” Surprisingly, polls historically have not shown any particular pattern of favoring one side or the other. The errors that favored Democratic candidates in the past eight years may be a result of the growth of political polarization, along with declining trust among conservatives in news organizations and other institutions that conduct polls.

Whatever the cause, the fact that Trump is again the nominee of the Republican Party means that pollsters must be especially careful to make sure all segments of the population are properly represented in surveys.

The real margin of error is often about double the one reported. A typical election poll sample of about 1,000 people has a margin of sampling error that’s about plus or minus 3 percentage points. That number expresses the uncertainty that results from taking a sample of the population rather than interviewing everyone . Random samples are likely to differ a little from the population just by chance, in the same way that the quality of your hand in a card game varies from one deal to the next.

The problem is that sampling error is not the only kind of error that affects a poll. Those other kinds of error, in fact, can be as large or larger than sampling error. Consequently, the reported margin of error can lead people to think that polls are more accurate than they really are.

There are three other, equally important sources of error in polling: noncoverage error , where not all the target population has a chance of being sampled; nonresponse error, where certain groups of people may be less likely to participate; and measurement error, where people may not properly understand the questions or misreport their opinions. Not only does the margin of error fail to account for those other sources of potential error, putting a number only on sampling error implies to the public that other kinds of error do not exist.

Several recent studies show that the average total error in a poll estimate may be closer to twice as large as that implied by a typical margin of sampling error. This hidden error underscores the fact that polls may not be precise enough to call the winner in a close election.

Other important things to remember

Transparency in how a poll was conducted is associated with better accuracy . The polling industry has several platforms and initiatives aimed at promoting transparency in survey methodology. These include AAPOR’s transparency initiative and the Roper Center archive . Polling organizations that participate in these organizations have less error, on average, than those that don’t participate, an analysis by FiveThirtyEight found .

Participation in these transparency efforts does not guarantee that a poll is rigorous, but it is undoubtedly a positive signal. Transparency in polling means disclosing essential information, including the poll’s sponsor, the data collection firm, where and how participants were selected, modes of interview, field dates, sample size, question wording, and weighting procedures.

There is evidence that when the public is told that a candidate is extremely likely to win, some people may be less likely to vote . Following the 2016 election, many people wondered whether the pervasive forecasts that seemed to all but guarantee a Hillary Clinton victory – two modelers put her chances at 99% – led some would-be voters to conclude that the race was effectively over and that their vote would not make a difference. There is scientific research to back up that claim: A team of researchers found experimental evidence that when people have high confidence that one candidate will win, they are less likely to vote. This helps explain why some polling analysts say elections should be covered using traditional polling estimates and margins of error rather than speculative win probabilities (also known as “probabilistic forecasts”).

National polls tell us what the entire public thinks about the presidential candidates, but the outcome of the election is determined state by state in the Electoral College . The 2000 and 2016 presidential elections demonstrated a difficult truth: The candidate with the largest share of support among all voters in the United States sometimes loses the election. In those two elections, the national popular vote winners (Al Gore and Hillary Clinton) lost the election in the Electoral College (to George W. Bush and Donald Trump). In recent years, analysts have shown that Republican candidates do somewhat better in the Electoral College than in the popular vote because every state gets three electoral votes regardless of population – and many less-populated states are rural and more Republican.

For some, this raises the question: What is the use of national polls if they don’t tell us who is likely to win the presidency? In fact, national polls try to gauge the opinions of all Americans, regardless of whether they live in a battleground state like Pennsylvania, a reliably red state like Idaho or a reliably blue state like Rhode Island. In short, national polls tell us what the entire citizenry is thinking. Polls that focus only on the competitive states run the risk of giving too little attention to the needs and views of the vast majority of Americans who live in uncompetitive states – about 80%.

Fortunately, this is not how most pollsters view the world . As the noted political scientist Sidney Verba explained, “Surveys produce just what democracy is supposed to produce – equal representation of all citizens.”

• Survey Methods
• Trust, Facts & Democracy
• Voter Files

Scott Keeter is a senior survey advisor at Pew Research Center .

Courtney Kennedy is Vice President of Methods and Innovation at Pew Research Center .

How do people in the U.S. take Pew Research Center surveys, anyway?

How public polling has changed in the 21st century, what 2020’s election poll errors tell us about the accuracy of issue polling, a field guide to polling: election 2020 edition, methods 101: how is polling done around the world, most popular.

901 E St. NW, Suite 300 Washington, DC 20004 USA (+1) 202-419-4300 | Main (+1) 202-857-8562 | Fax (+1) 202-419-4372 |  Media Inquiries

Research Topics

• Email Newsletters

ABOUT PEW RESEARCH CENTER  Pew Research Center is a nonpartisan fact tank that informs the public about the issues, attitudes and trends shaping the world. It conducts public opinion polling, demographic research, media content analysis and other empirical social science research. Pew Research Center does not take policy positions. It is a subsidiary of  The Pew Charitable Trusts .

© 2024 Pew Research Center

Developing Surveys on Questionable Research Practices: Four Challenging Design Problems

• Open access
• Published: 02 September 2024

Cite this article

You have full access to this open access article

• Christian Berggren   ORCID: orcid.org/0000-0002-4233-5138 1 ,
• Bengt Gerdin   ORCID: orcid.org/0000-0001-8360-5387 2 &
• Solmaz Filiz Karabag   ORCID: orcid.org/0000-0002-3863-1073 1 , 3  

9 Altmetric

Explore all metrics

The exposure of scientific scandals and the increase of dubious research practices have generated a stream of studies on Questionable Research Practices (QRPs), such as failure to acknowledge co-authors, selective presentation of findings, or removal of data not supporting desired outcomes. In contrast to high-profile fraud cases, QRPs can be investigated using quantitative, survey-based methods. However, several design issues remain to be solved. This paper starts with a review of four problems in the QRP research: the problem of precision and prevalence, the problem of social desirability bias, the problem of incomplete coverage, and the problem of controversiality, sensitivity and missing responses. Various ways to handle these problems are discussed based on a case study of the design of a large, cross-field QRP survey in the social and medical sciences in Sweden. The paper describes the key steps in the design process, including technical and cognitive testing and repeated test versions to arrive at reliable survey items on the prevalence of QRPs and hypothesized associated factors in the organizational and normative environments. Partial solutions to the four problems are assessed, unresolved issues are discussed, and tradeoffs that resist simple solutions are articulated. The paper ends with a call for systematic comparisons of survey designs and item quality to build a much-needed cumulative knowledge trajectory in the field of integrity studies.

Similar content being viewed by others

Design, Run, and Interpret Survey-Based Research in the Fields of Academic Integrity and Misconduct

Dirty data: the effects of screening respondents who provide low-quality data in survey research, explore related subjects.

• Medical Ethics

Avoid common mistakes on your manuscript.

Introduction

The public revelations of research fraud and non-replicable findings (Berggren & Karabag, 2019 ; Levelt et al., 2012 ; Nosek et al., 2022 ) have created a lively interest in studying research integrity. Most studies in this field tend to focus on questionable research practices, QRPs, rather than blatant fraud, which is less common and hard to study with rigorous methods (Butler et al., 2017 ). Despite the significant contributions of this research about the incidence of QRPs in various countries and contexts, several issues still need to be addressed regarding the challenges of designing precise and valid survey instruments and achieving satisfactory response rates in this sensitive area. While studies in management (Hinkin, 1998 ; Lietz, 2010 ), behavioral sciences, psychology (Breakwell et al., 2020 ), sociology (Brenner, 2020 ), and education (Hill et al., 2022 ) have provided guidelines to design surveys, they rarely discuss how to develop, test, and use surveys targeting sensitive and controversial issues such as organizational or individual corruption (Lin & Yu, 2020 ), fraud (Lawlor et al., 2021 ), and misconduct. The aim of this study is to contribute to a systematic discussion of challenges facing survey designers in these areas and, by way of a detailed case study, highlight alternative ways to increase participation and reliability of surveys focusing on questionable research practices, scientific norms, and organizational climate.

The following section starts with a literature-based review of four important problems:

the lack of conceptual consensus and precise measurements,

the problem of social desirability bias.

the difficulty of covering both quantitative and qualitative research fields.

the problem of controversiality and sensitivity.

Section 3 presents an in-depth case study of developing and implementing a survey on QRPs in the social and medical sciences in Sweden 2018–2021, designed to target these problems. Its first results were presented in this journal (Karabag et al., 2024 ). The section also describes the development process and the survey content and highlights the general design challenges. Section 4 returns to the four problems by discussing partial solutions, difficult tradeoffs, and remaining issues.

Four Design Problems in the Study of Questionable Research Practices

Extant QRP studies have generated an impressive body of knowledge regarding the occurrence and complexities of questionable practices, their increasing trend in several academic fields, and the difficulty of mitigating them with conventional interventions such as ethics courses and espousal of integrity policies (Gopalakrishna et al., 2022 ; Karabag et al., 2024 ; Necker, 2014 ). However, investigations on the prevalence of QRPs have so far lacked systematic problem analysis. Below, four main problems are discussed.

The Problem of Conceptual Clarity and Measurement Precision

Studies of QRP prevalence in the literature exhibit high levels of questionable behaviors but also considerable variation in their estimates. This is illustrated in the examples below:

“42% hade collected more data after inspecting whether results were statistically significant… and 51% had reported an unexpected finding as though it had been hypothesized from the start (HARKing)”( Fraser et al., 2018 , p. 1) , “51 , 3% of respondents engaging frequently in at least one QRP” ( Gopalakrishna et al., 2022 , p. 1) , “…one third of the researchers stated that for the express purpose of supporting hypotheses with statistical significance they engaged in post hoc exclusion of data” ( Banks et al., 2016 , p. 10).

On a general level, QRPs constitute deviations from the responsible conduct of research, that are not severe enough to be defined as fraud and fabrication (Steneck, 2006 ). Within these borders, there is no conceptual consensus regarding specific forms of QRPs (Bruton et al., 2020 ; Xie et al., 2021 ). This has resulted in a considerable variation in prevalence estimates (Agnoli et al., 2017 ; Artino et al. Jr, 2019 ; Fiedler & Schwarz, 2016 ). Many studies emphasize the role of intentionality, implying a purpose to support a specific assertion with biased evidence (Banks et al., 2016 ). This tends to be backed by reports of malpractices in quantitative research, such as p-hacking or HARKing, where unexpected findings or results from an exploratory analysis are reported as having been predicted from the start (Andrade, 2021 ). Other QRP studies, however, build on another, often implicit conceptual definition and include practices that could instead be defined as sloppy or under-resourced research, e.g. insufficient attention to equipment, deficient supervision of junior co-workers, inadequate note-keeping of the research process, or use of inappropriate research designs (Gopalakrishna et al., 2022 ). Alternatively, those studies include behaviors such as “Fashion-determined choice of research topic”, “Instrumental and marketable approach”, and “Overselling methods, data or results” (Ravn & Sørensen, 2021 , p. 30; Vermeulen & Hartmann, 2015 ) which may be opportunistic or survivalist but not necessarily involve intentions to mislead.

To shed light on the prevalence of QRPs in different environments, the first step is to conceptualize and delimit the practices to be considered. The next step is to operationalize the conceptual approach into useful indicators and, if needed, to reformulate and reword the indicators into unambiguous, easily understood items (Hinkin, 1995 , 1998 ). The importance of careful item design has been demonstrated by Fiedler and Schwarz ( 2016 ). They show how the perceived QRP prevalence changes by adding specifications to well-known QRP items. Such specifications include: “ failing to report all dependent measures that are relevant for a finding ”, “ selectively reporting studies related to a specific finding that ‘’worked’ ” (Fiedler & Schwarz, 2016 , p. 46, italics in original ), or “collecting more data after seeing whether results were significant in order to render non-significant results significant ” (Fiedler & Schwarz, 2016 , p. 49, italics in original ). These specifications demonstrate the importance of precision in item design, the need for item tests before applications in a large-scale survey, and as the case study in Sect. 3 indicates, the value of statistically analyzing the selected items post-implementation.

The Problem of Social Desirability

Case studies of publicly exposed scientific misconduct have the advantage of explicitness and possible triangulation of sources (Berggren & Karabag, 2019 ; Huistra & Paul, 2022 ). Opinions may be contradictory, but researchers/investigators may often approach a variety of stakeholders and compare oral statements with documents and other sources (Berggren & Karabag, 2019 ). By contrast, quantitative studies of QRPs need to rely on non-public sources in the form of statements and appraisals of survey respondents for the dependent variables and for potentially associated factors such as publication pressure, job insecurity, or competitive climate.

Many QRP surveys use items that target the respondents’ personal attitudes and preferences regarding the dependent variables, indicating QRP prevalence, as well as the explanatory variables. This has the advantage that the respondents presumably know their own preferences and practices. A significant disadvantage, however, concerns social desirability, which in this context means the tendency of respondents to portray themselves, sometimes inadvertently, in more positive ways than justified by their behavior. The extent of this problem was indicated in a meta-study by Fanelli ( 2009 ), which demonstrated major differences between answers to sensitive survey questions that targeted the respondents’ own behavior and questions that focused on the behavior of their colleagues. In the case study below, the pros and cons of the latter indirect approaches are analyzed.

The Problem of Covering Both Quantitative and Qualitative Research

Studies of QRP prevalence are dominated by quantitative research approaches, where there exists a common understanding of the meaning of facts, proper procedures and scientific evidence. Several research fields, also in the social and medical sciences, include qualitative approaches — case studies, interpretive inquiries, or discourse analysis — where assessments of ‘truth’ and ‘evidence’ may be different or more complex to evaluate.

This does not mean that all qualitative endeavors are equal or that deceit—such as presenting fabricated interview quotes or referring to non-existent protocols —is accepted. However, while there are defined criteria for reporting qualitative research, such as the Consolidated Criteria for Reporting Qualitative Research (COREQ) (Tong et al., 2007 ) or the Standards for Reporting Qualitative Research (SRQR checklist) (O’Brien et al., 2014 ), the field of qualitative research encompasses a wide range of different approaches. This includes comparative case studies that offer detailed evidence to support their claims—such as the differences between British and Japanese factories (Dore, 1973 /2011)—as well as discourse analyses and interpretive studies, where the concept of ‘evidence’ is more fluid and hard to apply. The generative richness of the analysis is a key component of their quality (Flick, 2013 ). This intra-field variation makes it hard to pin down and agree upon general QRP items to capture such behaviors in qualitative research. Some researchers have tried to interpret and report qualitative research by means of quantified methods (Ravn & Sørensen, 2021 ), but so far, these attempts constitute a marginal phenomenon. Consequently, the challenges of measuring the prevalence of QRPs (or similar issues) in the variegated field of qualitative research remain largely unexplored.

The Problem of Institutional Controversiality and Personal Sensitivity

Science and academia depend on public trust for funding and executing research. This makes investigations of questionable behaviors a controversial issue for universities and may lead to institutional refusal/non-response. This resistance was experienced by the designers of a large-scale survey of norms and practices in the Dutch academia when several universities decided not to take part, referring to the potential danger of negative publicity (de Vrieze, 2021 ). A Flemish survey on academic careers encountered similar participation problems (Aubert Bonn & Pinxten, 2019 ). Another study on universities’ willingness to solicit whistleblowers for participation revealed that university officers, managers, and lawyers tend to feel obligated to protect their institution’s reputation (Byrn et al., 2016 ). Such institutional actors may resist participation to avoid the exposure of potentially negative information about their institutions and management practices, which might damage the university’s brand (Byrn et al., 2016 ; Downes, 2017 ).

QRP surveys involve sensitive and potentially intrusive questions also from a respondent’s personal perspective that can lead to a reluctance to participate and non-response behavior (Roberts & John, 2014 ; Tourangeau & Yan, 2007 ). Studies show that willingness to participate declines for surveys covering sensitive issues such as misconduct, crime, and corruption, compared to less sensitive ones like leisure activities (cf. Tourangeau et al., 2010 ). The method of survey administration—whether face-to-face, over the phone, via the web, or paper-based—can influence the perceived sensitivity and response rate (Siewert & Udani, 2016 ; Szolnoki & Hoffmann, 2013 ). In the case study below, the survey did not require any institutional support. Instead, the designers focused on minimizing the individual sensitivity problem by avoiding questions about the respondents’ personal practices. To manage this, they concentrated on their colleagues’ behaviors (see Sect. 4.2). Even if a respondent agrees to participate, they may not answer the QRP items due to insufficient knowledge about her colleagues’ practices or a lack of motivation to answer critical questions about their colleagues’ practices (Beatty & Herrmann, 2002 ; Yan & Curtin, 2010 ). Additionally, a significant time gap between observing specific QRPs in the respondent’s research environment and receiving the survey may make it difficult to recall and accurately respond to the questions. Such issues may also result in non-response problems.

Addressing the Problems: Case Study of a Cross-Field QRP Survey – Design Process, Survey Content, Design Challenges

This section presents a case study of the way these four problems were addressed in a cross-field survey intended to capture QRP prevalence and associated factors across the social and medical sciences in Sweden. The account is based on the authors’ intensive involvement in the design and analysis of the survey, including the technical and cognitive testing, and post-implementation analysis of item quality, missing responses, and open respondent comments. The theoretical background and the substantive results of the study are presented in a separate paper (Karabag et al., 2024 ). Method and language experts at Statistics Sweden, a government agency responsible for public statistics in Sweden, supported the testing procedures, the stratified respondent sampling and administered the survey roll-out.

The Survey Design Process – Repeated Testing and Prototyping

The design process included four steps of testing, revising, and prototyping, which allowed the researchers to iteratively improve the survey and plan the roll-out.

Step 1: Development of the Baseline Survey

This step involved searching the literature and creating a list of alternative constructs concerning the key concepts in the planned survey. Based on the study’s aim, the first and third authors compared these constructs and examined how they had been itemized in the literature. After two rounds of discussions, they agreed on construct formulations and relevant ways to measure them, rephrased items if deemed necessary, and designed new items in areas where the extant literature did not provide any guidance. In this way, Survey Version 1 was compiled.

Step 2: Pre-Testing by Means of a Large Convenience Sample

In the second step, this survey version was reviewed by two experts in organizational behavior at Linköping University. This review led to minor adjustments and the creation of Survey Version 2 , which was used for a major pretest. The aim was both to check the quality of individual items and to garner enough responses for a factor analysis that could be used to build a preliminary theoretical model. This dual aim required a larger sample than suggested in the literature on pretesting (Perneger et al., 2015 ). At the same time, it was essential to minimize the contamination of the planned target population in Sweden. To accomplish this, the authors used their access to a community of organization scholars to administer Survey Version 2 to 200 European management researchers.

This mass pre-testing yielded 163 responses. The data were used to form preliminary factor structures and test a structural equation model. Feedback from a few of the respondents highlighted conceptual issues and duplicated questions. Survey Version 3 was developed and prepared for detailed pretesting based on this feedback.

Step 3: Focused Pre-Testing and Technical Assessment

This step focused on the pre-testing and technical assessment. The participants in this step’s pretesting were ten researchers (six in the social sciences and four in the medical sciences) at five Swedish universities: Linköping, Uppsala, Gothenburg, Gävle, and Stockholm School of Economics. Five of those researchers mainly used qualitative research methods, two used both qualitative and quantitative methods, and three used quantitative methods. In addition, Statistics Sweden conducted a technical assessment of the survey items, focusing on wording, sequence, and response options. Footnote 1 Based on feedback from the ten pretest participants and the Statistics Sweden assessment, Survey Version 4 was developed, translated into Swedish, and reviewed by two researchers with expertise in research ethics and scientific misconduct.

It should be highlighted that Swedish academia is predominantly bilingual. While most researchers have Swedish as their mother tongue, many are more proficient in English, and a minority have limited or no knowledge of Swedish. During the design process, the two language versions were compared item by item and slightly adjusted by skilled bilingual researchers. This task was relatively straightforward since most items and concepts were derived from previously published literature in English. Notably, the Swedish versions of key terms and concepts have long been utilized within Swedish academia (see for example Berggren, 2016 ; Hasselberg, 2012 ). To secure translation quality, the language was controlled by a language expert at Statistics Sweden.

Step 4: Cognitive Interviews by Survey and Measurement Experts

Next, cognitive interviews (Willis, 2004 ) were organized with eight researchers from the social and medical sciences and conducted by an expert from Statistics Sweden (Wallenborg Likidis, 2019 ). The participants included four women and four men, ranging in age from 30 to 60. They were two doctoral students, two lecturers, and four professors, representing five different universities and colleges. Additionally, two participants had a non-Nordic background. To ensure confidentiality, no connections are provided between these characteristics and the individual participants.

An effort was made to achieve a distribution of gender, age, subject, employment, and institution. Four social science researchers primarily used qualitative research methods, while the remaining four employed qualitative and quantitative methods. Additionally, four respondents completed the Swedish version of the survey, and four completed the English version.

The respondents completed the survey in the presence of a methods expert from Statistics Sweden, who observed their entire response process. The expert noted spontaneous reactions and recorded instances where respondents hesitated or struggled to understand an item. After the survey, the expert conducted a structured interview with all eight participants, addressing details in each section of the survey, including the missive for recruiting respondents. Some respondents provided oral feedback while reading the cover letter and answering the questions, while others offered feedback during the subsequent interview.

During the cognitive interview process, the methods expert continuously communicated suggestions for improvements to the design team. A detailed test protocol confirmed that most items were sufficiently strong, although a few required minor modifications. The research team then finalized Survey Version 5 , which included both English and Swedish versions (for the complete survey, see Supplementary Material S1).

Although the test successfully captured a diverse range of participants, it would have been desirable to conduct additional tests of the English survey with more non-Nordic participants; as it stands, only one such test was conducted. Despite the participants’ different approaches to completing the survey, the estimated time to complete it was approximately 15–20 min. No significant time difference was observed between completing the survey in Swedish and English.

Design Challenges – the Dearth of an Item-Specific Public Quality Discussion

The design decision to employ survey items from the relevant literature as much as possible was motivated by a desire to increase comparability with previous studies of questionable research practices. However, this approach came with several challenges. Survey-based studies of QRPs rely on the respondents’ subjective assessments, with no possibility to compare the answers with other sources. Thus, an open discussion of survey problems would be highly valuable. However, although published studies usually present the items used in the surveys, there is seldom any analysis of the problems and tradeoffs involved when using a particular type of item or response format and meager information about item validity. Few studies, for example, contain any analysis that clarifies which items that measured the targeted variables with sufficient precision and which items that failed to do so.

Another challenge when using existing survey studies is the lack of information regarding the respondents’ free-text comments about the survey’s content and quality. This could be because the survey did not contain any open questions or because the authors of the report could not statistically analyze the answers. As seen below, however, open respondent feedback on a questionnaire involving sensitive or controversial aspects may provide important feedback regarding problems that did not surface during the pretest process, which by necessity targets much smaller samples.

Survey Content

The survey started with questions about the respondent’s current employment and research environment. It ended with background questions on the respondents’ positions and the extent of their research activity, plus space for open comments about the survey. The core content of the survey consisted of sections on the organizational climate (15 items), scientific norms (13 items), good and questionable research practices (16 items), perceptions of fairness in the academic system (4 items), motivation for conducting research (8 items), ethics training and policies (5 items); and questions on the quality of the research environment and the respondent’s perceived job security.

Sample and Response Rate

All researchers, teachers, and Ph.D. students employed at Swedish universities are registered by Statistics Sweden. To ensure balanced representation and perspectives from both large universities and smaller university colleges, the institutions were divided into three strata based on the number of researchers, teachers, and Ph.D. students: more than 1,000 individuals (7 universities and university colleges), 500–999 individuals (3 institutions), and fewer than 500 individuals (29 institutions). From these strata, Statistics Sweden randomly sampled 35%, 45%, and 50% of the relevant employees, resulting in a sample of 10,047 individuals. After coverage analysis and exclusion of wrongly included, 9,626 individuals remained.

The selected individuals received a personal postal letter with a missive in both English and Swedish informing them about the project and the survey and notifying them that they could respond on paper or online. The online version provided the option to answer in either English or Swedish. The paper version was available only in English to reduce the cost of production and posting. The missive provided the recipients with comprehensive information about the study and what their involvement would entail. It emphasized the voluntary character of participation and their right to withdraw from the survey at any time, adding: “If you do not want to answer the questions , we kindly ask you to contact us. Then you will not receive any reminders.” Sixty-three individuals used this decline option. In line with standard Statistics Sweden procedures, survey completion implied an agreement to participation and to the publication of anonymized results and indicated participants’ understanding of the terms provided (Duncan & Cheng, 2021 ). An email address was provided for respondents to request study outputs or for any other reason. The survey was open for data collection for two months, during which two reminders were sent to non-responders who had not opted out.

Once Statistics Sweden had collected the answers, they were anonymized and used to generate data files delivered to the authors. Statistics Sweden also provided anonymized information about age, gender, and type of employment of each respondent in the dataset delivered to the researchers. Of the targeted individuals, 3,295 responded, amounting to an overall response rate of 34.2%. An analysis of missing value patterns revealed that 290 of the respondents either lacked data for an entire factor or had too many missing values dispersed over several survey sections. After removing these 290 responses, we used SPSS algorithms (IBM-SPSS Statistics 27) to analyze the remaining missing values, which were randomly distributed and constituted less than 5% of the data. These values were replaced using the program’s imputation program (Madley-Dowd et al., 2019 ). The final dataset consisted of 3,005 individuals, evenly distributed between female and male respondents (53,5% vs. 46,5%) and medical and social scientists (51,3% vs. 48,5%). An overview of the sample and the response rate is provided in Table  1 , which can also be found in (Karabag et al., 2024 ). As shown in Table  1 , the proportion of male and female respondents, as well as the proportion of respondents from medical and social science, and the age distribution of the respondents compared well with the original selection frame from Statistics Sweden.

Revisiting the Four Problems. Partial Solutions and Remaining Issues

Managing the precision problem - the value of factor analyses.

As noted above, the lack of conceptual consensus and standard ways to measure QRPs has resulted in a huge variation in estimated prevalence. In the case studied here, the purpose was to investigate deviations from research integrity and not low-quality research in general. This conceptual focus implied that selected survey items regarding QRP should build on the core aspect of intention, as suggested by Banks et al. ( 2016 , p. 323): “design, analytic, or reporting practices that have been questioned because of the potential for the practice to be employed with the purpose of presenting biased evidence in favor of an assertion”. After scrutinizing the literature, five items were selected as general indicators of QRP, irrespective of the research approach (see Table  2 ).

An analysis of the survey responses indicated that the general QRP indicators worked well in terms of understandability and precision. Considering the sensitive nature of the items, features that typically yield very high rates of missing data (Fanelli, 2009 ; Tourangeau & Yan, 2007 ), our missing rates of 11–21% must be considered modest. In addition, there were a few critical comments on the item formulation in the open response section at the end of the survey (see below).

Regarding the explanatory (independent) variables, the survey was inspired by studies showing the importance of the organizational climate and the normative environment within academia (Anderson et al., 2010 ). Organizational climate can be measured in several ways; the studied survey focused on items related to a collegial versus a competitive climate. The analysis of the normative environment was inspired by the classical norms of science articulated by Robert Merton in his CUDOS framework: communism (communalism), universalism, disinterestedness, and organized skepticism (Merton, 1942 /1973). This framework has been extensively discussed and challenged but remains a key reference (Anderson et al., 2010 ; Chalmers & Glasziou, 2009 ; Kim & Kim, 2018 ; Macfarlane & Cheng, 2008 ). Moreover, we were inspired by the late work of Merton on the ambivalence and ambiguities of scientists (Merton, 1942 /1973), and the counter norms suggested by Mitroff ( 1974 ). Thus, the survey involved a composite set of items to capture the contradictory normative environment in academia: classical norms as well as their counter norms.

To reduce the problems of social desirability bias and personal sensitivity, the survey design avoided items about the respondent’s personal adherence to explicit ideals, which are common in many surveys (Gopalakrishna et al., 2022 ). Instead, the studied survey focused on the normative preferences and attitudes within the respondent’s environment. This necessitated the identification, selection, and refinement of 3–4 items for each potentially relevant norm/counter-norm. The selection process was used in previous studies of norm subscription in various research communities (Anderson et al., 2007 ; Braxton, 1993 ; Bray & von Storch, 2017 ). For the norm “skepticism”, we consulted studies in the accounting literature of the three key elements of professional skepticism: questioning mind, suspension of judgment and search for knowledge (Hurtt, 2010 ).

The first analytical step after receiving the completed survey set from Statistics Sweden was to conduct a set of factor analyses to assess the quality and validity of the survey items related to the normative environment and the organizational climate. These analyses suggested three clearly identifiable factors related to the normative environment: (1) a counter norm factor combining Mitroff’s particularism and dogmatism (‘Biasedness’ in the further analysis), and two Mertonian factors: (2) Skepticism and (3) Openness, a variant of Merton’s Communalism (see Table  3 ). A fourth Merton factor, Disinterestedness, could not be identified in our analysis.

The analytical process for organizational climate involved reducing the number of items from 15 to 11 (see Table 4 ). Here, the factor analysis suggested two clearly identifiable factors, one related to collegiality and the other related to competition (see Table  4 ). Overall, the factor analyses suggested that the design efforts had paid off in terms of high item quality, robust factor loadings, and a very limited need to remove any items.

In a parallel step, the open comments were assessed as an indication of how the study was perceived by the respondents (see Table  5 ). Of the 3005 respondents, 622 provided comprehensible comments, and many of them were extensive. 187 comments were related to the respondents’ own employment/role, 120 were related to the respondents’ working conditions and research environment, and 98 were related to the academic environment and atmosphere. Problems in knowing details of collegial practices were mentioned in 82 comments.

Reducing Desirability Bias - the Challenge of Nonresponse

It is well established that studies on topics where the respondent has anything embarrassing or sensitive to report suffer from more missing responses than studies on neutral subjects and that respondents may edit the information they provide on sensitive topics (Tourangeau & Yan, 2007 ). Such a social desirability bias is applicable for QRP studies which explicitly target the respondents’ personal attitudes and behaviors. To reduce this problem, the studied survey applied a non-self-format focusing on the behaviors and preferences of the respondents’ colleagues. Relevant survey items from published studies were rephrased from self-format designs to non-self-questions about practices in the respondent’s environment, using the format: “In my research environment, colleagues…” followed by a five-step incremental response format from “(1) never” to “(5) always”. In a similar way the survey avoided “should”-statements about ideal normative values: “Scientists and scholars should critically examine…”. Instead, the survey used items intended to indicate the revealed preferences in the respondent’s normative environment regarding universalism versus particularism or openness versus secrecy.

As indicated by Fanelli ( 2009 ), these redesign efforts probably reduced the social desirability bias significantly. At the same time, however, the redesign seemed to increase a problem not discussed by Fanelli ( 2009 ): an increased uncertainty problem related to the respondents’ difficulties of knowing the practices of their colleagues in questionable areas. This issue was indicated by the open comment at the end of the studied survey, where 13% of the 622 respondents pointed out that they lacked sufficient knowledge about the behavior of their colleagues to answer the QRP questions (see Table  5 ). One respondent wrote:

“It’s difficult to answer questions about ‘colleagues in my research area’ because I don’t have an insight into their research practices; I can only make informed guesses and generalizations. Therefore, I am forced to answer ‘don’t know’ to a lot of questions”.

Regarding the questions on general QRPs, the rate of missing responses varied between 11% and 21%. As for the questions targeting specific QRP practices in quantitative and qualitative research, the rate of missing responses ranged from 38 to 49%. Unfortunately, the non-response alternative to these questions (“Don’t know/not relevant”) combined the two issues: the lack of knowledge and the lack of relevance. Thus, we don’t know what part of the missing responses related to a non-presence of the specific research approach in the respondent’s environment and what part signaled a lack of knowledge about collegial practices in this environment.

Measuring QRPs in Qualitative Research - the Limited Role of Pretests

Studies of QRP prevalence focus on quantitative research approaches, where there exists a common understanding of the interpretation of scientific evidence, clearly recommended procedures, and established QRP items related to compliance with these procedures. In the heterogenous field of qualitative research, there are several established standards for reporting the research (O’Brien et al., 2014 ; Tong et al., 2007 ), but, as noted above, hardly any commonly accepted survey items that capture behaviors that fulfill the criteria for QRPs. As a result, the studied survey project designed such items from the start during the survey development process. After technical and cognitive tests, four items were selected. See Table  6 .

Despite the series of pretests, however, the first two of these items met severe criticism from a few respondents in the survey’s open commentary section. Here, qualitative researchers argued that the items were unduly influenced by the truth claims in quantitative studies, whereas their research dealt with interpretation and discourse analysis. Thus, they rejected the items regarding selective usage of respondents and of interview quotes as indicators of questionable practices:

“The alternative regarding using quotes is a bit misleading. Supporting your results by quotes is a way to strengthen credibility in a qualitative method….” “The question about dubious practices is off target for us, who work with interpretation rather than solid truths. You can present new interpretations, but normally that does not imply that previous ‘findings’ should be considered incorrect.” “The questions regarding qualitative research were somewhat irrelevant. Often this research is not guided by a given hypothesis, and researchers may use a convenient sample without this resulting in lower quality.”

One comment focused on other problems related to qualitative research:

“Several questions do not quite capture the ethical dilemmas we wrestle with. For example , is the issue of dishonesty and ‘inaccuracies’ a little misplaced for us who work with interpretation? …At the same time , we have a lot of ethical discussions , which , for example , deal with power relations between researchers and ‘researched’ , participant observation/informal contacts and informed consent (rather than patients participating in a study)”.

Unfortunately, the survey received these comments and criticism only after the full-scale rollout and not during the pretest rounds. Thus, we had no chance to replace the contested items with other formulations or contemplate a differentiation of the subsection to target specific types of qualitative research with appropriate questions. Instead, we had to limit the post-roll-out survey analysis to the last two items in Table  6 , although they captured devious behaviors rather than gray zone practices.

Why then was this criticism of QRP items related to qualitative research not exposed in the pretest phase? This is a relevant question, also for future survey designers. An intuitive answer could be that the research team only involved quantitative researchers. However, as highlighted above, the pretest participants varied in their research methods: some exclusively used qualitative methods, others employed mixed methods, and some utilized quantitative methods. This diversity suggests that the selection of test participants was appropriate. Moreover, all three members of the research team had experience of both quantitative and qualitative studies. However, as discussed above, the field of qualitative research involves several different types of research, with different goals and methods – from detailed case studies grounded in original empirical fieldwork to participant observations of complex organizational phenomena to discursive re-interpretations of previous studies. Of the 3,005 respondents who answered the survey in a satisfactory way, only 16 respondents, or 0,5%, had any critical comments about the QRP items related to qualitative research. A failure to capture the objections from such a small proportion in a pretest phase is hardly surprising. The general problem could be compared with the challenge of detecting negative side-effects in drug development. Although the pharmaceutical firms conduct large-scale tests of candidate drugs before government approval, doctors nevertheless detect new side-effects when the medicine is rolled out to significantly more people than the test populations – and report these less frequent problems in the additional drug information (Galeano et al., 2020 ; McNeil et al., 2010 ).

In the social sciences, the purpose of pre-testing is to identify problems related to ambiguities and bias in item formulation and survey format and initiate a search for relevant solutions. A pre-test on a small, selected subsample cannot guarantee that all respondent problems during the full-scale data collection will be detected. The pretest aims to reduce errors to acceptable levels and ensure that the respondents will understand the language and terminology chosen. Pretesting in survey development is also essential to help the researchers to assess the overall flow and structure of the survey, and to make necessary adjustments to enhance respondent engagement and data quality (Ikart, 2019 ; Presser & Blair, 1994 ).

In our view, more pretests would hardly solve the epistemological challenge of formulating generally acceptable QRP items for qualitative research. The open comments studied here suggest that there is no one-size-fits-all solution. If this is right, the problem should rather be reformulated to a question of identifying different strands of qualitative research with diverse views of integrity and evidence which need to be measured with different measures. To address this challenge in a comprehensive way, however, goes far beyond the current study.

Controversiality and Collegial sensitivity - the Challenge of Predicting Nonresponse

Studies of research integrity, questionable research practices, and misconduct in science tend to be organizationally controversial and personally sensitive. If university leaders are asked to support such studies, there is a considerable risk that the answer will be negative. In the case studied here, the survey roll-out was not dependent on any active organizational participation since Statistics Sweden possessed all relevant respondent information in-house. This, we assumed, would take the controversiality problem off the agenda. Our belief was supported by the non-existent complaints regarding a potential negativity bias from the pretest participants. Instead, the problem surfaced when the survey was rolled out, and all the respondents contemplated the survey. The open comment section at the end of the survey provided insights into this reception.

Many respondents provided positive feedback, reflected in 30 different comments such as:

“Thank you for doing this survey. I really hope it will lead to changes because it is needed”. “This is an important survey. However , there are conflicting norms , such as those you cite in the survey , /concerning/ for example , data protection. How are researchers supposed to be open when we cannot share data for re-analysis?” “I am glad that the problems with egoism and non-collegiality are addressed in this manner ”.

Several of them asked for more critical questions regarding power, self-interest, and leadership:

“What I lack in the survey were items regarding academic leadership. Otherwise, I am happy that someone is doing research on these issues”. “A good survey but needs to be complemented with questions regarding researchers who put their commercial interests above research and exploit academic grants for commercial purposes”.

A small minority criticized the survey for being overly negative towards academia:

“A major part of the survey feels very negative and /conveys/ the impression that you have a strong pre-understanding of academia as a horrible environments”. “Some of the questions are uncomfortable and downright suggestive. Why such a negative attitude towards research?” “The questions have a tendency to make us /the respondents/ informers. An unpleasant feeling when you are supposed to lay information against your university”. “Many questions are hard to answer, and I feel that they measure my degree of suspicion against my closest colleagues and their motivation … Several questions I did not want to answer since they contain a negative interpretation of behaviors which I don’t consider as automatically negative”.

A few of these respondents stated that they abstained from answering some of the ‘negative questions’, since they did not want to report on or slander their colleagues. The general impact is hard to assess. Only 20% of the respondents offered open survey comments, and only seven argued that questions were “negative”. The small number explains why the issue of negativity did not show up during the testing process. However, a perceived sense of negativity may have affected the willingness to answer among more respondents than those who provided free test comments.

Conclusion - The Needs for a Cumulative Knowledge Trajectory in Integrity Studies

In the broad field of research integrity studies, investigations of QRPs in different contexts and countries play an important role. The comparability of the results, however, depends on the conceptual focus of the survey design and the quality of the survey items. This paper starts with a discussion of four common problems in QRP research: the problems of precision, social desirability, incomplete coverage, and organizational controversiality and sensitivity. This is followed by a case study of how these problems were addressed in a detailed survey design process. An assessment of the solutions employed in the studied survey design reveals progress as well as unresolved issues.

Overall, the paper shows that the problem and challenges of precision could be effectively managed through explicit conceptual definitions and careful item design.

The problem of social desirability bias was probably reduced by means of a non-self-response format referring to preferences and behaviors among colleagues instead of personal behaviors. However, an investigation of open respondent comments indicated that the reduced risk of social bias came at the expense of higher uncertainty due to the respondents’ lack of insight in the concrete practices of their colleagues.

The problem of incomplete coverage of QRPs in qualitative research, the authors initially linked to “the lack of standard items” to capture QRPs in qualitative studies. Open comments at the end of the survey, however, suggested that the lack of such standards would not be easily managed by the design of new items. Rather, it seems to be an epistemological challenge related to the multifarious nature of the qualitative research field, where the understanding of ‘evidence’ is unproblematic in some qualitative sub-fields but contested in others. This conjecture and other possible explanations will hopefully be addressed in forthcoming epistemological and empirical studies.

Regarding the problem of controversiality and sensitivity, previous studies show that QRP research is a controversial and sensitive area for academic executives and university brand managers. The case study discussed here indicates that this is a sensitive subject also for rank-and-file researchers who may hesitate to answer, even when the questions do not target the respondents’ own practices but the practices and preferences of their colleagues. Future survey designers may need to engage in framing, presenting, and balancing sensitive items to reduce respondent suspicions and minimize the rate of missing responses. Reflections on the case indicate that this is doable but requires thoughtful design, as well as repeated tests, including feedback from a broad selection of prospective participants.

In conclusion, the paper suggests that more resources should be spent on the systematic evaluation of different survey designs and item formulations. In the long term, such investments in method development will yield a higher proportion of robust and comparable studies. This would mitigate the problems discussed here and contribute to the creation of a much-needed cumulative knowledge trajectory in research integrity studies.

An issue not covered here is that surveys, however finely developed, only give quantitative information about patterns, behaviors, and structures. An understanding of underlying thoughts and perspectives requires other procedures. Thus, methods that integrate and triangulate qualitative and quantitative data —known as mixed methods (Karabag & Berggren, 2016 ; Ordu & Yılmaz, 2024 ; Smajic et al., 2022 )— may give a deeper and more complete picture of the phenomenon of QRP.

Data Availability

The data supporting the findings of this study are available from the corresponding author, upon reasonable request.

Wallenborg Likidis ( 2019 ). Academic norms and scientific attitudes: Metrology Review of a survey for doctoral students , researchers and academic teachers (In Swedish: Akademiska normer och vetenskapliga förhallningssätt. Mätteknisk granskning av en enkät till doktorander , forskare och akademiska lärare) . Prod.nr. 8,942,146, Statistics Sweden, Örebro.

Agnoli, F., Wicherts, J. M., Veldkamp, C. L., Albiero, P., & Cubelli, R. (2017). Questionable research practices among Italian research psychologists. PLoS One , 12(3), e0172792.

Anderson, M. S., Ronning, E. A., De Vries, R., & Martinson, B. C. (2007). The perverse effects of competition on scientists’ work and relationships. Science and Engineering Ethics , 13 , 437–461.

Article   Google Scholar  

Anderson, M. S., Ronning, E. A., Devries, R., & Martinson, B. C. (2010). Extending the Mertonian norms: Scientists’ subscription to norms of Research. The Journal of Higher Education , 81 (3), 366–393. https://doi.org/10.1353/jhe.0.0095

Andrade, C. (2021). HARKing, cherry-picking, p-hacking, fishing expeditions, and data dredging and mining as questionable research practices. The Journal of Clinical Psychiatry , 82 (1), 25941.

ArtinoJr, A. R., Driessen, E. W., & Maggio, L. A. (2019). Ethical shades of gray: International frequency of scientific misconduct and questionable research practices in health professions education. Academic Medicine , 94 (1), 76–84.

Aubert Bonn, N., & Pinxten, W. (2019). A decade of empirical research on research integrity: What have we (not) looked at? Journal of Empirical Research on Human Research Ethics , 14 (4), 338–352.

Banks, G. C., O’Boyle Jr, E. H., Pollack, J. M., White, C. D., Batchelor, J. H., Whelpley, C. E., & Adkins, C. L. (2016). Questions about questionable research practices in the field of management: A guest commentary. Journal of Management , 42 (1), 5–20.

Beatty, P., & Herrmann, D. (2002). To answer or not to answer: Decision processes related to survey item nonresponse. Survey Nonresponse , 71 , 86.

Google Scholar  

Berggren, C. (2016). Scientific Publishing: History, practice, and ethics (in Swedish: Vetenskaplig Publicering: Historik, Praktik Och Etik) . Studentlitteratur AB.

Berggren, C., & Karabag, S. F. (2019). Scientific misconduct at an elite medical institute: The role of competing institutional logics and fragmented control. Research Policy , 48 (2), 428–443. https://doi.org/10.1016/j.respol.2018.03.020

Braxton, J. M. (1993). Deviancy from the norms of science: The effects of anomie and alienation in the academic profession. Research in Higher Education , 54 (2), 213–228. https://www.jstor.org/stable/40196105

Bray, D., & von Storch, H. (2017). The normative orientations of climate scientists. Science and Engineering Ethics , 23 (5), 1351–1367.

Breakwell, G. M., Wright, D. B., & Barnett, J. (2020). Research questions, design, strategy and choice of methods. Research Methods in Psychology , 1–30.

Brenner, P. S. (2020). Why survey methodology needs sociology and why sociology needs survey methodology: Introduction to understanding survey methodology: Sociological theory and applications. In Understanding survey methodology: Sociological theory and applications (pp. 1–11). https://doi.org/10.1007/978-3-030-47256-6_1

Bruton, S. V., Medlin, M., Brown, M., & Sacco, D. F. (2020). Personal motivations and systemic incentives: Scientists on questionable research practices. Science and Engineering Ethics , 26 (3), 1531–1547.

Butler, N., Delaney, H., & Spoelstra, S. (2017). The gray zone: Questionable research practices in the business school. Academy of Management Learning & Education , 16 (1), 94–109.

Byrn, M. J., Redman, B. K., & Merz, J. F. (2016). A pilot study of universities’ willingness to solicit whistleblowers for participation in a study. AJOB Empirical Bioethics , 7 (4), 260–264.

Chalmers, I., & Glasziou, P. (2009). Avoidable waste in the production and reporting of research evidence. The Lancet , 374 (9683), 86–89.

de Vrieze, J. (2021). Large survey finds questionable research practices are common. Science . https://doi.org/10.1126/science.373.6552.265

Dore, R. P. (1973/2011). British Factory Japanese Factory: The origins of National Diversity in Industrial Relations, with a New Afterword . University of California Press/Routledge.

Downes, M. (2017). University scandal, reputation and governance. International Journal for Educational Integrity , 13 , 1–20.

Duncan, L. J., & Cheng, K. F. (2021). Public perception of NHS general practice during the first six months of the COVID-19 pandemic in England. F1000Research , 10 .

Fanelli, D. (2009). How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS One , 4(5), e5738.

Fiedler, K., & Schwarz, N. (2016). Questionable research practices revisited. Social Psychological and Personality Science , 7 (1), 45–52.

Flick, U. (2013). The SAGE Handbook of Qualitative Data Analysis . sage.

Fraser, H., Parker, T., Nakagawa, S., Barnett, A., & Fidler, F. (2018). Questionable research practices in ecology and evolution. PLoS One , 13(7), e0200303.

Galeano, D., Li, S., Gerstein, M., & Paccanaro, A. (2020). Predicting the frequencies of drug side effects. Nature Communications , 11 (1), 4575.

Gopalakrishna, G., Ter Riet, G., Vink, G., Stoop, I., Wicherts, J. M., & Bouter, L. M. (2022). Prevalence of questionable research practices, research misconduct and their potential explanatory factors: A survey among academic researchers in the Netherlands. PLoS One , 17 (2), e0263023.

Hasselberg, Y. (2012). Science as Work: Norms and Work Organization in Commodified Science (in Swedish: Vetenskap Som arbete: Normer och arbetsorganisation i den kommodifierade vetenskapen) . Gidlunds förlag.

Hill, J., Ogle, K., Gottlieb, M., Santen, S. A., & ArtinoJr, A. R. (2022). Educator’s blueprint: a how-to guide for collecting validity evidence in survey‐based research. AEM Education and Training , 6(6), e10835.

Hinkin, T. R. (1995). A review of scale development practices in the study of organizations. Journal of Management , 21 (5), 967–988.

Hinkin, T. R. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods , 1 (1), 104–121.

Huistra, P., & Paul, H. (2022). Systemic explanations of scientific misconduct: Provoked by spectacular cases of norm violation? Journal of Academic Ethics , 20 (1), 51–65.

Hurtt, R. K. (2010). Development of a scale to measure professional skepticism. Auditing: A Journal of Practice & Theory , 29 (1), 149–171.

Ikart, E. M. (2019). Survey questionnaire survey pretesting method: An evaluation of survey questionnaire via expert reviews technique. Asian Journal of Social Science Studies , 4 (2), 1.

Karabag, S. F., & Berggren, C. (2016). Misconduct, marginality and editorial practices in management, business and economics journals. PLoS One , 11 (7), e0159492. https://doi.org/10.1371/journal.pone.0159492

Karabag, S. F., Berggren, C., Pielaszkiewicz, J., & Gerdin, B. (2024). Minimizing questionable research practices–the role of norms, counter norms, and micro-organizational ethics discussion. Journal of Academic Ethics , 1–27. https://doi.org/10.1007/s10805-024-09520-z

Kim, S. Y., & Kim, Y. (2018). The ethos of Science and its correlates: An empirical analysis of scientists’ endorsement of Mertonian norms. Science Technology and Society , 23 (1), 1–24. https://doi.org/10.1177/0971721817744438

Lawlor, J., Thomas, C., Guhin, A. T., Kenyon, K., Lerner, M. D., Consortium, U., & Drahota, A. (2021). Suspicious and fraudulent online survey participation: Introducing the REAL framework. Methodological Innovations , 14 (3), 20597991211050467.

Levelt, W. J., Drenth, P., & Noort, E. (2012). Flawed science: The fraudulent research practices of social psychologist Diederik Stapel (in Dutch: Falende wetenschap: De frauduleuze onderzoekspraktijken van social-psycholoog Diederik Stapel) . Commissioned by the Tilburg University, University of Amsterdam and the University of Groningen. https://doi.org/http://hdl.handle.net/11858/00-001M-0000-0010-258A-9

Lietz, P. (2010). Research into questionnaire design: A summary of the literature. International Journal of Market Research , 52 (2), 249–272.

Lin, M. W., & Yu, C. (2020). Can corruption be measured? Comparing global versus local perceptions of corruption in East and Southeast Asia. In Regional comparisons in comparative policy analysis studies (pp. 90–107). Routledge.

Macfarlane, B., & Cheng, M. (2008). Communism, universalism and disinterestedness: Re-examining contemporary support among academics for Merton’s scientific norms. Journal of Academic Ethics , 6 , 67–78.

Madley-Dowd, P., Hughes, R., Tilling, K., & Heron, J. (2019). The proportion of missing data should not be used to guide decisions on multiple imputation. Journal of Clinical Epidemiology , 110 , 63–73.

McNeil, J. J., Piccenna, L., Ronaldson, K., & Ioannides-Demos, L. L. (2010). The value of patient-centred registries in phase IV drug surveillance. Pharmaceutical Medicine , 24 , 281–288.

Merton, R. K. (1942/1973). The normative structure of science. In The sociology of science: Theoretical and empirical investigations . The University of Chicago Press.

Mitroff, I. I. (1974). Norms and counter-norms in a select group of the Apollo Moon scientists: A case study of the ambivalence of scientists. American Sociological Review , 39 (4), 579–595. https://doi.org/10.2307/2094423

Necker, S. (2014). Scientific misbehavior in economics. Research Policy , 43 (10), 1747–1759. https://doi.org/10.1016/j.respol.2014.05.002

Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., & Nuijten, M. B. (2022). Replicability, robustness, and reproducibility in psychological science. Annual Review of Psychology , 73 (1), 719–748.

O’Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014). Standards for reporting qualitative research: A synthesis of recommendations. Academic Medicine , 89 (9). https://journals.lww.com/academicmedicine/fulltext/2014/09000/standards_for_reporting_qualitative_research__a.21.aspx

Ordu, Y., & Yılmaz, S. (2024). Examining the impact of dramatization simulation on nursing students’ ethical attitudes: A mixed-method study. Journal of Academic Ethics , 1–13.

Perneger, T. V., Courvoisier, D. S., Hudelson, P. M., & Gayet-Ageron, A. (2015). Sample size for pre-tests of questionnaires. Quality of life Research , 24 , 147–151.

Presser, S., & Blair, J. (1994). Survey pretesting: Do different methods produce different results? Sociological Methodology , 73–104.

Ravn, T., & Sørensen, M. P. (2021). Exploring the gray area: Similarities and differences in questionable research practices (QRPs) across main areas of research. Science and Engineering Ethics , 27 (4), 40.

Roberts, D. L., & John, F. A. S. (2014). Estimating the prevalence of researcher misconduct: a study of UK academics within biological sciences. PeerJ , 2 , e562.

Siewert, W., & Udani, A. (2016). Missouri municipal ethics survey: Do ethics measures work at the municipal level? Public Integrity , 18 (3), 269–289.

Smajic, E., Avdic, D., Pasic, A., Prcic, A., & Stancic, M. (2022). Mixed methodology of scientific research in healthcare. Acta Informatica Medica , 30 (1), 57–60. https://doi.org/10.5455/aim.2022.30.57-60

Steneck, N. H. (2006). Fostering integrity in research: Definitions, current knowledge, and future directions. Science and Engineering Ethics , 12 , 53–74.

Szolnoki, G., & Hoffmann, D. (2013). Online, face-to-face and telephone surveys—comparing different sampling methods in wine consumer research. Wine Economics and Policy , 2 (2), 57–66.

Tong, A., Sainsbury, P., & Craig, J. (2007). Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care , 19 (6), 349–357. https://doi.org/10.1093/intqhc/mzm042

Tourangeau, R., & Yan, T. (2007). Sensitive questions in surveys. Psychological Bulletin , 133 (5), 859.

Tourangeau, R., Groves, R. M., & Redline, C. D. (2010). Sensitive topics and reluctant respondents: Demonstrating a link between nonresponse bias and measurement error. Public Opinion Quarterly , 74 (3), 413–432.

Vermeulen, I., & Hartmann, T. (2015). Questionable research and publication practices in communication science. Communication Methods and Measures , 9 (4), 189–192.

Wallenborg Likidis, J. (2019). Academic norms and scientific attitudes: Metrology review of a survey for doctoral students, researchers and academic teachers (In Swedish: Akademiska normer och vetenskapliga förhallningssätt. Mätteknisk granskning av en enkät till doktorander, forskare och akademiska lärare) . Prod.nr. 8942146, Statistics Sweden, Örebro.

Willis, G. B. (2004). Cognitive interviewing: A tool for improving questionnaire design . Sage Publications.

Xie, Y., Wang, K., & Kong, Y. (2021). Prevalence of research misconduct and questionable research practices: A systematic review and meta-analysis. Science and Engineering Ethics , 27 (4), 41.

Yan, T., & Curtin, R. (2010). The relation between unit nonresponse and item nonresponse: A response continuum perspective. International Journal of Public Opinion Research , 22 (4), 535–551.

Download references

Acknowledgements

We thank Jennica Wallenborg Likidis, Statistics Sweden, for providing expert support in the survey design. We are grateful to colleagues Ingrid Johansson Mignon, Cecilia Enberg, Anna Dreber Almenberg, Andrea Fried, Sara Liin, Mariano Salazar, Lars Bengtsson, Harriet Wallberg, Karl Wennberg, and Thomas Magnusson, who joined the pretest or cognitive tests. We also thank Ksenia Onufrey, Peter Hedström, Jan-Ingvar Jönsson, Richard Öhrvall, Kerstin Sahlin, and David Ludvigsson for constructive comments or suggestions.

Open access funding provided by Linköping University. Swedish Forte: Research Council for Health, Working Life and Welfare ( https://www.vr.se/swecris?#/project/2018-00321_Forte ) Grant No. 2018-00321.

Open access funding provided by Linköping University.

Author information

Authors and affiliations.

Department of Management and Engineering [IEI], Linköping University, Linköping, SE-581 83, Sweden

Christian Berggren & Solmaz Filiz Karabag

Department of Surgical Sciences, Uppsala University, Uppsala University Hospital, entrance 70, Uppsala, SE-751 85, Sweden

Bengt Gerdin

Department of Civil and Industrial Engineering, Uppsala University, Box 169, Uppsala, SE-751 04, Sweden

Solmaz Filiz Karabag

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization: CB. Survey Design: SFK, CB, Methodology: SFK, BG, CB. Visualization: SFK, BG. Funding acquisition: SFK. Project administration and management: SFK. Writing – original draft: CB. Writing – review & editing: CB, BG, SFK. Approval of the final manuscript: SFK, BG, CB.

Corresponding author

Correspondence to Solmaz Filiz Karabag .

Ethics declarations

Ethics approval and consent to participate.

The Swedish Act concerning the Ethical Review of Research Involving Humans (2003:460) defines the type of studies which requires an ethics approval. In line with the General Data Protection Regulation (EU 2016/67), the act is applicable for studies that collect personal data that reveal racial or ethnic origin, political opinions, trade union membership, religious or philosophical beliefs, or health and sexual orientation. The present study does not involve any of the above, why no formal ethical permit was required. The ethical aspects of the project and its compliance with the guidelines of the Swedish Research Council (2017) were also part of the review process at the project’s public funding agency Forte.

Competing Interests

The authors declare that they have no competing interests.

Supporting Information

The complete case study survey of social and medical science researchers in Sweden 2020.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Berggren, C., Gerdin, B. & Karabag, S.F. Developing Surveys on Questionable Research Practices: Four Challenging Design Problems. J Acad Ethics (2024). https://doi.org/10.1007/s10805-024-09565-0

Download citation

Accepted : 23 August 2024

Published : 02 September 2024

DOI : https://doi.org/10.1007/s10805-024-09565-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Questionable Research Practices
• Normative Environment
• Organizational Climate
• Survey Development
• Design Problems
• Problem of Incomplete Coverage
• Survey Design Process
• Baseline Survey
• Pre-testing
• Technical Assessment
• Cognitive Interviews
• Social Desirability
• Sensitivity
• Organizational Controversiality
• Challenge of Nonresponse
• Qualitative Research
• Quantitative Research
• Find a journal
• Publish with us
• Track your research

Navigation Menu

Search code, repositories, users, issues, pull requests..., provide feedback.

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly.

To see all available qualifiers, see our documentation .

• Notifications You must be signed in to change notification settings

An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.

stanford-oval/storm

Folders and files.

Repository files navigation

Storm: synthesis of topic outlines through retrieval and multi-perspective question asking.

| Research preview | Paper | Website |

Latest News 🔥

• [2024/07] You can now install our package with pip install knowledge-storm !
• [2024/07] We add VectorRM to support grounding on user-provided documents, complementing existing support of search engines ( YouRM , BingSearch ). (check out #58 )
• [2024/07] We release demo light for developers a minimal user interface built with streamlit framework in Python, handy for local development and demo hosting (checkout #54 )
• [2024/06] We will present STORM at NAACL 2024! Find us at Poster Session 2 on June 17 or check our presentation material .
• [2024/05] We add Bing Search support in rm.py . Test STORM with GPT-4o - we now configure the article generation part in our demo using GPT-4o model.
• [2024/04] We release refactored version of STORM codebase! We define interface for STORM pipeline and reimplement STORM-wiki (check out src/storm_wiki ) to demonstrate how to instantiate the pipeline. We provide API to support customization of different language models and retrieval/search integration.

Overview (Try STORM now!)

While the system cannot produce publication-ready articles that often require a significant number of edits, experienced Wikipedia editors have found it helpful in their pre-writing stage.

Try out our live research preview to see how STORM can help your knowledge exploration journey and please provide feedback to help us improve the system 🙏!

How STORM works

STORM breaks down generating long articles with citations into two steps:

• Pre-writing stage : The system conducts Internet-based research to collect references and generates an outline.
• Writing stage : The system uses the outline and references to generate the full-length article with citations.

STORM identifies the core of automating the research process as automatically coming up with good questions to ask. Directly prompting the language model to ask questions does not work well. To improve the depth and breadth of the questions, STORM adopts two strategies:

• Perspective-Guided Question Asking : Given the input topic, STORM discovers different perspectives by surveying existing articles from similar topics and uses them to control the question-asking process.
• Simulated Conversation : STORM simulates a conversation between a Wikipedia writer and a topic expert grounded in Internet sources to enable the language model to update its understanding of the topic and ask follow-up questions.

Based on the separation of the two stages, STORM is implemented in a highly modular way using dspy .

Installation

To install the knowledge storm library, use pip install knowledge-storm .

You could also install the source code which allows you to modify the behavior of STORM engine directly.

Clone the git repository.

Install the required packages.

The STORM knowledge curation engine is defined as a simple Python STORMWikiRunner class.

As STORM is working in the information curation layer, you need to set up the information retrieval module and language model module to create a STORMWikiRunner instance. Here is an example of using You.com search engine and OpenAI models.

Currently, our package support:

• OpenAIModel , AzureOpenAIModel , ClaudeModel , VLLMClient , TGIClient , TogetherClient , OllamaClient , GoogleModel , DeepSeekModel , GroqModel as language model components
• YouRM , BingSearch , VectorRM , SerperRM , BraveRM , SearXNG , DuckDuckGoSearchRM , and TavilySearchRM as retrieval module components

🌟 PRs for integrating more language models into knowledge_storm/lm.py and search engines/retrievers into knowledge_storm/rm.py are highly appreciated!

The STORMWikiRunner instance can be evoked with the simple run method:

• do_research : if True, simulate conversations with difference perspectives to collect information about the topic; otherwise, load the results.
• do_generate_outline : if True, generate an outline for the topic; otherwise, load the results.
• do_generate_article : if True, generate an article for the topic based on the outline and the collected information; otherwise, load the results.
• do_polish_article : if True, polish the article by adding a summarization section and (optionally) removing duplicate content; otherwise, load the results.

Quick Start with Example Scripts

We provide scripts in our examples folder as a quick start to run STORM with different configurations.

To run STORM with gpt family models with default configurations:

• We suggest using secrets.toml to set up the API keys. Create a file secrets.toml under the root directory and add the following content: # Set up OpenAI API key. OPENAI_API_KEY= " your_openai_api_key " # If you are using the API service provided by OpenAI, include the following line: OPENAI_API_TYPE= " openai " # If you are using the API service provided by Microsoft Azure, include the following lines: OPENAI_API_TYPE= " azure " AZURE_API_BASE= " your_azure_api_base_url " AZURE_API_VERSION= " your_azure_api_version " # Set up You.com search API key. YDC_API_KEY= " your_youcom_api_key "
• Run the following command. python examples/run_storm_wiki_gpt.py \ --output-dir $OUTPUT_DIR \ --retriever you \ --do-research \ --do-generate-outline \ --do-generate-article \ --do-polish-article

To run STORM using your favorite language models or grounding on your own corpus: Check out examples/README.md .

Customization of the Pipeline

If you have installed the source code, you can customize STORM based on your own use case. STORM engine consists of 4 modules:

• Knowledge Curation Module: Collects a broad coverage of information about the given topic.
• Outline Generation Module: Organizes the collected information by generating a hierarchical outline for the curated knowledge.
• Article Generation Module: Populates the generated outline with the collected information.
• Article Polishing Module: Refines and enhances the written article for better presentation.

The interface for each module is defined in knowledge_storm/interface.py , while their implementations are instantiated in knowledge_storm/storm_wiki/modules/* . These modules can be customized according to your specific requirements (e.g., generating sections in bullet point format instead of full paragraphs).

Replicate NAACL2024 result

Please switch to the branch NAACL-2024-code-backup

Paper Experiments

The FreshWiki dataset used in our experiments can be found in ./FreshWiki .

Run the following commands under ./src .

Pre-writing Stage

For batch experiment on FreshWiki dataset:

• --engine (choices=[ gpt-4 , gpt-35-turbo ]): the LLM engine used for generating the outline
• --do-research : if True, simulate conversation to research the topic; otherwise, load the results.
• --max-conv-turn : the maximum number of questions for each information-seeking conversation
• STORM also uses a general conversation to collect basic information about the topic. So, the maximum number of QA pairs is max_turn * (max_perspective + 1) . 💡 Reducing max_turn or max_perspective can speed up the process and reduce the cost but may result in less comprehensive outline.
• The parameter will not have any effect if --disable-perspective is set (the perspective-driven question asking is disabled).

To run the experiment on a single topic:

• The script will ask you to enter the Topic and the Ground truth url that will be excluded. If you do not have any url to exclude, leave that field empty.

The generated outline will be saved in {output_dir}/{topic}/storm_gen_outline.txt and the collected references will be saved in {output_dir}/{topic}/raw_search_results.json .

Writing Stage

• --do-polish-article : if True, polish the article by adding a summarization section and removing duplicate content if --remove-duplicate is set True.
• The script will ask you to enter the Topic . Please enter the same topic as the one used in the pre-writing stage.

The generated article will be saved in {output_dir}/{topic}/storm_gen_article.txt and the references corresponding to citation index will be saved in {output_dir}/{topic}/url_to_info.json . If --do-polish-article is set, the polished article will be saved in {output_dir}/{topic}/storm_gen_article_polished.txt .

Customize the STORM Configurations

We set up the default LLM configuration in LLMConfigs in src/modules/utils.py . You can use set_conv_simulator_lm() , set_question_asker_lm() , set_outline_gen_lm() , set_article_gen_lm() , set_article_polish_lm() to override the default configuration. These functions take in an instance from dspy.dsp.LM or dspy.dsp.HFModel .

Automatic Evaluation

In our paper, we break down the evaluation into two parts: outline quality and full-length article quality.

Outline Quality

We introduce heading soft recall and heading entity recall to evaluate the outline quality. This makes it easier to prototype methods for pre-writing.

Run the following command under ./eval to compute the metrics on FreshWiki dataset:

Full-length Article Quality

eval/eval_article_quality.py provides the entry point of evaluating full-length article quality using ROUGE, entity recall, and rubric grading. Run the following command under eval to compute the metrics:

Use the Metric Yourself

The similarity-based metrics (i.e., ROUGE, entity recall, and heading entity recall) are implemented in eval/metrics.py .

For rubric grading, we use the prometheus-13b-v1.0 introduced in this paper . eval/evaluation_prometheus.py provides the entry point of using the metric.

Roadmap & Contributions

Our team is actively working on:

• Human-in-the-Loop Functionalities: Supporting user participation in the knowledge curation process.
• Information Abstraction: Developing abstractions for curated information to support presentation formats beyond the Wikipedia-style report.

If you have any questions or suggestions, please feel free to open an issue or pull request. We welcome contributions to improve the system and the codebase!

Contact person: Yijia Shao and Yucheng Jiang

Acknowledgement

We would like to thank Wikipedia for their excellent open-source content. The FreshWiki dataset is sourced from Wikipedia, licensed under the Creative Commons Attribution-ShareAlike (CC BY-SA) license.

We are very grateful to Michelle Lam for designing the logo for this project and Dekun Ma for leading the UI development.

Please cite our paper if you use this code or part of it in your work:

Contributors 18

• Python 100.0%

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• NEWS EXPLAINER
• 28 August 2024

Mpox is spreading rapidly. Here are the questions researchers are racing to answer

• Sara Reardon

You can also search for this author in PubMed   Google Scholar

You have full access to this article via your institution.

Monkeypox virus particles (shown in this coloured electron micrograph) can spread through close contact with people and animals. Credit: NIAID/Science Photo Library

When the World Health Organization (WHO) declared a public-health emergency over mpox earlier this month , it was because a concerning form of the virus that causes the disease had spread to multiple African countries where it had never been seen before. Since then, two people travelling to Africa — one from Sweden and one from Thailand — have become infected with that type of virus, called clade Ib, and brought it back to their countries.

Monkeypox virus: dangerous strain gains ability to spread through sex, new data suggest

Although researchers have known about the current outbreak since late last year, the need for answers about it is now more pressing than ever. The Democratic Republic of the Congo (DRC) has spent decades grappling with monkeypox clade I virus — the lineage to which Ib belongs. But in the past, clade I infections usually arose when a person came into contact with wild animals, and outbreaks would fizzle out.

Clade Ib seems to be different, and is spreading largely through contact between humans, including through sex . Around 18,000 suspected cases of mpox, many of them among children, and at least 600 deaths potentially attributable to the disease have been reported this year in the DRC alone.

How does this emergency compare with one declared in 2022, when mpox cases spread around the globe? How is this virus behaving compared with the version that triggered that outbreak, a type called clade II? And will Africa be able to rein this one in? Nature talks to researchers about information they are rushing to gather.

Is clade Ib more deadly than the other virus types?

It’s hard to determine, says Jason Kindrachuk, a virologist at the University of Manitoba in Winnipeg, Canada. He says that the DRC is experiencing two outbreaks simultaneously. The clade I virus, which has been endemic in forested regions of the DRC for decades, circulates in rural regions, where people get it from animals. That clade was renamed Ia after the discovery of clade Ib. Studies in animals suggest that clade I is deadlier than clade II 1 — but Kindrachuk says that it’s hard to speculate on what that means for humans at this point.

Even when not fatal, mpox can trigger fevers, aches and painful fluid-filled skin lesions.

Growing mpox outbreak prompts WHO to declare global health emergency

Although many reports state that 10% of clade I infections in humans are fatal, infectious-disease researcher Laurens Liesenborghs at the Institute of Tropical Medicine in Antwerp, Belgium, doubts that this figure is accurate. Even the WHO’s latest estimate of a 3.5% fatality rate for people with mpox in the DRC might be high.

There are many reasons that fatality estimates might be unreliable, Liesenborghs says. For one, surveillance data capture only the most severe cases; many people who are less ill might not seek care at hospitals or through physicians, so their infections go unreported.

Another factor that can confound fatality rates is a secondary health condition. For example, people living with HIV — who can represent a large proportion of the population in many African countries — die from mpox at twice the rate of the general population 2 , especially if their HIV is untreated. And the relatively high death rate among children under age 5 could be partly because of malnutrition, which is common among kids in rural parts of the DRC, Liesenborghs says.

Is clade Ib more transmissible than other types?

The clade Ib virus has garnered particular attention because epidemiological data suggest that it transmits more readily between people than previous strains did, including through sexual activity, whereas clade Ia mostly comes from animals. An analysis posted ahead of peer review on the preprint server medRxiv 3 shows that clade Ib’s genome contains genetic mutations that seem to have been induced by the human immune system, suggesting that it has been in humans for some time. Clade Ia genomes have fewer of these mutations.

But Liesenborghs says that the mutations and clades might not be the most important factor in understanding how monkeypox virus spreads. Although distinguishing Ia from Ib is useful in tracking the disease, he says, the severity and transmissibility of the disease could be affected more by the region where the virus is circulating and the people there. Clade Ia, for instance, seems to be more common in sparsely populated rural regions where it is less likely to spread far. Clade Ib is cropping up in densely populated areas and spreading more readily.

Jean Nachega, an infectious-disease physician at the University of Pittsburgh in Pennsylvania, says that scientists don’t understand many aspects of mpox transmission — they haven’t even determined which animal serves as a reservoir for the virus in the wild, although rodents are able to carry it. “We have to be very humble,” Nachega says.

How effective are vaccines against the clade I virus?

Just as was the case during the COVID-19 pandemic, health experts are looking to vaccines to help curb this mpox outbreak. Although there are no vaccines designed specifically against the monkeypox virus, there are two vaccines proven to ward off a related poxvirus — the one that causes smallpox. Jynneos, made by biotechnology company Bavarian Nordic in Hellerup, Denmark, contains a type of poxvirus that can’t replicate but can trigger an immune response. LC16m8, made by pharmaceutical company KM Biologics in Kumamoto, Japan, contains a live — but weakened — version of a different poxvirus strain.

Hopes dashed for drug aimed at monkeypox virus spreading in Africa

Still, it’s unclear how effective these smallpox vaccines are against mpox generally. Dimie Ogoina, an infectious-disease specialist at Niger Delta University in Wilberforce Island, Nigeria, points out that vaccines have been tested only against clade II virus in European and US populations, because these shots were distributed by wealthy nations during the 2022 global outbreak . And those recipients were primarily young, healthy men who have sex with men, a population that was particularly susceptible during that outbreak. One study in the United States found that one dose of Jynneos was 80% effective at preventing the disease in at-risk people, whereas two doses were 82% effective 4 ; the WHO recommends getting both jabs.

People in Africa infected with either the clade Ia or Ib virus — especially children and those with compromised immune systems — might respond differently. However, one study in the DRC found that the Jynneos vaccine generally raised antibodies against mpox in about 1,000 health-care workers who received it 5 .

But researchers are trying to fill in some data gaps. A team in the DRC is about to launch a clinical trial of Jynneos in people who have come into close contact with the monkeypox virus — but have not shown symptoms — to see whether it can prevent future infection, or improve outcomes if an infection arises.

Will the vaccines help to rein in the latest outbreak?

Mpox vaccines have been largely unavailable in Africa, but several wealthy countries have pledged to donate doses to the DRC and other affected African nations. The United States has offered 50,000 Jynneos doses from its national stockpile, and the European Union has ordered 175,000, with individual member countries pledging extra doses. Bavarian Nordic has also added another 40,000. Japan has offered 3.5 million doses of LC16m8 — for which only one jab is recommended instead of two.

Monkeypox in Africa: the science the world ignored

None of them have arrived yet, though, says Espoir Bwenge Malembaka, an epidemiologist at the Catholic University of Bukavu in the DRC. Low- and middle-income nations cannot receive vaccines until the WHO has deemed the jabs safe and effective. And the WHO has not given its thumbs up yet. It is evaluating data from vaccine manufacturers, delaying donors’ ability to send the vaccines.

Even when the vaccines arrive, Bwenge Malembaka says, “it’s really a drop in the bucket”. The Africa Centres for Disease Control and Prevention in Addis Ababa, Ethiopia, estimates that 10 million doses are needed to rein in the outbreak.

Bwenge Malembaka says that the uncertainty over vaccine arrival has made it difficult for the government to form a distribution plan. “I don’t know how one can go about this kind of challenge,” he says. Bwenge Malembaka suspects that children are likely to receive doses first, because they are highly vulnerable to clade I, but officials haven’t decided which regions to target. It’s also unclear how the government would prioritize other vulnerable populations such as sex workers, who have been affected by clade Ib. Their profession is criminalized in the DRC, so they might not be able to come forward for treatment.

Researchers lament that public-health organizations didn’t provide vaccines and other resources as soon as the clade I outbreak was identified, especially given lessons learnt from the 2022 global mpox outbreak. “The opportunity was there a couple months ago to cut this transmission chain, but resources weren’t available,” Liesenborghs says. “Now, it will be more challenging to tackle this outbreak, and the population at risk is much broader.”

Nature 633 , 16-17 (2024)

doi: https://doi.org/10.1038/d41586-024-02793-9

Americo, J. L., Earl, P. L. & Moss, B. Proc. Natl Acad. Sci. USA 120 , e2220415120 (2023).

Article   PubMed   Google Scholar  

Yinka-Ogunleye, A. et al. BMJ Glob. Health 8 , e013126 (2023).

Kinganda-Lusamaki, E. et al. Preprint at medRxiv https://doi.org/10.1101/2024.08.13.24311951 (2024).

Yeganeh, N. et al. Vaccine 42 , 125987 (2024).

Priyamvada, L. et al. Vaccine 40 , 7321–7327 (2022).

Download references

Reprints and permissions

Related Articles

• Public health

Mapping glycoprotein structure reveals Flaviviridae evolutionary history

Article 04 SEP 24

Farmed fur animals harbour viruses with zoonotic spillover potential

Mysterious Oropouche virus is spreading: what you should know

News Q&A 26 AUG 24

Found: a brain-wiring pattern linked to depression

News 04 SEP 24

The hepatitis C virus envelope protein complex is a dimer of heterodimers

How rival weight-loss drugs fare at treating obesity, diabetes and more

News 03 SEP 24

What accelerates brain ageing? This AI ‘brain clock’ points to answers

News 27 AUG 24

Extreme heat is a huge killer — these local approaches can keep people safe

News 22 AUG 24

Postdoctoral Associate- Genetic Epidemiology

Houston, Texas (US)

Baylor College of Medicine (BCM)

NOMIS Foundation ETH Postdoctoral Fellowship

The NOMIS Foundation ETH Fellowship Programme supports postdoctoral researchers at ETH Zurich within the Centre for Origin and Prevalence of Life ...

Zurich, Canton of Zürich (CH)

Centre for Origin and Prevalence of Life at ETH Zurich

13 PhD Positions at Heidelberg University

GRK2727/1 – InCheck Innate Immune Checkpoints in Cancer and Tissue Damage

Heidelberg, Baden-Württemberg (DE) and Mannheim, Baden-Württemberg (DE)

Medical Faculties Mannheim & Heidelberg and DKFZ, Germany

Postdoctoral Associate- Environmental Epidemiology

Open faculty positions at the state key laboratory of brain cognition & brain-inspired intelligence.

The laboratory focuses on understanding the mechanisms of brain intelligence and developing the theory and techniques of brain-inspired intelligence.

Shanghai, China

CAS Center for Excellence in Brain Science and Intelligence Technology (CEBSIT)

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

• SUGGESTED TOPICS
• The Magazine
• Newsletters
• Managing Yourself
• Managing Teams
• Work-life Balance
• The Big Idea
• Data & Visuals
• Reading Lists
• Case Selections
• HBR Learning
• Topic Feeds
• Account Settings
• Email Preferences

How to Support an Employee in Distress

• Reut Livne-Tarandach
• Hooria Jazaieri

Focus on their experience — not your own.

Recent research has a counterintuitive finding: People who have suffered troubles at work may not be effective in helping other employees experiencing similar distress. Those who have not endured the same thing are more likely to be more effective. In trying to help employees in distress, leaders should  focus on each person’s experience, not their own; validate their pain; and get the facts and ask questions to learn how they might help the employee. Leaders should also consider having someone who has not suffered the same problem mentor the person.

Think back to the last time you were struggling at work. If you chose to share your distress with a colleague, to whom did you turn? Many of us intuitively look to our colleagues who have experienced our same struggles, but our recently published research suggests you should rethink this approach. In a series of three studies, more than 600 employees from multiple industries across the United States told us about their experiences in sharing their work-related angst with others and how they responded to others who were experiencing such difficulties. Our finding: It is not always wise to seek support from colleagues who have been in the same situation.

• Reut Livne-Tarandach is the Louis F. Capalbo Chair of Business Administration and an associate professor of management at Manhattan College’s O’Malley School of Business. She is also a faculty affiliate of the Center for Positive Organizations at the University of Michigan’s Stephen M. Ross School of Business and a research fellow at the University of Louisville’s Center for Positive Leadership.
• Hooria Jazaieri is an assistant professor of management at Santa Clara University’s Leavey School of Business. She also serves as a science advisor at the Greater Good Science Center at the University of California, Berkeley, and is a licensed psychotherapist in California.

Partner Center