drawbacks of empirical research

Empirical Research: Advantages, Drawbacks and Differences with Non-Empirical Research

Based on the purpose and available resources, researchers conduct empirical or non-empirical research. Researchers employ both of these methods in various fields using qualitative, quantitative, or secondary data. Let's look at the characteristics of empirical research and see how it is different from non-empirical research.

The empirical study is evidence-based research. That is to say, it uses evidence, experiment, or observation to test the hypotheses. It is a systematic collection and analysis of data. Empirical research allows researchers to find new and thorough insights into the issue.  Mariam-Webster dictionary defines the word "empirical" as:

                "originating in or based on observation or experience"

               "relying on experience or observation alone often without due regard for system and theory"

               "capable of being verified or disproved by observation or experiment"

Unlike non-empirical research, it does not just rely on theories but also tries to find the reasoning behind those theories in order to prove them. Non-empirical research is based on theories and logic, and researchers don't attempt to test them.  Although empirical research mostly depends on primary data, secondary data can also be beneficial for the theory side of the research.  The empirical research process includes the following:

• Defining the issue
• Theory generation and research questions
• If available, studying existing theories about the issue
• Choosing appropriate data collection methods  such as experiment or observation
• Data gathering
• Data coding , analysis, and evaluation
• Data Interpretation and result
• Reporting and publishing  the findings

Benefits of empirical research

• Empirical research aims to find the meaning behind a particular phenomenon. In other words, it seeks answers to how and why something works the way it is.
• By identifying the reasons why something happens, it is possible to replicate or prevent similar events.
• The flexibility of the research allows the researchers to change certain aspects of the research and adjust them to new goals. 
• It is more reliable because it represents a real-life experience and not just theories.
• Data collected through empirical research may be less biased because the researcher is there during the collection process. In contrast, it is sometimes impossible to verify the accuracy of data in non-empirical research.

Drawbacks of empirical research

• It can be time-consuming depending on the research subject.
• It is not a cost-effective way of data collection in most cases because of the possible expensive methods of data gathering. Moreover, it may require traveling between multiple locations.
• Lack of evidence and research subjects may not yield the desired result. A small sample size prevents generalization because it may not be enough to represent the target audience.
• It isn't easy to get information on sensitive topics, and also, researchers may need participants' consent to use the data.

In most scientific fields, acting based solely on theories (or logic) is not enough. Empirical research makes it possible to measure the reliability of the theory before applying it. Researchers sometimes alternate between the two forms of research, as non-empirical research provides them with important information about the phenomenon, while empirical research helps them use that information to test the theory.

• Earth Science
• Physical Science
• Social Science
• Medical Science
• Mathematics
• Paleontology

What are the Advantages and Disadvantages of Empiricism

Empiricism is the theory of knowledge that claims that most or all our knowledge is obtained through sensory experience, rather than through rational deduction or innateness. Empiricists such as John Locke and David Hume emphasize the role of evidence and experience as the main way of justifying our knowledge claims. Therefore knowledge gained a priori is considered by empiricists to be inferior to knowledge gained a posteriori. John Locke, along with many other empiricists, postulated the idea of Tabula Rasa, or the argument that we are a blank slate at birth and all the ideas and concepts that we have, build up as we experience more and more things.

The main strength of using empiricism as a way of finding truth is that rationalism doesn’t necessarily account for the way that the world really works, whereas empiricism does. Empiricism is widely used in science as a method of proving and disproving theories. This is backed up by Galileo who stated that beliefs must be tested empirically in order to check that they work within the laws of physics. An example of this is Aristotle’s theory of motion in which he used rational thought to explain the motion of objects. He argued that each of the four terrestrial (or worldly) elements move toward their natural place and that heavier things fell faster than lighter things. Galileo disputed this, arguing that it was air resistance that was responsible for how fast things fell. This was later tested empirically on the moon when an astronaut dropped a feather and a hammer and they hit the ground at the same time. This is a strong argument for empiricism because it shows that it is much easier to see if something is true if it is tested than if reason is used alone.

However, rationalists dispute the role of empirical evidence based on its claim that we can acquire knowledge through our senses. This is because sense data is indirect and there has to be mediation between sensation and perception. There is also no way of knowing if what we are seeing is reality. Many people have experienced hallucinations or lucid dreams in their lives, in which they have been convinced of the existence of things that don’t exist. If it feels like reality while you are in the dream, how do you know what you’re experiencing now isn’t also a dream? Descartes was a rationalist and argued that there is no way of knowing if the things we are seeing and experiencing are real. If this is the case, we cannot claim to have knowledge that when we saw the hammer and the feather fall at the same speed, that this was actually the case. The only thing, according to rationalists, that we can be sure of are things that logically cannot fail to be true and only our rational minds can provide us with this information. Empiricism cannot be proved to be accurate.

David Hume argues against the claim that sense data is not accurate. A strong argument supporting Hume’s empiricism is that rationalism can only link ideas, whereas empiricism can link facts and is therefore a more useful tool in justifying knowledge claims. Under normal circumstances our senses do not lie and the more we repeat something the better idea we have of it. Rationalism is not useful in proving things to be true because it relies on logic that may or may not be true in itself and cannot account for the real world. Empiricism claims that experience can show whether a phenomenon repeats itself and therefore it abides by certain laws or it happened randomly, which is why it can be considered such a good foundation as a way of uncovering and proving facts. Rationalism, on the other hand, can only give us ideas, that may appear to be correct at first but without experimentation there is no way of telling whether the claim is correct.

Although empiricism is strong in the subject of physics, it cannot be used for complicated mathematics and algebra. Some mathematical equations are impossible to prove using the empiricist method because there is no physical way of showing the complications of the equation in an observational experiment. Rationalism solves this problem by using logic to suggest that whilst some things cannot be tested in the real world, they cannot logically fail to be true. This is true in the case of not just mathematics, but a lot of science. There are some things that at this moment in time cannot be tested scientifically, but a working hypothesis has still been created and only rationalism could have produced this. For instance, if we use the example of Galileo who disproved Aristotle’s theory of motion without having the ability of physically going to the moon and testing it himself. He did this by using noticing the logical fallacy of Aristotle’s argument and correcting it so that it was more coherent. Empiricism is only useful if it is possible for one to actually experience something and as there is much we cannot experience ourselves, rationalism is an important source for a great deal of knowledge.

Overall, it is clear that empiricism has both strengths and weaknesses. It is vital in our understanding of the world and in proving or disproving beliefs, but it cannot be used for everything, especially for answering questions based on intangible things such as the mind or theoretical mathematics. For these kinds of things rationalism would be better used and the most justified knowledge claims are those that cohere to both rational thought and empirical evidence.

Related posts:

Empirical Research: A Comprehensive Guide for Academics 

Empirical research relies on gathering and studying real, observable data. The term ’empirical’ comes from the Greek word ’empeirikos,’ meaning ‘experienced’ or ‘based on experience.’ So, what is empirical research? Instead of using theories or opinions, empirical research depends on real data obtained through direct observation or experimentation. 

Why Empirical Research?

Empirical research plays a key role in checking or improving current theories, providing a systematic way to grow knowledge across different areas. By focusing on objectivity, it makes research findings more trustworthy, which is critical in research fields like medicine, psychology, economics, and public policy. In the end, the strengths of empirical research lie in deepening our awareness of the world and improving our capacity to tackle problems wisely. 1,2  

Qualitative and Quantitative Methods

There are two main types of empirical research methods – qualitative and quantitative. 3,4 Qualitative research delves into intricate phenomena using non-numerical data, such as interviews or observations, to offer in-depth insights into human experiences. In contrast, quantitative research analyzes numerical data to spot patterns and relationships, aiming for objectivity and the ability to apply findings to a wider context. 

Steps for Conducting Empirical Research

When it comes to conducting research, there are some simple steps that researchers can follow. 5,6  

• Create Research Hypothesis:  Clearly state the specific question you want to answer or the hypothesis you want to explore in your study. 
• Examine Existing Research:  Read and study existing research on your topic. Understand what’s already known, identify existing gaps in knowledge, and create a framework for your own study based on what you learn. 
• Plan Your Study:  Decide how you’ll conduct your research—whether through qualitative methods, quantitative methods, or a mix of both. Choose suitable techniques like surveys, experiments, interviews, or observations based on your research question. 
• Develop Research Instruments:  Create reliable research collection tools, such as surveys or questionnaires, to help you collate data. Ensure these tools are well-designed and effective. 
• Collect Data:  Systematically gather the information you need for your research according to your study design and protocols using the chosen research methods. 
• Data Analysis:  Analyze the collected data using suitable statistical or qualitative methods that align with your research question and objectives. 
• Interpret Results:  Understand and explain the significance of your analysis results in the context of your research question or hypothesis. 
• Draw Conclusions:  Summarize your findings and draw conclusions based on the evidence. Acknowledge any study limitations and propose areas for future research. 

Advantages of Empirical Research

Empirical research is valuable because it stays objective by relying on observable data, lessening the impact of personal biases. This objectivity boosts the trustworthiness of research findings. Also, using precise quantitative methods helps in accurate measurement and statistical analysis. This precision ensures researchers can draw reliable conclusions from numerical data, strengthening our understanding of the studied phenomena. 4  

Disadvantages of Empirical Research

While empirical research has notable strengths, researchers must also be aware of its limitations when deciding on the right research method for their study.4 One significant drawback of empirical research is the risk of oversimplifying complex phenomena, especially when relying solely on quantitative methods. These methods may struggle to capture the richness and nuances present in certain social, cultural, or psychological contexts. Another challenge is the potential for confounding variables or biases during data collection, impacting result accuracy.  

Tips for Empirical Writing

In empirical research, the writing is usually done in research papers, articles, or reports. The empirical writing follows a set structure, and each section has a specific role. Here are some tips for your empirical writing. 7   

• Define Your Objectives:  When you write about your research, start by making your goals clear. Explain what you want to find out or prove in a simple and direct way. This helps guide your research and lets others know what you have set out to achieve. 
• Be Specific in Your Literature Review:  In the part where you talk about what others have studied before you, focus on research that directly relates to your research question. Keep it short and pick studies that help explain why your research is important. This part sets the stage for your work. 
• Explain Your Methods Clearly : When you talk about how you did your research (Methods), explain it in detail. Be clear about your research plan, who took part, and what you did; this helps others understand and trust your study. Also, be honest about any rules you follow to make sure your study is ethical and reproducible. 
• Share Your Results Clearly : After doing your empirical research, share what you found in a simple way. Use tables or graphs to make it easier for your audience to understand your research. Also, talk about any numbers you found and clearly state if they are important or not. Ensure that others can see why your research findings matter. 
• Talk About What Your Findings Mean:  In the part where you discuss your research results, explain what they mean. Discuss why your findings are important and if they connect to what others have found before. Be honest about any problems with your study and suggest ideas for more research in the future. 
• Wrap It Up Clearly:  Finally, end your empirical research paper by summarizing what you found and why it’s important. Remind everyone why your study matters. Keep your writing clear and fix any mistakes before you share it. Ask someone you trust to read it and give you feedback before you finish. 

References:  

• Empirical Research in the Social Sciences and Education, Penn State University Libraries. Available online at  https://guides.libraries.psu.edu/emp  
• How to conduct empirical research, Emerald Publishing. Available online at  https://www.emeraldgrouppublishing.com/how-to/research-methods/conduct-empirical-research  
• Empirical Research: Quantitative & Qualitative, Arrendale Library, Piedmont University. Available online at  https://library.piedmont.edu/empirical-research  
• Bouchrika, I.  What Is Empirical Research? Definition, Types & Samples  in 2024. Research.com, January 2024. Available online at  https://research.com/research/what-is-empirical-research  
• Quantitative and Empirical Research vs. Other Types of Research. California State University, April 2023. Available online at  https://libguides.csusb.edu/quantitative  
• Empirical Research, Definitions, Methods, Types and Examples, Studocu.com website. Available online at  https://www.studocu.com/row/document/uganda-christian-university/it-research-methods/emperical-research-definitions-methods-types-and-examples/55333816  
• Writing an Empirical Paper in APA Style. Psychology Writing Center, University of Washington. Available online at  https://psych.uw.edu/storage/writing_center/APApaper.pdf  

Paperpal is an AI writing assistant that help academics write better, faster with real-time suggestions for in-depth language and grammar correction. Trained on millions of research manuscripts enhanced by professional academic editors, Paperpal delivers human precision at machine speed.  

Try it for free or upgrade to  Paperpal Prime , which unlocks unlimited access to premium features like academic translation, paraphrasing, contextual synonyms, consistency checks and more. It’s like always having a professional academic editor by your side! Go beyond limitations and experience the future of academic writing.  Get Paperpal Prime now at just US$19 a month!  

Related Reads:

• How to Write a Scientific Paper in 10 Steps 
• What is a Literature Review? How to Write It (with Examples)
• What is an Argumentative Essay? How to Write It (With Examples)
• Ethical Research Practices For Research with Human Subjects

Ethics in Science: Importance, Principles & Guidelines 

Presenting research data effectively through tables and figures, you may also like, academic integrity vs academic dishonesty: types & examples, dissertation printing and binding | types & comparison , what is a dissertation preface definition and examples , the ai revolution: authors’ role in upholding academic..., the future of academia: how ai tools are..., how to write a research proposal: (with examples..., how to write your research paper in apa..., how to choose a dissertation topic, how to write a phd research proposal, how to write an academic paragraph (step-by-step guide).

• Skip to main content
• Skip to primary sidebar
• Skip to footer
• QuestionPro

• Solutions Industries Gaming Automotive Sports and events Education Government Travel & Hospitality Financial Services Healthcare Cannabis Technology Use Case AskWhy Communities Audience Contactless surveys Mobile LivePolls Member Experience GDPR Positive People Science 360 Feedback Surveys
• Resources Blog eBooks Survey Templates Case Studies Training Help center

Home Market Research

Empirical Research: Definition, Methods, Types and Examples

Content Index

Empirical research: Definition

Empirical research: origin, quantitative research methods, qualitative research methods, steps for conducting empirical research, empirical research methodology cycle, advantages of empirical research, disadvantages of empirical research, why is there a need for empirical research.

Empirical research is defined as any research where conclusions of the study is strictly drawn from concretely empirical evidence, and therefore “verifiable” evidence.

This empirical evidence can be gathered using quantitative market research and  qualitative market research  methods.

For example: A research is being conducted to find out if listening to happy music in the workplace while working may promote creativity? An experiment is conducted by using a music website survey on a set of audience who are exposed to happy music and another set who are not listening to music at all, and the subjects are then observed. The results derived from such a research will give empirical evidence if it does promote creativity or not.

LEARN ABOUT: Behavioral Research

You must have heard the quote” I will not believe it unless I see it”. This came from the ancient empiricists, a fundamental understanding that powered the emergence of medieval science during the renaissance period and laid the foundation of modern science, as we know it today. The word itself has its roots in greek. It is derived from the greek word empeirikos which means “experienced”.

In today’s world, the word empirical refers to collection of data using evidence that is collected through observation or experience or by using calibrated scientific instruments. All of the above origins have one thing in common which is dependence of observation and experiments to collect data and test them to come up with conclusions.

LEARN ABOUT: Causal Research

Types and methodologies of empirical research

Empirical research can be conducted and analysed using qualitative or quantitative methods.

• Quantitative research : Quantitative research methods are used to gather information through numerical data. It is used to quantify opinions, behaviors or other defined variables . These are predetermined and are in a more structured format. Some of the commonly used methods are survey, longitudinal studies, polls, etc
• Qualitative research:   Qualitative research methods are used to gather non numerical data.  It is used to find meanings, opinions, or the underlying reasons from its subjects. These methods are unstructured or semi structured. The sample size for such a research is usually small and it is a conversational type of method to provide more insight or in-depth information about the problem Some of the most popular forms of methods are focus groups, experiments, interviews, etc.

Data collected from these will need to be analysed. Empirical evidence can also be analysed either quantitatively and qualitatively. Using this, the researcher can answer empirical questions which have to be clearly defined and answerable with the findings he has got. The type of research design used will vary depending on the field in which it is going to be used. Many of them might choose to do a collective research involving quantitative and qualitative method to better answer questions which cannot be studied in a laboratory setting.

LEARN ABOUT: Qualitative Research Questions and Questionnaires

Quantitative research methods aid in analyzing the empirical evidence gathered. By using these a researcher can find out if his hypothesis is supported or not.

• Survey research: Survey research generally involves a large audience to collect a large amount of data. This is a quantitative method having a predetermined set of closed questions which are pretty easy to answer. Because of the simplicity of such a method, high responses are achieved. It is one of the most commonly used methods for all kinds of research in today’s world.

Previously, surveys were taken face to face only with maybe a recorder. However, with advancement in technology and for ease, new mediums such as emails , or social media have emerged.

For example: Depletion of energy resources is a growing concern and hence there is a need for awareness about renewable energy. According to recent studies, fossil fuels still account for around 80% of energy consumption in the United States. Even though there is a rise in the use of green energy every year, there are certain parameters because of which the general population is still not opting for green energy. In order to understand why, a survey can be conducted to gather opinions of the general population about green energy and the factors that influence their choice of switching to renewable energy. Such a survey can help institutions or governing bodies to promote appropriate awareness and incentive schemes to push the use of greener energy.

Learn more: Renewable Energy Survey Template Descriptive Research vs Correlational Research

• Experimental research: In experimental research , an experiment is set up and a hypothesis is tested by creating a situation in which one of the variable is manipulated. This is also used to check cause and effect. It is tested to see what happens to the independent variable if the other one is removed or altered. The process for such a method is usually proposing a hypothesis, experimenting on it, analyzing the findings and reporting the findings to understand if it supports the theory or not.

For example: A particular product company is trying to find what is the reason for them to not be able to capture the market. So the organisation makes changes in each one of the processes like manufacturing, marketing, sales and operations. Through the experiment they understand that sales training directly impacts the market coverage for their product. If the person is trained well, then the product will have better coverage.

• Correlational research: Correlational research is used to find relation between two set of variables . Regression analysis is generally used to predict outcomes of such a method. It can be positive, negative or neutral correlation.

LEARN ABOUT: Level of Analysis

For example: Higher educated individuals will get higher paying jobs. This means higher education enables the individual to high paying job and less education will lead to lower paying jobs.

• Longitudinal study: Longitudinal study is used to understand the traits or behavior of a subject under observation after repeatedly testing the subject over a period of time. Data collected from such a method can be qualitative or quantitative in nature.

For example: A research to find out benefits of exercise. The target is asked to exercise everyday for a particular period of time and the results show higher endurance, stamina, and muscle growth. This supports the fact that exercise benefits an individual body.

• Cross sectional: Cross sectional study is an observational type of method, in which a set of audience is observed at a given point in time. In this type, the set of people are chosen in a fashion which depicts similarity in all the variables except the one which is being researched. This type does not enable the researcher to establish a cause and effect relationship as it is not observed for a continuous time period. It is majorly used by healthcare sector or the retail industry.

For example: A medical study to find the prevalence of under-nutrition disorders in kids of a given population. This will involve looking at a wide range of parameters like age, ethnicity, location, incomes  and social backgrounds. If a significant number of kids coming from poor families show under-nutrition disorders, the researcher can further investigate into it. Usually a cross sectional study is followed by a longitudinal study to find out the exact reason.

• Causal-Comparative research : This method is based on comparison. It is mainly used to find out cause-effect relationship between two variables or even multiple variables.

For example: A researcher measured the productivity of employees in a company which gave breaks to the employees during work and compared that to the employees of the company which did not give breaks at all.

LEARN ABOUT: Action Research

Some research questions need to be analysed qualitatively, as quantitative methods are not applicable there. In many cases, in-depth information is needed or a researcher may need to observe a target audience behavior, hence the results needed are in a descriptive analysis form. Qualitative research results will be descriptive rather than predictive. It enables the researcher to build or support theories for future potential quantitative research. In such a situation qualitative research methods are used to derive a conclusion to support the theory or hypothesis being studied.

LEARN ABOUT: Qualitative Interview

• Case study: Case study method is used to find more information through carefully analyzing existing cases. It is very often used for business research or to gather empirical evidence for investigation purpose. It is a method to investigate a problem within its real life context through existing cases. The researcher has to carefully analyse making sure the parameter and variables in the existing case are the same as to the case that is being investigated. Using the findings from the case study, conclusions can be drawn regarding the topic that is being studied.

For example: A report mentioning the solution provided by a company to its client. The challenges they faced during initiation and deployment, the findings of the case and solutions they offered for the problems. Such case studies are used by most companies as it forms an empirical evidence for the company to promote in order to get more business.

• Observational method:   Observational method is a process to observe and gather data from its target. Since it is a qualitative method it is time consuming and very personal. It can be said that observational research method is a part of ethnographic research which is also used to gather empirical evidence. This is usually a qualitative form of research, however in some cases it can be quantitative as well depending on what is being studied.

For example: setting up a research to observe a particular animal in the rain-forests of amazon. Such a research usually take a lot of time as observation has to be done for a set amount of time to study patterns or behavior of the subject. Another example used widely nowadays is to observe people shopping in a mall to figure out buying behavior of consumers.

• One-on-one interview: Such a method is purely qualitative and one of the most widely used. The reason being it enables a researcher get precise meaningful data if the right questions are asked. It is a conversational method where in-depth data can be gathered depending on where the conversation leads.

For example: A one-on-one interview with the finance minister to gather data on financial policies of the country and its implications on the public.

• Focus groups: Focus groups are used when a researcher wants to find answers to why, what and how questions. A small group is generally chosen for such a method and it is not necessary to interact with the group in person. A moderator is generally needed in case the group is being addressed in person. This is widely used by product companies to collect data about their brands and the product.

For example: A mobile phone manufacturer wanting to have a feedback on the dimensions of one of their models which is yet to be launched. Such studies help the company meet the demand of the customer and position their model appropriately in the market.

• Text analysis: Text analysis method is a little new compared to the other types. Such a method is used to analyse social life by going through images or words used by the individual. In today’s world, with social media playing a major part of everyone’s life, such a method enables the research to follow the pattern that relates to his study.

For example: A lot of companies ask for feedback from the customer in detail mentioning how satisfied are they with their customer support team. Such data enables the researcher to take appropriate decisions to make their support team better.

Sometimes a combination of the methods is also needed for some questions that cannot be answered using only one type of method especially when a researcher needs to gain a complete understanding of complex subject matter.

We recently published a blog that talks about examples of qualitative data in education ; why don’t you check it out for more ideas?

Learn More: Data Collection Methods: Types & Examples

Since empirical research is based on observation and capturing experiences, it is important to plan the steps to conduct the experiment and how to analyse it. This will enable the researcher to resolve problems or obstacles which can occur during the experiment.

Step #1: Define the purpose of the research

This is the step where the researcher has to answer questions like what exactly do I want to find out? What is the problem statement? Are there any issues in terms of the availability of knowledge, data, time or resources. Will this research be more beneficial than what it will cost.

Before going ahead, a researcher has to clearly define his purpose for the research and set up a plan to carry out further tasks.

Step #2 : Supporting theories and relevant literature

The researcher needs to find out if there are theories which can be linked to his research problem . He has to figure out if any theory can help him support his findings. All kind of relevant literature will help the researcher to find if there are others who have researched this before, or what are the problems faced during this research. The researcher will also have to set up assumptions and also find out if there is any history regarding his research problem

Step #3: Creation of Hypothesis and measurement

Before beginning the actual research he needs to provide himself a working hypothesis or guess what will be the probable result. Researcher has to set up variables, decide the environment for the research and find out how can he relate between the variables.

Researcher will also need to define the units of measurements, tolerable degree for errors, and find out if the measurement chosen will be acceptable by others.

Step #4: Methodology, research design and data collection

In this step, the researcher has to define a strategy for conducting his research. He has to set up experiments to collect data which will enable him to propose the hypothesis. The researcher will decide whether he will need experimental or non experimental method for conducting the research. The type of research design will vary depending on the field in which the research is being conducted. Last but not the least, the researcher will have to find out parameters that will affect the validity of the research design. Data collection will need to be done by choosing appropriate samples depending on the research question. To carry out the research, he can use one of the many sampling techniques. Once data collection is complete, researcher will have empirical data which needs to be analysed.

LEARN ABOUT: Best Data Collection Tools

Step #5: Data Analysis and result

Data analysis can be done in two ways, qualitatively and quantitatively. Researcher will need to find out what qualitative method or quantitative method will be needed or will he need a combination of both. Depending on the unit of analysis of his data, he will know if his hypothesis is supported or rejected. Analyzing this data is the most important part to support his hypothesis.

Step #6: Conclusion

A report will need to be made with the findings of the research. The researcher can give the theories and literature that support his research. He can make suggestions or recommendations for further research on his topic.

A.D. de Groot, a famous dutch psychologist and a chess expert conducted some of the most notable experiments using chess in the 1940’s. During his study, he came up with a cycle which is consistent and now widely used to conduct empirical research. It consists of 5 phases with each phase being as important as the next one. The empirical cycle captures the process of coming up with hypothesis about how certain subjects work or behave and then testing these hypothesis against empirical data in a systematic and rigorous approach. It can be said that it characterizes the deductive approach to science. Following is the empirical cycle.

• Observation: At this phase an idea is sparked for proposing a hypothesis. During this phase empirical data is gathered using observation. For example: a particular species of flower bloom in a different color only during a specific season.
• Induction: Inductive reasoning is then carried out to form a general conclusion from the data gathered through observation. For example: As stated above it is observed that the species of flower blooms in a different color during a specific season. A researcher may ask a question “does the temperature in the season cause the color change in the flower?” He can assume that is the case, however it is a mere conjecture and hence an experiment needs to be set up to support this hypothesis. So he tags a few set of flowers kept at a different temperature and observes if they still change the color?
• Deduction: This phase helps the researcher to deduce a conclusion out of his experiment. This has to be based on logic and rationality to come up with specific unbiased results.For example: In the experiment, if the tagged flowers in a different temperature environment do not change the color then it can be concluded that temperature plays a role in changing the color of the bloom.
• Testing: This phase involves the researcher to return to empirical methods to put his hypothesis to the test. The researcher now needs to make sense of his data and hence needs to use statistical analysis plans to determine the temperature and bloom color relationship. If the researcher finds out that most flowers bloom a different color when exposed to the certain temperature and the others do not when the temperature is different, he has found support to his hypothesis. Please note this not proof but just a support to his hypothesis.
• Evaluation: This phase is generally forgotten by most but is an important one to keep gaining knowledge. During this phase the researcher puts forth the data he has collected, the support argument and his conclusion. The researcher also states the limitations for the experiment and his hypothesis and suggests tips for others to pick it up and continue a more in-depth research for others in the future. LEARN MORE: Population vs Sample

LEARN MORE: Population vs Sample

There is a reason why empirical research is one of the most widely used method. There are a few advantages associated with it. Following are a few of them.

• It is used to authenticate traditional research through various experiments and observations.
• This research methodology makes the research being conducted more competent and authentic.
• It enables a researcher understand the dynamic changes that can happen and change his strategy accordingly.
• The level of control in such a research is high so the researcher can control multiple variables.
• It plays a vital role in increasing internal validity .

Even though empirical research makes the research more competent and authentic, it does have a few disadvantages. Following are a few of them.

• Such a research needs patience as it can be very time consuming. The researcher has to collect data from multiple sources and the parameters involved are quite a few, which will lead to a time consuming research.
• Most of the time, a researcher will need to conduct research at different locations or in different environments, this can lead to an expensive affair.
• There are a few rules in which experiments can be performed and hence permissions are needed. Many a times, it is very difficult to get certain permissions to carry out different methods of this research.
• Collection of data can be a problem sometimes, as it has to be collected from a variety of sources through different methods.

LEARN ABOUT:  Social Communication Questionnaire

Empirical research is important in today’s world because most people believe in something only that they can see, hear or experience. It is used to validate multiple hypothesis and increase human knowledge and continue doing it to keep advancing in various fields.

For example: Pharmaceutical companies use empirical research to try out a specific drug on controlled groups or random groups to study the effect and cause. This way, they prove certain theories they had proposed for the specific drug. Such research is very important as sometimes it can lead to finding a cure for a disease that has existed for many years. It is useful in science and many other fields like history, social sciences, business, etc.

LEARN ABOUT: 12 Best Tools for Researchers

With the advancement in today’s world, empirical research has become critical and a norm in many fields to support their hypothesis and gain more knowledge. The methods mentioned above are very useful for carrying out such research. However, a number of new methods will keep coming up as the nature of new investigative questions keeps getting unique or changing.

Create a single source of real data with a built-for-insights platform. Store past data, add nuggets of insights, and import research data from various sources into a CRM for insights. Build on ever-growing research with a real-time dashboard in a unified research management platform to turn insights into knowledge.

LEARN MORE         FREE TRIAL

MORE LIKE THIS

Participant Engagement: Strategies + Improving Interaction

Sep 12, 2024

Employee Recognition Programs: A Complete Guide

Sep 11, 2024

A guide to conducting agile qualitative research for rapid insights with Digsite 

Cultural Insights: What it is, Importance + How to Collect?

Sep 10, 2024

Other categories

• Academic Research
• Artificial Intelligence
• Assessments
• Brand Awareness
• Case Studies
• Communities
• Consumer Insights
• Customer effort score
• Customer Engagement
• Customer Experience
• Customer Loyalty
• Customer Research
• Customer Satisfaction
• Employee Benefits
• Employee Engagement
• Employee Retention
• Friday Five
• General Data Protection Regulation
• Insights Hub
• Life@QuestionPro
• Market Research
• Mobile diaries
• Mobile Surveys
• New Features
• Online Communities
• Question Types
• Questionnaire
• QuestionPro Products
• Release Notes
• Research Tools and Apps
• Revenue at Risk
• Survey Templates
• Training Tips
• Tuesday CX Thoughts (TCXT)
• Uncategorized
• What’s Coming Up
• Workforce Intelligence

What is Empirical Research? Definition, Types, and More

Navigate data's complexities with empirical research, distinguishing truth from speculation. Explore types, methods, and more.

Godi Yeshaswi

January 12, 2024

In this Article

Research is crucial in many fields, involving a systematic exploration to confirm facts or draw specific conclusions. Empirical research, widely applied in different areas, aims to validate new facts. Grasping the significance of empirical research and knowing how to carry it out can aid in making decisions backed by a thorough investigation. 

What Do You Mean by Empirical Research?

The empirical research method is a study based on observation and direct experience to understand phenomena and draw conclusions based on real-world observations.

Empirical Research Examples

Consider a scenario where a study aims to determine if people add a product to their online cart due to product ratings. To investigate this, an experiment is carried out using an online shopping attitude survey . One group of participants is exposed to ratings, while another group is not exposed to any product ratings. The researchers then observe the behavior of these groups. The findings from this research will provide concrete evidence on whether product ratings impact the decision to purchase.

Types of Methodologies for Empirical Research

Quantitative research.

Quantitative research collects numerical data to analyze specific behaviors, opinions, or defined variables . Here are some methods used in quantitative empirical research:

This calls for collecting information from a group of people using a questionnaire. When conducting surveys, it's essential to pose straightforward, brief, and easy questions for participants to respond to. Survey participants can provide their answers through various channels, whether it be on paper, online through emails, or on social media. Administering surveys is generally a straightforward approach to obtaining information, whether from the general public or a specific audience.

Experimental Research

This process includes forming an idea and checking it through experimentation. Researchers can change one variable and see how it impacts other variables, helping them figure out if there's a clear connection. They can then examine the findings to confirm if their initial idea is correct.

Longitudinal Study

A longitudinal study involves observing a subject's characteristics or actions by testing them repeatedly over a period. The data collected from this method can be either qualitative or quantitative. For instance, marketers could track the buying patterns of a particular demographic, such as young adults, over several years. By repeatedly collecting data on their product design preferences, brand loyalty, and spending habits, researchers can gain insights into how these factors evolve over time. 

Cross-Sectional Research

Cross-sectional research is a way of studying people by looking at them during a particular time. In this method, researchers pick a group of individuals with similar characteristics, excluding the ones they are studying. This helps ensure that any findings are likely caused by the variable under investigation. For instance, researchers assess consumer preferences for different packaging designs at a specific time. Participants from the target market evaluate various options, providing immediate feedback. This approach offers a quick snapshot of consumer opinions on packaging, helping companies make informed decisions based on current preferences.

Correlational Research

Correlational research is a method used to find connections and prevalence among different factors. It often uses regression as a statistical tool to predict outcomes, showing whether there's a negative, neutral, or positive correlation between variables. For example, researchers might explore the relationship between how much time individuals spend watching television and their overall well-being. By collecting data on both variables from a diverse group of participants, the researchers can analyze whether there is a correlation between the time spent watching TV and factors like happiness or stress levels.

Qualitative Research

Qualitative research is useful for collecting information that isn't in numbers or can't be measured easily. It usually involves semi-structured or unstructured approaches, letting researchers uncover personal meanings, reasons, and opinions from participants. Qualitative empirical research often involves a small group of people and conversational methods to get detailed information and deeper insights into a problem. Examples of methods used in qualitative research include:

Observational Method

It involves watching and collecting descriptive information about a subject. The observational method gives researchers personal insights, helping them form detailed opinions about their studies. It's commonly used in ethnographic research, which looks at the culture of different groups of people.

One-on-One Interview

This is an entirely qualitative method that includes directly talking to a subject. Researchers often use it to get accurate and meaningful information about a subject. It's a conversational approach where specific questions are asked to guide the discussion. 

Focus Group

Focus groups are employed when researchers seek answers to questions of why, what, and how. A small group is typically chosen, and in-person interaction may not be necessary. If an in-person discussion is involved, a moderator is usually required. This method is commonly utilized by product companies to gather information about their brands and products.

For instance, in media/ad testing with focus groups, a company may evaluate a new soft drink advertisement. A small group views different ad versions and discusses their impressions, preferences, and memorable elements. This feedback helps the company refine its advertising strategy before a wider campaign launch.

Text Analysis

This qualitative empirical research method enables the analysis of an individual's social life . It's a contemporary approach leveraging the growing importance of social media and technology. Researchers can examine the specific words and images an individual uses to draw meaningful conclusions.

How to Conduct Empirical Research?

Empirical research relies on observation and experiences, so planning and analysis are crucial. Let’s take an example of media/ad/shopper testing as the research base to understand the steps to conduct empirical research - 

Step 1: Define the Research Objective

Clearly outline the study's goal, such as evaluating the effectiveness of a new packaging design for a consumer product or an advertisement of a new series. Consider potential issues with the resources schedule and ensure the study's benefits justify the costs.

Step 2: Review Relevant Literature and Theories

Identify theories or previous studies on consumer responses to packaging changes or new series ad releases. Understand how these insights can inform the study's outcomes.

Step 3: Formulate Hypothesis and Measurements

Develop an initial hypothesis, considering variables like consumer perception, brand appeal, and market competitiveness. Define units of measurement, such as consumer preferences and purchasing behavior, ensuring they align with industry standards.

Step 4: Define Research Design, Methodology, and Data Collection Techniques

Choose an appropriate research approach, whether qualitative research or quantitative research , to assess consumer reactions to the new packaging. Consider using focus groups and one-on-one interviews for in-depth insights and gather data on consumer reactions.

Step #5: Conduct Data Analysis and Frame the Results

Analyze the collected data, considering both quantitative metrics and qualitative feedback from focus groups and interviews. Assess whether the new packaging positively influences consumer perceptions and purchasing decisions. 

Evaluate consumer research tools powered by Insights AI that are powered by AI to give you unbiased feedback considering the emotions and behaviour of the respondent. 

Step 6: Draw Conclusions

Prepare a comprehensive report presenting the findings, including the impact of the new packaging or advertisement on consumer behavior. If sharing the results widely, convert the report into an article for publication and recommend further research areas in the packaging and media testing domain. Use a plagiarism checker to ensure the originality and credibility of the research.

You can also utilize the Gen AI feature in Decode to draw conclusions from your studies by just asking the Decode co-pilot, a virtual assistant. 

{{cta-button}}

Empirical Research Cycle

Observation

A media researcher observes audience reactions to a new television show by monitoring social media comments, ratings, and viewership numbers. This initial data collection serves as the basis for forming hypotheses about the show's popularity.

Based on the observations, the researcher may induce a hypothesis that suggests the show's popularity is linked to its engaging storyline and relatable characters. This assumption is then examined and tested against the collected data.

Using deductive reasoning, the researcher concludes that if the show's popularity is consistently associated with positive audience engagement and high ratings, it can be inferred that engaging content is a significant factor.

To test the hypothesis, the researcher designs a survey asking viewers about their reasons for liking the show and analyzes the responses. Statistical methods are employed to determine if there's a significant correlation between positive viewer feedback and the show's popularity.

In the final stage, the researcher evaluates the survey results, considering the empirical data, viewer comments, and any challenges encountered during the research. The findings are used to draw conclusions about the factors contributing to the show's success, and this information becomes the basis for further media testing or content development.

{{cta-case}}

Advantages and Disadvantages of Empirical Research

Advantages of empirical research.

Empirical research is widely used for several reasons, and here are some of its advantages:

• Authentication of Traditional Research: It validates traditional research through experiments and observations.
• Enhanced Competence and Authenticity: This methodology enhances the competency and authenticity of the conducted research.
• Adaptability to Dynamic Changes: Researchers can understand and adapt to dynamic changes by utilizing empirical research and adjusting their strategies accordingly.
• High Control Level: Empirical research offers a high level of control, allowing researchers to manage multiple variables.
• Increased Internal Validity: It plays a crucial role in boosting internal validity, ensuring the accuracy of the research outcomes.

Disadvantages of Empirical Research

While empirical research brings competency and authenticity, it also has some drawbacks:

• Time-Consuming Nature: Collecting data from various sources and dealing with numerous parameters can make this research time-consuming requiring patience.
• Costly Endeavor: Conducting research in different locations or environments may lead to increased expenses.
• Permission Challenges: Obtaining consent for certain experimental methods can be difficult, as there are strict rules governing their execution.
• Data Collection Challenges: Collecting data from various sources through different methods can be problematic at times.

Bottom Line

In a world full of data, empirical research is crucial for finding out what's true. It involves carefully observing and experiencing things to draw conclusions based on real-world evidence. This type of research uses both numbers (quantitative) and descriptions (qualitative) to understand various topics.

To conduct empirical research, you need a step-by-step plan. This includes setting clear goals, looking at existing research, making educated guesses (hypotheses), picking the right methods, analyzing data, and reaching sensible conclusions.

The research cycle involves watching, making guesses, drawing logical conclusions, testing those guesses, and finally evaluating everything.

While empirical research has benefits like proving traditional research, increasing competence, and adapting to changes, it also has challenges like being time-consuming, expensive, and dealing with permission and data collection issues.

In summary, understanding and using empirical research helps us make informed decisions in different fields by carefully studying and validating information through a systematic process.

Frequently Asked Questions

What do you mean by empirical research.

Empirical research is a type of study that relies on observing and measuring real-life phenomena as directly witnessed by the researcher. The collected data can be analyzed in relation to a theory or hypothesis, but the conclusions are grounded in actual experiences.

Theoretical vs Empirical Research

Empirical refers to information derived from observations or personal experiences, while theoretical is associated with ideas and hypotheses. In research contexts, these terms are commonly used to describe data, methods, or probabilities.

What are the benefits of Empirical Research?

Empirical research strives to understand the significance of a specific phenomenon. In simpler terms, it seeks to uncover how and why something operates the way it does. By pinpointing the reasons behind occurrences, it becomes feasible to reproduce or avoid similar events.

Is Empirical quantitative or qualitative?

Empirical research is often thought of as the same as quantitative research, but to be precise, it's any research that relies on direct observation.

Empirical Method Psychology Example

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.

With lots of unique blocks, you can easily build a page without coding.

Click on Study templates

Start from scratch

Add blocks to the content

Saving the Template

Publish the Template

Suppose a researcher aims to investigate the impact of listening to happy music on promoting prosocial behavior. In this scenario, an empirical analysis could involve conducting an experiment where one group of participants is exposed to happy music while another group is not exposed to any music at all.

Yeshaswi is a dedicated and enthusiastic individual with a strong affinity for tech and all things content. When he's not at work, he channels his passion into his love for football, especially for F.C. Barcelona and the GOAT, Lionel Messi. Instead of hitting the town for parties, he prefers to spend quality time cuddling with his Golden Retriever, Oreo.

Product Marketing Specialist

Related Articles

Top AI Events You Do Not Want to Miss in 2024

Here are all the top AI events for 2024, curated in one convenient place just for you.

Top Insights Events You Do Not Want to Miss in 2024

Here are all the top Insights events for 2024, curated in one convenient place just for you.

Top CX Events You Do Not Want to Miss in 2024

Here are all the top CX events for 2024, curated in one convenient place just for you.

How to Build an Experience Map: A Complete Guide

An experience map is essential for businesses, as it highlights the customer journey, uncovering insights to improve user experiences and address pain points. Read to find more!

Everything You Need to Know about Intelligent Scoring

Are you curious about Intelligent Scoring and how it differs from regular scoring? Discover its applications and benefits. Read on to learn more!

Qualitative Research Methods and Its Advantages In Modern User Research

Discover how to leverage qualitative research methods, including moderated sessions, to gain deep user insights and enhance your product and UX decisions.

The 10 Best Customer Experience Platforms to Transform Your CX

Explore the top 10 CX platforms to revolutionize customer interactions, enhance satisfaction, and drive business growth.

TAM SAM SOM: What It Means and How to Calculate It?

Understanding TAM, SAM, SOM helps businesses gauge market potential. Learn their definitions and how to calculate them for better business decisions and strategy.

Understanding Likert Scales: Advantages, Limitations, and Questions

Using Likert scales can help you understand how your customers view and rate your product. Here's how you can use them to get the feedback you need.

Mastering the 80/20 Rule to Transform User Research

Find out how the Pareto Principle can optimize your user research processes and lead to more impactful results with the help of AI.

Understanding Consumer Psychology: The Science Behind What Makes Us Buy

Gain a comprehensive understanding of consumer psychology and learn how to apply these insights to inform your research and strategies.

A Guide to Website Footers: Best Design Practices & Examples

Explore the importance of website footers, design best practices, and how to optimize them using UX research for enhanced user engagement and navigation.

Customer Effort Score: Definition, Examples, Tips

A great customer score can lead to dedicated, engaged customers who can end up being loyal advocates of your brand. Here's what you need to know about it.

How to Detect and Address User Pain Points for Better Engagement

Understanding user pain points can help you provide a seamless user experiences that makes your users come back for more. Here's what you need to know about it.

What is Quota Sampling? Definition, Types, Examples, and How to Use It?

Discover Quota Sampling: Learn its process, types, and benefits for accurate consumer insights and informed marketing decisions. Perfect for researchers and brand marketers!

What Is Accessibility Testing? A Comprehensive Guide

Ensure inclusivity and compliance with accessibility standards through thorough testing. Improve user experience and mitigate legal risks. Learn more.

Maximizing Your Research Efficiency with AI Transcriptions

Explore how AI transcription can transform your market research by delivering precise and rapid insights from audio and video recordings.

Understanding the False Consensus Effect: How to Manage it

The false consensus effect can cause incorrect assumptions and ultimately, the wrong conclusions. Here's how you can overcome it.

5 Banking Customer Experience Trends to Watch Out for in 2024

Discover the top 5 banking customer experience trends to watch out for in 2024. Stay ahead in the evolving financial landscape.

The Ultimate Guide to Regression Analysis: Definition, Types, Usage & Advantages

Master Regression Analysis: Learn types, uses & benefits in consumer research for precise insights & strategic decisions.

EyeQuant Alternative

Meet Qatalyst, your best eyequant alternative to improve user experience and an AI-powered solution for all your user research needs.

EyeSee Alternative

Embrace the Insights AI revolution: Meet Decode, your innovative solution for consumer insights, offering a compelling alternative to EyeSee.

Skeuomorphism in UX Design: Is It Dead?

Skeuomorphism in UX design creates intuitive interfaces using familiar real-world visuals to help users easily understand digital products. Do you know how?

Top 6 Wireframe Tools and Ways to Test Your Designs

Wireframe tools assist designers in planning and visualizing the layout of their websites. Look through this list of wireframing tools to find the one that suits you best.

Revolutionizing Customer Interaction: The Power of Conversational AI

Conversational AI enhances customer service across various industries, offering intelligent, context-aware interactions that drive efficiency and satisfaction. Here's how.

User Story Mapping: A Powerful Tool for User-Centered Product Development

Learn about user story mapping and how it can be used for successful product development with this blog.

What is Research Hypothesis: Definition, Types, and How to Develop

Read the blog to learn how a research hypothesis provides a clear and focused direction for a study and helps formulate research questions.

Understanding Customer Retention: How to Keep Your Customers Coming Back

Understanding customer retention is key to building a successful brand that has repeat, loyal customers. Here's what you need to know about it.

Demographic Segmentation: How Brands Can Use it to Improve Marketing Strategies

Read this blog to learn what demographic segmentation means, its importance, and how it can be used by brands.

Mastering Product Positioning: A UX Researcher's Guide

Read this blog to understand why brands should have a well-defined product positioning and how it affects the overall business.

Discrete Vs. Continuous Data: Everything You Need To Know

Explore the differences between discrete and continuous data and their impact on business decisions and customer insights.

50+ Employee Engagement Survey Questions

Understand how an employee engagement survey provides insights into employee satisfaction and motivation, directly impacting productivity and retention.

What is Experimental Research: Definition, Types & Examples

Understand how experimental research enables researchers to confidently identify causal relationships between variables and validate findings, enhancing credibility.

A Guide to Interaction Design

Interaction design can help you create engaging and intuitive user experiences, improving usability and satisfaction through effective design principles. Here's how.

Exploring the Benefits of Stratified Sampling

Understanding stratified sampling can improve research accuracy by ensuring diverse representation across key subgroups. Here's how.

A Guide to Voice Recognition in Enhancing UX Research

Learn the importance of using voice recognition technology in user research for enhanced user feedback and insights.

The Ultimate Figma Design Handbook: Design Creation and Testing

The Ultimate Figma Design Handbook covers setting up Figma, creating designs, advanced features, prototyping, and testing designs with real users.

The Power of Organization: Mastering Information Architectures

Understanding the art of information architectures can enhance user experiences by organizing and structuring digital content effectively, making information easy to find and navigate. Here's how.

Convenience Sampling: Examples, Benefits, and When To Use It

Read the blog to understand how convenience sampling allows for quick and easy data collection with minimal cost and effort.

What is Critical Thinking, and How Can it be Used in Consumer Research?

Learn how critical thinking enhances consumer research and discover how Decode's AI-driven platform revolutionizes data analysis and insights.

How Business Intelligence Tools Transform User Research & Product Management

This blog explains how Business Intelligence (BI) tools can transform user research and product management by providing data-driven insights for better decision-making.

What is Face Validity? Definition, Guide and Examples

Read this blog to explore face validity, its importance, and the advantages of using it in market research.

What is Customer Lifetime Value, and How To Calculate It?

Read this blog to understand how Customer Lifetime Value (CLV) can help your business optimize marketing efforts, improve customer retention, and increase profitability.

Systematic Sampling: Definition, Examples, and Types

Explore how systematic sampling helps researchers by providing a structured method to select representative samples from larger populations, ensuring efficiency and reducing bias.

Understanding Selection Bias: A Guide

Selection bias can affect the type of respondents you choose for the study and ultimately the quality of responses you receive. Here’s all you need to know about it.

A Guide to Designing an Effective Product Strategy

Read this blog to explore why a well-defined product strategy is required for brands while developing or refining a product.

A Guide to Minimum Viable Product (MVP) in UX: Definition, Strategies, and Examples

Discover what an MVP is, why it's crucial in UX, strategies for creating one, and real-world examples from top companies like Dropbox and Airbnb.

Asking Close Ended Questions: A Guide

Asking the right close ended questions is they key to getting quantitiative data from your users. Her's how you should do it.

Creating Website Mockups: Your Ultimate Guide to Effective Design

Read this blog to learn website mockups- tools, examples and how to create an impactful website design.

Understanding Your Target Market And Its Importance In Consumer Research

Read this blog to learn about the importance of creating products and services to suit the needs of your target audience.

What Is a Go-To-Market Strategy And How to Create One?

Check out this blog to learn how a go-to-market strategy helps businesses enter markets smoothly, attract more customers, and stand out from competitors.

What is Confirmation Bias in Consumer Research?

Learn how confirmation bias affects consumer research, its types, impacts, and practical tips to avoid it for more accurate and reliable insights.

Market Penetration: The Key to Business Success

Understanding market penetration is key to cracking the code to sustained business growth and competitive advantage in any industry. Here's all you need to know about it.

How to Create an Effective User Interface

Having a simple, clear user interface helps your users find what they really want, improving the user experience. Here's how you can achieve it.

Product Differentiation and What It Means for Your Business

Discover how product differentiation helps businesses stand out with unique features, innovative designs, and exceptional customer experiences.

What is Ethnographic Research? Definition, Types & Examples

Read this blog to understand Ethnographic research, its relevance in today’s business landscape and how you can leverage it for your business.

Product Roadmap: The 2024 Guide [with Examples]

Read this blog to understand how a product roadmap can align stakeholders by providing a clear product development and delivery plan.

Product Market Fit: Making Your Products Stand Out in a Crowded Market

Delve into the concept of product-market fit, explore its significance, and equip yourself with practical insights to achieve it effectively.

Consumer Behavior in Online Shopping: A Comprehensive Guide

Ever wondered how online shopping behavior can influence successful business decisions? Read on to learn more.

How to Conduct a First Click Test?

Why are users leaving your site so fast? Learn how First Click Testing can help. Discover quick fixes for frustration and boost engagement.

What is Market Intelligence? Methods, Types, and Examples

Read the blog to understand how marketing intelligence helps you understand consumer behavior and market trends to inform strategic decision-making.

What is a Longitudinal Study? Definition, Types, and Examples

Is your long-term research strategy unclear? Learn how longitudinal studies decode complexity. Read on for insights.

What Is the Impact of Customer Churn on Your Business?

Understanding and reducing customer churn is the key to building a healthy business that keeps customers satisfied. Here's all you need to know about it.

The Ultimate Design Thinking Guide

Discover the power of design thinking in UX design for your business. Learn the process and key principles in our comprehensive guide.

100+ Yes Or No Survey Questions Examples

Yes or no survey questions simplify responses, aiding efficiency, clarity, standardization, quantifiability, and binary decision-making. Read some examples!

What is Customer Segmentation? The ULTIMATE Guide

Explore how customer segmentation targets diverse consumer groups by tailoring products, marketing, and experiences to their preferred needs.

Crafting User-Centric Websites Through Responsive Web Design

Find yourself reaching for your phone instead of a laptop for regular web browsing? Read on to find out what that means & how you can leverage it for business.

How Does Product Placement Work? Examples and Benefits

Read the blog to understand how product placement helps advertisers seek subtle and integrated ways to promote their products within entertainment content.

The Importance of Reputation Management, and How it Can Make or Break Your Brand

A good reputation management strategy is crucial for any brand that wants to keep its customers loyal. Here's how brands can focus on it.

A Comprehensive Guide to Human-Centered Design

Are you putting the human element at the center of your design process? Read this blog to understand why brands must do so.

How to Leverage Customer Insights to Grow Your Business

Genuine insights are becoming increasingly difficult to collect. Read on to understand the challenges and what the future holds for customer insights.

The Complete Guide to Behavioral Segmentation

Struggling to reach your target audience effectively? Discover how behavioral segmentation can transform your marketing approach. Read more in our blog!

Creating a Unique Brand Identity: How to Make Your Brand Stand Out

Creating a great brand identity goes beyond creating a memorable logo - it's all about creating a consistent and unique brand experience for your cosnumers. Here's everything you need to know about building one.

Understanding the Product Life Cycle: A Comprehensive Guide

Understanding the product life cycle, or the stages a product goes through from its launch to its sunset can help you understand how to market it at every stage to create the most optimal marketing strategies.

Empathy vs. Sympathy in UX Research

Are you conducting UX research and seeking guidance on conducting user interviews with empathy or sympathy? Keep reading to discover the best approach.

What is Exploratory Research, and How To Conduct It?

Read this blog to understand how exploratory research can help you uncover new insights, patterns, and hypotheses in a subject area.

First Impressions & Why They Matter in User Research

Ever wonder if first impressions matter in user research? The answer might surprise you. Read on to learn more!

Cluster Sampling: Definition, Types & Examples

Read this blog to understand how cluster sampling tackles the challenge of efficiently collecting data from large, spread-out populations.

Top Six Market Research Trends

Curious about where market research is headed? Read on to learn about the changes surrounding this field in 2024 and beyond.

Lyssna Alternative

Meet Qatalyst, your best lyssna alternative to usability testing, to create a solution for all your user research needs.

What is Feedback Loop? Definition, Importance, Types, and Best Practices

Struggling to connect with your customers? Read the blog to learn how feedback loops can solve your problem!

UI vs. UX Design: What’s The Difference?

Learn how UI solves the problem of creating an intuitive and visually appealing interface and how UX addresses broader issues related to user satisfaction and overall experience with the product or service.

The Impact of Conversion Rate Optimization on Your Business

Understanding conversion rate optimization can help you boost your online business. Read more to learn all about it.

Insurance Questionnaire: Tips, Questions and Significance

Leverage this pre-built customizable questionnaire template for insurance to get deep insights from your audience.

UX Research Plan Template

Read on to understand why you need a UX Research Plan and how you can use a fully customizable template to get deep insights from your users!

Brand Experience: What it Means & Why It Matters

Have you ever wondered how users navigate the travel industry for your research insights? Read on to understand user experience in the travel sector.

Validity in Research: Definitions, Types, Significance, and Its Relationship with Reliability

Is validity ensured in your research process? Read more to explore the importance and types of validity in research.

The Role of UI Designers in Creating Delightful User Interfaces

UI designers help to create aesthetic and functional experiences for users. Here's all you need to know about them.

Top Usability Testing Tools to Try

Using usability testing tools can help you understand user preferences and behaviors and ultimately, build a better digital product. Here are the top tools you should be aware of.

Understanding User Experience in Travel Market Research

Ever wondered how users navigate the travel industry for your research insights? Read on to understand user experience in the travel sector.

Top 10 Customer Feedback Tools You’d Want to Try

Explore the top 10 customer feedback tools for analyzing feedback, empowering businesses to enhance customer experience.

10 Best UX Communities on LinkedIn & Slack for Networking & Collaboration

Discover the significance of online communities in UX, the benefits of joining communities on LinkedIn and Slack, and insights into UX career advancement.

The Role of Customer Experience Manager in Consumer Research

This blog explores the role of Customer Experience Managers, their skills, their comparison with CRMs, their key metrics, and why they should use a consumer research platform.

Product Review Template

Learn how to conduct a product review and get insights with this template on the Qatalyst platform.

What Is the Role of a Product Designer in UX?

Product designers help to create user-centric digital experiences that cater to users' needs and preferences. Here's what you need to know about them.

Top 10 Customer Journey Mapping Tools For Market Research in 2024

Explore the top 10 tools in 2024 to understand customer journeys while conducting market research.

Generative AI and its Use in Consumer Research

Ever wondered how Generative AI fits in within the research space? Read on to find its potential in the consumer research industry.

All You Need to Know About Interval Data: Examples, Variables, & Analysis

Understand how interval data provides precise numerical measurements, enabling quantitative analysis and statistical comparison in research.

How to Use Narrative Analysis in Research

Find the advantages of using narrative analysis and how this method can help you enrich your research insights.

A Guide to Asking the Right Focus Group Questions

Moderated discussions with multiple participants to gather diverse opinions on a topic.

Maximize Your Research Potential

Experience why teams worldwide trust our Consumer & User Research solutions.

Book a Demo

• Form Builder
• Survey Maker
• AI Form Generator
• AI Survey Tool
• AI Quiz Maker
• Store Builder
• WordPress Plugin

HubSpot CRM

Google Sheets

Google Analytics

Microsoft Excel

• Popular Forms
• Job Application Form Template
• Rental Application Form Template
• Hotel Accommodation Form Template
• Online Registration Form Template
• Employment Application Form Template
• Application Forms
• Booking Forms
• Consent Forms
• Contact Forms
• Donation Forms
• Customer Satisfaction Surveys
• Employee Satisfaction Surveys
• Evaluation Surveys
• Feedback Surveys
• Market Research Surveys
• Personality Quiz Template
• Geography Quiz Template
• Math Quiz Template
• Science Quiz Template
• Vocabulary Quiz Template

Try without registration Quick Start

Read engaging stories, how-to guides, learn about forms.app features.

Inspirational ready-to-use templates for getting started fast and powerful.

Spot-on guides on how to use forms.app and make the most out of it.

See the technical measures we take and learn how we keep your data safe and secure.

• Integrations
• Help Center
• Sign In Sign Up Free
• What is empirical research: Methods, types & examples

Defne Çobanoğlu

Having opinions on matters based on observation is okay sometimes. Same as having theories on the subject you want to solve. However, some theories need to be tested. Just like Robert Oppenheimer says, “Theory will take you only so far .” 

In that case, when you have your research question ready and you want to make sure it is correct, the next step would be experimentation. Because only then you can test your ideas and collect tangible information. Now, let us start with the empirical research definition:

• What is empirical research?

Empirical research is a research type where the aim of the study is based on finding concrete and provable evidence . The researcher using this method to draw conclusions can use both quantitative and qualitative methods. Different than theoretical research, empirical research uses scientific experimentation and investigation. 

Using experimentation makes sense when you need to have tangible evidence to act on whatever you are planning to do. As the researcher, you can be a marketer who is planning on creating a new ad for the target audience, or you can be an educator who wants the best for the students. No matter how big or small, data gathered from the real world using this research helps break down the question at hand. 

• When to use empirical research?

Empirical research methods are used when the researcher needs to gather data analysis on direct, observable, and measurable data. Research findings are a great way to make grounded ideas. Here are some situations when one may need to do empirical research:

1. When quantitative or qualitative data is needed

There are times when a researcher, marketer, or producer needs to gather data on specific research questions to make an informed decision. And the concrete data gathered in the research process gives a good starting point.

2. When you need to test a hypothesis

When you have a hypothesis on a subject, you can test the hypothesis through observation or experiment. Making a planned study is a great way to collect information and test whether or not your hypothesis is correct.

3. When you want to establish causality

Experimental research is a good way to explore whether or not there is any correlation between two variables. Researchers usually establish causality by changing a variable and observing if the independent variable changes accordingly.

• Types of empirical research

The aim of empirical research is to collect information about a subject from the people by doing experimentation and other data collection methods. However, the methods and data collected are divided into two groups: one collects numerical data, and the other one collects opinion-like data. Let us see the difference between these two types:

Quantitative research

Quantitative research methods are used to collect data in a numerical way. Therefore, the results gathered by these methods will be numbers, statistics, charts, etc. The results can be used to quantify behaviors, opinions, and other variables. Quantitative research methods are surveys, questionnaires, and experimental research.

Qualitiative research

Qualitative research methods are not used to collect numerical answers, instead, they are used to collect the participants’ reasons, opinions, and other meaningful aspects. Qualitative research methods include case studies, observations, interviews, focus groups, and text analysis.

• 5 steps to conduct empirical research

Necessary steps for empirical research

When you want to collect direct and concrete data on a subject, empirical research is a great way to go. And, just like every other project and research, it is best to have a clear structure in mind. This is even more important in studies that may take a long time, such as experiments that take years. Let us look at a clear plan on how to do empirical research:

1. Define the research question

The very first step of every study is to have the question you will explore ready. Because you do not want to change your mind in the middle of the study after investing and spending time on the experimentation.

2. Go through relevant literature

This is the step where you sit down and do a desk research where you gather relevant data and see if other researchers have tried to explore similar research questions. If so, you can see how well they were able to answer the question or what kind of difficulties they faced during the research process.

3. Decide on the methodology

Once you are done going through the relevant literature, you can decide on which method or methods you can use. The appropriate methods are observation, experimentation, surveys, interviews, focus groups, etc.

4. Do data analysis

When you get to this step, it means you have successfully gathered enough data to make a data analysis. Now, all you need to do is look at the data you collected and make an informed analysis.

5. Conclusion

This is the last step, where you are finished with the experimentation and data analysis process. Now, it is time to decide what to do with this information. You can publish a paper and make informed decisions about whatever your goal is.

• Empirical research methodologies

Some essential methodologies to conduct empirical research

The aim of this type of research is to explore brand-new evidence and facts. Therefore, the methods should be primary and gathered in real life, directly from the people. There is more than one method for this goal, and it is up to the researcher to use which one(s). Let us see the methods of empirical research: 

• Observation

The method of observation is a great way to collect information on people without the effect of interference. The researcher can choose the appropriate area, time, or situation and observe the people and their interactions with one another. The researcher can be just an outside observer or can be a participant as an observer or a full participant.

• Experimentation

The experimentation process can be done in the real world by intervening in some elements to unify the environment for all participants. This method can also be done in a laboratory environment. The experimentation process is good for being able to change the variables according to the aim of the study.

The case study method is done by making an in-depth analysis of already existing cases. When the parameters and variables are similar to the research question at hand, it is wise to go through what was researched before.

• Focus groups

The case study method is done by using a group of individuals or multiple groups and using their opinions, characteristics, and responses. The scientists gather the data from this group and generalize it to the whole population.

Surveys are an effective way to gather data directly from people. It is a systematic approach to collecting information. If it is done in an online setting as an online survey , it would be even easier to reach out to people and ask their opinions in open-ended or close-ended questions.

Interviews are similar to surveys as you are using questions to collect information and opinions of the people. Unlike a survey, this process is done face-to-face, as a phone call, or as a video call.

• Advantages of empirical research

Empirical research is effective for many reasons, and helps researchers from numerous fields. Here are some advantages of empirical research to have in mind for your next research:

• Empirical research improves the internal validity of the study.
• Empirical evidence gathered from the study is used to authenticate the research question.
• Collecting provable evidence is important for the success of the study.
• The researcher is able to make informed decisions based on the data collected using empirical research.
• Disadvantages of empirical research

After learning about the positive aspects of empirical research, it is time to mention the negative aspects. Because this type may not be suitable for everyone and the researcher should be mindful of the disadvantages of empirical research. Here are the disadvantages of empirical research:

• As it is similar to other research types, a case study where experimentation is included will be time-consuming no matter what. It has more steps and variables than concluding a secondary research.
• There are a lot of variables that need to be controlled and considered. Therefore, it may be a challenging task to be mindful of all the details.
• Doing evidence-based research can be expensive if you need to complete it on a large scale.
• When you are conducting an experiment, you may need some waivers and permissions.
• Frequently asked questions about empirical research

Empirical research is one of the many research types, and there may be some questions in mind about its similarities and differences to other research types.

Is empirical research qualitative or quantitative?

The data collected by empirical research can be qualitative, quantitative, or a mix of both. It is up to the aim of researcher to what kind of data is needed and searched for.

Is empirical research the same as quantitative research?

As quantitative research heavily relies on data collection methods of observation and experimentation, it is, in nature, an empirical study. Some professors may even use the terms interchangeably. However, that does not mean that empirical research is only a quantitative one.

What is the difference between theoretical and empirical research?

Empirical studies are based on data collection to prove theories or answer questions, and it is done by using methods such as observation and experimentation. Therefore, empirical research relies on finding evidence that backs up theories. On the other hand, theoretical research relies on theorizing on empirical research data and trying to make connections and correlations.

What is the difference between conceptual and empirical research?

Conceptual research is about thoughts and ideas and does not involve any kind of experimentation. Empirical research, on the other hand, works with provable data and hard evidence.

What is the difference between empirical vs applied research?

Some scientists may use these two terms interchangeably however, there is a difference between them. Applied research involves applying theories to solve real-life problems. On the other hand, empirical research involves the obtaining and analysis of data to test hypotheses and theories.

• Final words

Empirical research is a good means when the goal of your study is to find concrete data to go with. You may need to do empirical research when you need to test a theory, establish causality, or need qualitative/quantitative data. For example, you are a scientist and want to know if certain colors have an effect on people’s moods, or you are a marketer and want to test your theory on ad places on websites. 

In both scenarios, you can collect information by using empirical research methods and make informed decisions afterward. These are just the two of empirical research examples. This research type can be applied to many areas of work life and social sciences. Lastly, for all your research needs, you can visit forms.app to use its many useful features and over 1000 form and survey templates!

Defne is a content writer at forms.app. She is also a translator specializing in literary translation. Defne loves reading, writing, and translating professionally and as a hobby. Her expertise lies in survey research, research methodologies, content writing, and translation.

• Form Features
• Data Collection

Table of Contents

Related posts.

A full guide to the PEST analysis

Fatih Serdar Çıtak

How to improve your business with a web form

5 consent form examples that will save your day

• Voxco Online
• Voxco Panel Management
• Voxco Panel Portal
• Voxco Audience
• Voxco Mobile Offline
• Voxco Dialer Cloud
• Voxco Dialer On-premise
• Voxco TCPA Connect
• Voxco Analytics
• Voxco Text & Sentiment Analysis

• 40+ question types
• Drag-and-drop interface
• Skip logic and branching
• Multi-lingual survey
• Text piping
• Question library
• CSS customization
• White-label surveys
• Customizable ‘Thank You’ page
• Customizable survey theme
• Reminder send-outs
• Survey rewards
• Social media
• Website surveys
• Correlation analysis
• Cross-tabulation analysis
• Trend analysis
• Real-time dashboard
• Customizable report
• Email address validation
• Recaptcha validation
• SSL security

Take a peek at our powerful survey features to design surveys that scale discoveries.

Download feature sheet.

• Hospitality
• Academic Research
• Customer Experience
• Employee Experience
• Product Experience
• Market Research
• Social Research
• Data Analysis

Explore Voxco 

Need to map Voxco’s features & offerings? We can help!

Watch a Demo 

Download Brochures 

Get a Quote

• NPS Calculator
• CES Calculator
• A/B Testing Calculator
• Margin of Error Calculator
• Sample Size Calculator
• CX Strategy & Management Hub
• Market Research Hub
• Patient Experience Hub
• Employee Experience Hub
• NPS Knowledge Hub
• Market Research Guide
• Customer Experience Guide
• Survey Research Guides
• Survey Template Library
• Webinars and Events
• Feature Sheets
• Try a sample survey
• Professional Services

Get exclusive insights into research trends and best practices from top experts! Access Voxco’s ‘State of Research Report 2024 edition’ .

We’ve been avid users of the Voxco platform now for over 20 years. It gives us the flexibility to routinely enhance our survey toolkit and provides our clients with a more robust dataset and story to tell their clients.

VP Innovation & Strategic Partnerships, The Logit Group

• Client Stories
• Voxco Reviews
• Why Voxco Research?
• Careers at Voxco
• Vulnerabilities and Ethical Hacking

Explore Regional Offices

• Survey Software The world’s leading omnichannel survey software
• Online Survey Tools Create sophisticated surveys with ease.
• Mobile Offline Conduct efficient field surveys.
• Text Analysis
• Close The Loop
• Automated Translations
• NPS Dashboard
• CATI Manage high volume phone surveys efficiently
• Cloud/On-premise Dialer TCPA compliant Cloud on-premise dialer
• IVR Survey Software Boost productivity with automated call workflows.
• Analytics Analyze survey data with visual dashboards
• Panel Manager Nurture a loyal community of respondents.
• Survey Portal Best-in-class user friendly survey portal.
• Voxco Audience Conduct targeted sample research in hours.
• Predictive Analytics
• Customer 360
• Customer Loyalty
• Fraud & Risk Management
• AI/ML Enablement Services
• Credit Underwriting

Find the best survey software for you! (Along with a checklist to compare platforms)

Get Buyer’s Guide

• 100+ question types
• SMS surveys
• Financial Services
• Banking & Financial Services
• Retail Solution
• Risk Management
• Customer Lifecycle Solutions
• Net Promoter Score
• Customer Behaviour Analytics
• Customer Segmentation
• Data Unification

Explore Voxco 

Watch a Demo 

Download Brochures 

• CX Strategy & Management Hub
• The Voxco Guide to Customer Experience
• Professional services
• Blogs & White papers
• Case Studies

Find the best customer experience platform

Uncover customer pain points, analyze feedback and run successful CX programs with the best CX platform for your team.

Get the Guide Now

VP Innovation & Strategic Partnerships, The Logit Group

• Why Voxco Intelligence?
• Our clients
• Client stories
• Featuresheets

Dive into Empirical Research: An Insightful Overview

• April 1, 2021

Market Research Tool kit

Get started with Voxco’s Market Research Toolkit.

Market Research trends guide + Online Surveys guide + Agile MArket Research Guide + 5 Market Research Templates 

SHARE THE ARTICLE ON

What is Empirical research?

In empirical study, conclusions of the study are drawn from concrete empirical evidence. This evidence is also referred to as “verifiable” evidence. This evidence is gathered either through quantitative market research or qualitative market research methods.

An example of empirical analysis  would be if a researcher was interested in finding out whether listening to happy music promotes prosocial behaviour. An experiment could be conducted where one group of the audience is exposed to happy music and the other is not exposed to music at all. The participants could be given an opportunity to either help a stranger with something or not. The results are then evaluated to find whether happy music increases prosaically behavior or not.

See Voxco survey software in action with a Free demo.

What is an Empirical Study?

The origin of empirical methods starts from the quote “I will not believe it unless I see it myself.” Empirical observation emerged during the renaissance with medieval science. The word empirical is derived from the Greek word ‘empeirikos’ meaning ‘experienced’.

The word empirical, in today’s day and age, refers to collecting empirical data through methods of observation, experience, or by specific scientific instruments. All of these methods are dependent on observation and experiments which are used to collect data and test the same for arriving at conclusions. Online survey tools are an extremely effective technique which can be used for empirical methods.

Types and methodologies of empirical research

Empirical study uses qualitative or quantitative methods to conduct research and analyze results. 

• Quantitative research: Quantitative research is referred to as the process of collecting as well as analyzing numerical data. It is generally used to find patterns, averages, predictions, as well as cause-effect relationships between the variables being studied. It is also used to generalize the results of a particular study to the population in consideration.

• Qualitative research: Qualitative research can be defined as a method used for market research which aims at obtaining data through open-ended questions and conversations with the intended consumers. This method aims at establishing not only “what” people think but “how” they come to that opinion as well as “why” they think so.

Step by Step guide to Descriptive Research

Get ready to uncover the how, when, what, and where questions in a research problem

The empirical data that is collected from either of these methods has to be analyzed. Empirical evidence is analyzed using qualitative or quantitative methods. These methods are used to answer empirical questions that are clearly defined. The type of research design used by the researcher depends on the field and the nature of the problem. Some researchers use a combination of quantitative and qualitative methods to answer the questions set for the research.

Quantitative research methods

Quantitative research methods help in the analysis of the empirical evidence that has been gathered. By using these methods researchers can find support for their hypotheses.

• Survey research: Survey research is the most common and widely used tool for quantitative research. Surveys are used to gather data by asking relevant questions to the respondents who are thought to have the relevant information we are seeking to acquire. Generally, a formal list of questionnaires is prepared which is circulated to the respondents and they can self-report their thoughts. Researchers use a non-disguised approach so that the participants of the survey know exactly what they are answering. In general, respondents are asked questions regarding their demographic details, and the opinion that the researcher is interested in studying. Surveys can be conducted through online polls, paper-pencil questionnaires, web-intercept surveys, etc. 

For example: In market research, customers are deemed as the most important part of the organisation. It is a known fact that satisfied customers will help your organisation grow directly by remaining loyal to your company and also by becoming an advocate for your brand. Researchers can use customer satisfaction survey templates to assess their brand’s value and how likely their customers are to recommend their brand to others.

• Experimental research : This is one of the most recommended and reliant research methods in natural as well as social sciences. As the name suggests, experimental research (also known as experimentation) is usually based on one of more theories as its driving principle or rationale. In this method, the theory which is under study has not yet proven, it is merely a speculation. Thus, an experiment is performed in order to either prove or disprove the theory. If the results of the experiment are in line with the prediction made by the theory, then the theory is supported. If not, then the theory is refuted. 

For instance, if a researcher wants to study whether their dandruff protection product is successful in curing dandruff, and the only difference between the two groups under study is the product of interest (one group uses the product while group 2 uses a placebo), then dandruff could be considered as the dependent variable and the product curing it would be called an independent variable. Now, the independent variable, here, is “manipulated” in the sense that one group is exposed to it and one is not. All things being constant, if the product cures dandruff in group 1 as opposed to the group that is using a placebo, the experimental research findings are successful. This will help in establishing a cause and effect relationship, the product is “causing” the treatment (“effect”) of dandruff.

• Correlational research : A correlation refers to an association or a relationship between two entities. A correlational research studies how one entity impacts the other and what are the changes that are observed when either one of them changes.  correlation coefficient ranges from -1 to +1. A correlation coefficient of +1 indicates a perfect positive correlation whereas a correlation coefficient of -1 indicates a perfect negative correlation between two variables. A correlation coefficient of 0 indicates that there is no relationship between the variables under study.

Some examples of correlational research questions: 

• What is the relationship between gender and the purchase of a particular product under study?
• The relationship between stress and burnout in employees of an organisation.
• The relationship between choosing to work from home and the level of corona-phobia in employees.
• Longitudinal study : Longitudinal surveys, on the other hand, involve studying variables for a long period of time and observing the changes in them from time to time. Here, the data is collected from the respondents at the beginning of the study, and then the researcher collects data at different time intervals until the end of the study. Longitudinal surveys are more popularly used in medicinal science to understand and evaluate the effects of medicines, or vaccines, in the long-run on participants. Because longitudinal surveys take place for several years, researchers can establish the sequence of events that may affect the variable under study.

For example: If researchers want to understand how smoking affects the development of cancer in later stages of life, they would choose participants who are different from other observable variables but similar in one: smoking. In this case, researchers would observe the participants who started smoking from adolescence into later adulthood and examine the changes in their body that are caused due to smoking. They can see how smoking has influenced the immunity of participants, their reaction to stress, and other variables relevant to the researcher. Over time, researchers can also observe the effects of quitting smoking if some participants decide to quit smoking later in their life. This will help researchers understand the interaction between health and smoking in more detail.

• Cross sectional: In cross-sectional surveys, the study takes place at a single point in time. Hence, cross-sectional surveys do not entail the manipulation of the variables under study, and are limited in that way. Cross-sectional surveys allow researchers to study various characteristics, such as the demographic structure of the consumers, their interests, and attitudes, all at once. It aims to provide information about the population at the current moment in time. For example, cross-sectional surveys will tell us how the consumer is responding and feeling about the product at the present moment. It does not study the other variables that may affect the consumers’ reactions to the product in the future.

For example: Let us consider a researcher who is aiming to study developmental psychology. He/she may select groups of people who are of different ages but study them at one point in time. In this way, the difference between the groups will be attributed to their age differences instead of other variables that may happen over time.

Qualitative research methods

A qualitative approach is more appropriate when tackling some research questions. This is especially true if the researcher wishes to observe the behaviors of the target audience in-depth. The results here are in descriptive form. Qualitative research is not predictive in nature. It enables researchers to build and support their theories to advance future potential quantitative research. Qualitative research methods are used to come up with conclusions to support the theory or hypothesis under study.

• Case study: Case studies have evolved to become a valuable method for qualitative research. It is used for explaining a case of an organization or an entity. This is one of the simplest ways of conducting research because it involves an exhaustive understanding of the data collected and the interpretation of the same. 

For example: For example; let’s assume that a researcher is interested in understanding how to effectively solve the problems of turnover in organizations. While exploring, he came across an organization that had high rates of turnover and was able to solve the problem by the end of the year. The researcher can study this case in detail and come up with methods that increased the chances of success for this organization.

• Observational method :  When doing qualitative research, maintaining the existing records can be a valuable source of information in the future. This data can be used in new research and also provide insights for the same. Observation is one of the common aspects that is used in every method we described above. It can be systematic or naturalistic. Qualitative observation of respondents’ answers, or their behaviors in particular settings can yield enriching insights. Hence, observation in qualitative research is used to gather information about relevant characteristics that the researcher is interested in studying.

For instance, if a smartphone brand wants to see how customers react to its products in a showroom, observers may be hired to note the same. The observers can use the recorded observations to evaluate and draw inferences about the customers.

• One-on-one interview : Interviewing people of interest is one of the most common practices in qualitative research. Here, there is an in-depth personal interview carried out either face-to-face or through online mediums with one respondent at a time. This is a conversational method of gathering information and it invites the researcher with an opportunity to get a detailed response from the respondent.

For example: A one-on-one interview with an environmentalist will help to gather data on the current climate crisis in the world. 

• Focus groups :  Another most commonly used method in qualitative research apart from interviewing people is focus group. In this method, data is usually conducted once a researcher includes a limited number of consumers (usually ranging from 6 to 10) from the target market and forms a group. 

For example: Let’s assume a researcher wants to explore what are qualities consumers value when buying a laptop. This could be the display quality, battery life, brand value, or even the color. The researcher can make a focus group of people who buy laptops regularly and understand the dynamics a consumer considers when buying electronic devices.

• Text analysis : In text analysis, researchers analyze the social life of the respondents in the study and aim to decode the actions and the words of the respondents. Hence, text analysis is distinct from other qualitative research methods as it focuses on the social life of the respondents. In the last decade or so, text analysis has become increasingly popular due to the analysis of what consumers share on social media platforms in the form of blogs, images, and other texts. 

For example: Companies ask their customers to give detailed feedback on how satisfied they are with their customer support team. This data helps them make appropriate decisions to improve their team.

Sometimes researchers use a combination of methods to answer the questions. This is especially true when researchers tackle complex subject matters.

Exploratory Research Guide

Conducting exploratory research seems tricky but an effective guide can help.

Steps for conducting empirical research

Since empirical methods are based on observation and capturing experiences, it is important to plan the steps to conduct the experiment and how to analyze it. This will enable the researcher to resolve problems or obstacles which can occur during the experiment.

Step #1: Define the purpose of the research

The very first step is for the researcher to identify the area of research and the problem can be addressed by finding out ways to solve it. The researcher should come up with various questions regarding what is the problem, who will benefit from the research, how should they go about the process, etc. The researchers should explore the purpose of the research in detail.

Step #2 : Supporting theories and relevant literature

After exploring and finding out the purpose of the research, the researcher must aim to find if there are existing theories that have addressed this before. The researcher has to figure out whether any previous studies can help them support their research. During this stage of empirical study, the researcher should aim at finding all relevant literature that will help them understand the problem at hand. The researcher should also come up with his/her own set of assumptions or problem statements that they wish to explore. 

Step #3: Creation of Hypothesis and measurement

If the researcher is aiming to solve a problem the problem has not been resolved efficiently in previous research, then the researcher creates his/her own problem statement. This problem statement, also called hypothesis, will be based on the questions that the researcher came up with while identifying the area of concern. The researcher can also form a hypothesis on the basis of prior research they found and studied during the literature review phase of the study.

Step #4: Methodology, research design and empirical data collection

Here the researcher has to define the strategies to be used for conducting the research. They can set up experiments in collecting data that can help them come up with probable hypotheses. On the basis of the hypotheses, researchers can decide whether they will require experimental or non-experimental methods for the conduction of the research. The research design will depend upon the field in which the research is to be conducted. The researchers will need to find parameters that can affect the validity of the research design. Researchers also need to choose appropriate methods of data collection, which in turn depends on the research question. There are many sampling methods that can be used by the researcher. Once, the data is collected, it has to be analysed.

Step #5: Data Analysis and result

Data can be analyzed either qualitatively and quantitatively. Researchers will need to decide which method they will employ depending upon the nature of the empirical data collected. Researchers can also use a combination of both for their study. On the basis of the analysis, the hypothesis will either be supported or rejected. Data analysis is the most important aspect of empirical observation.

Step #6: Conclusion

The researcher will have to collate the findings and make a report based on the empirical observations. The researcher can use previous theories and literature to support their hypothesis and lineage of findings. The researcher can also make recommendations for future research on similar issues.

Advantages of Empirical research

The advantages of empirical study are highlighted below:

• Used for authentication. Empirical study is used to authenticate previous findings of experiments and empirical observations. This research methodology makes the conducted study more authentic and accurate. 
• Empirical approach is useful for understanding dynamic changes. Due to the detailed process of literature review, empirical analysis is used in helping researchers understand dynamic changes in the field. It also enables them to strategies accordingly.
• Provides a level of control . Empirical approach empowers researchers to demonstrate a level of control by allowing them to control multiple variables under study.
• Empirical methods Increase internal validity . The high level of control in the research process makes an empirical method demonstrate high internal validity.

Disadvantages of Empirical research

Empirical approach is not without its limitations. Some of them include:

• Time consuming . Empirical studies are time consuming because it requires researchers to collect data through multiple sources. It also requires them to assess various parameters involved in the research. 
• Empirical approach is Expensive. The researcher may have to conduct the research at different locations or environments which may be expensive.
• Difficult to acquire consent/permission. Sometimes empirical studies may be difficult to conduct due to the rules that are to be followed when conducting it.
• Data collection in the empirical approach can be a problem. Since empirical data has to be collected from different methods and sources, it can pose a problem to the researchers.

Transform your insight generation process

Create an actionable feedback collection process.

Why is there a need for empirical research?

Because most people today only believe in their experiences, empirical observation is increasingly becoming important. It is used to validate various hypotheses or refute them in the face of evidence. It also increases human knowledge and advances scientific progression. 

For instance, empirical analysis is used by pharmaceutical companies to test specific drugs. This is done by administering the drug on an experimental group, while giving a placebo to the control group. This is done to prove theories about the proposed drug and check its efficacy. This is the most crucial way in which leading evidence for various drugs have been found for many years. 

Empirical methods are used not just in medical science, but also in history, social science, market research, etc.

In today’s world it has become critical to conduct empirical analysis in order to support hypotheses and gather knowledge in several fields. The methods under empirical studies mentioned above help researchers to carry out research.

Explore Voxco Survey Software

+ Omnichannel Survey Software 

+ Online Survey Software 

+ CATI Survey Software 

+ IVR Survey Software 

+ Market Research Tool

+ Customer Experience Tool 

+ Product Experience Software 

+ Enterprise Survey Software 

Customer Experience Management Software

Customer Experience Management Software : Definition, Benefits & Features Try a free Voxco Online sample survey! Unlock your Sample Survey SHARE THE ARTICLE ON Table

Market Survey

Market Survey: Uncovering Your Target Market SHARE THE ARTICLE ON Table of Contents In a competitive yet saturated business landscape, staying ahead of the curve

How are polls conducted?

How are polls conducted? SHARE THE ARTICLE ON Table of Contents Conducting a poll means running a survey through a sample of the population. The

Non-leading survey questions

Non-leading Survey Questions SHARE THE ARTICLE ON Table of Contents What are leading and non-leading questions? Talking about leading questions, leading questions in surveys subtly

Purposive Sampling

Purposive Sampling: Advantages and Disadvantages in Research SHARE THE ARTICLE ON Table of Contents In the vast landscape of research methodologies, purposive sampling stands out

Improving your business’s customer focus

Improving your business’s customer focus SHARE THE ARTICLE ON Share on facebook Share on twitter Share on linkedin Table of Contents Customer focus follows an

We use cookies in our website to give you the best browsing experience and to tailor advertising. By continuing to use our website, you give us consent to the use of cookies. Read More

What are the advantages and disadvantages of an empirical study?

An empirical study is that which has statistical research in the form of the primary. This primary data is usually collected in the form of survey questionnaires or interviews. Focus group discussions, observation methods, and financial statement analysis are some other ways to collect primary data.

• Since an empirical study contributes to existing knowledge, it adds high value to the research paper.
• The methodology is flexible. You can change the sample size, sampling type, data collection methods, and analysis methods as necessary.
• Fewer rules are to be followed as they are flexible to incorporate. Empirical papers can be presented in many ways. For example, you can eliminate the literature review. You can skip testing the hypothesis and base the analysis on frequency tables and cross-tabulations only.
• It saves a lot of time.

However, there are certain disadvantages too.

• Empirical studies are lengthy. Depending upon the number of variables and data analysis methods used, primary data analysis cannot be fit in less than 3000 words.
• Results can be unpredictable. This problem can be averted if suitable measures are taken beforehand.
• The presentation of data can be tricky. For instance, do you need to present graphs, or tables, or both? Do you need to present the formula for the test used? Do you need to show a sample size calculation? This is because there is no universal format for the presentation of empirical data.
• Priya Chetty

I am a management graduate with specialisation in Marketing and Finance. I have over 12 years' experience in research and analysis. This includes fundamental and applied research in the domains of management and social sciences. I am well versed with academic research principles. Over the years i have developed a mastery in different types of data analysis on different applications like SPSS, Amos, and NVIVO. My expertise lies in inferring the findings and creating actionable strategies based on them. 

Over the past decade I have also built a profile as a researcher on Project Guru's Knowledge Tank division. I have penned over 200 articles that have earned me 400+ citations so far. My Google Scholar profile can be accessed here . 

I now consult university faculty through Faculty Development Programs (FDPs) on the latest developments in the field of research. I also guide individual researchers on how they can commercialise their inventions or research findings. Other developments im actively involved in at Project Guru include strengthening the "Publish" division as a bridge between industry and academia by bringing together experienced research persons, learners, and practitioners to collaboratively work on a common goal. 

• Click to share on Twitter (Opens in new window)
• Click to share on Facebook (Opens in new window)
• Click to share on LinkedIn (Opens in new window)
• Click to share on WhatsApp (Opens in new window)
• Click to share on Telegram (Opens in new window)

Notify me of follow-up comments by email.

proofreading

• What is Empirical Research Study? [Examples & Method]

The bulk of human decisions relies on evidence, that is, what can be measured or proven as valid. In choosing between plausible alternatives, individuals are more likely to tilt towards the option that is proven to work, and this is the same approach adopted in empirical research. 

In empirical research, the researcher arrives at outcomes by testing his or her empirical evidence using qualitative or quantitative methods of observation, as determined by the nature of the research. An empirical research study is set apart from other research approaches by its methodology and features hence; it is important for every researcher to know what constitutes this investigation method. 

What is Empirical Research? 

Empirical research is a type of research methodology that makes use of verifiable evidence in order to arrive at research outcomes. In other words, this  type of research relies solely on evidence obtained through observation or scientific data collection methods. 

Empirical research can be carried out using qualitative or quantitative observation methods , depending on the data sample, that is, quantifiable data or non-numerical data . Unlike theoretical research that depends on preconceived notions about the research variables, empirical research carries a scientific investigation to measure the experimental probability of the research variables 

Characteristics of Empirical Research

• Research Questions

An empirical research begins with a set of research questions that guide the investigation. In many cases, these research questions constitute the research hypothesis which is tested using qualitative and quantitative methods as dictated by the nature of the research.

In an empirical research study, the research questions are built around the core of the research, that is, the central issue which the research seeks to resolve. They also determine the course of the research by highlighting the specific objectives and aims of the systematic investigation. 

• Definition of the Research Variables

The research variables are clearly defined in terms of their population, types, characteristics, and behaviors. In other words, the data sample is clearly delimited and placed within the context of the research. 

• Description of the Research Methodology

 An empirical research also clearly outlines the methods adopted in the systematic investigation. Here, the research process is described in detail including the selection criteria for the data sample, qualitative or quantitative research methods plus testing instruments. 

An empirical research is usually divided into 4 parts which are the introduction, methodology, findings, and discussions. The introduction provides a background of the empirical study while the methodology describes the research design, processes, and tools for the systematic investigation. 

The findings refer to the research outcomes and they can be outlined as statistical data or in the form of information obtained through the qualitative observation of research variables. The discussions highlight the significance of the study and its contributions to knowledge. 

Uses of Empirical Research

Without any doubt, empirical research is one of the most useful methods of systematic investigation. It can be used for validating multiple research hypotheses in different fields including Law, Medicine, and Anthropology. 

• Empirical Research in Law : In Law, empirical research is used to study institutions, rules, procedures, and personnel of the law, with a view to understanding how they operate and what effects they have. It makes use of direct methods rather than secondary sources, and this helps you to arrive at more valid conclusions.
• Empirical Research in Medicine : In medicine, empirical research is used to test and validate multiple hypotheses and increase human knowledge.
• Empirical Research in Anthropology : In anthropology, empirical research is used as an evidence-based systematic method of inquiry into patterns of human behaviors and cultures. This helps to validate and advance human knowledge.

Discover how Extrapolation Powers statistical research: Definition, examples, types, and applications explained.

The Empirical Research Cycle

The empirical research cycle is a 5-phase cycle that outlines the systematic processes for conducting and empirical research. It was developed by Dutch psychologist, A.D. de Groot in the 1940s and it aligns 5 important stages that can be viewed as deductive approaches to empirical research. 

In the empirical research methodological cycle, all processes are interconnected and none of the processes is more important than the other. This cycle clearly outlines the different phases involved in generating the research hypotheses and testing these hypotheses systematically using the empirical data. 

• Observation: This is the process of gathering empirical data for the research. At this stage, the researcher gathers relevant empirical data using qualitative or quantitative observation methods, and this goes ahead to inform the research hypotheses.
• Induction: At this stage, the researcher makes use of inductive reasoning in order to arrive at a general probable research conclusion based on his or her observation. The researcher generates a general assumption that attempts to explain the empirical data and s/he goes on to observe the empirical data in line with this assumption.
• Deduction: This is the deductive reasoning stage. This is where the researcher generates hypotheses by applying logic and rationality to his or her observation.
• Testing: Here, the researcher puts the hypotheses to test using qualitative or quantitative research methods. In the testing stage, the researcher combines relevant instruments of systematic investigation with empirical methods in order to arrive at objective results that support or negate the research hypotheses.
• Evaluation: The evaluation research is the final stage in an empirical research study. Here, the research outlines the empirical data, the research findings and the supporting arguments plus any challenges encountered during the research process.

This information is useful for further research. 

Learn about qualitative data: uncover its types and examples here.

Examples of Empirical Research 

• An empirical research study can be carried out to determine if listening to happy music improves the mood of individuals. The researcher may need to conduct an experiment that involves exposing individuals to happy music to see if this improves their moods.

The findings from such an experiment will provide empirical evidence that confirms or refutes the hypotheses. 

• An empirical research study can also be carried out to determine the effects of a new drug on specific groups of people. The researcher may expose the research subjects to controlled quantities of the drug and observe research subjects to controlled quantities of the drug and observe the effects over a specific period of time to gather empirical data.
• Another example of empirical research is measuring the levels of noise pollution found in an urban area to determine the average levels of sound exposure experienced by its inhabitants. Here, the researcher may have to administer questionnaires or carry out a survey in order to gather relevant data based on the experiences of the research subjects.
• Empirical research can also be carried out to determine the relationship between seasonal migration and the body mass of flying birds. A researcher may need to observe the birds and carry out necessary observation and experimentation in order to arrive at objective outcomes that answer the research question.

Empirical Research Data Collection Methods

Empirical data can be gathered using qualitative and quantitative data collection methods. Quantitative data collection methods are used for numerical data gathering while qualitative data collection processes are used to gather empirical data that cannot be quantified, that is, non-numerical data. 

The following are common methods of gathering data in empirical research

• Survey/ Questionnaire

A survey is a method of data gathering that is typically employed by researchers to gather large sets of data from a specific number of respondents with regards to a research subject. This method of data gathering is often used for quantitative data collection , although it can also be deployed during quantitative research.

A survey contains a set of questions that can range from close-ended to open-ended questions together with other question types that revolve around the research subject. A survey can be administered physically or with the use of online data-gathering platforms like Formplus. 

Empirical data can also be collected by carrying out an experiment. An experiment is a controlled simulation in which one or more of the research variables is manipulated using a set of interconnected processes in order to confirm or refute the research hypotheses.

An experiment is a useful method of measuring causality; that is cause and effect between dependent and independent variables in a research environment. It is an integral data gathering method in an empirical research study because it involves testing calculated assumptions in order to arrive at the most valid data and research outcomes. 

T he case study method is another common data gathering method in an empirical research study. It involves sifting through and analyzing relevant cases and real-life experiences about the research subject or research variables in order to discover in-depth information that can serve as empirical data.

• Observation

The observational method is a method of qualitative data gathering that requires the researcher to study the behaviors of research variables in their natural environments in order to gather relevant information that can serve as empirical data.

How to collect Empirical Research Data with Questionnaire

With Formplus, you can create a survey or questionnaire for collecting empirical data from your research subjects. Formplus also offers multiple form sharing options so that you can share your empirical research survey to research subjects via a variety of methods.

Here is a step-by-step guide of how to collect empirical data using Formplus:

Sign in to Formplus

In the Formplus builder, you can easily create your empirical research survey by dragging and dropping preferred fields into your form. To access the Formplus builder, you will need to create an account on Formplus. 

Once you do this, sign in to your account and click on “Create Form ” to begin. 

Unlock the secrets of Quantitative Data: Click here to explore the types and examples.

Edit Form Title

Click on the field provided to input your form title, for example, “Empirical Research Survey”.

Edit Form  

• Click on the edit button to edit the form.
• Add Fields: Drag and drop preferred form fields into your form in the Formplus builder inputs column. There are several field input options for survey forms in the Formplus builder.
• Edit fields
• Click on “Save”
• Preview form.

Customize Form

Formplus allows you to add unique features to your empirical research survey form. You can personalize your survey using various customization options. Here, you can add background images, your organization’s logo, and use other styling options. You can also change the display theme of your form. 

• Share your Form Link with Respondents

Formplus offers multiple form sharing options which enables you to easily share your empirical research survey form with respondents. You can use the direct social media sharing buttons to share your form link to your organization’s social media pages. 

You can send out your survey form as email invitations to your research subjects too. If you wish, you can share your form’s QR code or embed it on your organization’s website for easy access. 

Empirical vs Non-Empirical Research

Empirical and non-empirical research are common methods of systematic investigation employed by researchers. Unlike empirical research that tests hypotheses in order to arrive at valid research outcomes, non-empirical research theorizes the logical assumptions of research variables. 

Definition: Empirical research is a research approach that makes use of evidence-based data while non-empirical research is a research approach that makes use of theoretical data. 

Method: In empirical research, the researcher arrives at valid outcomes by mainly observing research variables, creating a hypothesis and experimenting on research variables to confirm or refute the hypothesis. In non-empirical research, the researcher relies on inductive and deductive reasoning to theorize logical assumptions about the research subjects.

The major difference between the research methodology of empirical and non-empirical research is while the assumptions are tested in empirical research, they are entirely theorized in non-empirical research. 

Data Sample: Empirical research makes use of empirical data while non-empirical research does not make use of empirical data. Empirical data refers to information that is gathered through experience or observation. 

Unlike empirical research, theoretical or non-empirical research does not rely on data gathered through evidence. Rather, it works with logical assumptions and beliefs about the research subject. 

Data Collection Methods : Empirical research makes use of quantitative and qualitative data gathering methods which may include surveys, experiments, and methods of observation. This helps the researcher to gather empirical data, that is, data backed by evidence.  

Non-empirical research, on the other hand, does not make use of qualitative or quantitative methods of data collection . Instead, the researcher gathers relevant data through critical studies, systematic review and meta-analysis. 

Advantages of Empirical Research 

• Empirical research is flexible. In this type of systematic investigation, the researcher can adjust the research methodology including the data sample size, data gathering methods plus the data analysis methods as necessitated by the research process.
• It helps the research to understand how the research outcomes can be influenced by different research environments.
• Empirical research study helps the researcher to develop relevant analytical and observation skills that can be useful in dynamic research contexts.
• This type of research approach allows the researcher to control multiple research variables in order to arrive at the most relevant research outcomes.
• Empirical research is widely considered as one of the most authentic and competent research designs.
• It improves the internal validity of traditional research using a variety of experiments and research observation methods.

Disadvantages of Empirical Research 

• An empirical research study is time-consuming because the researcher needs to gather the empirical data from multiple resources which typically takes a lot of time.
• It is not a cost-effective research approach. Usually, this method of research incurs a lot of cost because of the monetary demands of the field research.
• It may be difficult to gather the needed empirical data sample because of the multiple data gathering methods employed in an empirical research study.
• It may be difficult to gain access to some communities and firms during the data gathering process and this can affect the validity of the research.
• The report from an empirical research study is intensive and can be very lengthy in nature.

Conclusion 

Empirical research is an important method of systematic investigation because it gives the researcher the opportunity to test the validity of different assumptions, in the form of hypotheses, before arriving at any findings. Hence, it is a more research approach. 

There are different quantitative and qualitative methods of data gathering employed during an empirical research study based on the purpose of the research which include surveys, experiments, and various observatory methods. Surveys are one of the most common methods or empirical data collection and they can be administered online or physically. 

You can use Formplus to create and administer your online empirical research survey. Formplus allows you to create survey forms that you can share with target respondents in order to obtain valuable feedback about your research context, question or subject. 

In the form builder, you can add different fields to your survey form and you can also modify these form fields to suit your research process. Sign up to Formplus to access the form builder and start creating powerful online empirical research survey forms. 

Connect to Formplus, Get Started Now - It's Free!

• advantage of empirical research
• disadvantages of empirical resarch
• empirical research characteristics
• empirical research cycle
• empirical research method
• example of empirical research
• uses of empirical research
• busayo.longe

You may also like:

Research Questions: Definitions, Types + [Examples]

A comprehensive guide on the definition of research questions, types, importance, good and bad research question examples

Recall Bias: Definition, Types, Examples & Mitigation

This article will discuss the impact of recall bias in studies and the best ways to avoid them during research.

Extrapolation in Statistical Research: Definition, Examples, Types, Applications

In this article we’ll look at the different types and characteristics of extrapolation, plus how it contrasts to interpolation.

What is Pure or Basic Research? + [Examples & Method]

Simple guide on pure or basic research, its methods, characteristics, advantages, and examples in science, medicine, education and psychology

Formplus - For Seamless Data Collection

Collect data the right way with a versatile data collection tool. try formplus and transform your work productivity today..

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Published: 15 September 2022

Interviews in the social sciences

• Eleanor Knott   ORCID: orcid.org/0000-0002-9131-3939 1 ,
• Aliya Hamid Rao   ORCID: orcid.org/0000-0003-0674-4206 1 ,
• Kate Summers   ORCID: orcid.org/0000-0001-9964-0259 1 &
• Chana Teeger   ORCID: orcid.org/0000-0002-5046-8280 1  

Nature Reviews Methods Primers volume  2 , Article number:  73 ( 2022 ) Cite this article

738k Accesses

90 Citations

190 Altmetric

Metrics details

• Interdisciplinary studies

In-depth interviews are a versatile form of qualitative data collection used by researchers across the social sciences. They allow individuals to explain, in their own words, how they understand and interpret the world around them. Interviews represent a deceptively familiar social encounter in which people interact by asking and answering questions. They are, however, a very particular type of conversation, guided by the researcher and used for specific ends. This dynamic introduces a range of methodological, analytical and ethical challenges, for novice researchers in particular. In this Primer, we focus on the stages and challenges of designing and conducting an interview project and analysing data from it, as well as strategies to overcome such challenges.

Similar content being viewed by others

The fundamental importance of method to theory

How ‘going online’ mediates the challenges of policy elite interviews

Participatory action research

Introduction.

In-depth interviews are a qualitative research method that follow a deceptively familiar logic of human interaction: they are conversations where people talk with each other, interact and pose and answer questions 1 . An interview is a specific type of interaction in which — usually and predominantly — a researcher asks questions about someone’s life experience, opinions, dreams, fears and hopes and the interview participant answers the questions 1 .

Interviews will often be used as a standalone method or combined with other qualitative methods, such as focus groups or ethnography, or quantitative methods, such as surveys or experiments. Although interviewing is a frequently used method, it should not be viewed as an easy default for qualitative researchers 2 . Interviews are also not suited to answering all qualitative research questions, but instead have specific strengths that should guide whether or not they are deployed in a research project. Whereas ethnography might be better suited to trying to observe what people do, interviews provide a space for extended conversations that allow the researcher insights into how people think and what they believe. Quantitative surveys also give these kinds of insights, but they use pre-determined questions and scales, privileging breadth over depth and often overlooking harder-to-reach participants.

In-depth interviews can take many different shapes and forms, often with more than one participant or researcher. For example, interviews might be highly structured (using an almost survey-like interview guide), entirely unstructured (taking a narrative and free-flowing approach) or semi-structured (using a topic guide ). Researchers might combine these approaches within a single project depending on the purpose of the interview and the characteristics of the participant. Whatever form the interview takes, researchers should be mindful of the dynamics between interviewer and participant and factor these in at all stages of the project.

In this Primer, we focus on the most common type of interview: one researcher taking a semi-structured approach to interviewing one participant using a topic guide. Focusing on how to plan research using interviews, we discuss the necessary stages of data collection. We also discuss the stages and thought-process behind analysing interview material to ensure that the richness and interpretability of interview material is maintained and communicated to readers. The Primer also tracks innovations in interview methods and discusses the developments we expect over the next 5–10 years.

We wrote this Primer as researchers from sociology, social policy and political science. We note our disciplinary background because we acknowledge that there are disciplinary differences in how interviews are approached and understood as a method.

Experimentation

Here we address research design considerations and data collection issues focusing on topic guide construction and other pragmatics of the interview. We also explore issues of ethics and reflexivity that are crucial throughout the research project.

Research design

Participant selection.

Participants can be selected and recruited in various ways for in-depth interview studies. The researcher must first decide what defines the people or social groups being studied. Often, this means moving from an abstract theoretical research question to a more precise empirical one. For example, the researcher might be interested in how people talk about race in contexts of diversity. Empirical settings in which this issue could be studied could include schools, workplaces or adoption agencies. The best research designs should clearly explain why the particular setting was chosen. Often there are both intrinsic and extrinsic reasons for choosing to study a particular group of people at a specific time and place 3 . Intrinsic motivations relate to the fact that the research is focused on an important specific social phenomenon that has been understudied. Extrinsic motivations speak to the broader theoretical research questions and explain why the case at hand is a good one through which to address them empirically.

Next, the researcher needs to decide which types of people they would like to interview. This decision amounts to delineating the inclusion and exclusion criteria for the study. The criteria might be based on demographic variables, like race or gender, but they may also be context-specific, for example, years of experience in an organization. These should be decided based on the research goals. Researchers should be clear about what characteristics would make an individual a candidate for inclusion in the study (and what would exclude them).

The next step is to identify and recruit the study’s sample . Usually, many more people fit the inclusion criteria than can be interviewed. In cases where lists of potential participants are available, the researcher might want to employ stratified sampling , dividing the list by characteristics of interest before sampling.

When there are no lists, researchers will often employ purposive sampling . Many researchers consider purposive sampling the most useful mode for interview-based research since the number of interviews to be conducted is too small to aim to be statistically representative 4 . Instead, the aim is not breadth, via representativeness, but depth via rich insights about a set of participants. In addition to purposive sampling, researchers often use snowball sampling . Both purposive and snowball sampling can be combined with quota sampling . All three types of sampling aim to ensure a variety of perspectives within the confines of a research project. A goal for in-depth interview studies can be to sample for range, being mindful of recruiting a diversity of participants fitting the inclusion criteria.

Study design

The total number of interviews depends on many factors, including the population studied, whether comparisons are to be made and the duration of interviews. Studies that rely on quota sampling where explicit comparisons are made between groups will require a larger number of interviews than studies focused on one group only. Studies where participants are interviewed over several hours, days or even repeatedly across years will tend to have fewer participants than those that entail a one-off engagement.

Researchers often stop interviewing when new interviews confirm findings from earlier interviews with no new or surprising insights (saturation) 4 , 5 , 6 . As a criterion for research design, saturation assumes that data collection and analysis are happening in tandem and that researchers will stop collecting new data once there is no new information emerging from the interviews. This is not always possible. Researchers rarely have time for systematic data analysis during data collection and they often need to specify their sample in funding proposals prior to data collection. As a result, researchers often draw on existing reports of saturation to estimate a sample size prior to data collection. These suggest between 12 and 20 interviews per category of participant (although researchers have reported saturation with samples that are both smaller and larger than this) 7 , 8 , 9 . The idea of saturation has been critiqued by many qualitative researchers because it assumes that meaning inheres in the data, waiting to be discovered — and confirmed — once saturation has been reached 7 . In-depth interview data are often multivalent and can give rise to different interpretations. The important consideration is, therefore, not merely how many participants are interviewed, but whether one’s research design allows for collecting rich and textured data that provide insight into participants’ understandings, accounts, perceptions and interpretations.

Sometimes, researchers will conduct interviews with more than one participant at a time. Researchers should consider the benefits and shortcomings of such an approach. Joint interviews may, for example, give researchers insight into how caregivers agree or debate childrearing decisions. At the same time, they may be less adaptive to exploring aspects of caregiving that participants may not wish to disclose to each other. In other cases, there may be more than one person interviewing each participant, such as when an interpreter is used, and so it is important to consider during the research design phase how this might shape the dynamics of the interview.

Data collection

Semi-structured interviews are typically organized around a topic guide comprised of an ordered set of broad topics (usually 3–5). Each topic includes a set of questions that form the basis of the discussion between the researcher and participant (Fig.  1 ). These topics are organized around key concepts that the researcher has identified (for example, through a close study of prior research, or perhaps through piloting a small, exploratory study) 5 .

a | Elaborated topics the researcher wants to cover in the interview and example questions. b | An example topic arc. Using such an arc, one can think flexibly about the order of topics. Considering the main question for each topic will help to determine the best order for the topics. After conducting some interviews, the researcher can move topics around if a different order seems to make sense.

Topic guide

One common way to structure a topic guide is to start with relatively easy, open-ended questions (Table  1 ). Opening questions should be related to the research topic but broad and easy to answer, so that they help to ease the participant into conversation.

After these broad, opening questions, the topic guide may move into topics that speak more directly to the overarching research question. The interview questions will be accompanied by probes designed to elicit concrete details and examples from the participant (see Table  1 ).

Abstract questions are often easier for participants to answer once they have been asked more concrete questions. In our experience, for example, questions about feelings can be difficult for some participants to answer, but when following probes concerning factual experiences these questions can become less challenging. After the main themes of the topic guide have been covered, the topic guide can move onto closing questions. At this stage, participants often repeat something they have said before, although they may sometimes introduce a new topic.

Interviews are especially well suited to gaining a deeper insight into people’s experiences. Getting these insights largely depends on the participants’ willingness to talk to the researcher. We recommend designing open-ended questions that are more likely to elicit an elaborated response and extended reflection from participants rather than questions that can be answered with yes or no.

Questions should avoid foreclosing the possibility that the participant might disagree with the premise of the question. Take for example the question: “Do you support the new family-friendly policies?” This question minimizes the possibility of the participant disagreeing with the premise of this question, which assumes that the policies are ‘family-friendly’ and asks for a yes or no answer. Instead, asking more broadly how a participant feels about the specific policy being described as ‘family-friendly’ (for example, a work-from-home policy) allows them to express agreement, disagreement or impartiality and, crucially, to explain their reasoning 10 .

For an uninterrupted interview that will last between 90 and 120 minutes, the topic guide should be one to two single-spaced pages with questions and probes. Ideally, the researcher will memorize the topic guide before embarking on the first interview. It is fine to carry a printed-out copy of the topic guide but memorizing the topic guide ahead of the interviews can often make the interviewer feel well prepared in guiding the participant through the interview process.

Although the topic guide helps the researcher stay on track with the broad areas they want to cover, there is no need for the researcher to feel tied down by the topic guide. For instance, if a participant brings up a theme that the researcher intended to discuss later or a point the researcher had not anticipated, the researcher may well decide to follow the lead of the participant. The researcher’s role extends beyond simply stating the questions; it entails listening and responding, making split-second decisions about what line of inquiry to pursue and allowing the interview to proceed in unexpected directions.

Optimizing the interview

The ideal place for an interview will depend on the study and what is feasible for participants. Generally, a place where the participant and researcher can both feel relaxed, where the interview can be uninterrupted and where noise or other distractions are limited is ideal. But this may not always be possible and so the researcher needs to be prepared to adapt their plans within what is feasible (and desirable for participants).

Another key tool for the interview is a recording device (assuming that permission for recording has been given). Recording can be important to capture what the participant says verbatim. Additionally, it can allow the researcher to focus on determining what probes and follow-up questions they want to pursue rather than focusing on taking notes. Sometimes, however, a participant may not allow the researcher to record, or the recording may fail. If the interview is not recorded we suggest that the researcher takes brief notes during the interview, if feasible, and then thoroughly make notes immediately after the interview and try to remember the participant’s facial expressions, gestures and tone of voice. Not having a recording of an interview need not limit the researcher from getting analytical value from it.

As soon as possible after each interview, we recommend that the researcher write a one-page interview memo comprising three key sections. The first section should identify two to three important moments from the interview. What constitutes important is up to the researcher’s discretion 9 . The researcher should note down what happened in these moments, including the participant’s facial expressions, gestures, tone of voice and maybe even the sensory details of their surroundings. This exercise is about capturing ethnographic detail from the interview. The second part of the interview memo is the analytical section with notes on how the interview fits in with previous interviews, for example, where the participant’s responses concur or diverge from other responses. The third part consists of a methodological section where the researcher notes their perception of their relationship with the participant. The interview memo allows the researcher to think critically about their positionality and practice reflexivity — key concepts for an ethical and transparent research practice in qualitative methodology 11 , 12 .

Ethics and reflexivity

All elements of an in-depth interview can raise ethical challenges and concerns. Good ethical practice in interview studies often means going beyond the ethical procedures mandated by institutions 13 . While discussions and requirements of ethics can differ across disciplines, here we focus on the most pertinent considerations for interviews across the research process for an interdisciplinary audience.

Ethical considerations prior to interview

Before conducting interviews, researchers should consider harm minimization, informed consent, anonymity and confidentiality, and reflexivity and positionality. It is important for the researcher to develop their own ethical sensitivities and sensibilities by gaining training in interview and qualitative methods, reading methodological and field-specific texts on interviews and ethics and discussing their research plans with colleagues.

Researchers should map the potential harm to consider how this can be minimized. Primarily, researchers should consider harm from the participants’ perspective (Box  1 ). But, it is also important to consider and plan for potential harm to the researcher, research assistants, gatekeepers, future researchers and members of the wider community 14 . Even the most banal of research topics can potentially pose some form of harm to the participant, researcher and others — and the level of harm is often highly context-dependent. For example, a research project on religion in society might have very different ethical considerations in a democratic versus authoritarian research context because of how openly or not such topics can be discussed and debated 15 .

The researcher should consider how they will obtain and record informed consent (for example, written or oral), based on what makes the most sense for their research project and context 16 . Some institutions might specify how informed consent should be gained. Regardless of how consent is obtained, the participant must be made aware of the form of consent, the intentions and procedures of the interview and potential forms of harm and benefit to the participant or community before the interview commences. Moreover, the participant must agree to be interviewed before the interview commences. If, in addition to interviews, the study contains an ethnographic component, it is worth reading around this topic (see, for example, Murphy and Dingwall 17 ). Informed consent must also be gained for how the interview will be recorded before the interview commences. These practices are important to ensure the participant is contributing on a voluntary basis. It is also important to remind participants that they can withdraw their consent at any time during the interview and for a specified period after the interview (to be decided with the participant). The researcher should indicate that participants can ask for anything shared to be off the record and/or not disseminated.

In terms of anonymity and confidentiality, it is standard practice when conducting interviews to agree not to use (or even collect) participants’ names and personal details that are not pertinent to the study. Anonymizing can often be the safer option for minimizing harm to participants as it is hard to foresee all the consequences of de-anonymizing, even if participants agree. Regardless of what a researcher decides, decisions around anonymity must be agreed with participants during the process of gaining informed consent and respected following the interview.

Although not all ethical challenges can be foreseen or planned for 18 , researchers should think carefully — before the interview — about power dynamics, participant vulnerability, emotional state and interactional dynamics between interviewer and participant, even when discussing low-risk topics. Researchers may then wish to plan for potential ethical issues, for example by preparing a list of relevant organizations to which participants can be signposted. A researcher interviewing a participant about debt, for instance, might prepare in advance a list of debt advice charities, organizations and helplines that could provide further support and advice. It is important to remember that the role of an interviewer is as a researcher rather than as a social worker or counsellor because researchers may not have relevant and requisite training in these other domains.

Box 1 Mapping potential forms of harm

Social: researchers should avoid causing any relational detriment to anyone in the course of interviews, for example, by sharing information with other participants or causing interview participants to be shunned or mistreated by their community as a result of participating.

Economic: researchers should avoid causing financial detriment to anyone, for example, by expecting them to pay for transport to be interviewed or to potentially lose their job as a result of participating.

Physical: researchers should minimize the risk of anyone being exposed to violence as a result of the research both from other individuals or from authorities, including police.

Psychological: researchers should minimize the risk of causing anyone trauma (or re-traumatization) or psychological anguish as a result of the research; this includes not only the participant but importantly the researcher themselves and anyone that might read or analyse the transcripts, should they contain triggering information.

Political: researchers should minimize the risk of anyone being exposed to political detriment as a result of the research, such as retribution.

Professional/reputational: researchers should minimize the potential for reputational damage to anyone connected to the research (this includes ensuring good research practices so that any researchers involved are not harmed reputationally by being involved with the research project).

The task here is not to map exhaustively the potential forms of harm that might pertain to a particular research project (that is the researcher’s job and they should have the expertise most suited to mapping such potential harms relative to the specific project) but to demonstrate the breadth of potential forms of harm.

Ethical considerations post-interview

Researchers should consider how interview data are stored, analysed and disseminated. If participants have been offered anonymity and confidentiality, data should be stored in a way that does not compromise this. For example, researchers should consider removing names and any other unnecessary personal details from interview transcripts, password-protecting and encrypting files and using pseudonyms to label and store all interview data. It is also important to address where interview data are taken (for example, across borders in particular where interview data might be of interest to local authorities) and how this might affect the storage of interview data.

Examining how the researcher will represent participants is a paramount ethical consideration both in the planning stages of the interview study and after it has been conducted. Dissemination strategies also need to consider questions of anonymity and representation. In small communities, even if participants are given pseudonyms, it might be obvious who is being described. Anonymizing not only the names of those participating but also the research context is therefore a standard practice 19 . With particularly sensitive data or insights about the participant, it is worth considering describing participants in a more abstract way rather than as specific individuals. These practices are important both for protecting participants’ anonymity but can also affect the ability of the researcher and others to return ethically to the research context and similar contexts 20 .

Reflexivity and positionality

Reflexivity and positionality mean considering the researcher’s role and assumptions in knowledge production 13 . A key part of reflexivity is considering the power relations between the researcher and participant within the interview setting, as well as how researchers might be perceived by participants. Further, researchers need to consider how their own identities shape the kind of knowledge and assumptions they bring to the interview, including how they approach and ask questions and their analysis of interviews (Box  2 ). Reflexivity is a necessary part of developing ethical sensibility as a researcher by adapting and reflecting on how one engages with participants. Participants should not feel judged, for example, when they share information that researchers might disagree with or find objectionable. How researchers deal with uncomfortable moments or information shared by participants is at their discretion, but they should consider how they will react both ahead of time and in the moment.

Researchers can develop their reflexivity by considering how they themselves would feel being asked these interview questions or represented in this way, and then adapting their practice accordingly. There might be situations where these questions are not appropriate in that they unduly centre the researchers’ experiences and worldview. Nevertheless, these prompts can provide a useful starting point for those beginning their reflexive journey and developing an ethical sensibility.

Reflexivity and ethical sensitivities require active reflection throughout the research process. For example, researchers should take care in interview memos and their notes to consider their assumptions, potential preconceptions, worldviews and own identities prior to and after interviews (Box  2 ). Checking in with assumptions can be a way of making sure that researchers are paying close attention to their own theoretical and analytical biases and revising them in accordance with what they learn through the interviews. Researchers should return to these notes (especially when analysing interview material), to try to unpack their own effects on the research process as well as how participants positioned and engaged with them.

Box 2 Aspects to reflect on reflexively

For reflexive engagement, and understanding the power relations being co-constructed and (re)produced in interviews, it is necessary to reflect, at a minimum, on the following.

Ethnicity, race and nationality, such as how does privilege stemming from race or nationality operate between the researcher, the participant and research context (for example, a researcher from a majority community may be interviewing a member of a minority community)

Gender and sexuality, see above on ethnicity, race and nationality

Social class, and in particular the issue of middle-class bias among researchers when formulating research and interview questions

Economic security/precarity, see above on social class and thinking about the researcher’s relative privilege and the source of biases that stem from this

Educational experiences and privileges, see above

Disciplinary biases, such as how the researcher’s discipline/subfield usually approaches these questions, possibly normalizing certain assumptions that might be contested by participants and in the research context

Political and social values

Lived experiences and other dimensions of ourselves that affect and construct our identity as researchers

In this section, we discuss the next stage of an interview study, namely, analysing the interview data. Data analysis may begin while more data are being collected. Doing so allows early findings to inform the focus of further data collection, as part of an iterative process across the research project. Here, the researcher is ultimately working towards achieving coherence between the data collected and the findings produced to answer successfully the research question(s) they have set.

The two most common methods used to analyse interview material across the social sciences are thematic analysis 21 and discourse analysis 22 . Thematic analysis is a particularly useful and accessible method for those starting out in analysis of qualitative data and interview material as a method of coding data to develop and interpret themes in the data 21 . Discourse analysis is more specialized and focuses on the role of discourse in society by paying close attention to the explicit, implicit and taken-for-granted dimensions of language and power 22 , 23 . Although thematic and discourse analysis are often discussed as separate techniques, in practice researchers might flexibly combine these approaches depending on the object of analysis. For example, those intending to use discourse analysis might first conduct thematic analysis as a way to organize and systematize the data. The object and intention of analysis might differ (for example, developing themes or interrogating language), but the questions facing the researcher (such as whether to take an inductive or deductive approach to analysis) are similar.

Preparing data

Data preparation is an important step in the data analysis process. The researcher should first determine what comprises the corpus of material and in what form it will it be analysed. The former refers to whether, for example, alongside the interviews themselves, analytic memos or observational notes that may have been taken during data collection will also be directly analysed. The latter refers to decisions about how the verbal/audio interview data will be transformed into a written form, making it suitable for processes of data analysis. Typically, interview audio recordings are transcribed to produce a written transcript. It is important to note that the process of transcription is one of transformation. The verbal interview data are transformed into a written transcript through a series of decisions that the researcher must make. The researcher should consider the effect of mishearing what has been said or how choosing to punctuate a sentence in a particular way will affect the final analysis.

Box  3 shows an example transcript excerpt from an interview with a teacher conducted by Teeger as part of her study of history education in post-apartheid South Africa 24 (Box  3 ). Seeing both the questions and the responses means that the reader can contextualize what the participant (Ms Mokoena) has said. Throughout the transcript the researcher has used square brackets, for example to indicate a pause in speech, when Ms Mokoena says “it’s [pause] it’s a difficult topic”. The transcription choice made here means that we see that Ms Mokoena has taken time to pause, perhaps to search for the right words, or perhaps because she has a slight apprehension. Square brackets are also included as an overt act of communication to the reader. When Ms Mokoena says “ja”, the English translation (“yes”) of the word in Afrikaans is placed in square brackets to ensure that the reader can follow the meaning of the speech.

Decisions about what to include when transcribing will be hugely important for the direction and possibilities of analysis. Researchers should decide what they want to capture in the transcript, based on their analytic focus. From a (post)positivist perspective 25 , the researcher may be interested in the manifest content of the interview (such as what is said, not how it is said). In that case, they may choose to transcribe intelligent verbatim . From a constructivist perspective 25 , researchers may choose to record more aspects of speech (including, for example, pauses, repetitions, false starts, talking over one another) so that these features can be analysed. Those working from this perspective argue that to recognize the interactional nature of the interview setting adequately and to avoid misinterpretations, features of interaction (pauses, overlaps between speakers and so on) should be preserved in transcription and therefore in the analysis 10 . Readers interested in learning more should consult Potter and Hepburn’s summary of how to present interaction through transcription of interview data 26 .

The process of analysing semi-structured interviews might be thought of as a generative rather than an extractive enterprise. Findings do not already exist within the interview data to be discovered. Rather, researchers create something new when analysing the data by applying their analytic lens or approach to the transcripts. At a high level, there are options as to what researchers might want to glean from their interview data. They might be interested in themes, whereby they identify patterns of meaning across the dataset 21 . Alternatively, they may focus on discourse(s), looking to identify how language is used to construct meanings and therefore how language reinforces or produces aspects of the social world 27 . Alternatively, they might look at the data to understand narrative or biographical elements 28 .

A further overarching decision to make is the extent to which researchers bring predetermined framings or understandings to bear on their data, or instead begin from the data themselves to generate an analysis. One way of articulating this is the extent to which researchers take a deductive approach or an inductive approach to analysis. One example of a truly inductive approach is grounded theory, whereby the aim of the analysis is to build new theory, beginning with one’s data 6 , 29 . In practice, researchers using thematic and discourse analysis often combine deductive and inductive logics and describe their process instead as iterative (referred to also as an abductive approach ) 30 , 31 . For example, researchers may decide that they will apply a given theoretical framing, or begin with an initial analytic framework, but then refine or develop these once they begin the process of analysis.

Box 3 Excerpt of interview transcript (from Teeger 24 )

Interviewer : Maybe you could just start by talking about what it’s like to teach apartheid history.

Ms Mokoena : It’s a bit challenging. You’ve got to accommodate all the kids in the class. You’ve got to be sensitive to all the racial differences. You want to emphasize the wrongs that were done in the past but you also want to, you know, not to make kids feel like it’s their fault. So you want to use the wrongs of the past to try and unite the kids …

Interviewer : So what kind of things do you do?

Ms Mokoena : Well I normally highlight the fact that people that were struggling were not just the blacks, it was all the races. And I give examples of the people … from all walks of life, all races, and highlight how they suffered as well as a result of apartheid, particularly the whites… . What I noticed, particularly my first year of teaching apartheid, I noticed that the black kids made the others feel responsible for what happened… . I had a lot of fights…. A lot of kids started hating each other because, you know, the others are white and the others were black. And they started saying, “My mother is a domestic worker because she was never allowed an opportunity to get good education.” …

Interviewer : I didn’t see any of that now when I was observing.

Ms Mokoena : … Like I was saying I think that because of the re-emphasis of the fact that, look, everybody did suffer one way or the other, they sort of got to see that it was everybody’s struggle … . They should now get to understand that that’s why we’re called a Rainbow Nation. Not everybody agreed with apartheid and not everybody suffered. Even all the blacks, not all blacks got to feel what the others felt . So ja [yes], it’s [pause] it’s a difficult topic, ja . But I think if you get the kids to understand why we’re teaching apartheid in the first place and you show the involvement of all races in all the different sides , then I think you have managed to teach it properly. So I think because of my inexperience then — that was my first year of teaching history — so I think I — maybe I over-emphasized the suffering of the blacks versus the whites [emphasis added].

Reprinted with permission from ref. 24 , Sage Publications.

From data to codes

Coding data is a key building block shared across many approaches to data analysis. Coding is a way of organizing and describing data, but is also ultimately a way of transforming data to produce analytic insights. The basic practice of coding involves highlighting a segment of text (this may be a sentence, a clause or a longer excerpt) and assigning a label to it. The aim of the label is to communicate some sort of summary of what is in the highlighted piece of text. Coding is an iterative process, whereby researchers read and reread their transcripts, applying and refining their codes, until they have a coding frame (a set of codes) that is applied coherently across the dataset and that captures and communicates the key features of what is contained in the data as it relates to the researchers’ analytic focus.

What one codes for is entirely contingent on the focus of the research project and the choices the researcher makes about the approach to analysis. At first, one might apply descriptive codes, summarizing what is contained in the interviews. It is rarely desirable to stop at this point, however, because coding is a tool to move from describing the data to interpreting the data. Suppose the researcher is pursuing some version of thematic analysis. In that case, it might be that the objects of coding are aspects of reported action, emotions, opinions, norms, relationships, routines, agreement/disagreement and change over time. A discourse analysis might instead code for different types of speech acts, tropes, linguistic or rhetorical devices. Multiple types of code might be generated within the same research project. What is important is that researchers are aware of the choices they are making in terms of what they are coding for. Moreover, through the process of refinement, the aim is to produce a set of discrete codes — in which codes are conceptually distinct, as opposed to overlapping. By using the same codes across the dataset, the researcher can capture commonalities across the interviews. This process of refinement involves relabelling codes and reorganizing how and where they are applied in the dataset.

From coding to analysis and writing

Data analysis is also an iterative process in which researchers move closer to and further away from the data. As they move away from the data, they synthesize their findings, thus honing and articulating their analytic insights. As they move closer to the data, they ground these insights in what is contained in the interviews. The link should not be broken between the data themselves and higher-order conceptual insights or claims being made. Researchers must be able to show evidence for their claims in the data. Figure  2 summarizes this iterative process and suggests the sorts of activities involved at each stage more concretely.

As well as going through steps 1 to 6 in order, the researcher will also go backwards and forwards between stages. Some stages will themselves be a forwards and backwards processing of coding and refining when working across different interview transcripts.

At the stage of synthesizing, there are some common quandaries. When dealing with a dataset consisting of multiple interviews, there will be salient and minority statements across different participants, or consensus or dissent on topics of interest to the researcher. A strength of qualitative interviews is that we can build in these nuances and variations across our data as opposed to aggregating them away. When exploring and reporting data, researchers should be asking how different findings are patterned and which interviews contain which codes, themes or tropes. Researchers should think about how these variations fit within the longer flow of individual interviews and what these variations tell them about the nature of their substantive research interests.

A further consideration is how to approach analysis within and across interview data. Researchers may look at one individual code, to examine the forms it takes across different participants and what they might be able to summarize about this code in the round. Alternatively, they might look at how a code or set of codes pattern across the account of one participant, to understand the code(s) in a more contextualized way. Further analysis might be done according to different sampling characteristics, where researchers group together interviews based on certain demographic characteristics and explore these together.

When it comes to writing up and presenting interview data, key considerations tend to rest on what is often termed transparency. When presenting the findings of an interview-based study, the reader should be able to understand and trace what the stated findings are based upon. This process typically involves describing the analytic process, how key decisions were made and presenting direct excerpts from the data. It is important to account for how the interview was set up and to consider the active part that the researcher has played in generating the data 32 . Quotes from interviews should not be thought of as merely embellishing or adding interest to a final research output. Rather, quotes serve the important function of connecting the reader directly to the underlying data. Quotes, therefore, should be chosen because they provide the reader with the most apt insight into what is being discussed. It is good practice to report not just on what participants said, but also on the questions that were asked to elicit the responses.

Researchers have increasingly used specialist qualitative data analysis software to organize and analyse their interview data, such as NVivo or ATLAS.ti. It is important to remember that such software is a tool for, rather than an approach or technique of, analysis. That said, software also creates a wide range of possibilities in terms of what can be done with the data. As researchers, we should reflect on how the range of possibilities of a given software package might be shaping our analytical choices and whether these are choices that we do indeed want to make.

Applications

This section reviews how and why in-depth interviews have been used by researchers studying gender, education and inequality, nationalism and ethnicity and the welfare state. Although interviews can be employed as a method of data collection in just about any social science topic, the applications below speak directly to the authors’ expertise and cutting-edge areas of research.

When it comes to the broad study of gender, in-depth interviews have been invaluable in shaping our understanding of how gender functions in everyday life. In a study of the US hedge fund industry (an industry dominated by white men), Tobias Neely was interested in understanding the factors that enable white men to prosper in the industry 33 . The study comprised interviews with 45 hedge fund workers and oversampled women of all races and men of colour to capture a range of experiences and beliefs. Tobias Neely found that practices of hiring, grooming and seeding are key to maintaining white men’s dominance in the industry. In terms of hiring, the interviews clarified that white men in charge typically preferred to hire people like themselves, usually from their extended networks. When women were hired, they were usually hired to less lucrative positions. In terms of grooming, Tobias Neely identifies how older and more senior men in the industry who have power and status will select one or several younger men as their protégés, to include in their own elite networks. Finally, in terms of her concept of seeding, Tobias Neely describes how older men who are hedge fund managers provide the seed money (often in the hundreds of millions of dollars) for a hedge fund to men, often their own sons (but not their daughters). These interviews provided an in-depth look into gendered and racialized mechanisms that allow white men to flourish in this industry.

Research by Rao draws on dozens of interviews with men and women who had lost their jobs, some of the participants’ spouses and follow-up interviews with about half the sample approximately 6 months after the initial interview 34 . Rao used interviews to understand the gendered experience and understanding of unemployment. Through these interviews, she found that the very process of losing their jobs meant different things for men and women. Women often saw job loss as being a personal indictment of their professional capabilities. The women interviewed often referenced how years of devaluation in the workplace coloured their interpretation of their job loss. Men, by contrast, were also saddened by their job loss, but they saw it as part and parcel of a weak economy rather than a personal failing. How these varied interpretations occurred was tied to men’s and women’s very different experiences in the workplace. Further, through her analysis of these interviews, Rao also showed how these gendered interpretations had implications for the kinds of jobs men and women sought to pursue after job loss. Whereas men remained tied to participating in full-time paid work, job loss appeared to be a catalyst pushing some of the women to re-evaluate their ties to the labour force.

In a study of workers in the tech industry, Hart used interviews to explain how individuals respond to unwanted and ambiguously sexual interactions 35 . Here, the researcher used interviews to allow participants to describe how these interactions made them feel and act and the logics of how they interpreted, classified and made sense of them 35 . Through her analysis of these interviews, Hart showed that participants engaged in a process she termed “trajectory guarding”, whereby they sought to monitor unwanted and ambiguously sexual interactions to avoid them from escalating. Yet, as Hart’s analysis proficiently demonstrates, these very strategies — which protect these workers sexually — also undermined their workplace advancement.

Drawing on interviews, these studies have helped us to understand better how gendered mechanisms, gendered interpretations and gendered interactions foster gender inequality when it comes to paid work. Methodologically, these studies illuminate the power of interviews to reveal important aspects of social life.

Nationalism and ethnicity

Traditionally, nationalism has been studied from a top-down perspective, through the lens of the state or using historical methods; in other words, in-depth interviews have not been a common way of collecting data to study nationalism. The methodological turn towards everyday nationalism has encouraged more scholars to go to the field and use interviews (and ethnography) to understand nationalism from the bottom up: how people talk about, give meaning, understand, navigate and contest their relation to nation, national identification and nationalism 36 , 37 , 38 , 39 . This turn has also addressed the gap left by those studying national and ethnic identification via quantitative methods, such as surveys.

Surveys can enumerate how individuals ascribe to categorical forms of identification 40 . However, interviews can question the usefulness of such categories and ask whether these categories are reflected, or resisted, by participants in terms of the meanings they give to identification 41 , 42 . Categories often pitch identification as a mutually exclusive choice; but identification might be more complex than such categories allow. For example, some might hybridize these categories or see themselves as moving between and across categories 43 . Hearing how people talk about themselves and their relation to nations, states and ethnicities, therefore, contributes substantially to the study of nationalism and national and ethnic forms of identification.

One particular approach to studying these topics, whether via everyday nationalism or alternatives, is that of using interviews to capture both articulations and narratives of identification, relations to nationalism and the boundaries people construct. For example, interviews can be used to gather self–other narratives by studying how individuals construct I–we–them boundaries 44 , including how participants talk about themselves, who participants include in their various ‘we’ groupings and which and how participants create ‘them’ groupings of others, inserting boundaries between ‘I/we’ and ‘them’. Overall, interviews hold great potential for listening to participants and understanding the nuances of identification and the construction of boundaries from their point of view.

Education and inequality

Scholars of social stratification have long noted that the school system often reproduces existing social inequalities. Carter explains that all schools have both material and sociocultural resources 45 . When children from different backgrounds attend schools with different material resources, their educational and occupational outcomes are likely to vary. Such material resources are relatively easy to measure. They are operationalized as teacher-to-student ratios, access to computers and textbooks and the physical infrastructure of classrooms and playgrounds.

Drawing on Bourdieusian theory 46 , Carter conceptualizes the sociocultural context as the norms, values and dispositions privileged within a social space 45 . Scholars have drawn on interviews with students and teachers (as well as ethnographic observations) to show how schools confer advantages on students from middle-class families, for example, by rewarding their help-seeking behaviours 47 . Focusing on race, researchers have revealed how schools can remain socioculturally white even as they enrol a racially diverse student population. In such contexts, for example, teachers often misrecognize the aesthetic choices made by students of colour, wrongly inferring that these students’ tastes in clothing and music reflect negative orientations to schooling 48 , 49 , 50 . These assessments can result in disparate forms of discipline and may ultimately shape educators’ assessments of students’ academic potential 51 .

Further, teachers and administrators tend to view the appropriate relationship between home and school in ways that resonate with white middle-class parents 52 . These parents are then able to advocate effectively for their children in ways that non-white parents are not 53 . In-depth interviews are particularly good at tapping into these understandings, revealing the mechanisms that confer privilege on certain groups of students and thereby reproduce inequality.

In addition, interviews can shed light on the unequal experiences that young people have within educational institutions, as the views of dominant groups are affirmed while those from disadvantaged backgrounds are delegitimized. For example, Teeger’s interviews with South African high schoolers showed how — because racially charged incidents are often framed as jokes in the broader school culture — Black students often feel compelled to ignore and keep silent about the racism they experience 54 . Interviews revealed that Black students who objected to these supposed jokes were coded by other students as serious or angry. In trying to avoid such labels, these students found themselves unable to challenge the racism they experienced. Interviews give us insight into these dynamics and help us see how young people understand and interpret the messages transmitted in schools — including those that speak to issues of inequality in their local school contexts as well as in society more broadly 24 , 55 .

The welfare state

In-depth interviews have also proved to be an important method for studying various aspects of the welfare state. By welfare state, we mean the social institutions relating to the economic and social wellbeing of a state’s citizens. Notably, using interviews has been useful to look at how policy design features are experienced and play out on the ground. Interviews have often been paired with large-scale surveys to produce mixed-methods study designs, therefore achieving both breadth and depth of insights.

In-depth interviews provide the opportunity to look behind policy assumptions or how policies are designed from the top down, to examine how these play out in the lives of those affected by the policies and whose experiences might otherwise be obscured or ignored. For example, the Welfare Conditionality project used interviews to critique the assumptions that conditionality (such as, the withdrawal of social security benefits if recipients did not perform or meet certain criteria) improved employment outcomes and instead showed that conditionality was harmful to mental health, living standards and had many other negative consequences 56 . Meanwhile, combining datasets from two small-scale interview studies with recipients allowed Summers and Young to critique assumptions around the simplicity that underpinned the design of Universal Credit in 2020, for example, showing that the apparently simple monthly payment design instead burdened recipients with additional money management decisions and responsibilities 57 .

Similarly, the Welfare at a (Social) Distance project used a mixed-methods approach in a large-scale study that combined national surveys with case studies and in-depth interviews to investigate the experience of claiming social security benefits during the COVID-19 pandemic. The interviews allowed researchers to understand in detail any issues experienced by recipients of benefits, such as delays in the process of claiming, managing on a very tight budget and navigating stigma and claiming 58 .

These applications demonstrate the multi-faceted topics and questions for which interviews can be a relevant method for data collection. These applications highlight not only the relevance of interviews, but also emphasize the key added value of interviews, which might be missed by other methods (surveys, in particular). Interviews can expose and question what is taken for granted and directly engage with communities and participants that might otherwise be ignored, obscured or marginalized.

Reproducibility and data deposition

There is a robust, ongoing debate about reproducibility in qualitative research, including interview studies. In some research paradigms, reproducibility can be a way of interrogating the rigour and robustness of research claims, by seeing whether these hold up when the research process is repeated. Some scholars have suggested that although reproducibility may be challenging, researchers can facilitate it by naming the place where the research was conducted, naming participants, sharing interview and fieldwork transcripts (anonymized and de-identified in cases where researchers are not naming people or places) and employing fact-checkers for accuracy 11 , 59 , 60 .

In addition to the ethical concerns of whether de-anonymization is ever feasible or desirable, it is also important to address whether the replicability of interview studies is meaningful. For example, the flexibility of interviews allows for the unexpected and the unforeseen to be incorporated into the scope of the research 61 . However, this flexibility means that we cannot expect reproducibility in the conventional sense, given that different researchers will elicit different types of data from participants. Sharing interview transcripts with other researchers, for instance, downplays the contextual nature of an interview.

Drawing on Bauer and Gaskell, we propose several measures to enhance rigour in qualitative research: transparency, grounding interpretations and aiming for theoretical transferability and significance 62 .

Researchers should be transparent when describing their methodological choices. Transparency means documenting who was interviewed, where and when (without requiring de-anonymization, for example, by documenting their characteristics), as well as the questions they were asked. It means carefully considering who was left out of the interviews and what that could mean for the researcher’s findings. It also means carefully considering who the researcher is and how their identity shaped the research process (integrating and articulating reflexivity into whatever is written up).

Second, researchers should ground their interpretations in the data. Grounding means presenting the evidence upon which the interpretation relies. Quotes and extracts should be extensive enough to allow the reader to evaluate whether the researcher’s interpretations are grounded in the data. At each step, researchers should carefully compare their own explanations and interpretations with alternative explanations. Doing so systematically and frequently allows researchers to become more confident in their claims. Here, researchers should justify the link between data and analysis by using quotes to justify and demonstrate the analytical point, while making sure the analytical point offers an interpretation of quotes (Box  4 ).

An important step in considering alternative explanations is to seek out disconfirming evidence 4 , 63 . This involves looking for instances where participants deviate from what the majority are saying and thus bring into question the theory (or explanation) that the researcher is developing. Careful analysis of such examples can often demonstrate the salience and meaning of what appears to be the norm (see Table  2 for examples) 54 . Considering alternative explanations and paying attention to disconfirming evidence allows the researcher to refine their own theories in respect of the data.

Finally, researchers should aim for theoretical transferability and significance in their discussions of findings. One way to think about this is to imagine someone who is not interested in the empirical study. Articulating theoretical transferability and significance usually takes the form of broadening out from the specific findings to consider explicitly how the research has refined or altered prior theoretical approaches. This process also means considering under what other conditions, aside from those of the study, the researcher thinks their theoretical revision would be supported by and why. Importantly, it also includes thinking about the limitations of one’s own approach and where the theoretical implications of the study might not hold.

Box 4 An example of grounding interpretations in data (from Rao 34 )

In an article explaining how unemployed men frame their job loss as a pervasive experience, Rao writes the following: “Unemployed men in this study understood unemployment to be an expected aspect of paid work in the contemporary United States. Robert, a white unemployed communications professional, compared the economic landscape after the Great Recession with the tragic events of September 11, 2001:

Part of your post-9/11 world was knowing people that died as a result of terrorism. The same thing is true with the [Great] Recession, right? … After the Recession you know somebody who was unemployed … People that really should be working.

The pervasiveness of unemployment rendered it normal, as Robert indicates.”

Here, the link between the quote presented and the analytical point Rao is making is clear: the analytical point is grounded in a quote and an interpretation of the quote is offered 34 .

Limitations and optimizations

When deciding which research method to use, the key question is whether the method provides a good fit for the research questions posed. In other words, researchers should consider whether interviews will allow them to successfully access the social phenomena necessary to answer their question(s) and whether the interviews will do so more effectively than other methods. Table  3 summarizes the major strengths and limitations of interviews. However, the accompanying text below is organized around some key issues, where relative strengths and weaknesses are presented alongside each other, the aim being that readers should think about how these can be balanced and optimized in relation to their own research.

Breadth versus depth of insight

Achieving an overall breadth of insight, in a statistically representative sense, is not something that is possible or indeed desirable when conducting in-depth interviews. Instead, the strength of conducting interviews lies in their ability to generate various sorts of depth of insight. The experiences or views of participants that can be accessed by conducting interviews help us to understand participants’ subjective realities. The challenge, therefore, is for researchers to be clear about why depth of insight is the focus and what we should aim to glean from these types of insight.

Naturalistic or artificial interviews

Interviews make use of a form of interaction with which people are familiar 64 . By replicating a naturalistic form of interaction as a tool to gather social science data, researchers can capitalize on people’s familiarity and expectations of what happens in a conversation. This familiarity can also be a challenge, as people come to the interview with preconceived ideas about what this conversation might be for or about. People may draw on experiences of other similar conversations when taking part in a research interview (for example, job interviews, therapy sessions, confessional conversations, chats with friends). Researchers should be aware of such potential overlaps and think through their implications both in how the aims and purposes of the research interview are communicated to participants and in how interview data are interpreted.

Further, some argue that a limitation of interviews is that they are an artificial form of data collection. By taking people out of their daily lives and asking them to stand back and pass comment, we are creating a distance that makes it difficult to use such data to say something meaningful about people’s actions, experiences and views. Other approaches, such as ethnography, might be more suitable for tapping into what people actually do, as opposed to what they say they do 65 .

Dynamism and replicability

Interviews following a semi-structured format offer flexibility both to the researcher and the participant. As the conversation develops, the interlocutors can explore the topics raised in much more detail, if desired, or pass over ones that are not relevant. This flexibility allows for the unexpected and the unforeseen to be incorporated into the scope of the research.

However, this flexibility has a related challenge of replicability. Interviews cannot be reproduced because they are contingent upon the interaction between the researcher and the participant in that given moment of interaction. In some research paradigms, replicability can be a way of interrogating the robustness of research claims, by seeing whether they hold when they are repeated. This is not a useful framework to bring to in-depth interviews and instead quality criteria (such as transparency) tend to be employed as criteria of rigour.

Accessing the private and personal

Interviews have been recognized for their strength in accessing private, personal issues, which participants may feel more comfortable talking about in a one-to-one conversation. Furthermore, interviews are likely to take a more personable form with their extended questions and answers, perhaps making a participant feel more at ease when discussing sensitive topics in such a context. There is a similar, but separate, argument made about accessing what are sometimes referred to as vulnerable groups, who may be difficult to make contact with using other research methods.

There is an associated challenge of anonymity. There can be types of in-depth interview that make it particularly challenging to protect the identities of participants, such as interviewing within a small community, or multiple members of the same household. The challenge to ensure anonymity in such contexts is even more important and difficult when the topic of research is of a sensitive nature or participants are vulnerable.

Increasingly, researchers are collaborating in large-scale interview-based studies and integrating interviews into broader mixed-methods designs. At the same time, interviews can be seen as an old-fashioned (and perhaps outdated) mode of data collection. We review these debates and discussions and point to innovations in interview-based studies. These include the shift from face-to-face interviews to the use of online platforms, as well as integrating and adapting interviews towards more inclusive methodologies.

Collaborating and mixing

Qualitative researchers have long worked alone 66 . Increasingly, however, researchers are collaborating with others for reasons such as efficiency, institutional incentives (for example, funding for collaborative research) and a desire to pool expertise (for example, studying similar phenomena in different contexts 67 or via different methods). Collaboration can occur across disciplines and methods, cases and contexts and between industry/business, practitioners and researchers. In many settings and contexts, collaboration has become an imperative 68 .

Cheek notes how collaboration provides both advantages and disadvantages 68 . For example, collaboration can be advantageous, saving time and building on the divergent knowledge, skills and resources of different researchers. Scholars with different theoretical or case-based knowledge (or contacts) can work together to build research that is comparative and/or more than the sum of its parts. But such endeavours also carry with them practical and political challenges in terms of how resources might actually be pooled, shared or accounted for. When undertaking such projects, as Morse notes, it is worth thinking about the nature of the collaboration and being explicit about such a choice, its advantages and its disadvantages 66 .

A further tension, but also a motivation for collaboration, stems from integrating interviews as a method in a mixed-methods project, whether with other qualitative researchers (to combine with, for example, focus groups, document analysis or ethnography) or with quantitative researchers (to combine with, for example, surveys, social media analysis or big data analysis). Cheek and Morse both note the pitfalls of collaboration with quantitative researchers: that quality of research may be sacrificed, qualitative interpretations watered down or not taken seriously, or tensions experienced over the pace and different assumptions that come with different methods and approaches of research 66 , 68 .

At the same time, there can be real benefits of such mixed-methods collaboration, such as reaching different and more diverse audiences or testing assumptions and theories between research components in the same project (for example, testing insights from prior quantitative research via interviews, or vice versa), as long as the skillsets of collaborators are seen as equally beneficial to the project. Cheek provides a set of questions that, as a starting point, can be useful for guiding collaboration, whether mixed methods or otherwise. First, Cheek advises asking all collaborators about their assumptions and understandings concerning collaboration. Second, Cheek recommends discussing what each perspective highlights and focuses on (and conversely ignores or sidelines) 68 .

A different way to engage with the idea of collaboration and mixed methods research is by fostering greater collaboration between researchers in the Global South and Global North, thus reversing trends of researchers from the Global North extracting knowledge from the Global South 69 . Such forms of collaboration also align with interview innovations, discussed below, that seek to transform traditional interview approaches into more participatory and inclusive (as part of participatory methodologies).

Digital innovations and challenges

The ongoing COVID-19 pandemic has centred the question of technology within interview-based fieldwork. Although conducting synchronous oral interviews online — for example, via Zoom, Skype or other such platforms — has been a method used by a small constituency of researchers for many years, it became (and remains) a necessity for many researchers wanting to continue or start interview-based projects while COVID-19 prevents face-to-face data collection.

In the past, online interviews were often framed as an inferior form of data collection for not providing the kinds of (often necessary) insights and forms of immersion face-to-face interviews allow 70 , 71 . Online interviews do tend to be more decontextualized than interviews conducted face-to-face 72 . For example, it is harder to recognize, engage with and respond to non-verbal cues 71 . At the same time, they broaden participation to those who might not have been able to access or travel to sites where interviews would have been conducted otherwise, for example people with disabilities. Online interviews also offer more flexibility in terms of scheduling and time requirements. For example, they provide more flexibility around precarious employment or caring responsibilities without having to travel and be away from home. In addition, online interviews might also reduce discomfort between researchers and participants, compared with face-to-face interviews, enabling more discussion of sensitive material 71 . They can also provide participants with more control, enabling them to turn on and off the microphone and video as they choose, for example, to provide more time to reflect and disconnect if they so wish 72 .

That said, online interviews can also introduce new biases based on access to technology 72 . For example, in the Global South, there are often urban/rural and gender gaps between who has access to mobile phones and who does not, meaning that some population groups might be overlooked unless researchers sample mindfully 71 . There are also important ethical considerations when deciding between online and face-to-face interviews. Online interviews might seem to imply lower ethical risks than face-to-face interviews (for example, they lower the chances of identification of participants or researchers), but they also offer more barriers to building trust between researchers and participants 72 . Interacting only online with participants might not provide the information needed to assess risk, for example, participants’ access to a private space to speak 71 . Just because online interviews might be more likely to be conducted in private spaces does not mean that private spaces are safe, for example, for victims of domestic violence. Finally, online interviews prompt further questions about decolonizing research and engaging with participants if research is conducted from afar 72 , such as how to include participants meaningfully and challenge dominant assumptions while doing so remotely.

A further digital innovation, modulating how researchers conduct interviews and the kinds of data collected and analysed, stems from the use and integration of (new) technology, such as WhatsApp text or voice notes to conduct synchronous or asynchronous oral or written interviews 73 . Such methods can provide more privacy, comfort and control to participants and make recruitment easier, allowing participants to share what they want when they want to, using technology that already forms a part of their daily lives, especially for young people 74 , 75 . Such technology is also emerging in other qualitative methods, such as focus groups, with similar arguments around greater inclusivity versus traditional offline modes. Here, the digital challenge might be higher for researchers than for participants if they are less used to such technology 75 . And while there might be concerns about the richness, depth and quality of written messages as a form of interview data, Gibson reports that the reams of transcripts that resulted from a study using written messaging were dense with meaning to be analysed 75 .

Like with online and face-to-face interviews, it is important also to consider the ethical questions and challenges of using such technology, from gaining consent to ensuring participant safety and attending to their distress, without cues, like crying, that might be more obvious in a face-to-face setting 75 , 76 . Attention to the platform used for such interviews is also important and researchers should be attuned to the local and national context. For example, in China, many platforms are neither legal nor available 76 . There, more popular platforms — like WeChat — can be highly monitored by the government, posing potential risks to participants depending on the topic of the interview. Ultimately, researchers should consider trade-offs between online and offline interview modalities, being attentive to the social context and power dynamics involved.

The next 5–10 years

Continuing to integrate (ethically) this technology will be among the major persisting developments in interview-based research, whether to offer more flexibility to researchers or participants, or to diversify who can participate and on what terms.

Pushing the idea of inclusion even further is the potential for integrating interview-based studies within participatory methods, which are also innovating via integrating technology. There is no hard and fast line between researchers using in-depth interviews and participatory methods; many who employ participatory methods will use interviews at the beginning, middle or end phases of a research project to capture insights, perspectives and reflections from participants 77 , 78 . Participatory methods emphasize the need to resist existing power and knowledge structures. They broaden who has the right and ability to contribute to academic knowledge by including and incorporating participants not only as subjects of data collection, but as crucial voices in research design and data analysis 77 . Participatory methods also seek to facilitate local change and to produce research materials, whether for academic or non-academic audiences, including films and documentaries, in collaboration with participants.

In responding to the challenges of COVID-19, capturing the fraught situation wrought by the pandemic and the momentum to integrate technology, participatory researchers have sought to continue data collection from afar. For example, Marzi has adapted an existing project to co-produce participatory videos, via participants’ smartphones in Medellin, Colombia, alongside regular check-in conversations/meetings/interviews with participants 79 . Integrating participatory methods into interview studies offers a route by which researchers can respond to the challenge of diversifying knowledge, challenging assumptions and power hierarchies and creating more inclusive and collaborative partnerships between participants and researchers in the Global North and South.

Brinkmann, S. & Kvale, S. Doing Interviews Vol. 2 (Sage, 2018). This book offers a good general introduction to the practice and design of interview-based studies.

Silverman, D. A Very Short, Fairly Interesting And Reasonably Cheap Book About Qualitative Research (Sage, 2017).

Yin, R. K. Case Study Research And Applications: Design And Methods (Sage, 2018).

Small, M. L. How many cases do I need?’ On science and the logic of case selection in field-based research. Ethnography 10 , 5–38 (2009). This article convincingly demonstrates how the logic of qualitative research differs from quantitative research and its goal of representativeness.

Google Scholar  

Gerson, K. & Damaske, S. The Science and Art of Interviewing (Oxford Univ. Press, 2020).

Glaser, B. G. & Strauss, A. L. The Discovery Of Grounded Theory: Strategies For Qualitative Research (Aldine, 1967).

Braun, V. & Clarke, V. To saturate or not to saturate? Questioning data saturation as a useful concept for thematic analysis and sample-size rationales. Qual. Res. Sport Exerc. Health 13 , 201–216 (2021).

Guest, G., Bunce, A. & Johnson, L. How many interviews are enough? An experiment with data saturation and variability. Field Methods 18 , 59–82 (2006).

Vasileiou, K., Barnett, J., Thorpe, S. & Young, T. Characterising and justifying sample size sufficiency in interview-based studies: systematic analysis of qualitative health research over a 15-year period. BMC Med. Res. Methodol. 18 , 148 (2018).

Silverman, D. How was it for you? The Interview Society and the irresistible rise of the (poorly analyzed) interview. Qual. Res. 17 , 144–158 (2017).

Jerolmack, C. & Murphy, A. The ethical dilemmas and social scientific tradeoffs of masking in ethnography. Sociol. Methods Res. 48 , 801–827 (2019).

MathSciNet   Google Scholar  

Reyes, V. Ethnographic toolkit: strategic positionality and researchers’ visible and invisible tools in field research. Ethnography 21 , 220–240 (2020).

Guillemin, M. & Gillam, L. Ethics, reflexivity and “ethically important moments” in research. Qual. Inq. 10 , 261–280 (2004).

Summers, K. For the greater good? Ethical reflections on interviewing the ‘rich’ and ‘poor’ in qualitative research. Int. J. Soc. Res. Methodol. 23 , 593–602 (2020). This article argues that, in qualitative interview research, a clearer distinction needs to be drawn between ethical commitments to individual research participants and the group(s) to which they belong, a distinction that is often elided in existing ethics guidelines.

Yusupova, G. Exploring sensitive topics in an authoritarian context: an insider perspective. Soc. Sci. Q. 100 , 1459–1478 (2019).

Hemming, J. in Surviving Field Research: Working In Violent And Difficult Situations 21–37 (Routledge, 2009).

Murphy, E. & Dingwall, R. Informed consent, anticipatory regulation and ethnographic practice. Soc. Sci. Med. 65 , 2223–2234 (2007).

Kostovicova, D. & Knott, E. Harm, change and unpredictability: the ethics of interviews in conflict research. Qual. Res. 22 , 56–73 (2022). This article highlights how interviews need to be considered as ethically unpredictable moments where engaging with change among participants can itself be ethical.

Andersson, R. Illegality, Inc.: Clandestine Migration And The Business Of Bordering Europe (Univ. California Press, 2014).

Ellis, R. What do we mean by a “hard-to-reach” population? Legitimacy versus precarity as barriers to access. Sociol. Methods Res. https://doi.org/10.1177/0049124121995536 (2021).

Article   Google Scholar  

Braun, V. & Clarke, V. Thematic Analysis: A Practical Guide (Sage, 2022).

Alejandro, A. & Knott, E. How to pay attention to the words we use: the reflexive review as a method for linguistic reflexivity. Int. Stud. Rev. https://doi.org/10.1093/isr/viac025 (2022).

Alejandro, A., Laurence, M. & Maertens, L. in International Organisations and Research Methods: An Introduction (eds Badache, F., Kimber, L. R. & Maertens, L.) (Michigan Univ. Press, in the press).

Teeger, C. “Both sides of the story” history education in post-apartheid South Africa. Am. Sociol. Rev. 80 , 1175–1200 (2015).

Crotty, M. The Foundations Of Social Research: Meaning And Perspective In The Research Process (Routledge, 2020).

Potter, J. & Hepburn, A. Qualitative interviews in psychology: problems and possibilities. Qual. Res. Psychol. 2 , 281–307 (2005).

Taylor, S. What is Discourse Analysis? (Bloomsbury Publishing, 2013).

Riessman, C. K. Narrative Analysis (Sage, 1993).

Corbin, J. M. & Strauss, A. Grounded theory research: Procedures, canons and evaluative criteria. Qual. Sociol. 13 , 3–21 (1990).

Timmermans, S. & Tavory, I. Theory construction in qualitative research: from grounded theory to abductive analysis. Sociol. Theory 30 , 167–186 (2012).

Fereday, J. & Muir-Cochrane, E. Demonstrating rigor using thematic analysis: a hybrid approach of inductive and deductive coding and theme development. Int. J. Qual. Meth. 5 , 80–92 (2006).

Potter, J. & Hepburn, A. Eight challenges for interview researchers. Handb. Interview Res. 2 , 541–570 (2012).

Tobias Neely, M. Fit to be king: how patrimonialism on Wall Street leads to inequality. Socioecon. Rev. 16 , 365–385 (2018).

Rao, A. H. Gendered interpretations of job loss and subsequent professional pathways. Gend. Soc. 35 , 884–909 (2021). This article used interview data from unemployed men and women to illuminate how job loss becomes a pivotal moment shaping men’s and women’s orientation to paid work, especially in terms of curtailing women’s participation in paid work.

Hart, C. G. Trajectory guarding: managing unwanted, ambiguously sexual interactions at work. Am. Sociol. Rev. 86 , 256–278 (2021).

Goode, J. P. & Stroup, D. R. Everyday nationalism: constructivism for the masses. Soc. Sci. Q. 96 , 717–739 (2015).

Antonsich, M. The ‘everyday’ of banal nationalism — ordinary people’s views on Italy and Italian. Polit. Geogr. 54 , 32–42 (2016).

Fox, J. E. & Miller-Idriss, C. Everyday nationhood. Ethnicities 8 , 536–563 (2008).

Yusupova, G. Cultural nationalism and everyday resistance in an illiberal nationalising state: ethnic minority nationalism in Russia. Nations National. 24 , 624–647 (2018).

Kiely, R., Bechhofer, F. & McCrone, D. Birth, blood and belonging: identity claims in post-devolution Scotland. Sociol. Rev. 53 , 150–171 (2005).

Brubaker, R. & Cooper, F. Beyond ‘identity’. Theory Soc. 29 , 1–47 (2000).

Brubaker, R. Ethnicity Without Groups (Harvard Univ. Press, 2004).

Knott, E. Kin Majorities: Identity And Citizenship In Crimea And Moldova From The Bottom-Up (McGill Univ. Press, 2022).

Bucher, B. & Jasper, U. Revisiting ‘identity’ in international relations: from identity as substance to identifications in action. Eur. J. Int. Relat. 23 , 391–415 (2016).

Carter, P. L. Stubborn Roots: Race, Culture And Inequality In US And South African Schools (Oxford Univ. Press, 2012).

Bourdieu, P. in Cultural Theory: An Anthology Vol. 1, 81–93 (eds Szeman, I. & Kaposy, T.) (Wiley-Blackwell, 2011).

Calarco, J. M. Negotiating Opportunities: How The Middle Class Secures Advantages In School (Oxford Univ. Press, 2018).

Carter, P. L. Keepin’ It Real: School Success Beyond Black And White (Oxford Univ. Press, 2005).

Carter, P. L. ‘Black’ cultural capital, status positioning and schooling conflicts for low-income African American youth. Soc. Probl. 50 , 136–155 (2003).

Warikoo, N. K. The Diversity Bargain Balancing Acts: Youth Culture in the Global City (Univ. California Press, 2011).

Morris, E. W. “Tuck in that shirt!” Race, class, gender and discipline in an urban school. Sociol. Perspect. 48 , 25–48 (2005).

Lareau, A. Social class differences in family–school relationships: the importance of cultural capital. Sociol. Educ. 60 , 73–85 (1987).

Warikoo, N. Addressing emotional health while protecting status: Asian American and white parents in suburban America. Am. J. Sociol. 126 , 545–576 (2020).

Teeger, C. Ruptures in the rainbow nation: how desegregated South African schools deal with interpersonal and structural racism. Sociol. Educ. 88 , 226–243 (2015). This article leverages ‘ deviant ’ cases in an interview study with South African high schoolers to understand why the majority of participants were reluctant to code racially charged incidents at school as racist.

Ispa-Landa, S. & Conwell, J. “Once you go to a white school, you kind of adapt” black adolescents and the racial classification of schools. Sociol. Educ. 88 , 1–19 (2015).

Dwyer, P. J. Punitive and ineffective: benefit sanctions within social security. J. Soc. Secur. Law 25 , 142–157 (2018).

Summers, K. & Young, D. Universal simplicity? The alleged simplicity of Universal Credit from administrative and claimant perspectives. J. Poverty Soc. Justice 28 , 169–186 (2020).

Summers, K. et al. Claimants’ Experiences Of The Social Security System During The First Wave Of COVID-19 . https://www.distantwelfare.co.uk/winter-report (2021).

Desmond, M. Evicted: Poverty And Profit In The American City (Crown Books, 2016).

Reyes, V. Three models of transparency in ethnographic research: naming places, naming people and sharing data. Ethnography 19 , 204–226 (2018).

Robson, C. & McCartan, K. Real World Research (Wiley, 2016).

Bauer, M. W. & Gaskell, G. Qualitative Researching With Text, Image And Sound: A Practical Handbook (SAGE, 2000).

Lareau, A. Listening To People: A Practical Guide To Interviewing, Participant Observation, Data Analysis And Writing It All Up (Univ. Chicago Press, 2021).

Lincoln, Y. S. & Guba, E. G. Naturalistic Inquiry (Sage, 1985).

Jerolmack, C. & Khan, S. Talk is cheap. Sociol. Methods Res. 43 , 178–209 (2014).

Morse, J. M. Styles of collaboration in qualitative inquiry. Qual. Health Res. 18 , 3–4 (2008).

ADS   Google Scholar  

Lamont, M. et al. Getting Respect: Responding To Stigma And Discrimination In The United States, Brazil And Israel (Princeton Univ. Press, 2016).

Cheek, J. Researching collaboratively: implications for qualitative research and researchers. Qual. Health Res. 18 , 1599–1603 (2008).

Botha, L. Mixing methods as a process towards indigenous methodologies. Int. J. Soc. Res. Methodol. 14 , 313–325 (2011).

Howlett, M. Looking at the ‘field’ through a zoom lens: methodological reflections on conducting online research during a global pandemic. Qual. Res. https://doi.org/10.1177/1468794120985691 (2021).

Reñosa, M. D. C. et al. Selfie consents, remote rapport and Zoom debriefings: collecting qualitative data amid a pandemic in four resource-constrained settings. BMJ Glob. Health 6 , e004193 (2021).

Mwambari, D., Purdeková, A. & Bisoka, A. N. Covid-19 and research in conflict-affected contexts: distanced methods and the digitalisation of suffering. Qual. Res. https://doi.org/10.1177/1468794121999014 (2021).

Colom, A. Using WhatsApp for focus group discussions: ecological validity, inclusion and deliberation. Qual. Res. https://doi.org/10.1177/1468794120986074 (2021).

Kaufmann, K. & Peil, C. The mobile instant messaging interview (MIMI): using WhatsApp to enhance self-reporting and explore media usage in situ. Mob. Media Commun. 8 , 229–246 (2020).

Gibson, K. Bridging the digital divide: reflections on using WhatsApp instant messenger interviews in youth research. Qual. Res. Psychol. 19 , 611–631 (2020).

Lawrence, L. Conducting cross-cultural qualitative interviews with mainland Chinese participants during COVID: lessons from the field. Qual. Res. https://doi.org/10.1177/1468794120974157 (2020).

Ponzoni, E. Windows of understanding: broadening access to knowledge production through participatory action research. Qual. Res. 16 , 557–574 (2016).

Kong, T. S. Gay and grey: participatory action research in Hong Kong. Qual. Res. 18 , 257–272 (2018).

Marzi, S. Participatory video from a distance: co-producing knowledge during the COVID-19 pandemic using smartphones. Qual. Res. https://doi.org/10.1177/14687941211038171 (2021).

Kvale, S. & Brinkmann, S. InterViews: Learning The Craft Of Qualitative Research Interviewing (Sage, 2008).

Rao, A. H. The ideal job-seeker norm: unemployment and marital privileges in the professional middle-class. J. Marriage Fam. 83 , 1038–1057 (2021).

Rivera, L. A. Ivies, extracurriculars and exclusion: elite employers’ use of educational credentials. Res. Soc. Stratif. Mobil. 29 , 71–90 (2011).

Download references

Acknowledgements

The authors are grateful to the MY421 team and students for prompting how best to frame and communicate issues pertinent to in-depth interview studies.

Author information

Authors and affiliations.

Department of Methodology, London School of Economics, London, UK

Eleanor Knott, Aliya Hamid Rao, Kate Summers & Chana Teeger

You can also search for this author in PubMed   Google Scholar

Contributions

The authors contributed equally to all aspects of the article.

Corresponding author

Correspondence to Eleanor Knott .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Peer review

Peer review information.

Nature Reviews Methods Primers thanks Jonathan Potter and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

A pre-written interview outline for a semi-structured interview that provides both a topic structure and the ability to adapt flexibly to the content and context of the interview and the interaction between the interviewer and participant. Others may refer to the topic guide as an interview protocol.

Here we refer to the participants that take part in the study as the sample. Other researchers may refer to the participants as a participant group or dataset.

This involves dividing a population into smaller groups based on particular characteristics, for example, age or gender, and then sampling randomly within each group.

A sampling method where the guiding logic when deciding who to recruit is to achieve the most relevant participants for the research topic, in terms of being rich in information or insights.

Researchers ask participants to introduce the researcher to others who meet the study’s inclusion criteria.

Similar to stratified sampling, but participants are not necessarily randomly selected. Instead, the researcher determines how many people from each category of participants should be recruited. Recruitment can happen via snowball or purposive sampling.

A method for developing, analysing and interpreting patterns across data by coding in order to develop themes.

An approach that interrogates the explicit, implicit and taken-for-granted dimensions of language as well as the contexts in which it is articulated to unpack its purposes and effects.

A form of transcription that simplifies what has been said by removing certain verbal and non-verbal details that add no further meaning, such as ‘ums and ahs’ and false starts.

The analytic framework, theoretical approach and often hypotheses, are developed prior to examining the data and then applied to the dataset.

The analytic framework and theoretical approach is developed from analysing the data.

An approach that combines deductive and inductive components to work recursively by going back and forth between data and existing theoretical frameworks (also described as an iterative approach). This approach is increasingly recognized not only as a more realistic but also more desirable third alternative to the more traditional inductive versus deductive binary choice.

A theoretical apparatus that emphasizes the role of cultural processes and capital in (intergenerational) social reproduction.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Cite this article.

Knott, E., Rao, A.H., Summers, K. et al. Interviews in the social sciences. Nat Rev Methods Primers 2 , 73 (2022). https://doi.org/10.1038/s43586-022-00150-6

Download citation

Accepted : 14 July 2022

Published : 15 September 2022

DOI : https://doi.org/10.1038/s43586-022-00150-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Development of a digital intervention for psychedelic preparation (dipp).

• Rosalind G. McAlpine
• Matthew D. Sacchet
• Sunjeev K. Kamboj

Scientific Reports (2024)

‘Life Minus Illness = Recovery’: A Phenomenological Study About Experiences and Meanings of Recovery Among Individuals with Serious Mental Illness from Southern India

• Srishti Hegde
• Shalini Quadros
• Vinita A. Acharya

Community Mental Health Journal (2024)

Is e-business breaking down barriers for Bangladesh’s young female entrepreneurs during the COVID-19 pandemic? A qualitative study

• Md. Fouad Hossain Sarker
• Sayed Farrukh Ahmed
• Md. Salman Sohel

SN Social Sciences (2024)

Between the dog and the wolf: an interpretative phenomenological analysis of bicultural, sexual minority people’s lived experiences

• Emelie Louise Miller
• Ingrid Zakrisson

Discover Psychology (2024)

Acknowledging that Men are Moral and Harmed by Gender Stereotypes Increases Men’s Willingness to Engage in Collective Action on Behalf of Women

• Alexandra Vázquez
• Lucía López-Rodríguez
• Marco Brambilla

Sex Roles (2024)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List

A comparison of results of empirical studies of supplementary search techniques and recommendations in review methodology handbooks: a methodological review

Chris cooper.

1 PenTAG, University of Exeter Medical School, Exeter, England

Andrew Booth

2 HEDS, School of Health and Related Research (ScHARR), University of Sheffield, Sheffield, England

Nicky Britten

3 Institute of Health Research, University of Exeter Medical School, Exeter, England

Ruth Garside

4 European Centre for Environment and Human Health, University of Exeter Medical School, Truro, England

Associated Data

Not applicable

The purpose and contribution of supplementary search methods in systematic reviews is increasingly acknowledged. Numerous studies have demonstrated their potential in identifying studies or study data that would have been missed by bibliographic database searching alone.

What is less certain is how supplementary search methods actually work, how they are applied, and the consequent advantages, disadvantages and resource implications of each search method.

The aim of this study is to compare current practice in using supplementary search methods with methodological guidance.

Four methodological handbooks in informing systematic review practice in the UK were read and audited to establish current methodological guidance.

Studies evaluating the use of supplementary search methods were identified by searching five bibliographic databases. Studies were included if they (1) reported practical application of a supplementary search method (descriptive) or (2) examined the utility of a supplementary search method (analytical) or (3) identified/explored factors that impact on the utility of a supplementary method, when applied in practice.

Thirty-five studies were included in this review in addition to the four methodological handbooks. Studies were published between 1989 and 2016, and dates of publication of the handbooks ranged from 1994 to 2014.

Five supplementary search methods were reviewed: contacting study authors, citation chasing, handsearching, searching trial registers and web searching.

Conclusions

There is reasonable consistency between recommended best practice (handbooks) and current practice (methodological studies) as it relates to the application of supplementary search methods.

The methodological studies provide useful information on the effectiveness of the supplementary search methods, often seeking to evaluate aspects of the method to improve effectiveness or efficiency. In this way, the studies advance the understanding of the supplementary search methods. Further research is required, however, so that a rational choice can be made about which supplementary search strategies should be used, and when.

The purpose and contribution of supplementary search methods in systematic reviews are increasingly acknowledged. Numerous studies have demonstrated their potential in identifying studies or study data that would have been missed by bibliographic database searching alone [ 1 – 8 ].

It is commonly believed that the inclusion of supplementary search methods adds value to the process of comprehensive study identification in systematic reviews. The methodological handbooks for systematic review methodology, such as The Cochrane or CRD Handbooks, provide practical (although limited) instruction on how to undertake each supplementary search method, and empirical studies have evaluated the effectiveness and efficiencies of these search methods. What is perhaps less certain is how supplementary search methods actually work, and what the advantages, disadvantages and resource implications of each search method are.

The aim of this study is to compare empirical studies of supplementary search techniques to the recommendations in methodological handbooks.

By re-considering the best practice guidance of methodological handbooks for systematic review, and reviewing how this guidance has been interpreted and evaluated within current practice by authors, this study seeks to identify claimed advantages, claimed disadvantages and resource requirements of using supplementary search methods.

The research question for this study

The research question for this study is how do empirical studies of supplementary search techniques compare to the recommendations in review methodology handbooks?

This study aims to produce a structured methodological overview of methodological handbooks on the conduct of supplementary searches in systematic reviews. In addition, we reviewed studies that report on the utility and practice of supplementary searches. In order to identify this literature, a systematic approach to study identification, study selection and data extraction was used, which is set out below. These two types of literature—handbooks and practical explorations of applying supplementary search strategies—were then compared. The advantages, disadvantages and resource requirements of each method were evaluated.

Study identification

We selected the following methodological handbooks as the most influential handbooks in informing systematic review practice in the UK. The current editions of each handbook were read and audited to establish current methodological guidance:

• The Cochrane Handbook for Systematic Reviews of Interventions (version 5.10, March 2011) [ 9 ];
• Systematic Reviews: CRD’s guidance for undertaking review in health care (2009) [ 10 ];
• The Campbell Information Retrieval Methods Group guide to information retrieval (October 2009) [ 11 ]; and
• The NICE manual to developing NICE guidelines (October 2014) [ 12 ].

The following five search methods, supplementary to database searches, were identified from these handbooks:

• Contacting study authors or experts

Citation chasing

Handsearching.

• Trial register searching
• Web searching.

In order to compare the existing handbook guidance to current practice, we identified studies that describe and/or evaluate how these methods are applied in practice. Studies were identified by searching five bibliographic databases: MEDLINE, EMBASE, LISTA, ASSIA and Web of Science in July 2016. Forward citation chasing was applied to studies meeting inclusion at full text, and the bibliographies were appraised. Tables of included studies were examined if aggregated within systematic reviews. The search syntax for bibliographic database searching is included as a supplementary file.

Study selection

Studies were downloaded into Endnote X6 where manual de-duplication was performed. Studies were single screened by CC using the inclusion criteria below:

Inclusion criteria

For inclusion in this review, a study was required to:

• (i) Report practical application of a supplementary search method (descriptive)
• (ii) Examine the utility of a supplementary search method (analytical)
• (iii) Identify/explore factors that impact on the utility of a supplementary method when applied in practice

Exclusion criteria

The following studies were excluded:

• i) Studies reporting the use of supplementary search methods but not discussing the practical application of the method (such as listing their use to identify studies in a systematic review, i.e. ‘we handsearched the following journals’)
• ii) Studies reported as abstracts or on-going studies
• iii) Systematic reviews or reviews, in which case tables of included studies were examined to identify eligible primary studies

Data extraction

The following data were extracted: citation details, study design, claimed advantages, claimed disadvantages and resource requirements.

Thirty-five studies were included in this review in addition to the four methodological handbooks. Studies were published between 1989 and 2016, and handbooks were published between 1994 and 2014. Table  1 summarises which studies cited which handbooks as their source of methodological reference. The handbooks audited for this study cited only three studies: Eysenbach et al. (2001) was cited in The Cochrane Handbook, Hetherington et al. (1989) was cited in The Cochrane Handbook and The Campbell Handbook and Papaioannou et al. (2010) was cited in The Campbell Handbook (Table  1 ).

Studies citing handbooks: handbooks citing studies

Studies in italics are ones cited by the handbooks identified using the symbol ‘**’ for informing their guidance

NR not reported

The results were categorised by the supplementary search methods and reported in five domains: (1) what the method is used for, (2) what the evidence says, (3) claimed advantages, (4) claimed disadvantages and (5) resource requirements. A summary of these results is presented in Table  2 .

Overview of results

Contacting study authors

The handbooks focus on identifying contact details and considering how to request studies or study data [ 9 , 10 , 13 ]. The studies evaluate the effectiveness of methods to make contact and elicit a response. Six empirical studies were included [ 6 , 14 – 18 ].

What it is used for

It is used for identifying unpublished or on-going studies [ 10 ]; identifying missing, incomplete, discordant or unreported study data, or completed but unpublished studies [ 9 , 13 , 14 , 16 – 18 ]; and asking study authors (or topic experts) to review a list of studies included at full text in a review, to see whether any studies had been inadvertently overlooked [ 9 , 10 ].

What the evidence says

Two handbooks and one study provided detail on identifying contact details [ 6 , 9 , 10 ]. The Cochrane Handbook suggests that review authors should contact the original investigators, identifying contact details from study reports, recent publications, staff listings or a search of the internet [ 13 ]. Colleagues, relevant research organisations and specialist libraries can also be a valuable source of author information and contact details [ 9 , 10 ]. A study by McManus et al. used a questionnaire, primarily to request study data or references, but also to ask recipients to recommend the names of other authors to contact [ 6 ]. A study by Hetherington et al. contacted authors and experts by letter in an attempt to identify unpublished trials [ 17 ].

Two studies reported using a multi-stage protocol to contact authors and request data: Selph et al. devised and followed a protocol that used both e-mail and telephone contact with the corresponding authors at defined stages over a period of 15 days [ 16 ]. Gibson et al. devised a similar protocol, although focused on e-mail contact, targeting first the corresponding authors and finally the last author and statisticians by e-mail and then telephone (statisticians were contacted due to the specific focus of the case study) [ 14 ]. Selph et al. contacted 45 authors and 28 (62%) provided study data [ 16 ], and Gibson et al. contacted 146 authors and 46 (31.5%) provided study data [ 14 ].

Two studies claimed that e-mail was considered an effective method of contact [ 14 , 15 ]. O’Leary reported a response rate of 73% using e-mail contact, finding that more responses were obtained from an institutional address compared to a hotmail address (86 vs 57%, p  = 0.02) [ 15 ]. Conversely, Reveiz et al. achieved a 7.5% response rate from contacting 525 study authors to identify RCTs but identified 10 unpublished RCTs and links to 21 unregistered and on-going RCTs [ 18 ]. Gibson et al. found that e-mail was most likely to receive a reply when compared to letter (hazard ratio [HR] = 2.5; 95% confidence interval [CI] = 1.3–4.0) but that a combined approach of letter and e-mail, whilst generating a higher response rate, was not statistically different from e-mail alone (73 vs 47%, p  = 0.36. One hundred forty-six authors were contacted overall and 46 responded) [ 14 ].

Hetherington et al. sent letters to 42,000 obstetricians and paediatricians in 18 countries in an attempt to identify unpublished controlled trials in perinatal medicine [ 17 ]. Responses were received from 481 individuals indicating they would provide details concerning unpublished studies, and 453 questionnaires were completed and returned which identified 481 unpublished trials [ 17 ].

Chapter Seven of The Cochrane Handbook offers guidance on how to set out requests for studies or study data when contacting study authors [ 13 ]. The guidance suggests considering if the request is open-ended, or seeking specific information, and whether (therefore) to include a (uncompleted or partially completed) data collection form or request specific data (i.e. individual patient data) [ 13 ]. McManus et al. evaluated the use of a questionnaire to identify studies, study data and the names of relevant authors to contact for a systematic review [ 6 ]. The questionnaire resulted in the identification of 1057 references unique to the review, but no unpublished data were offered [ 6 ].

Two handbooks recommend submitting a list of included studies to authors [ 9 ] or topic experts [ 10 ] to identify any potentially missing studies. The Cochrane Handbook suggests including the review’s inclusion criteria as a guide to authors [ 9 ].

Claimed advantages

Five studies claimed that identifying additional published or unpublished studies, study data or references is possible by contacting study authors [ 6 , 14 , 16 – 18 ]. McManus et al. identified 23 references (out of 75 included in the review overall) by contacting study authors [ 6 ]; Reveiz et al. identified 10 unpublished RCTs and 21 unregistered or on-going RCTs [ 18 ]; two studies stated that they identified additional study data but did not separate their findings from contacting study authors from other methods of study identification [ 14 , 16 ]; and Hetherington et al. identified 481 unpublished trials by contacting 42,000 obstetricians and paediatricians in 17 countries [ 17 ].

O’Leary found that more detailed study information was provided as a result of contacting study authors [ 14 ].

Claimed disadvantages

The CRD handbook claims that contacting authors/experts offers no guarantee of obtaining relevant information [ 10 ]. Selph et al. found that, whilst identifying additional studies or study data is possible, contacting study authors is challenging and, despite extensive effort, missing data remains likely [ 16 ].

Hetherington et al. claimed that methodologically sound trials were not reported through author contact, even by the investigators responsible for them. This was attributed, anecdotally, to the possibility that the trials yielded results that the investigators found disappointing [ 17 ].

Reveiz et al. reported low response rates. Of 525 study authors contacted, only 40 (7.5%) replied [ 18 ].

Two studies and one handbook claimed that contacting authors/experts is time consuming for researchers [ 10 , 14 , 16 ]. Selph et al. noted that this method is time consuming for the study authors too, who must identify the data requested [ 16 ].

Gibson et al. claimed that contacting authors/experts may be less successful for older studies, given the increased possibility that authors’ contact details are out of date [ 14 ]. Gibson et al. reported a 78% (CI = 0.107–0.479) reduction in the odds of response if the article was 10 years old or older [ 14 ].

Resource requirements

Gibson et al. claimed that additional resources were required to undertake author contact [ 14 ]. No specific details of the costs or time implications were recorded.

Gibson et al. recorded the duration between the information request and response [ 14 ]. This averaged 14 ± 22 days (median = 6 days) and was shortest for e-mail (3 ± 3 days; median = 1 day) compared to e-mail plus letter (13 ± 12 days; median = 9 days) and letter only (27 ± 30 days; median = 10 days) [ 14 ].

Selph et al. reported that all authors who provided data did so by the third attempt, suggesting that repeated attempts to elicit studies or study data may be ineffective [ 16 ].

The handbooks provide a brief overview of the method and list some of the tools commonly used [ 9 , 10 ]. The studies typically evaluate the effectiveness of the tools used to undertake the search methods. Nine studies assessing the use of citation chasing were included [ 1 – 3 , 19 – 24 ].

It is used for identifying further studies, and clusters or networks of studies, that cite or are cited by a primary study [ 10 ].

Two studies provided detail on the application of the search method [ 1 , 3 ]. The studies noted that backward citation searching is undertaken by reviewing bibliographies of relevant or included studies and forward citation chasing is undertaken by checking if a study, already known to be relevant, has since been cited by another study [ 1 , 3 ].

Three tools for electronic citation searching dominate the studies: Web of Science, Scopus and Google Scholar. The first two are subscription databases, and Google Scholar is presently free [ 19 ].

Four studies claimed that an advantage of citation chasing is that it is not limited by keywords or indexing as is bibliographic database searching [ 2 , 3 , 20 , 21 ]. Accordingly, four studies claimed the following advantages: Robinson et al. claimed that a small initial number of studies can create a network [ 21 ]; Hinde et al. claimed that citation searching can help inform researchers of parallel topics that may be missed by the focus of bibliographic database searches [ 2 ]; Janssens and Gwinn claimed that citation searching may be valuable in topic areas where there is no consistent terminology, so searches focus on links between studies rather than keywords [ 20 ]; and Papaioannou et al. reported that citation searching facilitated ‘serendipitous study identification’ due to the unstructured nature of citations [ 3 ].

One study appraised the quality of the studies identified through citation searching (and by other search methods) [ 3 ]. Papaioannou et al. reported that citation searching identified high-quality studies in their case study, although they do not define which quality appraisal tool was used to appraise study quality, so it is not clear if this observation is empirically derived [ 3 ].

Three studies stated that citation searching is reliant on the currency, accuracy and completeness of the underlying citation network [ 1 , 21 , 22 ]. Levay et al. identified ‘linking lag’, namely the delay between a study being cited and the citation being recorded in a citation database, which impacts on the currency of results [ 1 ]; Janssens and Gwinn stated that the accuracy and efficiency of citation searching depends on study authors citing studies, which means that selective citation of studies could cause relevant studies to be missed in citation searching [ 20 ]; Robinson et al. reported limited returns from citation searching where ‘broken citation links’ created ‘island’ studies which makes for incomplete citation networks and study identification [ 21 ].

Two studies questioned the efficiency of citation searching [ 2 , 22 ]. Wright et al. screened 4161 studies to identify one study (yield rate of 0.0002) [ 22 ], and Hinde et al. screened 4529 citations to identify 76 relevant studies (yield rate of 0.0168) [ 2 ]. Wright et al. specifically recorded the time to undertake citation chasing in their study (discussed below in resource use), [ 22 ] whereas Hinde et al. did not report the time taken to search but state that the search was ‘very time consuming’ [ 2 ].

Two studies claimed that replicability of citation searching strategies could be affected by the choice of the tools used [ 1 , 24 ]. Levay et al. questioned the replicability of Google Scholar, since search returns are controlled by Google’s algorithm, meaning that the results returned will change over time and cannot be replicated [ 1 ]. Bramer et al. found reproducibility of citation searching to be low, due to inaccurate or incomplete reporting of citation search strategies by study authors [ 24 ].

Two studies recorded the time taken to citation search, and one study commented on the time needed [ 1 , 3 , 22 ]. Levay et al. reported that citation searching the same 46 studies in Web of Science and Google Scholar took 79 h (Web of Science = 4 h and Google Scholar 75 h) to identify and de-duplicate 783 studies (Web of Science = 46 studies and Google Scholar = 737 studies) [ 1 ]. Wright et al. reported that citation chasing the same 40 studies in Web of Science, Medline, Google Scholar and Scopus took 5 days in total (2 days to download 1680 results from Google Scholar; 1 day to download 2481 results from Web of Science, Scopus and Medline; and 2 days to screen all the studies) [ 22 ]. Both studies commented on the administrative burden of exporting studies from Google Scholar which accounted for the majority of time searching in both cases [ 1 , 22 ]. Conversely, Papaioannou et al. claimed reference tracking and citation searching to be minimally time intensive, yielding unique and high-quality studies. The number of studies citation chased, the time taken to search and the tool used to appraise study quality were not reported [ 3 ].

One study provided data on the costs involved in citation chasing [ 1 ]. Levay et al. reported that the staff time to search Web of Science for 4 h cost between £88 and £136 and the 75 h to search Google Scholar cost between £1650 and £2550, based on staff grades ranging from £22–£34 per hour (all UK Sterling: 2012) [ 1 ].

The handbooks focus on where to handsearch [ 9 , 10 ], and they provide guidance on who should do this [ 9 ]. The studies have a similar focus but they have sought to evaluate effectiveness compared with other search methods [ 25 – 28 ] as well as to evaluate the effectiveness and/or the efficiency of handsearchers in identifying studies [ 29 , 30 ]. Twelve studies were included [ 25 – 36 ].

It is used for ensuring the complete identification of studies or publication types that are not routinely indexed in, or identified by, searches of bibliographic databases, including recently published studies [ 10 ].

Handsearching involves a manual, page-by-page, examination of the entire contents of relevant journals, conference proceedings and abstracts [ 9 , 10 , 27 , 31 ].

Two handbooks and six studies provide detail on selecting journals to handsearch [ 9 , 10 , 25 , 27 , 30 – 33 ]. Three strategies were identified, as set out below.

Using databases (or database search results) to identify journals to handsearch

The handbooks suggest that bibliographic databases can be used to identify which journals to handsearch [ 9 , 10 ]. The Cochrane Handbook, with its focus on identifying studies reporting randomised controlled trails (RCTs), suggests that searches of The Cochrane CENTRAL database, MEDLINE and EMBASE can be used to identify journals that return the greatest number of studies by study design in the relevant topic area of research [ 9 ]. Variations of this approach to selecting journals to handsearch were utilised in three studies [ 25 , 30 , 31 ]. The CRD Handbook suggests analysing the relevant results of the review’s bibliographic database searches in order to identify journals that contain the largest number of relevant studies [ 10 ].

Handsearching journals not indexed in bibliographic databases

The Cochrane Handbook suggests that journals not indexed in MEDLINE or EMBASE should be considered for handsearching [ 9 ]. A study by Blümle et al. considered this strategy necessary to obtain a complete search [ 32 ].

Contacting experts to identify journals to handsearch

Two studies contacted experts to develop a list of journals to handsearch [ 30 , 31 ]. Armstrong et al. contacted organisations to develop a list of non-indexed journals to handsearch (in addition to database searching), and Langham et al. used a combination of database searches, contacting organisations and searches of library shelves to identity relevant journals (in addition to database searching) [ 30 , 31 ]. A list of possible journals to handsearch was provided to professional contacts to appraise and identify any missing journals [ 30 ]. Neither study specifically reports the number of journals identified by experts to handsearch, when compared to the number of journals to handsearch identified by database searching, and there is no discussion of the effectiveness of either method in identifying journals to handsearch.

Five studies explored specifically where or which sections of a journal to handsearch [ 25 , 27 , 28 , 31 , 33 ]. A study by Hopewell et al. handsearched full reports, short reports, editorials, correspondence sections, meeting abstracts and supplements [ 27 ]. Hopewell et al. found that, of the 369 reports uniquely identified by handsearching, 92% were abstracts and/or published in the supplement of journals [ 27 ]; two studies reported the greatest value in searching supplement editions of journals [ 28 , 31 ], since these are not routinely indexed in databases [ 28 ]. Armstrong et al. identified three studies (out of 131) through searching supplement editions of journals [ 31 ], and Jadad et al. identified 162 eligible RCTs from a total of 2889 abstracts reported in four journals [ 28 ]; Croft et al. claimed value in searching the correspondence section of journals but they did not record the effect of handsearching this section in terms of identification of studies [ 33 ]; and Adams et al. reported handsearching book reviews and identifying one study [ 25 ].

Table  3 summarises a claimed advantage of handsearching, since the studies demonstrate that handsearching identifies studies missed through database searching. Where the studies reported the reason that the studies were missed by database searching (the advantage of handsearching), these are summarised in Table  3 .

Handsearching results

Table  3 also summarises a claimed disadvantage of handsearching since, even though this method is often defined as a ‘gold standard’, the studies demonstrate that database searching can identify studies missed by handsearching. Where the studies reported the reason that the studies were missed by handsearching (the disadvantage over database searching), these are summarised in Table  3 .

Two studies claimed that the precision of handsearching was low when compared to the precision found in database searching [ 25 , 28 ]. Table  3 records the relative precision between handsearching and MEDLINE searching. Two studies claimed that the time needed to handsearch, and access to resources (including handsearchers), was a disadvantage of handsearching [ 31 , 36 ].

Seven studies reported detail on the time taken to handsearch [ 25 , 28 , 29 , 31 , 33 , 34 , 36 ]. There was no agreement between the studies on how long handsearching takes. The range was between 6 min [ 36 ] and 1 h [ 29 ] per journal handsearched. It is not possible to calculate an average, since not all studies reported their handsearching as time per journal handsearched. One study reported handsearching in ‘two hour bursts’ across 3 months in order to focus concentration, but the detail of how often these ‘bursts’ occurred and the effectiveness relative to ‘non-burst’ handsearching is not reported [ 33 ].

Jadad et al. reported the time taken specifically to handsearch the supplement editions [ 28 ]. Two thousand, eight hundred and eighty-nine abstracts were handsearched in 172 min with an average of 1.1 min per eligible study identified [ 28 ].

The use of volunteers [ 29 , 30 ] or experienced handsearchers [ 27 , 31 ] varied in studies. Due to the varied outcome measures used between the studies, it is not possible to aggregate the effectiveness of experienced handsearchers against volunteers. Moher et al., however, specifically sought to test the effectiveness of volunteers in identifying RCTs, finding that volunteers with minimal training can contribute to handsearching [ 29 ]. Conversely, a study by Langham et al. discussed a possible explanation of their volunteer handsearcher missing studies was a lack of specific knowledge to identify RCTs [ 30 ], which suggests experience or training is necessary. Milne and Thorogood suggested that handsearching may need to be undertaken by more than one person [ 36 ].

Five studies provided data on training given to handsearchers [ 25 , 27 , 29 , 30 , 34 ]. This included specific training on RCTs [ 27 , 29 ], a 2-h training session [ 29 , 34 ] and an information pack including guidelines to handsearching, developed by experienced handsearchers, and a thesaurus of terms to identify RCTs [ 30 ].This data was reported narratively, and supporting information, such as the information pack reported in the study by Langham et al., was not provided in the studies. [ 30 ].

Two studies provided guidance on approaches to handsearching if resources were limited [ 27 , 28 ]. Hopewell et al. claimed that, where resources are limited (and it was accepted that studies would be missed), and the aim of searching is the comprehensive identification of studies reporting RCTs, handsearching is best targeted on journals not indexed in MEDLINE and journals published before 1991 (the year the publication type indexing term for RCTs was introduced into MEDLINE [ 37 ]) [ 27 ]. Jadad et al., in a study focused on identifying RCTs, claimed that a combination of MEDLINE searches with selective handsearching of abstracts of letters may be a good alternative to comprehensive handsearching [ 28 ].

Armstrong et al. claimed that researchers handsearching for non-randomised study designs may need more time to handsearch. No guidance on speculative timing was given [ 31 ].

Moher et al. provided data on costs. Moher et al. recorded costs for photocopying (10–15 Cents Canadian per page) and car parking (10 Dollars Canadian) in their 1995 study assessing the use of volunteers to handsearch [ 29 ].

Searching trial registers

The handbooks focus on the benefit of searching registers [ 10 ], with The Cochrane Handbook providing specific guidance on where to search [ 9 ]. The studies focused on the searching of the registers [ 38 ] and the advantages and disadvantages of doing so [ 39 , 40 ]. Three studies were included [ 38 – 40 ].

It is used for identifying unpublished, recently completed or on-going trials [ 9 , 10 , 39 , 40 ] and keeping a track of any adaptations to trial protocols and reported study outcomes [ 39 , 40 ]. Trials that have been stopped, or were unable to reach optimal recruitment, can also be identified.

The Cochrane Handbook includes a comprehensive list of trial registers to search [ 9 ]. Distinctions are made between national and international trial registers (which hold trials of any population or intervention), subject (i.e. population)-specific registers and pharmaceutical/industry trial registers [ 9 ]. There is a further distinction between on-going, completed trial registers and result registers. Glanville et al. also drew a distinction between trial registers (e.g. ClinicalTrials.gov) and portals to trial registers (e.g. WHO) [ 38 ].

Glanville et al. explored the need to search trial registers as a complementary search method to comprehensive searches of bibliographic databases [ 38 ]. Glanville et al. reported that, in both ClinicalTrials.gov and WHO International Clinical Trials Registry Platform (ICTRP), their ‘highly sensitive single concept search’ of the basic interface offered the greatest reliability in identifying known records. The methods of searching are explored in greater detail in this study [ 38 ].

Two studies claimed that searching trial registers will identify unique studies or study data [ 39 , 40 ]. Van est. et al. reported that, in four out of 80 Cochrane reviews included in their study, primary studies were identified and included from a prospective search of a trial register search [ 39 ]. Jones et al. reported that, of 29 studies to record registry search results in their study, 15 found at least one relevant study through searching a register [ 40 ].

Two studies claimed that searching of trial registers facilitates checking of a priori outcome measures against reported final outcome measures [ 39 , 40 ]. Jones et al. suggested that the comparison of registered trials (and trial data) against published trials (and data) will aid the understanding of any potential bias in the trials [ 40 ].

Jones et al. noted that an advantage of trial registers is that they often include contact details for trial investigators, thereby facilitating author contact [ 40 ].

Two studies concluded that trial registers must be searched in combination with other bibliographic resources [ 38 , 39 ]. Glanville et al. concluded that trial registers lag behind major bibliographic databases in terms of their search interfaces [ 38 ].

None were reported.

Web searching

The handbooks report limited guidance for web searching. The CRD Handbook suggests that web searching may be a useful means of identifying grey literature [ 10 ], and The Campbell Handbook provides some guidance on how to undertake web searches, including a list of grey literature websites [ 11 ]. The studies explored the role of web searching in systematic reviews. Five studies were included [ 41 – 45 ].

What is it used for

It is used for identifying published or unpublished studies not indexed or included in bibliographic databases, or studies missed by database (or other) search methods, identifying and retrieving grey literature and identifying study protocols and on-going studies [ 10 , 11 , 42 , 45 ].

The CRD Handbook makes a separation between a search of the internet through a ‘search engine’ and searches of specific and relevant websites [ 10 ]. It considers the latter to be more practical than a general search of the World Wide Web in systematic reviews [ 10 ].

The Campbell Handbook provides guidance on searching using a search engine [ 11 ], and Eysenbach et al. reported the results of a pilot study to assess the search features of 11 search engines for use in searching for systematic reviews [ 45 ]. 1 The Campbell Handbook suggests that, when using search engines, researchers should use the advanced search function. In some cases, this allows searchers to use Boolean logic and employ strategies to limit searches, such as precise phrases like “control group” [ 11 ].

Godin et al. reported the development and use of a web-searching protocol to identify grey literature as part of study identification in a systematic review [ 43 ]. Godin et al. broke their web searching into three parts: first, searches using Google for documents published on the internet; secondly, searches using custom Google search engines; and thirdly, browsing targeted websites of relevant organisations and agencies [ 43 ].

Two studies identified studies uniquely by web searching [ 43 , 45 ]. Eysenbach et al. identified 14 unpublished, on-going or recently finished trials, and at least nine were considered relevant for four systematic reviews [ 45 ]. Godin et al. identified 302 potentially relevant reports of which 15 were included in their systematic review [ 43 ].

Three studies commented on the types of study or study data identified [ 42 , 43 , 45 ]. Eysenbach et al. claimed that internet searches may identify ‘hints’ to on-going or recently completed studies via grey literature [ 45 ]; Godin et al. uniquely identified report literature [ 43 ]; and Stansfield et al. suggested that web searching may identify studies not identified from ‘traditional’ database searches [ 42 ].

Five studies discussed the disadvantages of web searching [ 41 – 45 ]. The studies drew illustrative comparisons between database searching and web searching in order to highlight the disadvantages of web searching:

Three studies commented on searching using a web search engine: Eysenbach et al. reported that current search engines are limited by functionality and that they cover only a fraction of the visible web [ 45 ]; Mahood et al. claimed that their chosen search engines could not accommodate either full or modified search strategies, nor did they support controlled indexing [ 44 ]; and Godin et al. claimed that, in contrast to systematic searches of bibliographic databases, where one search strategy combining all search terms would be used, Google searches may require several search enquiries containing multiple combinations of search terms [ 43 ].

Three studies commented on the number of studies returned through web searching [ 43 – 45 ]. Godin et al. claimed that searching Google can be overwhelming due to the amount of information and lack of consistent organisation of websites [ 43 ]; Mahood et al. had to limit their web searches to title only in order to control search returns [ 44 ], and Eysenbach et al. recorded recall of between 0 and 43.6%, finding references to published studies and precision for hints to published or unpublished studies ranged between 0 and 20.2% [ 45 ].

Three studies commented on the search returns [ 42 , 43 , 45 ]. Eysenbach et al. and Stansfield et al. commented on the lack of abstracts when web searching, which impacts on the precision of web searching and volume of studies identified [ 42 , 45 ], and Godin et al. claimed that it was impossible to screen all results from a Google search, so researchers were reliant on page ranking [ 43 ].

Three studies claimed potential issues with the reliability of items identified through web searching [ 41 , 43 , 45 ]. Godin et al. discussed the possibility of bias created in web searching, where search results are presented depending on geographic location or previous search history [ 43 ]; Briscoe reported that algorithms used by search engines change over time and according to the user, which will influence the identification of studies and impact the transparency and replicability of search reporting [ 41 ]; and Eysenbach et al. reported identifying a study published on-line that differed in reporting to the copy published in the peer-reviewed journal, where adverse event data was omitted in the on-line version [ 45 ].

Stansfield et al. claimed that the lack of functionality to export search results presented a challenge to web searchers [ 42 ]. Three studies claimed that web searching presented difficulties in transparent search reporting [ 41 , 43 , 44 ].

Two studies discussed time taken to web-search [ 43 , 45 ]. Eysenbach et al. reported searching 429 returned search result pages in 21 h [ 45 ], and Godin et al. reports custom Google searching taking 7.9 h and targeted web searches taking 9–11 h, both timings being specific to the case studies in question [ 43 ].

Stansfield et al. discussed planning when to undertake web searching [ 42 ]. Stansfield et al. linked planning a web search to the time-frame and resources available in order to inform where to search [ 42 ].

Mahood et al. claimed that large yields of studies can be difficult and time consuming to explore, sort, manage and process for inclusion [ 44 ]. Mahood et al. initially had to limit their web searching to title only (as a method to control volume) before eventually rejecting their web searching due to concerns about reproducibility and ability to manage search returns [ 44 ].

No studies reported any data relating to the costs involved in web searching.

The discussion will focus on two elements inherent in the research question of this study: how does current supplementary search practice compare with recommended best practice and what are the implications of the evidence for searching using these supplementary methods.

The advent of e-mail (and more specifically the standardised reporting of e-mail addresses for corresponding study authors) would appear to have improved the efficiency of contacting study authors [ 10 , 11 ], although it is possible that it has not altered the effectiveness [ 46 ]. Identifying additional studies or data (the effectiveness) is conditional upon a reply, whatever the method of contact. The guidance of the handbooks, to consider how best to set out requests for studies or study data, is well made but seldom explored in the studies themselves. Whilst making contact is important, which the studies evaluate exploring techniques to improve the rate of reply would be a valuable contribution to improve the efficiency and effectiveness of identifying studies or study data through author contact.

When to contact study authors is worthy of consideration, since the studies included in this review reported a delay between asking for studies or study data and a response. Sufficient time should be allowed between identifying the need for author contact, making contact, a response being provided and the study or data being integrated into the review (with all the methodological implications considered). A recognition for the need of this method, combined with the realisation that this method takes time to yield results, is important. It is perhaps for this reason that, whilst contacting authors is common in systematic reviews, it is not a method of study identification that is undertaken as a matter of course [ 47 ].

The concept of contacting authors could also be understood more broadly than simply contacting with a view to requesting known studies or data. Whilst in contact with authors, requests for unpublished, linked or forthcoming studies are not unreasonable requests, and authors can assist with the interpretation of specific elements of studies or topics, in order to aid the process of critical appraisal. Furthermore, Ogilvie et al. found the value in contacting experts was the link to better reports of studies already identified [ 48 ]. This highlights the potential flexibility of the search method: it is not only the chance to identify known studies or study data but also it offers the opportunity to speak with experts.

The advantages and disadvantages (and resource requirements) were most clearly stated for this supplementary search method. The handbooks, and some studies, suggested and found advantages and disadvantages in the methods and tools.

The Cochrane Handbook suggested that there is little evidence to support the methodology of citation searching, since the citation of studies ‘is far from objective’ [ 49 ]. The studies included in this review suggested that the reasons for ‘non-citation’ are unclear and could range from selective citation (i.e. selective reporting) to pragmatic reasons, such as a review of trials being cited instead of each individual trial reviewed [ 21 ]. Furthermore, a high number of citations for a study should not necessarily be confused as an indicator of study quality [ 50 , 51 ] or a complete citation network. Non-citation of studies, or ‘linking lag’ [ 1 ], forces a break in citational networks [ 1 , 2 , 21 ], meaning it becomes unclear when (or if all) studies have cited a primary study [ 20 ]. There is presently no method to assess the completeness of citational networks and no certainty as to the comprehension of any citation chasing.

There is little common agreement between the studies as to which tool (or combination of tools) is superior in citation chasing, since the relative merits of each resource depend greatly upon the topic of review, the data range of the resource and the currency of the results (c.f. [ 1 , 23 , 24 , 52 – 54 ]). A study that evaluated the tools (Web of Science, SCOPUS and Google Scholar), how the tools are best searched, how the platform hosts select data for inclusion and the advantages and disadvantages of use would make clearer statements on when (or if) to use which tools.

There are, undoubtedly, advantages to citation searching. The citational link is neutral, in the sense that it only links the studies but it does not explain the nature of the link. This is important, since a citation search will identify any study linked to the primary study, including erratum studies and studies that dispute or disagree with the primary study, and it should also link different publication types, such as editorial content, reviews or grey literature. This could not only aid interpretation of studies but also it could help researchers explore the idea of study impact. Furthermore, as reported in the ‘ Results ’ section, a citation search links by citation and it is not beholden to the use of ‘the correct’ search terms or database indexing. It may, therefore, as Papaioannou et al. reported, facilitate serendipitous study identification [ 3 ], suggesting that citation chasing is valuable in scoping review topics, to aid development of searches, and review searches, in order to ensure all studies have been identified.

The nature of bi-directional citation chasing suggests that, given the relative specificity, this method could possibly be used to efficiently update systematic reviews using known includes as the citations to chase [ 20 ]. Researchers have had positive, although incomplete, success trialling this method, and studies suggest that citation chasing alone is not a substitute for standard update searches [ 55 , 56 ].

The evidence on handsearching can be summarised as (1) selecting where to handsearch, (2) what to handsearch and (3) who does the handsearching. In relation to 1, the handbooks advocate selecting journals to handsearch on the basis of the number of relevant studies included from journals identified in database searching. This approach means handsearching is a supplementary method to database searching, since to undertake handsearching—following this method—database searches define the list of journals to handsearch.

Studies included in this review provided empirical evidence that handsearching journals identified by database searching was effective in identifying studies missed by poor indexing, lack of study design or omission of key search terms, or where sections of journals are not indexed on databases. In this way, this approach to selecting journals to handsearch could be categorised as a ‘safety net search’, since it aims to identify studies missed by deficiencies in literature searching and database indexing. This approach to selecting journals to handsearch, even though it is effective, could be argued to be a duplication of effort, since the journals being handsearched have already been ‘searched’ through the bibliographic databases. This is likely why the studies recorded low precision (compared to database searches) and why handsearching takes longer [ 28 ].

The Cochrane Handbook and three studies suggested alternative ways to identify journals to handsearch: namely, selecting journals not indexed on MEDLINE or EMBASE [ 9 , 32 ]—a suggestion that is easily changed to read ‘primary databases’ relevant to the field of study (i.e. ERIC for reviews of educational topics)—and contacting experts, contacting organisations and searches of library shelves [ 30 , 31 ]. Neither the study by Armstrong et al. nor the study by Langham et al. listed the journals identified by method of identification, so it is not clear if there were differences between the list of journals provided by experts when compared to those provided by databases [ 30 , 31 ]. This review did not identify any studies that compared the use of databases to identify journals to handsearch as against these alternative methods but such a study may be of value if efficiencies could be found in practice.

It may be that, in reviews in which a comprehensive identification of studies is required, identifying journals to handsearch should be done both by using databases and contacting experts or organisations. The former being to cover any deficiencies in the database searching and the latter to capture any unique journals or conferences known to experts but not indexed in databases.

Selecting what to handsearch and who should handsearch was another notable difference between the handbooks and studies. The studies included in this review identified studies uniquely from handsearching various sections of journals (from abstracts through to book reviews), and the studies used volunteers, provided training to handsearchers, and used experienced handsearchers to handsearch, with varying degrees of success and failure since handsearching relates to effectively identifying studies when compared to database searching. The Cochrane Collaboration arguably has one of the longest track-records of handsearching projects (c/f [ 37 ]), and it is their recommendation that handsearching is the page-by-page examination of the entire contents of a journal [ 9 , 10 ] by a well-trained handsearcher [ 9 ]. Handsearching is commonly referred to and used as a ‘gold standard’ comparator to establish effectiveness of other search methods. Given that every study included in this review uniquely identified studies by handsearching but also missed studies by handsearching too, a reminder of what constitutes handsearching is likely warranted.

Trial registers

The handbooks provide guidance on where to search and the studies focused on the effectiveness of study identification in selected registers and/or the practicalities of searching registers. In this way, the studies advance the guidance of the handbooks, since they provide empirically derived case-studies of searching the registers. The implications for searching, however, are clear: searching trial registers should still be undertaken in combination with bibliographic database searching [ 38 , 57 ]. Even despite the aims of the International Committee of Medical Journal Editors [ 58 ], comprehensive and prospective registration of trials—and keeping the trial data up to date—is still not common place. It is unclear what pressure (if any) is put upon trial managers who do not prospectively register their trials and, in fact, if there is any active penalty if trial managers do not do so. Until this issue is resolved, the comprehension of registers will remain uncertain and a combination of bibliographic database searching (to identify published trials) and searches of trial registers (to identify recruiting, on-going or completed trials) is required.

The advantages of searching trial registers are worthy of discussion. Registered trials include an e-mail address for trial managers, which can facilitate author contact, and the studies concluded that more consistent searching of trial registers may improve identification of publication and outcome reporting bias [ 40 , 59 ]. If trial managers were using the portals correctly, it would also be a practical method of reporting results and sharing study data, perhaps akin to a ‘project website’, as recommend in the Cochrane Handbook [ 9 ]. The variability of the search interfaces is notably a disadvantage and something upon which could be improved. Glanville et al. observed that the search interfaces lag behind major bibliographic databases [ 38 ]. If the registers themselves are hard to search (and in some cases impossible to export data from), they are less likely to be searched. Trial managers and information specialists/researchers could usefully work together with the registers to develop the interfaces in order to meet the needs of all who use them. The use of trial registers may be broader than only researchers [ 60 ].

In their 2001 study, Eysenbach et al. stated that the role of the internet for identifying studies for systematic reviews is less clear when compared to other methods of study identification [ 45 ]. The handbooks do not update this view, and very few studies were identified in this review which improve upon Eysenbach et al.’s claim. The studies have attempted to take on Eysenbach et al.’s suggestion that a systematic investigation to evaluate the usefulness of the internet for locating evidence is needed. Mahood et al., however, had to abandon their attempts to web-search [ 44 ], but Godin et al. took this work a little further in their case study with reference to identifying grey literature [ 43 ].

The comparative lack of guidance in the handbooks could stem either from a lack of certain knowledge of how to web-search or perhaps a lack of certainty of how to do this systematically, such that web searching could be replicable, and therefore, be included as a method to identify studies without introducing bias. Researchers are exploring the idea of how far web searching can meet the need to be replicable and transparent but still functional [ 41 ]. Further guidance is undoubtedly needed on this supplementary search method.

Limitations

The date range and age of the handbooks and studies included in this review could be considered a limitation of this study.

Comparative and non-comparative case studies form the evidence base for this study. The studies included in this review have been taken at face-value, and no formal quality appraisal has been undertaken since no suitable tool exists. Furthermore, supplementary search methods are typically evaluated in the context of effectiveness, which is potentially a limited test of the contribution they may offer in the process of study identification. Different thresholds of effectiveness and efficiency may apply in the use of supplementary search methods in systematic reviews of qualitative studies when compared to reviews of RCTs, for example.

The studies themselves do not necessarily correlate to the concepts of claimed advantages and disadvantages. In most cases, proposed advantages and disadvantages have not been tested in practice.

Whilst we have aimed to comprehensively identify and review studies for inclusion, the use of supplementary search methods is a broad field of study and it is possible that some completed studies may have been inadvertently missed or overlooked. It is possible that standard systematic review techniques, such as double-screening, would have minimised this risk, but we are confident that, whilst a more systematic approach may have improved the rigour of the study, it is unlikely to alter the conclusions below.

Current supplementary search practice aligns methodologically with recommended best practice. The search methods as recommended in the handbooks are perceptibly the same methods as used in the studies identified in this review. The difference between the handbooks and the studies is of purpose: the studies sought to test the search methods or tools used to undertake the search methods.

The causal inference between methods (as presented in the handbooks) and results (as found in the studies) could be usefully tested to develop our understanding of these supplementary search methods. Further research is needed to better understand these search methods. Specifically, consistency in measuring outcomes, so the results can be generalised and trends identified, which would provide a link not only to better effectiveness data but also to efficiency data, offers researchers a better understanding of the value of using these search methods, or not.

All of the studies discussed in this review claimed to identify additional includable material for their reviews using supplementary search methods that would have been missed using database searches alone. Few of the studies, however, reported the resources required to identify these unique studies. Further, none of the studies used a common framework or provided information that allows a common metric to be calculated. It is not, therefore, possible to compare the resources required to identify any extra study with each search method. This, alongside the use of comparative and non-comparative case studies as the primary study design to test effectiveness, limits our ability to generalise the results of the studies and so reliably interpret the broader efficiency of these search methods. Researchers could usefully consider reporting the amount of time taken to undertake each search method in their search reporting [ 28 , 61 ].

Value versus impact?

Identifying unique studies is commonly interpreted as adding value to the review and the process of searching in and of itself. Only three studies sought to extend this, appraising either the quality of the studies identified or the contribution of the studies to the synthesis as a way of considering the value of the additional studies [ 3 , 16 , 45 ]. In reviews of effectiveness, where all studies should be identified so as to generate a reliable estimate of effect, study value might be a moot point but, in resource-limited situations, or for reviews where a comprehensive identification of studies is less important, study value is an important metric in understanding the contribution of supplementary search methods and the extent to which researchers invest time in undertaking them.

Time + value

Comparing the time taken to search, with a summary estimate of the contribution or value of the studies identified uniquely, against the total number of studies identified, could alter how researchers value supplementary searches. It would permit some basic form of retrospective cost-effectiveness analysis, which would ultimately move literature searching beyond simply claiming that more studies were identified to explaining what studies were identified, at what cost and to what value.

Acknowledgements

CC is grateful to Danica Cooper for her proof-reading and comments. CC is grateful for feedback from Jo Varley-Campbell and the EST and IS team at Exeter Medical School: Jo Thompson-Coon, Rebecca Abbott, Rebecca Whear, Morwenna Rogers, Alison Bethel, Simon Briscoe and Sophie Robinson. CC is grateful to Juan Talens-Bou and Jenny Lowe for their assistance in full-text retrieval.

CC is grateful to Chris Hyde for his help in stimulating the development of this study and his on-going guidance.

This work was funded as part of a PenTAG NIHR Health Technology Assessment Grant.

Availability of data and materials

Abbreviations.

Authors’ contributions

CC conceived, designed and undertook the study as a part of his PhD. AB, NB and RG provided comments on, and discussed, the study in draft as part of CC’s PhD supervision. All authors have approved this manuscript prior to submission.

Authors’ information

CC is a p/t PhD student exploring the use of tailored literature searches in complex systematic reviews. This publication forms a part of his PhD thesis.

Ethics approval and consent to participate

Consent for publication, competing interests.

AB and RG are associate editors of systematic reviews.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

1 Eysenbach et al. recommend Alta Vista but this search engine no longer exists.

Contributor Information

Chris Cooper, Email: [email protected] .

Andrew Booth, Email: [email protected] .

Nicky Britten, Email: [email protected] .

Ruth Garside, Email: [email protected] .

Advertisement

Meditation Benefits and Drawbacks: Empirical Codebook and Implications for Teaching

• Original Research
• Published: 14 January 2019
• Volume 3 , pages 207–220, ( 2019 )

Cite this article

• Thomas Anderson   ORCID: orcid.org/0000-0002-2387-5219 1 ,
• Mallika Suresh 1 &
• Norman AS Farb 1  

3534 Accesses

28 Citations

13 Altmetric

Explore all metrics

Meditation has become a cultural phenomenon, and modern scientific research on the topic has exploded. Thousands of scientific articles report various benefits of meditation including clinical, physiological and well-being outcomes. Despite these benefits, drop-out rates in mindfulness-based interventions remain a problem and little work has studied the drawbacks of meditation. Reports of adverse reactions to meditation have emerged and critical voices have begun advocating caution, rather than enthusiasm, for meditation training. Furthermore, the experiences of meditators outside interventions and conventional lab studies are not well understood. Here we develop an empirical codebook of and framework for meditation benefits and drawbacks (MBDs), discussing the actionable implications for meditation training in the real world. These data reveal the major drawbacks hindering real-world meditators, including several less intuitive drawbacks. We also report the major benefits of meditation and generate a structural framework from which they can be understood in parallel with drawbacks. We investigate whether meditation styles affect MBDs and report comparisons between current and former meditators. These results bring cogent structure to the variety of meditation outcomes laypeople experience. As the number of meditators continues to increase, we need such structures to inform how meditation is taught, ensuring ethically informed consent and optimizing practice-fit. Mixed-methods research such as this study allows a greater practical understanding of how meditation is experienced in situ. This work complements the literature on the clinical benefits and neurophysiological mechanisms of meditation and this framework will inform clinicians, researchers and meditation teachers as best practices are reviewed in the coming years.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Toward a Framework for Reporting and Differentiating Key Features of Meditation- and Mindfulness-Based Interventions

A Framework for the Empirical Investigation of Mindfulness Meditative Development

Dose–response Relationship of Reported Lifetime Meditation Practice with Mental Health and Wellbeing: a Cross-sectional Study

Anderson, T. (2017). Reddit user ‘oredna’, Thomas Anderson, PhD student. Retrieved 10 September 2018, from https://www.reddit.com/user/oredna/

Anderson, T., & Farb, N. A. S. (2018). Personalising practice using preferences for meditation anchor modality. Frontiers in Psychology, 9 . https://doi.org/10.3389/fpsyg.2018.02521 .

Anderson, T., Suresh, M., & Farb, N. (2016). Meditation benefits and drawbacks: empirical codebook and implications for teaching. Retrieved 8 September 2018, from https://osf.io/b2hxu/

Brown, K. W., Ryan, R. M., & Creswell, J. D. (2007). Mindfulness: theoretical foundations and evidence for its salutary effects. Psychological Inquiry, 18 (4), 211–237. https://doi.org/10.1080/10478400701598298 .

Article   Google Scholar  

Carver, C. S., & White, T. L. (1994). Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: the BIS/BAS scales. Journal of Personality and Social Psychology, 67 , 319–333.

Castillo, R. J. (1990). Depersonalization and meditation. Psychiatry, 53 (2), 158–168.

Article   PubMed   Google Scholar  

Cebolla, A., Demarzo, M., Martins, P., Soler, J., & Garcia-Campayo, J. (2017). Unwanted effects: is there a negative side of meditation? A multicentre survey. PLoS One, 12 (9), e0183137. https://doi.org/10.1371/journal.pone.0183137 .

Article   PubMed   PubMed Central   Google Scholar  

Compson, J. (2018). Adverse meditation experiences: navigating Buddhist and secular frameworks for addressing them. Mindfulness, 9 (5), 1358–1369. https://doi.org/10.1007/s12671-017-0878-8 .

Corbin, J. M. (2017). Grounded theory. The Journal of Positive Psychology, 12 (3), 301–302. https://doi.org/10.1080/17439760.2016.1262614 .

Corbin, J. M., & Strauss, A. (1990). Grounded theory research: procedures, canons, and evaluative criteria. Qualitative Sociology, 13 (1), 3–21. https://doi.org/10.1007/BF00988593 .

Couper, M. (2002). Safety of medicines—a guide to detecting and reporting adverse drug reactions—why health professionals need to take action. Retrieved 23 January 2018, from http://apps.who.int/medicinedocs/en/d/Jh2992e/2.html

Crane, C., & Williams, J. M. G. (2010). Factors associated with attrition from mindfulness-based cognitive therapy in patients with a history of suicidal depression. Mindfulness, 1 (1), 10–20 https://doi.org/10.1007/s12671-010-0003-8 .

Crescentini, C., Fabbro, F., & Tomasino, B. (2017). Editorial special topic: enhancing brain and cognition through meditation. Journal of Cognitive Enhancement, 1 (2), 81–83. https://doi.org/10.1007/s41465-017-0033-4 .

Davidson, R. J., & Kaszniak, A. W. (2015). Conceptual and methodological issues in research on mindfulness and meditation. American Psychologist, 70 (7), 581–592.

Dobkin, P. L., Irving, J. A., & Amar, S. (2012). For whom may participation in a mindfulness-based stress reduction program be contraindicated? Mindfulness, 3 (1), 44–50. https://doi.org/10.1007/s12671-011-0079-9 .

Dresler, M., Sandberg, A., Ohla, K., Bublitz, C., Trenado, C., Mroczko-Wąsowicz, A., et al. (2013). Non-pharmacological cognitive enhancement. Neuropharmacology, 64 , 529–543. https://doi.org/10.1016/j.neuropharm.2012.07.002 .

Eberth, J., & Sedlmeier, P. (2012). The effects of mindfulness meditation: a meta-analysis. Mindfulness, 3 (3), 174–189. https://doi.org/10.1007/s12671-012-0101-x .

Elliot, A. J., & Church, M. A. (1997). A hierarchical model of approach and avoidance achievement motivation. Journal of Personality and Social Psychology, 72 (1), 218–232. https://doi.org/10.1037/0022-3514.72.1.218 .

Farb, N., Anderson, T., & Segal, Z. V. (2018). Contemplative practice, well being, and social function. Retrieved 18 June 2018, from https://osf.io/w2f86/

Farias, M., & Wikholm, C. (2016). Has the science of mindfulness lost its mind? BJPsych Bulletin, 40 (6), 329–332. https://doi.org/10.1192/pb.bp.116.053686 .

Farias, M., Wikholm, C., & Delmonte, R. (2016). What is mindfulness-based therapy good for? The Lancet Psychiatry, 3 (11), 1012–1013. https://doi.org/10.1016/S2215-0366(16)30211-5 .

Fenwick, P. (1983). Can we still recommend meditation? British Medical Journal (Clinical Research Ed.), 287 (6403), 1401.

Fortier, M. S., Hogg, W., O’Sullivan, T. L., Blanchard, C., Reid, R. D., Sigal, R. J., et al. (2007). The physical activity counselling (PAC) randomized controlled trial: rationale, methods, and interventions. Applied Physiology, Nutrition, and Metabolism = Physiologie Appliquee, Nutrition Et Metabolisme, 32 (6), 1170–1185. https://doi.org/10.1139/H07-075 .

Gosling, S. D., Rentfrow, P. J., & Swann, W. B. (2003). A very brief measure of the big-five personality domains. Journal of Research in Personality, 37 (6), 504–528. https://doi.org/10.1016/S0092-6566(03)00046-1 .

Goyal, M., Singh, S., Sibinga, E. M. S., Gould, N. F., Rowland-Seymour, A., Sharma, R., et al. (2014). Meditation programs for psychological stress and well-being: a systematic review and meta-analysis. JAMA Internal Medicine, 174 (3), 357–368. https://doi.org/10.1001/jamainternmed.2013.13018 .

Grossman, P., Niemann, L., Schmidt, S., & Walach, H. (2004). Mindfulness-based stress reduction and health benefits: a meta-analysis. Journal of Psychosomatic Research, 57 (1), 35–43. https://doi.org/10.1016/S0022-3999(03)00573-7 .

Jensen, C. G., Vangkilde, S., Frokjaer, V., & Hasselbalch, S. G. (2012). Mindfulness training affects attention—or is it attentional effort? Journal of Experimental Psychology: General, 141 (1), 106–123. https://doi.org/10.1037/a0024931 .

Jha, A. P., Krompinger, J., & Baime, M. J. (2007). Mindfulness training modifies subsystems of attention. Cognitive, Affective, & Behavioral Neuroscience, 7 (2), 109–119.

Josipovic, Z., & Baars, B. J. (2015). Editorial: what can neuroscience learn from contemplative practices? Frontiers in Psychology, 6 . https://doi.org/10.3389/fpsyg.2015.01731 .

Kabat-Zinn, J. (2013). Full catastrophe living , Revised Edition : How to cope with stress, pain and illness using mindfulness meditation . Little, Brown Book Group.

Khoury, B., Lecomte, T., Fortin, G., Masse, M., Therien, P., Bouchard, V., et al. (2013). Mindfulness-based therapy: a comprehensive meta-analysis. Clinical Psychology Review, 33 (6), 763–771. https://doi.org/10.1016/j.cpr.2013.05.005 .

Kuijpers, H. J. H., van der Heijden, F. M. M. A., Tuinier, S., & Verhoeven, W. M. A. (2007). Meditation-induced psychosis. Psychopathology, 40 (6), 461–464. https://doi.org/10.1159/000108125 .

Lindahl, J. R., Fisher, N. E., Cooper, D. J., Rosen, R. K., & Britton, W. B. (2017). The varieties of contemplative experience: a mixed-methods study of meditation-related challenges in Western Buddhists. PLoS One, 12 (5), e0176239. https://doi.org/10.1371/journal.pone.0176239 .

Lustyk, M. K. B., Chawla, N., Nolan, R. S., & Marlatt, G. A. (2009). Mindfulness meditation research: issues of participant screening, safety procedures, and researcher training. Advances in Mind-Body Medicine, 24 (1), 20–30.

PubMed   Google Scholar  

Matejka, J., Glueck, M., Grossman, T., & Fitzmaurice, G. (2016). The effect of visual appearance on the performance of continuous sliders and visual analogue scales. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (pp. 5421–5432). New York: ACM. https://doi.org/10.1145/2858036.2858063

Mesmer-Magnus, J., Manapragada, A., Viswesvaran, C., & Allen, J. W. (2017). Trait mindfulness at work: a meta-analysis of the personal and professional correlates of trait mindfulness. Human Performance, 0 (0), 1–20. https://doi.org/10.1080/08959285.2017.1307842 .

Mrazek, M. D., Franklin, M. S., Phillips, D. T., Baird, B., & Schooler, J. W. (2013). Mindfulness training improves working memory capacity and GRE performance while reducing mind wandering. Psychological Science, 24 (5), 776–781. https://doi.org/10.1177/0956797612459659 .

Ng, B. Y. (1999). Qigong-induced mental disorders: a review. The Australian and New Zealand Journal of Psychiatry, 33 (2), 197–206. https://doi.org/10.1046/j.1440-1614.1999.00536.x .

NRS. (2018). Social Grade | National Readership Survey. Retrieved 9 September 2018, from http://www.nrs.co.uk/nrs-print/lifestyle-and-classification-data/social-grade/

Peer, E., Brandimarte, L., Samat, S., & Acquisti, A. (2017). Beyond the Turk: alternative platforms for crowdsourcing behavioral research. Journal of Experimental Social Psychology, 70 , 153–163. https://doi.org/10.1016/j.jesp.2017.01.006 .

Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. The American Psychologist, 55 (1), 68–78.

Sattelberg, W. (2018). The demographics of Reddit: who uses the site? Retrieved 22 August 2018, from https://www.techjunkie.com/demographics-reddit/

Shapiro, D. H. (1992). Adverse effects of meditation: a preliminary investigation of long-term meditators. International Journal of Psychosomatics: Official Publication of the International Psychosomatics Institute, 39 (1–4), 62–67.

Google Scholar  

Smallwood, J., & Schooler, J. (2006). The restless mind. Psychological Bulletin, 132 (6), 946–958. https://doi.org/10.1037/0033-2909.132.6.946 .

Stol, K., Ralph, D. P., & FitzGerald, B. (2016). Grounded theory in software engineering research: a critical review and guidelines. https://doi.org/10.1145/2884781.2884833

Stürmer, S., Snyder, M., & Omoto, A. M. (2005). Prosocial emotions and helping: the moderating role of group membership. Journal of Personality and Social Psychology, 88 (3), 532–546. https://doi.org/10.1037/0022-3514.88.3.532 .

Tanaka, K. K. (2007). The individual in relation to the Sangha in American Buddhism: an examination of ‘“privatized religion”’. Buddhist-Christian Studies, 27 (1), 115–127. https://doi.org/10.1353/bcs.2007.0028 .

Tang, Y.-Y., Ma, Y., Wang, J., Fan, Y., Feng, S., Lu, Q., et al. (2007). Short-term meditation training improves attention and self-regulation. Proceedings of the National Academy of Sciences of the United States of America, 104 (43), 17152–17156. https://doi.org/10.1073/pnas.0707678104 .

Van Dam, N. T., van Vugt, M. K., Vago, D. R., Schmalzl, L., Saron, C. D., Olendzki, A., et al. (2018). Mind the hype: a critical evaluation and prescriptive agenda for research on mindfulness and meditation. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 13 (1), 36–61. https://doi.org/10.1177/1745691617709589 .

West, M. (1979). Meditation. The British Journal of Psychiatry, 135 (5), 457–467. https://doi.org/10.1192/bjp.135.5.457 .

Wong, S. Y. S., Chan, J. Y. C., Zhang, D., Lee, E. K. P., & Tsoi, K. K. F. (2018). The safety of mindfulness-based interventions: a systematic review of randomized controlled trials. Mindfulness, 9 (5), 1344–1357. https://doi.org/10.1007/s12671-018-0897-0 .

Zeidan, F., Johnson, S. K., Diamond, B. J., David, Z., & Goolkasian, P. (2010). Mindfulness meditation improves cognition: evidence of brief mental training. Consciousness and Cognition, 19 (2), 597–605. https://doi.org/10.1016/j.concog.2010.03.014 .

Download references

Author information

Authors and affiliations.

Department of Psychology, University of Toronto Mississauga, 3359 Mississauga Road, Mississauga, ON, L5L 1C6, Canada

Thomas Anderson, Mallika Suresh & Norman AS Farb

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Thomas Anderson .

Ethics declarations

All research was conducted under informed consent in accord with the Declaration of Helsinki. Participated volunteered their responses without financial compensation.

Conflict of Interest

The authors declare that there is no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Anderson, T., Suresh, M. & Farb, N.A. Meditation Benefits and Drawbacks: Empirical Codebook and Implications for Teaching. J Cogn Enhanc 3 , 207–220 (2019). https://doi.org/10.1007/s41465-018-00119-y

Download citation

Received : 10 September 2018

Accepted : 21 December 2018

Published : 14 January 2019

Issue Date : 15 June 2019

DOI : https://doi.org/10.1007/s41465-018-00119-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Mindfulness
• Negative outcomes
• Attention training
• Qualitative
• Open science
• Find a journal
• Publish with us
• Track your research

Meditation Benefits and Drawbacks: Empirical Codebook and Implications for Teaching

• Journal of Cognitive Enhancement

• University of Toronto

• McMaster University

Abstract and Figures

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations

• Tatyana Veksler

• Amit Bernstein
• Adam C. Barton

• Erik Jönhagen

• Julieta Galante
• Yanying Chen
• Tianyang Wang
• Mindfulness

• Emily C. Fewster
• Brianna Fitapelli
• Natalia Van Doren
• Eshaan Rawat
• Daniel M. Blonigen

• Kim Johnston

• Samuel Yeung-Shan Wong

• David J. Cooper
• Willoughby B. Britton

• Laura Brandimarte
• Sonam Samat

• Josh W. Allen
• Juliet Corbin
• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

Peer-Reviewed Empirical Articles

It’s Tutorial Thursday! In this series, we explore APA’s library of video tutorials available on the  APA Publishing Training YouTube channel . Please feel free to link to or embed our videos in your library websites or LibGuides, course management systems, or other locations where students, faculty, and researchers will find them.

The first video, Finding High Quality Original Research in APA PsycInfo , is an introduction for all platforms that explains what an empirical study is and what it means for a study to be published in a peer-reviewed journal.

The second video demonstrates how to quickly and easily find peer-reviewed empirical articles using the filters in APA PsycInfo. 

Select the video for the platform or website that hosts APA PsycInfo at your institution. 

• Peer-Reviewed Empirical Articles – Searching APA PsycInfo on APA PsycNet
• Peer-Reviewed Empirical Articles – Searching APA PsycInfo on EBSCOhost 
• Peer-Reviewed Empirical Articles – Searching APA PsycInfo on Ovid
• Peer-Reviewed Empirical Articles – Searching APA PsycInfo on ProQuest

By the end of this  2-part tutorial  you will be able to:

• Explain what it means for an article to be considered an empirical study.
• Describe the process that an article goes through to be published in a peer-reviewed journal.
• Use the filters in APA PsycInfo to access empirical studies from peer-reviewed journals.
• Determine if an article found in APA PsycInfo is an empirical study from a peer-reviewed journal.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

• Login for Applicant
• Login for ICSSR Officials

Guidelines for the Award of ICSSR Collaborative Empirical Research Project on Solar Energy and Sustainability

• Google Plus

You are here

1. introduction.

1.1 Bharat, as one of the world’s largest economies, is progressing towards major energy transformation. The Government of India has introduced several significant initiatives to promote solar energy as part of its broader renewable energy and sustainability efforts. The National Solar Mission (NSM), launched in 2010, aims to achieve large-scale solar deployment with a target of 100 GW by 2022. The (PM KUSUM) Pradhan Mantri Kisan Urja Suraksha evam Utthaan Mahabhiyaan scheme was introduced in 2019 to support farmers in installing solar pumps and power plants, while the Rooftop Solar Programme sets a goal of 40 GW in rooftop solar capacity. Solar Park Development focuses on large-scale projects with an additional 40 GW capacity. India, as a co-founder of the International Solar Alliance (ISA), leads global efforts in solar energy promotion. The Pradhan Mantri Janjati Adivasi Nyaya Maha Abhiyan (PM JANMAN) emphasizes solar energy for tribal communities, while the PLI Scheme boosts domestic solar manufacturing. Collectively, these initiatives contribute to India's renewable energy targets of 175 GW by 2022 and 450 GW by 2030.

1.2 The nation’s rapidly growing population and increasing industrialisation have led to a surge in energy needs over the past half-century. Traditional fossil fuels, which are both limited and harmful to the environment, are no longer a viable option. Given these challenges, solar energy emerges as a beacon of hope. India’s geographical advantage of abundant sunlight makes solar energy not just a promising option, but a potential game-changer. Empirical research on solar energy and sustainability can identify optimal strategies for transitioning to solar power and reducing the carbon footprint, which will facilitate combating climate change. While solar technology is making massive advances, efficiency, storage, and cost pose a significant challenge. To address these challenges, empirical research can lead to advances by testing new materials, increasing photovoltaic cell efficiency, and improving storage costs. Innovation in these areas is absolutely essential for making solar energy more efficient and attractive for widespread use.

1.3 The solar energy sector holds significant potential for job creation in areas such as manufacturing, installation, maintenance, and research. Empirical studies can map the job creation landscape, identify skill gaps, and suggest educational and training programs to prepare the workforce for the emerging green economy.

1.4 Furthermore, the economic impacts of solar energy investments can attract further investments, both domestic and international, thereby bolstering India’s economy. Empirical research can also address issues emerging from India's heavy reliance on imported fossil fuels. To do so, it can identify practical ways to harness domestic solar energy, enhancing energy security and independence. Furthermore, the research can offer decentralised solar solutions to alleviate energy needs in rural and remote areas. Such an approach will reduce regional economic inequalities. A combination of traditional knowledge and modern technologies supported by robust empirical data can support sustainable and culturally appropriate energy solutions, paving the way for India's brighter and more sustainable future.

1.5 The researchers' role in conducting empirical research is pivotal for effective policymaking for sustainable energy development. By providing robust empirical evidence, researchers play a crucial role in offering insights into approaches to integrate solar power into the national grid, design effective subsidy frameworks, and create incentives for solar energy producers and consumers. Understanding the socio-economic impacts of solar energy adoption allows for policies that maximise benefits while minimizing potential drawbacks. This data-driven approach ensures that policies are technically sound, socially equitable, and economically beneficial.

1.6 In this context, the ICSSR has identified Solar Energy and Sustainability as a key area for empirical research, with the goal of generating outcomes and insights on various aspects of solar energy transformation in both urban and rural sectors.

i. Proposal Requirements:

1.7 Proposals must ensure a significant sample size for research to objectively assess the various dimensions of Solar energy and sustainability issues. Detailed geographical coverage (villages, blocks, and districts) should be shown using charts and GIS-based maps to illustrate project locations.

1.8 Proposals should encompass fact-based and action-oriented research, including a thorough background of the specific research area and a relevant literature review leading to well-defined hypotheses, objectives, and research questions. The proposal must outline a systematic research methodology, specifying sampling methods, data sources, interview schedules/questionnaires, and tools and software for analysis. The study should address quantitative and qualitative approaches, focusing on actionable and applied analysis, and include specific scope and limitations with mitigation strategies.

ii. Research Team Composition:

1.9 The research team should include three to six scholars to develop a comprehensive study, emphasising collaboration among multiple institutions. Researchers from various disciplines, institutions, and regions are encouraged to collaborate on this research.

1.10 The Project Director will be responsible for completing the study successfully and utilising the funds. The Project Director and Co-Project Directors should be from the social and human sciences.

iii. Duration, Budget and Disbursal of Grant

1.11 The project duration will be 10 to 12 months, with a budget of up to Rs. 15 Lakhs. The grant will be disbursed in instalments as deemed fit by the ICSSR. The affiliating institution should open/maintain a dedicated bank account for the ICSSR grant (Scheme Code-0877) that is duly registered at the EAT Module of the PFMS portal.

iv. Methodological Approaches:

1.12 A multi-method approach will be employed to provide a comprehensive analysis. Quantitative studies will deploy statistical methods to analyse existing large-scale datasets (or the data gathered by the researchers) on solar energy production, consumption, and impacts that will offer insights into overall trends. To complement the quantitative exercise, qualitative research will be conducted through in-depth interviews, focus-group discussions, and case studies to capture the nuanced experiences and perspectives of the stakeholders, ranging from policymakers to local users. Additionally, a comparative regional analysis will assess the performance and challenges of solar energy implementation across various states, identifying region-specific solutions and best practices tailored to diverse environmental and socio-economic contexts.

2. ELIGIBILITY

2.1 Researchers who are permanently employed or retired as faculty from UGC Recognized Indian Universities/Deemed to be Universities/ Affiliated Colleges/Institutions under (2) F or 12(B), ICSSR Research Institutes, ICSSR Recognised Institutes and Institutes of National Importance as defined by the Ministry of Education (MoE) are eligible to apply. The applicants should have substantial research experience, which is demonstrable through publications of books/research papers/reports. The Project Director and Co-Project Director must also hold a Ph.D. Degree.

2.2 In exceptional cases, Independent researchers with PhD degrees who are not affiliated permanently with any institution mentioned in Clause 2.1 but have produced at least two sole-author books published by reputed publishers and/or 05 articles in peer-reviewed journals can also be considered as Co-Project Directors. Such scholars will be required to collaborate with a faculty from institutions given in 2.1 above.

2.3 Further, those researchers with PhD degrees who are in contractual appointment in academic/research institutions mentioned in Clause 2.1 and have produced at least two sole-author books published by reputed publishers and/or 05 articles in peer-reviewed journals may also apply as Co-Project Directors. In the event of their contract expiry, they may continue as Co-Project Directors until the completion of the project.

2.4 Senior and retired government and defence officers (having not less than 05 years of regular service) possessing a Ph. D. degree in any social science discipline and having produced at least two sole-author books published by reputed publishers and/or 05 articles in peer-reviewed journals can also apply as Co-Project Directors, in collaboration with a faculty from institutions given in 2.1 above.

2.5 Non-academic participants/stakeholders/local community may also be part of the research team in the capacity of key informants.

3. How to Apply

3.1 The applications shall will be invited through an advertisement on ICSSR website and shall will be promoted through social media platforms of ICSSR.

3.2 The applicants must submit an online application along with the research proposal, annexures, and other required documents in the prescribed format duly forwarded by the Competent Authorities of the affiliated university/college/institute. Hard copies of the same must be submitted within ten days of the last date of submission of the online application. The online application form will be available on the ICSSR website from September 13th, 2024. The last date for online submission is October 13th, 2024.

3.3 Research proposals and final reports should be in English or Hindi. The application form should be filled out in Hindi using Arial/ Mangal Unicode (Devanagari) font.

3.4 Researchers can apply for only one project at a time. For any ongoing or completed project with the ICSSR, the cooling-off period for applying to another project will be one year for the Project Director. The date of cooling period will be calculated from the date of submission of the final report. However, this will not be applicable for minor projects/short-term empirical research projects of duration equal to or less than 12 months. For ICSSR Research Institutes, the cooling period will not be applicable.

4. Procedure for Awards

4.1 The procedure for awarding the projects will be in multiple phases before the declaration of final results. All applications submitted to the ICSSR will be screened and evaluated by the expert committee following a blind review process. Shortlisted applicants will be invited to interact/present at ICSSR (in person or online).

4.2 The expert committee(s) shall make a recommendation(s) for awarding studies and suggest the budget for the proposed studies after interacting with the shortlisted applicants.

4.3 The merit list of selected candidates for Projects will be published on the ICSSR website.

4.4 Only the selected candidates and their affiliating universities shall be informed individually through a provisional award letter clearly specifying the formalities and documents required for joining the Project.

5 Budget and Heads of Expenditure

5.1 The amount will be disbursed in instalments, depending on the funds, phases and duration of the study, as indicated in the Award Letter. ICSSR reserves the right, based on expert opinion, to make changes in the research design, budget and duration of the project.

5.2 The detailed budget estimates and the proportionate Heads of Expenditure for these proposals are to be prepared by the Institute / Project Director/group of scholars.

i. Allocation of Heads of Expenditure

5.3        A. The remuneration for the Research Staff must be according to the ICSSR guidelines.

B. The proportionate allocation of expenditure for the budget heads such as Fieldwork (Travel / Logistics / Boarding, Survey Preparation or Consultancy, etc.); Equipment and Study material (Computer, Printer, Source Material, Books, Journals, Software, Data Sets, workshop/seminar/publication etc.); and Contingency charges etc. shall be as per the ICSSR guidelines given below;

* The project investigator may with the permission of the institution the re-appropriate expenditure from one sub-head to another subject to a maximum of 10% of the particular budget heads. If the study necessitates re-appropriation beyond 10% it may be done only after the approval of the ICSSR.

C. Affiliating Institutional Overhead Charges @ 10% over and above on the awarded grant of the project, subject to a maximum limit of Rs. 2 00,000/-, will be released by the ICSSR after successfully completing the project.

5.4. Project Staff shall be engaged/appointed per the rules by the affiliating institution of the Project Director on a full/ part-time basis during the research work. The project director may decide the duration. The consolidated monthly remuneration/emoluments of the project staff must be according to the following guidelines:

5.5 Selection of Research Staff should be made through an advertisement published on the respective institute’s website and a selection committee, duly approved by the Competent Authority of the institution, consisting of (1) Project Director; (2) One external subject Expert (from outside the institute where the project is located); (3) Dean of the faculty in case of University /Principal in case of College and (4) Head of the Department of the Project Director.

5.6 The rules of affiliating institutes/universities shall be applied to all field work-related expenses of the project director, co-project director (s), and project personnel.

5.7 All equipment and books purchased out of the project fund shall be the property of the affiliating institution. A detailed stock report duly signed by the Head of the Institute / Registrar / Principal must be submitted to the ICSSR. However, ICSSR may request books and/ or equipment if it so requires.

6. Joining and Release of Grants

6.1 The Project Director has to join the project as per the date notified by the ICSSR by submitting the requisite documents, such as an ‘undertaking’ on a Rs.100 stamp paper duly verified by a notary, declaration in prescribed format on a Rs.100 stamp paper duly verified by a notary, Grant-in-Aid bill towards the first instalment on or before the given deadline and Registration Mandate Form of PFMS Account of those affiliating/administering institutions, which have not linked their accounts to PFMS for ICSSR grant. The joining period can be extended only in exceptional circumstances up to a maximum of three months by the ICSSR.

6.2 The total grant awarded for the Research Project will be released to the affiliating institution in instalments, as mentioned in the award letter. The ICSSR will decide in accordance with the overall project requirements.

6.3 The final instalment will be released after receipt of recommendations of the expert for acceptance of the Final Report: Audited statement of accounts (AC) in prescribed format with utilization certificate (UC) in form 12A of GFR for the entire approved project amount duly signed by the Finance Officer/Registrar/Director of the affiliating Institution; and at least five published research papers in the UGC care and Scopus Indexed journals. A detailed stock report duly signed by the Head of the Institute / Registrar / Principal has to be submitted to the ICSSR. The Finance Officer and charted accountant will sign the utilisation certificate of institutions whose accounts are not audited by CAG/AG.

6.4 The Overhead Charges to the affiliating institution will be released after the acceptance of the Final Report, along with the receipt of the final audited Statement of Accounts and Utilisation Certificate in prescribed formats, which the ICSSR shall verify.

6.5 The Project Director will ensure that their expenditure conforms to the approved budget heads and relevant rules. The Audited Statement of Accounts with Utilization Certificate in Form 12A of GFR is mandatory for the entire approved amount for the project.

7 Monitoring of Research Projects

7.1 Research undertaken by a Project Director will be monitored by submitting periodic progress reports in the prescribed format. The project may be discontinued/terminated if research progress is unsatisfactory or any ICSSR rules are violated. In such cases, the entire amount must be refunded with a 10% penal interest.

7.2 The scholar/awardee must acknowledge the support of ICSSR in all their publications resulting from the project output, such as Research Papers, Journal Articles, and Articles in Edited Books, etc., and they must submit a copy of the same to the ICSSR during the course of or after completion of the project. In case of absence of acknowledgement by the scholars, they will be blacklisted, and they will not be able to apply for any schemes of ICSSR in the future. Papers published in Conference/Seminar proceedings will not be considered as they are not peer-reviewed. However, proceedings published by Scopus-indexed / UGC care-listed journals can be considered.

7.3 All project-related queries will be addressed to the Project Director/ Affiliating Institution for their timely reply.

7.4 The ICSSR may, at any time, ask for verification of accounts and other relevant documents related to the Project.

7.5 The ICSSR reserves the right to change the affiliating institute if it is found that the institute is not cooperating with the scholar and is not facilitating the timely completion of the study.

7.6 The final report submitted by the Project Director is mandatorily evaluated by an Expert appointed by the ICSSR before the release of the final instalment is considered.

7.7 The Project Director shall be personally responsible for the project's timely completion. Any member of the project staff, including the project director, cannot submit the project proposal/final report for the award of any University degree/diploma or funding by any institution. However, ICSSR will have no objection if any project staff member utilises the project data for research purposes, provided there are due acknowledgements to ICSSR.

7.8 If the researchers do not submit the requisite documents and the final report in time or the project is not completed in the stipulated period, the scholars will be blacklisted, and legal recourse will be initiated for recovery of the released grant.

7.9 As per the Ministry of Education's (MoE) directions, the amount of grant sanctioned is to be utilized within the stipulated duration of the project. Any amount of the grant remaining unspent shall be refunded to the ICSSR immediately upon the expiration of the project's duration. Suppose the Project Director fails to utilize the grant for the purpose for which the same has been sanctioned/or fails to submit the audited statement of expenditure within the stipulated period. In that case, he/she will be required to refund the grant amount with interest thereon @ 10% per annum.

8 Completion of the Study

8.1 On completion of the study, the Project Director should submit:

A. Two hard copies of the Final Report along with softcopy in both PDF and Word formats;

B. Hard copy of abstract in 500 words along with softcopy in both PDF and word formats;

C. Hard copy of the Executive Summary of the final report in 5000 words along with softcopy in both PDF and Word formats;

D. Similarity index sheet (Plagiarism check) for the final report.

8.2 If the expert suggests any changes in the reports at the time of evaluation, the Project Director shall incorporate the changes within the stipulated time and should submit the following:

A. Soft copy of the modified final report in both PDF and Word formats along with two hard copies;

B. Five copies of the executive summary;

C. Softcopies of (if any) Data Sets, along with well-defined data definitions and other important information for documentation.

8.3 ICSSR checks every report for plagiarism and generates a similarity report. As a policy, ICSSR does not accept reports with a similarity beyond 10 per cent on the similarity index. Scholars must get their final report checked by their affiliating institutions for similarity index and attach a certified report of the same at the time of submission.

8.4 The scholar's final report will be considered satisfactory only after the expert appointed by the ICSSR makes the final recommendation of acceptance.

9. Obligations of the Affiliating Institution

9.1 The affiliating institution must give an undertaking in the prescribed format contained in the Application Form to administer and manage the ICSSR grant.

9.2 It is also required to provide the requisite research infrastructure to the scholar and maintain proper accounts.

9.3 The affiliating institution should open and maintain a dedicated bank account for the ICSSR grant (Scheme Code-0877) duly registered at the EAT Module of the PFMS portal for the release of the grant without any delay. This account should be retained for all projects awarded by ICSSR. 

9.4 The affiliating institution will be under obligation to ensure submission of the final report and an Audited Statement of Accounts and Utilization Certificate (in the prescribed Proforma GFR 12-A) duly certified by the Competent Authority of the institution, including the refund of any unspent balance.

9.5 The affiliating institution shall make suitable arrangements for preserving data relating to the study, such as filled-in schedules, tabulation sheets, manuscripts, reports, etc. The ICSSR reserves the right to demand raw data or such parts of the survey as it deems fit.

9.6 If a Project Director leaves/discontinues the project before completion of the tenure, the affiliating institution shall inform ICSSR immediately and refund the entire amount with a penal interest @ 10% per annum. The unutilised funds pending with the institutions for all projects must be returned to the ICSSR immediately. In case the universities/ institutions do not abide by the rules of the ICSSR, they shall be blacklisted for applying to schemes of ICSSR in the future.

9.7 In case a Project Director passes away before the project's completion, the affiliating institution shall immediately inform ICSSR by submitting a copy of the death certificate and settle the accounts immediately by expediting the refund of any unspent balance.

10. Other Conditions

10.1 The duration of the project includes the time for Final Report writing. In exceptional circumstances, if the ICSSR is satisfied with the progress of the work, including quality publications, an extension may be granted (up to three months for Minor Projects up to six months for Major Projects) without any additional grant. If an extension is needed beyond the above-mentioned period, the matter will be brought up with the competent authority of ICSSR for a decision. If the extension is required, the Project Director must request at least three months before the end of the stipulated tenure for a no-cost extension with a copy of the progress report and reasons for the delay with documentary evidence. Retrospective extension will not be permitted.

10.2 The contingency grant may be utilized for stationery, computer typing-related costs, specialised assistance such as data analysis and consultation for field trips, etc., related to the research work.

10.3 Defaulters of any previous fellowship/project/programme/grant of the ICSSR will not be eligible for consideration. No scholar can participate in a research project or ICSSR fellowship.

10.4 Foreign trips are not permissible within the awarded budget of a project. However, the Project Director may undertake data collection outside India in exceptional cases and if warranted by the needs of the proposal. For this, they must apply separately for consideration under the Data Collection Scheme of the ICSSR International Collaboration Division. However, ICSSR will not be bound to support such data collection from abroad, and the decision of the ICSSR will be final. In either case, the completion of the study should not be consequent upon such data collection support.

10.5 Any request for an additional grant over the sanctioned budget will not be considered.

10.6 The procurement of equipment/assets for the research project is allowed only if it was originally proposed, does not surpass the permissible amount, and adheres to the regulations of the affiliating institution.

10.7 The project director cannot make any changes in the research design at any stage.

10.8 Regarding Transfer of a Project/Appointment of Substitute Project Director:

A. At the request of a university/institute, the ICSSR may permit the appointment of a Substitute Project Director in exceptional circumstances.

B. The ICSSR may also appoint a Substitute Project Director if it is convinced that the project's original awardee will not be able to carry out the study successfully.

C. The ICSSR may transfer the place of the Project from one affiliating institution to another subject to submission of the following:

● Satisfactory progress report (s);

● No objection certificate from both the previous and the new university/institute;

● Audited statement of account, utilisation certificate, and unspent balance, if any.

● However, no transfer of project / Project director should be requested in the last six months of the study.

D. Overhead charges will be apportioned proportionally among the institutes as per the grant released or as may be finally decided by the ICSSR.

E. In case of superannuation of the Project Director and if the institution's rules require, the project transfer to a serving faculty member may be done with prior approval of the ICSSR. The Project's credit shall belong to the original Project Director.

10.9 Consideration under other call(s) would require a fresh proposal.

10.10 The Council reserves the right to reject any application without assigning any reason. It is not responsible for postal delays or losses.

10.11 Incomplete applications will not be considered in any respect.

10.12 The ICSSR has the final authority regarding the interpretation of these guidelines or any other issue.

10.13 The ICSSR will not entertain queries until the final declaration of results against a call. Lobbying for an award will lead to disqualification.

10.14 The ICSSR retains all rights to publish any project funded by it, contingent upon the recommendation by expert(s) appointed by ICSSR for publication. ICSSR shall hold copyright for the final report and outcomes of the project. Any publication or dissemination of research findings shall solely be at the discretion of ICSSR.