what makes a marketing research study valid

Frequently Asked Questions

What is reliability and validity in research.

Reliability in research refers to the consistency and stability of measurements or findings. Validity relates to the accuracy and truthfulness of results, measuring what the study intends to. Both are crucial for trustworthy and credible research outcomes.

What is validity?

Validity in research refers to the extent to which a study accurately measures what it intends to measure. It ensures that the results are truly representative of the phenomena under investigation. Without validity, research findings may be irrelevant, misleading, or incorrect, limiting their applicability and credibility.

What is reliability?

Reliability in research refers to the consistency and stability of measurements over time. If a study is reliable, repeating the experiment or test under the same conditions should produce similar results. Without reliability, findings become unpredictable and lack dependability, potentially undermining the study’s credibility and generalisability.

What is reliability in psychology?

In psychology, reliability refers to the consistency of a measurement tool or test. A reliable psychological assessment produces stable and consistent results across different times, situations, or raters. It ensures that an instrument’s scores are not due to random error, making the findings dependable and reproducible in similar conditions.

What is test retest reliability?

Test-retest reliability assesses the consistency of measurements taken by a test over time. It involves administering the same test to the same participants at two different points in time and comparing the results. A high correlation between the scores indicates that the test produces stable and consistent results over time.

How to improve reliability of an experiment?

• Standardise procedures and instructions.
• Use consistent and precise measurement tools.
• Train observers or raters to reduce subjective judgments.
• Increase sample size to reduce random errors.
• Conduct pilot studies to refine methods.
• Repeat measurements or use multiple methods.
• Address potential sources of variability.

What is the difference between reliability and validity?

Reliability refers to the consistency and repeatability of measurements, ensuring results are stable over time. Validity indicates how well an instrument measures what it’s intended to measure, ensuring accuracy and relevance. While a test can be reliable without being valid, a valid test must inherently be reliable. Both are essential for credible research.

Are interviews reliable and valid?

Interviews can be both reliable and valid, but they are susceptible to biases. The reliability and validity depend on the design, structure, and execution of the interview. Structured interviews with standardised questions improve reliability. Validity is enhanced when questions accurately capture the intended construct and when interviewer biases are minimised.

Are IQ tests valid and reliable?

IQ tests are generally considered reliable, producing consistent scores over time. Their validity, however, is a subject of debate. While they effectively measure certain cognitive skills, whether they capture the entirety of “intelligence” or predict success in all life areas is contested. Cultural bias and over-reliance on tests are also concerns.

Are questionnaires reliable and valid?

Questionnaires can be both reliable and valid if well-designed. Reliability is achieved when they produce consistent results over time or across similar populations. Validity is ensured when questions accurately measure the intended construct. However, factors like poorly phrased questions, respondent bias, and lack of standardisation can compromise their reliability and validity.

You May Also Like

This post provides the key disadvantages of secondary research so you know the limitations of secondary research before making a decision.

In correlational research, a researcher measures the relationship between two or more variables or sets of scores without having control over the variables.

In historical research, a researcher collects and analyse the data, and explain the events that occurred in the past to test the truthfulness of observations.

USEFUL LINKS

LEARNING RESOURCES

COMPANY DETAILS

• How It Works

Instant insights, infinite possibilities

Validity in research: a guide to measuring the right things

Last updated

27 February 2023

Reviewed by

Cathy Heath

Short on time? Get an AI generated summary of this article instead

Validity is necessary for all types of studies ranging from market validation of a business or product idea to the effectiveness of medical trials and procedures. So, how can you determine whether your research is valid? This guide can help you understand what validity is, the types of validity in research, and the factors that affect research validity.

Make research less tedious

Dovetail streamlines research to help you uncover and share actionable insights

• What is validity?

In the most basic sense, validity is the quality of being based on truth or reason. Valid research strives to eliminate the effects of unrelated information and the circumstances under which evidence is collected. 

Validity in research is the ability to conduct an accurate study with the right tools and conditions to yield acceptable and reliable data that can be reproduced. Researchers rely on carefully calibrated tools for precise measurements. However, collecting accurate information can be more of a challenge.

Studies must be conducted in environments that don't sway the results to achieve and maintain validity. They can be compromised by asking the wrong questions or relying on limited data. 

Why is validity important in research?

Research is used to improve life for humans. Every product and discovery, from innovative medical breakthroughs to advanced new products, depends on accurate research to be dependable. Without it, the results couldn't be trusted, and products would likely fail. Businesses would lose money, and patients couldn't rely on medical treatments. 

While wasting money on a lousy product is a concern, lack of validity paints a much grimmer picture in the medical field or producing automobiles and airplanes, for example. Whether you're launching an exciting new product or conducting scientific research, validity can determine success and failure.

• What is reliability?

Reliability is the ability of a method to yield consistency. If the same result can be consistently achieved by using the same method to measure something, the measurement method is said to be reliable. For example, a thermometer that shows the same temperatures each time in a controlled environment is reliable.

While high reliability is a part of measuring validity, it's only part of the puzzle. If the reliable thermometer hasn't been properly calibrated and reliably measures temperatures two degrees too high, it doesn't provide a valid (accurate) measure of temperature. 

Similarly, if a researcher uses a thermometer to measure weight, the results won't be accurate because it's the wrong tool for the job. 

• How are reliability and validity assessed?

While measuring reliability is a part of measuring validity, there are distinct ways to assess both measurements for accuracy. 

How is reliability measured?

These measures of consistency and stability help assess reliability, including:

Consistency and stability of the same measure when repeated multiple times and conditions

Consistency and stability of the measure across different test subjects

Consistency and stability of results from different parts of a test designed to measure the same thing

How is validity measured?

Since validity refers to how accurately a method measures what it is intended to measure, it can be difficult to assess the accuracy. Validity can be estimated by comparing research results to other relevant data or theories.

The adherence of a measure to existing knowledge of how the concept is measured

The ability to cover all aspects of the concept being measured

The relation of the result in comparison with other valid measures of the same concept

• What are the types of validity in a research design?

Research validity is broadly gathered into two groups: internal and external. Yet, this grouping doesn't clearly define the different types of validity. Research validity can be divided into seven distinct groups.

Face validity : A test that appears valid simply because of the appropriateness or relativity of the testing method, included information, or tools used.

Content validity : The determination that the measure used in research covers the full domain of the content.

Construct validity : The assessment of the suitability of the measurement tool to measure the activity being studied.

Internal validity : The assessment of how your research environment affects measurement results. This is where other factors can’t explain the extent of an observed cause-and-effect response.

External validity : The extent to which the study will be accurate beyond the sample and the level to which it can be generalized in other settings, populations, and measures.

Statistical conclusion validity: The determination of whether a relationship exists between procedures and outcomes (appropriate sampling and measuring procedures along with appropriate statistical tests).

Criterion-related validity : A measurement of the quality of your testing methods against a criterion measure (like a “gold standard” test) that is measured at the same time.

• Examples of validity

Like different types of research and the various ways to measure validity, examples of validity can vary widely. These include:

A questionnaire may be considered valid because each question addresses specific and relevant aspects of the study subject.

In a brand assessment study, researchers can use comparison testing to verify the results of an initial study. For example, the results from a focus group response about brand perception are considered more valid when the results match that of a questionnaire answered by current and potential customers.

A test to measure a class of students' understanding of the English language contains reading, writing, listening, and speaking components to cover the full scope of how language is used.

• Factors that affect research validity

Certain factors can affect research validity in both positive and negative ways. By understanding the factors that improve validity and those that threaten it, you can enhance the validity of your study. These include:

Random selection of participants vs. the selection of participants that are representative of your study criteria

Blinding with interventions the participants are unaware of (like the use of placebos)

Manipulating the experiment by inserting a variable that will change the results

Randomly assigning participants to treatment and control groups to avoid bias

Following specific procedures during the study to avoid unintended effects

Conducting a study in the field instead of a laboratory for more accurate results

Replicating the study with different factors or settings to compare results

Using statistical methods to adjust for inconclusive data

What are the common validity threats in research, and how can their effects be minimized or nullified?

Research validity can be difficult to achieve because of internal and external threats that produce inaccurate results. These factors can jeopardize validity.

History: Events that occur between an early and later measurement

Maturation: The passage of time in a study can include data on actions that would have naturally occurred outside of the settings of the study

Repeated testing: The outcome of repeated tests can change the outcome of followed tests

Selection of subjects: Unconscious bias which can result in the selection of uniform comparison groups

Statistical regression: Choosing subjects based on extremes doesn't yield an accurate outcome for the majority of individuals

Attrition: When the sample group is diminished significantly during the course of the study

Maturation: When subjects mature during the study, and natural maturation is awarded to the effects of the study

While some validity threats can be minimized or wholly nullified, removing all threats from a study is impossible. For example, random selection can remove unconscious bias and statistical regression. 

Researchers can even hope to avoid attrition by using smaller study groups. Yet, smaller study groups could potentially affect the research in other ways. The best practice for researchers to prevent validity threats is through careful environmental planning and t reliable data-gathering methods. 

• How to ensure validity in your research

Researchers should be mindful of the importance of validity in the early planning stages of any study to avoid inaccurate results. Researchers must take the time to consider tools and methods as well as how the testing environment matches closely with the natural environment in which results will be used.

The following steps can be used to ensure validity in research:

Choose appropriate methods of measurement

Use appropriate sampling to choose test subjects

Create an accurate testing environment

How do you maintain validity in research?

Accurate research is usually conducted over a period of time with different test subjects. To maintain validity across an entire study, you must take specific steps to ensure that gathered data has the same levels of accuracy. 

Consistency is crucial for maintaining validity in research. When researchers apply methods consistently and standardize the circumstances under which data is collected, validity can be maintained across the entire study.

Is there a need for validation of the research instrument before its implementation?

An essential part of validity is choosing the right research instrument or method for accurate results. Consider the thermometer that is reliable but still produces inaccurate results. You're unlikely to achieve research validity without activities like calibration, content, and construct validity.

• Understanding research validity for more accurate results

Without validity, research can't provide the accuracy necessary to deliver a useful study. By getting a clear understanding of validity in research, you can take steps to improve your research skills and achieve more accurate results.

Should you be using a customer insights hub?

Do you want to discover previous research faster?

Do you share your research findings with others?

Do you analyze research data?

Start for free today, add your research, and get to key insights faster

Editor’s picks

Last updated: 18 April 2023

Last updated: 27 February 2023

Last updated: 22 August 2024

Last updated: 5 February 2023

Last updated: 16 August 2024

Last updated: 9 March 2023

Last updated: 30 April 2024

Last updated: 12 December 2023

Last updated: 11 March 2024

Last updated: 4 July 2024

Last updated: 6 March 2024

Last updated: 5 March 2024

Last updated: 13 May 2024

Latest articles

Related topics, .css-je19u9{-webkit-align-items:flex-end;-webkit-box-align:flex-end;-ms-flex-align:flex-end;align-items:flex-end;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-box-flex-wrap:wrap;-webkit-flex-wrap:wrap;-ms-flex-wrap:wrap;flex-wrap:wrap;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;row-gap:0;text-align:center;max-width:671px;}@media (max-width: 1079px){.css-je19u9{max-width:400px;}.css-je19u9>span{white-space:pre;}}@media (max-width: 799px){.css-je19u9{max-width:400px;}.css-je19u9>span{white-space:pre;}} decide what to .css-1kiodld{max-height:56px;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}@media (max-width: 1079px){.css-1kiodld{display:none;}} build next, decide what to build next, log in or sign up.

Get started for free

Validity in Research: Definitions, Types, Significance, and Its Relationship with Reliability

Is validity ensured in your research process? Read more to explore the importance and types of validity in research.

Godi Yeshaswi

April 4, 2024

In this Article

In market research , ensuring the accuracy of your measurements is very important. The measurement of accuracy is called validity in research. It's about making sure your study truly represents what it aims to. Accuracy is important whether you're using measurement tools or tools for gathering information like surveys , questionnaires , and interviews.

How do you conclude that your research is valid, or how do you identify validity in research? This article gives deep insights into the concept of validity, explores various types of validity in research, and examines the factors influencing research validity.

What is Validity in Research?

Validity in research refers to the accuracy and significance of your data. When your research is valid, it means that your methodology and measurement technique are valid and capable of gathering results that align with the research objective. To establish validity, statistical techniques and concepts are used - Pearson Correlation Coeffecient and Cronobach’s Alpha

For example, think of a scenario in the consumer packaged goods (CPG) industry where a company is testing a new product packaging design . If the research consistently shows that consumers prefer the new packaging and their purchasing behavior supports this preference, then the study exhibits high validity. In this example, the research produces valid results that genuinely reflect consumer preferences, indicating the validity of the study.

Types of Validity in Research

Construct Validity

Construct validity ensures that the method used to measure something aligns with what you're trying to measure. For instance, imagine a consumer packaged goods (CPG) company wants to gauge customer satisfaction with a new product. They create a survey to measure this, but they need to ensure the questions in the survey genuinely reflect satisfaction with the product, not some other aspect like price perception or brand loyalty.

To establish construct validity, the company must carefully design the survey questions based on existing knowledge about customer satisfaction indicators. They should include questions specifically related to product performance, features, and overall experience, excluding irrelevant factors.

In this example, evidence supporting the construct validity of the survey could include correlations between positive survey responses and repeat purchases or recommendations to friends, indicating that the survey accurately measures customer satisfaction with the product.

Content Validity

Content validity examines whether a test adequately represents all aspects of the topic being measured.

For a media and entertainment company conducting a viewer survey on a new television show, content validity ensures that the survey questions cover all essential aspects of the viewers' experience. This includes aspects like plot engagement, character development, production quality, and overall enjoyment. If the survey overlooks key elements or includes irrelevant questions, the validity of the results is compromised, potentially leading to biased findings.

Face Validity

Face validity evaluates how appropriate the content of a test appears at first glance. It shares similarities with content validity but is a more informal and subjective evaluation. Since face validity relies on personal judgment, it's often seen as the least robust form of validity. Nonetheless, it can offer insights during the early stages of method development.

For example, imagine a media and entertainment company creating a survey to assess viewer satisfaction with a new TV show. Face validity would involve reviewing the survey questions to ensure they seem relevant and aligned with the goal of measuring viewer satisfaction. While face validity alone may not guarantee accuracy, it can provide valuable feedback for refining the survey before administering it to a larger audience.

Criterion Validity

Criterion validity, also known as criterion-related validity, assesses how well a test measures the specific outcome it was intended to measure. This outcome could be a disease, behavior, or performance. Concurrent validity checks if a test correlates with criterion variables simultaneously.

For instance, if a company wants to measure the effectiveness of a new employee training program, concurrent validity would involve comparing the test results of employees who have completed the program to their actual job performance ratings during the same time period.

Importance of Validity in Research

Validity in research is crucial because it ensures that the data and findings obtained are accurate, meaningful, and reliable. Here's why validity is important:

Credibility  

Validity enhances the credibility of research findings. It ensures that the conclusions drawn from the study accurately represent the phenomenon being studied, making the research more trustworthy.

Accurate Decision Making  

Valid research enables researchers, policymakers, practitioners, and other stakeholders to make informed decisions based on reliable evidence. Without validity, decisions may be based on flawed or misleading information.

Generalizability 

Valid research allows for the generalization of findings to broader populations or contexts. When validity is established, researchers can confidently apply their findings beyond the specific study sample or setting.

Prevents Bias 

Validity helps to mitigate bias and ensures that the research accurately reflects the true nature of the phenomenon under investigation. This is essential for maintaining the integrity of the research process.

Reliability vs Validity in Research

In data collection and research, two fundamental ideas are important: reliability and validity. These concepts are essential for maintaining the integrity of research outcomes, ensuring that the information gathered and the conclusions reached are both meaningful and dependable. To grasp their true significance in research, let's explore the core of these concepts of reliability and validity.

What is Reliability?

Reliability in research means that collecting data is consistent and dependable, like having a steady hand that produces the same result for a given task.

In the research context, reliability ensures that the results would be consistent each time if the same study were repeated using the same reliable method. It's similar to having different researchers independently conduct the same experiment and obtain outcomes that match perfectly.

For example, consider a consumer packaged goods (CPG) company testing the weight of its product packaging. If they use a scale to measure the weight multiple times and consistently get the same result, it indicates reliability. This means that whether it's the same person or different individuals using the scale, they can rely on consistent measurement, ensuring the data's integrity.

Difference between Reliability and Validity in Research

Final Thoughts

In conclusion, reliability and validity are fundamental concepts in research that ensure data collection and analysis integrity and accuracy. Reliability ensures consistency and dependability in measurements, while validity ensures that the measurements accurately capture the intended construct or phenomenon being studied. 

Understanding and addressing both reliability and validity are essential for producing meaningful and trustworthy research findings. By carefully considering and applying these concepts, researchers can enhance their research's credibility, accuracy, and generalizability, ultimately contributing to informed decision-making and advancing knowledge in their respective fields.

Therefore, choosing the right consumer research tools like Decode is very important as it enhances data collection for both validity as well as reliability.  

{{cta-white}} {{cta-button}}

Frequently Asked Questions

What are the 4 types of validity.

Validity can be proven by establishing a clear connection between the test and its intended measurement. This can be achieved by demonstrating that a study possesses one or more of the four types of validity: content validity, criterion-related validity, construct validity, and/or face validity.

What is reliability and validity?

Reliability and validity both assess how effectively a method measures something: Reliability focuses on the consistency of a measure (whether the results can be replicated under similar conditions), while validity pertains to the accuracy of a measure (whether the results truly represent what they are intended to measure).

What is reliability in research?

Reliability in research refers to the extent to which a study can be reproduced or replicated. If a study can be repeated and consistently yields the same results, it is deemed reliable. Reliability can be observed over time or across different groups of participants.

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.

With lots of unique blocks, you can easily build a page without coding.

Click on Study templates

Start from scratch

Add blocks to the content

Saving the Template

Publish the Template

Yeshaswi is a dedicated and enthusiastic individual with a strong affinity for tech and all things content. When he's not at work, he channels his passion into his love for football, especially for F.C. Barcelona and the GOAT, Lionel Messi. Instead of hitting the town for parties, he prefers to spend quality time cuddling with his Golden Retriever, Oreo.

Product Marketing Specialist

Related Articles

Top AI Events You Do Not Want to Miss in 2024

Here are all the top AI events for 2024, curated in one convenient place just for you.

Top Insights Events You Do Not Want to Miss in 2024

Here are all the top Insights events for 2024, curated in one convenient place just for you.

Top CX Events You Do Not Want to Miss in 2024

Here are all the top CX events for 2024, curated in one convenient place just for you.

How to Build an Experience Map: A Complete Guide

An experience map is essential for businesses, as it highlights the customer journey, uncovering insights to improve user experiences and address pain points. Read to find more!

Everything You Need to Know about Intelligent Scoring

Are you curious about Intelligent Scoring and how it differs from regular scoring? Discover its applications and benefits. Read on to learn more!

Qualitative Research Methods and Its Advantages In Modern User Research

Discover how to leverage qualitative research methods, including moderated sessions, to gain deep user insights and enhance your product and UX decisions.

The 10 Best Customer Experience Platforms to Transform Your CX

Explore the top 10 CX platforms to revolutionize customer interactions, enhance satisfaction, and drive business growth.

TAM SAM SOM: What It Means and How to Calculate It?

Understanding TAM, SAM, SOM helps businesses gauge market potential. Learn their definitions and how to calculate them for better business decisions and strategy.

Understanding Likert Scales: Advantages, Limitations, and Questions

Using Likert scales can help you understand how your customers view and rate your product. Here's how you can use them to get the feedback you need.

Mastering the 80/20 Rule to Transform User Research

Find out how the Pareto Principle can optimize your user research processes and lead to more impactful results with the help of AI.

Understanding Consumer Psychology: The Science Behind What Makes Us Buy

Gain a comprehensive understanding of consumer psychology and learn how to apply these insights to inform your research and strategies.

A Guide to Website Footers: Best Design Practices & Examples

Explore the importance of website footers, design best practices, and how to optimize them using UX research for enhanced user engagement and navigation.

Customer Effort Score: Definition, Examples, Tips

A great customer score can lead to dedicated, engaged customers who can end up being loyal advocates of your brand. Here's what you need to know about it.

How to Detect and Address User Pain Points for Better Engagement

Understanding user pain points can help you provide a seamless user experiences that makes your users come back for more. Here's what you need to know about it.

What is Quota Sampling? Definition, Types, Examples, and How to Use It?

Discover Quota Sampling: Learn its process, types, and benefits for accurate consumer insights and informed marketing decisions. Perfect for researchers and brand marketers!

What Is Accessibility Testing? A Comprehensive Guide

Ensure inclusivity and compliance with accessibility standards through thorough testing. Improve user experience and mitigate legal risks. Learn more.

Maximizing Your Research Efficiency with AI Transcriptions

Explore how AI transcription can transform your market research by delivering precise and rapid insights from audio and video recordings.

Understanding the False Consensus Effect: How to Manage it

The false consensus effect can cause incorrect assumptions and ultimately, the wrong conclusions. Here's how you can overcome it.

5 Banking Customer Experience Trends to Watch Out for in 2024

Discover the top 5 banking customer experience trends to watch out for in 2024. Stay ahead in the evolving financial landscape.

The Ultimate Guide to Regression Analysis: Definition, Types, Usage & Advantages

Master Regression Analysis: Learn types, uses & benefits in consumer research for precise insights & strategic decisions.

EyeQuant Alternative

Meet Qatalyst, your best eyequant alternative to improve user experience and an AI-powered solution for all your user research needs.

EyeSee Alternative

Embrace the Insights AI revolution: Meet Decode, your innovative solution for consumer insights, offering a compelling alternative to EyeSee.

Skeuomorphism in UX Design: Is It Dead?

Skeuomorphism in UX design creates intuitive interfaces using familiar real-world visuals to help users easily understand digital products. Do you know how?

Top 6 Wireframe Tools and Ways to Test Your Designs

Wireframe tools assist designers in planning and visualizing the layout of their websites. Look through this list of wireframing tools to find the one that suits you best.

Revolutionizing Customer Interaction: The Power of Conversational AI

Conversational AI enhances customer service across various industries, offering intelligent, context-aware interactions that drive efficiency and satisfaction. Here's how.

User Story Mapping: A Powerful Tool for User-Centered Product Development

Learn about user story mapping and how it can be used for successful product development with this blog.

What is Research Hypothesis: Definition, Types, and How to Develop

Read the blog to learn how a research hypothesis provides a clear and focused direction for a study and helps formulate research questions.

Understanding Customer Retention: How to Keep Your Customers Coming Back

Understanding customer retention is key to building a successful brand that has repeat, loyal customers. Here's what you need to know about it.

Demographic Segmentation: How Brands Can Use it to Improve Marketing Strategies

Read this blog to learn what demographic segmentation means, its importance, and how it can be used by brands.

Mastering Product Positioning: A UX Researcher's Guide

Read this blog to understand why brands should have a well-defined product positioning and how it affects the overall business.

Discrete Vs. Continuous Data: Everything You Need To Know

Explore the differences between discrete and continuous data and their impact on business decisions and customer insights.

50+ Employee Engagement Survey Questions

Understand how an employee engagement survey provides insights into employee satisfaction and motivation, directly impacting productivity and retention.

What is Experimental Research: Definition, Types & Examples

Understand how experimental research enables researchers to confidently identify causal relationships between variables and validate findings, enhancing credibility.

A Guide to Interaction Design

Interaction design can help you create engaging and intuitive user experiences, improving usability and satisfaction through effective design principles. Here's how.

Exploring the Benefits of Stratified Sampling

Understanding stratified sampling can improve research accuracy by ensuring diverse representation across key subgroups. Here's how.

A Guide to Voice Recognition in Enhancing UX Research

Learn the importance of using voice recognition technology in user research for enhanced user feedback and insights.

The Ultimate Figma Design Handbook: Design Creation and Testing

The Ultimate Figma Design Handbook covers setting up Figma, creating designs, advanced features, prototyping, and testing designs with real users.

The Power of Organization: Mastering Information Architectures

Understanding the art of information architectures can enhance user experiences by organizing and structuring digital content effectively, making information easy to find and navigate. Here's how.

Convenience Sampling: Examples, Benefits, and When To Use It

Read the blog to understand how convenience sampling allows for quick and easy data collection with minimal cost and effort.

What is Critical Thinking, and How Can it be Used in Consumer Research?

Learn how critical thinking enhances consumer research and discover how Decode's AI-driven platform revolutionizes data analysis and insights.

How Business Intelligence Tools Transform User Research & Product Management

This blog explains how Business Intelligence (BI) tools can transform user research and product management by providing data-driven insights for better decision-making.

What is Face Validity? Definition, Guide and Examples

Read this blog to explore face validity, its importance, and the advantages of using it in market research.

What is Customer Lifetime Value, and How To Calculate It?

Read this blog to understand how Customer Lifetime Value (CLV) can help your business optimize marketing efforts, improve customer retention, and increase profitability.

Systematic Sampling: Definition, Examples, and Types

Explore how systematic sampling helps researchers by providing a structured method to select representative samples from larger populations, ensuring efficiency and reducing bias.

Understanding Selection Bias: A Guide

Selection bias can affect the type of respondents you choose for the study and ultimately the quality of responses you receive. Here’s all you need to know about it.

A Guide to Designing an Effective Product Strategy

Read this blog to explore why a well-defined product strategy is required for brands while developing or refining a product.

A Guide to Minimum Viable Product (MVP) in UX: Definition, Strategies, and Examples

Discover what an MVP is, why it's crucial in UX, strategies for creating one, and real-world examples from top companies like Dropbox and Airbnb.

Asking Close Ended Questions: A Guide

Asking the right close ended questions is they key to getting quantitiative data from your users. Her's how you should do it.

Creating Website Mockups: Your Ultimate Guide to Effective Design

Read this blog to learn website mockups- tools, examples and how to create an impactful website design.

Understanding Your Target Market And Its Importance In Consumer Research

Read this blog to learn about the importance of creating products and services to suit the needs of your target audience.

What Is a Go-To-Market Strategy And How to Create One?

Check out this blog to learn how a go-to-market strategy helps businesses enter markets smoothly, attract more customers, and stand out from competitors.

What is Confirmation Bias in Consumer Research?

Learn how confirmation bias affects consumer research, its types, impacts, and practical tips to avoid it for more accurate and reliable insights.

Market Penetration: The Key to Business Success

Understanding market penetration is key to cracking the code to sustained business growth and competitive advantage in any industry. Here's all you need to know about it.

How to Create an Effective User Interface

Having a simple, clear user interface helps your users find what they really want, improving the user experience. Here's how you can achieve it.

Product Differentiation and What It Means for Your Business

Discover how product differentiation helps businesses stand out with unique features, innovative designs, and exceptional customer experiences.

What is Ethnographic Research? Definition, Types & Examples

Read this blog to understand Ethnographic research, its relevance in today’s business landscape and how you can leverage it for your business.

Product Roadmap: The 2024 Guide [with Examples]

Read this blog to understand how a product roadmap can align stakeholders by providing a clear product development and delivery plan.

Product Market Fit: Making Your Products Stand Out in a Crowded Market

Delve into the concept of product-market fit, explore its significance, and equip yourself with practical insights to achieve it effectively.

Consumer Behavior in Online Shopping: A Comprehensive Guide

Ever wondered how online shopping behavior can influence successful business decisions? Read on to learn more.

How to Conduct a First Click Test?

Why are users leaving your site so fast? Learn how First Click Testing can help. Discover quick fixes for frustration and boost engagement.

What is Market Intelligence? Methods, Types, and Examples

Read the blog to understand how marketing intelligence helps you understand consumer behavior and market trends to inform strategic decision-making.

What is a Longitudinal Study? Definition, Types, and Examples

Is your long-term research strategy unclear? Learn how longitudinal studies decode complexity. Read on for insights.

What Is the Impact of Customer Churn on Your Business?

Understanding and reducing customer churn is the key to building a healthy business that keeps customers satisfied. Here's all you need to know about it.

The Ultimate Design Thinking Guide

Discover the power of design thinking in UX design for your business. Learn the process and key principles in our comprehensive guide.

100+ Yes Or No Survey Questions Examples

Yes or no survey questions simplify responses, aiding efficiency, clarity, standardization, quantifiability, and binary decision-making. Read some examples!

What is Customer Segmentation? The ULTIMATE Guide

Explore how customer segmentation targets diverse consumer groups by tailoring products, marketing, and experiences to their preferred needs.

Crafting User-Centric Websites Through Responsive Web Design

Find yourself reaching for your phone instead of a laptop for regular web browsing? Read on to find out what that means & how you can leverage it for business.

How Does Product Placement Work? Examples and Benefits

Read the blog to understand how product placement helps advertisers seek subtle and integrated ways to promote their products within entertainment content.

The Importance of Reputation Management, and How it Can Make or Break Your Brand

A good reputation management strategy is crucial for any brand that wants to keep its customers loyal. Here's how brands can focus on it.

A Comprehensive Guide to Human-Centered Design

Are you putting the human element at the center of your design process? Read this blog to understand why brands must do so.

How to Leverage Customer Insights to Grow Your Business

Genuine insights are becoming increasingly difficult to collect. Read on to understand the challenges and what the future holds for customer insights.

The Complete Guide to Behavioral Segmentation

Struggling to reach your target audience effectively? Discover how behavioral segmentation can transform your marketing approach. Read more in our blog!

Creating a Unique Brand Identity: How to Make Your Brand Stand Out

Creating a great brand identity goes beyond creating a memorable logo - it's all about creating a consistent and unique brand experience for your cosnumers. Here's everything you need to know about building one.

Understanding the Product Life Cycle: A Comprehensive Guide

Understanding the product life cycle, or the stages a product goes through from its launch to its sunset can help you understand how to market it at every stage to create the most optimal marketing strategies.

Empathy vs. Sympathy in UX Research

Are you conducting UX research and seeking guidance on conducting user interviews with empathy or sympathy? Keep reading to discover the best approach.

What is Exploratory Research, and How To Conduct It?

Read this blog to understand how exploratory research can help you uncover new insights, patterns, and hypotheses in a subject area.

First Impressions & Why They Matter in User Research

Ever wonder if first impressions matter in user research? The answer might surprise you. Read on to learn more!

Cluster Sampling: Definition, Types & Examples

Read this blog to understand how cluster sampling tackles the challenge of efficiently collecting data from large, spread-out populations.

Top Six Market Research Trends

Curious about where market research is headed? Read on to learn about the changes surrounding this field in 2024 and beyond.

Lyssna Alternative

Meet Qatalyst, your best lyssna alternative to usability testing, to create a solution for all your user research needs.

What is Feedback Loop? Definition, Importance, Types, and Best Practices

Struggling to connect with your customers? Read the blog to learn how feedback loops can solve your problem!

UI vs. UX Design: What’s The Difference?

Learn how UI solves the problem of creating an intuitive and visually appealing interface and how UX addresses broader issues related to user satisfaction and overall experience with the product or service.

The Impact of Conversion Rate Optimization on Your Business

Understanding conversion rate optimization can help you boost your online business. Read more to learn all about it.

Insurance Questionnaire: Tips, Questions and Significance

Leverage this pre-built customizable questionnaire template for insurance to get deep insights from your audience.

UX Research Plan Template

Read on to understand why you need a UX Research Plan and how you can use a fully customizable template to get deep insights from your users!

Brand Experience: What it Means & Why It Matters

Have you ever wondered how users navigate the travel industry for your research insights? Read on to understand user experience in the travel sector.

The Role of UI Designers in Creating Delightful User Interfaces

UI designers help to create aesthetic and functional experiences for users. Here's all you need to know about them.

Top Usability Testing Tools to Try

Using usability testing tools can help you understand user preferences and behaviors and ultimately, build a better digital product. Here are the top tools you should be aware of.

Understanding User Experience in Travel Market Research

Ever wondered how users navigate the travel industry for your research insights? Read on to understand user experience in the travel sector.

Top 10 Customer Feedback Tools You’d Want to Try

Explore the top 10 customer feedback tools for analyzing feedback, empowering businesses to enhance customer experience.

10 Best UX Communities on LinkedIn & Slack for Networking & Collaboration

Discover the significance of online communities in UX, the benefits of joining communities on LinkedIn and Slack, and insights into UX career advancement.

The Role of Customer Experience Manager in Consumer Research

This blog explores the role of Customer Experience Managers, their skills, their comparison with CRMs, their key metrics, and why they should use a consumer research platform.

Product Review Template

Learn how to conduct a product review and get insights with this template on the Qatalyst platform.

What Is the Role of a Product Designer in UX?

Product designers help to create user-centric digital experiences that cater to users' needs and preferences. Here's what you need to know about them.

Top 10 Customer Journey Mapping Tools For Market Research in 2024

Explore the top 10 tools in 2024 to understand customer journeys while conducting market research.

Generative AI and its Use in Consumer Research

Ever wondered how Generative AI fits in within the research space? Read on to find its potential in the consumer research industry.

All You Need to Know About Interval Data: Examples, Variables, & Analysis

Understand how interval data provides precise numerical measurements, enabling quantitative analysis and statistical comparison in research.

How to Use Narrative Analysis in Research

Find the advantages of using narrative analysis and how this method can help you enrich your research insights.

A Guide to Asking the Right Focus Group Questions

Moderated discussions with multiple participants to gather diverse opinions on a topic.

Maximize Your Research Potential

Experience why teams worldwide trust our Consumer & User Research solutions.

Book a Demo

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• The 4 Types of Validity in Research | Definitions & Examples

The 4 Types of Validity in Research | Definitions & Examples

Published on September 6, 2019 by Fiona Middleton . Revised on June 22, 2023.

Validity tells you how accurately a method measures something. If a method measures what it claims to measure, and the results closely correspond to real-world values, then it can be considered valid. There are four main types of validity:

• Construct validity : Does the test measure the concept that it’s intended to measure?
• Content validity : Is the test fully representative of what it aims to measure?
• Face validity : Does the content of the test appear to be suitable to its aims?
• Criterion validity : Do the results accurately measure the concrete outcome they are designed to measure?

In quantitative research , you have to consider the reliability and validity of your methods and measurements.

Note that this article deals with types of test validity, which determine the accuracy of the actual components of a measure. If you are doing experimental research, you also need to consider internal and external validity , which deal with the experimental design and the generalizability of results.

Table of contents

Construct validity, content validity, face validity, criterion validity, other interesting articles, frequently asked questions about types of validity.

Construct validity evaluates whether a measurement tool really represents the thing we are interested in measuring. It’s central to establishing the overall validity of a method.

What is a construct?

A construct refers to a concept or characteristic that can’t be directly observed, but can be measured by observing other indicators that are associated with it.

Constructs can be characteristics of individuals, such as intelligence, obesity, job satisfaction, or depression; they can also be broader concepts applied to organizations or social groups, such as gender equality, corporate social responsibility, or freedom of speech.

There is no objective, observable entity called “depression” that we can measure directly. But based on existing psychological research and theory, we can measure depression based on a collection of symptoms and indicators, such as low self-confidence and low energy levels.

What is construct validity?

Construct validity is about ensuring that the method of measurement matches the construct you want to measure. If you develop a questionnaire to diagnose depression, you need to know: does the questionnaire really measure the construct of depression? Or is it actually measuring the respondent’s mood, self-esteem, or some other construct?

To achieve construct validity, you have to ensure that your indicators and measurements are carefully developed based on relevant existing knowledge. The questionnaire must include only relevant questions that measure known indicators of depression.

The other types of validity described below can all be considered as forms of evidence for construct validity.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Content validity assesses whether a test is representative of all aspects of the construct.

To produce valid results, the content of a test, survey or measurement method must cover all relevant parts of the subject it aims to measure. If some aspects are missing from the measurement (or if irrelevant aspects are included), the validity is threatened and the research is likely suffering from omitted variable bias .

A mathematics teacher develops an end-of-semester algebra test for her class. The test should cover every form of algebra that was taught in the class. If some types of algebra are left out, then the results may not be an accurate indication of students’ understanding of the subject. Similarly, if she includes questions that are not related to algebra, the results are no longer a valid measure of algebra knowledge.

Face validity considers how suitable the content of a test seems to be on the surface. It’s similar to content validity, but face validity is a more informal and subjective assessment.

You create a survey to measure the regularity of people’s dietary habits. You review the survey items, which ask questions about every meal of the day and snacks eaten in between for every day of the week. On its surface, the survey seems like a good representation of what you want to test, so you consider it to have high face validity.

As face validity is a subjective measure, it’s often considered the weakest form of validity. However, it can be useful in the initial stages of developing a method.

Criterion validity evaluates how well a test can predict a concrete outcome, or how well the results of your test approximate the results of another test.

What is a criterion variable?

A criterion variable is an established and effective measurement that is widely considered valid, sometimes referred to as a “gold standard” measurement. Criterion variables can be very difficult to find.

What is criterion validity?

To evaluate criterion validity, you calculate the correlation between the results of your measurement and the results of the criterion measurement. If there is a high correlation, this gives a good indication that your test is measuring what it intends to measure.

A university professor creates a new test to measure applicants’ English writing ability. To assess how well the test really does measure students’ writing ability, she finds an existing test that is considered a valid measurement of English writing ability, and compares the results when the same group of students take both tests. If the outcomes are very similar, the new test has high criterion validity.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Ecological validity

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Face validity and content validity are similar in that they both evaluate how suitable the content of a test is. The difference is that face validity is subjective, and assesses content at surface level.

When a test has strong face validity, anyone would agree that the test’s questions appear to measure what they are intended to measure.

For example, looking at a 4th grade math test consisting of problems in which students have to add and multiply, most people would agree that it has strong face validity (i.e., it looks like a math test).

On the other hand, content validity evaluates how well a test represents all the aspects of a topic. Assessing content validity is more systematic and relies on expert evaluation. of each question, analyzing whether each one covers the aspects that the test was designed to cover.

A 4th grade math test would have high content validity if it covered all the skills taught in that grade. Experts(in this case, math teachers), would have to evaluate the content validity by comparing the test to the learning objectives.

Criterion validity evaluates how well a test measures the outcome it was designed to measure. An outcome can be, for example, the onset of a disease.

Criterion validity consists of two subtypes depending on the time at which the two measures (the criterion and your test) are obtained:

• Concurrent validity is a validation strategy where the the scores of a test and the criterion are obtained at the same time .
• Predictive validity is a validation strategy where the criterion variables are measured after the scores of the test.

Convergent validity and discriminant validity are both subtypes of construct validity . Together, they help you evaluate whether a test measures the concept it was designed to measure.

• Convergent validity indicates whether a test that is designed to measure a particular construct correlates with other tests that assess the same or similar construct.
• Discriminant validity indicates whether two tests that should not be highly related to each other are indeed not related. This type of validity is also called divergent validity .

You need to assess both in order to demonstrate construct validity. Neither one alone is sufficient for establishing construct validity.

The purpose of theory-testing mode is to find evidence in order to disprove, refine, or support a theory. As such, generalizability is not the aim of theory-testing mode.

Due to this, the priority of researchers in theory-testing mode is to eliminate alternative causes for relationships between variables . In other words, they prioritize internal validity over external validity , including ecological validity .

It’s often best to ask a variety of people to review your measurements. You can ask experts, such as other researchers, or laypeople, such as potential participants, to judge the face validity of tests.

While experts have a deep understanding of research methods , the people you’re studying can provide you with valuable insights you may have missed otherwise.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Middleton, F. (2023, June 22). The 4 Types of Validity in Research | Definitions & Examples. Scribbr. Retrieved September 9, 2024, from https://www.scribbr.com/methodology/types-of-validity/

Is this article helpful?

Fiona Middleton

Other students also liked, reliability vs. validity in research | difference, types and examples, construct validity | definition, types, & examples, external validity | definition, types, threats & examples, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

• Privacy Policy

Home » Validity – Types, Examples and Guide

Validity – Types, Examples and Guide

Table of Contents

Validity is a fundamental concept in research, referring to the extent to which a test, measurement, or study accurately reflects or assesses the specific concept that the researcher is attempting to measure. Ensuring validity is crucial as it determines the trustworthiness and credibility of the research findings.

Research Validity

Research validity pertains to the accuracy and truthfulness of the research. It examines whether the research truly measures what it claims to measure. Without validity, research results can be misleading or erroneous, leading to incorrect conclusions and potentially flawed applications.

How to Ensure Validity in Research

Ensuring validity in research involves several strategies:

• Clear Operational Definitions : Define variables clearly and precisely.
• Use of Reliable Instruments : Employ measurement tools that have been tested for reliability.
• Pilot Testing : Conduct preliminary studies to refine the research design and instruments.
• Triangulation : Use multiple methods or sources to cross-verify results.
• Control Variables : Control extraneous variables that might influence the outcomes.

Types of Validity

Validity is categorized into several types, each addressing different aspects of measurement accuracy.

Internal Validity

Internal validity refers to the degree to which the results of a study can be attributed to the treatments or interventions rather than other factors. It is about ensuring that the study is free from confounding variables that could affect the outcome.

External Validity

External validity concerns the extent to which the research findings can be generalized to other settings, populations, or times. High external validity means the results are applicable beyond the specific context of the study.

Construct Validity

Construct validity evaluates whether a test or instrument measures the theoretical construct it is intended to measure. It involves ensuring that the test is truly assessing the concept it claims to represent.

Content Validity

Content validity examines whether a test covers the entire range of the concept being measured. It ensures that the test items represent all facets of the concept.

Criterion Validity

Criterion validity assesses how well one measure predicts an outcome based on another measure. It is divided into two types:

• Predictive Validity : How well a test predicts future performance.
• Concurrent Validity : How well a test correlates with a currently existing measure.

Face Validity

Face validity refers to the extent to which a test appears to measure what it is supposed to measure, based on superficial inspection. While it is the least scientific measure of validity, it is important for ensuring that stakeholders believe in the test’s relevance.

Importance of Validity

Validity is crucial because it directly affects the credibility of research findings. Valid results ensure that conclusions drawn from research are accurate and can be trusted. This, in turn, influences the decisions and policies based on the research.

Examples of Validity

• Internal Validity : A randomized controlled trial (RCT) where the random assignment of participants helps eliminate biases.
• External Validity : A study on educational interventions that can be applied to different schools across various regions.
• Construct Validity : A psychological test that accurately measures depression levels.
• Content Validity : An exam that covers all topics taught in a course.
• Criterion Validity : A job performance test that predicts future job success.

Where to Write About Validity in A Thesis

In a thesis, the methodology section should include discussions about validity. Here, you explain how you ensured the validity of your research instruments and design. Additionally, you may discuss validity in the results section, interpreting how the validity of your measurements affects your findings.

Applications of Validity

Validity has wide applications across various fields:

• Education : Ensuring assessments accurately measure student learning.
• Psychology : Developing tests that correctly diagnose mental health conditions.
• Market Research : Creating surveys that accurately capture consumer preferences.

Limitations of Validity

While ensuring validity is essential, it has its limitations:

• Complexity : Achieving high validity can be complex and resource-intensive.
• Context-Specific : Some validity types may not be universally applicable across all contexts.
• Subjectivity : Certain types of validity, like face validity, involve subjective judgments.

By understanding and addressing these aspects of validity, researchers can enhance the quality and impact of their studies, leading to more reliable and actionable results.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Content Validity – Measurement and Examples

Criterion Validity – Methods, Examples and...

Face Validity – Methods, Types, Examples

External Validity – Threats, Examples and Types

Test-Retest Reliability – Methods, Formula and...

Internal Validity – Threats, Examples and Guide

Validity & Reliability In Research

A Plain-Language Explanation (With Examples)

By: Derek Jansen (MBA) | Expert Reviewer: Kerryn Warren (PhD) | September 2023

Validity and reliability are two related but distinctly different concepts within research. Understanding what they are and how to achieve them is critically important to any research project. In this post, we’ll unpack these two concepts as simply as possible.

This post is based on our popular online course, Research Methodology Bootcamp . In the course, we unpack the basics of methodology  using straightfoward language and loads of examples. If you’re new to academic research, you definitely want to use this link to get 50% off the course (limited-time offer).

Overview: Validity & Reliability

• The big picture
• Validity 101
• Reliability 101 
• Key takeaways

First, The Basics…

First, let’s start with a big-picture view and then we can zoom in to the finer details.

Validity and reliability are two incredibly important concepts in research, especially within the social sciences. Both validity and reliability have to do with the measurement of variables and/or constructs – for example, job satisfaction, intelligence, productivity, etc. When undertaking research, you’ll often want to measure these types of constructs and variables and, at the simplest level, validity and reliability are about ensuring the quality and accuracy of those measurements .

As you can probably imagine, if your measurements aren’t accurate or there are quality issues at play when you’re collecting your data, your entire study will be at risk. Therefore, validity and reliability are very important concepts to understand (and to get right). So, let’s unpack each of them.

What Is Validity?

In simple terms, validity (also called “construct validity”) is all about whether a research instrument accurately measures what it’s supposed to measure .

For example, let’s say you have a set of Likert scales that are supposed to quantify someone’s level of overall job satisfaction. If this set of scales focused purely on only one dimension of job satisfaction, say pay satisfaction, this would not be a valid measurement, as it only captures one aspect of the multidimensional construct. In other words, pay satisfaction alone is only one contributing factor toward overall job satisfaction, and therefore it’s not a valid way to measure someone’s job satisfaction.

Oftentimes in quantitative studies, the way in which the researcher or survey designer interprets a question or statement can differ from how the study participants interpret it . Given that respondents don’t have the opportunity to ask clarifying questions when taking a survey, it’s easy for these sorts of misunderstandings to crop up. Naturally, if the respondents are interpreting the question in the wrong way, the data they provide will be pretty useless . Therefore, ensuring that a study’s measurement instruments are valid – in other words, that they are measuring what they intend to measure – is incredibly important.

There are various types of validity and we’re not going to go down that rabbit hole in this post, but it’s worth quickly highlighting the importance of making sure that your research instrument is tightly aligned with the theoretical construct you’re trying to measure .  In other words, you need to pay careful attention to how the key theories within your study define the thing you’re trying to measure – and then make sure that your survey presents it in the same way.

For example, sticking with the “job satisfaction” construct we looked at earlier, you’d need to clearly define what you mean by job satisfaction within your study (and this definition would of course need to be underpinned by the relevant theory). You’d then need to make sure that your chosen definition is reflected in the types of questions or scales you’re using in your survey . Simply put, you need to make sure that your survey respondents are perceiving your key constructs in the same way you are. Or, even if they’re not, that your measurement instrument is capturing the necessary information that reflects your definition of the construct at hand.

If all of this talk about constructs sounds a bit fluffy, be sure to check out Research Methodology Bootcamp , which will provide you with a rock-solid foundational understanding of all things methodology-related. Remember, you can take advantage of our 60% discount offer using this link.

Need a helping hand?

What Is Reliability?

As with validity, reliability is an attribute of a measurement instrument – for example, a survey, a weight scale or even a blood pressure monitor. But while validity is concerned with whether the instrument is measuring the “thing” it’s supposed to be measuring, reliability is concerned with consistency and stability . In other words, reliability reflects the degree to which a measurement instrument produces consistent results when applied repeatedly to the same phenomenon , under the same conditions .

As you can probably imagine, a measurement instrument that achieves a high level of consistency is naturally more dependable (or reliable) than one that doesn’t – in other words, it can be trusted to provide consistent measurements . And that, of course, is what you want when undertaking empirical research. If you think about it within a more domestic context, just imagine if you found that your bathroom scale gave you a different number every time you hopped on and off of it – you wouldn’t feel too confident in its ability to measure the variable that is your body weight 🙂

It’s worth mentioning that reliability also extends to the person using the measurement instrument . For example, if two researchers use the same instrument (let’s say a measuring tape) and they get different measurements, there’s likely an issue in terms of how one (or both) of them are using the measuring tape. So, when you think about reliability, consider both the instrument and the researcher as part of the equation.

As with validity, there are various types of reliability and various tests that can be used to assess the reliability of an instrument. A popular one that you’ll likely come across for survey instruments is Cronbach’s alpha , which is a statistical measure that quantifies the degree to which items within an instrument (for example, a set of Likert scales) measure the same underlying construct . In other words, Cronbach’s alpha indicates how closely related the items are and whether they consistently capture the same concept . 

Recap: Key Takeaways

Alright, let’s quickly recap to cement your understanding of validity and reliability:

• Validity is concerned with whether an instrument (e.g., a set of Likert scales) is measuring what it’s supposed to measure
• Reliability is concerned with whether that measurement is consistent and stable when measuring the same phenomenon under the same conditions.

In short, validity and reliability are both essential to ensuring that your data collection efforts deliver high-quality, accurate data that help you answer your research questions . So, be sure to always pay careful attention to the validity and reliability of your measurement instruments when collecting and analysing data. As the adage goes, “rubbish in, rubbish out” – make sure that your data inputs are rock-solid.

Psst… there’s more!

This post is an extract from our bestselling short course, Methodology Bootcamp . If you want to work smart, you don't want to miss this .

THE MATERIAL IS WONDERFUL AND BENEFICIAL TO ALL STUDENTS.

THE MATERIAL IS WONDERFUL AND BENEFICIAL TO ALL STUDENTS AND I HAVE GREATLY BENEFITED FROM THE CONTENT.

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology
• Reliability vs Validity in Research | Differences, Types & Examples

Reliability vs Validity in Research | Differences, Types & Examples

Published on 3 May 2022 by Fiona Middleton . Revised on 10 October 2022.

Reliability and validity are concepts used to evaluate the quality of research. They indicate how well a method , technique, or test measures something. Reliability is about the consistency of a measure, and validity is about the accuracy of a measure.

It’s important to consider reliability and validity when you are creating your research design , planning your methods, and writing up your results, especially in quantitative research .

Table of contents

Understanding reliability vs validity, how are reliability and validity assessed, how to ensure validity and reliability in your research, where to write about reliability and validity in a thesis.

Reliability and validity are closely related, but they mean different things. A measurement can be reliable without being valid. However, if a measurement is valid, it is usually also reliable.

What is reliability?

Reliability refers to how consistently a method measures something. If the same result can be consistently achieved by using the same methods under the same circumstances, the measurement is considered reliable.

What is validity?

Validity refers to how accurately a method measures what it is intended to measure. If research has high validity, that means it produces results that correspond to real properties, characteristics, and variations in the physical or social world.

High reliability is one indicator that a measurement is valid. If a method is not reliable, it probably isn’t valid.

However, reliability on its own is not enough to ensure validity. Even if a test is reliable, it may not accurately reflect the real situation.

Validity is harder to assess than reliability, but it is even more important. To obtain useful results, the methods you use to collect your data must be valid: the research must be measuring what it claims to measure. This ensures that your discussion of the data and the conclusions you draw are also valid.

Prevent plagiarism, run a free check.

Reliability can be estimated by comparing different versions of the same measurement. Validity is harder to assess, but it can be estimated by comparing the results to other relevant data or theory. Methods of estimating reliability and validity are usually split up into different types.

Types of reliability

Different types of reliability can be estimated through various statistical methods.

Types of validity

The validity of a measurement can be estimated based on three main types of evidence. Each type can be evaluated through expert judgement or statistical methods.

To assess the validity of a cause-and-effect relationship, you also need to consider internal validity (the design of the experiment ) and external validity (the generalisability of the results).

The reliability and validity of your results depends on creating a strong research design , choosing appropriate methods and samples, and conducting the research carefully and consistently.

Ensuring validity

If you use scores or ratings to measure variations in something (such as psychological traits, levels of ability, or physical properties), it’s important that your results reflect the real variations as accurately as possible. Validity should be considered in the very earliest stages of your research, when you decide how you will collect your data .

• Choose appropriate methods of measurement

Ensure that your method and measurement technique are of high quality and targeted to measure exactly what you want to know. They should be thoroughly researched and based on existing knowledge.

For example, to collect data on a personality trait, you could use a standardised questionnaire that is considered reliable and valid. If you develop your own questionnaire, it should be based on established theory or the findings of previous studies, and the questions should be carefully and precisely worded.

• Use appropriate sampling methods to select your subjects

To produce valid generalisable results, clearly define the population you are researching (e.g., people from a specific age range, geographical location, or profession). Ensure that you have enough participants and that they are representative of the population.

Ensuring reliability

Reliability should be considered throughout the data collection process. When you use a tool or technique to collect data, it’s important that the results are precise, stable, and reproducible.

• Apply your methods consistently

Plan your method carefully to make sure you carry out the same steps in the same way for each measurement. This is especially important if multiple researchers are involved.

For example, if you are conducting interviews or observations, clearly define how specific behaviours or responses will be counted, and make sure questions are phrased the same way each time.

• Standardise the conditions of your research

When you collect your data, keep the circumstances as consistent as possible to reduce the influence of external factors that might create variation in the results.

For example, in an experimental setup, make sure all participants are given the same information and tested under the same conditions.

It’s appropriate to discuss reliability and validity in various sections of your thesis or dissertation or research paper. Showing that you have taken them into account in planning your research and interpreting the results makes your work more credible and trustworthy.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

Middleton, F. (2022, October 10). Reliability vs Validity in Research | Differences, Types & Examples. Scribbr. Retrieved 9 September 2024, from https://www.scribbr.co.uk/research-methods/reliability-or-validity/

Is this article helpful?

Fiona Middleton

Other students also liked, the 4 types of validity | types, definitions & examples, a quick guide to experimental design | 5 steps & examples, sampling methods | types, techniques, & examples.

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

10.2 Steps in the Marketing Research Process

Learning objective.

• Describe the basic steps in the marketing research process and the purpose of each step.

The basic steps used to conduct marketing research are shown in Figure 10.6 “Steps in the Marketing Research Process” . Next, we discuss each step.

Figure 10.6 Steps in the Marketing Research Process

Step 1: Define the Problem (or Opportunity)

There’s a saying in marketing research that a problem half defined is a problem half solved. Defining the “problem” of the research sounds simple, doesn’t it? Suppose your product is tutoring other students in a subject you’re a whiz at. You have been tutoring for a while, and people have begun to realize you’re darned good at it. Then, suddenly, your business drops off. Or it explodes, and you can’t cope with the number of students you’re being asked help. If the business has exploded, should you try to expand your services? Perhaps you should subcontract with some other “whiz” students. You would send them students to be tutored, and they would give you a cut of their pay for each student you referred to them.

Both of these scenarios would be a problem for you, wouldn’t they? They are problems insofar as they cause you headaches. But are they really the problem? Or are they the symptoms of something bigger? For example, maybe your business has dropped off because your school is experiencing financial trouble and has lowered the number of scholarships given to incoming freshmen. Consequently, there are fewer total students on campus who need your services. Conversely, if you’re swamped with people who want you to tutor them, perhaps your school awarded more scholarships than usual, so there are a greater number of students who need your services. Alternately, perhaps you ran an ad in your school’s college newspaper, and that led to the influx of students wanting you to tutor them.

Businesses are in the same boat you are as a tutor. They take a look at symptoms and try to drill down to the potential causes. If you approach a marketing research company with either scenario—either too much or too little business—the firm will seek more information from you such as the following:

• In what semester(s) did your tutoring revenues fall (or rise)?
• In what subject areas did your tutoring revenues fall (or rise)?
• In what sales channels did revenues fall (or rise): Were there fewer (or more) referrals from professors or other students? Did the ad you ran result in fewer (or more) referrals this month than in the past months?
• Among what demographic groups did your revenues fall (or rise)—women or men, people with certain majors, or first-year, second-, third-, or fourth-year students?

The key is to look at all potential causes so as to narrow the parameters of the study to the information you actually need to make a good decision about how to fix your business if revenues have dropped or whether or not to expand it if your revenues have exploded.

The next task for the researcher is to put into writing the research objective. The research objective is the goal(s) the research is supposed to accomplish. The marketing research objective for your tutoring business might read as follows:

To survey college professors who teach 100- and 200-level math courses to determine why the number of students referred for tutoring dropped in the second semester.

This is admittedly a simple example designed to help you understand the basic concept. If you take a marketing research course, you will learn that research objectives get a lot more complicated than this. The following is an example:

“To gather information from a sample representative of the U.S. population among those who are ‘very likely’ to purchase an automobile within the next 6 months, which assesses preferences (measured on a 1–5 scale ranging from ‘very likely to buy’ to ‘not likely at all to buy’) for the model diesel at three different price levels. Such data would serve as input into a forecasting model that would forecast unit sales, by geographic regions of the country, for each combination of the model’s different prices and fuel configurations (Burns & Bush, 2010).”

Now do you understand why defining the problem is complicated and half the battle? Many a marketing research effort is doomed from the start because the problem was improperly defined. Coke’s ill-fated decision to change the formula of Coca-Cola in 1985 is a case in point: Pepsi had been creeping up on Coke in terms of market share over the years as well as running a successful promotional campaign called the “Pepsi Challenge,” in which consumers were encouraged to do a blind taste test to see if they agreed that Pepsi was better. Coke spent four years researching “the problem.” Indeed, people seemed to like the taste of Pepsi better in blind taste tests. Thus, the formula for Coke was changed. But the outcry among the public was so great that the new formula didn’t last long—a matter of months—before the old formula was reinstated. Some marketing experts believe Coke incorrectly defined the problem as “How can we beat Pepsi in taste tests?” instead of “How can we gain market share against Pepsi?” (Burns & Bush, 2010)

New Coke Is It! 1985

(click to see video)

This video documents the Coca-Cola Company’s ill-fated launch of New Coke in 1985.

1985 Pepsi Commercial—“They Changed My Coke”

This video shows how Pepsi tried to capitalize on the blunder.

Step 2: Design the Research

The next step in the marketing research process is to do a research design. The research design is your “plan of attack.” It outlines what data you are going to gather and from whom, how and when you will collect the data, and how you will analyze it once it’s been obtained. Let’s look at the data you’re going to gather first.

There are two basic types of data you can gather. The first is primary data. Primary data is information you collect yourself, using hands-on tools such as interviews or surveys, specifically for the research project you’re conducting. Secondary data is data that has already been collected by someone else, or data you have already collected for another purpose. Collecting primary data is more time consuming, work intensive, and expensive than collecting secondary data. Consequently, you should always try to collect secondary data first to solve your research problem, if you can. A great deal of research on a wide variety of topics already exists. If this research contains the answer to your question, there is no need for you to replicate it. Why reinvent the wheel?

Sources of Secondary Data

Your company’s internal records are a source of secondary data. So are any data you collect as part of your marketing intelligence gathering efforts. You can also purchase syndicated research. Syndicated research is primary data that marketing research firms collect on a regular basis and sell to other companies. J.D. Power & Associates is a provider of syndicated research. The company conducts independent, unbiased surveys of customer satisfaction, product quality, and buyer behavior for various industries. The company is best known for its research in the automobile sector. One of the best-known sellers of syndicated research is the Nielsen Company, which produces the Nielsen ratings. The Nielsen ratings measure the size of television, radio, and newspaper audiences in various markets. You have probably read or heard about TV shows that get the highest (Nielsen) ratings. (Arbitron does the same thing for radio ratings.) Nielsen, along with its main competitor, Information Resources, Inc. (IRI), also sells businesses scanner-based research . Scanner-based research is information collected by scanners at checkout stands in stores. Each week Nielsen and IRI collect information on the millions of purchases made at stores. The companies then compile the information and sell it to firms in various industries that subscribe to their services. The Nielsen Company has also recently teamed up with Facebook to collect marketing research information. Via Facebook, users will see surveys in some of the spaces in which they used to see online ads (Rappeport, Gelles, 2009).

By contrast, MarketResearch.com is an example of a marketing research aggregator. A marketing research aggregator is a marketing research company that doesn’t conduct its own research and sell it. Instead, it buys research reports from other marketing research companies and then sells the reports in their entirety or in pieces to other firms. Check out MarketResearch.com’s Web site. As you will see there are a huge number of studies in every category imaginable that you can buy for relatively small amounts of money.

Figure 10.7

Market research aggregators buy research reports from other marketing research companies and then resell them in part or in whole to other companies so they don’t have to gather primary data.

Source: http://www.marketresearch.com .

Your local library is a good place to gather free secondary data. It has searchable databases as well as handbooks, dictionaries, and books, some of which you can access online. Government agencies also collect and report information on demographics, economic and employment data, health information, and balance-of-trade statistics, among a lot of other information. The U.S. Census Bureau collects census data every ten years to gather information about who lives where. Basic demographic information about sex, age, race, and types of housing in which people live in each U.S. state, metropolitan area, and rural area is gathered so that population shifts can be tracked for various purposes, including determining the number of legislators each state should have in the U.S. House of Representatives. For the U.S. government, this is primary data. For marketing managers it is an important source of secondary data.

The Survey Research Center at the University of Michigan also conducts periodic surveys and publishes information about trends in the United States. One research study the center continually conducts is called the “Changing Lives of American Families” ( http://www.isr.umich.edu/home/news/research-update/2007-01.pdf ). This is important research data for marketing managers monitoring consumer trends in the marketplace. The World Bank and the United Nations are two international organizations that collect a great deal of information. Their Web sites contain many free research studies and data related to global markets. Table 10.1 “Examples of Primary Data Sources versus Secondary Data Sources” shows some examples of primary versus secondary data sources.

Table 10.1 Examples of Primary Data Sources versus Secondary Data Sources

Gauging the Quality of Secondary Data

When you are gathering secondary information, it’s always good to be a little skeptical of it. Sometimes studies are commissioned to produce the result a client wants to hear—or wants the public to hear. For example, throughout the twentieth century, numerous studies found that smoking was good for people’s health. The problem was the studies were commissioned by the tobacco industry. Web research can also pose certain hazards. There are many biased sites that try to fool people that they are providing good data. Often the data is favorable to the products they are trying to sell. Beware of product reviews as well. Unscrupulous sellers sometimes get online and create bogus ratings for products. See below for questions you can ask to help gauge the credibility of secondary information.

Gauging the Credibility of Secondary Data: Questions to Ask

• Who gathered this information?
• For what purpose?
• What does the person or organization that gathered the information have to gain by doing so?
• Was the information gathered and reported in a systematic manner?
• Is the source of the information accepted as an authority by other experts in the field?
• Does the article provide objective evidence to support the position presented?

Types of Research Design

Now let’s look specifically at the types of research designs that are utilized. By understanding different types of research designs, a researcher can solve a client’s problems more quickly and efficiently without jumping through more hoops than necessary. Research designs fall into one of the following three categories:

• Exploratory research design
• Descriptive research design
• Causal research design (experiments)

An exploratory research design is useful when you are initially investigating a problem but you haven’t defined it well enough to do an in-depth study of it. Perhaps via your regular market intelligence, you have spotted what appears to be a new opportunity in the marketplace. You would then do exploratory research to investigate it further and “get your feet wet,” as the saying goes. Exploratory research is less structured than other types of research, and secondary data is often utilized.

One form of exploratory research is qualitative research. Qualitative research is any form of research that includes gathering data that is not quantitative, and often involves exploring questions such as why as much as what or how much . Different forms, such as depth interviews and focus group interviews, are common in marketing research.

The depth interview —engaging in detailed, one-on-one, question-and-answer sessions with potential buyers—is an exploratory research technique. However, unlike surveys, the people being interviewed aren’t asked a series of standard questions. Instead the interviewer is armed with some general topics and asks questions that are open ended, meaning that they allow the interviewee to elaborate. “How did you feel about the product after you purchased it?” is an example of a question that might be asked. A depth interview also allows a researcher to ask logical follow-up questions such as “Can you tell me what you mean when you say you felt uncomfortable using the service?” or “Can you give me some examples?” to help dig further and shed additional light on the research problem. Depth interviews can be conducted in person or over the phone. The interviewer either takes notes or records the interview.

Focus groups and case studies are often utilized for exploratory research as well. A focus group is a group of potential buyers who are brought together to discuss a marketing research topic with one another. A moderator is used to focus the discussion, the sessions are recorded, and the main points of consensus are later summarized by the market researcher. Textbook publishers often gather groups of professors at educational conferences to participate in focus groups. However, focus groups can also be conducted on the telephone, in online chat rooms, or both, using meeting software like WebEx. The basic steps of conducting a focus group are outlined below.

The Basic Steps of Conducting a Focus Group

• Establish the objectives of the focus group. What is its purpose?
• Identify the people who will participate in the focus group. What makes them qualified to participate? How many of them will you need and what they will be paid?
• Obtain contact information for the participants and send out invitations (usually e-mails are most efficient).
• Develop a list of questions.
• Choose a facilitator.
• Choose a location in which to hold the focus group and the method by which it will be recorded.
• Conduct the focus group. If the focus group is not conducted electronically, include name tags for the participants, pens and notepads, any materials the participants need to see, and refreshments. Record participants’ responses.
• Summarize the notes from the focus group and write a report for management.

A case study looks at how another company solved the problem that’s being researched. Sometimes multiple cases, or companies, are used in a study. Case studies nonetheless have a mixed reputation. Some researchers believe it’s hard to generalize, or apply, the results of a case study to other companies. Nonetheless, collecting information about companies that encountered the same problems your firm is facing can give you a certain amount of insight about what direction you should take. In fact, one way to begin a research project is to carefully study a successful product or service.

Two other types of qualitative data used for exploratory research are ethnographies and projective techniques. In an ethnography , researchers interview, observe, and often videotape people while they work, live, shop, and play. The Walt Disney Company has recently begun using ethnographers to uncover the likes and dislikes of boys aged six to fourteen, a financially attractive market segment for Disney, but one in which the company has been losing market share. The ethnographers visit the homes of boys, observe the things they have in their rooms to get a sense of their hobbies, and accompany them and their mothers when they shop to see where they go, what the boys are interested in, and what they ultimately buy. (The children get seventy-five dollars out of the deal, incidentally.) (Barnes, 2009)

Projective techniques are used to reveal information research respondents might not reveal by being asked directly. Asking a person to complete sentences such as the following is one technique:

People who buy Coach handbags __________.

(Will he or she reply with “are cool,” “are affluent,” or “are pretentious,” for example?)

KFC’s grilled chicken is ______.

Or the person might be asked to finish a story that presents a certain scenario. Word associations are also used to discern people’s underlying attitudes toward goods and services. Using a word-association technique, a market researcher asks a person to say or write the first word that comes to his or her mind in response to another word. If the initial word is “fast food,” what word does the person associate it with or respond with? Is it “McDonald’s”? If many people reply that way, and you’re conducting research for Burger King, that could indicate Burger King has a problem. However, if the research is being conducted for Wendy’s, which recently began running an advertising campaign to the effect that Wendy’s offerings are “better than fast food,” it could indicate that the campaign is working.

Completing cartoons is yet another type of projective technique. It’s similar to finishing a sentence or story, only with the pictures. People are asked to look at a cartoon such as the one shown in Figure 10.8 “Example of a Cartoon-Completion Projective Technique” . One of the characters in the picture will have made a statement, and the person is asked to fill in the empty cartoon “bubble” with how they think the second character will respond.

Figure 10.8 Example of a Cartoon-Completion Projective Technique

In some cases, your research might end with exploratory research. Perhaps you have discovered your organization lacks the resources needed to produce the product. In other cases, you might decide you need more in-depth, quantitative research such as descriptive research or causal research, which are discussed next. Most marketing research professionals advise using both types of research, if it’s feasible. On the one hand, the qualitative-type research used in exploratory research is often considered too “lightweight.” Remember earlier in the chapter when we discussed telephone answering machines and the hit TV sitcom Seinfeld ? Both product ideas were initially rejected by focus groups. On the other hand, relying solely on quantitative information often results in market research that lacks ideas.

The Stone Wheel—What One Focus Group Said

Watch the video to see a funny spoof on the usefulness—or lack of usefulness—of focus groups.

Descriptive Research

Anything that can be observed and counted falls into the category of descriptive research design. A study using a descriptive research design involves gathering hard numbers, often via surveys, to describe or measure a phenomenon so as to answer the questions of who , what , where , when , and how . “On a scale of 1–5, how satisfied were you with your service?” is a question that illustrates the information a descriptive research design is supposed to capture.

Physiological measurements also fall into the category of descriptive design. Physiological measurements measure people’s involuntary physical responses to marketing stimuli, such as an advertisement. Elsewhere, we explained that researchers have gone so far as to scan the brains of consumers to see what they really think about products versus what they say about them. Eye tracking is another cutting-edge type of physiological measurement. It involves recording the movements of a person’s eyes when they look at some sort of stimulus, such as a banner ad or a Web page. The Walt Disney Company has a research facility in Austin, Texas, that it uses to take physical measurements of viewers when they see Disney programs and advertisements. The facility measures three types of responses: people’s heart rates, skin changes, and eye movements (eye tracking) (Spangler, 2009).

Figure 10.9

A woman shows off her headgear for an eye-tracking study. The gear’s not exactly a fashion statement but . . .

lawrencegs – Google Glass – CC BY 2.0.

A strictly descriptive research design instrument—a survey, for example—can tell you how satisfied your customers are. It can’t, however, tell you why. Nor can an eye-tracking study tell you why people’s eyes tend to dwell on certain types of banner ads—only that they do. To answer “why” questions an exploratory research design or causal research design is needed (Wagner, 2007).

Causal Research

Causal research design examines cause-and-effect relationships. Using a causal research design allows researchers to answer “what if” types of questions. In other words, if a firm changes X (say, a product’s price, design, placement, or advertising), what will happen to Y (say, sales or customer loyalty)? To conduct causal research, the researcher designs an experiment that “controls,” or holds constant, all of a product’s marketing elements except one (or using advanced techniques of research, a few elements can be studied at the same time). The one variable is changed, and the effect is then measured. Sometimes the experiments are conducted in a laboratory using a simulated setting designed to replicate the conditions buyers would experience. Or the experiments may be conducted in a virtual computer setting.

You might think setting up an experiment in a virtual world such as the online game Second Life would be a viable way to conduct controlled marketing research. Some companies have tried to use Second Life for this purpose, but the results have been somewhat mixed as to whether or not it is a good medium for marketing research. The German marketing research firm Komjuniti was one of the first “real-world” companies to set up an “island” in Second Life upon which it could conduct marketing research. However, with so many other attractive fantasy islands in which to play, the company found it difficult to get Second Life residents, or players, to voluntarily visit the island and stay long enough so meaningful research could be conducted. (Plus, the “residents,” or players, in Second Life have been known to protest corporations invading their world. When the German firm Komjuniti created an island in Second Life to conduct marketing research, the residents showed up waving signs and threatening to boycott the island.) (Wagner, 2007)

Why is being able to control the setting so important? Let’s say you are an American flag manufacturer and you are working with Walmart to conduct an experiment to see where in its stores American flags should be placed so as to increase their sales. Then the terrorist attacks of 9/11 occur. In the days afterward, sales skyrocketed—people bought flags no matter where they were displayed. Obviously, the terrorist attacks in the United States would have skewed the experiment’s data.

An experiment conducted in a natural setting such as a store is referred to as a field experiment . Companies sometimes do field experiments either because it is more convenient or because they want to see if buyers will behave the same way in the “real world” as in a laboratory or on a computer. The place the experiment is conducted or the demographic group of people the experiment is administered to is considered the test market . Before a large company rolls out a product to the entire marketplace, it will often place the offering in a test market to see how well it will be received. For example, to compete with MillerCoors’ sixty-four-calorie beer MGD 64, Anheuser-Busch recently began testing its Select 55 beer in certain cities around the country (McWilliams, 2009).

Figure 10.10

Select 55 beer: Coming soon to a test market near you? (If you’re on a diet, you have to hope so!)

Martine – Le champagne – CC BY-NC 2.0.

Many companies use experiments to test all of their marketing communications. For example, the online discount retailer O.co (formerly called Overstock.com) carefully tests all of its marketing offers and tracks the results of each one. One study the company conducted combined twenty-six different variables related to offers e-mailed to several thousand customers. The study resulted in a decision to send a group of e-mails to different segments. The company then tracked the results of the sales generated to see if they were in line with the earlier experiment it had conducted that led it to make the offer.

Step 3: Design the Data-Collection Forms

If the behavior of buyers is being formally observed, and a number of different researchers are conducting observations, the data obviously need to be recorded on a standardized data-collection form that’s either paper or electronic. Otherwise, the data collected will not be comparable. The items on the form could include a shopper’s sex; his or her approximate age; whether the person seemed hurried, moderately hurried, or unhurried; and whether or not he or she read the label on products, used coupons, and so forth.

The same is true when it comes to surveying people with questionnaires. Surveying people is one of the most commonly used techniques to collect quantitative data. Surveys are popular because they can be easily administered to large numbers of people fairly quickly. However, to produce the best results, the questionnaire for the survey needs to be carefully designed.

Questionnaire Design

Most questionnaires follow a similar format: They begin with an introduction describing what the study is for, followed by instructions for completing the questionnaire and, if necessary, returning it to the market researcher. The first few questions that appear on the questionnaire are usually basic, warm-up type of questions the respondent can readily answer, such as the respondent’s age, level of education, place of residence, and so forth. The warm-up questions are then followed by a logical progression of more detailed, in-depth questions that get to the heart of the question being researched. Lastly, the questionnaire wraps up with a statement that thanks the respondent for participating in the survey and information and explains when and how they will be paid for participating. To see some examples of questionnaires and how they are laid out, click on the following link: http://cas.uah.edu/wrenb/mkt343/Project/Sample%20Questionnaires.htm .

How the questions themselves are worded is extremely important. It’s human nature for respondents to want to provide the “correct” answers to the person administering the survey, so as to seem agreeable. Therefore, there is always a hazard that people will try to tell you what you want to hear on a survey. Consequently, care needs to be taken that the survey questions are written in an unbiased, neutral way. In other words, they shouldn’t lead a person taking the questionnaire to answer a question one way or another by virtue of the way you have worded it. The following is an example of a leading question.

Don’t you agree that teachers should be paid more ?

The questions also need to be clear and unambiguous. Consider the following question:

Which brand of toothpaste do you use ?

The question sounds clear enough, but is it really? What if the respondent recently switched brands? What if she uses Crest at home, but while away from home or traveling, she uses Colgate’s Wisp portable toothpaste-and-brush product? How will the respondent answer the question? Rewording the question as follows so it’s more specific will help make the question clearer:

Which brand of toothpaste have you used at home in the past six months? If you have used more than one brand, please list each of them 1 .

Sensitive questions have to be asked carefully. For example, asking a respondent, “Do you consider yourself a light, moderate, or heavy drinker?” can be tricky. Few people want to admit to being heavy drinkers. You can “soften” the question by including a range of answers, as the following example shows:

How many alcoholic beverages do you consume in a week ?

• __0–5 alcoholic beverages
• __5–10 alcoholic beverages
• __10–15 alcoholic beverages

Many people don’t like to answer questions about their income levels. Asking them to specify income ranges rather than divulge their actual incomes can help.

Other research question “don’ts” include using jargon and acronyms that could confuse people. “How often do you IM?” is an example. Also, don’t muddy the waters by asking two questions in the same question, something researchers refer to as a double-barreled question . “Do you think parents should spend more time with their children and/or their teachers?” is an example of a double-barreled question.

Open-ended questions , or questions that ask respondents to elaborate, can be included. However, they are harder to tabulate than closed-ended questions , or questions that limit a respondent’s answers. Multiple-choice and yes-and-no questions are examples of closed-ended questions.

Testing the Questionnaire

You have probably heard the phrase “garbage in, garbage out.” If the questions are bad, the information gathered will be bad, too. One way to make sure you don’t end up with garbage is to test the questionnaire before sending it out to find out if there are any problems with it. Is there enough space for people to elaborate on open-ended questions? Is the font readable? To test the questionnaire, marketing research professionals first administer it to a number of respondents face to face. This gives the respondents the chance to ask the researcher about questions or instructions that are unclear or don’t make sense to them. The researcher then administers the questionnaire to a small subset of respondents in the actual way the survey is going to be disseminated, whether it’s delivered via phone, in person, by mail, or online.

Getting people to participate and complete questionnaires can be difficult. If the questionnaire is too long or hard to read, many people won’t complete it. So, by all means, eliminate any questions that aren’t necessary. Of course, including some sort of monetary incentive for completing the survey can increase the number of completed questionnaires a market researcher will receive.

Step 4: Specify the Sample

Once you have created your questionnaire or other marketing study, how do you figure out who should participate in it? Obviously, you can’t survey or observe all potential buyers in the marketplace. Instead, you must choose a sample. A sample is a subset of potential buyers that are representative of your entire target market, or population being studied. Sometimes market researchers refer to the population as the universe to reflect the fact that it includes the entire target market, whether it consists of a million people, a hundred thousand, a few hundred, or a dozen. “All unmarried people over the age of eighteen who purchased Dirt Devil steam cleaners in the United States during 2011” is an example of a population that has been defined.

Obviously, the population has to be defined correctly. Otherwise, you will be studying the wrong group of people. Not defining the population correctly can result in flawed research, or sampling error. A sampling error is any type of marketing research mistake that results because a sample was utilized. One criticism of Internet surveys is that the people who take these surveys don’t really represent the overall population. On average, Internet survey takers tend to be more educated and tech savvy. Consequently, if they solely constitute your population, even if you screen them for certain criteria, the data you collect could end up being skewed.

The next step is to put together the sampling frame , which is the list from which the sample is drawn. The sampling frame can be put together using a directory, customer list, or membership roster (Wrenn et. al., 2007). Keep in mind that the sampling frame won’t perfectly match the population. Some people will be included on the list who shouldn’t be. Other people who should be included will be inadvertently omitted. It’s no different than if you were to conduct a survey of, say, 25 percent of your friends, using friends’ names you have in your cell phone. Most of your friends’ names are likely to be programmed into your phone, but not all of them. As a result, a certain degree of sampling error always occurs.

There are two main categories of samples in terms of how they are drawn: probability samples and nonprobability samples. A probability sample is one in which each would-be participant has a known and equal chance of being selected. The chance is known because the total number of people in the sampling frame is known. For example, if every other person from the sampling frame were chosen, each person would have a 50 percent chance of being selected.

A nonprobability sample is any type of sample that’s not drawn in a systematic way. So the chances of each would-be participant being selected can’t be known. A convenience sample is one type of nonprobability sample. It is a sample a researcher draws because it’s readily available and convenient to do so. Surveying people on the street as they pass by is an example of a convenience sample. The question is, are these people representative of the target market?

For example, suppose a grocery store needed to quickly conduct some research on shoppers to get ready for an upcoming promotion. Now suppose that the researcher assigned to the project showed up between the hours of 10 a.m. and 12 p.m. on a weekday and surveyed as many shoppers as possible. The problem is that the shoppers wouldn’t be representative of the store’s entire target market. What about commuters who stop at the store before and after work? Their views wouldn’t be represented. Neither would people who work the night shift or shop at odd hours. As a result, there would be a lot of room for sampling error in this study. For this reason, studies that use nonprobability samples aren’t considered as accurate as studies that use probability samples. Nonprobability samples are more often used in exploratory research.

Lastly, the size of the sample has an effect on the amount of sampling error. Larger samples generally produce more accurate results. The larger your sample is, the more data you will have, which will give you a more complete picture of what you’re studying. However, the more people surveyed or studied, the more costly the research becomes.

Statistics can be used to determine a sample’s optimal size. If you take a marketing research or statistics class, you will learn more about how to determine the optimal size.

Of course, if you hire a marketing research company, much of this work will be taken care of for you. Many marketing research companies, like ResearchNow, maintain panels of prescreened people they draw upon for samples. In addition, the marketing research firm will be responsible for collecting the data or contracting with a company that specializes in data collection. Data collection is discussed next.

Step 5: Collect the Data

As we have explained, primary marketing research data can be gathered in a number of ways. Surveys, taking physical measurements, and observing people are just three of the ways we discussed. If you’re observing customers as part of gathering the data, keep in mind that if shoppers are aware of the fact, it can have an effect on their behavior. For example, if a customer shopping for feminine hygiene products in a supermarket aisle realizes she is being watched, she could become embarrassed and leave the aisle, which would adversely affect your data. To get around problems such as these, some companies set up cameras or two-way mirrors to observe customers. Organizations also hire mystery shoppers to work around the problem. A mystery shopper is someone who is paid to shop at a firm’s establishment or one of its competitors to observe the level of service, cleanliness of the facility, and so forth, and report his or her findings to the firm.

Make Extra Money as a Mystery Shopper

Watch the YouTube video to get an idea of how mystery shopping works.

Survey data can be collected in many different ways and combinations of ways. The following are the basic methods used:

• Face-to-face (can be computer aided)
• Telephone (can be computer aided or completely automated)
• Mail and hand delivery
• E-mail and the Web

A face-to-face survey is, of course, administered by a person. The surveys are conducted in public places such as in shopping malls, on the street, or in people’s homes if they have agreed to it. In years past, it was common for researchers in the United States to knock on people’s doors to gather survey data. However, randomly collected door-to-door interviews are less common today, partly because people are afraid of crime and are reluctant to give information to strangers (McDaniel & Gates, 1998).

Nonetheless, “beating the streets” is still a legitimate way questionnaire data is collected. When the U.S. Census Bureau collects data on the nation’s population, it hand delivers questionnaires to rural households that do not have street-name and house-number addresses. And Census Bureau workers personally survey the homeless to collect information about their numbers. Face-to-face surveys are also commonly used in third world countries to collect information from people who cannot read or lack phones and computers.

A plus of face-to-face surveys is that they allow researchers to ask lengthier, more complex questions because the people being surveyed can see and read the questionnaires. The same is true when a computer is utilized. For example, the researcher might ask the respondent to look at a list of ten retail stores and rank the stores from best to worst. The same question wouldn’t work so well over the telephone because the person couldn’t see the list. The question would have to be rewritten. Another drawback with telephone surveys is that even though federal and state “do not call” laws generally don’t prohibit companies from gathering survey information over the phone, people often screen such calls using answering machines and caller ID.

Probably the biggest drawback of both surveys conducted face-to-face and administered over the phone by a person is that they are labor intensive and therefore costly. Mailing out questionnaires is costly, too, and the response rates can be rather low. Think about why that might be so: if you receive a questionnaire in the mail, it is easy to throw it in the trash; it’s harder to tell a market researcher who approaches you on the street that you don’t want to be interviewed.

By contrast, gathering survey data collected by a computer, either over the telephone or on the Internet, can be very cost-effective and in some cases free. SurveyMonkey and Zoomerang are two Web sites that will allow you to create online questionnaires, e-mail them to up to one hundred people for free, and view the responses in real time as they come in. For larger surveys, you have to pay a subscription price of a few hundred dollars. But that still can be extremely cost-effective. The two Web sites also have a host of other features such as online-survey templates you can use to create your questionnaire, a way to set up automatic reminders sent to people who haven’t yet completed their surveys, and tools you can use to create graphics to put in your final research report. To see how easy it is to put together a survey in SurveyMonkey, click on the following link: http://help.surveymonkey.com/app/tutorials/detail/a_id/423 .

Like a face-to-face survey, an Internet survey can enable you to show buyers different visuals such as ads, pictures, and videos of products and their packaging. Web surveys are also fast, which is a major plus. Whereas face-to-face and mailed surveys often take weeks to collect, you can conduct a Web survey in a matter of days or even hours. And, of course, because the information is electronically gathered it can be automatically tabulated. You can also potentially reach a broader geographic group than you could if you had to personally interview people. The Zoomerang Web site allows you to create surveys in forty different languages.

Another plus for Web and computer surveys (and electronic phone surveys) is that there is less room for human error because the surveys are administered electronically. For instance, there’s no risk that the interviewer will ask a question wrong or use a tone of voice that could mislead the respondents. Respondents are also likely to feel more comfortable inputting the information into a computer if a question is sensitive than they would divulging the information to another person face-to-face or over the phone. Given all of these advantages, it’s not surprising that the Internet is quickly becoming the top way to collect primary data. However, like mail surveys, surveys sent to people over the Internet are easy to ignore.

Lastly, before the data collection process begins, the surveyors and observers need to be trained to look for the same things, ask questions the same way, and so forth. If they are using rankings or rating scales, they need to be “on the same page,” so to speak, as to what constitutes a high ranking or a low ranking. As an analogy, you have probably had some teachers grade your college papers harder than others. The goal of training is to avoid a wide disparity between how different observers and interviewers record the data.

Figure 10.11

Training people so they know what constitutes different ratings when they are collecting data will improve the quality of the information gathered in a marketing research study.

Ricardo Rodriquez – Satisfaction survey – CC BY-NC-ND 2.0.

For example, if an observation form asks the observers to describe whether a shopper’s behavior is hurried, moderately hurried, or unhurried, they should be given an idea of what defines each rating. Does it depend on how much time the person spends in the store or in the individual aisles? How fast they walk? In other words, the criteria and ratings need to be spelled out.

Collecting International Marketing Research Data

Gathering marketing research data in foreign countries poses special challenges. However, that doesn’t stop firms from doing so. Marketing research companies are located all across the globe, in fact. Eight of the ten largest marketing research companies in the world are headquartered in the United States. However, five of these eight firms earn more of their revenues abroad than they do in the United States. There’s a reason for this: many U.S. markets were saturated, or tapped out, long ago in terms of the amount that they can grow. Coke is an example. As you learned earlier in the book, most of the Coca-Cola Company’s revenues are earned in markets abroad. To be sure, the United States is still a huge market when it comes to the revenues marketing research firms generate by conducting research in the country: in terms of their spending, American consumers fuel the world’s economic engine. Still, emerging countries with growing middle classes, such as China, India, and Brazil, are hot new markets companies want to tap.

What kind of challenges do firms face when trying to conduct marketing research abroad? As we explained, face-to-face surveys are commonly used in third world countries to collect information from people who cannot read or lack phones and computers. However, face-to-face surveys are also common in Europe, despite the fact that phones and computers are readily available. In-home surveys are also common in parts of Europe. By contrast, in some countries, including many Asian countries, it’s considered taboo or rude to try to gather information from strangers either face-to-face or over the phone. In many Muslim countries, women are forbidden to talk to strangers.

And how do you figure out whom to research in foreign countries? That in itself is a problem. In the United States, researchers often ask if they can talk to the heads of households to conduct marketing research. But in countries in which domestic servants or employees are common, the heads of households aren’t necessarily the principal shoppers; their domestic employees are (Malhotra).

Translating surveys is also an issue. Have you ever watched the TV comedians Jay Leno and David Letterman make fun of the English translations found on ethnic menus and products? Research tools such as surveys can suffer from the same problem. Hiring someone who is bilingual to translate a survey into another language can be a disaster if the person isn’t a native speaker of the language to which the survey is being translated.

One way companies try to deal with translation problems is by using back translation. When back translation is used, a native speaker translates the survey into the foreign language and then translates it back again to the original language to determine if there were gaps in meaning—that is, if anything was lost in translation. And it’s not just the language that’s an issue. If the research involves any visual images, they, too, could be a point of confusion. Certain colors, shapes, and symbols can have negative connotations in other countries. For example, the color white represents purity in many Western cultures, but in China, it is the color of death and mourning (Zouhali-Worrall, 2008). Also, look back at the cartoon-completion exercise in Figure 10.8 “Example of a Cartoon-Completion Projective Technique” . What would women in Muslim countries who aren’t allowed to converse with male sellers think of it? Chances are, the cartoon wouldn’t provide you with the information you’re seeking if Muslim women in some countries were asked to complete it.

One way marketing research companies are dealing with the complexities of global research is by merging with or acquiring marketing research companies abroad. The Nielsen Company is the largest marketing research company in the world. The firm operates in more than a hundred countries and employs more than forty thousand people. Many of its expansions have been the result of acquisitions and mergers.

Step 6: Analyze the Data

Step 6 involves analyzing the data to ensure it’s as accurate as possible. If the research is collected by hand using a pen and pencil, it’s entered into a computer. Or respondents might have already entered the information directly into a computer. For example, when Toyota goes to an event such as a car show, the automaker’s marketing personnel ask would-be buyers to complete questionnaires directly on computers. Companies are also beginning to experiment with software that can be used to collect data using mobile phones.

Once all the data is collected, the researchers begin the data cleaning , which is the process of removing data that have accidentally been duplicated (entered twice into the computer) or correcting data that have obviously been recorded wrong. A program such as Microsoft Excel or a statistical program such as Predictive Analytics Software (PASW, which was formerly known as SPSS) is then used to tabulate, or calculate, the basic results of the research, such as the total number of participants and how collectively they answered various questions. The programs can also be used to calculate averages, such as the average age of respondents, their average satisfaction, and so forth. The same can done for percentages, and other values you learned about, or will learn about, in a statistics course, such as the standard deviation, mean, and median for each question.

The information generated by the programs can be used to draw conclusions, such as what all customers might like or not like about an offering based on what the sample group liked or did not like. The information can also be used to spot differences among groups of people. For example, the research might show that people in one area of the country like the product better than people in another area. Trends to predict what might happen in the future can also be spotted.

If there are any open-ended questions respondents have elaborated upon—for example, “Explain why you like the current brand you use better than any other brand”—the answers to each are pasted together, one on top of another, so researchers can compare and summarize the information. As we have explained, qualitative information such as this can give you a fuller picture of the results of the research.

Part of analyzing the data is to see if it seems sound. Does the way in which the research was conducted seem sound? Was the sample size large enough? Are the conclusions that become apparent from it reasonable?

The two most commonly used criteria used to test the soundness of a study are (1) validity and (2) reliability. A study is valid if it actually tested what it was designed to test. For example, did the experiment you ran in Second Life test what it was designed to test? Did it reflect what could really happen in the real world? If not, the research isn’t valid. If you were to repeat the study, and get the same results (or nearly the same results), the research is said to be reliable . If you get a drastically different result if you repeat the study, it’s not reliable. The data collected, or at least some it, can also be compared to, or reconciled with, similar data from other sources either gathered by your firm or by another organization to see if the information seems on target.

Stage 7: Write the Research Report and Present Its Findings

If you end up becoming a marketing professional and conducting a research study after you graduate, hopefully you will do a great job putting the study together. You will have defined the problem correctly, chosen the right sample, collected the data accurately, analyzed it, and your findings will be sound. At that point, you will be required to write the research report and perhaps present it to an audience of decision makers. You will do so via a written report and, in some cases, a slide or PowerPoint presentation based on your written report.

The six basic elements of a research report are as follows.

• Title Page . The title page explains what the report is about, when it was conducted and by whom, and who requested it.
• Table of Contents . The table of contents outlines the major parts of the report, as well as any graphs and charts, and the page numbers on which they can be found.
• Executive Summary . The executive summary summarizes all the details in the report in a very quick way. Many people who receive the report—both executives and nonexecutives—won’t have time to read the entire report. Instead, they will rely on the executive summary to quickly get an idea of the study’s results and what to do about those results.

Methodology and Limitations . The methodology section of the report explains the technical details of how the research was designed and conducted. The section explains, for example, how the data was collected and by whom, the size of the sample, how it was chosen, and whom or what it consisted of (e.g., the number of women versus men or children versus adults). It also includes information about the statistical techniques used to analyze the data.

Every study has errors—sampling errors, interviewer errors, and so forth. The methodology section should explain these details, so decision makers can consider their overall impact. The margin of error is the overall tendency of the study to be off kilter—that is, how far it could have gone wrong in either direction. Remember how newscasters present the presidential polls before an election? They always say, “This candidate is ahead 48 to 44 percent, plus or minus 2 percent.” That “plus or minus” is the margin of error. The larger the margin of error is, the less likely the results of the study are accurate. The margin of error needs to be included in the methodology section.

• Findings . The findings section is a longer, fleshed-out version of the executive summary that goes into more detail about the statistics uncovered by the research that bolster the study’s findings. If you have related research or secondary data on hand that back up the findings, it can be included to help show the study did what it was designed to do.
• Recommendations . The recommendations section should outline the course of action you think should be taken based on the findings of the research and the purpose of the project. For example, if you conducted a global market research study to identify new locations for stores, make a recommendation for the locations (Mersdorf, 2009).

As we have said, these are the basic sections of a marketing research report. However, additional sections can be added as needed. For example, you might need to add a section on the competition and each firm’s market share. If you’re trying to decide on different supply chain options, you will need to include a section on that topic.

As you write the research report, keep your audience in mind. Don’t use technical jargon decision makers and other people reading the report won’t understand. If technical terms must be used, explain them. Also, proofread the document to ferret out any grammatical errors and typos, and ask a couple of other people to proofread behind you to catch any mistakes you might have missed. If your research report is riddled with errors, its credibility will be undermined, even if the findings and recommendations you make are extremely accurate.

Many research reports are presented via PowerPoint. If you’re asked to create a slideshow presentation from the report, don’t try to include every detail in the report on the slides. The information will be too long and tedious for people attending the presentation to read through. And if they do go to the trouble of reading all the information, they probably won’t be listening to the speaker who is making the presentation.

Instead of including all the information from the study in the slides, boil each section of the report down to key points and add some “talking points” only the presenter will see. After or during the presentation, you can give the attendees the longer, paper version of the report so they can read the details at a convenient time, if they choose to.

Key Takeaway

Step 1 in the marketing research process is to define the problem. Businesses take a look at what they believe are symptoms and try to drill down to the potential causes so as to precisely define the problem. The next task for the researcher is to put into writing the research objective, or goal, the research is supposed to accomplish. Step 2 in the process is to design the research. The research design is the “plan of attack.” It outlines what data you are going to gather, from whom, how, and when, and how you’re going to analyze it once it has been obtained. Step 3 is to design the data-collection forms, which need to be standardized so the information gathered on each is comparable. Surveys are a popular way to gather data because they can be easily administered to large numbers of people fairly quickly. However, to produce the best results, survey questionnaires need to be carefully designed and pretested before they are used. Step 4 is drawing the sample, or a subset of potential buyers who are representative of your entire target market. If the sample is not correctly selected, the research will be flawed. Step 5 is to actually collect the data, whether it’s collected by a person face-to-face, over the phone, or with the help of computers or the Internet. The data-collection process is often different in foreign countries. Step 6 is to analyze the data collected for any obvious errors, tabulate the data, and then draw conclusions from it based on the results. The last step in the process, Step 7, is writing the research report and presenting the findings to decision makers.

Review Questions

• Explain why it’s important to carefully define the problem or opportunity a marketing research study is designed to investigate.
• Describe the different types of problems that can occur when marketing research professionals develop questions for surveys.
• How does a probability sample differ from a nonprobability sample?
• What makes a marketing research study valid? What makes a marketing research study reliable?
• What sections should be included in a marketing research report? What is each section designed to do?

1 “Questionnaire Design,” QuickMBA , http://www.quickmba.com/marketing/research/qdesign (accessed December 14, 2009).

Barnes, B., “Disney Expert Uses Science to Draw Boy Viewers,” New York Times , April 15, 2009, http://www.nytimes.com/2009/04/14/arts/television/14boys.html?pagewanted=1&_r=1 (accessed December 14, 2009).

Burns A. and Ronald Bush, Marketing Research , 6th ed. (Upper Saddle River, NJ: Prentice Hall, 2010), 85.

Malhotra, N., Marketing Research: An Applied Approach , 6th ed. (Upper Saddle River, NJ: Prentice Hall), 764.

McDaniel, C. D. and Roger H. Gates, Marketing Research Essentials , 2nd ed. (Cincinnati: South-Western College Publishing, 1998), 61.

McWilliams, J., “A-B Puts Super-Low-Calorie Beer in Ring with Miller,” St. Louis Post-Dispatch , August 16, 2009, http://www.stltoday.com/business/next-matchup-light-weights-a-b-puts-super-low-calorie/article_47511bfe-18ca-5979-bdb9-0526c97d4edf.html (accessed April 13, 2012).

Mersdorf, S., “How to Organize Your Next Survey Report,” Cvent , August 24, 2009, http://survey.cvent.com/blog/cvent-survey/0/0/how-to-organize-your-next-survey-report (accessed December 14, 2009).

Rappeport A. and David Gelles, “Facebook to Form Alliance with Nielsen,” Financial Times , September 23, 2009, 16.

Spangler, T., “Disney Lab Tracks Feelings,” Multichannel News 30, no. 30 (August 3, 2009): 26.

Wagner, J., “Marketing in Second Life Doesn’t Work…Here Is Why!” GigaOM , April 4, 2007, http://gigaom.com/2007/04/04/3-reasons-why-marketing-in-second-life-doesnt-work (accessed December 14, 2009).

Wrenn, B., Robert E. Stevens, and David L. Loudon, Marketing Research: Text and Cases , 2nd ed. (Binghamton, NY: Haworth Press, 2007), 180.

Zouhali-Worrall, M., “Found in Translation: Avoiding Multilingual Gaffes,” CNNMoney.com , July 14, 2008, http://money.cnn.com/2008/07/07/smallbusiness/language_translation.fsb/index.htm (accessed December 14, 2009).

Principles of Marketing Copyright © 2015 by University of Minnesota is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

Marketing Research

What is marketing research.

Marketing research is the systematic process of collecting, analyzing, and interpreting data about a market, product, or service to inform marketing decisions and strategies. It involves understanding customer needs, preferences, and behaviors, as well as assessing market trends, competition, and potential opportunities.

Common Terms and Definitions

Primary Research : Original research conducted directly by a company or its representatives to gather specific information about a market, product, or service.

Secondary Research : Research that relies on existing data and information from external sources, such as government reports, industry publications, or online databases.

Qualitative Research : Research that focuses on gathering non-numerical data, such as opinions, attitudes, and motivations, through methods like interviews, focus groups, and observations.

Quantitative Research : Research that involves collecting and analyzing numerical data to measure and quantify market trends, consumer behaviors, and other variables.

Sample : A subset of a larger population selected for research purposes to represent the characteristics and opinions of the entire group.

Survey : A research method that involves collecting data from a sample of respondents through a structured questionnaire or interview.

Types of Marketing Research

Market Segmentation Research : Research that aims to identify and profile distinct groups of consumers with similar needs, preferences, or behaviors.

Product Research : Research conducted to assess the viability, features, and potential success of a new or existing product.

Pricing Research : Research that focuses on determining the optimal price point for a product or service based on factors like consumer willingness to pay, competition, and production costs.

Advertising Research : Research that evaluates the effectiveness of advertising campaigns, messages, and media channels in reaching and influencing target audiences.

Brand Research : Research that assesses consumer perceptions, associations, and loyalty towards a brand, as well as its positioning relative to competitors.

The Marketing Research Process

• Define the research problem and objectives.
• Develop a research plan and design.
• Collect primary and/or secondary data.
• Analyze and interpret the data.
• Present findings and recommendations.
• Implement and evaluate marketing decisions based on research insights.

Common Questions and Answers

What is the difference between primary and secondary research?

Primary research involves collecting original data directly from sources, while secondary research relies on existing data and information from external sources. Primary research is more tailored to specific research objectives but can be time-consuming and expensive, whereas secondary research is often more cost-effective and efficient but may not fully address unique research needs.

When should a company conduct marketing research?

Companies should conduct marketing research when they need to make informed decisions about entering new markets, developing or improving products, setting prices, designing advertising campaigns, or understanding changes in consumer behavior and market trends. Regular research helps businesses stay competitive and responsive to evolving customer needs and market conditions.

How can marketing research help a company gain a competitive advantage?

Marketing research provides valuable insights into consumer preferences, market trends, and competitor strategies, enabling companies to identify unmet needs, develop targeted products and campaigns, and differentiate themselves in the market. By making data-driven decisions based on research findings, companies can optimize their marketing efforts, improve customer satisfaction, and ultimately gain a competitive edge.

Marketing research is a critical tool for businesses to understand their customers, markets, and competition, and make informed decisions that drive growth and profitability. By mastering the key concepts, types, and processes of marketing research, you will be well-equipped to gather and analyze data, generate valuable insights, and develop effective marketing strategies in your career.

Marketing Strategy and Planning

Integrated marketing communications, marketing analytics and metrics, ethical and sustainable marketing, international marketing, brand management.

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur.

Block quote

Ordered list

Unordered list

Superscript

Breadcrumbs Section. Click here to navigate to respective pages.

Establishing validity and reliability in case study research projects

DOI link for Establishing validity and reliability in case study research projects

Click here to navigate to parent product.

This chapter investigates in depth the issues of establishing reliability and validity in case study approaches to marketing research. IT will be particularly valuable to postgraduates in marketing research who are using the case study approach.

It explores the difference between reliability (replicability of results) and validity (match to the research aim).

The tests of validity, credibility and reliability in case studies investigated include construct validity, internal validity, external validity, reliability, credibility, transferability, confirmability and dependability.

It identifies that problems with validity and reliability cannot be removed completely, but their negative effects can be reduced by using tests described in the chapter.

• Privacy Policy
• Terms & Conditions
• Cookie Policy
• Taylor & Francis Online
• Taylor & Francis Group
• Students/Researchers
• Librarians/Institutions

Connect with us

Registered in England & Wales No. 3099067 5 Howick Place | London | SW1P 1WG © 2024 Informa UK Limited

Marketing Research and Market Intelligence

Steps in the marketing research process, review questions.

• Explain why it's important to carefully define the problem or opportunity a marketing research study is designed to investigate.
• Describe the different types of problems that can occur when marketing research professionals develop questions for surveys.
• How does a probability sample differ from a nonprobability sample?
• What makes a marketing research study valid? What makes a marketing research study reliable?
• What sections should be included in a marketing research report? What is each section designed to do?

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Write for Us
• BMJ Journals

You are here

• Volume 18, Issue 2
• Issues of validity and reliability in qualitative research
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Helen Noble 1 ,
• Joanna Smith 2
• 1 School of Nursing and Midwifery, Queens's University Belfast , Belfast , UK
• 2 School of Human and Health Sciences, University of Huddersfield , Huddersfield , UK
• Correspondence to Dr Helen Noble School of Nursing and Midwifery, Queens's University Belfast, Medical Biology Centre, 97 Lisburn Rd, Belfast BT9 7BL, UK; helen.noble{at}qub.ac.uk

https://doi.org/10.1136/eb-2015-102054

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Evaluating the quality of research is essential if findings are to be utilised in practice and incorporated into care delivery. In a previous article we explored ‘bias’ across research designs and outlined strategies to minimise bias. 1 The aim of this article is to further outline rigour, or the integrity in which a study is conducted, and ensure the credibility of findings in relation to qualitative research. Concepts such as reliability, validity and generalisability typically associated with quantitative research and alternative terminology will be compared in relation to their application to qualitative research. In addition, some of the strategies adopted by qualitative researchers to enhance the credibility of their research are outlined.

Are the terms reliability and validity relevant to ensuring credibility in qualitative research?

Although the tests and measures used to establish the validity and reliability of quantitative research cannot be applied to qualitative research, there are ongoing debates about whether terms such as validity, reliability and generalisability are appropriate to evaluate qualitative research. 2–4 In the broadest context these terms are applicable, with validity referring to the integrity and application of the methods undertaken and the precision in which the findings accurately reflect the data, while reliability describes consistency within the employed analytical procedures. 4 However, if qualitative methods are inherently different from quantitative methods in terms of philosophical positions and purpose, then alterative frameworks for establishing rigour are appropriate. 3 Lincoln and Guba 5 offer alternative criteria for demonstrating rigour within qualitative research namely truth value, consistency and neutrality and applicability. Table 1 outlines the differences in terminology and criteria used to evaluate qualitative research.

• View inline

Terminology and criteria used to evaluate the credibility of research findings

What strategies can qualitative researchers adopt to ensure the credibility of the study findings?

Unlike quantitative researchers, who apply statistical methods for establishing validity and reliability of research findings, qualitative researchers aim to design and incorporate methodological strategies to ensure the ‘trustworthiness’ of the findings. Such strategies include:

Accounting for personal biases which may have influenced findings; 6

Acknowledging biases in sampling and ongoing critical reflection of methods to ensure sufficient depth and relevance of data collection and analysis; 3

Meticulous record keeping, demonstrating a clear decision trail and ensuring interpretations of data are consistent and transparent; 3 , 4

Establishing a comparison case/seeking out similarities and differences across accounts to ensure different perspectives are represented; 6 , 7

Including rich and thick verbatim descriptions of participants’ accounts to support findings; 7

Demonstrating clarity in terms of thought processes during data analysis and subsequent interpretations 3 ;

Engaging with other researchers to reduce research bias; 3

Respondent validation: includes inviting participants to comment on the interview transcript and whether the final themes and concepts created adequately reflect the phenomena being investigated; 4

Data triangulation, 3 , 4 whereby different methods and perspectives help produce a more comprehensive set of findings. 8 , 9

Table 2 provides some specific examples of how some of these strategies were utilised to ensure rigour in a study that explored the impact of being a family carer to patients with stage 5 chronic kidney disease managed without dialysis. 10

Strategies for enhancing the credibility of qualitative research

In summary, it is imperative that all qualitative researchers incorporate strategies to enhance the credibility of a study during research design and implementation. Although there is no universally accepted terminology and criteria used to evaluate qualitative research, we have briefly outlined some of the strategies that can enhance the credibility of study findings.

• Sandelowski M
• Lincoln YS ,
• Barrett M ,
• Mayan M , et al
• Greenhalgh T
• Lingard L ,

Twitter Follow Joanna Smith at @josmith175 and Helen Noble at @helnoble

Competing interests None.

Read the full text or download the PDF:

• Marketplace
• Future Proof

5 reasons why marketing research is important to your business

Marketing research is a critical tool that businesses and organisations use to gather valuable information about their target market and industry. By conducting research, companies can gain insights into customer needs, preferences, and behaviour, as well as market trends and the competitive landscape. In this article, we'll explore five reasons why marketing research is important for businesses and organisations.

5 reasons why marketing research is important

Is your company on the fence about investing in marketing research? If so, read on for five reasons why marketing research is important to the viability and longevity of your business.

1. Marketing research helps businesses understand their customers.

Knowing your customers is the first step in running a successful business. And while this may sound simple enough, it can be difficult to look inside the consumer psyche—one whose needs, preferences, and behaviours are ever-evolving.

Fortunately, marketing research can give you a peek behind the curtain, helping you understand what motivates current as well as potential customers. A marketing research survey , for example, can provide insight into pricing sensitivity among a segment of your target population. The data yielded may suggest that boomers are more likely to purchase a product based on price alone while millennials are more motivated by the values of the respective company. Based on this information, you can tailor your offerings to better meet these needs.

Let’s take a closer look at what else marketing research can tell you about your target audience.

a. Marketing research reveals demographic data.

Through conducting surveys, you can better understand characteristics of your buyers, like age, gender, race, and ethnicity. You can also collect socioeconomic information like education, income, and employment status. These variables can help you engage in audience segmentation, which essentially involves tweaking your brand messaging to appeal to various subgroups.

b. Marketing research illustrates purchasing behaviour.

Marketing research tools like surveys, transaction tracking, and social media tracking can also shed light on when, why, how, and what consumers are buying. A better understanding of purchasing behaviour can help your company develop brand messaging that strikes a chord with its target audience. If consumers tend to shop between the hours of 5 and 8 p.m., for instance, your company may send an email coupon at 4:45 p.m. to prime that behaviour.

c. Marketing research helps companies appeal to potential customers.

It’s important to learn as much as possible about your current customers. However, it’s equally important to learn about people who are at the beginning of the customer journey. A deeper understanding of what motivates these individuals can help your business turn clicks into conversions.

2. Marketing research helps businesses manage risk.

Risk is an intrinsic part of the business world. However, by understanding market trends and customer behaviour, businesses can anticipate changes in demand and adjust their strategies accordingly. Market research can also help businesses identify potential obstacles or challenges and develop contingency plans to mitigate those risks.

For example, let’s say a company hopes to introduce a new product to the market. Before investing significant time and financial resources into product development, market researchers may conduct a series of surveys to determine if consumers are even interested in the respective product. This mitigates the risk of introducing a good for which there is no demand.

Other examples in which marketing research can be used as a tool for mitigating risk include:

• Conducting ad testing to determine if brand messaging resonates with consumers
• Hosting product development focus groups to receive feedback before revealing a new product to the larger market
• Organising customer satisfaction surveys at various points of the customer journey to determine any issues affecting sales
• Administering price testing to determine the ideal price for a new product

3. Marketing research helps businesses stay competitive.

Marketing research gives companies a competitive edge in many ways, one being a greater awareness of the industry itself. By understanding what other businesses are doing, your company can develop strategies to differentiate itself and stay ahead of the curve. For example, if research indicates that laundry detergent manufacturers are increasing prices, a company can either decide to a) keep up with the times and increase costs, b) keep costs the same, or c) decrease pricing to appeal to a specific target market.

Marketing research also helps companies determine areas where they may be falling behind. If several healthcare conglomerates are expanding telehealth services, it would be wise for a hospital to also make moves to provide digital appointments. Otherwise, patients may take their business elsewhere.

How else can marketing research help your business stay competitive?

a. Marketing research aids in trendspotting.

It’s important for companies to keep up with industry patterns. However, it’s also important for companies to predict patterns before they even happen. The fashion industry provides a good example of this. Trend forecasters analyse decades—centuries even—of socio-cultural data in an attempt to be the first company to put out a ‘hot’ new product that appeals to modern sensibilities.

b. Marketing research allows businesses to identify gaps in the industry.

Similarly, marketing research can help companies realise a demand that has yet to be satisfied. Through marketing research surveys, for instance, an organic snack food company may see that consumers are craving more savoury gluten-free snack options. Using this data, the company can one-up competitors by introducing a new product.

4. Marketing research gives businesses the tools to measure success.

Net profit is an important indicator of a company’s success. However, if profits are plummeting, it can be difficult to know if this is because of your company’s performance or industry-wide factors like increased materials costs and decreased demand. Because of this, businesses must rely on a more holistic approach to evaluating success.

Fortunately, marketing research can be a helpful tool if you want to measure business health using more than dollar signs. Here’s how marketing research can help you gauge company success.

a. Marketing research provides insight into customer satisfaction.

Sure, your company could be making money. But if your customers aren’t satisfied, your financial gains are likely to be short-lived. To determine if customers are happy with your products or services, you can conduct a customer feedback survey questionnaire.

Alternatively, you may choose to conduct a longitudinal panel survey . Unlike an ad hoc survey, which takes a snapshot of consumer opinions and behaviours, longitudinal studies (like monthly or quarterly trackers) monitor how consumer attitudes change over time. This tool is especially helpful if you want to see how customer satisfaction changes after the introduction of new protocols or training methods.

b. Marketing research helps businesses assess employee satisfaction.

A financially profitable company isn’t necessarily successful if its employees aren’t satisfied. To assess this metric of success, you can conduct surveys, in-person interviews, or focus groups. The results from these marketing research assessments may also shed light on the efficiency of your staff.

c. Marketing research provides insight into the performance of competitors.

It can be constructive to compare your company’s performance to that of competitors. This can help you understand if dips or rises in profit are unique to your company or simply an industry trend.

5. Marketing research helps businesses make informed decisions.

For decades, companies have made key business decisions based on gut feelings alone. But the new age of marketing revolves around data-driven decision-making: a process in which business decisions are informed by metrics and analytics. This contemporary model helps companies make solid choices that reduce risk while improving performance.

Since marketing research lends itself to data generation, it’s an integral part of this equation. For example, data yielded from a brand awareness survey may illustrate that men aged 50 to 65 in a specific geographic area are the least familiar with the brand. Using this information, the company can invest marketing dollars in a way that will yield true results.

Other examples of using marketing research to support data-driven decision-making include:

• During a focus group, a company discovers that 60% of women are dissatisfied with the playback speed options of an audiobook app. In response, software developers dedicate time and financial resources to adjusting that feature.
• Survey data reveals that as the price of microwaves increases, so does customers’ perceived value of the product. With this in mind, a company chooses to increase prices by 30%.
• A popular food chain conducts a series of phone interviews to assess customer service at various locations. Afterward, the company mandates that employees at the restaurants with the lowest reviews take customer service training courses.

All this to say, marketing research has many different applications. By gathering data about customer preferences, market trends, and industry dynamics, businesses can make decisions based on facts rather than assumptions or intuition. This can help businesses avoid costly mistakes and increase their chances of success.

Conduct high-quality marketing research with Kantar

Marketing research is a critical tool that businesses and organisations can use to gather valuable information about their target market and industry. By conducting research, businesses can gain insights into customer needs and behaviour, stay competitive, make informed decisions, manage risk, and measure success.

And during VUCA times (volatile, uncertain, complex and ambiguous), keeping a consistent pulse on consumer sentiment and behaviour is essential to driving the right decisions for business growth.

However, in today's rapidly changing business landscape, marketing research can feel complicated and overwhelming. That’s where Kantar comes in. As an industry leader, we provide tips and tools to help you navigate each step of the marketing research process, from survey design to data visualisation.

Want to get started? Speak to our award-winning team  to learn how we can help you conduct marketing research.

For research tips direct to your inbox each month, sign up using the form below.

Want more like this?

Read: Survey sample questions and templates  

Read: Everything you need to know about data quality  

Read: Guide to market research surveys

Run your survey research with your rules

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Bras Pneumol
• v.44(3); May-Jun 2018

Internal and external validity: can you apply research study results to your patients?

Cecilia maria patino.

1 . Methods in Epidemiologic, Clinical, and Operations Research-MECOR-program, American Thoracic Society/Asociación Latinoamericana del Tórax, Montevideo, Uruguay.

2 . Department of Preventive Medicine, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA.

Juliana Carvalho Ferreira

3 . Divisão de Pneumologia, Instituto do Coração, Hospital das Clínicas, Faculdade de Medicina, Universidade de São Paulo, São Paulo (SP) Brasil.

CLINICAL SCENARIO

In a multicenter study in France, investigators conducted a randomized controlled trial to test the effect of prone vs. supine positioning ventilation on mortality among patients with early, severe ARDS. They showed that prolonged prone-positioning ventilation decreased 28-day mortality [hazard ratio (HR) = 0.39; 95% CI: 0.25-0.63]. 1

STUDY VALIDITY

The validity of a research study refers to how well the results among the study participants represent true findings among similar individuals outside the study. This concept of validity applies to all types of clinical studies, including those about prevalence, associations, interventions, and diagnosis. The validity of a research study includes two domains: internal and external validity.

Internal validity is defined as the extent to which the observed results represent the truth in the population we are studying and, thus, are not due to methodological errors. In our example, if the authors can support that the study has internal validity, they can conclude that prone positioning reduces mortality among patients with severe ARDS. The internal validity of a study can be threatened by many factors, including errors in measurement or in the selection of participants in the study, and researchers should think about and avoid these errors.

Once the internal validity of the study is established, the researcher can proceed to make a judgment regarding its external validity by asking whether the study results apply to similar patients in a different setting or not ( Figure 1 ). In the example, we would want to evaluate if the results of the clinical trial apply to ARDS patients in other ICUs. If the patients have early, severe ARDS, probably yes, but the study results may not apply to patients with mild ARDS . External validity refers to the extent to which the results of a study are generalizable to patients in our daily practice, especially for the population that the sample is thought to represent.

Lack of internal validity implies that the results of the study deviate from the truth, and, therefore, we cannot draw any conclusions; hence, if the results of a trial are not internally valid, external validity is irrelevant. 2 Lack of external validity implies that the results of the trial may not apply to patients who differ from the study population and, consequently, could lead to low adoption of the treatment tested in the trial by other clinicians.

INCREASING VALIDITY OF RESEARCH STUDIES

To increase internal validity, investigators should ensure careful study planning and adequate quality control and implementation strategies-including adequate recruitment strategies, data collection, data analysis, and sample size. External validity can be increased by using broad inclusion criteria that result in a study population that more closely resembles real-life patients, and, in the case of clinical trials, by choosing interventions that are feasible to apply. 2

To enrich the lives of those living with home IV nutrition or tube feeding through advocacy, education, community and innovation.

• Bylaws and Financials
• Board of Trustees
• Donate to Support Oley
• Get Involved
• Help - Sign In
• Spread the Word About Oley
• Joan Bishop Lifetime Achievement Award
• Ambassador Network
• Consumer/Patient Profiles
• Oley Award Winners
• Award Nomination Form
• Oley-Inspire Forum
• Support Groups
• Caregivers/Parents
• Diet, Hydration, Diarrhea
• Sign Up Today, for Healthcare Professionals
• View the Directory
• HEN (Tube Feeding)
• HPN (IV Nutrition)
• Motility Disorders
• Short Bowel
• Transplantation
• Sleep on TPN
• Insurance/Disability
• Legal Resources
• Adjusting to Tube Feeding/IV Nutrition
• Emergency Preparedness
• Ideas & Products
• Restaurant/Bathroom Cards
• Medical Air Transport Serivces
• Sample Travel Letter for Tube & IV Consumers
• Travel/Hospital Packet
• Travel Tips
• Tube Feeding Tips
• Links to Other Organizations
• Online Store
• HPN Research Prize
• HPN Research Prize Submission Form
• HPEN Research Report
• Join a Study
• 4EverBold Contest
• Kyle R. Noble Scholarship (Education)
• Oley Tim Weaver Camp Scholarship
• Oley Award Nomination Form
• Catheter Infection Guide
• Save My Line
• Tube Feeding Troubleshooting
• 2025 Charleston, SC
• Why Attend?
• 2012-2024 Conferences
• Community Connections
• Previous Mini Meetings
• Oley ASPEN Symposium
• Feeding Tube Awareness Week
• HPN Awareness Week
• HPEN Drug/Product Shortages
• FDA Listening Session
• Legislation
• Self Advocacy
• Tell Others
• Enteral Donations
• Industry Partners

8/8/2024 Access Issues

8/8/2024 Malnutrition and Older Adults

8/8/2024 Discontinuation of Kangaroo™ePumps, Kangaroo™Joey pumps, and associated accessories

5/13/2024 Oley President Interviewed for Women in Leadership Series

9/16/2024 » 9/20/2024 Malnutrition Awareness Week

9/24/2024 How to Do College with a Disability…You Have No IDEA in College

Profile Home  

Manage Profile

Quick Links

• Shop Online
• Enteral Donation Program

New Mailing Address as of 8/21/23  The Oley Foundation Albany Medical Center, MC-28 43 New Scotland Ave.  Albany, NY 12208  518-262-5079

[email protected] 

Physical address as of 8/21/23

Albany Medical Center

66 Hackett Blvd.

Albany, NY 12209