searching for research papers

A free, AI-powered research tool for scientific literature

• Dennis O'Dea
• Electronegativity

New & Improved API for Developers

Introducing semantic reader in beta.

Stay Connected With Semantic Scholar Sign Up What Is Semantic Scholar? Semantic Scholar is a free, AI-powered research tool for scientific literature, based at Ai2.

Reference management. Clean and simple.

The top list of academic search engines

1. Google Scholar

4. science.gov, 5. semantic scholar, 6. baidu scholar, get the most out of academic search engines, frequently asked questions about academic search engines, related articles.

Academic search engines have become the number one resource to turn to in order to find research papers and other scholarly sources. While classic academic databases like Web of Science and Scopus are locked behind paywalls, Google Scholar and others can be accessed free of charge. In order to help you get your research done fast, we have compiled the top list of free academic search engines.

Google Scholar is the clear number one when it comes to academic search engines. It's the power of Google searches applied to research papers and patents. It not only lets you find research papers for all academic disciplines for free but also often provides links to full-text PDF files.

• Coverage: approx. 200 million articles
• Abstracts: only a snippet of the abstract is available
• Related articles: ✔
• References: ✔
• Cited by: ✔
• Links to full text: ✔
• Export formats: APA, MLA, Chicago, Harvard, Vancouver, RIS, BibTeX

BASE is hosted at Bielefeld University in Germany. That is also where its name stems from (Bielefeld Academic Search Engine).

• Coverage: approx. 136 million articles (contains duplicates)
• Abstracts: ✔
• Related articles: ✘
• References: ✘
• Cited by: ✘
• Export formats: RIS, BibTeX

CORE is an academic search engine dedicated to open-access research papers. For each search result, a link to the full-text PDF or full-text web page is provided.

• Coverage: approx. 136 million articles
• Links to full text: ✔ (all articles in CORE are open access)
• Export formats: BibTeX

Science.gov is a fantastic resource as it bundles and offers free access to search results from more than 15 U.S. federal agencies. There is no need anymore to query all those resources separately!

• Coverage: approx. 200 million articles and reports
• Links to full text: ✔ (available for some databases)
• Export formats: APA, MLA, RIS, BibTeX (available for some databases)

Semantic Scholar is the new kid on the block. Its mission is to provide more relevant and impactful search results using AI-powered algorithms that find hidden connections and links between research topics.

• Coverage: approx. 40 million articles
• Export formats: APA, MLA, Chicago, BibTeX

Although Baidu Scholar's interface is in Chinese, its index contains research papers in English as well as Chinese.

• Coverage: no detailed statistics available, approx. 100 million articles
• Abstracts: only snippets of the abstract are available
• Export formats: APA, MLA, RIS, BibTeX

RefSeek searches more than one billion documents from academic and organizational websites. Its clean interface makes it especially easy to use for students and new researchers.

• Coverage: no detailed statistics available, approx. 1 billion documents
• Abstracts: only snippets of the article are available
• Export formats: not available

Consider using a reference manager like Paperpile to save, organize, and cite your references. Paperpile integrates with Google Scholar and many popular databases, so you can save references and PDFs directly to your library using the Paperpile buttons:

Google Scholar is an academic search engine, and it is the clear number one when it comes to academic search engines. It's the power of Google searches applied to research papers and patents. It not only let's you find research papers for all academic disciplines for free, but also often provides links to full text PDF file.

Semantic Scholar is a free, AI-powered research tool for scientific literature developed at the Allen Institute for AI. Sematic Scholar was publicly released in 2015 and uses advances in natural language processing to provide summaries for scholarly papers.

BASE , as its name suggest is an academic search engine. It is hosted at Bielefeld University in Germany and that's where it name stems from (Bielefeld Academic Search Engine).

CORE is an academic search engine dedicated to open access research papers. For each search result a link to the full text PDF or full text web page is provided.

Science.gov is a fantastic resource as it bundles and offers free access to search results from more than 15 U.S. federal agencies. There is no need any more to query all those resources separately!

Explore millions of high-quality primary sources and images from around the world, including artworks, maps, photographs, and more.

Explore migration issues through a variety of media types

• Part of Street Art Graphics
• Part of The Journal of Economic Perspectives, Vol. 34, No. 1 (Winter 2020)
• Part of Cato Institute (Aug. 3, 2021)
• Part of University of California Press
• Part of Open: Smithsonian National Museum of African American History & Culture
• Part of Indiana Journal of Global Legal Studies, Vol. 19, No. 1 (Winter 2012)
• Part of R Street Institute (Nov. 1, 2020)
• Part of Leuven University Press
• Part of UN Secretary-General Papers: Ban Ki-moon (2007-2016)
• Part of Perspectives on Terrorism, Vol. 12, No. 4 (August 2018)
• Part of Leveraging Lives: Serbia and Illegal Tunisian Migration to Europe, Carnegie Endowment for International Peace (Mar. 1, 2023)
• Part of UCL Press

Harness the power of visual materials—explore more than 3 million images now on JSTOR.

Enhance your scholarly research with underground newspapers, magazines, and journals.

Explore collections in the arts, sciences, and literature from the world’s leading museums, archives, and scholars.

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

• We're Hiring!
• Help Center

Download 55 million PDFs for free

Explore our top research interests.

Engineering

Anthropology

• Earth Sciences

• Computer Science

• Mathematics

• Health Sciences

Join 271 million academics and researchers

Track your impact.

Share your work with other academics, grow your audience and track your impact on your field with our robust analytics

Discover new research

Get access to millions of research papers and stay informed with the important topics around the world

Publish your work

Publish your research with fast and rigorous service through Academia.edu Journals. Get instant worldwide dissemination of your work

Unlock the most powerful tools with Academia Premium

Work faster and smarter with advanced research discovery tools

Search the full text and citations of our millions of papers. Download groups of related papers to jumpstart your research. Save time with detailed summaries and search alerts.

• Advanced Search
• PDF Packages of 37 papers
• Summaries and Search Alerts

Share your work, track your impact, and grow your audience

Get notified when other academics mention you or cite your papers. Track your impact with in-depth analytics and network with members of your field.

• Mentions and Citations Tracking
• Advanced Analytics
• Publishing Tools

Real stories from real people

Used by academics at over 15,000 universities

Get started and find the best quality research

• Academia.edu Journals
•   We're Hiring!
•   Help Center
• Find new research papers in:
• Cognitive Science
• Academia ©2024

Unfortunately we don't fully support your browser. If you have the option to, please upgrade to a newer version or use Mozilla Firefox , Microsoft Edge , Google Chrome , or Safari 14 or newer. If you are unable to, and need support, please send us your feedback .

We'd appreciate your feedback. Tell us what you think!   opens in new tab/window

Scopus Search

Scopus quickly delivers the information you're looking for from over 92m records. Updated daily, Scopus features state-of-the-art search tools and filters to empower research efficiency.

Increase research efficiency

Having access to comprehensive content and high-quality data is only effective if you can easily find the information you need. Uncovering trends, discovering sources and potential collaborators, and building deeper insights require effective search tools that can identify the right results.

Identify trends for key topics

Scopus’ literature search is built to distill massive amounts of information down to the most relevant documents and information in less time.

With Scopus you can search and filter results in the following ways:

Document search : Search directly from the homepage and use detailed search options to ensure you find the document(s) you want

Author search : Search for a specific author by name or by Open Research and Contributor Identifier ID (ORCID)

Affiliation search : Identify and assess an affiliation’s scholarly output, collaborating institutions and top authors

Advanced search : Narrow the scope of your search using field codes, proximity operators and/or Boolean operators

Refine results : Scopus makes it easy to refine your results list to specific categories of documents

Language interface : The Scopus interface is available in Chinese and Japanese; content is not localized, but you can switch the interface to one of these language options (and switch back to English, the default language) at the bottom of any Scopus page

Quick reference guide

Learn how to easily start your search from the homepage and use all the features in Scopus with this handy Quick Reference Guide.

Download the guide   opens in new tab/window .

Learn how Scopus can help your organization achieve its goals.

Related links

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

The PMC website is updating on October 15, 2024. Learn More or Try it out now .

PubMed Central (PMC) Home Page

PubMed Central ® (PMC) is a free full-text archive of biomedical and life sciences journal literature at the U.S. National Institutes of Health's National Library of Medicine (NIH/NLM)

Discover a digital archive of scholarly articles, spanning centuries of scientific research.

Learn how to find and read articles of interest to you.

Collections

Browse the PMC Journal List or learn about some of PMC's unique collections.

For Authors

Navigate the PMC submission methods to comply with a funder mandate, expand access, and ensure preservation.

For Publishers

Learn about deposit options for journals and publishers and the PMC selection process.

For Developers

Find tools for bulk download, text mining, and other machine analysis.

10.2 MILLION articles are archived in PMC.

Content provided in part by:, full participation journals.

Journals deposit the complete contents of each issue or volume.

NIH Portfolio Journals

Journals deposit all NIH-funded articles as defined by the NIH Public Access Policy.

Selective Deposit Programs

Publisher deposits a subset of articles from a collection of journals.

Sept. 16, 2024

Pmc website update coming october 15.

On October 15, 2024, PMC will transition to a new website running on cloud architecture. This update, available for preview …

March 21, 2024

Preview upcoming improvements to pmc.

We are pleased to announce the availability of a preview of improvements planned for the PMC website. These…

We are pleased to announce the availability of a preview of improvements planned for the PMC website. These improvements will become the default in October 2024.

28 Best Academic Search Engines That make your research easier

If you’re a researcher or scholar, you know that conducting effective online research is a critical part of your job. And if you’re like most people, you’re always on the lookout for new and better ways to do it. 

This article aims to give you an edge over researchers that rely mainly on Google for their entire research process.

Table of Contents

#1. Google Scholar

Google Scholar is an academic search engine that indexes the full text or metadata of scholarly literature across an array of publishing formats and disciplines.

#2. ERIC (Education Resources Information Center) 

ERIC indexes over a million articles, reports, conference papers, and other resources on all aspects of education from early childhood to higher education. So, search results are more relevant to Education on ERIC. 

ERIC is a free online database of education-related literature. 

#3. Wolfram Alpha

Wolfram Alpha can also be used to find academic articles. Just type in your keywords and Wolfram Alpha will generate a list of academic articles that match your query.

#4. iSEEK Education 

iSEEK is a search engine targeting students, teachers, administrators, and caregiver. It’s designed to be safe with editor-reviewed content.

iSEEK Education is free to use.

#5. BASE (Bielefeld Academic Search Engine)

CORE is an academic search engine that focuses on open access research papers. A link to the full text PDF or complete text web page is supplied for each search result. It’s academic search engine dedicated to open access research papers.

You might also like:

#7. Science.gov

#8. semantic scholar, #9. refseek.

This is one of the free search engines that feels like Yahoo with a massive directory. It could be good when you are just looking for research ideas from unexpected angles. It could lead you to some other database that you might not know such as the CIA The World Factbook, which is a great reference tool.

#10. ResearchGate 

A mixture of social networking site + forum + content databases where researchers can build their profile, share research papers, and interact with one another.

#11. DataONE Search (formerly CiteULike) 

#12. dataelixir , #13. lazyscholar – browser extension, #14. citeseerx – digital library from penstate, #15. the lens – patents search , #16. fatcat – wiki for bibliographic catalog , #17. lexis web – legal database, #18. infotopia – part of the vlrc family, #19. virtual learning resources center, #21. worldwidescience.

Over 70 countries’ databases are used on the website. When a user enters a query, it contacts databases from all across the world and shows results in both English and translated journals and academic resources.

#22. Google Books

A user can browse thousands of books on Google Books, from popular titles to old titles, to find pages that include their search terms. You can look through pages, read online reviews, and find out where to buy a hard copy once you find the book you are interested in.

#23. DOAJ (Directory of Open Access Journals)

#24. baidu scholar, #25. pubmed central, #26. medline®.

MEDLINE® is a paid subscription database for life sciences and biomedicine that includes more than 28 million citations to journal articles. For finding reliable, carefully chosen health information, Medline Plus provides a powerful search tool and even a dictionary.

Defunct Academic Search Engines 

#27. microsoft academic  .

Microsoft Academic

#28. Scizzle

Final thoughts.

There are many academic search engines that can help researchers and scholars find the information they need. This list provides a variety of options, starting with more familiar engines and moving on to less well-known ones. 

10 thoughts on “28 Best Academic Search Engines That make your research easier”

Thank you so much Joannah..I have found this information useful to me as librarian in an academic library

You are welcome! We are happy to hear that!

Thank You Team, for providing a comprehensive list of academic search engines that can help make research easier for students and scholars. The variety of search engines included offers a range of options for finding scholarly articles, journals, and other academic resources. The article also provides a brief summary of each search engine’s features, which helps in determining which one is the best fit for a specific research topic. Overall, this article is a valuable resource for anyone looking for a quick and easy way to access a wealth of academic information.

We appreciate your support and thank you for your kind words. We will continue to provide valuable resources for students and researchers in the future. Please let us know if you have any further questions or suggestions.

No more questions Thank You

I cannot thank you enough!!! thanks alot 🙂

Typography animation is a technique that combines text and motion to create visually engaging and dynamic animations. It involves animating individual letters, words, or phrases in various ways to convey a message, evoke emotions, or enhance the visual impact of a design or video. – Typography Animation Techniques Tools and Online Software {43}

Expontum – Helps researchers quickly find knowledge gaps and identify what research projects have been completed before. Expontum is free, open access, and available to all globally with no paid versions of the site. Automated processes scan research article information 24/7 so this website is constantly updating. By looking at over 35 million research publications (240 million by the end of 2023), the site has 146 million tagged research subjects and 122 million tagged research attributes. Learn more about methodology and sources on the Expontum About Page ( https://www.expontum.com/about.php )

Hey Ryan, I clicked and checked your site and thought it was very relevant to our reader. Thank you for sharing. And, we will be reviewing your site soon.

Sounds good! Thanks, Joannah!

Leave a Comment Cancel reply

Analyze research papers at superhuman speed

Search for research papers, get one sentence abstract summaries, select relevant papers and search for more like them, extract details from papers into an organized table.

Find themes and concepts across many papers

Don't just take our word for it.

Tons of features to speed up your research

Upload your own pdfs, orient with a quick summary, view sources for every answer, ask questions to papers, research for the machine intelligence age, pick a plan that's right for you, get in touch, enterprise and institutions, common questions. great answers., how do researchers use elicit.

Over 2 million researchers have used Elicit. Researchers commonly use Elicit to:

• Speed up literature review
• Find papers they couldn’t find elsewhere
• Automate systematic reviews and meta-analyses
• Learn about a new domain

Elicit tends to work best for empirical domains that involve experiments and concrete results. This type of research is common in biomedicine and machine learning.

What is Elicit not a good fit for?

Elicit does not currently answer questions or surface information that is not written about in an academic paper. It tends to work less well for identifying facts (e.g. "How many cars were sold in Malaysia last year?") and in theoretical or non-empirical domains.

What types of data can Elicit search over?

Elicit searches across 125 million academic papers from the Semantic Scholar corpus, which covers all academic disciplines. When you extract data from papers in Elicit, Elicit will use the full text if available or the abstract if not.

How accurate are the answers in Elicit?

A good rule of thumb is to assume that around 90% of the information you see in Elicit is accurate. While we do our best to increase accuracy without skyrocketing costs, it’s very important for you to check the work in Elicit closely. We try to make this easier for you by identifying all of the sources for information generated with language models.

How can you get in contact with the team?

You can email us at [email protected] or post in our Slack community ! We log and incorporate all user comments, and will do our best to reply to every inquiry as soon as possible.

What happens to papers uploaded to Elicit?

When you upload papers to analyze in Elicit, those papers will remain private to you and will not be shared with anyone else.

How accurate is Elicit?

Training our models on specific tasks, searching over academic papers, making it easy to double-check answers, save time, think more. try elicit for free..

AI Search Engine for Research

Find & understand the best science, faster.

Try an example search

See how it works

Used by researchers at the world’s top institutes

Why Consensus?

Consensus responsibly uses AI to help you conduct research faster.

Extensive research coverage

Search through over 200M research papers across every domain of science & academia.

Time-saving AI insights

Get instant insights with Copilot, the Consensus Meter and more. We leverage both OpenAI & custom LLMs.

Find the most relevant papers

Our proprietary academic search tools & filters help you find the most relevant and reliable research papers, faster.

Results connected to science

Everything we show in our product is cited. You're always only one click away from the underlying research paper.

Who Consensus helps most...

Whether you’re conducting a systematic review or just fact-checking a friend, if you need insights from the literature, Consensus is for you.

Students & researchers

Streamline your literature review process. Quickly see the direction of current findings, and surface the best papers.

Science organizations

Quickly check ingredients, chemicals, or molecules. Understand mechanisms of action, and stay up to date with new research.

Clinicians & doctors

Get answers to patients’ questions that you can trust, share information they can digest, and easily cite your references.

Universities & schools

Students & researchers at over 5,000 universities worldwide search with Consensus. We partner with libraries, higher learning institutes, and universities.

Writers & journalists

Source evidence-based insights on your topic, understand connected fields, and see related suggested searches.

Health & fitness experts

Easily check out the science regarding supplement safety, diet types, and exercise science outcomes.

2,000,000 +

Researchers, students, doctors, professionals, and evidence-based humans choose Consensus.

"I can make sense of what’s out there a lot faster with Consensus. I jump into different topics with the summary & Copilot before diving deeper. The interface makes it so easy to review individual papers and see what they’re about."

"It's not every day I find a tool that truly helps with my work. Consensus blew me away when I started using it, I was learning things I had never encountered before. This is an AI product that isn't hype."

"No more endless scrolling and scanning research papers. Simply ask a question and Consensus gives you AI-powered summaries of the top 5-10 articles"

Consensus has been featured in

Consensus vs ChatGPT

ChatGPT predicts the most likely language that should follow. Consensus helps you find & understand the best science, faster.

Results directly connected to scientific papers

Fully machine-generated, prone to hallucinations

Consensus' time-saving features

Consensus Meter

Quickly see the scientific consensus & gain topic context and direction. See exactly which papers were included.

Simply include in your search - ask Copilot to adopt a style, draft content, format, create lists, and more. Read a referenced topic synthesis.

Paper-level Insights

We extract key insights and answers. Locate the most helpful papers and digest their insights faster.

Search Filters

Filter by sample size, study design, methodology, if the paper is open access, a human or animal study (and many more filters).

Quality Indicators

Focus on the best papers - intuitive labels for citations, journal quality, and study type.

Study Snapshot

Our Study Snapshot quickly shows key information like Population, Sample size, Methods, etc. - all within the results page.

How we created the ultimate search engine for science

Search: Find the best papers

Purpose-built academic & vector search functionality. Consensus utilizes important factors like study design, sample size, population details, and more to rank the best research higher.

Dedicated research LLMs

Our proprietary LLMs read research like an expert - we also leverage the best-in-class models from OpenAI. Consensus generates AI insights at both the search and paper level.

What’s new at Consensus

Introducing the Consensus API - Embed Evidence-Based Results

Offer instant access to the most relevant academic papers. Seamlessly integrate peer-reviewed citations into your projects.

Announcing Our $11.5M Series A Fundraise

This funding will accelerate our mission of creating a world where expert knowledge is within everyone’s reach.

• Find journals
• My journals

Register Sign in

Register or sign-in in order to manage your journal lists

Sign in or register to save a journal

To save a journal and create lists, you need to sign in to your Elsevier account.

Find the right journal for your research

Looking for the best journal match for your paper? Search the world's leading source of academic journals using your abstract or your keywords and other details.

Check if you're eligible for open access (OA) savings.

How do I search in Scopus?

You can search in Scopus by document, author, or organization:

Scopus allows you to search for publications based on search terms relating to specific parts of a document (e.g., title, author, keywords, ISSN).

To search for a document:

• Go to the Scopus homepage . The default page is the Document search.
• Select the fields to search within from the Search within drop-down.
• Enter your terms in the Search documents field.

Note: To add search terms, select 'Add search field' for an additional search term line.

• Select 'Search' . For information about how to work with document search results, see document search results .

Document search tips:

Use boolean operators to combine different search queries and proximity operators to find words near/within a specified distance of each other.

Boolean operators - OR, AND, AND NOT

Rules for using Boolean operators:

• AND NOT e.g., KEY(mouse AND NOT cat OR dog) is interpreted as KEY((mouse) AND NOT (cat OR dog))
• AND NOT should always be used at the end of the query.
• To search for a specific phrase, enclose the terms in double quotes (" ") or for an exact match use braces ({}).

Proximity operators - W/ n , PRE/ n

You can choose between two Proximity operators to find words within a certain distance from each other: Pre/ n specifies a word order whereas W/ n does not.

Tips for proximity operators:

There are two ways of searching for phrases, an exact search and a loose/approximate phrase, depending on how exact a match you want to find.

Loose/approximate phrases

Double quotation marks are important when searching for a loose/approximate phrase.

• Loose phrase: TITLE-ABS-KEY( "heart attack") searches for documents where heart attack appear together in the title, abstract, or keywords.
• Not a loose phrase: TITLE-ABS-KEY( heart attack) searches for documents where heart and attack appear together or separately in the title, abstract, or keywords.
• Dots and hyphens are treated as intentional. When a dot/hyphen is used, it is ignored and the search terms are treated as a loose phrase
• heart-attack or heart.attack is searched as "heart attack"
• Wildcards work: "criminal* liab*" finds criminally liable and criminal liability .
• Plurals and spelling variants are included: heart attack includes heart attacks , anesthesia includes anaesthesia .
• Double quotation marks can be used to search specifically for stop words and special characters: "crocodiles with alligators" will return results such as: Crocodiles with alligators are among the largest reptiles .
• title-abs-key (*/art) is searched as title-abs-key(art)
• abs(iwv-*) is searched as abs(iwv)

Exact phrase

To find documents that contain an exact phrase, enclose the phrase in braces: {oyster toadfish} .

RESULT: This includes any stop words , spaces, and punctuation which you included in the braces. For example:

• {heart-attack} and {heart attack} will return different results because the dash is included.
• Wildcards are searched as actual characters, e.g., {health care?} returns results such as: Who pays for health care?

You can search for accented characters either with or without the accent. The results contain both variants.

Example:  España and Espana are both found whether you entered espana or españa .

This also applies to special characters.

Letters from the Greek alphabet and their spelled-out equivalents – alpha; α

Special characters that have common equivalents like π r⊃2; - pr2

Special characters with no common equivalents, punctuation, and spacing are ignored.

To search specifically for a special character or a punctuation mark, enclose it in braces {π}.

Using the singular form of a word in your search retrieves the singular, plural, and possessive forms of most words.

Scopus applies word stemming to fields containing text (not to names, affiliations, dates, or numbers). Word stemming ensures that different occurrences of a word are found.

Example : criterion finds criteria and criterion

Use these filters to reduce your search results:

Scopus coverage focuses on primary document types from serial publications. Primary means that the author is identical to the researcher in charge of the presented findings. Scopus does not include secondary document types, where the author is not identical to the person behind the presented research, such as obituaries and book reviews.

Document types covered in Scopus

• Article or Review
• Book or Book Chapter
• Book Chapter
• Article or Conference Paper
• Conference Paper
• Conference Review
• Short Survey
• Business Article or Press

Document types not covered in Scopus

• Book reviews
• Conference meeting

For more in-depth information about document types, see the Scopus Content Coverage Guide .

The author search helps you find documents written by a specific person in Scopus , even if the author is listed inconsistently. For example, an author may be cited as Smith, J in one document, but as Smith, John in another. The Scopus Author Identifier allows you to identify between different authors in author search results.

You can also search using an Open Researcher and Contributor ID (ORCID). For more information, see How do I search for authors using ORCID?

• From the Scopus homepage , select the 'Authors' tab.
• From the 'Search using' dropdown, select 'Author name' to search by name or select 'ORCID' .
• Enter the last name of the author in the 'Enter last name' field. Alternatively, enter the ORCID number. The last name of the author is required. You can also enter a first name or initials, and an affilation name to further narrow your results.
• Select 'Search' .

Author search tips

Author Search results include any available author name variations that match your search criteria. For example, searching for Smith, J will also produce Smith, John in the results.

Here is how to limit your search results (e.g., for Smith, J.):

Unsure about spelling the author’s name? Use Wildcards to replace letters with unknowns:

The hyphen is treated as punctuation and therefore ignored if it is not in an exact phrase. Wildcards must be used with words because they cannot be standalone. When an hyphen is placed between a wildcard and a word, the wildcard will be dropped.

• Author last name: Smith-*
• Author last name: Jones and Affiliation: *-smithsonian

It is also possible to search an Author in Advanced Search with their author ID. For example: AU-ID(000000000)

Multiple author IDs can be searched as well: For example: AU-ID(000000000) OR AU-ID(111111111) OR AU-ID(222222222)

Any author search field Codes can be used with OR between them to search multiple authors.

An organization search returns a list of organizations with links to documents and a summary of the organization's research areas, collaborations, and publications.

To search for documents and authors within those organizations:

• From the Scopus homepage , select the 'Organizations' tab.
• Enter the name of an organization in the 'Search organizations' field.
• Select the arrow to search.

Affiliation search tips

• AND NOT e.g., ‘KEY(mouse AND NOT cat OR dog)’ is interpreted as ‘KEY((mouse) AND NOT (cat OR dog))’
• To search for a specific phrase, enclose the terms in double quotes (" ") or, for an exact match, brackets ({}).

Use these characters (wildcards) to find variations of a word:

Note: The hyphen is treated as punctuation and therefore ignored if it is not in an exact phrase. Wildcards must be used with words because they cannot be standalone. When an hyphen is placed between a wildcard and a word, the wildcard will be dropped.

• Affiliation name: micro-*

For example, you can enter Técnicas or Tecnicas for your search. Searching for Tecnicas returns results for e as well as é.

Here are some different field codes you could use:

You can combine two or more searches with the operators OR , AND , and AND NOT using Combine queries.

• From the Scopus homepage , select the 'Search history' tab.
• Choose two or more searches and select 'Combine queries' .
• From the Combine queries page, select the desired operator from the operator dropdown for each query combination. Note: Select an operator from the Change all operators dropdown to syncronize all operators.
• Select 'Show results' to view the results of the combined query.

Was this answer helpful?

Thank you for your feedback, it will help us serve you better. If you require assistance, please scroll down and use one of the contact options to get in touch.

Help us to help you:

Thank you for your feedback!

• Why was this answer not helpful?
• It was hard to understand / follow.
• It did not answer my question.
• The solution did not work.
• There was a mistake in the answer.
• Feel free to leave any comments below: Please enter your feedback to submit this form

Related Articles:

• How to conduct a basic search tutorial
• How can I best use the Advanced search?
• How do I work with document search results?
• Scopus tutorials
• What is the Scopus Author Identifier?

For further assistance:

You may be searching for:

• How does artificial intelligence enhance education?
• How can technology promote mental health and well-being?
• Why do people spread fake news?
• How does globalization impact cultural diversity?
• How to prepare for future pandemics?

Free AI Search Engine for Research

Quickly find and gain valuable insights from 300 million+ research papers.

• Sustainable cities
• Food security
• AI in healthcare

Why HIX Scholar?

300 million+ scholarly papers

Access over 300 million authoritative and reputable academic papers. Find the most reliable and up-to-date information in one place.

Save over 50% of research time

Instantly find the most relevant papers related to any research topic or question, cutting hours spent searching different databases.

Free access for all

Knowledge should be accessible to all. You can use HIX Scholar's robust search engine and vast scholarly resources without any cost.

Covers all academic disciplines

Whether you are looking for research papers in physics, literature, humanities, or any other field, HIX Scholar has got you covered.

We Empower All Knowledge Seekers

Find reliable sources for research papers and school projects quickly and easily.

Find relevant academic resources to incorporate into lectures and classes.

Find technical papers and studies to support engineering projects and advancements.

Researchers

Conduct efficient and effective literature reviews for research projects.

Legal Professionals

Gather valuable information from scholarly articles to support legal cases and arguments.

Health Practitioners

Access up-to-date medical literature to inform diagnoses, treatments, and patient care.

Try Our AI Search Engine for Research for Free

Search and retrieve relevant information from hundreds of millions of papers.

This website uses cookies to ensure you get the best experience. Learn more about DOAJ’s privacy policy.

Hide this message

You are using an outdated browser. Please upgrade your browser to improve your experience and security.

The Directory of Open Access Journals

Directory of Open Access Journals

Find open access journals & articles.

Doaj in numbers.

80 languages

135 countries represented

13,752 journals without APCs

20,920 journals

10,461,097 article records

Quick search

About the directory.

DOAJ is a unique and extensive index of diverse open access journals from around the world, driven by a growing community, and is committed to ensuring quality content is freely available online for everyone.

DOAJ is committed to keeping its services free of charge, including being indexed, and its data freely available.

→ About DOAJ

→ How to apply

DOAJ is twenty years old in 2023.

Fund our 20th anniversary campaign

DOAJ is independent. All support is via donations.

82% from academic organisations

18% from contributors

Support DOAJ

Publishers don't need to donate to be part of DOAJ.

News Service

Meet the doaj team: head of editorial and deputy head of editorial (quality), vacancy: operations manager, press release: pubscholar joins the movement to support the directory of open access journals, new major version of the api to be released.

→ All blog posts

We would not be able to work without our volunteers, such as these top-performing editors and associate editors.

→ Meet our volunteers

Librarianship, Scholarly Publishing, Data Management

Brisbane, Australia (Chinese, English)

Adana, Türkiye (Turkish, English)

Humanities, Social Sciences

Natalia Pamuła

Toruń, Poland (Polish, English)

Medical Sciences, Nutrition

Pablo Hernandez

Caracas, Venezuela (Spanish, English)

Research Evaluation

Paola Galimberti

Milan, Italy (Italian, German, English)

Social Sciences, Humanities

Dawam M. Rohmatulloh

Ponorogo, Indonesia (Bahasa Indonesia, English, Dutch)

Systematic Entomology

Kadri Kıran

Edirne, Türkiye (English, Turkish, German)

Library and Information Science

Nataliia Kaliuzhna

Kyiv, Ukraine (Ukrainian, Russian, English, Polish)

Recently-added journals

DOAJ’s team of managing editors, editors, and volunteers work with publishers to index new journals. As soon as they’re accepted, these journals are displayed on our website freely accessible to everyone.

→ See Atom feed

→ A log of journals added (and withdrawn)

→ DOWNLOAD all journals as CSV

• Nubihar Akademi
• Eastern Europe-Regional Studies
• International Journal of Electronics and Communications System
• Phytopathogenomics and Disease Control
• Journal of Pediatrics: Clinical Practice
• Revista Aurora
• JEM Reports
• Journal of Cycling and Micromobility Research
• Responsive Materials
• Indoor Environments
• Revija za kriminologiju i krivično pravo
• Interdisciplinary Cultural and Humanities Review
• Inter Litteras
• مجلة دراسات و بحوث التربیة النوعیة
• Journal of Food and Dairy Sciences

WeChat QR code

• Boston University Libraries

Open Access for Social Work Research

• Directories and Indexes
• Subject Guides for Your Research

Social Work Research Support

Lucy Flamm, Social Work Librarian, is available for appointments and workshops relating to social work research including identifying sources, citations, and data analysis. Lucy can be contacted via email at [email protected]. 

Using Technology

• Boston University has a laptop loan program if you need assistance staying connected with technology throughout your coursework. Learn more here. 
• The Boston Public Library also has a 3-week laptop lending program. Learn more here.  Anyone can get a free Boston Public Library card with proof of Massachusetts address. 

Need help? Ask us!

Related Subject Guides

Additional subject guides across the Library are available to support your research. A full list is available here , with guides most relevant to the social work community listed below:

• Abortion, Contraception, and Reproductive Rights by Lucy Flamm Last Updated Sep 12, 2024 665 views this year
• APA Citations (7th edition) by Lucy Flamm Last Updated Sep 12, 2024 340 views this year
• Boston Data and Statistics by Lucy Flamm Last Updated Sep 12, 2024 1140 views this year
• Children, Youth, and Families by Lucy Flamm Last Updated Sep 12, 2024 21 views this year
• Course Reserves for the School of Social Work by Lucy Flamm Last Updated Sep 11, 2024 65 views this year
• Databases for Social Work Research by Lucy Flamm Last Updated Sep 12, 2024 8 views this year
• Death and Dying by Lucy Flamm Last Updated Sep 12, 2024 183 views this year
• Food Justice by Lucy Flamm Last Updated Sep 12, 2024 622 views this year
• Gerontology by Lucy Flamm Last Updated Sep 12, 2024 62 views this year
• Immigration by Lucy Flamm Last Updated Sep 16, 2024 356 views this year
• Library Resources and Services for Social Work Faculty and Instructors by Lucy Flamm Last Updated Sep 12, 2024 140 views this year
• Literature Reviews in Social Work by Lucy Flamm Last Updated Sep 16, 2024 682 views this year
• Music Therapy by Lucy Flamm Last Updated Sep 12, 2024 361 views this year
• Open Access for Social Work Research by Lucy Flamm Last Updated Sep 16, 2024 0 views this year
• People-First Communication by Lucy Flamm Last Updated Sep 6, 2024 29 views this year
• Psychology & Brain Sciences by Kate Silfen Last Updated Aug 12, 2024 655 views this year
• Race and Justice by Lucy Flamm Last Updated Sep 16, 2024 41 views this year
• Social Work by Lucy Flamm Last Updated Sep 16, 2024 1942 views this year
• Social Work Policy by Lucy Flamm Last Updated Sep 12, 2024 499 views this year
• Substance Use and Dependence by Lucy Flamm Last Updated Sep 12, 2024 102 views this year
• Systematic Reviews in the Social Sciences by Lucy Flamm Last Updated Sep 12, 2024 1270 views this year
• Tests and Measures by Lucy Flamm Last Updated Sep 12, 2024 24 views this year
• Youth Justice by Lucy Flamm Last Updated Sep 12, 2024 50 views this year

• << Previous: Subject Guides for Your Research
• Last Updated: Sep 16, 2024 2:56 PM
• URL: https://library.bu.edu/c.php?g=1425086

The Increasing Cost of Buying American

The latest resurgence in the U.S. of policies aimed at reducing imports and bolstering domestic production has included the expansion of Buy American provisions. While some of these are new and untested, in this paper we evaluate long-standing procurement limitations on the purchase of foreign products by the U.S. Federal Government. We use procurement micro-data to first map and measure the positive employment effects of government purchases. We then calibrate a quantitative trade model adapted to include features relevant to the Buy American Act: a government sector, policy barriers in final and intermediate goods, labor force participation, and external economies of scale. We show that current Buy American provisions on final goods purchase have created up to 100,000 jobs at a cost of between $111,500 and $137,700 per job. However, the recently announced tightening of the policy on the use of foreign inputs will create fewer jobs at a higher cost of $154,000 to $237,800 per job. We also find scant evidence of the use of Buy American rules as an effective industrial policy.

We thank Vidya Venkatachalam and Bohan Wang for excellent research assistance. We appreciate comments by seminar and conference participants at LMU Munich, Boston University, Duke University, Hong Kong University of Science and Technology, the University of Hong Kong, Singapore Management University, Peking University HSBC Business School, Yale University and the CESIfo Venice Summer Institute. We also thank Andrés Rodriguéz-Clare and Steve Tadelis for very helpful suggestions. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.

MARC RIS BibTeΧ

Download Citation Data

More from NBER

In addition to working papers , the NBER disseminates affiliates’ latest findings through a range of free periodicals — the NBER Reporter , the NBER Digest , the Bulletin on Retirement and Disability , the Bulletin on Health , and the Bulletin on Entrepreneurship  — as well as online conference reports , video lectures , and interviews .

Peer Reviewed

GPT-fabricated scientific papers on Google Scholar: Key features, spread, and implications for preempting evidence manipulation

Article metrics.

CrossRef Citations

Altmetric Score

PDF Downloads

Academic journals, archives, and repositories are seeing an increasing number of questionable research papers clearly produced using generative AI. They are often created with widely available, general-purpose AI applications, most likely ChatGPT, and mimic scientific writing. Google Scholar easily locates and lists these questionable papers alongside reputable, quality-controlled research. Our analysis of a selection of questionable GPT-fabricated scientific papers found in Google Scholar shows that many are about applied, often controversial topics susceptible to disinformation: the environment, health, and computing. The resulting enhanced potential for malicious manipulation of society’s evidence base, particularly in politically divisive domains, is a growing concern.

Swedish School of Library and Information Science, University of Borås, Sweden

Department of Arts and Cultural Sciences, Lund University, Sweden

Division of Environmental Communication, Swedish University of Agricultural Sciences, Sweden

Research Questions

• Where are questionable publications produced with generative pre-trained transformers (GPTs) that can be found via Google Scholar published or deposited?
• What are the main characteristics of these publications in relation to predominant subject categories?
• How are these publications spread in the research infrastructure for scholarly communication?
• How is the role of the scholarly communication infrastructure challenged in maintaining public trust in science and evidence through inappropriate use of generative AI?

research note Summary

• A sample of scientific papers with signs of GPT-use found on Google Scholar was retrieved, downloaded, and analyzed using a combination of qualitative coding and descriptive statistics. All papers contained at least one of two common phrases returned by conversational agents that use large language models (LLM) like OpenAI’s ChatGPT. Google Search was then used to determine the extent to which copies of questionable, GPT-fabricated papers were available in various repositories, archives, citation databases, and social media platforms.
• Roughly two-thirds of the retrieved papers were found to have been produced, at least in part, through undisclosed, potentially deceptive use of GPT. The majority (57%) of these questionable papers dealt with policy-relevant subjects (i.e., environment, health, computing), susceptible to influence operations. Most were available in several copies on different domains (e.g., social media, archives, and repositories).
• Two main risks arise from the increasingly common use of GPT to (mass-)produce fake, scientific publications. First, the abundance of fabricated “studies” seeping into all areas of the research infrastructure threatens to overwhelm the scholarly communication system and jeopardize the integrity of the scientific record. A second risk lies in the increased possibility that convincingly scientific-looking content was in fact deceitfully created with AI tools and is also optimized to be retrieved by publicly available academic search engines, particularly Google Scholar. However small, this possibility and awareness of it risks undermining the basis for trust in scientific knowledge and poses serious societal risks.

Implications

The use of ChatGPT to generate text for academic papers has raised concerns about research integrity. Discussion of this phenomenon is ongoing in editorials, commentaries, opinion pieces, and on social media (Bom, 2023; Stokel-Walker, 2024; Thorp, 2023). There are now several lists of papers suspected of GPT misuse, and new papers are constantly being added. 1 See for example Academ-AI, https://www.academ-ai.info/ , and Retraction Watch, https://retractionwatch.com/papers-and-peer-reviews-with-evidence-of-chatgpt-writing/ . While many legitimate uses of GPT for research and academic writing exist (Huang & Tan, 2023; Kitamura, 2023; Lund et al., 2023), its undeclared use—beyond proofreading—has potentially far-reaching implications for both science and society, but especially for their relationship. It, therefore, seems important to extend the discussion to one of the most accessible and well-known intermediaries between science, but also certain types of misinformation, and the public, namely Google Scholar, also in response to the legitimate concerns that the discussion of generative AI and misinformation needs to be more nuanced and empirically substantiated  (Simon et al., 2023).

Google Scholar, https://scholar.google.com , is an easy-to-use academic search engine. It is available for free, and its index is extensive (Gusenbauer & Haddaway, 2020). It is also often touted as a credible source for academic literature and even recommended in library guides, by media and information literacy initiatives, and fact checkers (Tripodi et al., 2023). However, Google Scholar lacks the transparency and adherence to standards that usually characterize citation databases. Instead, Google Scholar uses automated crawlers, like Google’s web search engine (Martín-Martín et al., 2021), and the inclusion criteria are based on primarily technical standards, allowing any individual author—with or without scientific affiliation—to upload papers to be indexed (Google Scholar Help, n.d.). It has been shown that Google Scholar is susceptible to manipulation through citation exploits (Antkare, 2020) and by providing access to fake scientific papers (Dadkhah et al., 2017). A large part of Google Scholar’s index consists of publications from established scientific journals or other forms of quality-controlled, scholarly literature. However, the index also contains a large amount of gray literature, including student papers, working papers, reports, preprint servers, and academic networking sites, as well as material from so-called “questionable” academic journals, including paper mills. The search interface does not offer the possibility to filter the results meaningfully by material type, publication status, or form of quality control, such as limiting the search to peer-reviewed material.

To understand the occurrence of ChatGPT (co-)authored work in Google Scholar’s index, we scraped it for publications, including one of two common ChatGPT responses (see Appendix A) that we encountered on social media and in media reports (DeGeurin, 2024). The results of our descriptive statistical analyses showed that around 62% did not declare the use of GPTs. Most of these GPT-fabricated papers were found in non-indexed journals and working papers, but some cases included research published in mainstream scientific journals and conference proceedings. 2 Indexed journals mean scholarly journals indexed by abstract and citation databases such as Scopus and Web of Science, where the indexation implies journals with high scientific quality. Non-indexed journals are journals that fall outside of this indexation. More than half (57%) of these GPT-fabricated papers concerned policy-relevant subject areas susceptible to influence operations. To avoid increasing the visibility of these publications, we abstained from referencing them in this research note. However, we have made the data available in the Harvard Dataverse repository.

The publications were related to three issue areas—health (14.5%), environment (19.5%) and computing (23%)—with key terms such “healthcare,” “COVID-19,” or “infection”for health-related papers, and “analysis,” “sustainable,” and “global” for environment-related papers. In several cases, the papers had titles that strung together general keywords and buzzwords, thus alluding to very broad and current research. These terms included “biology,” “telehealth,” “climate policy,” “diversity,” and “disrupting,” to name just a few.  While the study’s scope and design did not include a detailed analysis of which parts of the articles included fabricated text, our dataset did contain the surrounding sentences for each occurrence of the suspicious phrases that formed the basis for our search and subsequent selection. Based on that, we can say that the phrases occurred in most sections typically found in scientific publications, including the literature review, methods, conceptual and theoretical frameworks, background, motivation or societal relevance, and even discussion. This was confirmed during the joint coding, where we read and discussed all articles. It became clear that not just the text related to the telltale phrases was created by GPT, but that almost all articles in our sample of questionable articles likely contained traces of GPT-fabricated text everywhere.

Evidence hacking and backfiring effects

Generative pre-trained transformers (GPTs) can be used to produce texts that mimic scientific writing. These texts, when made available online—as we demonstrate—leak into the databases of academic search engines and other parts of the research infrastructure for scholarly communication. This development exacerbates problems that were already present with less sophisticated text generators (Antkare, 2020; Cabanac & Labbé, 2021). Yet, the public release of ChatGPT in 2022, together with the way Google Scholar works, has increased the likelihood of lay people (e.g., media, politicians, patients, students) coming across questionable (or even entirely GPT-fabricated) papers and other problematic research findings. Previous research has emphasized that the ability to determine the value and status of scientific publications for lay people is at stake when misleading articles are passed off as reputable (Haider & Åström, 2017) and that systematic literature reviews risk being compromised (Dadkhah et al., 2017). It has also been highlighted that Google Scholar, in particular, can be and has been exploited for manipulating the evidence base for politically charged issues and to fuel conspiracy narratives (Tripodi et al., 2023). Both concerns are likely to be magnified in the future, increasing the risk of what we suggest calling evidence hacking —the strategic and coordinated malicious manipulation of society’s evidence base.

The authority of quality-controlled research as evidence to support legislation, policy, politics, and other forms of decision-making is undermined by the presence of undeclared GPT-fabricated content in publications professing to be scientific. Due to the large number of archives, repositories, mirror sites, and shadow libraries to which they spread, there is a clear risk that GPT-fabricated, questionable papers will reach audiences even after a possible retraction. There are considerable technical difficulties involved in identifying and tracing computer-fabricated papers (Cabanac & Labbé, 2021; Dadkhah et al., 2023; Jones, 2024), not to mention preventing and curbing their spread and uptake.

However, as the rise of the so-called anti-vaxx movement during the COVID-19 pandemic and the ongoing obstruction and denial of climate change show, retracting erroneous publications often fuels conspiracies and increases the following of these movements rather than stopping them. To illustrate this mechanism, climate deniers frequently question established scientific consensus by pointing to other, supposedly scientific, studies that support their claims. Usually, these are poorly executed, not peer-reviewed, based on obsolete data, or even fraudulent (Dunlap & Brulle, 2020). A similar strategy is successful in the alternative epistemic world of the global anti-vaccination movement (Carrion, 2018) and the persistence of flawed and questionable publications in the scientific record already poses significant problems for health research, policy, and lawmakers, and thus for society as a whole (Littell et al., 2024). Considering that a person’s support for “doing your own research” is associated with increased mistrust in scientific institutions (Chinn & Hasell, 2023), it will be of utmost importance to anticipate and consider such backfiring effects already when designing a technical solution, when suggesting industry or legal regulation, and in the planning of educational measures.

Recommendations

Solutions should be based on simultaneous considerations of technical, educational, and regulatory approaches, as well as incentives, including social ones, across the entire research infrastructure. Paying attention to how these approaches and incentives relate to each other can help identify points and mechanisms for disruption. Recognizing fraudulent academic papers must happen alongside understanding how they reach their audiences and what reasons there might be for some of these papers successfully “sticking around.” A possible way to mitigate some of the risks associated with GPT-fabricated scholarly texts finding their way into academic search engine results would be to provide filtering options for facets such as indexed journals, gray literature, peer-review, and similar on the interface of publicly available academic search engines. Furthermore, evaluation tools for indexed journals 3 Such as LiU Journal CheckUp, https://ep.liu.se/JournalCheckup/default.aspx?lang=eng . could be integrated into the graphical user interfaces and the crawlers of these academic search engines. To enable accountability, it is important that the index (database) of such a search engine is populated according to criteria that are transparent, open to scrutiny, and appropriate to the workings of  science and other forms of academic research. Moreover, considering that Google Scholar has no real competitor, there is a strong case for establishing a freely accessible, non-specialized academic search engine that is not run for commercial reasons but for reasons of public interest. Such measures, together with educational initiatives aimed particularly at policymakers, science communicators, journalists, and other media workers, will be crucial to reducing the possibilities for and effects of malicious manipulation or evidence hacking. It is important not to present this as a technical problem that exists only because of AI text generators but to relate it to the wider concerns in which it is embedded. These range from a largely dysfunctional scholarly publishing system (Haider & Åström, 2017) and academia’s “publish or perish” paradigm to Google’s near-monopoly and ideological battles over the control of information and ultimately knowledge. Any intervention is likely to have systemic effects; these effects need to be considered and assessed in advance and, ideally, followed up on.

Our study focused on a selection of papers that were easily recognizable as fraudulent. We used this relatively small sample as a magnifying glass to examine, delineate, and understand a problem that goes beyond the scope of the sample itself, which however points towards larger concerns that require further investigation. The work of ongoing whistleblowing initiatives 4 Such as Academ-AI, https://www.academ-ai.info/ , and Retraction Watch, https://retractionwatch.com/papers-and-peer-reviews-with-evidence-of-chatgpt-writing/ . , recent media reports of journal closures (Subbaraman, 2024), or GPT-related changes in word use and writing style (Cabanac et al., 2021; Stokel-Walker, 2024) suggest that we only see the tip of the iceberg. There are already more sophisticated cases (Dadkhah et al., 2023) as well as cases involving fabricated images (Gu et al., 2022). Our analysis shows that questionable and potentially manipulative GPT-fabricated papers permeate the research infrastructure and are likely to become a widespread phenomenon. Our findings underline that the risk of fake scientific papers being used to maliciously manipulate evidence (see Dadkhah et al., 2017) must be taken seriously. Manipulation may involve undeclared automatic summaries of texts, inclusion in literature reviews, explicit scientific claims, or the concealment of errors in studies so that they are difficult to detect in peer review. However, the mere possibility of these things happening is a significant risk in its own right that can be strategically exploited and will have ramifications for trust in and perception of science. Society’s methods of evaluating sources and the foundations of media and information literacy are under threat and public trust in science is at risk of further erosion, with far-reaching consequences for society in dealing with information disorders. To address this multifaceted problem, we first need to understand why it exists and proliferates.

Finding 1: 139 GPT-fabricated, questionable papers were found and listed as regular results on the Google Scholar results page. Non-indexed journals dominate.

Most questionable papers we found were in non-indexed journals or were working papers, but we did also find some in established journals, publications, conferences, and repositories. We found a total of 139 papers with a suspected deceptive use of ChatGPT or similar LLM applications (see Table 1). Out of these, 19 were in indexed journals, 89 were in non-indexed journals, 19 were student papers found in university databases, and 12 were working papers (mostly in preprint databases). Table 1 divides these papers into categories. Health and environment papers made up around 34% (47) of the sample. Of these, 66% were present in non-indexed journals.

Finding 2: GPT-fabricated, questionable papers are disseminated online, permeating the research infrastructure for scholarly communication, often in multiple copies. Applied topics with practical implications dominate.

The 20 papers concerning health-related issues are distributed across 20 unique domains, accounting for 46 URLs. The 27 papers dealing with environmental issues can be found across 26 unique domains, accounting for 56 URLs.  Most of the identified papers exist in multiple copies and have already spread to several archives, repositories, and social media. It would be difficult, or impossible, to remove them from the scientific record.

As apparent from Table 2, GPT-fabricated, questionable papers are seeping into most parts of the online research infrastructure for scholarly communication. Platforms on which identified papers have appeared include ResearchGate, ORCiD, Journal of Population Therapeutics and Clinical Pharmacology (JPTCP), Easychair, Frontiers, the Institute of Electrical and Electronics Engineer (IEEE), and X/Twitter. Thus, even if they are retracted from their original source, it will prove very difficult to track, remove, or even just mark them up on other platforms. Moreover, unless regulated, Google Scholar will enable their continued and most likely unlabeled discoverability.

A word rain visualization (Centre for Digital Humanities Uppsala, 2023), which combines word prominences through TF-IDF 5 Term frequency–inverse document frequency , a method for measuring the significance of a word in a document compared to its frequency across all documents in a collection. scores with semantic similarity of the full texts of our sample of GPT-generated articles that fall into the “Environment” and “Health” categories, reflects the two categories in question. However, as can be seen in Figure 1, it also reveals overlap and sub-areas. The y-axis shows word prominences through word positions and font sizes, while the x-axis indicates semantic similarity. In addition to a certain amount of overlap, this reveals sub-areas, which are best described as two distinct events within the word rain. The event on the left bundles terms related to the development and management of health and healthcare with “challenges,” “impact,” and “potential of artificial intelligence”emerging as semantically related terms. Terms related to research infrastructures, environmental, epistemic, and technological concepts are arranged further down in the same event (e.g., “system,” “climate,” “understanding,” “knowledge,” “learning,” “education,” “sustainable”). A second distinct event further to the right bundles terms associated with fish farming and aquatic medicinal plants, highlighting the presence of an aquaculture cluster.  Here, the prominence of groups of terms such as “used,” “model,” “-based,” and “traditional” suggests the presence of applied research on these topics. The two events making up the word rain visualization, are linked by a less dominant but overlapping cluster of terms related to “energy” and “water.”

The bar chart of the terms in the paper subset (see Figure 2) complements the word rain visualization by depicting the most prominent terms in the full texts along the y-axis. Here, word prominences across health and environment papers are arranged descendingly, where values outside parentheses are TF-IDF values (relative frequencies) and values inside parentheses are raw term frequencies (absolute frequencies).

Finding 3: Google Scholar presents results from quality-controlled and non-controlled citation databases on the same interface, providing unfiltered access to GPT-fabricated questionable papers.

Google Scholar’s central position in the publicly accessible scholarly communication infrastructure, as well as its lack of standards, transparency, and accountability in terms of inclusion criteria, has potentially serious implications for public trust in science. This is likely to exacerbate the already-known potential to exploit Google Scholar for evidence hacking (Tripodi et al., 2023) and will have implications for any attempts to retract or remove fraudulent papers from their original publication venues. Any solution must consider the entirety of the research infrastructure for scholarly communication and the interplay of different actors, interests, and incentives.

We searched and scraped Google Scholar using the Python library Scholarly (Cholewiak et al., 2023) for papers that included specific phrases known to be common responses from ChatGPT and similar applications with the same underlying model (GPT3.5 or GPT4): “as of my last knowledge update” and/or “I don’t have access to real-time data” (see Appendix A). This facilitated the identification of papers that likely used generative AI to produce text, resulting in 227 retrieved papers. The papers’ bibliographic information was automatically added to a spreadsheet and downloaded into Zotero. 6 An open-source reference manager, https://zotero.org .

We employed multiple coding (Barbour, 2001) to classify the papers based on their content. First, we jointly assessed whether the paper was suspected of fraudulent use of ChatGPT (or similar) based on how the text was integrated into the papers and whether the paper was presented as original research output or the AI tool’s role was acknowledged. Second, in analyzing the content of the papers, we continued the multiple coding by classifying the fraudulent papers into four categories identified during an initial round of analysis—health, environment, computing, and others—and then determining which subjects were most affected by this issue (see Table 1). Out of the 227 retrieved papers, 88 papers were written with legitimate and/or declared use of GPTs (i.e., false positives, which were excluded from further analysis), and 139 papers were written with undeclared and/or fraudulent use (i.e., true positives, which were included in further analysis). The multiple coding was conducted jointly by all authors of the present article, who collaboratively coded and cross-checked each other’s interpretation of the data simultaneously in a shared spreadsheet file. This was done to single out coding discrepancies and settle coding disagreements, which in turn ensured methodological thoroughness and analytical consensus (see Barbour, 2001). Redoing the category coding later based on our established coding schedule, we achieved an intercoder reliability (Cohen’s kappa) of 0.806 after eradicating obvious differences.

The ranking algorithm of Google Scholar prioritizes highly cited and older publications (Martín-Martín et al., 2016). Therefore, the position of the articles on the search engine results pages was not particularly informative, considering the relatively small number of results in combination with the recency of the publications. Only the query “as of my last knowledge update” had more than two search engine result pages. On those, questionable articles with undeclared use of GPTs were evenly distributed across all result pages (min: 4, max: 9, mode: 8), with the proportion of undeclared use being slightly higher on average on later search result pages.

To understand how the papers making fraudulent use of generative AI were disseminated online, we programmatically searched for the paper titles (with exact string matching) in Google Search from our local IP address (see Appendix B) using the googlesearch – python library(Vikramaditya, 2020). We manually verified each search result to filter out false positives—results that were not related to the paper—and then compiled the most prominent URLs by field. This enabled the identification of other platforms through which the papers had been spread. We did not, however, investigate whether copies had spread into SciHub or other shadow libraries, or if they were referenced in Wikipedia.

We used descriptive statistics to count the prevalence of the number of GPT-fabricated papers across topics and venues and top domains by subject. The pandas software library for the Python programming language (The pandas development team, 2024) was used for this part of the analysis. Based on the multiple coding, paper occurrences were counted in relation to their categories, divided into indexed journals, non-indexed journals, student papers, and working papers. The schemes, subdomains, and subdirectories of the URL strings were filtered out while top-level domains and second-level domains were kept, which led to normalizing domain names. This, in turn, allowed the counting of domain frequencies in the environment and health categories. To distinguish word prominences and meanings in the environment and health-related GPT-fabricated questionable papers, a semantically-aware word cloud visualization was produced through the use of a word rain (Centre for Digital Humanities Uppsala, 2023) for full-text versions of the papers. Font size and y-axis positions indicate word prominences through TF-IDF scores for the environment and health papers (also visualized in a separate bar chart with raw term frequencies in parentheses), and words are positioned along the x-axis to reflect semantic similarity (Skeppstedt et al., 2024), with an English Word2vec skip gram model space (Fares et al., 2017). An English stop word list was used, along with a manually produced list including terms such as “https,” “volume,” or “years.”

• Artificial Intelligence
• / Search engines

Cite this Essay

Haider, J., Söderström, K. R., Ekström, B., & Rödl, M. (2024). GPT-fabricated scientific papers on Google Scholar: Key features, spread, and implications for preempting evidence manipulation. Harvard Kennedy School (HKS) Misinformation Review . https://doi.org/10.37016/mr-2020-156

• / Appendix B

Bibliography

Antkare, I. (2020). Ike Antkare, his publications, and those of his disciples. In M. Biagioli & A. Lippman (Eds.), Gaming the metrics (pp. 177–200). The MIT Press. https://doi.org/10.7551/mitpress/11087.003.0018

Barbour, R. S. (2001). Checklists for improving rigour in qualitative research: A case of the tail wagging the dog? BMJ , 322 (7294), 1115–1117. https://doi.org/10.1136/bmj.322.7294.1115

Bom, H.-S. H. (2023). Exploring the opportunities and challenges of ChatGPT in academic writing: A roundtable discussion. Nuclear Medicine and Molecular Imaging , 57 (4), 165–167. https://doi.org/10.1007/s13139-023-00809-2

Cabanac, G., & Labbé, C. (2021). Prevalence of nonsensical algorithmically generated papers in the scientific literature. Journal of the Association for Information Science and Technology , 72 (12), 1461–1476. https://doi.org/10.1002/asi.24495

Cabanac, G., Labbé, C., & Magazinov, A. (2021). Tortured phrases: A dubious writing style emerging in science. Evidence of critical issues affecting established journals . arXiv. https://doi.org/10.48550/arXiv.2107.06751

Carrion, M. L. (2018). “You need to do your research”: Vaccines, contestable science, and maternal epistemology. Public Understanding of Science , 27 (3), 310–324. https://doi.org/10.1177/0963662517728024

Centre for Digital Humanities Uppsala (2023). CDHUppsala/word-rain [Computer software]. https://github.com/CDHUppsala/word-rain

Chinn, S., & Hasell, A. (2023). Support for “doing your own research” is associated with COVID-19 misperceptions and scientific mistrust. Harvard Kennedy School (HSK) Misinformation Review, 4 (3). https://doi.org/10.37016/mr-2020-117

Cholewiak, S. A., Ipeirotis, P., Silva, V., & Kannawadi, A. (2023). SCHOLARLY: Simple access to Google Scholar authors and citation using Python (1.5.0) [Computer software]. https://doi.org/10.5281/zenodo.5764801

Dadkhah, M., Lagzian, M., & Borchardt, G. (2017). Questionable papers in citation databases as an issue for literature review. Journal of Cell Communication and Signaling , 11 (2), 181–185. https://doi.org/10.1007/s12079-016-0370-6

Dadkhah, M., Oermann, M. H., Hegedüs, M., Raman, R., & Dávid, L. D. (2023). Detection of fake papers in the era of artificial intelligence. Diagnosis , 10 (4), 390–397. https://doi.org/10.1515/dx-2023-0090

DeGeurin, M. (2024, March 19). AI-generated nonsense is leaking into scientific journals. Popular Science. https://www.popsci.com/technology/ai-generated-text-scientific-journals/

Dunlap, R. E., & Brulle, R. J. (2020). Sources and amplifiers of climate change denial. In D.C. Holmes & L. M. Richardson (Eds.), Research handbook on communicating climate change (pp. 49–61). Edward Elgar Publishing. https://doi.org/10.4337/9781789900408.00013

Fares, M., Kutuzov, A., Oepen, S., & Velldal, E. (2017). Word vectors, reuse, and replicability: Towards a community repository of large-text resources. In J. Tiedemann & N. Tahmasebi (Eds.), Proceedings of the 21st Nordic Conference on Computational Linguistics (pp. 271–276). Association for Computational Linguistics. https://aclanthology.org/W17-0237

Google Scholar Help. (n.d.). Inclusion guidelines for webmasters . https://scholar.google.com/intl/en/scholar/inclusion.html

Gu, J., Wang, X., Li, C., Zhao, J., Fu, W., Liang, G., & Qiu, J. (2022). AI-enabled image fraud in scientific publications. Patterns , 3 (7), 100511. https://doi.org/10.1016/j.patter.2022.100511

Gusenbauer, M., & Haddaway, N. R. (2020). Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources. Research Synthesis Methods , 11 (2), 181–217.   https://doi.org/10.1002/jrsm.1378

Haider, J., & Åström, F. (2017). Dimensions of trust in scholarly communication: Problematizing peer review in the aftermath of John Bohannon’s “Sting” in science. Journal of the Association for Information Science and Technology , 68 (2), 450–467. https://doi.org/10.1002/asi.23669

Huang, J., & Tan, M. (2023). The role of ChatGPT in scientific communication: Writing better scientific review articles. American Journal of Cancer Research , 13 (4), 1148–1154. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10164801/

Jones, N. (2024). How journals are fighting back against a wave of questionable images. Nature , 626 (8000), 697–698. https://doi.org/10.1038/d41586-024-00372-6

Kitamura, F. C. (2023). ChatGPT is shaping the future of medical writing but still requires human judgment. Radiology , 307 (2), e230171. https://doi.org/10.1148/radiol.230171

Littell, J. H., Abel, K. M., Biggs, M. A., Blum, R. W., Foster, D. G., Haddad, L. B., Major, B., Munk-Olsen, T., Polis, C. B., Robinson, G. E., Rocca, C. H., Russo, N. F., Steinberg, J. R., Stewart, D. E., Stotland, N. L., Upadhyay, U. D., & Ditzhuijzen, J. van. (2024). Correcting the scientific record on abortion and mental health outcomes. BMJ , 384 , e076518. https://doi.org/10.1136/bmj-2023-076518

Lund, B. D., Wang, T., Mannuru, N. R., Nie, B., Shimray, S., & Wang, Z. (2023). ChatGPT and a new academic reality: Artificial Intelligence-written research papers and the ethics of the large language models in scholarly publishing. Journal of the Association for Information Science and Technology, 74 (5), 570–581. https://doi.org/10.1002/asi.24750

Martín-Martín, A., Orduna-Malea, E., Ayllón, J. M., & Delgado López-Cózar, E. (2016). Back to the past: On the shoulders of an academic search engine giant. Scientometrics , 107 , 1477–1487. https://doi.org/10.1007/s11192-016-1917-2

Martín-Martín, A., Thelwall, M., Orduna-Malea, E., & Delgado López-Cózar, E. (2021). Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: A multidisciplinary comparison of coverage via citations. Scientometrics , 126 (1), 871–906. https://doi.org/10.1007/s11192-020-03690-4

Simon, F. M., Altay, S., & Mercier, H. (2023). Misinformation reloaded? Fears about the impact of generative AI on misinformation are overblown. Harvard Kennedy School (HKS) Misinformation Review, 4 (5). https://doi.org/10.37016/mr-2020-127

Skeppstedt, M., Ahltorp, M., Kucher, K., & Lindström, M. (2024). From word clouds to Word Rain: Revisiting the classic word cloud to visualize climate change texts. Information Visualization , 23 (3), 217–238. https://doi.org/10.1177/14738716241236188

Swedish Research Council. (2017). Good research practice. Vetenskapsrådet.

Stokel-Walker, C. (2024, May 1.). AI Chatbots Have Thoroughly Infiltrated Scientific Publishing . Scientific American. https://www.scientificamerican.com/article/chatbots-have-thoroughly-infiltrated-scientific-publishing/

Subbaraman, N. (2024, May 14). Flood of fake science forces multiple journal closures: Wiley to shutter 19 more journals, some tainted by fraud. The Wall Street Journal . https://www.wsj.com/science/academic-studies-research-paper-mills-journals-publishing-f5a3d4bc

The pandas development team. (2024). pandas-dev/pandas: Pandas (v2.2.2) [Computer software]. Zenodo. https://doi.org/10.5281/zenodo.10957263

Thorp, H. H. (2023). ChatGPT is fun, but not an author. Science , 379 (6630), 313–313. https://doi.org/10.1126/science.adg7879

Tripodi, F. B., Garcia, L. C., & Marwick, A. E. (2023). ‘Do your own research’: Affordance activation and disinformation spread. Information, Communication & Society , 27 (6), 1212–1228. https://doi.org/10.1080/1369118X.2023.2245869

Vikramaditya, N. (2020). Nv7-GitHub/googlesearch [Computer software]. https://github.com/Nv7-GitHub/googlesearch

This research has been supported by Mistra, the Swedish Foundation for Strategic Environmental Research, through the research program Mistra Environmental Communication (Haider, Ekström, Rödl) and the Marcus and Amalia Wallenberg Foundation [2020.0004] (Söderström).

Competing Interests

The authors declare no competing interests.

The research described in this article was carried out under Swedish legislation. According to the relevant EU and Swedish legislation (2003:460) on the ethical review of research involving humans (“Ethical Review Act”), the research reported on here is not subject to authorization by the Swedish Ethical Review Authority (“etikprövningsmyndigheten”) (SRC, 2017).

This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided that the original author and source are properly credited.

Data Availability

All data needed to replicate this study are available at the Harvard Dataverse: https://doi.org/10.7910/DVN/WUVD8X

Acknowledgements

The authors wish to thank two anonymous reviewers for their valuable comments on the article manuscript as well as the editorial group of Harvard Kennedy School (HKS) Misinformation Review for their thoughtful feedback and input.

This week: the arXiv Accessibility Forum

Help | Advanced Search

Computer Science > Computation and Language

Title: can llms generate novel research ideas a large-scale human study with 100+ nlp researchers.

Abstract: Recent advancements in large language models (LLMs) have sparked optimism about their potential to accelerate scientific discovery, with a growing number of works proposing research agents that autonomously generate and validate new ideas. Despite this, no evaluations have shown that LLM systems can take the very first step of producing novel, expert-level ideas, let alone perform the entire research process. We address this by establishing an experimental design that evaluates research idea generation while controlling for confounders and performs the first head-to-head comparison between expert NLP researchers and an LLM ideation agent. By recruiting over 100 NLP researchers to write novel ideas and blind reviews of both LLM and human ideas, we obtain the first statistically significant conclusion on current LLM capabilities for research ideation: we find LLM-generated ideas are judged as more novel (p < 0.05) than human expert ideas while being judged slightly weaker on feasibility. Studying our agent baselines closely, we identify open problems in building and evaluating research agents, including failures of LLM self-evaluation and their lack of diversity in generation. Finally, we acknowledge that human judgements of novelty can be difficult, even by experts, and propose an end-to-end study design which recruits researchers to execute these ideas into full projects, enabling us to study whether these novelty and feasibility judgements result in meaningful differences in research outcome.

Submission history

Access paper:.

• HTML (experimental)
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

GLBC 1200.19 Women in American Pop Music at Webster University Library

Recommended databases, google scholar, how to recognize a scholarly, peer-reviewed article.

• Videos and Music
• Citations and Writing
• Services for Faculty and Staff
• Student Self-Assessment Survey
• Academic Search Premier (EBSCO) A scholarly multidisciplinary database of periodical articles, most with full-text, through Ebscohost.
• Fine Arts (Gale OneFile) Fine Arts and Music Collection provides access to scholarly journals and magazines that support research in areas including drama, music, art history, and film making. The database emphasizes full-text content for publications included in the Wilson Art Index and RILM bibliography.
• Biography (Gale in Context) A comprehensive biographical reference database that provides more than 300,000 narrative biographies on noteworthy individuals from around the world, throughout history and across all disciplines and subject areas. Information is drawn from nearly 80 Gale biographical sources such as Contemporary Authors, Dictionary of American Biography, Scribner's Encyclopedia of American Lives, Encyclopedia of World Biography, as well as full-text magazine and newspaper articles.

Google Scholar is a simple tool to search for scholarly (peer-reviewed) articles, and see how many times an article has been cited in other research. And when you hit a paywall, it can connect to Webster University Library's resources. If you are using a computer with an off-campus IP address, follow these instructions to connect Google Scholar to library resources:

• Go to Google Scholar
• Click on Settings.

What is a scholarly, peer-reviewed journal article?

Scholarly articles are sometimes "peer-reviewed" or "refereed" because they are evaluated by other scholars or experts in the field before being accepted for publication.  A scholarly article is commonly an experimental or research study, or an in-depth theoretical or literature review. It is usually many more pages than a magazine article.

The clearest and most reliable indicator of a scholarly article is the presence of references or citations. Look for a list of works cited, a reference list, and/or numbered footnotes or endnotes. Citations are not merely a check against plagiarism. They set the article in the context of a scholarly discussion and provide useful suggestions for further research. 

Many of our databases allow you to limit your search to just scholarly articles. This is a useful feature, but it is not 100% accurate in terms of what it includes and what it excludes. You should still check to see if the article has references or citations.

The table below compares some of the differences between magazines (e.g. Psychology Today) and journals (e.g Journal of Abnormal Psychology).

How to find scholarly, peer-reviewed articles

• FAQ: How do I find peer-reviewed or scholarly articles?
• FAQ: How can I tell if an article is peer-reviewed?
• << Previous: Home
• Next: Books >>
• Last Updated: Sep 16, 2024 10:59 AM
• URL: https://library.webster.edu/womeninpop