Using Grey Literature in your Research: Grey Literature Searches - Methods

Google provides access to a huge number of potential grey literature sources, and does not require a priori knowledge of where on the internet they are located.

Weaknesses:

• Poor reproducibility: Search results depend on browser version, search history, and geographic location.

• Search results are too numerous to screen, so a stopping rule is needed.

• Search strings cannot be too long or complex (max 150 words, does not support parentheses), so will need to simplify and possibly run multiple searches with different combinations of terms and document each one.

Tips:

• See also this e-learning module from Radboud University, Smarter Searches with Google.

• Using the advanced search function to filter can greatly increase the proportion of relevant results:

Google Scholar indexes any webpages that meet its specific criteria (e.g. includes title page, author list, reference list) and contains a large amount of grey literature as well as traditional publications from all fields of study.

Weaknesses:

• Just as for Google, search strings cannot be too long or complex (max 256 characters).
• Displays only the first 1000 search results and there is a lack of transparency about search result rankings.
• Higher proportion of duplicate records compared to traditional academic databases [1].
• Contains both grey and traditional academic literature.

Tips

• The top-ranked Google Scholar search results consist mainly of traditional publications, whereas the proportion of grey literature peaks much later: In one study the grey literature peak was around page 35 of results [1]. Thus, search well beyond the 30^th page of results to maximize the number of relevant grey literature sources.
• Searching based on the title only, rather than the full text, may deliver a greater proportion of grey literature [1]. You can check the “title only” option using Advanced Search.

See this list of databases as a starting point.

Weaknesses:

• Each database has different search capabilities, so you will need to adapt your search strings for each one and document the changes.
• There is not a lot of overlap between grey literature databases; searching several databases is important for adequate coverage [2].

Experts can point you in the direction of relevant grey literature sources, including specialist websites, organizations, and unpublished or ongoing studies.

Tips:

• For the broadest coverage, contact experts from within and also outside of academia (e.g. policy, industry, practice).
• Personal communications and general calls on email listservs or social media are the most typical ways to take this step.
• It is best to reach out to experts early in the search process, because responses may be slow and the advice you receive can help shape your strategy.

Hand-searching involves browsing a website or printed material to find relevant documents. Researchers who use this method often report that it delivers large number of unique documents [e.g., 2,3].

Weaknesses:

• Hand-searching is time-consuming.

Tips for identifying websites of interest:

• Google search for organizations producing relevant grey literature in your topic area. Be sure to document this search for reproducibility.
• Ask experts to recommend organizations (see box above).
• Look at the proceedings of past conferences in your field of study, which are often not well-indexed in databases [4].
• As a starting point, here is a list of organizations that produce grey literature in a variety of fields.

For a detailed discussion on searching websites, see this 2016 paper by Stansfield, Dickson and Bangpan [5].

For a set of key sources, look at both their reference lists and the articles that cited them, using, for example, Web of Science or Google Scholar. Relevant grey literature is often referred to within traditional articles, as well as in other grey literature.

This method is also known as Snowballing – see this Radboud University guide on Literature Searching at the Nijmegen School of Management for more details.