Evidence-informed decision-making (EIDM) uses a variety of tools to support the policy-making process including evidence maps and systematic reviews. These tools allow researchers to synthesise research and other evidence into policy-relevant frameworks. Both evidence maps (EM) and systematic reviews (SR) require the expertise of sector professionals, methodology experts, and information or search specialists. This blog explains the role of the information/search specialist in the production of EMs and SRs.
Systematic reviews and evidence maps
In their 2005 book, ‘Systematic reviews need systematic searchers’, McGowan and Sampson describe an SR as “a review that uses systematic and explicit methods to identify, select, and critically appraise relevant research and to collect and analyse data from the studies that are included in the review”. While Miake-Lye and colleagues define EMs in their 2016 article ‘What is an evidence map?’ define an EM “An evidence map is a systematic search of a broad field to identify gaps in knowledge and/or future research needs that presents results in a user-friendly format, often a visual figure or graph, or a searchable database”. From the above, we see that both SR and EM focus on systematically gathering all relevant research and other studies that are available on a specific topic. The search specialist plays a critical role in the first phase of any synthesis project as errors in the search process potentially result in a biased or otherwise incomplete evidence base. Hence, the success of both EM and SR hinge on the success of the initial phase of searching.
Search specialists
Search specialists usually have some kind of background in library and information management so they are experts at finding and organising material. Unlike traditional library work where librarians assist individual users with library information queries, a search specialist on an evidence synthesis project will work with multiple parties including sector professionals, methodology experts, and in some cases policy-makers. This diverse array of actors is collectively called the review team.
Steps in the searching phase of SR and EM
There are four steps to ensuring a systematic search as the foundation of a SR or EM.
- The first step is to define the concepts used in the systematic review/evidence map and then to develop the search strategy in consultation with the review team.
- The second step is to identify the key sources of information, including grey literature and academic databases. Searches should be run on a number of different databases; there is no set number of databases as the search sources will depend on the specific topic. There will be some overlap between the databases but it is important to search more than one database as the coverage needs to be complete, and different databases index different articles and each database has a different way of indexing.
- The third step is to develop the search strings. This is an iterative process which involves extensive consultation with the review team. The search specialist will develop the search strings based on the keywords and topics that the review team put together. S/he then runs a pilot search and identifies tricky terms that need to be refined, or finds gaps in the literature search that show that the search string is not strong enough to gather all the relevant research that is available. Typically a number of relevant articles are identified by the review team and are used to test the search strategy, i.e. if the search strategy recalls these key studies then it is on the right track.
- Unlike a traditional literature search, a SR or EM search is more stylised. The final step in the search phase that a search specialist is involved in is to run the finalised search strings on multiple databases and extracted using citation management software such as Endnote or Refworks. The cited software platforms are able to reference from different sources and convert references to a common format, removing duplicate references, and saving citations in a database.
Once the search strings are finalised, the Information Specialist has no need to review the searches. They export them into the reference management software and share these with the review team. The researchers are then responsible for taking the research forward by looking at what studies should be included or excluded.
From the above process, we see that the role of the search specialist throughout the search process is to record the search strategies used, databases searched, dates of coverage provided by each database at the time of the search, and the number of items downloaded from each database. The two reasons for this exhaustive process are 1) transparency: other review teams should be able to satisfy themselves that the review is not open to bias; and 2) reproducibility: other review teams should be able to replicate the methods and arrive at the same results.
What to look out for in an information specialist
Engaging - As a search specialist, it is critical to engage with the sector experts to understand what the key concepts are within the search and from there to develop search strings which are refined enough to exclude irrelevant material but broad enough to capture any and all research articles that cover the topic. This is the most fundamental expertise to search for when looking for an information specialist.
Helpful - In general, search specialists confine themselves to running the searches on academic databases and do not do searches for grey literature. Sector experts typically get involved in grey literature searching as they are more able to decide on the relevance of a particular report or study. However, the search specialist can guide and assist in the searching process.
Precision - Search specialists need to be precise and thorough in their work. In order for a search to be successful, search specialist need to be able to work with conceptual issues, while simultaneously being able to focus on small details (for example, spelling in the search strings, use of inverted commas, etc.) because one mistake can render the whole search invalid.
The role of a search specialist is key in the first phases of developing successful EM and SR. Their hard work at the beginning of the process makes the rest of the process (sorting and appraising) stronger and easier to do.