Once the search results are obtained the analysis phase of the systematic review commences where each paper must be judged on validity and quality. The decisions on which papers to include and exclude are based on the eligibility criteria specified in the protocol. 

Best Practice Tip

It is recommended that you use the PRISMA flow diagram to record the number of search results from each database and the changes to those numbers as the screening process is conducted. This diagram should be included in the reported findings of the systematic review. 

What is PRISMA?
PRISMA is the Preferred Reporting Items for Systematic reviews and Meta-Analyses. PRISMA is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. 

Screening papers 

The screening process of a systematic review will usually involve more than one reviewer, with the reviewers assessing each paper against the eligibility criteria for inclusion or exclusion from the review as defined in the protocol. 

Each reviewer will need to provide a rationale for how the papers match the eligibility criteria or have some value to the systematic review. This is done separately by each reviewer to ensure minimal bias. The results are then compared. 

The screening is done incrementally in two phases. An agreement must be reached by all reviewers, at each phase, on which papers are to be included and excluded.

Phase 1 – Titles and abstracts of each paper are assessed against the eligibility criteria.

Phase 2 – The process is repeated where the full text of each of the remaining papers are assessed against the eligibility criteria.

The remaining papers are those that will contribute to the systematic review. 

Critical appraisal 

The critical appraisal process examines the validity and applicability of the studies. It is important to critically evaluate the literature to: 

• assess the benefits and strengths of the research against flaws and weaknesses 
• decide whether studies have been undertaken in a way that makes their findings reliable 
• make sense of the results
• know what these results mean in the context of the clinical decision being made
• assess the usefulness of the evidence for clinical decisions. 

There are several checklists available to assist in the process of determining the quality of the studies.  A checklist should be chosen to assess sources of bias that are likely to affect results in relation to the research question.

Take a look at the following sites for examples of critical appraisal tools that can be used to assess the quality, validity, and bias of papers.  

• CASP (Critical Appraisal Skills Programme) Checklists 
• Joanna Briggs Critical Appraisal Tools 
• Risk of bias assessment tools from the University of Bristol