Systematic reviews and meta-analysis are the paramount level of evidence, based on which clinical guidelines are formatted. Furthermore, they guide clinicians to provide optimal evidence-based care to their patients (Gopalakrishnan and Ganesh Kumar, 2013).

Systematic Reviews are constructed to identify, appraise and summarize all the available evidence in accordance with a pre-determined criterion to answer a focused question.

Systematic reviews form the highest level of evidence (Phillips et al 2008, Evans 2003, Fleisher et al 2005 and Guyatt et al 2000) and are imperative to the dynamic changes in medicine. An understanding and implementation of systematic reviews is mandatory for all healthcare professionals.

It is important that abstracts of a Systematic Review should ideally adhere to the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) guidelines.

According to the Cochrane Collaboration the researchers must follow a strict and explicit methodology in order to minimize bias and produce reliable findings that can be used in decision making. The standard of systematic reviews and meta-analysis published from 2009-2016 has been described as “medium” thereby affecting the Evidence based practice in Endodontics (Kattan et al 2018).

The aim of this editorial is to direct authors writing reviews to produce manuscripts with minimal issues and the ease of understanding by the reader.

WHAT IS THE AIM OF A SYSTEMATIC REVIEW?

• A Presentation that enables clear understanding by the In a systematic review the criteria for including or excluding a study are clearly outlined such that the reader can analyze the quality of the review, methodology and the potential for bias.
• To guide clinical practice by providing high-quality research evidence
• To aid research proposals
• Apart from identifying, appraising studies and summarizing results, systematic reviews identify reasons for disparity across studies and indicate limitations of current
• Satisfactory Methodology and Reporting Criteria

TYPES OF SYSTEMATIC STUDIES

• Qualitative- summarized results of primary studies not arranged in a statistical format.
• Quantitative-also known as meta-analysis. Aggregated primary studies are arranged

Depending on the nature of the data, the results of a systematic review can be summarized in text or graphic form. In graphic form, it is common for different trials to be depicted in a plot where the point estimate and 95% confidence interval for each study are presented on an individual line. When results are mathematically combined (a process sometimes referred to as pooling), this is referred to as meta-analysis. Graphically, the pooled result is often presented as a diamond at the bottom of the plot [1-6].

STRENGTHS OF A SYSTEMATIC REVIEW

• Systematic reviews are vital for evidence-based medicine as they help healthcare professionals make vital decisions regarding patients care. These decisions combine the preferences and expectations of the patients along with the clinicians’ personal experience in treating patients and best available research evidence in the literature. Thus, healthcare policies are designed by the review of available evidence [7]. Referring to a well conducted systematic review is a good way to familiarize with the best available research evidence for a focused clinical question.
• Systematic reviews can also help us to establish whether the findings of various primary studies are consistent and can be applied across various population groups.
• Reduction of false positive or false negative results leading to prompt introduction of treatments [8].
• Hence, combining the data from multiple well-conducted randomized control trials will provide a more accurate estimate of the effect (under investigation) rather than an individual study. When this data is combined, the size of the overall sample is increased, enhancing the statistical power of the analysis and as a result the size of the confidence interval is reduced. It is also easier to communicate a pooled summary of the clinical effect being investigated rather than to describe the results for each of the individual studies. Sometimes, if the treatment effect in small trials shows a non-significant trend toward efficacy, then pooling the results may establish the benefits of therapy [9].
• A useful tool towards research funding enabling us to investigate what we know and what we don’t [10]!

SYSTEMATIC REVIEW METHODOLOGY

A good systematic review is the one which has a clear methodology. The methodology involves the following steps.

Formatting a focused question

There should be clear instructions on the question to be answered with inclusion and exclusion criteria. Framework by [11] Hassing 1999 in formatting a question (PICO/PECO) is as follows.

Patient Population

E/I Exposure/Intervention Comparison:

Outcome: Modifications are added if any changes are made to the protocol such as defining populations, outcomes, interventions or study designs.

Inclusion and Exclusion Criteria: Exclusion and inclusion criteria should be determined related to the question cautious of the introduction of any bias.

THE SEARCH BEGINS

Commonly used are Medline/PubMed, Web of Science, Embase, Metapress, Scirus, Biosis, eTblast, Cochrane databases of randomized trials or systematic reviews, Google Scholar, DARE (Database of Abstracts of Reviews of Effectiveness), HTA (Health Technology Assessment Database, LILACS (language specific database) and SIGLE (System for information on grey literature in European Achieve).

Search should be exhaustive using multiple resources online and printed without language barriers.

QUALITY ASSESSMENT

Quality assessment and minimizing bias is an integral part of the systematic review. Some points to be kept in mind are acknowledging all the evidence and seeking expert opinion.

These quality assessments are used for analyzing heterogeneity and decisions regarding suitability of meta-analysis. They will also aid in assessment of interference and future recommendations.

EVIDENCE SUMMARY

Synthesis of the data consists of tabulation of the characteristics of the study, quality and the use of statistics to explore differences between various studies and their combined effects.

Synthesis and summarizing of the evidence should be done by reviewers’ independently [12]. Advance planning should be implemented for determining heterogeneity. Failure to format a meta-analysis should be promptly replaced by a sub-meta-analysis.

RESULTS OF SYSTEMATIC REVIEW

Publication bias should be explored. Heterogeneity should be explored to determine if the summary can be trusted, if not, the effects generated in high-quality studies should be utilized for generating inferences.

SYSTEMATIC REVIEWS IN ENDODONTICS

The published data in dentistry is increasing exponentially. However, it is difficult for the clinician to ascertain if the information is accurate and valuable. Systematic reviews are a remedy to this problem as they are ranked as the highest level of evidence. Unfortunately, emphasis is currently being placed on reviewing the quality of these reviews and making dentists’ aware of the shortcomings of some of these reviews.

LIMITATIONS OF SYSTEMATIC REVIEWS

• Systematic reviews are subject to flaws. Bias can be introduced at any step leading to inaccuracy. Examples are bias based on selection criteria, performance, attrition and detection. [13] stated that some studies lacked a concrete source of bias.
• A good systematic review may be rendered useless if it is based on a poorly conducted research and hence it is important that reviews must be well-versed with the research methodology and have strict and carefully designed criteria for studies to be included in the systematic review.
• “Garbage in, garbage out” a phrase used to describe poorly conducted systematic studies. If meta-analysis includes a misleading original study then the conclusion should expect criticism.
• “Quality assessment affects the estimate”. Methodological criteria tailed to the type of study should be used for quality assessment.
• “Mixing apples and oranges”. Lack of homogeneity in the studies can cause the readers to apply the conclusions to their clinical practice without considering the differences which can be hazardous. Some authors expressed their concern regarding modifications that made comparison impossible.
• Most of the systematic reviews are based on published data with publication bias towards studies which show significant differences in results; uninteresting studies are usually not published. According to the oxford center of evidence-based medicine the importance of publication bias must be taken into consideration. The exact frequency of publication bias remains unknown.
• Inappropriate data handling can lead to misleading conclusions. For example, assumption that dropouts are always clinical failures is not true!
• Finally, important consideration must be given to the design of clinical protocols to rule out bias towards a treatment. This may impact on the final result and reinforce already existing biased conclusion.
• Some studies did not publish conclusions!

Some authors have questioned the quality of systematic studies in endodontics. Spanberg [13] stated that good endodontic literature is rare and will take some time to improve. To support this statement, he included three studies which failed in their objectives and were doubted as being a part of the literature. In response to this letter, Sathorn blamed the editors and reviews for publishing these studies.

An assessment into the methodology of systematic reviews in dentistry carried out by Glenny [14] highlighted that a substantial number of the reviews were substandard and the conclusions may be misguiding to dental professionals. The author reviewed 65 systematic reviews between 1990-2001 and found serious shortcomings in the search criteria, quality assessment of studies, no examination of heterogeneity and inappropriate pooling of data.

A review of the quality of 16 meta-analysis reports in endodontology published between 2001-2009 showed that the published meta-analysis had high AMSTAR scores [quality scoring system for systematic reviews] in most of the areas but weaker areas included failure to disclose the conduct of assessment of publication bias. An editorial: Is it time to recycle “garbage in-garbage out” [15] systematic reviews pointed the shortcomings in the above review especially with the inappropriate use of AMSTAR scoring system (as discussed below).

Endodontics are working hard to publish systematic reviews and meta-analysis which is useful for the practitioners. But is this a global trend towards good quality research?

This has highlighted the importance of availability of rigorous quality assessment tools for assessment of systematic reviews to ensure that dental care professionals are provided with highest level of evidence regarding an intervention which would enable them to make essential decisions regarding patient care.

QUESTIONS TO BE ASKED WHEN INTERPRETING AS SYSTEMATIC REVIEW [12].

1. Does the objective of this review answer my question?
2. What were the search methods? Were they comprehensive?
3. Any additional research that has been done since then?
4. What were the selection criteria for the studies?
5. What was done to avoid bias?
6. What were the criteria to validate the studies?
7. Are the conclusions and results the same?

QUALITY OF SYSTEMATIC REVIEWS

Mohar [16] defined quality as a plausibility that the design of a systematic review will bring about unbiased results.

Quality assessment of systematic reviews ranges from a simple checklist which was developed in 1990s to a complex and cumbersome checklist known as Overview Quality Assessment Questionnaire (OQAQ).

This was followed by the release of Quality of Reporting of Meta-analyses (QUOROM) statement in 1996 by Moher et al and the implementation of QUOROM tool which is a combination of a flow chart and checklist to assess the. quality of a meta-analysis.

QUOROM tools saw an improvement in the quality of systematic reviews and meta-analysis. The release of Consolidated Standards of Reporting Trials (CONSORT) statement by Begg [17] saw a similar improvement in the quality of studies being conducted which empowered the clinician with best available evidence. The increased attention of the journal editors and reviewers to the general methodological quality of reports also played a role. The AMSTAR tool is currently the most widely used tool in assessing the quality of methodology of systematic reviews.

AMSTAR TOOL

The AMSTAR tool (Table 1 (Appendix 1)) which is currently the most widely used tool in assessing the quality of methodology of systematic was developed in 2007. This is a 11-item questionnaire each item is given a score of 1 if the criteria are met and score of 0 if criteria is not met, is unclear, or is not applicable.

AMSTAR quality assessments are divided into three ranges

1. High (9-11)
2. Medium (5-8)
3. Low (0-4)

It provides a summary score, which is helpful for clinicians making decisions [18].

According to Shea [19], careful psychometric assessments determined that AMSTAR has good face and content validity for measuring the methodological quality of systematic reviews and their clinical relevance.

As of this date, AMSTAR has been used by professional health care associations and other policy institutions, and has gained in respectability, reliability, reproducibility. However, AMSTAR has only been tested on randomized control trials evaluating treatment interventions its use is limited to evaluate diagnostic, prognostic studies.

There have been concerns regarding inappropriate use of AMSTAR in evaluating systematic reviews in Endodontology. In a review of meta-analysis using AMSTAR Suebnukarn 2010 revealed that the meta-analysis of various studies in endodontics had high AMSTAR scores. He was criticized for the inappropriate use of AMSTAR, in observational studies, non-randomized controlled trials [20].

CONCLUSION

In the current flood of publications, a great volume of systematic reviews and meta- analysis are available. Systematic reviews have become prominent in endodontics due to the rapidly growing emphasis on evidence-based medicine [21-23].

Ironically not all of the systematic reviews are of high quality. Hence readers must develop a critical attitude to interpret various publications.

Furthermore, clinicians performing systematic reviews and meta-analysis should stay focused to the question, have a vigorous protocol in place, thoroughly investigate clinical and statistical heterogeneity with attempts to minimize bias [24-27].

Journal reviewers and editors shouldn’t overlook poorly conducted reviews.

The development of AMSTAR tool has provided us with the ability to assess the methodological quality of the systematic review which is an essential determinant in the overall quality of a systematic review. This has helped the clinician to use evidence-based medicine in everyday practice [20].

Systematic reviews are powerful toots that influence clinical arena. On the contrary if improperly performed can be misleading.

With the increasing awareness of systematic reviews is this an awakening of a new trend towards good quality endodontics?

1. Gopalakrishnan S, Ganeshkumar P (2013) Systematic reviews and meta-analysis: Understanding the best evidence in primary healthcare. J Family Med Prim Care 2: 9-14.
2. Phillips B, Ball C, Sackett D (2008) Oxford Centre for Evidence-Based Medicine Levels of Evidence.
3. Evans D (2003) Hierarchy of evidence: A framework for ranking evidence evaluating healthcare interventions. J Clin Nurs 12(1): 77-84.
4. Fleisher LA, Bass EB, McKeown P (2005) Methodological approach: Guidelines for management of atrial fibrillation post-cardiac surgery. Chest 128.
5. Guyatt GH, Haynes RB, Jaeschke RZ, Cook DJ, Green L, et al. (2000) Users' Guides to the Medical Literature: XXV. Evidence-based medicine: principles for applying the Users' Guides to patient care: Evidence-Based Medicine Working Group. JAMA 284(10): 1290-1296.
6. Kattan S, Lee SM, Kohli MR, Setzer FC, Karabucak B (2018) Methodological quality assessment of meta-analyses in Endodontics. J Endod 44: 22-31.
7. Garg AX, Hackam D, Tonelli M (2008) Systematic review and meta- analysis: When one study is just not enough. Clin J Am Soc Nephrol 3: 253-260.
8. Zwahlen M, Renehan A, Egger M (2008) Meta-analysis in medical research: Potentials and limitations. Urol Oncol 26: 320-329.
9. Pogue J, Yusuf S (1998) Overcoming the limitations of current meta-analysis of randomized controlled trials. Lancet 351: 47-52.
10. Yuan Y, Hunt RH (2009) Systematic Reviews: The Good, the Bad, and the Ugly. Am J Gastroenterol 104: 1086-1092.
11. Hassig RA (1999) Evidence-based medicine. University of Connecticut Health Center. V.XIII, ¾.
12. Clarke M (2008) Interpreting the results of systematic reviews. Semin Hematol 45(3): 176-180.
13. Spångberg LS (2007) Systematic reviews in endodontics-examples of GIGO. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 103: 723-724.
14. Glenny AM, Esposito M, Coulthard P, Worthington HV (2003) The assessment of systematic reviews in dentistry. Eur J Oral Sci 111: 85-92.
15. Fedorowicz Z, Byron PS (2011) Climate change in endodontics: Is it time to recycle “garbage in–garbage out” systematic reviews? Oral Surg Oral Med Oral Pathol Oral Radiol Endod 112: 515-517.
16. Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, et al. (1995) Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials 16: 62-73.
17. Begg C, Cho M, Eastwood S, Horton R, Moher D, et al. (1996) Improving the quality of reporting of randomised controlled trials: The CONSORT statement. JAMA 276(8): 637-639.
18. Byroni PS, Fedorowicz Z, Jagannath VA, Sharif MO (2011) An AMSTAR assessment of the Methodological interventions published in the Journal of Applied Oral Science. J Appl Oral Sci 19(5): 440-447.
19. Shea BJ, Hamel C, Wells GA, Bouter LM, Kristjansson E (2009) AMSTAR is a valid and reliable measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol 62(10): 1013-1020.
20. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, et al. (2007) Development of AMSTAR: A: Measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 7: 10.
21. Liesegang TJ (2001) Evidence-based medicine: Principles for applying the users’ guides to patient care. Evidence-based medicine working group. Am J Ophthalmol 284(10): 1290-1296.
22. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, et al. (1999) Improving the quality of reports of meta-analyses of randomized controlled trials: the QUOROM statement. Lancet 354: 1896-1900.
23. Palma S, Rodriguez MD (2005) Assessment of publication bias in meta-analyses of cardiovascular diseases. J Epidemiol Commun Health 59: 864-869.
24. Khan KS, Kunz R, Kleijne J, Antes G (2003) Five steps to conducting a systematic review. J R Soc Med 96: 118-121.
25. Oxford (2011) Centre for Evidence-Based Medicine. (OCEBM). OCEBM Levels of Evidence Working Group. The Oxford 2011 Levels of Evidence. Available online at: https://www.cebm.ox.ac.uk/resources/levels-of-evidence/ocebm-levels-of-evidence
26. Suebnukarn S, Ngamboonsirisingh S, Rattanabanlang A (2010) A systematic evaluation of the quality of meta-analyses in endodontics. J Endod 36(4): 602-608.
27. Whiting P, Rutjes AWS, Reitsma JB, Bossuyt PMM, Kleijnen J (2003) The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol 3: 2-13.