Methods

Canadian Diabetes Association Clinical Practice Guidelines Expert Committee

Gillian Booth MD, MSc, FRCPC Alice Y.Y. Cheng MD, FRCPC

  • Key Messages
  • Recommendations
  • Figures
  • Full Text
  • References

Process

Following the process used to develop previous Canadian Diabetes Association clinical practice guidelines (1,2), an Executive Committee, Steering Committee and Expert Committee with broad expertise and geographic representation were assembled. In total, 120 volunteers, including health professionals from family medicine, endocrinology, internal medicine, infectious disease, neurology, nephrology, cardiology, urology, psychology, obstetrics, ophthalmology, pediatrics, nursing, dietetics, pharmacy, exercise physiology and others, as well as people with diabetes, participated in the guideline development process.

The following basic principles were adopted to ensure that the values and empirical basis underlying each recommendation were explicitly identified and to facilitate the critical scrutiny and analysis of each recommendation by other organizations and individuals.

Elements covered by the Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument were incorporated into the guideline development process.

  • Each recommendation had to address a clinically important question related to 1 or more of the following: detection, prognosis, prevention or management of diabetes and its sequelae. Health benefits, risks and side effects of interventions were considered in formulating the recommendations. Patient preferences and values were sought from expert panel members with diabetes and the literature (where available).
  • Whenever possible, each recommendation had to be justified by the strongest clinically relevant, empirical evidence that could be identified; the citation(s) reporting this evidence had to be noted adjacent to the relevant guideline.
  • The strength of this evidence, based on prespecified criteria from the epidemiological literature and other guidelines processes, had to be noted (3–8) .
  • Each recommendation had to be assigned a grade based on the available evidence, its methodological strength and its applicability to the Canadian population.
  • Each recommendation had to be approved by the Steering Committee and Executive Committee, with 100% consensus.
  • Guidelines based on biological or mechanistic reasoning, expert opinion or consensus had to be explicitly identified and graded as such; harmonization was sought with other Canadian guideline bodies, including the Canadian Cardiovascular Society (CCS), the Canadian Hypertension Education Program (CHEP), the Canadian Cardiovascular Harmonization of National Guidelines Endeavour (C-CHANGE) and the Society of Obstetricians and Gynecologists of Canada (SOGC).

Identifying and Appraising the Evidence

Authors for each chapter were assembled based on their relevant fields of expertise. Each chapter had 1 lead author, 1 or 2 “evidence resource” persons trained or experienced in clinical epidemiology or clinical research methodology, and additional authors, as needed. At the outset of the process, committee members from each section of the guidelines attended a workshop on evidence-based methodology, in order to ensure a consistent approach to the development of recommendations. Committee members identified clinically important questions related to diagnosis, prognosis, prevention and treatment of diabetes and its complications, which were used as a basis for our literature search strategy (outlined below).

Authors were to explicitly define A) the population to which a guideline would apply; B) the test, risk factor or intervention being addressed; C) the “gold standard” test or relevant intervention to which the test or intervention in question was compared; and D) the clinically relevant outcomes being targeted. This information was used to develop specific, clinically relevant questions that were the focus of literature searches. For each question, individual strategies were developed combining diabetes terms with methodological terms. A librarian with expertise in literature reviews performed a comprehensive search of the relevant English-language, published, peer-reviewed literature using validated search strategies ( http://hiru.mcmaster.ca/hiru/ ) of electronic databases (MEDLINE, EMBASE, CINAHL, the Cochrane Central Register of Trials, and PsycINFO [where appropriate]). This was complemented by the authors' own manual and electronic searches.

For topics that were covered in the 2008 guidelines, the literature searches focused on new evidence published since those guidelines, including literature published in September 2007 or later. For new topics, the search time frame included the literature published since 1990 or earlier, where relevant. Updated literature searches were performed at regular intervals throughout the development process.

Key citations retrieved from the literature searches were then reviewed. Each citation that was used to formulate or revise a recommendation was assigned a level of evidence according to the prespecified criteria in Table 1, reflecting the methodological quality of the paper. When evaluating papers, authors were required to use standardized checklists that highlighted the most important elements of a well-conducted study. The level of evidence was then determined by the cited paper's objectives, methodological rigour, susceptibility to bias and generalizability ( Table 1 ). Because they could not be critically appraised, meeting abstracts, narrative review articles, news reports and other sources could not be used to support recommendations. Papers evaluating the cost effectiveness of therapies or diagnostic tests also were not included.

A number of considerations were made when evaluating the evidence within a given area. For example, people with diabetes are at high risk for several sequelae that are not exclusive to diabetes (e.g. cardiovascular disease, renal failure and erectile dysfunction). As such, some evidence relating to these problems was identified that either excluded, did not report on or did not focus on people with diabetes.

Whenever such evidence was identified, a level was assigned using the approach described above. Higher levels were assigned if A) people with diabetes comprised a predefined subgroup; B) the results in the diabetes subgroup were unlikely to have occurred by chance; and C) the evidence was generated in response to questions that were formulated prior to the analysis of the results. Lower levels (than those indicated in Table 1 ) were assigned to evidence that did not meet these criteria.

Table 1
Criteria for assigning levels of evidence to the published studies
RCT, randomized, controlled trial.
∗ In cases where such blinding was not possible or was impractical (e.g. intensive vs. conventional insulin therapy), the blinding of individuals who assessed and adjudicated study outcomes was felt to be sufficient.
Level Criteria
Studies of diagnosis
Level 1
  1. Independent interpretation of test results (without knowledge of the result of the diagnostic or gold standard)
  2. Independent interpretation of the diagnostic standard (without knowledge of the test result)
  3. Selection of people suspected (but not known) to have the disorder
  4. Reproducible description of both the test and diagnostic standard
  5. At least 50 patients with and 50 patients without the disorder
Level 2 Meets 4 of the Level 1 criteria
Level 3 Meets 3 of the Level 1 criteria
Level 4 Meets 1 or 2 of the Level 1 criteria
Studies of treatment and prevention
Level 1A
  1. Comprehensive search for evidence
  2. Authors avoided bias in selecting articles for inclusion
  3. Authors assessed each article for validity
  4. Reports clear conclusions that are supported by the data and appropriate analyses
OR
Appropriately designed RCT with adequate power to answer the question posed by the investigators
  1. Patients were randomly allocated to treatment groups
  2. Follow-up at least 80% complete
  3. Patients and investigators were blinded to the treatment
  4. Patients were analyzed in the treatment groups to which they were assigned
  5. The sample size was large enough to detect the outcome of interest
Level 1B Nonrandomized clinical trial or cohort study with indisputable results
Level 2 RCT or systematic overview that does not meet Level 1 criteria
Level 3 Nonrandomized clinical trial or cohort study; systematic overview or meta-analysis of level 3 studies
Level 4 Other
Studies of prognosis
Level 1
  1. Inception cohort of patients with the condition of interest but free of the outcome of interest
  2. Reproducible inclusion/exclusion criteria
  3. Follow-up of at least 80% of subjects
  4. Statistical adjustment for extraneous prognostic factors (confounders)
  5. Reproducible description of outcome measures
Level 2 Meets criterion a) above, plus 3 of the other 4 criteria
Level 3 Meets criterion a) above, plus 2 of the other criteria
Level 4 Meets criterion a) above, plus 1 of the other criteria

Guideline Development

Expert Committee members evaluated the relevant literature, and guidelines were developed and initially reviewed by the Expert Committee. In the absence of new evidence since the publication of the 2008 clinical practice guidelines, recommendations from the 2008 document were not changed.

The studies used to develop and support each recommendation are cited beside the level of evidence. In some cases, key citations that influenced the final recommendation were not assigned the same level of evidence but rather were of varying levels of evidence. In those circumstances, all relevant studies were cited, regardless of the grading assigned to the recommendation. The final grading depended on the overall evidence available, including the relative strengths of the studies from a methodological perspective and the studies' findings. Studies with conflicting outcomes were also considered and cited in the final recommendation where relevant. Further details on the grading process are described below.

Finally, several treatment recommendations were based on evidence generated from the use of one therapeutic agent from a given class (e.g. one of the statins). Whenever evidence relating to 1 or more agents from a recognized class of agents was available, the recommendation was written so as to be relevant to the class, but specifically studied therapeutic agents were identified within the recommendation and/or cited reference(s). Only medications with Health Canada Notice of Compliance granted by February 15, 2013 were included in the recommendations.

Grading the Recommendations

After formulating new recommendations or modifying existing ones based on new evidence, each recommendation was assigned a grade from A through D ( Table 2). The highest possible grade that a recommendation could have was based on the strength of evidence that supported the recommendation (i.e. the highest level of evidence assigned to studies on which the recommendation was based). However, the assigned grading was lowered in some cases, for example, if the evidence was found not to be applicable to the Canadian population or, if based on the consensus of the Steering and Executive Committees, there were additional concerns regarding the recommendation. In some situations, the grading also was lowered for subgroups that were not well represented in the study or in whom the beneficial effect of an intervention was less clear. Grading also was lowered if the findings from relevant (and equally rigorous) studies on the topic were conflicting. Thus, a recommendation based on Level 1 evidence, deemed to be very applicable to Canadians and supported by strong consensus, was assigned a grade of A. A recommendation not deemed to be applicable to Canadians, or judged to require further supporting evidence, was assigned a lower grade. Where available, the number of patients that would need to be treated in order to prevent 1 clinical event (number needed to treat [NNT]) or to cause an adverse event (number needed to harm [NNH]) was considered in assessing the impact of a particular intervention. The degree to which evidence derived from other populations was felt to be relevant to diabetes also was reflected in the wording and grading of the recommendation. Finally, in the absence of Level 1, 2 or 3 supporting evidence, or if the recommendation was based on the consensus of the Steering and Executive Committees, the highest grade that could be assigned was D.

Table 2
Criteria for assigning grades of recommendations for clinical practice
Grade Criteria
Grade A The best evidence was at Level 1
Grade B The best evidence was at Level 2
Grade C The best evidence was at Level 3
Grade D The best evidence was at Level 4 or consensus

Interpreting the Assigned Grade of a Recommendation

The grade assigned to each recommendation is closely linked to the methodological rigour and robustness of the relevant clinical research. Therefore, as noted above, a high grade reflects a high degree of confidence that following the recommendation will lead to the desired outcome. Similarly, a lower grade reflects weaker evidence and a greater possibility that the recommendation will change when more evidence is generated in the future. Of note, the assigned grade contains no subjective information regarding the importance of the recommendation or how strongly members of the committee felt about it; it only contains information regarding the evidence upon which the recommendation is based. Thus, many Grade D recommendations were deemed to be very important to the contemporary management of diabetes, based on clinical experience, case series, physiological evidence and current concepts of disease pathophysiology. However, the paucity of clinical evidence addressing the areas of therapy, prevention, diagnosis or prognosis precluded the assignment of a higher grade.

Clearly, clinicians need to base clinical decisions on the best available relevant evidence that addresses clinical situations. However, they also frequently are faced with having to act in the absence of clinical evidence, and there are many situations where good clinical evidence may be impossible, impractical or too expensive to generate (which implies that it would be impossible to develop Grade A recommendations). For example, it took the United Kingdom Prospective Diabetes Study (UKPDS) Group >20 years to collect and publish Level 1 evidence leading to a Grade A recommendation in support of the role of tight glycemic control to reduce microvascular disease in people with type 2 diabetes. Prior to the publication of the UKPDS results, the recommendation for glycemic control to prevent microvascular consequences was a Grade B recommendation (9) .

Varying grades of recommendations, therefore, reflect varying degrees of certainty regarding the strength of inference that can be drawn from the evidence in support of the recommendation. Therefore, these evidence-based guidelines and their graded recommendations are designed to satisfy 2 important needs: 1) the explicit identification of the best research upon which the recommendation is based and an assessment of its scientific relevance and quality (captured by the assignment of a level of evidence to each citation); and 2) the explicit assignment of strength of the recommendation based on this evidence (captured by the grade). In this way, they provide a convenient summary of the evidence to facilitate clinicians in the task of “weighting” and incorporating ever increasing evidence into their daily clinical decision making. They also facilitate the ability of clinicians, healthcare planners, healthcare providers and society, in general, to critically examine any recommendation and arrive at their own conclusions regarding its appropriateness. Thus, these guidelines facilitate their own scrutiny by others according to the same principles that they use to scrutinize the literature.

It is important to note that the system chosen for grading recommendations differs from the approach used in some other guideline documents, such as the one pertaining to the periodic health examination in Canada, in which harmful practices were assigned a grade of D (8) . In this Canadian Diabetes Association guidelines document, recommendation to avoid any harmful practices would be graded in the same manner as all other recommendations. However, it should be noted that the authors of these guidelines focused on clinical practices that were thought to be potentially beneficial and did not seek out evidence regarding the harmfulness of interventions.

External Peer Review and Independent Methodological Review

In May 2012, a draft document was circulated nationally and internationally for review by numerous stakeholders and experts in relevant fields. This input was then considered by the Executive and Steering Committees, and revisions were made accordingly. Subsequently, a panel of 6 methodologists, who were not directly involved with the initial review and assessment of the evidence, independently reviewed each recommendation, its assigned grade and supportive citations. Based on this review, the wording, assigned level of evidence and grade of each recommendation were reassessed and modified as necessary. Revised recommendations were reviewed and approved by the Executive and Steering Committees. Selected recommendations were presented at a public forum at the Canadian Diabetes Association/Canadian Society of Endocrinology and Metabolism Professional Conference and Annual Meetings in Vancouver, British Columbia, on October 13, 2012.

Disclosure of Duality of Interest

Committee members were volunteers and received no remuneration or honoraria for their participation. Members of all committees signed an annual duality of interest form listing all financial interests or relationships with manufacturer(s) of any commercial product(s) and/or provider(s) of commercial services. Dualities of interest were discussed during deliberations where relevant. In the case of a potential duality or outright conflict of interest, committee members removed themselves from discussions. Funding for the development of the guidelines was provided from the general funds of the Canadian Diabetes Association and from unrestricted educational grants from Novo Nordisk Canada Inc, Eli Lilly Canada Inc, Merck Canada Inc, Bristol-Myers Squibb and AstraZeneca, and Novartis Pharmaceuticals Canada Inc. These companies were not involved in any aspect of guideline development, literature interpretation, the decision to publish or any other aspect related to the publication of these guidelines, and they did not have access to guideline meetings, guideline drafts or committee deliberations.

Guideline Updates

A process to update the full guidelines will commence within 5 years and will be published in 2018. Updates to individual chapters may be published sooner in the event of significant changes in evidence supporting the recommendations. The Executive and Steering Committees of the 2013 revision will continue to remain intact to deliberate any potential updates to individual chapters until such time as the Executive and Steering Committees for the 2018 revision have been created.

Other Relevant Guidelines

Introduction, p. S1

References

  1. Canadian Diabetes Association Clinical Practice Guideline Expert Committee Canadian Diabetes Association 2003 clinical practice guidelines for the prevention and management of diabetes in Canada Can J Diabetes 27 suppl 2 2003 S1 S152
  2. Canadian Diabetes Association Clinical Practice Guideline Expert Committee Canadian Diabetes Association 2008 clinical practice guidelines for the prevention and management of diabetes in Canada Can J Diabetes 32 suppl 1 2008 S1 S201
  3. S.E. Straus F.A. McAlister What is the prognosis? H.C. Gerstein R.B. Haynes Evidence-based Diabetes Care 2001 BC Decker Inc Hamilton, ON 6 12
  4. American Medical Association Users' Guides to the Medical Literature: Essentials of Evidence-based Clinical Practice 2001 American Medical Association Chicago, IL
  5. R. Jaeschke G.H. Guyatt How should diagnostic tests be chosen and used? H.C. Gerstein R.B. Haynes Evidence-based Diabetes Care 2001 BC Decker Inc Hamilton, ON 13 23
  6. A.M. Holbrook J.-A. Clarke C. Raymond How should a particular problem be managed? Incorporating evidence about therapies into practice H.C. Gerstein R.B. Haynes Evidence-based Diabetes Care 2001 BC Decker Inc Hamilton, ON 24 47
  7. S.B. Harris S.M. Webster-Bogaert Evidence-based clinical practice guidelines H.C. Gerstein R.B. Haynes Evidence-based Diabetes Care 2001 BC Decker Inc Hamilton, ON 48 61
  8. R. Goldbloom R.N. Battista The periodic health examination: 1. Introduction CMAJ 134 1986 721 723
  9. S. Meltzer L. Leiter D. Daneman 1998 clinical practice guidelines for the management of diabetes in Canada CMAJ 159 suppl 8 1998 S1 S29
 
Reproduced with permission from Canadian Journal of Diabetes © 2013 Canadian Diabetes Association. To cite this article, please refer to For citation.
 

*The Canadian Diabetes Association is the registered owner of the name Diabetes Canada. All content on guidelines.diabetes.ca, CPG Apps and in our online store remains exactly the same. For questions, contact communication@diabetes.ca.