Nonpharmacological interventions on glycated haemoglobin in youth with type 1 diabetes: a Bayesian network meta-analysis – Cardiovascular Diabetology

Two researchers (JM-P and NH-U) independently carried out a rigorous systematic exploration to replicate the number of studies initially obtained. Any disparities found in the collection process were reconciled through consensus discussions involving a third researcher (AG-H).

Protocol and registration

The study was conducted according to the NMA extension of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA-NMA) guidelines [18]. The study protocol was registered in PROSPERO (CRD42023432586).

Eligibility criteria

The studies were included if they met all of the following criteria: (i) the mean age of the study cohort was required to be 18 years or younger, with a confirmed diagnosis of type 1 diabetes mellitus, regardless of disease duration; (ii) the study assessed a group enrolled in at least one of the following nonpharmacological intervention: physical exercise program, physical activity promotion, dietary interventions, nutritional education, behavioural therapy, or diabetes education; (iii) the experimental group was compared to a control group or another different intervention; (iv) the HbA1c measurement was one of the dependent variables; and (v) the study design was a randomised clinical trial (RCT).

Exclusion criteria were: (i) having comorbidities other than diabetes, (ii) receiving the same type of intervention between groups (e.g., two similar dietary interventions), and (iii) gray literature (e.g., conference proceedings or editorials). No specific criterion regarding the minimum duration of the program was set.

Information sources and search

Relevant articles were identified by title and abstract in the electronic databases PubMed, Web of Science, Scopus, and SPORTDiscus from inception to July 1, 2023. Search terms related to the disease “Diabetes Mellitus Type 1” were used. No limitation according to the date of publication was done.

Terms related to the dependent variable of interest and its abbreviation, glycated haemoglobin or HbA1c or A1c, were searched. Moreover, terms were included to specify the respective interventions (e.g., “exercise”, “physical activity”, “diet”, “nutritional education”, “behavioural therapy”, “diabetes education”, among others). All search strategies are shown in ESM Table 1. Additionally, a comprehensive literature review was conducted, which encompassed the identification of previous reviews addressing similar topics. Therefore, a rigorous analysis of additional studies was carried out using backward citation searching.

Study selection

First, two of the reviewers (J-MP and A-GH) inputted the citations gathered from the databases into the EndNote bibliographic management tool, and after removing duplicates, the articles were initially screened based on their titles and abstracts. Subsequently, the full text of the studies that met the criteria was evaluated. Each author generated individual lists of the selected studies at each stage. Once again, any discrepancies were resolved by a third researcher (N-HU) to achieve the final compilation of studies.

Interventions obtained were classified as: “Control (CON)”, “Aerobic exercise training (AER)”, “Behavioural therapy (BT)”, “Carbohydrate counting (CC)”, “Diabetic education (DEd)”, “Diet (DIET)”, “Family therapy (FT)”, “Multicomponent exercise training (MC)” (e.g., aerobic and strength training), “Nutritional education (NEd)”, “Physical activity promotion (PAP)”, “Resistance exercise training (RT)”, “Sleep education (SLEd)”, “Nutritional supplements (SUP)”, “AER + DIET”, “BT + FT”, “DEd + AER + DIET”, “DEd + BT”, “DEd + BT + FT”, and “NEd + BT”, “RT + SUP”. The explanation of the independent categories is explained in ESM Table 2. This systematic categorisation facilitated the grouping of studies into 20 distinct categories, enabling a comprehensive approach for conducting NMA [19].

Data collection process and data items

Two reviewers (J-MP and J-LG) established a database to extract results across four domains. The first encompassed study characteristics including authors, journal, and publication year. The second comprised participant demographics such as sex and age. The third involved intervention details like type, duration, and frequency. The fourth contained HbA1c data as the primary outcome and, when available, daily insulin dose requirements as a secondary outcome.

During data collection, the measurement units used in each study were noted (i.e., %, mmol/mol, U, and U/Kg). To maintain methodological consistency in our analysis and to precisely evaluate the actual effects of the interventions, the extracted data encompassed baseline and immediately post-intervention mean values alongside their corresponding standard deviations (SD). Follow-up measurements were excluded from consideration. For trials presenting results in figures, data was extracted using WebPlotDigitizer software (version 4.6). Means and SDs were estimated if needed [20,21,22].

Risk of bias within individual studies

Two reviewers (JM-P and IH-A) used pretested forms to assess the studies risk of bias based on the criteria outlined in the Cochrane Risk of Bias Tool 2.0 (RoB 2.0) [23]. This tool comprises five domains that assess aspects such as randomization (D1), implementation of interventions (D2), loss of data (D3), measurement of variables (D4), and reporting of results (D5). The RCTs were analysed individually and assigned as “high risk of bias” if one of the five domains was at high risk of bias; “moderate risk of bias” if at least one of the five domains had possible indications of bias; and “low risk of bias” if there were no concerns for any of the domains. Any disagreement in quality ratings was discussed. If consensus could not be reached, a third member of the review team (JL-G) was consulted.

Summary measures

Comprehensive meta-analysis software (v3; Biostat, Inc.; Englewood, New Jersey, United States) was used to synthesise the findings and summarise the numerical outcomes of individual studies. The standardised mean differences (Cohen’s d) (SMD) and the standard error (SE) were computed to determine the effect of interventions and control group independently for each RCT [24]. Subsequently, they were grouped into their respective categories for analysis within the NMA.

Network meta-analysis

All statistical analyses were carried out using the R statistical software (Version 4.1.1) from R Core Team in Vienna, Austria, and RStudio (Version 2021.09.2) from Posit in Boston, MA, USA. “BUGSnet” (Bayesian inference Using Gibbs Sampling to conduct NETwork meta-analysis) is a feature-rich R package to conduct Bayesian NMA in compliance with the best practice and reporting guidelines available [25]. These Bayesian analyses are carried out utilizing Just Another Gibbs Sampler (JAGS). The outputs generated are extensively customizable and encompass a range of visual aids and data summaries. These include network plots, tables detailing network attributes, league tables coupled with league heat plots, Surface Under Cumulative Ranking (SUCRA) plots, rankograms, forest plots, leverage plots, trace plots, and plots for comparing posterior mean deviances. The Confidence in Network Meta-Analysis (CINeMA) framework was employed to ascertain the final outcomes of the NMA, which are contextualised based on the bias of the comparisons, resulting in the certainty of the evidence [26]. CINeMA is a single web application that communicates to an R back-end server [27].

Initially, a narrative synthesis of the RCTs included in this study was performed, compiling the findings in an ad hoc table that delineates both direct and indirect comparisons. Subsequently, to enable the software to analyse the results, the CINeMA guidelines were adhered to by constructing a table in a long and continuous format [27].

Prior to performing the NMA, a meticulous evaluation of three fundamental assumptions was conducted as outlined by previous literature [28]: (a) to avoid potential bias in the comparative analyses, thereby prevention the appearance of heterogeneity and inconsistency, a comprehensive assessment of the likeness and comparability of the trials encompassed by the NMA was performed. The evaluation of similarity was rooted in the comparison of the baseline distribution of variables acknowledged as effect modifiers (e.g., sex, age) among samples within each intervention category; (b) to ensure the homogeneity of conditions associated with the respective studies, a comprehensive verification of the absence of heterogeneity within the outcomes of pairwise comparisons was undertaken. The magnitude and clinical significance of heterogeneity were quantified through the application of the τ² statistic, classifying its impact as having either a low degree of clinical relevance (29]; and (c) a meticulous scrutiny of consistency and transitivity was executed to ensure the absence of significant disparities between direct and indirect evidence. The validity of the similarity assumption or incoherence was ascertained via the implementation of the side or node-splitting methodology [19, 30].

To facilitate the analysis of the results, the following procedures were carried out: robustness was evaluated by employing a network geometry to visually represent the available evidence regarding HbA1c. In this figure, the diameter of the nodes is proportional to the number of participants involved in trials receiving the same type of intervention, and the thickness of the continuous line connecting the nodes is proportional to the number of trials directly comparing a pair of interventions [31].

The comparative assessment of the impact of interventions on HbA1c was executed via a Bayesian NMA, comparing the interactions between interventions and control groups [32]. The relative treatment effect that represents a clinically important difference was defined [26]. In this instance, relative SMD estimates below – 0.8 were regarded as clinically significant [24]; and the range between ± 0.8 is the “range of equivalence”, which corresponds to clinically unimportant differences between interventions. In addition, HbA1c data were reconverted to the same unit of measurement before re-analysis to understand the clinical impact on % HbA1c using mean differences (MD) [33].

To assess inconsistency, we employed the I² statistic, which spans the range of 0–100%. Based on the I² values, the classification of inconsistency ranged from not important (0–30%), moderate (30–60%), and substantial (60–75%) to considerable (75–100%) [34]. Corresponding p-values were also considered.

To identify the most effective intervention, rankograms were generated, which displays the likelihood of each treatment being the optimal choice. Furthermore, we derived estimates for the SUCRA metric for each intervention [31]. The SUCRA allocates a score between 0 and 1 to each intervention within the rankogram, facilitating their ranking. A SUCRA value close to 0 indicates the least favourable intervention, while a value near 1 signifies the most effective intervention. The SUCRA metric synthesises the information regarding the effectiveness of each intervention into a singular value, thereby simplifying the outcomes of NMA into a concise representation. The significance of SUCRA is given when a consistent disparity in preference exists between consecutive ranks throughout the entire rating scale. The SUCRA scores were reported as percentages to improve its interpretation.

Risk of bias across studies, comparisons and certainty of evidence

The level of confidence in the evidence within coherent networks was evaluated using the Grading of Recommendation, Assessment, Development, and Evaluation (GRADE) methodology. Four degrees of confidence were appraised, ranging from very low to high (reflecting either a strong possibility of substantial variations between the actual and estimated impact or a significant level of assurance in the similarity between the actual and estimated impact, respectively). The GRADE judgments were formulated considering the values yielded by each of the six domains of CINeMA, an online tool was designed by the Cochrane Group, which also aids in evaluating the quality and reliability of evidence generated from the simultaneous comparison of multiple interventions [26]. In accordance with the relevant protocol, each comparison was assessed for potential bias, including within-study bias, reporting bias, directionality, imprecision, heterogeneity and inconsistency. Confidence levels for each domain were initially high, but were subsequently adjusted for the specific methodological issues described below:

Within-study bias

Once the RoB 2.0 tool was integrated into the CINeMA framework, the bar chart provided by the system was analysed. Subsequently, the confidence level of each comparison was reduced using the weighted average risk of bias method. To do this, scores of – 1, 0 and 1 were assigned to low, moderate and high risk of bias, respectively [26]. After multiplying each percentage in the bar chart by the corresponding factor and summing the three components, the result of the bias to be reported was obtained.

Reporting bias

The Risk of Bias due to Missing Evidence in Network meta-analysis (ROB-MEN) was also used to simplify the assessment of the reporting bias domain associated with CINeMA [35]. This tool incorporates qualitative or quantitative approaches such as funnel plots, tests for small study effects or selection models into an overall assessment for comparative studies such as RCTs.

In addition, the recommendations of the Cochrane Handbook for Systematic Reviews of Interventions [36], Bayesian funnel plots were run to assess publication bias through asymmetry between all nonpharmacological therapies and controls. Independently, the same dynamics were followed for those comparisons with more than 10 studies (behavioural therapy vs control and diabetes education vs control). As these tests are not recommended for comparisons with less than 10 studies, it was decided to indicate “some concerns”.

Indirectness

In order to assess whether the available evidence is applicable to the clinical question [37], the weighted average method was selected taking into account the bar chart reported by the framework [27].

Imprecision

In the NMA, indirect evidence is added to direct evidence for gaining precision purposes [38]. A SMD = 0.8 was set as indicative of a clinically important effect and then imprecision was assessed using 95% credible intervals (CrI) estimated with the maximum likelihood method. “No concerns” were established if the 95% CrI was entirely on the effect side or entirely within the range of equivalence. The confidence level decreased to “some concern” when 95% CrI covered both clinically important and null effects. Conversely, comparisons were marked as of “major concern” if the 95% CrI limits extended beyond of the opposite equivalence range.

Heterogeneity

Considering the clinically significant effect threshold (SMD = 0.8), CINeMA assesses heterogeneity by examining whether the prediction and 95% CrI yield discordant conclusions regarding clinical relevance. Comparisons are considered of “some concern” if the prediction interval enters or leaves the zone of clinical significance once, and of “major concern” if it crosses this region twice.

Incoherence

We assessed the ability of the NMA to produce similar estimates when using direct comparisons versus indirect estimates. In addition, we examined the transitivity assumption, which ensures that the only systematic difference between two groups is the treatment and no other confounding factors. Considering the inconsistency factor and using the SIDE method [39], which separates direct and indirect estimates to check for significant differences in effect estimates, we identified “some concern” when direct and indirect 95% CrI reported discrepancies in effect despite to follow the same direction. In contrast, we reported “major concern” when 95% CrI indicated discrepancies due to changes in the direction of effect.

Confidence rating

The levels reported by CINeMA in each domain were translated to the levels commonly reported by GRADE as follows: high confidence, indicating that the true effect closely aligns with the estimated effect; moderate confidence, suggesting that the true effect is likely close to the estimate but could differ substantially; low confidence, indicating that the true effect may differ substantially from the estimate; and very low confidence, indicating that the true effect is likely substantially different from the estimate. In the final report, the interrelation between domains was considered, avoiding isolated judgments (e.g., substantial “heterogeneity” may impact increased “imprecision” in the relative treatment effects) [40].

Additional analyses

We conducted a further NMA using the same methodology to assess how interventions directly and indirectly affected participants daily insulin dose requirements.

In addition, when there were 10 or more studies for the same type of intervention [41], meta-regression analyses were conducted to determine the moderating effect of mean age, number of girls, duration of diabetes since diagnosis, and duration of interventions on the estimates obtained. Moreover, to determine the moderating effect of these same factors in the interventions at the global level, moderation analyses including all interventions (regardless of type) versus control were carried out (following the recommendation of one of the reviewers). All these analyses were performed with the function “bmr” of the “bayesmeta” package [42].

On the other hand, applying the same criteria as for the meta-regression analyses (i.e., for interventions with 10 or more studies and for interventions at global level), publication bias was assessed through funnel plots (using the function “funnel” of the “bayesmeta” package).