Download the full-sized PDF of The impact of bias on the magnitude of treatment effect estimates in oral health randomized trialsDownload the full-sized PDF



Permanent link (DOI):


Export to: EndNote  |  Zotero  |  Mendeley


This file is in the following communities:

Graduate Studies and Research, Faculty of


This file is in the following collections:

Theses and Dissertations

The impact of bias on the magnitude of treatment effect estimates in oral health randomized trials Open Access


Other title
Randomized controlled trial
Systematic review
Study quality
Effect size
Meta-epidemiological study
Type of item
Degree grantor
University of Alberta
Author or creator
Saltaji, Humam
Supervisor and department
Flores-Mir, Carlos
Cummings, Greta
Examining committee member and department
Compton, Sharon
Sharifzadeh-Amin, Maryam
Pandis, Nikolaos
Liven, Liran
Armijo-Olivo, Susan
Medical Sciences-Orthodontics

Date accepted
Graduation date
2016-06:Fall 2016
Doctor of Philosophy
Degree level
Background: There is emerging evidence that randomized trials are subject to biases. Flaws in the design of such trials can result in over- or underestimation of the treatment effect size. Aim: To examine the empirical evidence for bias, to quantify the extent of bias associated with methodology, and to make recommendations for reducing bias in oral health randomized trials. Methods and Analyses: The study was conducted in four interconnected phases. First, to develop a register of oral health systematic reviews, 1188 reviews published in the domain of oral health research were identified and described. In the second phase, a methodology study of 1114 therapeutic reviews (published between 1991 and 2014) was constructed using descriptive and multivariate logistic regression analyses to explore (a) how often and by what means risk of bias in trials had been assessed, and (b) factors associated with completed risk of bias assessments. In the third phase, from the register of reviews, all meta-analyses were selected that examined at least one continuous outcome and included a minimum of five oral health randomized trials (n = 64); this package consisted of 540 randomized trials that analyzed 137,957 patients. The risk of bias was examined with respect to the reporting and methodological characteristics of these trials in order to assess the state of oral health trials over time. In the fourth phase, using a two-level meta-meta-analytic approach with a random effects model to allow for intra- and intermeta-analysis heterogeneity, the impact of (a) bias associated with 22 methodological characteristics, and (b) specific features (such as dental specialty and type of outcome) on the magnitude of treatment effect estimates in oral health randomized trials was evaluated. Results: Risk of bias assessment of primary studies had not been made in a considerable portion of 1114 therapeutic reviews published between 1991 and 2014 (61.4%, n = 684). This occurred more often in reviews published after dissemination of the PRISMA statement (odds ratio = 1.55; 95% confidence interval [CI]: 1.17 to 2.06), and in reviews published in nondental journals (odds ratio = 0.28; 95% CI: 0.19 to 0.41). The results of the risk of bias and quality assessments were unfavorable in general, indicating substandard quality and high potential for bias in oral health trials. The proportion of trials judged as having a low risk of bias did not exceed 60% in the majority of the risk of bias domains, but this proportion has significantly increased over time. In the 540 oral health randomized trials examined, significantly larger treatment effect size estimates were identified in trials that had inadequate sequence generation (difference in treatment effect size estimates = 0.13; 95% CI: 0.01 to 0.25), inadequate allocation concealment (difference in treatment effect size estimates = 0.15; 95% CI: 0.02 to 0.27), lack of patient and assessor blinding (difference in treatment effect size estimates = 0.19; 95% CI: 0.06 to 0.32), inadequate reporting of the dropout rate (difference in treatment effect size estimates = 0.24; 95% CI: 0.05 to 0.43), and inappropriate influence of funders (difference in treatment effect size estimates = 0.10; 95% CI: 0.02 to 0.19). Although not statistically significant, a tendency toward exaggeration of the treatment effect size was found in the presence of imbalance in cointervention, inadequate compliance to treatment, incomplete outcome data, or having dropout without performing the intention-to-treat approach. In contrast, baseline imbalance, caregiver blinding, an acceptable dropout rate ( 20%), selective outcome reporting, and analysis based on an intention-to-treat approach, were not associated with inflated or underestimated treatment effect size. Conclusion: Bias was found to be, overall, associated with inflated treatment effect size in oral health randomized trials. Therefore, systematic reviewers would be advised to exclude trials conducted in the domains of dental, oral, and craniofacial research from meta-analyses or to perform sensitivity analyses based on the adequacy of these criteria. Because of the impact of bias on treatment effect size estimates, dental journal editors and reviewers should insist on adequate conduct and reporting of trial reports submitted for publication.
This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for the purpose of private, scholarly or scientific research. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.
Citation for previous publication
[1] Saltaji H, Cummings GG, Armijo-Olivo S, et al. A descriptive analysis of oral health systematic reviews published 1991-2012: cross sectional study. PloS One 2013;8(9):e74545. [2] Saltaji H, Armijo-Olivo S, Cummings GG, Amin M, Flores-Mir C. Methodological characteristics and treatment effect sizes in oral health randomised controlled trials: is there a relationship? Protocol for a meta-epidemiological study. BMJ Open 2014;4(2):e004527. [3] Saltaji H, Ospina MB, Armijo-Olivo S, Agarwal S, Cummings GG, Amin M, Flores-Mir C. Evaluation of risk of bias assessment of trials in systematic reviews of oral health interventions, 1991-2014: a methodology studyJ Am Dent Assoc. 2016 May 4. pii: S0002-8177(16)30299-9.

File Details

Date Uploaded
Date Modified
Audit Status
Audits have not yet been run on this file.
File format: pdf (Portable Document Format)
Mime type: application/pdf
File size: 2965374
Last modified: 2016:11:16 13:33:47-07:00
Filename: Saltaji_Humam_201611_PhD.pdf
Original checksum: a4c0e31a1b1811381d9a421e8ee7c2b3
Well formed: false
Valid: false
Status message: Invalid page tree node offset=1061776
Status message: Unexpected error in findFonts java.lang.ClassCastException: edu.harvard.hul.ois.jhove.module.pdf.PdfSimpleObject cannot be cast to edu.harvard.hul.ois.jhove.module.pdf.PdfDictionary offset=5004
Status message: Invalid Annotation list offset=2509416
Status message: Invalid outline dictionary item offset=2511431
Activity of users you follow
User Activity Date