Human Reproduction Update Advance Access originally published online on July 4, 2008
Human Reproduction Update 2008 14(5):537-538; doi:10.1093/humupd/dmn026
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Low dose aspirin and IVF: ‘Is it time for a meta-analysis’ continued?
Division of Endocrinology, Mubarak Al Kabeer Teaching Hospital, Jabriya, Kuwait
1 Correspondence address. E-mail: sardoi{at}gmx.net
In their reply, Dr Gelbaya and co-authors (Gelbaya et al., 2008
) defend their meta-analysis against the criticism of Ruopp et al. (2008) that the statistical approach and the validity of their meta-analysis on the use of low-dose aspirin in women undergoing in vitro fertilization (IVF) (Gelbaya et al., 2007) has reached a flawed interpretation of the results. The criticism was that random-effect models have an inherent loss of precision by introducing a between-study variance. Gelbaya et al. (2008) try to respond to this by claiming that the use of a random effects variance component of DerSimonian and Laird (1986) is the most appropriate and accurate method of analysis, accommodates diversity between studies and that their pooled estimate is more conservative. These claims are not true and several problems are evident with this meta-analysis result.
First, the random effects re-distribution of study weights is indeed random and bears no relationship to differences between these studies (Senn, 2007). This is expected since a common between-study variance is used to increase each study's variance but mainly affecting the studies with the highest precision. In other words, this model penalizes high-precision studies in the presence of heterogeneity to the benefit of low-precision studies. Since precision reflects sample size and not study quality, this would mean that the re-distribution of inverse variance weights in this method is independent of which studies have the more credible effects sizes and instead randomly assigns such credibility information based on the observed variance within each study. The result does not correlate with actual between study differences simply because changes in magnitude or direction of an effect size attributable to poor quality cannot be predicted by precision, thus confounding the ability of a common between-study variance to identify those studies that depart from quality.
Second, Gelbaya et al. did not bother to assess the methodologic quality of the studies, they include in their meta-analysis, which is a common flaw with meta-analyses (Moja et al., 2005). Even if we argue that the quality scores are subjective and not as precise as a common between-study variance that they use, this would still be better than the totally random assignment of study weights via a random effects model. The incorporation of quality scores has conventionally been used as a sensitivity criterion and for use in summary estimates from meta-analyses. The latter has been criticized by Greenland and O'Rourke (2001) on the grounds that it fails to account for the magnitude or direction of bias induced by a quality deficiency and the mixing of ‘objective’ measures with arbitrary judgements (Greenland and O'Rourke, 2001). Although the quality deficiency is easily overcome by the quality effects model (Doi and Thalib, 2008), the arbitrary judgements actually apply more to the random model since the inflation of the variance of more precise studies in this model bears no relationship to heterogeneity arising from differences in quality of individual studies. Indeed, Poole and Greenland (1999) again point out that the random-effects point estimates are not invariably closer to the null value nor are their P-values invariably larger than those of fixed-effects summaries (Poole and Greenland, 1999). They provide evidence that random-effects summaries are not predictably conservative according to either of these two connotations of the term. They also give an example in which the random-effects summaries are less conservative in both of these alternative senses and possibly more biased than the fixed-effects summaries (Poole and Greenland, 1999). This occurs because the random model re-distributes study weights based on varying precision alone, disregarding all other differences that constitute quality such as randomization, blinding, losses to follow-up, etc. In essence, therefore, even though a random model may seem more precise and less arbitrary than a quality-adjusted model, unfortunately it turns out to be more precise about nothing tangible.
I, therefore, strongly support Ruopp et al. (2008) and conclude that when heterogeneity has been detected, there is a strong case for investigating its possible origin, and re-distributing the weights based on such a determination (Doi and Thalib, 2008). Weight re-distribution based on the between-study variance will not be helpful in such cases since it is doubtful that any useful question can be formulated (even if the objectives of the studies are similar) that a random effects analysis could answer. The discussion of when to use random effects and when to use fixed effects in computing summary estimates should therefore be replaced by a discussion of whether summary estimates should be computed at all, under the random model, when the studies are not methodologically comparable or when their results are discernibly heterogeneous (Poole and Greenland, 1999). It is now time for the authors to reveal to us the differences in methodologic quality between the studies they included in their analysis and how sensitive their results were to these differences.
 |
References |
|---|
 TOP References |
|---|
DerSimonian R., Laird N. Meta-analysis in clinical trials. Control Clin Trials (1986) 7:177–188.[CrossRef][Web of Science][Medline]
Doi S. A., Thalib L. A quality-effects model for meta-analysis. Epidemiology (2008) 19:94–100.[Web of Science][Medline]
Gelbaya T. A., Kyrgiou M., Li T. C., Stern C., Nardo L. G. Low-dose aspirin for in vitro fertilization: a systematic review and meta-analysis. Hum Reprod Update (2007) 13:357–364.
Gelbaya T. A., Kyrgiou M., Nardo L. G. Reply: ‘Is it time for meta-analysis?’ Hum Reprod Update (2008) doi:10.1093/humupd/dmm051.
Greenland S., O'Rourke K. On the bias produced by quality scores in meta- analysis, and a hierarchical view of proposed solutions. Biostatistics (2001) 2:463–471.[Abstract]
Moja L. P., Telaro E., D'Amico R., Moschetti I., Coe L., Liberati A. Assessment of methodological quality of primary studies by systematic reviews: results of the metaquality cross sectional study. BMJ (2005) 330:1053.
Poole C., Greenland S. Random-effects meta-analyses are not always conservative. Am J Epidemiol (1999) 150:469–475.
Ruopp M. D., Whitcomb B. W., Schisterman E. F. Is it time for a meta-analysis? Hum Reprod Update (2008) 14:289.
Senn S. Trying to be precise about vagueness. Stat Med (2007) 26:1417–1430.[CrossRef][Web of Science][Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||