Human Reproduction Update Advance Access originally published online on February 7, 2008
Human Reproduction Update 2008 14(3):289-291; doi:10.1093/humupd/dmm051
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Reply: ‘Is it time for meta-analysis?’
1 Department of Reproductive Medicine, St Mary's Hospital, CMMC University Hospitals, Whitworth Park, Manchester M13 0JH, UK 2 Department of Obstetrics and Gynaecology, Lancashire Teaching Hospitals, Preston, UK
3 Correspondence address. Fax: +44-0161-224-0957; E-mail: luciano.nardo{at}cmmc.nhs.uk
In their letter, Dr Ruopp and co-authors question the statistical approach and the validity of our meta-analysis on the use of low-dose aspirin in women undergoing in vitro fertilization (IVF) (Gelbaya et al., 2007
). They also go some way to suggest that we have reached a flawed interpretation of the results. Their argument is that random-effects models have an inherent loss of precision by introducing an in-between study variance and that random-effects model should only be used when the absence of inter-study heterogeneity can be assumed.
With fixed-effects models, it is assumed that there is a sole common effect estimates for all studies, i.e. the true effect of treatment, in both magnitude and direction, is the same value in every study. This assumption implies that the observed differences among study results and the observed between-study variability are due solely to the play of chance, i.e. there is no statistical heterogeneity. With the range of data available in most meta-analyses, failure to reject the null hypothesis of homogeneity does not prove homogeneity. It is implausible that effect modifications and biases are exactly the same between studies. In the presence of any between-study heterogeneity, fixed-effects give tighter confidence intervals compared with random-effects. This leads to spuriously lower levels of statistical significance for the summary effects (Poole and Greenland, 1999; Ntzani et al., 2007).
When there is heterogeneity that cannot be readily explained, one analytical approach is to incorporate it into a random-effects model. A random-effects meta-analysis model involves an assumption that the effects being estimated in the different studies are not identical, but follow some distribution. The model represents our lack of knowledge about why real, or apparent, treatment effects differ by considering the differences as if they were random (The Cochrane Collaboration, 2006). The most popular estimator of the between-study variance is the DerSimonian–Laird estimator (DerSimonian and Laird, 1986) incorporating the assumption that different studies are estimating different, yet related, treatment effects (which was the case in our meta-analysis) and makes an adjustment to the study weights according to the extent of variation, or heterogeneity, among the varying treatment effects. The ‘fixed-effects’ model considers, often unreasonably, that this variability is exclusively due to random variation. Therefore, if all the studies were infinitely large they would give identical results. Conversely, the ‘random-effects’ model assumes a different underlying effect for each study and takes this into consideration as an additional source of variation, which leads to somewhat wider confidence intervals than the fixed-effects model and therefore corresponding claims of statistical significance are more conservative (Lau et al., 1998; Engels et al., 2000).
Random-effects accommodate diversity between studies, thus being by far preferable in the presence or anticipation of any between-study heterogeneity. This is the case in the majority of meta-analyses, so that the use of random-effects models is generally preferable compared to the fixed effects. When no heterogeneity exists, then both models show similar effects. The practice of using fixed effects inappropriately continues (Ioannidis et al., 2007a, b).
In our meta-analysis, the statistical and methodological heterogeneity (as revealed by the I2 statistics and the Q test) makes the use of the random-effects model the most appropriate and accurate method of analysis. The assumption behind the random effects model, that the effect of aspirin may have small random and normally distributed around zero differences between studies, seems more plausible than the fixed-effects assumption, that aspirin has a common identical effect. We made a concerted effort to define strictly the inclusion and exclusion criteria for our meta-analysis and minimize, as far as possible, the combinations of data from different clinical groups. We excluded studies that were not randomised and those that investigated the use of low-dose aspirin only in a specific subgroup of patients such as the poor responders, oocyte recipients or subjects who had frozen embryo transfer. However, the included studies have underlying differences in population and methodology, hence the assumption of a common effect is unrealistic.
We do agree that the random-effects model distributes to the studies weights that may not reflect differences in precision as prominently as the fixed-effects do. However, this should be problematic in a case of funnel plot asymmetry (Higgins and Spiegelhalter, 2002). In our analysis, where I2 ranges from 27% to 62%, we do not anticipate that the weights are so remarkably unfair to the big trials. The main concern with the suggestion by Dr Ruopp and colleagues is that the use of fixed-effects model in the presence of heterogeneity shrinks the confidence interval of the summary estimate and therefore may spuriously yield a statistically significant result (Schmidt et al., 2007). In the original protocol, we anticipated heterogeneity due to clinical and methodological differences between studies and set the random-effects model as the most realistic analysis scenario. The use of fixed-effects would simply suggest a ‘chasing for significance’ bias.
Analysis with the fixed-effects model did not alter substantially the findings of this study and reached similar conclusions. Noteworthy, another recently published meta-analysis on the use of aspirin in IVF that used the fixed-effects model for their analysis reached conclusions similar to ours (Khairy et al., 2007).
Dr Ruopp and colleagues also raise the question whether a meta-analysis for the use of aspirin in women undergoing IVF was premature. In the current climate of good and safe medical practice, there is no such a thing as ‘premature meta-analysis’. It is unethical and scientifically unacceptable to ignore the best available evidence that comes from meta-analysis, even if the evidence may not be seen as conclusive. As the absence of evidence is not evidence of absence, we think that our interpretation ‘the evidence evaluating the administration of low-dose aspirin for this purpose does not support its implementation in clinical practice’ remains accurate. It is important to highlight that we did not recommend policy changes and abandonment of the use of aspirin from clinical practice as Dr Ruopp and his colleagues have stated in their letter.
Furthermore, current evidence does not support the routine use of low-dose aspirin for women undergoing IVF treatment, especially in the absence of sufficient information on its safety when administered early from the day of embryo transfer and continued through the early stages of embryogenesis. A previously published meta-analysis (Kozer et al., 2002) demonstrated that the use of aspirin in the first trimester of pregnancy was associated with a significantly increased risk of gastroschisis (odds ratio, 2.37; 95% CI, 1.44–3.88).
In line with the available literature, our meta-analysis is well placed to inform fertility professionals that aspirin administration should not be routinely recommended to women undergoing assisted conception treatments and should only be offered within the context of clinical trials. Indeed, our views are different from those expressed by Dr Ruopp and collaborators in their letter to the Editor. Instead of keep on recommending aspirin in the absence of evidence, we suggest to wait for the evidence before unnecessarily prescribing ineffective drugs to patients.
 |
References |
|---|
 TOP References |
|---|
DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials (1986) 7:177–188.[CrossRef][Web of Science][Medline]
Engels EA, Schmid CH, Terrin N, Olkin I, Lau J. Heterogeneity and statistical significance in meta-analysis: an empirical study of 125 meta-analyses. Stat Med (2000) 19:1707–1728.[CrossRef][Web of Science][Medline]
Gelbaya TA, Kyrgiou M, Li TC, Stern C, Nardo LG. Low-dose aspirin for in vitro fertilization: a systematic review and meta-analysis. Hum Reprod Update (2007) 13:357–364.
Higgins JP, Spiegelhalter DJ. Being sceptical about meta-analyses: a Bayesian perspective on magnesium trails in myocardial infarction. Int J Epidemiol (2002) 31:96–104.
Khairy M, Banerjee K, El-Toukhy T, Coomarasamy A, Khalaf Y. Aspirin in women undergoing in vitro fertilization treatment: a systematic review and meta-analysis. Fertil Steril (2007) 88:822–831.[CrossRef][Medline]
Kozer E, Nikfar S, Costei A, Boskovic R, Nulman I, Koren G. Aspirin consumption during the first trimester of pregnancy and congenital anomalies: a meta-analysis. Am J Obstet Gynecol (2002) 187:1623–1630.[CrossRef][Web of Science][Medline]
Ioannidis JP. Molecular evidence-based medicine: evolution and integration of information in the genomic era. Eur J Clin Invest (2007) a 37:340–349.[CrossRef][Medline]
Ioannidis JP. Non-replication and inconsistency in the genome-wide association setting. Hum Hered (2007) b 64:203–213.[CrossRef][Medline]
Lau J, Ioannidis JP, Schmid CH. Summing up evidence: one answer is not always enough. Lancet (1998) 351:123–127.[CrossRef][Web of Science][Medline]
Ntzani EE, Rizos EC, Ioannidis JP. Genetic effects versus bias for candidate polymorphisms in myocardial infarction: case study and overview of large-scale evidence. Am J Epidemiol (2007) 165:973–984.
Poole C, Greenland S. Random-effects meta-analyses are not always conservative. Am J Epidemiol (1999) 150:469–475.
Schmidt FL, Oh IS, Hayes TL. Fixed- versus random-effects models in meta-analysis: Model properties and an empirical comparison of differences in results. Br J Math Stat Psychol (2007) 13. (Epub ahead of print).
The Cochrane Collaboration. Cochrane Handbook for Systematic Reviews of Interventions 4.2.6, 2006. www.cochrane.org/resources/handbook.
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||