Skip Navigation



Biostatistics Advance Access published online on April 20, 2009
Biostatistics, doi:10.1093/biostatistics/kxp011
This Article
Right arrow Full Text
Right arrow Full Text (PDF)
Right arrow Supplementary Material
Right arrow All Versions of this Article:
10/3/550    most recent
kxp011v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Disclaimer
Google Scholar
Right arrow Articles by van de Wiel, M. A.
Right arrow Articles by van Wieringen, W. N.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by van de Wiel, M. A.
Right arrow Articles by van Wieringen, W. N.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

© The Author 2009. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oxfordjournals.org.

Testing the prediction error difference between 2 predictors

Mark A. van de Wiel*

Department of Epidemiology and Biostatistics, VU University Medical Center, PO Box 7057, 1007 MB Amsterdam, The Netherlands and Department of Mathematics, VU University, De Boelelaan 1081a, 1081 HV Amsterdam, The Netherlands mark.vdwiel{at}vumc.nl

Johannes Berkhof

Department of Epidemiology and Biostatistics, VU University Medical Center, PO Box 7057, 1007 MB Amsterdam, The Netherlands

Wessel N. van Wieringen

Department of Epidemiology and Biostatistics, VU University Medical Center, PO Box 7057, 1007 MB Amsterdam, The Netherlands and Department of Mathematics, VU University, De Boelelaan 1081a, 1081 HV Amsterdam, The Netherlands

* To whom correspondence should be addressed.

We develop an inference framework for the difference in errors between 2 prediction procedures. The 2 procedures may differ in any aspect and possibly utilize different sets of covariates. We apply training and testing on the same data set, which is accommodated by sample splitting. For each split, both procedures predict the response of the same samples, which results in paired residuals to which a signed-rank test is applied. Multiple splits result in multiple p-values. The median p-value and the mean inverse normal transformed p-value are proposed as summary (test) statistics, for which bounds on the overall type I error rate under a variety of assumptions are proven. A simulation study is performed to check type I error control of the least conservative bound. Moreover, it confirms superior power of our method with respect to a one-split approach. Our inference framework is applied to genomic survival data sets to study 2 issues: compare lasso and ridge regression and decide upon use of both methylation and gene expression markers or the latter only. The framework easily accommodates any prediction paradigm and allows comparing any 2, possibly nonmodel-based, prediction procedures.

Keywords: Cross-validation; Microarray data; Model comparison; Nonparametric testing

Received October 6, 2008; revised January 23, 2009; revised February 26, 2009; accepted for publication March 24, 2009.


Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?




Disclaimer: Please note that abstracts for content published before 1996 were created through digital scanning and may therefore not exactly replicate the text of the original print issues. All efforts have been made to ensure accuracy, but the Publisher will not be held responsible for any remaining inaccuracies. If you require any further clarification, please contact our Customer Services Department.