Skip Navigation


Biostatistics Advance Access originally published online on December 18, 2007
Biostatistics 2008 9(3):458-466; doi:10.1093/biostatistics/kxm047
This Article
Right arrow Full Text
Right arrow Full Text (PDF)
Right arrow All Versions of this Article:
9/3/458    most recent
kxm047v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Disclaimer
Google Scholar
Right arrow Articles by Shi, J.
Right arrow Articles by Whittemore, A. S.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Shi, J.
Right arrow Articles by Whittemore, A. S.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

© The Author 2007. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oxfordjournals.org.

Significance levels for studies with correlated test statistics

Jianxin Shi and Douglas F. Levinson

Department of Psychiatry and Behavioral Science, Stanford University School of Medicine, Stanford, CA 94305, USA

Alice S. Whittemore*

Department of Health Research and Policy, Redwood Building, Room T204, Stanford University School of Medicine, Stanford, CA 94305, USA, alicesw{at}stanford.edu

* To whom correspondence should be addressed.

When testing large numbers of null hypotheses, one needs to assess the evidence against the global null hypothesis that none of the hypotheses is false. Such evidence typically is based on the test statistic of the largest magnitude, whose statistical significance is evaluated by permuting the sample units to simulate its null distribution. Efron (2007) has noted that correlation among the test statistics can induce substantial interstudy variation in the shapes of their histograms, which may cause misleading tail counts. Here, we show that permutation-based estimates of the overall significance level also can be misleading when the test statistics are correlated. We propose that such estimates be conditioned on a simple measure of the spread of the observed histogram, and we provide a method for obtaining conditional significance levels. We justify this conditioning using the conditionality principle described by Cox and Hinkley (1974). Application of the method to gene expression data illustrates the circumstances when conditional significance levels are needed.

Keywords: Conditional p-value; Gene expression data; Genome-wide association data; Multiple testing; Overall p-value

Received July 18, 2007; revised November 15, 2007; accepted for publication November 15, 2007.


Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?


This article has been cited by other articles:


Home page
 BioinformaticsHome page
V. Zuber and K. Strimmer
Gene ranking and biomarker discovery under correlation
Bioinformatics, October 15, 2009; 25(20): 2700 - 2707.
[Abstract] [Full Text] [PDF]


Disclaimer: Please note that abstracts for content published before 1996 were created through digital scanning and may therefore not exactly replicate the text of the original print issues. All efforts have been made to ensure accuracy, but the Publisher will not be held responsible for any remaining inaccuracies. If you require any further clarification, please contact our Customer Services Department.