B&H is appropriate for any number of tests. I believe it is, in this respect, the most general approach, although a subject matter expert might know of alternatives that can sometimes provide more power. Q-values can, when there are very large numbers of tests and a substantial fraction are not expected to be null, provide additional power. Storey's method is an appealing way to leverage the industry-standard B&H approach while still squeezing as much power as possible out of large studies with many thousands of simultaneous tests, but in order to work correctly, the method absolutely requires thousands of tests. Brad Efron has explored this topic quite thoroughly, and the notes from his course and book on the subject are available online ( http://www-stat.stanford.edu/~omkar/329/) if the topic is of great interest to you. If you consider the effect of covariance between tests and between subjects, many times, these multiple tests are not in fact independent -- but, those covariance matrices must themselves be estimated in order to make sense of the situation. Thus the line between estimation and testing can be quite blurry. Anyways, for your question, the answer I suggest is B&H (1995) as implemented in p.adjust(). On Wed, Jul 3, 2013 at 3:27 PM, Feng Tian <fengtian@bu.edu> wrote: > Hi Tim, > > Thank you very much for your reply. So BH(95) method is the most general > FDR-controlling method which can be applied to the case for small number of > testes, right? > > Feng > > On Wed, Jul 3, 2013 at 5:59 PM, Tim Triche, Jr. <tim.triche@gmail.com>wrote: > >> qvalues are not appropriate for an experiment where there is such a small >> number of tests. >> >> To compute q-values, the proportion of true nulls must be estimated, >> which requires enough data points (i.e., enough tests being done) to >> estimate the parameters of a two-component mixture model. I dimly recall >> the lower bound for such an estimate to be any good is around 3000 tests, >> and more is better. >> >> Furthermore, the estimate (mentioned above) is then simply applied to the >> B&H-corrected pvalues. >> >> So: >> >> p.adjust(your.p.values, method='fdr') >> >> is probably the most sensible choice for your experiment. >> >> Hope this helps, >> >> --t >> >> >> >> On Wed, Jul 3, 2013 at 1:55 PM, Feng Tian <fengtian@bu.edu> wrote: >> >>> Hi Dear all, >>> >>> I need to do multiple testing correction for my results which contains >>> about 200 tests. I find that there are two general methods to do FDR >>> controlling. One is the BH method and the other one is the Storey >>> (q-value). The results from two methods are quite different. Usually, >>> Storey method can get more significant results than BH method. So which I >>> should I use? Are there some guildlines for choosing appropriate method >>> to >>> do FDR controlling? >>> >>> Thanks and have a nice day. >>> >>> Feng >>> >>> [[alternative HTML version deleted]] >>> >>> _______________________________________________ >>> Bioconductor mailing list >>> Bioconductor@r-project.org >>> https://stat.ethz.ch/mailman/listinfo/bioconductor >>> Search the archives: >>> http://news.gmane.org/gmane.science.biology.informatics.conductor >>> >> >> >> >> -- >> *A model is a lie that helps you see the truth.* >> * >> * >> Howard Skipper<http: cancerres.aacrjournals.org="" content="" 31="" 9="" 1173.full.pdf=""> >> > > -- *A model is a lie that helps you see the truth.* * * Howard Skipper<http: cancerres.aacrjournals.org="" content="" 31="" 9="" 1173.full.pdf=""> [[alternative HTML version deleted]]