12.6 years ago by
There has been a fair amount of discussion of these issues already,
searching the mailing list will help to reveal the salient points.
The most important question here is what do *you* think p-value
correction is going to do for you? In my opinion (and lots of folks
seem to have different views), p-value corrections do two things for
Both are related to the observation that under a composite null (all
hypotheses are false), that the smallest p-value when testing 10K
hypotheses is much smaller than the smallest p-value when testing 5K.
And most of us need some help deciding/interpreting these outputs.
1) If you test some large number of hypotheses, p-value corrections
allow you to interpret the p-values in some holistic way. Here one, is
trying to answer the question of whether any (or how many) of the
hypotheses are truly false. And it sort of works, but basically the
"correction", is almost always a reduction in the significance level,
and so not only do you enrich the set of "called false" hypotheses for
truly false, you also make more errors of the other kind (not
hypotheses that are false).
2) If you have two experiments, one with 5K hypotheses, and one with
10K, then p-value corrections allow you to "align" the evidence, and
compare in some sensible way the two experiments.
I am not aware of any other contributions that these methods can make,
but perhaps others will enlighten us.
Now, when we turn our attention to GO, the problem is not one of
p-value correction, but one of philosophy, again in my view. Consider
the following situation (which does often arise).
Consider two nodes in the GO graph, with a parent child
and further consider a given set of data, where you have some set of
tested genes (which define your universe) and some set of genes you
decided are *special*. Next we find, that for these two nodes in the
graph, the same set of genes are annotated at both (for all genes in
organism this will not be true, but we didn't measure them all and we
only get to work with what we measured). So now, the two p-values from
your Hypergeometric test are identical. No amount of p-value
(or even p-value psychotherapy) will change that. So which node do you
report? This is entirely philosophy and not mathematics. Current
scientific practice is to report the more specific of the nodes, and
only make more general claims (eg I cured cancer, over less general
ones, I cured person X, who had cancer) when there is additional
evidence, over and above that needed for the specific claim. That is
point of the conditional analyses.
Now of course, a much better way to do the whole thing is to use
(eg the Category package), but then you will eventually end up back at
the same place. When you are dealing with dependent hypotheses, there
always going to be a philosophical, not just a mathematical, issue to
Mark W Kimpel wrote:
> Here's a question for the serious statisticians amongst us.
> The function hyperGTest of package "GOstats" implements a method
> to Alexa, et. al (2006) (elim method). Alexa, et. al claim that the
> used hypergeometric test on the entire ontology can't be analyzed
> FDR because of the highly interdependent nature of the DAG structure
> GO. The authors go on claim that their methods decrease this
> interdependence, but, as far as I can tell, never directly answer
> question as to whether the resultant p values can be corrected for
> For the purpose of the following discussion, assume that we are only
> working with one of the 3 major GO categories. While it is true that
> dependence has been decreased because a parent cannot reverse
> gene from its child, several children at the same level can share
> or can they? I"m not sure.
> If there is gene overlap at the lowest levels of the GO graph
> then it seems to me that there is still dependence and FDR cannot be
> assessed. Correct?
> if there is no gene overlap at the lowest levels of the GO graph
> structure, then it seems to me that these levels are independent and
> can be applied. Correct?
> Would someone who really knows GO answer the question about overlap
> genes at the lowest levels and then could a statistician answer the
> questions regarding dependence/independence and the applicability of
> applying an FDR method such as BH or the Storey qvalue?
Robert Gentleman, PhD
Program in Computational Biology
Division of Public Health Sciences
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N, M2-B876
PO Box 19024
Seattle, Washington 98109-1024
rgentlem at fhcrc.org