Is it ok to use decideTests on linearly dependent contrasts?
2
0
Entering edit mode
@ryan-c-thompson-5618
Last seen 8 months ago
Scripps Research, La Jolla, CA

I have an RNA-seq experiment with 4 time points, and I would like to identify genes that change significantly in one direction and then later change significantly in the other direction (broadly speaking, I'm looking to distinguish between genes that change transiently and genes whose change is maintained through the last time point). In order to do this, I need to look at (potentially) all 6 possible contrasts between time points. I would like to use a common significance threshold for all 6 contrasts, so I'm looking at limma's decideTests function with method = "global". However, 3 of these 6 contrasts are necessarily linear combinations of the other 3, and I don't know whether that is a problem for decideTests. Is it ok to use decideTests in this way or is there an assumption in decideTests that all the tested contrasts are linearly independent that would cause it to return a bad or unreliable result for this use case? If decideTests is not suitable, is there some other way I should be going about this?

limma decidetests • 764 views
ADD COMMENT
2
Entering edit mode
Aaron Lun ★ 28k
@alun
Last seen 1 hour ago
The city by the bay

The real question is whether the dependencies between tests are a problem for the Benjamini-Hochberg method. I think you're probably okay as the BH method is pretty robust to these correlations.

You can double-check for yourself by making some dummy contrasts for which the null hypothesis should be true. For example, put an arbitrary replicate from each time point into a group, and test for DE between groups (while blocking on the time point). If you have enough replicates per time point, you can mimic the dependency structure that you have in your actual use case. The null hypothesis should be true for all genes in each comparison between these artificially constructed groups, so you can apply the BH method (after throwing in a bunch of low p-values to simulate true positives) and estimate the empirical FDR. This should be below the nominal threshold if the BH method is behaving correctly.

ADD COMMENT
1
Entering edit mode
@gordon-smyth
Last seen 38 minutes ago
WEHI, Melbourne, Australia

I think the global FDR will likely be somewhat conservative overall, because you are adjusting for doing 6 x G tests when there are actually only 3 x G independent comparisons (because there are only 3 df for between time comparisons). The larger numerator in the adjusted p-value computation should cause a modest amount of conservativeness.

A more important concern occurs when some contrasts have lots of true DE while other contrasts have little or none. If this is so, then the global FDR adjustment will be conservative for the contrasts with lots of true DE and anti-conservative for contrasts with no true DE. The FDR will be controlled globally but not for each contrast individually. There is nothing published on this, it is just a phenomenon I have observed myself.

ADD COMMENT

Login before adding your answer.

Traffic: 684 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6