Hi,

I use goseq and with the Wallenius approximation I get 264 BH adjusted significant overrepresented categories, with the sampling (1000 rep) I get 0 at a cutoff of 0.05. I should be reasonable, sincere and always use resampling and if theres nothing then that's the result - right?

You should always use the "Wallenius" method in goseq() and never the "Sampling" method in your context. The Sampling method is only accurate when the number of sampling replicates (repcnt) is at least 10 times larger than 1/p where p is the smallest p-value. You might have some p-values around 10^(-30). So you would need to do 10^31 repeats in order to get sensible results. This is completely impossible. So use Wallenius.

If you use only 1000 sampling repeats, then the smallest possible sampling p-value that is obtainable is 1/1001. After you adjust for multiple testing across tens of thousands of GO categories, it becomes completely impossible to get any significant result, regardless of what your data looks like. So the analysis with repcnt=1000 is meaningless.

Hi,

The Wallenius and sampling methods should give about the same results, which is more or less what you see in the unadjusted p-value plot. I wonder if there is some minimum p-value you can obtain from the sampling method which is tied to the value of repcnt. Do you get more significant genes if you adjust repcnt up? If so, I think you could trust the results from the Wallenius method.

Cheers,

Nadia.