This was in answer to: BH vs BY in p.adjust + FDR interpretation

Dear Caroline,

You are becoming confused because you are making an assumption which is not true.  The methods BH and BY do not estimate the FDR.  What they do is to control the expected FDR to be less than a given value.  The term "control" means that they provide an upper bound rather than an estimate. The term "expected" means that they don't control the FDR for any given data set (because no one can do that), rather they provide a control of FDR in the long run.

Consider this thought experiment to show the difference between control and estimation.  Suppose you toss a die each day and decide to carry an umbrella unless the die comes up 6.  This would control the proportion of days on which you get wet to be less than 1/6 in the long run.  The proportion of days on which you actually get wet would probably be quite a bit less than 1/6, because most days it doesn't rain anyway, so 1/6 is not an estimate. On the other hand, the actual proportion days on which you get wet over any actual period of time could conceivably be more than 1/6, depending on how the die comes up and how much it rains.

The difference between BH and BY is that BY is valid for any level of dependence between genes while BH is theoretically valid only for weak or no dependence.  Hence BY is more conservative than BH.  In fact it is so conservative that almost no one uses it. While BH can theoretically fail to control the expected FDR for some dependence structures, simulations suggest that it is unlikely to fail for realistic scenarios or to fail by much.  This is why BH is the default method in limma.

The best reference is Rainer et al, Bioinformatics, 2003, which is cited in the limma User's Guide.

As for magnitude of change, this has no bearing on the calculations.

Best wishes
Gordon