Schurch NJ, Schofield P, Gierliński M, Cole C, Sherstnev A, Singh V, Wrobel N, Gharbi K, Simpson GG, Owen-Hughes T, Blaxter M, Barton GJ. (2016) How many biological replicates are needed in an RNA-seq experiment and which differential expression tool should you use? RNA
I'm looking into it and communicating with the authors. Just to say, at this point, I can't reproduce anything like their FPR (Figure 4) when I run DESeq2 on null subsets of their 42 WT samples (3 vs 3 up to 20 vs 20). I get similar DESeq2 numbers to edgeR for example. I'll provide more details as soon as I have them.
Here is an example of the plot I'm getting when I do null comparisons within the WT samples and here's my code to generate this plot. Note that after n=3 the 95% quantile is always under the red line of 0.05, similar to the edgeR GLM performance in their paper.
It was a bug in the estimation of FPR for DESeq2.
Here's the latest from the study authors:
Thanks to @mikelove we've identified a prob. w. Figs 4 & S6 in our paper. DESeq2 FDR perf. similar to DESeq/edgeR, not worse. 1/2 @gjbarton
— Nick Schurch (@nickschurch) April 1, 2016
Updated figures coming shortly! Watch this space... 2/2 @mikelove @gjbarton @sujaik @drchriscole @CSoneson
— Nick Schurch (@nickschurch) April 1, 2016
@stephenfloor @mikelove @nickschurch after correction to our code, FDR is well controlled by DEseq2. Now it's amongst best tools.
— Chris Cole (@drchriscole) April 1, 2016
hi Riccardo,
The authors have released new figures which show DESeq2 performing similarly to other top methods.
Here is another Bioc post explaining things:
Updated DESeq2 performance on highly replicated yeast RNA-seq data
Here is the senior author's blog post on the correction:
https://geoffbarton.wordpress.com/2016/04/21/how-many-biological-replicates-are-needed-in-an-rna-seq-experiment-and-which-differential-expression-tool-should-you-use/