I am trying to run edgeR and limma for very high sample sample size groups (group1 sample size = 236 and group2 sample size = 490). The glmQLFit() in edgeR and voom() + lmFit() in limma-voom takes about 2 min when using two confounding variables. For even higher sample sizes where each group has above 1000 samples, that function takes several minutes.

I am wondering can this function be optimized so as to reduce the compute time? Is there any way we can parallelize the computation in this function so as to decrease the computation time?

Thank you.

We recommend that you use voomLmFit() instead of voom+lmFit.

We are pretty proud of the speed of glmQLFit and voom, and we hadn't expected that a few minutes for a very large dataset would be too much a problem for users, considering the much larger computational overheads required to prepare the RNA-seq dataset in the first place.

Anyway, the short answer is that we don't offer parallelization options for either package. Our aim is that both packages should perform well on all platforms, whereas parallelization is very platform specific and, in our limited experience, offers modest gains considering the extra complications in terms of code and code dependencies. As developer, it is not something that I want to get into for a computation that only takes a few minutes in the first place, even on a laptop.

glmQLFit is already highly optimized by dropping almost everything down to clean C code. voom() could be speeded up considerably by dropping everything down to C. That is on our to-do list, but we haven't done that yet, and the function is already faster than competing methods. The slowest part of the voom pipeline is the possible use of duplicateCorrelation() for blocking. That is certainly a candidate for parallelization, but we are thinking in terms of algorithmic improvements rather going to computing technology in the first instance.