I use EdgeR since a while and I have noticed recently that the common dispersion estimated using estimateGLMCommonDisp() was slightly varying according to the row order of the count input (gene ID order). If I do the same analysis with exactly the same input table but by changing the row order, e.g. genes ordered by either decreasing or increasing mean of count across samples, I obtain slightly different dispersion values and p-values. As an example, I found 0.0260375, 0.02603552, 0.02603565 as common dispersion value for the same input table but with three different row order. This difference is very small but I was wondering what could be the reason of this variation?

By default, estimateGLMCommonDisp() takes a systematic sample of 10,000 rows to compute the common dispersion from. If you change the order of the genes in y, then the selection of genes can in some circumstances change slightly and this will change the estimated dispersion. As your post shows, the variation is so small as to be of no consequence but, if you want, you can avoid any variation by setting subset=Inf.

In recent years, we have been recommending estimateDisp instead of estimateGLMCommonDisp. estimateDisp doesn't subset the genes so the dispersion estimates will be unaffected by changes in the row order.