4.3 years ago by
For DESeq2, we have 3 options for the type of trend line: fitType = "parametric", "local" or "mean". These are described in
?estimateDispersions, but roughly the recommendation is: for decreasing gene-wise dispersion estimates over mean (using plotDispEsts) one should use parametric, unless the parametric fitting procedure does not work, in which case use "local" (local regression is actually automatically substituted with a message in the case that the parametric fitting procedure does not converge.) The "mean" option is useful when there is no apparent dependence of dispersion estimates over mean (using plotDispEsts). This choice does not depend on sample size, but on the apparent dependence of the gene-wise estimates (the MLE for each gene) on the mean of counts.
If you are referring to the the tagwise estimation in edgeR, the tagwise estimation is similar to the estimateDispersionsMAP() step in DESeq2, which is the last step in estimateDispersions(), which is automatically used by DESeq().