Hey,

with the method ""plotDispEsts" of the DESeq2 package one gets a two-dimensinal dispersion plot. Here I have the following question:

Can somebody explain or does somebody knows how the blue points ("final") are calculated? I expect they are calculated taking the black points ("gene-est") as starting point, but how?

If I understand it right:

Because we have the mean of the normalized (read) counts on the x-axis, and the dispersion on the y-axis one black point should represent the variability (== dispersion) of this read count, i.e. how often

occurs, right?

But how are noew the blue points ("final") calculated, by introducing a kind of cut-off to the black ones?

Many thanks in advance,

Juergen

This is best understood by reading the relevant sections of the DESeq2 paper:

http://genomebiology.biomedcentral.com/articles/10.1186/s13059-014-0550-8

"First, gene-wise MLEs are obtained using only the respective gene’s data (black dots). Then, a curve (red) is fit to the MLEs to capture the overall trend of dispersion-mean dependence. This fit is used as a prior mean for a second estimation round, which results in the final MAP estimates of dispersion (arrow heads). This can be understood as a shrinkage (along the blue arrows) of the noisy gene-wise estimates toward the consensus represented by the red line. The black points circled in blue are detected as dispersion outliers and not shrunk toward the prior (shrinkage would follow the dotted line)."

Then you can skip to the Methods for the full details, under "Estimation of dispersions".

The black points are alpha_i^gw

The red line is alpha_tr(mu-bar)

The blue points are alpha_i^MAP (except for the outliers, see following section, "Dispersion outliers")

Please see the section of the vignette "Access to all calculated values" if you want to know how to find these estimated values in the dds object.