Hiya,

I have 110 samples and am looking at sample clustering. Using rlog it seems to show outliers, which if I removed then throws up further outliers and so on ... after each sequential removal and re-run. If I use VST it seems that these extreme outliers do not exist or at least are not as obvious when looking at the PCA. Reading around it looks like VST is recommended for sample clustering outlier identification - am I correct in this thinking or is it best to keep removing samples based on sequential rlog repeats until no obvious outliers exist anymore?

Yes, I recommend VST for PCA and outlier identification. There's another related thread just posted, and perhaps the rlog issue is related to a preponderance of low count features. Either way, I would stick to VST.