Hi, -using Bioconductor v1.8.1 and limma v1.6.1 When I use backgroundCorrect(minimum) and then dupcor.series I don't get a correlation value (NaN), if I don't use the backgroundCorrect method I get a value but with a warning message. Does anyone have any suggestions as to what is happening here and how I can sort it out please? (I realize the experimental design is hopeless but I have to work with what I'm given). I don't have any missing/blank values but obviously have negative values after background subtraction. Code and warning are given below. Thanks, Helen > files=dir(pattern="*\\.gpr") > RG=read.maimages(files, columns=list(Rf="F635 Median", Gf="F532 Median", Rb="B635 Median", Gb="B532 Median"), wt.fun=wtflags(0)) > names(RG) > RG$genes=readGAL() > RG$printer=getLayout(RG$genes) > samples=read.table("sampleinformationa.txt", header=TRUE, sep="\t", as.is=TRUE) > samples > spottypes=readSpotTypes() > RG$genes$Status=controlStatus(spottypes, RG) > RGmin=backgroundCorrect(RG, method="minimum") > MAmin=normalizeWithinArrays(RGmin, RG$printer) > MAmin2=normalizeBetweenArrays(MAmin) > design=c(1,1,1,1) > cor=dupcor.series(MAmin2$M, design, ndups=2, spacing=1) Loading required package: statmod Attaching package 'statmod': The following object(s) are masked from package:limma : matvec vecmat > cor$cor [1] NaN > MAdef=normalizeWithinArrays(RG, RG$printer) > MAdef2=normalizeBetweenArrays(MAdef) > design=c(1,1,1,1) > cor=dupcor.series(MAdef2$M, design, ndups=2, spacing=1) Warning messages: 1: Too much damping - convergence tolerance not achievable in: glmgam.fit(dx, dy, start = start, tol = tol, maxit = maxit, trace = trace) 2: Too much damping - convergence tolerance not achievable in: glmgam.fit(dx, dy, start = start, tol = tol, maxit = maxit, trace = trace) 3: Too much damping - convergence tolerance not achievable in: glmgam.fit(dx, dy, start = start, tol = tol, maxit = maxit, trace = trace) [1] 0.7891024 [[alternative HTML version deleted]]

At 01:59 AM 16/04/2004, Helen Cattan wrote: >Hi, > >-using Bioconductor v1.8.1 and limma v1.6.1 >When I use backgroundCorrect(minimum) and then dupcor.series I don't get >a correlation value (NaN), if I don't use the backgroundCorrect method I >get a value but with a warning message. Does anyone have any suggestions >as to what is happening here and how I can sort it out please? (I >realize the experimental design is hopeless but I have to work with what >I'm given). I don't have any missing/blank values but obviously have >negative values after background subtraction. Code and warning are given >below. >Thanks, > >Helen > > > files=dir(pattern="*\\.gpr") > > RG=read.maimages(files, columns=list(Rf="F635 Median", Gf="F532 >Median", Rb="B635 Median", Gb="B532 Median"), wt.fun=wtflags(0)) > > names(RG) > > RG$genes=readGAL() > > RG$printer=getLayout(RG$genes) > > samples=read.table("sampleinformationa.txt", header=TRUE, sep="\t", >as.is=TRUE) > > samples > > spottypes=readSpotTypes() > > RG$genes$Status=controlStatus(spottypes, RG) > > RGmin=backgroundCorrect(RG, method="minimum") > > MAmin=normalizeWithinArrays(RGmin, RG$printer) No need to type 'RG$printer', the function will find it automatically. > > MAmin2=normalizeBetweenArrays(MAmin) > > design=c(1,1,1,1) > > cor=dupcor.series(MAmin2$M, design, ndups=2, spacing=1) >Loading required package: statmod > >Attaching package 'statmod': > > > The following object(s) are masked from package:limma : > > matvec vecmat > > > cor$cor >[1] NaN I don't know what the problem is, presumably it is a problem with method="minimum". Can you please type summary(MAmin2$M), summary(MAmin2$weights) and summary(cor$cor.genes) to give more information? > > MAdef=normalizeWithinArrays(RG, RG$printer) > > MAdef2=normalizeBetweenArrays(MAdef) > > design=c(1,1,1,1) > > cor=dupcor.series(MAdef2$M, design, ndups=2, spacing=1) >Warning messages: >1: Too much damping - convergence tolerance not achievable in: >glmgam.fit(dx, dy, start = start, tol = tol, maxit = maxit, trace = >trace) >2: Too much damping - convergence tolerance not achievable in: >glmgam.fit(dx, dy, start = start, tol = tol, maxit = maxit, trace = >trace) >3: Too much damping - convergence tolerance not achievable in: >glmgam.fit(dx, dy, start = start, tol = tol, maxit = maxit, trace = >trace) >[1] 0.7891024 The warnings are no problem. Gordon

Hi, Thanks for the reply. I've typed in what you suggested for more information. Done with and without background correction. I don't know what you are looking for - but hope this helps. After using RG=backgroundCorrect(RG, method="minimum") summary(MAmin2$M) QCmouse..CD4.depl.day.7..260803...590.530..liver. Min. :0 1st Qu.:0 Median :0 Mean :0 3rd Qu.:0 Max. :0 QCmouse..CD4.depl.day.8..10903.II....590.550..liver.4 Min. :0 1st Qu.:0 Median :0 Mean :0 3rd Qu.:0 Max. :0 QCmouse.liver..CD4.depl..1.6.ug..day.8..670.530..240703 Min. :0 1st Qu.:0 Median :0 Mean :0 3rd Qu.:0 Max. :0 QCmouse.liver.CD4.depl.day.8..10903.I....590.540.. Min. : NA 1st Qu.: NA Median : NA Mean : NaN 3rd Qu.: NA Max. : NA NA's :31200 > summary(MAmin2$weights) QCmouse..CD4.depl.day.7..260803...590.530..liver. Min. :0.0000 1st Qu.:1.0000 Median :1.0000 Mean :0.9873 3rd Qu.:1.0000 Max. :1.0000 QCmouse..CD4.depl.day.8..10903.II....590.550..liver.4 Min. :0.000 1st Qu.:1.000 Median :1.000 Mean :0.986 3rd Qu.:1.000 Max. :1.000 QCmouse.liver..CD4.depl..1.6.ug..day.8..670.530..240703 Min. :0.0000 1st Qu.:1.0000 Median :1.0000 Mean :0.9762 3rd Qu.:1.0000 Max. :1.0000 QCmouse.liver.CD4.depl.day.8..10903.I....590.540.. Min. :0.0000 1st Qu.:1.0000 Median :1.0000 Mean :0.9878 3rd Qu.:1.0000 Max. :1.0000 > > summary(cor$cor.genes) Length Class Mode 0 NULL NULL > And not using background correct > summary(MA1$M) QCmouse..CD4.depl.day.7..260803...590.530..liver. Min. :-5.188e+00 1st Qu.:-2.813e-01 Median :-1.425e-02 Mean : 2.424e-02 3rd Qu.: 2.949e-01 Max. : 5.752e+00 NA's : 1.326e+04 QCmouse..CD4.depl.day.8..10903.II....590.550..liver.4 Min. : -5.25729 1st Qu.: -0.24996 Median : -0.03117 Mean : 0.09563 3rd Qu.: 0.35874 Max. : 6.01569 NA's :8209.00000 QCmouse.liver..CD4.depl..1.6.ug..day.8..670.530..240703 Min. :-7.200e+00 1st Qu.:-2.703e-01 Median : 4.189e-03 Mean : 3.571e-02 3rd Qu.: 3.068e-01 Max. : 1.293e+01 NA's : 1.611e+04 QCmouse.liver.CD4.depl.day.8..10903.I....590.540.. Min. :-7.880e+00 1st Qu.:-2.729e-01 Median :-5.434e-04 Mean : 8.506e-02 3rd Qu.: 3.162e-01 Max. : 1.025e+01 NA's : 1.808e+04 > > summary(MA1$weights) QCmouse..CD4.depl.day.7..260803...590.530..liver. Min. :0.0000 1st Qu.:1.0000 Median :1.0000 Mean :0.9873 3rd Qu.:1.0000 Max. :1.0000 QCmouse..CD4.depl.day.8..10903.II....590.550..liver.4 Min. :0.000 1st Qu.:1.000 Median :1.000 Mean :0.986 3rd Qu.:1.000 Max. :1.000 QCmouse.liver..CD4.depl..1.6.ug..day.8..670.530..240703 Min. :0.0000 1st Qu.:1.0000 Median :1.0000 Mean :0.9762 3rd Qu.:1.0000 Max. :1.0000 QCmouse.liver.CD4.depl.day.8..10903.I....590.540.. Min. :0.0000 1st Qu.:1.0000 Median :1.0000 Mean :0.9878 3rd Qu.:1.0000 Max. :1.0000 > > summary(cor$cor.genes) Length Class Mode 0 NULL NULL Thanks, Helen -----Original Message----- From: Gordon Smyth [mailto:smyth@wehi.edu.au] Sent: 16 April 2004 00:18 To: Helen Cattan Cc: bioconductor@stat.math.ethz.ch Subject: Re: [BioC] limma - dupcor.series At 01:59 AM 16/04/2004, Helen Cattan wrote: >Hi, > >-using Bioconductor v1.8.1 and limma v1.6.1 >When I use backgroundCorrect(minimum) and then dupcor.series I don't get >a correlation value (NaN), if I don't use the backgroundCorrect method I >get a value but with a warning message. Does anyone have any suggestions >as to what is happening here and how I can sort it out please? (I >realize the experimental design is hopeless but I have to work with what >I'm given). I don't have any missing/blank values but obviously have >negative values after background subtraction. Code and warning are given >below. >Thanks, > >Helen > > > files=dir(pattern="*\\.gpr") > > RG=read.maimages(files, columns=list(Rf="F635 Median", Gf="F532 >Median", Rb="B635 Median", Gb="B532 Median"), wt.fun=wtflags(0)) > > names(RG) > > RG$genes=readGAL() > > RG$printer=getLayout(RG$genes) > > samples=read.table("sampleinformationa.txt", header=TRUE, sep="\t", >as.is=TRUE) > > samples > > spottypes=readSpotTypes() > > RG$genes$Status=controlStatus(spottypes, RG) > > RGmin=backgroundCorrect(RG, method="minimum") > > MAmin=normalizeWithinArrays(RGmin, RG$printer) No need to type 'RG$printer', the function will find it automatically. > > MAmin2=normalizeBetweenArrays(MAmin) > > design=c(1,1,1,1) > > cor=dupcor.series(MAmin2$M, design, ndups=2, spacing=1) >Loading required package: statmod > >Attaching package 'statmod': > > > The following object(s) are masked from package:limma : > > matvec vecmat > > > cor$cor >[1] NaN I don't know what the problem is, presumably it is a problem with method="minimum". Can you please type summary(MAmin2$M), summary(MAmin2$weights) and summary(cor$cor.genes) to give more information? > > MAdef=normalizeWithinArrays(RG, RG$printer) > > MAdef2=normalizeBetweenArrays(MAdef) > > design=c(1,1,1,1) > > cor=dupcor.series(MAdef2$M, design, ndups=2, spacing=1) >Warning messages: >1: Too much damping - convergence tolerance not achievable in: >glmgam.fit(dx, dy, start = start, tol = tol, maxit = maxit, trace = >trace) >2: Too much damping - convergence tolerance not achievable in: >glmgam.fit(dx, dy, start = start, tol = tol, maxit = maxit, trace = >trace) >3: Too much damping - convergence tolerance not achievable in: >glmgam.fit(dx, dy, start = start, tol = tol, maxit = maxit, trace = >trace) >[1] 0.7891024 The warnings are no problem. Gordon

At 05:01 AM 19/04/2004, Helen Cattan wrote: >Hi, > >Thanks for the reply. I've typed in what you suggested for more >information. Done with and without background correction. I don't know >what you are looking for - but hope this helps. The summary output shows that all your data is zero or NA after background correction, so you can hardly expect to do any linear modelling with it. Even without background correction, half your data is missing. You could update to the current version of limma from http://bioinf.wehi.edu.au/limma, which may be more robust in the presence of NAs, but I think you basically need to go back to your original data files and ask why there is so much missing data. Foreground and background intensities from Genepix should never be missing. >After using RG=backgroundCorrect(RG, method="minimum") > >summary(MAmin2$M) > QCmouse..CD4.depl.day.7..260803...590.530..liver. > Min. :0 > 1st Qu.:0 > Median :0 > Mean :0 > 3rd Qu.:0 > Max. :0 > > QCmouse..CD4.depl.day.8..10903.II....590.550..liver.4 > Min. :0 > 1st Qu.:0 > Median :0 > Mean :0 > 3rd Qu.:0 > Max. :0 > > QCmouse.liver..CD4.depl..1.6.ug..day.8..670.530..240703 > Min. :0 > 1st Qu.:0 > Median :0 > Mean :0 > 3rd Qu.:0 > Max. :0 All the above columns are entirely zero! > > QCmouse.liver.CD4.depl.day.8..10903.I....590.540.. > Min. : NA > 1st Qu.: NA > Median : NA > Mean : NaN > 3rd Qu.: NA > Max. : NA > NA's :31200 This column is entirely missing! >summary(MAmin2$weights) >QCmouse..CD4.depl.day.7..260803...590.530..liver. > Min. :0.0000 > 1st Qu.:1.0000 > Median :1.0000 > Mean :0.9873 > 3rd Qu.:1.0000 > Max. :1.0000 > QCmouse..CD4.depl.day.8..10903.II....590.550..liver.4 > Min. :0.000 > 1st Qu.:1.000 > Median :1.000 > Mean :0.986 > 3rd Qu.:1.000 > Max. :1.000 > QCmouse.liver..CD4.depl..1.6.ug..day.8..670.530..240703 > Min. :0.0000 > 1st Qu.:1.0000 > Median :1.0000 > Mean :0.9762 > 3rd Qu.:1.0000 > Max. :1.0000 > QCmouse.liver.CD4.depl.day.8..10903.I....590.540.. > Min. :0.0000 > 1st Qu.:1.0000 > Median :1.0000 > Mean :0.9878 > 3rd Qu.:1.0000 > Max. :1.0000 > > > > summary(cor$cor.genes) >Length Class Mode > 0 NULL NULL > > >And not using background correct > > > summary(MA1$M) > QCmouse..CD4.depl.day.7..260803...590.530..liver. > Min. :-5.188e+00 > 1st Qu.:-2.813e-01 > Median :-1.425e-02 > Mean : 2.424e-02 > 3rd Qu.: 2.949e-01 > Max. : 5.752e+00 > NA's : 1.326e+04 > QCmouse..CD4.depl.day.8..10903.II....590.550..liver.4 > Min. : -5.25729 > 1st Qu.: -0.24996 > Median : -0.03117 > Mean : 0.09563 > 3rd Qu.: 0.35874 > Max. : 6.01569 > NA's :8209.00000 > QCmouse.liver..CD4.depl..1.6.ug..day.8..670.530..240703 > Min. :-7.200e+00 > 1st Qu.:-2.703e-01 > Median : 4.189e-03 > Mean : 3.571e-02 > 3rd Qu.: 3.068e-01 > Max. : 1.293e+01 > NA's : 1.611e+04 > QCmouse.liver.CD4.depl.day.8..10903.I....590.540.. > Min. :-7.880e+00 > 1st Qu.:-2.729e-01 > Median :-5.434e-04 > Mean : 8.506e-02 > 3rd Qu.: 3.162e-01 > Max. : 1.025e+01 > NA's : 1.808e+04 Have of your values are missing! If you have indeed not background corrected then this shows that half of your foreground intensities are missing! Gordon > > summary(MA1$weights) > QCmouse..CD4.depl.day.7..260803...590.530..liver. > Min. :0.0000 > 1st Qu.:1.0000 > Median :1.0000 > Mean :0.9873 > 3rd Qu.:1.0000 > Max. :1.0000 > QCmouse..CD4.depl.day.8..10903.II....590.550..liver.4 > Min. :0.000 > 1st Qu.:1.000 > Median :1.000 > Mean :0.986 > 3rd Qu.:1.000 > Max. :1.000 > QCmouse.liver..CD4.depl..1.6.ug..day.8..670.530..240703 > Min. :0.0000 > 1st Qu.:1.0000 > Median :1.0000 > Mean :0.9762 > 3rd Qu.:1.0000 > Max. :1.0000 > QCmouse.liver.CD4.depl.day.8..10903.I....590.540.. > Min. :0.0000 > 1st Qu.:1.0000 > Median :1.0000 > Mean :0.9878 > 3rd Qu.:1.0000 > Max. :1.0000 > > > > summary(cor$cor.genes) >Length Class Mode > 0 NULL NULL > > > >Thanks, >Helen