Search
Question: EBImage crashing in for-loop
0
3.2 years ago by
United States
mk.applebee0 wrote:

I have a script that uses EBImage to identify and measure a swarming colony in series of time-lapse photos. When I run this script, it usually crashes. The number of cycles it takes to crash changes between runs, though it usually crashes at a step where computeFeatures.moment and computeFeatures.shape are called.  I thought this was a memory issue, but the number of iterations it takes to crash does not change significantly if I run the script on a computer with more memory (started with 4GB, have now tried 16 and 32 GB), and I have also added rm() and gc() commands to dump images as soon as they are no longer needed.  However, since I have done this and updated R, Rstudio, and EBImage, I am now averaging 8-9 images being processed instead of 4-5.

Sometimes RStudio prints an error message with the crash - "This application has requested the Runtime to terminate it in an unusual way.  Please contact the application's support team for more information." - but not always.  Googling this error message lead me to believe it has something to do with Microsoft Visual C++, which means it is way beyond my limited computer knowledge to solve.

(I also get this error message when I try to install EBImage on a more recent version of R (3.2.0) on the 16GB machine whenever I type in >source("http://bioconductor.org/biocLite.R") , though first I get an error message that says "Warning in install.packages: "lib = C:/ProgramFiles/R/R-3.2.0/library" is not writable." I get the "Runtime" error message when I force the stop, at which point Rstudio tells me that it lost connection with R and has to restart).

EDIT/UPDATE:

I have found a specific image that always crashes the analysis, at the compute.Features/compute.Moments step.  Here is a link to a dropbox folder that contains

1. two images (the background and subject photo)

2. the R script where all you need to do is fill in the directory paths to the images and where to put the output if the system doesn't crash

3. screenshots of the two types of crashes I get - with/without the "Runtime" error message. Both are from "computer 1"

Computer 1:

> sessionInfo()
R version 3.2.1 (2015-06-18)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] tools_3.2.1

Computer 2:

> sessionInfo()

R version 3.2.0 (2015-04-16)

Platform: x86_64-w64-mingw32/x64 (64-bit)

Running under: Windows 8 x64 (build 9200)

locale:

[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252

[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C

[5] LC_TIME=English_United States.1252

attached base packages:

[1] stats     graphics  grDevices utils     datasets  methods   base

loaded via a namespace (and not attached):

[1] tools_3.2.0

And finally, here is my function.

crashme <- function(){

picdir = "C:/Users/(fill in directory path)"     #location of pic files
savedir = "C:/Users/(fill in directory path)"  #directory to save output images into

bck=5                        #bck - image # of background photo
frst=15                      #frst - image # of first image to analyze
filelistL <- list.files(picdir, "jpg")              #list of pic file names
LL=65        #LL - last image# to analyze

library("EBImage")  #version 4.10.0

# load first photo, to be used for background subtraction from all others

#creat empty objects
allmlist <- list()
mlist <- c()

for (x in frst:LL) {

#print number of images left to process
print(paste0("---img #",(1+x-frst)," of ",(LL-frst+1),"---"))

# Background subtraction
print("0. Background Subtraction")
paranobg<- (para-paramed)

width<-dim(para)[1]/4   #dimension to be used for tiled image at end of loop
para<-resize(para,w=width) #resize for tiled image

#change neg values from background subtraction back to 0
print("1. remove neg values")
par0<-as.matrix(paranobg)
par0[par0<0]<-0
par0<-as.Image(par0)

#amplify stronger signals using gamma correction
print("2. gamma correction")
par0<-(par0^0.3)

#some global thresholding of weak signals to zero
print("3. remove weak noises")
par0<-as.matrix(par0)
par0[par0<0.23]<-0
par0<-as.Image(par0)
par2print<-resize(par0*10,w=width)  #increase contrast in output image, and resize

#blur image a bit to get rid of noise
print("4. running filter2")
f = makeBrush(11, shape='diamond', step=FALSE)
f = f/sum(f)
filt2<-filter2(par0,f)
rm(par0)
filt2print<-resize(filt2*10,w=width)  #increase contrast in output image

#generate first mask; now image is binary
rm(filt2)

kern <- makeBrush(9, shape='diamond')

print("8. computeFeatures 1")
m <- cbind(m2[,'s.area'],m)

if (length(m) != 0) {
print("9. objects found")
colnames(m)[1] <- "m.pxs"

low = 6000  #initial colony size appears to be about 8000 pixels

id<-which(m[,'m.pxs']<low | m[,'m.eccentricity'] > 0.65 | m[,'m.majoraxis']>2000)

mask4 <- rmObjects(mask3, id)  #removes objects that satisfy any conditions of "id"

m<- cbind(m2[,'s.area'],m)

if (length(m) != 0) {
print("11. true objects found")
colnames(m)[1] <- "m.pxs"

mlist=m[,c('m.cx','m.cy', 'm.pxs')]
allmlist[[x]] <- mlist

}else{print("11. No true objects found")}

}else{print("9-11. no objects found")}

#generate output image
writeImage(t1, file=paste(savedir, paste(substr(filelistL[x],1,nchar(filelistL)-4), "_tile.png",sep="")
, sep="/"))
gc()

} #end of for-loop
saveRDS(allmlist,file=paste(savedir,"allmlist.rds",sep="/"))
return(allmlist)
}
modified 3.2 years ago by Andrzej Oleś720 • written 3.2 years ago by mk.applebee0

Having the image files would probably be necessary to reproduce this. As stated in the faq you can upload the files to dropbox or another service and share the url in this thread, making sure the files are publicly accessible.

thank you!  here is a dropbox link, and the original post has been updated

Hi,

as already suggested by Dan, please provide a complete reproducible example including the sample image files.

Cheers,

Andrzej

2
3.2 years ago by
Andrzej Oleś720
Heidelberg, Germany
Andrzej Oleś720 wrote:

Thank you for a comprehensive bug report and for providing a dataset which triggers the crash. I was able to reproduce the error and I've applied a patch which fixes it in EBImage 4.10.1.

For the record: the bug was in C code of the ocontour function (called by computeFeatures.shape). There was an indexing problem if the contour length exceeded the size of the temporary buffer. I've fixed the index issue and also increased the buffer size to accommodate for larger contours.

Cheers,

Andrzej