Search
Question: EBImage crashing in for-loop
0
gravatar for mk.applebee
2.4 years ago by
United States
mk.applebee0 wrote:

I have a script that uses EBImage to identify and measure a swarming colony in series of time-lapse photos. When I run this script, it usually crashes. The number of cycles it takes to crash changes between runs, though it usually crashes at a step where computeFeatures.moment and computeFeatures.shape are called.  I thought this was a memory issue, but the number of iterations it takes to crash does not change significantly if I run the script on a computer with more memory (started with 4GB, have now tried 16 and 32 GB), and I have also added rm() and gc() commands to dump images as soon as they are no longer needed.  However, since I have done this and updated R, Rstudio, and EBImage, I am now averaging 8-9 images being processed instead of 4-5.

Sometimes RStudio prints an error message with the crash - "This application has requested the Runtime to terminate it in an unusual way.  Please contact the application's support team for more information." - but not always.  Googling this error message lead me to believe it has something to do with Microsoft Visual C++, which means it is way beyond my limited computer knowledge to solve.

(I also get this error message when I try to install EBImage on a more recent version of R (3.2.0) on the 16GB machine whenever I type in >source("http://bioconductor.org/biocLite.R") , though first I get an error message that says "Warning in install.packages: "lib = C:/ProgramFiles/R/R-3.2.0/library" is not writable." I get the "Runtime" error message when I force the stop, at which point Rstudio tells me that it lost connection with R and has to restart).

EDIT/UPDATE:

I have found a specific image that always crashes the analysis, at the compute.Features/compute.Moments step.  Here is a link to a dropbox folder that contains

1. two images (the background and subject photo)

2. the R script where all you need to do is fill in the directory paths to the images and where to put the output if the system doesn't crash

3. screenshots of the two types of crashes I get - with/without the "Runtime" error message. Both are from "computer 1"

 

Computer 1:

> sessionInfo()
R version 3.2.1 (2015-06-18)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252   
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] tools_3.2.1

Computer 2:

> sessionInfo()

R version 3.2.0 (2015-04-16)

Platform: x86_64-w64-mingw32/x64 (64-bit)

Running under: Windows 8 x64 (build 9200)


locale:

[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252   

[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C                          

[5] LC_TIME=English_United States.1252    


attached base packages:

[1] stats     graphics  grDevices utils     datasets  methods   base     


loaded via a namespace (and not attached):

[1] tools_3.2.0

 

And finally, here is my function. 

crashme <- function(){
  
  picdir = "C:/Users/(fill in directory path)"     #location of pic files
  savedir = "C:/Users/(fill in directory path)"  #directory to save output images into
  
  bck=5                        #bck - image # of background photo
  frst=15                      #frst - image # of first image to analyze
  filelistL <- list.files(picdir, "jpg")              #list of pic file names
  LL=65        #LL - last image# to analyze
  
  
  library("EBImage")  #version 4.10.0
  
  # load first photo, to be used for background subtraction from all others
  paramed <- readImage(paste(picdir,"/",filelistL[bck],sep=""))

  #creat empty objects
  allmlist <- list()
  mlist <- c()
  
  
  for (x in frst:LL) {   
    para <- readImage(paste(picdir,"/",filelistL[x],sep=""))
    
    
    #print number of images left to process
    print(paste0("---img #",(1+x-frst)," of ",(LL-frst+1),"---"))
    
    # Background subtraction
    print("0. Background Subtraction")
    paranobg<- (para-paramed)
    
    
    width<-dim(para)[1]/4   #dimension to be used for tiled image at end of loop
    para<-resize(para,w=width) #resize for tiled image
    
    #change neg values from background subtraction back to 0
    print("1. remove neg values")
    par0<-as.matrix(paranobg)
    par0[par0<0]<-0
    par0<-as.Image(par0)
    
    
    #amplify stronger signals using gamma correction
    print("2. gamma correction")
    par0<-(par0^0.3)
   
    
    #some global thresholding of weak signals to zero
    print("3. remove weak noises")
    par0<-as.matrix(par0)
    par0[par0<0.23]<-0
    par0<-as.Image(par0)
    par2print<-resize(par0*10,w=width)  #increase contrast in output image, and resize
    
    
    
    #blur image a bit to get rid of noise
    print("4. running filter2")
    f = makeBrush(11, shape='diamond', step=FALSE)
    f = f/sum(f)
    filt2<-filter2(par0,f)
    rm(par0)
    filt2print<-resize(filt2*10,w=width)  #increase contrast in output image
    
    
    #generate first mask; now image is binary
    print("5. making mask1")
    mask1<-thresh(filt2,w=600,h=600,offset=0.0015)
    rm(filt2)
    
    print("6. making mask2")
    kern <- makeBrush(9, shape='diamond')
    mask2 <- closing(opening(mask1, kern), kern)
    
    print("7. making mask3")
    mask1<-resize(mask1,w=width)  #for output image
    mask3 <- fillHull(bwlabel(mask2)) 
    rm(mask2)
    
    print("8. computeFeatures 1")
    m <- computeFeatures.moment(mask3)
    m2 <- computeFeatures.shape(mask3)
    m <- cbind(m2[,'s.area'],m)
    
    
    if (length(m) != 0) {
      print("9. objects found")
      colnames(m)[1] <- "m.pxs"   
      
      low = 6000  #initial colony size appears to be about 8000 pixels
      
      id<-which(m[,'m.pxs']<low | m[,'m.eccentricity'] > 0.65 | m[,'m.majoraxis']>2000)
      
      mask4 <- rmObjects(mask3, id)  #removes objects that satisfy any conditions of "id"
      print("10. made mask4")
      mask3<-resize(mask3,w=width)
      
      
      m <- computeFeatures.moment(mask4)
      m2 <- computeFeatures.shape(mask4)
      m<- cbind(m2[,'s.area'],m)
      mask4<-resize(mask4,w=width)
      
      if (length(m) != 0) {
        print("11. true objects found")
        colnames(m)[1] <- "m.pxs"
        
        mlist=m[,c('m.cx','m.cy', 'm.pxs')]
        allmlist[[x]] <- mlist 
        
       
      }else{print("11. No true objects found")}
      
      
    }else{print("9-11. no objects found")}
    
    
    
    #generate output image
    t1<-tile(combine(para,par2print,filt2print,mask1,mask3,mask4),nx = 3)
    writeImage(t1, file=paste(savedir, paste(substr(filelistL[x],1,nchar(filelistL)-4), "_tile.png",sep="")
                              , sep="/"))
    gc()
  
    
   } #end of for-loop
  saveRDS(allmlist,file=paste(savedir,"allmlist.rds",sep="/"))
  return(allmlist)
}
ADD COMMENTlink modified 2.4 years ago by Andrzej Oleś670 • written 2.4 years ago by mk.applebee0

Having the image files would probably be necessary to reproduce this. As stated in the faq you can upload the files to dropbox or another service and share the url in this thread, making sure the files are publicly accessible.

ADD REPLYlink written 2.4 years ago by Dan Tenenbaum ♦♦ 8.2k

thank you!  here is a dropbox link, and the original post has been updated

ADD REPLYlink written 2.4 years ago by mk.applebee0

Hi,

as already suggested by Dan, please provide a complete reproducible example including the sample image files.

Cheers,

Andrzej

ADD REPLYlink written 2.4 years ago by Andrzej Oleś670
2
gravatar for Andrzej Oleś
2.4 years ago by
Andrzej Oleś670
EMBL Heidelberg, Germany
Andrzej Oleś670 wrote:

Thank you for a comprehensive bug report and for providing a dataset which triggers the crash. I was able to reproduce the error and I've applied a patch which fixes it in EBImage 4.10.1.

For the record: the bug was in C code of the ocontour function (called by computeFeatures.shape). There was an indexing problem if the contour length exceeded the size of the temporary buffer. I've fixed the index issue and also increased the buffer size to accommodate for larger contours.

Cheers,

Andrzej

ADD COMMENTlink modified 2.0 years ago • written 2.4 years ago by Andrzej Oleś670

Thank you very much for the patch!  Everything is working beautifully now.

ADD REPLYlink written 2.4 years ago by mk.applebee0
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 2.2.0
Traffic: 145 users visited in the last hour