Question: minfi failed positions in EPIC array
0
gravatar for Giovanni Calice
3.2 years ago by
Italy
Giovanni Calice150 wrote:

Hi all,

I'm working on EPIC arrays methylation data by minfi version 1.16.0.

After reading the sheet, I import the EPIC data:

RGset <- read.450k.exp(targets = targets)
RGset@annotation <- c(array = "IlluminaHumanMethylationEPIC", annotation = "ilm10b2.hg19")

Now I've a strange case when I remove all failed positions with non-significant p-values ( >0.05):

detP <- detectionP(RGset)

RGset2 <- RGset[rowSums(detP < 0.05) == ncol(RGset),  ]

dim(RGset)
Features  Samples
 1052641       18 

dim(RGset2)
Features  Samples
   39866       18 

 

As you can see the Features number is reduced drastically. The simple commands to remove the failed positions are the same that I've always used on 450k arrays and worked every time good, for example on 18 Samples in 450k arrays the Features number of RGset is 622399, obviously, and the Features number of RGset2 is 620199 so a reasonable difference.

What's going on using the same commands on EPIC arrays?

Thanks in advance, Regards

 

Giovanni

sessionInfo()
R version 3.2.2 (2015-08-14)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 14.04.4 LTS

locale:
 [1] LC_CTYPE=it_IT.UTF-8       LC_NUMERIC=C               LC_TIME=it_IT.UTF-8       
 [4] LC_COLLATE=it_IT.UTF-8     LC_MONETARY=it_IT.UTF-8    LC_MESSAGES=it_IT.UTF-8   
 [7] LC_PAPER=it_IT.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
[10] LC_TELEPHONE=C             LC_MEASUREMENT=it_IT.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats4    parallel  stats     graphics  grDevices utils     datasets  methods  
[9] base     

other attached packages:
 [1] IlluminaHumanMethylationEPICmanifest_0.3.0
 [2] minfi_1.16.0                              
 [3] bumphunter_1.10.0                         
 [4] locfit_1.5-9.1                            
 [5] iterators_1.0.8                           
 [6] foreach_1.4.3                             
 [7] Biostrings_2.38.3                         
 [8] XVector_0.10.0                            
 [9] SummarizedExperiment_1.0.2                
[10] GenomicRanges_1.22.3                      
[11] GenomeInfoDb_1.6.1                        
[12] IRanges_2.4.6                             
[13] S4Vectors_0.8.7                           
[14] lattice_0.20-33                           
[15] Biobase_2.30.0                            
[16] BiocGenerics_0.16.1                       

loaded via a namespace (and not attached):
 [1] mclust_5.1              rgl_0.95.1441           base64_1.1             
 [4] Rcpp_0.12.3             corpcor_1.6.8           Rsamtools_1.22.0       
 [7] digest_0.6.9            plyr_1.8.3              futile.options_1.0.0   
[10] ellipse_0.3-8           RSQLite_1.0.0           ggplot2_2.0.0          
[13] zlibbioc_1.16.0         GenomicFeatures_1.22.8  annotate_1.48.0        
[16] preprocessCore_1.32.0   splines_3.2.2           BiocParallel_1.4.3     
[19] stringr_1.0.0           igraph_1.0.1            RCurl_1.95-4.7         
[22] biomaRt_2.26.1          munsell_0.4.2           rtracklayer_1.30.1     
[25] multtest_2.26.0         pkgmaker_0.22           GEOquery_2.36.0        
[28] quadprog_1.5-5          codetools_0.2-14        matrixStats_0.50.1     
[31] XML_3.98-1.3            reshape_0.8.5           GenomicAlignments_1.6.3
[34] MASS_7.3-45             bitops_1.0-6            grid_3.2.2             
[37] nlme_3.1-122            xtable_1.8-0            gtable_0.1.2           
[40] registry_0.3            DBI_0.3.1               magrittr_1.5           
[43] scales_0.3.0            stringi_1.0-1           genefilter_1.52.0      
[46] doRNG_1.6               limma_3.26.5            futile.logger_1.4.1    
[49] nor1mix_1.2-1           lambda.r_1.1.7          RColorBrewer_1.1-2     
[52] mixOmics_5.2.0          siggenes_1.44.0         tools_3.2.2            
[55] illuminaio_0.12.0       rngtools_1.2.4          survival_2.38-3        
[58] AnnotationDbi_1.32.3    colorspace_1.2-6        beanplot_1.2   

 

ADD COMMENTlink modified 3.1 years ago • written 3.2 years ago by Giovanni Calice150
Answer: minfi failed positions in EPIC array
0
gravatar for James W. MacDonald
3.1 years ago by
United States
James W. MacDonald50k wrote:

So this line

RGset2 <- RGset[rowSums(detP < 0.05) == ncol(RGset),  ]

is removing all probes with even a single sample that has a 'large' detection p-value. This is a really stringent criterion, especially given that you have 18 samples. You might be better served to exclude probes that have too many large detection p-values, rather than any large detection p-value, where 'too many' is some smallish percentage.

ADD COMMENTlink written 3.1 years ago by James W. MacDonald50k

Hi James,

Ok this is a stringent criterion but It worked fine on 450k arrays analyses I run, also on more than samples such as 50 or 60 with a reasonable post-filtering difference. I'll consider what you suggest about filtering probes with a smallish percentage of detection p-values (50%-20%). Only I wish there was not something bias the construction of RGSet reading raw data from IDAT files in EPIC array by this minfi version; in particular in this analysis the 18 samples are distributed on 8 arrays.

Sure full support for EPIC arrays requires greater minfi version and this would prevent 'possible' bias in reading EPIC raw data, since the probes number has almost doubled compared to 450k.

 

Many Thanks,

Giovanni

 

 

ADD REPLYlink modified 3.1 years ago • written 3.1 years ago by Giovanni Calice150
Answer: minfi failed positions in EPIC array
0
gravatar for Kasper Daniel Hansen
3.1 years ago by
United States
Kasper Daniel Hansen6.4k wrote:
Giovanni, You're using a version of minfi which is more than 6 months old, and is not suited at all for working with the EPIC array. I strongly recommend you update to either the latest release (1.18) or the latest devel (1.19.6 - just submitted), if you want to use EPIC arrays. Things are constantly improving. Using minfi 1.19.6 and minfiDataEPIC I get library(minfiDataEPIC) det = detectionP(RGsetEPIC) colSums(det <= 0.05) 200144450018_R04C01 200144450019_R07C01 200144450021_R05C01 865911 865895 866219 which looks good to me. Kasper
ADD COMMENTlink modified 3.1 years ago by Dan Tenenbaum8.2k • written 3.1 years ago by Kasper Daniel Hansen6.4k

Hi Kasper,

I conclude some analyses and then I update to the latest minfi release.

 

Best,

Giovanni

ADD REPLYlink written 3.1 years ago by Giovanni Calice150
Answer: minfi failed positions in EPIC array
0
gravatar for Giovanni Calice
3.1 years ago by
Italy
Giovanni Calice150 wrote:

Hi Kasper, Hi all

I update my configuration to R 3.3.1 / Bioconductor 3.3 and so to the latest minfi release 1.18.2.

I tested the Illumina Demo EPIC dataset

detectionDemoEPIC <- detectionP(RGsetDemoEPIC)
colSums(detectionDemoEPIC <= 0.05)
200144450018_R04C01 200144450019_R07C01 200144450021_R05C01
             865911              865895              866219 

and It looks good to me (but It was fine also by previous release).

...

ADD COMMENTlink modified 3.1 years ago • written 3.1 years ago by Giovanni Calice150

When I read again my dataset of 18 samples distributed on 8 arrays, I get

detectionMyDemoEPIC <- detectionP(RGsetMyDemoEPIC)
colSums(detectionMyDemoEPIC < 0.05)
200325580029_R02C01 200325580029_R08C01 200498360056_R05C01
             866759              866687               33801
200498360134_R02C01 200498360167_R02C01 200498360167_R07C01
              38362               36149               29778
200503840026_R04C01 200607100008_R07C01 200607100044_R04C01
              36616               40095               69074
200325580029_R06C01 200325580076_R03C01 200498360056_R08C01
             866761              866734               32136
200498360134_R05C01 200498360167_R05C01 200503840026_R02C01
              31250               29119               31337
200503840026_R07C01 200607100008_R05C01 200607100044_R08C01
              24907               35396               38388 

that seems not fine for samples of 6 arrays.

...

ADD REPLYlink written 3.1 years ago by Giovanni Calice150

So I tested all samples of these arrays 200498360056 and 200498360134

detectionMyDemoEPIC.array.200498360056 <- detectionP(RGsetMyDemoEPIC.array.200498360056)

colSums(detectionMyDemoEPIC.array.200498360056 < 0.05)
200498360056_R01C01 200498360056_R02C01 200498360056_R03C01
             866738              866768              866803
200498360056_R04C01 200498360056_R05C01 200498360056_R06C01
             866531              866541              866571
200498360056_R07C01 200498360056_R08C01
             866610              866497 

detectionMyDemoEPIC.array.200498360134 <- detectionP(RGsetMyDemoEPIC.array.200498360134)

colSums(detectionMyDemoEPIC.array.200498360134 < 0.05)
200498360134_R01C01 200498360134_R02C01 200498360134_R03C01
             866771              866782              866787
200498360134_R04C01 200498360134_R05C01 200498360134_R06C01
             866781              866784              866783
200498360134_R07C01 200498360134_R08C01
             866789              866785 

and the detection p-value estimates are fine also for 200498360056_R05C01, 200498360056_R08C01, 200498360134_R02C01, 200498360134_R05C01 that in the dataset of 18 samples distributed on 8 arrays aren't good.

...

ADD REPLYlink written 3.1 years ago by Giovanni Calice150

I can't understand this strange trend in detection p-values, why looks it good in one array dataset and doesn't it in multi-arrays dataset?

Any ideas, suggestions?

Thanks in advance,

Giovanni

sessionInfo()
R version 3.3.1 (2016-06-21)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 14.04.4 LTS

locale:
 [1] LC_CTYPE=it_IT.UTF-8       LC_NUMERIC=C               LC_TIME=it_IT.UTF-8        LC_COLLATE=it_IT.UTF-8     LC_MONETARY=it_IT.UTF-8   
 [6] LC_MESSAGES=it_IT.UTF-8    LC_PAPER=it_IT.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=it_IT.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats4    parallel  stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] BiocInstaller_1.22.3                       IlluminaHumanMethylationEPICmanifest_0.3.0 minfi_1.18.2                              
 [4] bumphunter_1.12.0                          locfit_1.5-9.1                             iterators_1.0.8                           
 [7] foreach_1.4.3                              Biostrings_2.40.2                          XVector_0.12.0                            
[10] SummarizedExperiment_1.2.3                 GenomicRanges_1.24.2                       GenomeInfoDb_1.8.1                        
[13] IRanges_2.6.1                              S4Vectors_0.10.1                           lattice_0.20-33                           
[16] Biobase_2.32.0                             BiocGenerics_0.18.0                       

loaded via a namespace (and not attached):
 [1] httr_1.2.1                                 nor1mix_1.2-1                              splines_3.3.1                             
 [4] doRNG_1.6                                  Rsamtools_1.24.0                           impute_1.46.0                             
 [7] RSQLite_1.0.0                              limma_3.28.14                              quadprog_1.5-5                            
[10] chron_2.3-47                               digest_0.6.9                               RColorBrewer_1.1-2                        
[13] qvalue_2.4.2                               colorspace_1.2-6                           fastICA_1.2-0                             
[16] preprocessCore_1.34.0                      Matrix_1.2-6                               plyr_1.8.4                                
[19] GEOquery_2.38.4                            siggenes_1.46.0                            XML_3.98-1.4                              
[22] biomaRt_2.28.0                             genefilter_1.54.2                          zlibbioc_1.18.0                           
[25] xtable_1.8-2                               scales_0.4.0                               RefFreeEWAS_2.0                           
[28] IlluminaHumanMethylation450kmanifest_0.4.0 BiocParallel_1.6.2                         openssl_0.9.4                             
[31] annotate_1.50.0                            beanplot_1.2                               pkgmaker_0.22                             
[34] mgcv_1.8-12                                ggplot2_2.1.0                              GenomicFeatures_1.24.3                    
[37] survival_2.39-5                            magrittr_1.5                               mclust_5.2                                
[40] RPMM_1.20                                  doParallel_1.0.10                          nlme_3.1-128                              
[43] MASS_7.3-45                                isva_1.8                                   tools_3.3.1                               
[46] registry_0.3                               data.table_1.9.6                           matrixStats_0.50.2                        
[49] stringr_1.0.0                              munsell_0.4.3                              cluster_2.0.4                             
[52] rngtools_1.2.4                             AnnotationDbi_1.34.3                       base64_2.0                                
[55] grid_3.3.1                                 RCurl_1.95-4.8                             marray_1.50.0                             
[58] bitops_1.0-6                               DNAcopy_1.46.0                             gtable_0.2.0                              
[61] codetools_0.2-14                           multtest_2.28.0                            DBI_0.4-1                                 
[64] reshape_0.8.5                              reshape2_1.4.1                             R6_2.1.2                                  
[67] illuminaio_0.14.0                          GenomicAlignments_1.8.3                    rtracklayer_1.32.1                        
[70] wateRmelon_1.16.0                          stringi_1.1.1                              sva_3.20.0                                
[73] Rcpp_0.12.5  
ADD REPLYlink modified 3.1 years ago • written 3.1 years ago by Giovanni Calice150
Please use 1.18.3 or greater, not 1.18.2
ADD REPLYlink modified 3.1 years ago by Dan Tenenbaum8.2k • written 3.1 years ago by Kasper Daniel Hansen6.4k

Hi Kasper,

I'm using the minfi latest dev version 1.19.2 and update all packages by biocLite()

sessionInfo()
R version 3.3.1 (2016-06-21)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 14.04.4 LTS

locale:
 [1] LC_CTYPE=it_IT.UTF-8       LC_NUMERIC=C               LC_TIME=it_IT.UTF-8       
 [4] LC_COLLATE=it_IT.UTF-8     LC_MONETARY=it_IT.UTF-8    LC_MESSAGES=it_IT.UTF-8   
 [7] LC_PAPER=it_IT.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
[10] LC_TELEPHONE=C             LC_MEASUREMENT=it_IT.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats4    parallel  stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] BiocInstaller_1.22.3                       IlluminaHumanMethylationEPICmanifest_0.3.0
 [3] minfi_1.19.2                               bumphunter_1.12.0                         
 [5] locfit_1.5-9.1                             iterators_1.0.8                           
 [7] foreach_1.4.3                              Biostrings_2.40.2                         
 [9] XVector_0.12.0                             SummarizedExperiment_1.2.3                
[11] GenomicRanges_1.24.2                       GenomeInfoDb_1.8.1                        
[13] IRanges_2.6.1                              S4Vectors_0.10.1                          
[15] lattice_0.20-33                            Biobase_2.32.0                            
[17] BiocGenerics_0.18.0                       

loaded via a namespace (and not attached):
 [1] genefilter_1.54.2       splines_3.3.1           beanplot_1.2           
 [4] rtracklayer_1.32.1      GenomicFeatures_1.24.3  chron_2.3-47           
 [7] XML_3.98-1.4            survival_2.39-5         DBI_0.4-1              
[10] BiocParallel_1.6.2      RColorBrewer_1.1-2      registry_0.3           
[13] rngtools_1.2.4          doRNG_1.6               matrixStats_0.50.2     
[16] plyr_1.8.4              pkgmaker_0.22           stringr_1.0.0          
[19] zlibbioc_1.18.0         codetools_0.2-14        biomaRt_2.28.0         
[22] AnnotationDbi_1.34.3    illuminaio_0.14.0       preprocessCore_1.34.0  
[25] Rcpp_0.12.5             xtable_1.8-2            openssl_0.9.4          
[28] limma_3.28.14           base64_2.0              annotate_1.50.0        
[31] Rsamtools_1.24.0        digest_0.6.9            stringi_1.1.1          
[34] nor1mix_1.2-1           grid_3.3.1              quadprog_1.5-5         
[37] GEOquery_2.38.4         tools_3.3.1             bitops_1.0-6           
[40] magrittr_1.5            siggenes_1.46.0         RCurl_1.95-4.8         
[43] RSQLite_1.0.0           MASS_7.3-45             Matrix_1.2-6           
[46] data.table_1.9.6        httr_1.2.1              reshape_0.8.5          
[49] R6_2.1.2                mclust_5.2              GenomicAlignments_1.8.3
[52] multtest_2.28.0         nlme_3.1-128

...

 

ADD REPLYlink modified 3.1 years ago • written 3.1 years ago by Giovanni Calice150
detectionMyDemoEPIC <- detectionP(RGsetMyDemoEPIC)
colSums(detectionMyDemoEPIC < 0.05)

200325580029_R02C01 200325580029_R08C01 200498360056_R05C01 200498360134_R02C01
             866759              866687               33801               38362
200498360167_R02C01 200498360167_R07C01 200503840026_R04C01 200607100008_R07C01
              36149               29778               36616               40095
200607100044_R04C01 200325580029_R06C01 200325580076_R03C01 200498360056_R08C01
              69074              866761              866734               32136
200498360134_R05C01 200498360167_R05C01 200503840026_R02C01 200503840026_R07C01
              31250               29119               31337               24907
200607100008_R05C01 200607100044_R08C01
              35396               38388 

and then

detectionMyDemoEPIC.array.200498360056 <- detectionP(RGsetMyDemoEPIC.array.200498360056)
colSums(detectionMyDemoEPIC.array.200498360056 < 0.05)
200498360056_R01C01 200498360056_R02C01 200498360056_R03C01 200498360056_R04C01
             866738              866768              866803              866531
200498360056_R05C01 200498360056_R06C01 200498360056_R07C01 200498360056_R08C01
             866541              866571              866610              866497 

detectionMyDemoEPIC.array.200498360134 <- detectionP(RGsetMyDemoEPIC.array.200498360134)
colSums(detectionMyDemoEPIC.array.200498360134 < 0.05)
200498360134_R01C01 200498360134_R02C01 200498360134_R03C01 200498360134_R04C01
             866771              866782              866787              866781
200498360134_R05C01 200498360134_R06C01 200498360134_R07C01 200498360134_R08C01
             866784              866783              866789              866785 
ADD REPLYlink written 3.1 years ago by Giovanni Calice150

As you can see the detection p-value of 200498360056_R05C01, 200498360056_R08C01, 200498360134_R02C01, 200498360134_R05C01 for example, is fine only when I import one array at a time.

So in my analysis on 8 arrays, since 4 samples have good detection p-value, how should I proceed?

dim(RGsetMyDemoEPIC)
Features  Samples
 1052641       18

RGset2 <- RGsetMyDemoEPIC[rowSums(detectionMyDemoEPIC < 0.05) >= 4,  ]
dim(RGset2)
Features  Samples
 1052358       18 

RGset2 <- RGsetMyDemoEPIC[rowSums(detectionMyDemoEPIC < 0.05) <= 4,  ]
dim(RGset2)
Features  Samples
  943734       18 

RGset2 <- RGsetMyDemoEPIC[rowSums(detectionMyDemoEPIC < 0.05) == 4,  ]
dim(RGset2)
Features  Samples
  943451       18 

 

What do you think about?

Thanks,

Giovanni

 

ADD REPLYlink written 3.1 years ago by Giovanni Calice150
This strongly suggests your problems will go away if you use minfi 1.18.3 Best, Kasper (Sent from my phone.)
ADD REPLYlink modified 3.1 years ago by Dan Tenenbaum8.2k • written 3.1 years ago by Kasper Daniel Hansen6.4k

Thanks Kasper,

Where can I download minfi_1.18.3.tar.gz package source?

Best,

Giovanni

 

ADD REPLYlink written 3.1 years ago by Giovanni Calice150
It should be available through biocLite as a binary http://bioconductor.org/checkResults/release/bioc-LATEST/minfi/ Kasper
ADD REPLYlink modified 3.1 years ago by Dan Tenenbaum8.2k • written 3.1 years ago by Kasper Daniel Hansen6.4k
biocLite("minfi", siteRepos = "http://bioconductor.org/checkResults/release/bioc-LATEST/minfi/")
BioC_mirror: https://bioconductor.org
Using Bioconductor 3.3 (BiocInstaller 1.22.3), R 3.3.1 (2016-06-21).
Installing package(s) ‘minfi’

Warning: unable to access index for repository http://bioconductor.org/checkResults/release/bioc-LATEST/minfi/src/contrib:
  cannot download all files

So it goes to download minfi_1.18.2.tar.gz

'https://bioconductor.org/packages/3.3/bioc/src/contrib/minfi_1.18.2.tar.gz'
Content type 'unknown' length 321295 bytes (313 KB)
==================================================
downloaded 313 KB

* installing *source* package ‘minfi’ ...
** R
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
* DONE (minfi)

The downloaded source packages are in
    ‘/tmp/RtmpB2KYvW/downloaded_packages’

Why doesn't it work?

Giovanni

ADD REPLYlink modified 3.1 years ago • written 3.1 years ago by Giovanni Calice150

I also tried install_git {devtools} and it works but the minfi version is 1.19.7

that is the latest devel and not the latest release suggested.

Thanks,

Giovanni

ADD REPLYlink written 3.1 years ago by Giovanni Calice150

Hi Kasper,

I'm checking frequently the possible new release of minfi  that now is 1.18.2

and I'm checking also the BUILD link

http://bioconductor.org/checkResults/release/bioc-LATEST/minfi/

where the package source is 1.18.4; but if I continue to "force" through biocLite

the installation of the package source 1.18.4, there is always the same message:

Installing package(s) ‘minfi’
Warning: unable to access index for repository http://bioconductor.org/checkResults/release/bioc-LATEST/minfi/src/contrib:
  cannot download all files

and so It goes to download minfi_1.18.2.tar.gz

'https://bioconductor.org/packages/3.3/bioc/src/contrib/minfi_1.18.2.tar.gz'
Content type 'unknown' length 321295 bytes (313 KB)
==================================================
downloaded 313 KB

* installing *source* package ‘minfi’ ...
** R
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
* DONE (minfi)

The downloaded source packages are in
    ‘/tmp/Rtmp4hQKcD/downloaded_packages’
Warning: unable to access index for repository http://bioconductor.org/checkResults/release/bioc-LATEST/minfi/src/contrib:
  cannot download all files

I'm sorry but I need to proceed with the analysis on EPIC arrays.

I do not know if I can help on debugging code in reading and detection p-value methods.

 

Thanks in advance, Best

Giovanni

 

 

ADD REPLYlink written 3.1 years ago by Giovanni Calice150
You are indeed right: version 1.18.4 should be available for Linux / OS X, but not for Windows for some reason which seems outside of my control (missing Biobase, S4Vectors, which is pretty weird). Bioconductor has had some problems with the build system lately, so perhaps a small heads up is necessary. I have cc'ed Herve who may be able to help. Finally, if you are keen on using EPIC, I would very much recommend you update to Bioc level and try to get minfi version 1.9.10 (which I just posted yesterday so it is not yet available through the build system) Best, Kasper
ADD REPLYlink modified 3.1 years ago by Dan Tenenbaum8.2k • written 3.1 years ago by Kasper Daniel Hansen6.4k

Dear Kasper,

Ok I just tried the last minfi devel version 1.19.10; I installed it by git{devtools} and it works.

But I got the same strange results, the detection p-value is fine only when I import one array at a time and not fine in my experiment of 18 samples distributed on 8 arrays:

detectionMyDemoEPIC.array.200498360056 <- detectionP(RGsetMyDemoEPIC.array.200498360056)
colSums(detectionMyDemoEPIC.array.200498360056 < 0.05)
200498360056_R01C01 200498360056_R02C01
             866738              866768
200498360056_R03C01 200498360056_R04C01
             866803              866531
200498360056_R05C01 200498360056_R06C01
             866541              866571
200498360056_R07C01 200498360056_R08C01
             866610              866497 

detectionMyDemoEPIC.array.200498360134 <- detectionP(RGsetMyDemoEPIC.array.200498360134)
colSums(detectionMyDemoEPIC.array.200498360134 < 0.05)
200498360134_R01C01 200498360134_R02C01
             866771              866782
200498360134_R03C01 200498360134_R04C01
             866787              866781
200498360134_R05C01 200498360134_R06C01
             866784              866783
200498360134_R07C01 200498360134_R08C01
             866789              866785 

...

ADD REPLYlink modified 3.1 years ago • written 3.1 years ago by Giovanni Calice150

and then

detectionMyDemoEPIC <- detectionP(RGsetMyDemoEPIC)
colSums(detectionMyDemoEPIC < 0.05)
200325580029_R02C01 200325580029_R08C01
             866759              866687
200498360056_R05C01 200498360134_R02C01
              33801               38362
200498360167_R02C01 200498360167_R07C01
              36149               29778
200503840026_R04C01 200607100008_R07C01
              36616               40095
200607100044_R04C01 200325580029_R06C01
              69074              866761
200325580076_R03C01 200498360056_R08C01
             866734               32136
200498360134_R05C01 200498360167_R05C01
              31250               29119
200503840026_R02C01 200503840026_R07C01
              31337               24907
200607100008_R05C01 200607100044_R08C01
              35396               38388 

...

ADD REPLYlink written 3.1 years ago by Giovanni Calice150

sessionInfo()
R version 3.3.1 (2016-06-21)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 14.04.4 LTS

locale:
 [1] LC_CTYPE=it_IT.UTF-8       LC_NUMERIC=C               LC_TIME=it_IT.UTF-8        LC_COLLATE=it_IT.UTF-8    
 [5] LC_MONETARY=it_IT.UTF-8    LC_MESSAGES=it_IT.UTF-8    LC_PAPER=it_IT.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=it_IT.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats4    parallel  stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] IlluminaHumanMethylationEPICmanifest_0.3.0 minfi_1.19.10                             
 [3] bumphunter_1.12.0                          locfit_1.5-9.1                            
 [5] iterators_1.0.8                            foreach_1.4.3                             
 [7] Biostrings_2.40.2                          XVector_0.12.0                            
 [9] SummarizedExperiment_1.2.3                 GenomicRanges_1.24.2                      
[11] GenomeInfoDb_1.8.3                         IRanges_2.6.1                             
[13] S4Vectors_0.10.2                           lattice_0.20-33                           
[15] Biobase_2.32.0                             BiocGenerics_0.18.0                       

...

ADD REPLYlink written 3.1 years ago by Giovanni Calice150

loaded via a namespace (and not attached):
[1] genefilter_1.54.2       splines_3.3.1           beanplot_1.2            rtracklayer_1.32.1      GenomicFeatures_1.24.4
[6] chron_2.3-47            XML_3.98-1.4            survival_2.39-5         DBI_0.4-1               BiocParallel_1.6.2    
[11] RColorBrewer_1.1-2      registry_0.3            rngtools_1.2.4          doRNG_1.6               matrixStats_0.50.2    
[16] plyr_1.8.4              pkgmaker_0.22           stringr_1.0.0           zlibbioc_1.18.0         codetools_0.2-14      
[21] biomaRt_2.28.0          AnnotationDbi_1.34.4    illuminaio_0.14.0       preprocessCore_1.34.0   Rcpp_0.12.5           
[26] xtable_1.8-2            openssl_0.9.4           limma_3.28.17           base64_2.0              annotate_1.50.0       
[31] Rsamtools_1.24.0        digest_0.6.9            stringi_1.1.1           nor1mix_1.2-1           grid_3.3.1            
[36] quadprog_1.5-5          GEOquery_2.38.4         tools_3.3.1             bitops_1.0-6            magrittr_1.5          
[41] siggenes_1.46.0         RCurl_1.95-4.8          RSQLite_1.0.0           MASS_7.3-45             Matrix_1.2-6          
[46] data.table_1.9.6        httr_1.2.1              reshape_0.8.5           R6_2.1.2                mclust_5.2            
[51] GenomicAlignments_1.8.4 multtest_2.28.0         nlme_3.1-128

...

ADD REPLYlink modified 3.1 years ago • written 3.1 years ago by Giovanni Calice150

Considering that version 1.18.4 should be available for Linux, is It possible to synchronize the Build Server and my Linux Client to download it (skipping Windows files) and avoiding the warning message

Warning: unable to access index for repository http://bioconductor.org/checkResults/release/bioc-LATEST/minfi/src/contrib: > cannot download all files

and so test the minfi 1.18.4 version?

 

Many Thanks, Best

Giovanni

ADD REPLYlink written 3.1 years ago by Giovanni Calice150
Did you re-create detectionMyDemoEPIC.array.200498360056 from scratch? The changes requires you to re-read the IDAT files using read.metharray. Best, Kasper
ADD REPLYlink modified 3.1 years ago by Dan Tenenbaum8.2k • written 3.1 years ago by Kasper Daniel Hansen6.4k

Hi Kasper,

Yes, of course, I remove all objects and re-read the .idat files then I re-create the detection p-value matrix; all this for each single array experiment (detectionMyDemoEPIC.array.200498360056, detectionMyDemoEPIC.array.200498360134) and for multiple arrays experiment (detectionMyDemoEPIC).

Today I'm also testing the new devel minfi version 1.19.11 but the result doesn't change.

Best,

Giovanni

ADD REPLYlink written 3.1 years ago by Giovanni Calice150
1
Giovanni, Thanks a lot for reporting this and also making your IDAT files available. I have confirmed this is a bug, which is fixed in minfi 1.19.12 (devel) minfi 1.18.6 (release) The devel version is available from GitHub; the other two versions have just been committed to the Bioconductor subversion repository and should be available "soon". This bug is critical; all users should update. It is caused by assuming that a specific ordering is constant across IDAT files with the same number of probes. This is no longer assumed, but the read.metharray function is now slower. The bug can be observed if you get very few (< 50.000 say) probes with a detectionP p-value < 0.05 (however note that this can also be caused by a failed array). The bug only happens if you read in IDAT files with different orderings in the same RGChannelSet; I assume this is rare. Best, Kasper On Wed, Jul 27, 2016 at 5:28 AM, Giovanni Calice [bioc] < noreply@bioconductor.org> wrote: > Activity on a post you are following on support.bioconductor.org > > User Giovanni Calice <https: support.bioconductor.org="" u="" 6415=""/> wrote Comment: > minfi failed positions in EPIC array > <https: support.bioconductor.org="" p="" 84363="" #85470="">: > > Hi Kasper, > > Yes, of course, I remove all objects and re-read the .idat files then I > re-create the detection p-value matrix; all this for each single array > experiment (detectionMyDemoEPIC.array.200498360056, > detectionMyDemoEPIC.array.200498360134) and for multiple arrays experiment > (detectionMyDemoEPIC). > > Today I'm also testing the new devel minfi version 1.19.11 but the result > doesn't change. > > Best, > > Giovanni > > ------------------------------ > > Post tags: minfi, illuminahumanmethylationepicanno.ilmn10b.hg19 > > You may reply via email or visit > C: minfi failed positions in EPIC array >
ADD REPLYlink written 3.1 years ago by Kasper Daniel Hansen6.4k

Dear Kasper,

Finally We did it, It works and works fine:

200325580029_R02C01 200325580029_R08C01
             866759              866687
200498360056_R05C01 200498360134_R02C01
             866541              866782
200498360167_R02C01 200498360167_R07C01
             866724              866601
200503840026_R04C01 200607100008_R07C01
             866605              866706
200607100044_R04C01 200325580029_R06C01
             866707              866761
200325580076_R03C01 200498360056_R08C01
             866734              866497
200498360134_R05C01 200498360167_R05C01
             866784              866739
200503840026_R02C01 200503840026_R07C01
             866580              866590
200607100008_R05C01 200607100044_R08C01
             866549              866717 

In some tests executed, I noticed that there was something strange in the orderings.

For what concerns read.metharray(), I don't notice an unusual delay reading in .idat files at least for 18 samples.

Thanks to you for everything, Best

Giovanni

ADD REPLYlink written 3.1 years ago by Giovanni Calice150
Could you send me the IDAT files? Best, Kasper (Sent from my phone.) > On Jul 27, 2016, at 11:28, Giovanni Calice [bioc] <noreply@bioconductor.org> wrote: > > Activity on a post you are following on support.bioconductor.org > User Giovanni Calice wrote Comment: minfi failed positions in EPIC array: > > Hi Kasper, > > Yes, of course, I remove all objects and re-read the .idat files then I re-create the detection p-value matrix; all this for each single array experiment (detectionMyDemoEPIC.array.200498360056, detectionMyDemoEPIC.array.200498360134) and for multiple arrays experiment (detectionMyDemoEPIC). > > Today I'm also testing the new devel minfi version 1.19.11 but the result doesn't change. > > Best, > > Giovanni > > Post tags: minfi, illuminahumanmethylationepicanno.ilmn10b.hg19 > > You may reply via email or visit C: minfi failed positions in EPIC array
ADD REPLYlink written 3.1 years ago by Kasper Daniel Hansen6.4k
Kasper Could I send .idat files as G Drive link on khansen@jhsph.edu or on your Gmail address (but it is not shown completely on Bioc Support site)? Best, Giovanni <khansen@jhsph.edu> 2016-07-27 11:50 GMT+02:00 Kasper Daniel Hansen [bioc] < noreply@bioconductor.org>: > Activity on a post you are following on support.bioconductor.org > > User Kasper Daniel Hansen <https: support.bioconductor.org="" u="" 2979=""/> > wrote Comment: minfi failed positions in EPIC array > <https: support.bioconductor.org="" p="" 84363="" #85471="">: > > Could you send me the IDAT files? Best, Kasper (Sent from my phone.) > On > Jul 27, 2016, at 11:28, Giovanni Calice [bioc] <noreply@bioconductor.org> > wrote: > > Activity on a post you are following on > support.bioconductor.org > User Giovanni Calice wrote Comment: minfi > failed positions in EPIC array: > > Hi Kasper, > > Yes, of course, I remove > all objects and re-read the .idat files then I re-create the detection > p-value matrix; all this for each single array experiment > (detectionMyDemoEPIC.array.200498360056, > detectionMyDemoEPIC.array.200498360134) and for multiple arrays experiment > (detectionMyDemoEPIC). > > Today I'm also testing the new devel minfi > version 1.19.11 but the result doesn't change. > > Best, > > Giovanni > > > Post tags: minfi, illuminahumanmethylationepicanno.ilmn10b.hg19 > > You may > reply via email or visit C: minfi failed positions in EPIC array > <https: support.bioconductor.org="" p="" 84363="" #85470=""> > > ------------------------------ > > Post tags: minfi, illuminahumanmethylationepicanno.ilmn10b.hg19 > > You may reply via email or visit > C: minfi failed positions in EPIC array >
ADD REPLYlink written 3.1 years ago by Giovanni Calice150
Send it to me on kasperdanielhansen@gmail.com or share it on either Dropbox or google drive, same email address. Best, Kasper (Sent from my phone.) > On Jul 27, 2016, at 12:51, Giovanni Calice [bioc] <noreply@bioconductor.org> wrote: > > Activity on a post you are following on support.bioconductor.org > User Giovanni Calice wrote Comment: minfi failed positions in EPIC array: > > Kasper Could I send .idat files as G Drive link on khansen@jhsph.edu or on your Gmail address (but it is not shown completely on Bioc Support site)? Best, Giovanni <khansen@jhsph.edu> 2016-07-27 11:50 GMT+02:00 Kasper Daniel Hansen [bioc] < noreply@bioconductor.org>: > Activity on a post you are following on support.bioconductor.org > > User Kasper Daniel Hansen <https: support.bioconductor.org="" u="" 2979=""/> > wrote Comment: minfi failed positions in EPIC array > <https: support.bioconductor.org="" p="" 84363="" #85471="">: > > Could you send me the IDAT files? Best, Kasper (Sent from my phone.) > On > Jul 27, 2016, at 11:28, Giovanni Calice [bioc] <noreply@bioconductor.org> > wrote: > > Activity on a post you are following on > support.bioconductor.org > User Giovanni Calice wrote Comment: minfi > failed positions in EPIC array: > > Hi Kasper, > > Yes, of course, I remove > all objects and re-read the .idat files then I re-create the detection > p-value matrix; all this for each single array experiment > (detectionMyDemoEPIC.array.200498360056, > detectionMyDemoEPIC.array.200498360134) and for multiple arrays experiment > (detectionMyDemoEPIC). > > Today I'm also testing the new devel minfi > version 1.19.11 but the result doesn't change. > > Best, > > Giovanni > > > Post tags: minfi, illuminahumanmethylationepicanno.ilmn10b.hg19 > > You may > reply via email or visit C: minfi failed positions in EPIC array > <https: support.bioconductor.org="" p="" 84363="" #85470=""> > > ------------------------------ > > Post tags: minfi, illuminahumanmethylationepicanno.ilmn10b.hg19 > > You may reply via email or visit > C: minfi failed positions in EPIC array > > Post tags: minfi, illuminahumanmethylationepicanno.ilmn10b.hg19 > > You may reply via email or visit C: minfi failed positions in EPIC array
ADD REPLYlink written 3.1 years ago by Kasper Daniel Hansen6.4k

Hi Kasper, Giovanni,

Yes we're experiencing difficulties with the builds on Windows lately so it can take more than usual for changes in a package to propagate (2-4 days in release, 4-7 days in devel). We're actively working on finding a solution.

In the case of minfi though, there seems to be a real error in the examples for bumphunter-methods and with the unit tests. See R CMD check result for Linux and Mac. I can reproduce this on my laptop. This prevents version 1.18.4 from propagating. Note that it seems that 1.18.3 didn't propagate either. The version currently available via biocLite() is 1.18.2, from May.

Cheers,

H.

ADD REPLYlink written 3.1 years ago by Hervé Pagès ♦♦ 14k

Hi Hervé,

 

Thanks for clarifications.

 

Best,

Giovanni

 

ADD REPLYlink written 3.1 years ago by Giovanni Calice150
Please log in to add an answer.

Help
Access

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.
Powered by Biostar version 16.09
Traffic: 254 users visited in the last hour