Problem installing package, version of dependency issue
1
0
Entering edit mode
@alexandreblais-23753
Last seen 10 weeks ago
Canada

Hello

I am facing this strange issue where installation of a package (GO.db) fails due to a dependency (AnnotationDbi) not having the correct version, even though in fact it is the latest available version. The dependency loads fine and shows properly in sessionInfo(). I should specify that this is on a large compute cluster. I install all my packages in a personal library, to which I point with .libPaths(). I am facing no such issue with over 100 other packages.

Thanks for any advice on how I may solve this problem.

Alex


R version 4.3.1 (2023-06-16) -- "Beagle Scouts"

> .libPaths("/project/6026769/R/x86_64-pc-linux-gnu-library/4.3.1")
> library("AnnotationDbi")
Error in library("AnnotationDbi") :
  there is no package called 'AnnotationDbi'

> BiocManager::install("AnnotationDbi", lib="/project/6026769/R/x86_64-pc-linux-gnu-library/4.3.1")
Bioconductor version 3.18 (BiocManager 1.30.22), R 4.3.1 (2023-06-16)
Installing package(s) 'AnnotationDbi'
trying URL 'https://bioconductor.org/packages/3.18/bioc/src/contrib/AnnotationDbi_1.64.1.tar.gz'
Content type 'application/x-gzip' length 4359564 bytes (4.2 MB)
==================================================
downloaded 4.2 MB

[[[snip]]]
* DONE (AnnotationDbi)

The downloaded source packages are in
        '/tmp/RtmpDciMJ7/downloaded_packages'
> library("AnnotationDbi")
Loading required package: stats4
Loading required package: BiocGenerics

Attaching package: 'BiocGenerics'
[[[snip]]]



> BiocManager::install("GO.db", lib="/project/6026769/R/x86_64-pc-linux-gnu-library/4.3.1")
Bioconductor version 3.18 (BiocManager 1.30.22), R 4.3.1 (2023-06-16)
Installing package(s) 'GO.db'
trying URL 'https://bioconductor.org/packages/3.18/data/annotation/src/contrib/GO.db_3.18.0.tar.gz'
Content type 'application/x-gzip' length 29013599 bytes (27.7 MB)
==================================================
downloaded 27.7 MB

* installing *source* package 'GO.db' ...
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
Error: package 'AnnotationDbi' 1.60.2 was found, but >= 1.63.2 is required by 'GO.db'
Execution halted
ERROR: lazy loading failed for package 'GO.db'
* removing '/project/6026769/R/x86_64-pc-linux-gnu-library/4.3.1/GO.db'

The downloaded source packages are in
        '/tmp/RtmpDciMJ7/downloaded_packages'
Warning message:
In install.packages(...) :
  installation of package 'GO.db' had non-zero exit status



sessionInfo( )
R version 4.3.1 (2023-06-16)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)

Matrix products: default
BLAS/LAPACK: FlexiBLAS IMKL;  LAPACK version 3.9.0

locale:
 [1] LC_CTYPE=en_CA.UTF-8       LC_NUMERIC=C
 [3] LC_TIME=en_CA.UTF-8        LC_COLLATE=en_CA.UTF-8
 [5] LC_MONETARY=en_CA.UTF-8    LC_MESSAGES=en_CA.UTF-8
 [7] LC_PAPER=en_CA.UTF-8       LC_NAME=C
 [9] LC_ADDRESS=C               LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_CA.UTF-8 LC_IDENTIFICATION=C

time zone: America/Vancouver
tzcode source: system (glibc)

attached base packages:
[1] stats4    stats     graphics  grDevices utils     datasets  methods
[8] base

other attached packages:
[1] AnnotationDbi_1.64.1 IRanges_2.36.0       S4Vectors_0.40.2
[4] Biobase_2.62.0       BiocGenerics_0.48.1

loaded via a namespace (and not attached):
 [1] crayon_1.5.2            vctrs_0.6.5             httr_1.4.7
 [4] cli_3.6.2               rlang_1.1.3             DBI_1.2.1
 [7] png_0.1-8               bit_4.0.5               RCurl_1.98-1.14
[10] Biostrings_2.70.1       KEGGREST_1.42.0         bitops_1.0-7
[13] fastmap_1.1.1           GenomeInfoDb_1.38.5     memoise_2.0.1
[16] BiocManager_1.30.22     compiler_4.3.1          RSQLite_2.3.4
[19] blob_1.2.4              XVector_0.42.0          R6_2.5.1
[22] GenomeInfoDbData_1.2.11 tools_4.3.1             bit64_4.0.5
[25] zlibbioc_1.48.0         cachem_1.0.8
AnnotationDbi GO.db • 1.2k views
ADD COMMENT
0
Entering edit mode
@james-w-macdonald-5106
Last seen 1 day ago
United States

Is AnnotationDbi also installed in a path that is upstream of your local path? What does .libPaths() return? Ideally the site-wide library dir would be after your local library dir.

0
Entering edit mode

My personal library is first in the list

> .libPaths()
[1] "/project/6026769/R/x86_64-pc-linux-gnu-library/4.3.1"
[2] "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/r/4.3.1/lib64/R/library"

However, I am not sure if I understand your first question, about "upstream of my local path". I do have the same package installed in a path at the same level "/project/6026769/R/x86_64-pc-linux-gnu-library/4.2.2" and more, all the way down to version 3.6 of R. Could this be what is causing me trouble? That would be unexpected for me, considering that libPaths explicitely indicates to go to a specific folder.

ADD REPLY
0
Entering edit mode

What I meant was that the sysadmin might have already installed AnnotationDbi in the site-wide library dir, and if your .libPaths variable had the site-wide dir first, then you would be comparing versus that one rather than your local version.

Having other libdirs that you use for other versions of R is fine (I have the same setup), so I wouldn't expect that to be a problem. I guess the obvious thing to try is

library(BiocManager)
valid()

And see if that picks something up. I wouldn't expect it to, but it's worth a try.

ADD REPLY
0
Entering edit mode

Thanks for helping me with this. Unfortunately (or not?), there doesn't seem to be any problem with Bioc

> library(BiocManager)
Bioconductor version 3.18 (BiocManager 1.30.22), R 4.3.1 (2023-06-16)
> valid()
[1] TRUE
ADD REPLY
0
Entering edit mode

And if you try to install GO.db a second time it still fails?

ADD REPLY
0
Entering edit mode

Yes, it still fails with the same error message about the version of AnnotationDbi being too low. No matter if I issue library("AnnotationDbi") ahead of the install call or not.

I realize this problem may be caused by some particular configuration of the cluster I am using. I don't have the skills to figure this out myself, and I am not expecting you to troubleshoot this kind of problem. I have asked the cluster tech support team for some help. I was provided with a working but "trivial" solution to load a different R module that comes pre-bundled with Bioconductor. Unfortunately, no clear explanation as to why the GO.db installation does not see my local AnnotationDbi library...

ADD REPLY
0
Entering edit mode

This is a weird one, and might be difficult to track down. These messages

* installing *source* package 'GO.db' ...
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading

all come from tools:::.install_packages, which is called 'outside' of the R session that you are installing from. In other words, all packages are installed using R CMD INSTALL, which is a shell script that calls another instance of the R binary to do the actual installation. When you use BiocManager::install it checks to ensure that the versions all line up, and then passes the correct packages to install to utils::install.packages, which then (among other things) generates the correct arguments for R CMD INSTALL and passes them off to that script using system2, which is a way to make system calls from within R.

What that means is you cannot do the obvious thing - run tools:::.install_packages under the debugger - to figure out why the 'wrong' version of AnnotationDbi is being identified, because it's being run by a different instance of R.

ADD REPLY
0
Entering edit mode

Pardon my ignorance, but would it be possible then to pass the location of my R libraries with something like export R_LIBS=/project/6026769/R/x86_64-pc-linux-gnu-library/4.3.1 or as a variable in the appropriate R CMD command?

ADD REPLY
1
Entering edit mode

It already does. If I install GO.db under the debugger (for install.packages), and proceed through the function, I end up here:

debug: args <- c(args0, get_install_opts(fil), "-l", shQuote(update[i, 
    2L]), getConfigureArgs(fil), getConfigureVars(fil), shQuote(fil))
Browse[2]> 
debug: status <- system2(cmd0, args, env = env, stdout = outfile, stderr = outfile, 
    timeout = tlim)
Browse[2]> cmd0
[1] "/share/apps/R/R-4.3.1/lib64/R/bin/R"
Browse[2]> args
[1] "CMD"                                                                                 
[2] "INSTALL"                                                                             
[3] "-l"                                                                                  
[4] "'/share/groups/edge/R/R-4.3.1'"                                                      
[5] "'/share/groups/edge/analyses/tmp/RtmpqCH5c3/downloaded_packages/GO.db_3.18.0.tar.gz'"
Browse[2]>

So what is being passed to system2 is the path to the R binary (cmd0), and a set of arguments, which will then create a system call of

/share/apps/R/R-4.3.1/lib64/R/bin/R CMD INSTALL -l /share/groups/edge/R/R-4.3.1 /share/groups/edge/R/RtmpqCH5c3/downloaded_packages/GO.db_3.18.0.tar.gz

which is already pointing to the correct library dir.

ADD REPLY
0
Entering edit mode

Could you check that AnnotationDbi is really installed with the right version in the desired library via installed.packages(lib.loc = "/project/6026769/R/x86_64-pc-linux-gnu-library/4.3.1") ? Is this with a fresh installation without loading anything? This might also cause problems if the package from the system is loaded before you install and try to load the new one in your personal/project library

ADD REPLY
0
Entering edit mode

I am not sure about your question regarding "fresh installation", as I work on a cluster where I load pre-installed modules for R and its dependencies. However, without loading anything, immediately after starting my R session, I get:

> pack_list <- installed.packages(lib.loc = "/project/6026769/R/x86_64-pc-linux-gnu-library/4.3.1")
> pack_list[1:5, c("LibPath", "Version")]
              LibPath                                                Version
annotate      "/project/6026769/R/x86_64-pc-linux-gnu-library/4.3.1" "1.80.0"
AnnotationDbi "/project/6026769/R/x86_64-pc-linux-gnu-library/4.3.1" "1.64.1"
AnnotationHub "/project/6026769/R/x86_64-pc-linux-gnu-library/4.3.1" "3.10.0"
ape           "/project/6026769/R/x86_64-pc-linux-gnu-library/4.3.1" "5.7-1"
aplot         "/project/6026769/R/x86_64-pc-linux-gnu-library/4.3.1" "0.2.2"
ADD REPLY
0
Entering edit mode

Yes, by fresh installation, I meant without loading anything before doing the installation steps. Now it seems you fulfill the GO.db requirements: package 'AnnotationDbi' 1.63.2 is installed. Can you try again without doing anything else: BiocManager::install("GO.db", lib = "/project/6026769/R/x86_64-pc-linux-gnu-library/4.3.1"), if it finds some missing dependencies BiocManager should install them.

If you still have problems I recommend to create an .Rprofile in the directory where you lunch the R process with a line reporting the libraries it has (just a print(.libPaths()\n should be enough but you might want to add print(packageVersion("AnnotationDbi")) . Maybe the way it loads R interferes with finding the package (which shouldn't happen).

ADD REPLY
0
Entering edit mode
[ablai2@cedar1 def-ablai2]$ cat .Rprofile
.libPaths("/project/6026769/R/x86_64-pc-linux-gnu-library/4.3.1")
print(.libPaths())
print(packageVersion("AnnotationDbi"))

and


R version 4.3.1 (2023-06-16) -- "Beagle Scouts"
Copyright (C) 2023 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

[1] "/project/6026769/R/x86_64-pc-linux-gnu-library/4.3.1"
[2] "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/r/4.3.1/lib64/R/library"
Error in packageVersion("AnnotationDbi") :
  could not find function "packageVersion"
> print(packageVersion("AnnotationDbi"))
[1] '1.64.1'

I am unsure why packageVersion fails when the .Rprofile script is run at the beginning of the R session, but the command works immediately afterwards.

ADD REPLY
0
Entering edit mode

Maybe utils is not loaded yet. It might work if you use utils::packageVersion. But anyway, does this mean that you still cannot install GO.db? Clever idea to put the .libPaths in the R profile, this might help to find the right library when installing packages

ADD REPLY
0
Entering edit mode
[ablai2@cedar1 def-ablai2]$ cat .Rprofile
.libPaths("/project/6026769/R/x86_64-pc-linux-gnu-library/4.3.1")
print(.libPaths())
print(utils::packageVersion("AnnotationDbi"))

and

[ablai2@cedar1 def-ablai2]$ R

R version 4.3.1 (2023-06-16) -- "Beagle Scouts"
Copyright (C) 2023 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

[1] "/project/6026769/R/x86_64-pc-linux-gnu-library/4.3.1"
[2] "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/r/4.3.1/lib64/R/library"
[1] '1.64.1'
> BiocManager::install("GO.db")
Bioconductor version 3.18 (BiocManager 1.30.22), R 4.3.1 (2023-06-16)
Installing package(s) 'GO.db'
trying URL 'https://bioconductor.org/packages/3.18/data/annotation/src/contrib/GO.db_3.18.0.tar.gz'
Content type 'application/x-gzip' length 29013599 bytes (27.7 MB)
==================================================
downloaded 27.7 MB

[1] "/project/6026769/R/x86_64-pc-linux-gnu-library/4.3.1"
[2] "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/r/4.3.1/lib64/R/library"
[1] '1.64.1'
* installing *source* package 'GO.db' ...
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
Error: package 'AnnotationDbi' 1.60.2 was found, but >= 1.63.2 is required by 'GO.db'
Execution halted
ERROR: lazy loading failed for package 'GO.db'
* removing '/project/6026769/R/x86_64-pc-linux-gnu-library/4.3.1/GO.db'

The downloaded source packages are in
        '/tmp/RtmpZtOpi8/downloaded_packages'
Installation paths not writeable, unable to update packages
  path: /cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/r/4.3.1/lib64/R/library
  packages:
    cluster, foreign, KernSmooth, lattice, MASS, Matrix, mgcv, nlme, rpart,
    spatial, survival
Old packages: 'ggfun', 'GOSemSim', 'shadowtext'
Update all/some/none? [a/s/n]:

Notice how:

  • the .Rprofile script is run a second time, when I issue the install command. I was not expecting that.
  • although my first-listed libPath is definitely writable, R seems to be trying to install at my second-listed libPath (which is indeed non-writable for me)

If I specify lib= in the install command, then it did not complain about the packages to update (they must be in my user library and up-to-date there) and it found other ones needing updating:

> BiocManager::install("GO.db", lib="/project/6026769/R/x86_64-pc-linux-gnu-library/4.3.1")
Bioconductor version 3.18 (BiocManager 1.30.22), R 4.3.1 (2023-06-16)
Installing package(s) 'GO.db'
trying URL 'https://bioconductor.org/packages/3.18/data/annotation/src/contrib/GO.db_3.18.0.tar.gz'
Content type 'application/x-gzip' length 29013599 bytes (27.7 MB)
==================================================
downloaded 27.7 MB

[1] "/project/6026769/R/x86_64-pc-linux-gnu-library/4.3.1"
[2] "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/r/4.3.1/lib64/R/library"
[1] '1.64.1'
* installing *source* package 'GO.db' ...
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
Error: package 'AnnotationDbi' 1.60.2 was found, but >= 1.63.2 is required by 'GO.db'
Execution halted
ERROR: lazy loading failed for package 'GO.db'
* removing '/project/6026769/R/x86_64-pc-linux-gnu-library/4.3.1/GO.db'

The downloaded source packages are in
        '/tmp/RtmpZtOpi8/downloaded_packages'
Old packages: 'ggfun', 'GOSemSim', 'shadowtext'
Update all/some/none? [a/s/n]: a
also installing the dependency 'GO.db'

GOsemSim depends on GO.db, so both fail.

ADD REPLY
0
Entering edit mode

I finally kind of solved it. I also had a .Rprofile file in my home (~) This ~/.Rprofile was establishing .libPaths to libraries for an earlier version of R (4.2.2) with of course the wrong version of AnnotationDbi

Although R was properly sourcing the .Rprofile located in my projects directory at the start of the session, it was somehow reverting to ~/.Rprofile for the installation. I was not expecting this, and frankly I don't understand.

Here is some proof:

[ablai2@cedar1 def-ablai2]$ echo 'print("Now just sourced the home Rprofile without warnings!")' >> ~/.Rprofile
[ablai2@cedar1 def-ablai2]$ echo 'print("This is the projects Rprofile talking.")' >> .Rprofile                                                   


[ablai2@cedar1 def-ablai2]$ R

R version 4.3.1 (2023-06-16) -- "Beagle Scouts"
Copyright (C) 2023 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

[1] "/project/6026769/R/x86_64-pc-linux-gnu-library/4.3.1"
[2] "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/r/4.3.1/lib64/R/library"
[1] '1.64.1'
[1] "This is the projects Rprofile talking."
> BiocManager::install("beepr", force=TRUE)
Bioconductor version 3.18 (BiocManager 1.30.22), R 4.3.1 (2023-06-16)
Installing package(s) 'beepr'
trying URL 'https://cloud.r-project.org/src/contrib/beepr_1.3.tar.gz'
Content type 'application/x-gzip' length 922244 bytes (900 KB)
==================================================
downloaded 900 KB

[1] "/project/6026769/R/x86_64-pc-linux-gnu-library/4.3.1"
[2] "/cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/r/4.3.1/lib64/R/library"
[1] '1.64.1'
[1] "This is the projects Rprofile talking."
* installing *source* package 'beepr' ...
** package 'beepr' successfully unpacked and MD5 sums checked
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
[1] "Now just sourced the home Rprofile without warnings!"
** help
*** installing help indices
** building package indices
[1] "Now just sourced the home Rprofile without warnings!"
** testing if installed package can be loaded from temporary location
[1] "Now just sourced the home Rprofile without warnings!"
** testing if installed package can be loaded from final location
[1] "Now just sourced the home Rprofile without warnings!"
** testing if installed package keeps a record of temporary installation path
* DONE (beepr)

The downloaded source packages are in
        '/tmp/RtmpQNqMU3/downloaded_packages'
Installation paths not writeable, unable to update packages
  path: /cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/r/4.3.1/lib64/R/library
  packages:
    cluster, foreign, KernSmooth, lattice, MASS, Matrix, mgcv, nlme, rpart,
    spatial, survival
Old packages: 'graphlayouts'
Update all/some/none? [a/s/n]: n

I should apologize to James and Lluis for the time wasted on this.

ADD REPLY
1
Entering edit mode

That's why I have a print/message in my .Rprofile: showing when I am running it (it helps debug this kind of problems) ;) Nice to see it solved

ADD REPLY

Login before adding your answer.

Traffic: 300 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6