The basic issue is that R has an annual release cycle, whereas Bioconductor has a twice-yearly release cycle. Also, Bioconductor has a 'devel' branch where new packages and features are introduced, and a 'release' branch where bug fixes and relative stability are important.

Previously, one would install a Bioconductor package by running source(".../biocLite.R") to read a file from the web. The file contained an installation script that was smart enough to figure out what version of R and Bioconductor were in use / appropriate for the person invoking the script. Sourcing an executable script from the web is an obvious security problem.

Our solution was to use a CRAN package BiocManager, so that users would start the ball rolling with an installation from CRAN, rather than sourcing from the web.

But how does a CRAN package know what version of Bioconductor is in use? Can we use BiocManager? No, because we don't have enough control of the version of BiocManager available on CRAN, e.g., everyone using the same version of R would get the same version of BiocManager and hence of Bioconductor. But there are two Bioconductor versions per R version, so that's not going to work!

BiocManager could write information to a cache on the user disk, but this is not a robust solution for a number of reasons. Is there any other way that R could keep track of version information? Yes, by installing a Bioconductor package (BiocVersion) whose sole purpose is to indicate the version of Bioconductor in use. You see this when you run BiocManager for the first time

/tmp$ mkdir eglib
/tmp$ R_LIBS_USER=eglib ~/bin/R-3-6-branch/bin/R --vanilla
> library("BiocManager")
Error in library("BiocManager") :
  there is no package called 'BiocManager'
> install.packages("BiocManager")
Installing package into '/private/tmp/eglib'
(as 'lib' is unspecified)
...
> BiocManager::install()
Bioconductor version 3.9 (BiocManager 1.30.4), R 3.6.0 Patched (2019-04-26
  r76431)
Installing package(s) 'BiocVersion'
trying URL 'https://bioconductor.org/packages/3.9/bioc/src/contrib/BiocVersion_3.9.0.tar.gz'
Content type 'application/x-gzip' length 984 bytes
==================================================
downloaded 984 bytes

* installing *source* package 'BiocVersion' ...
** using staged installation
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (BiocVersion)

The downloaded source packages are in
    '/private/var/folders/yn/gmsh_22s2c55v816r6d51fx1tnyl61/T/RtmpxaCwDL/downloaded_packages'

You were re-using your BiocVersion from a previous installation, and BiocManager was responding to the request to install a version of Bioconductor that required a different version of R.

One weakness in the current approach is that the BiocManager does not actually load BiocVersion, it just determines the package version (the scenario below continues from above, but is not quite the same as your scenario)

/tmp$ R_LIBS_USER=eglib ~/bin/R-3-5-branch/bin/R --quiet  # new library, old R
> packageVersion("BiocVersion")
[1] '3.9.0'

If BiocManager actually tried to load the package, R would have objected

> library("BiocVersion")
Error: This is R 3.5.3, package 'BiocVersion' needs >= 3.6.0

We will certainly make this process more robust, with a clearer indication to the user of what the underlying problem is.