I am not a Linux administrator, and I don't think the Linux administrators that we have understand our needs with regard to installing and keeping up to date programs that regularly update to newer versions. We have a few versions of R installed in various places on our server, as well a few versions of python installed in various places and despite this I can not with my sudo priviledges get pip installed and working and I am having problems with some Bioconductor packages. Asking the admin's to help means waiting a few weeks.

So, I have two questions.

First, is there any thing published such as a 'best practices' for installing R and Bioconductor on Linux for multiple users and then when a new version or R and/or Bioconductor appears how to install the new version so that there is no confusion between the versions ?

Second, is there a particular Linux distro that anyone would recommend as being better suited for using R, Bioconductor, python and Bioinformatics in general ? We have 8 cores and 16 GB of memory.
 

I wouldn't want to get involved in a 'best practices' flame war, so I'll just say what we do at my core, which works pretty well for us. I should note that our sysadmins are very responsive, and a request to get something installed usually takes less than an hour to be fulfilled, so I don't share your problems. I do however prefer to maintain my own R installs, and like to have many different versions laying around, so we don't do the usual install in the system directories.

Anyway, we have a directory set up to have read/write/execute permissions for the group of people who are using R, into which I install various versions of R. Note that this isn't in /usr or /opt or wherever; it's in a non-system location. Because everybody has permissions, anybody who wants to install a package can do so, without having to install in a personal directory, and the packages are then available for everybody to use. The upside being that you don't have all these personal library directories floating around, and the obvious downside being that somebody could bork the installation by using e.g., install.packages instead of biocLite. How well or poorly this works is directly dependent on how trustworthy your users are, so YMMV.

An alternative is to give read/write/execute to just one person, and read/execute to all others. The downside to that is people will either end up with personal library dirs, or they will continually bother the 'installer' to add packages.

I use Emacs/ESS, because I am old school, and I have this in my .emacs file:

(defun R-2.15.1 (&optional start-args)
  (interactive "P")(let ((inferior-R-program-name
                          "/data/oldR/R-2.15.1/bin/R"))(R start-args)))
(defun R-3.0.0 (&optional start-args)
  (interactive "P")(let ((inferior-R-program-name
                          "/data/oldR/R-3.0.0/bin/R"))(R start-args)))
(defun R-3.0.1 (&optional start-args)
  (interactive "P")(let ((inferior-R-program-name
                          "/data/oldR/R-3.0.1/bin/R"))(R start-args)))
(defun R-3.0.2 (&optional start-args)
  (interactive "P")(let ((inferior-R-program-name
                          "/data/oldR/R-3.0.2/bin/R"))(R start-args)))
(defun R-devel (&optional start-args)
  (interactive "P")(let ((inferior-R-program-name
                          "/data/oldR/R-devel/bin/R"))(R start-args)))

And so when I want to start R, I can just do M-x R to get the current version, or M-x R-3.0.2 or whatever to get some old version if I have to recapitulate an analysis I did with that version.

I'm sure the cool kids who use RStudio or whatnot will be able to chime in and say how the less Paleolithic among us do things like that.

I also happen to have a group-wide install dir that I use to install things that are fast moving, or that require more recent versions than are available from the distro repositories. Things like bowtie or subread, which may get bugfixes that I want to install quickly without having to bother the sysadmins.

So that is an example of what you may be able to do, given sysadmins who are cool with you installing things yourself.

As for the Linux distro, I am not sure it matters. For a single server probably any of the Debian variants will work just fine (we use Ubuntu, IIRC).