I am deploying multiple R versions on multiple virtual desktops. I've built 3.6.3 and 4.1.2 R from source on Ubuntu 18.04.3 LTS. None of them finds the system-wide Rprofile.site file in /etc/R or the system certificates in /usr/share/ca-certificates. However R (3.4.4) installed with APT has no such problems. I used Ansible, but for the sake of this question I reproduced the deployment for one host with a shell script.
#!/bin/bash
set -euo pipefail
# install build dependecies
(command -v apt && apt-get build-dep r-base) || (command -v dnf && dnf builddep R)
version='4.1.2'
major_version=$(echo "$version" | cut -c 1)
wget "https://cran.rstudio.com/src/base/R-$major_version/R-$version.tar.gz"
tar -xzf R-$version.tar.gz
cd R-$version
./configure \
--prefix=/opt/R/$version \
--sysconfdir=/etc/R \
--enable-R-shlib \
--with-pcre1 \
--with-blas \
--with-lapack
make -j 8
make install
Note: It should run on most Linux distros with APT or RPM package managers. Increase the -j argument of make, if you have the enough cores, but no time.
So I defined the installation prefix as /opt/R/$version , but I want it read config files from /etc/R (defined --sysconfdir=/etc/R). However when I open the R interactive shell (/opt/R/4.1.2/bin/R) to try install a package:
install.packages("remotes")
then I will be prompted to choose a R package mirror, but one already defined in /etc/R/Rprofile.site:
local({
r <- getOption("repos")
r["CRAN"] <- "https://cloud.r-project.org"
options(repos = r)
})
I can force the R shell to find the Rprofile.site file by defining it with the R_PROFILE environment variable.
export R_PROFILE=/etc/R/Rprofile.site
/opt/R/4.1.2/bin/R
then call install.packages("remotes") again in the R shell. Now no mirror selection prompt will be shown, but the following error:
Warning: unable to access index for repository https://cloud.r-project.org/src/contrib:
cannot open URL 'https://cloud.r-project.org/src/contrib/PACKAGES'
Warning message:
package ‘remotes’ is not available for this version of R
A version of this package for your version of R might be available elsewhere,
see the ideas at
https://cran.r-project.org/doc/manuals/r-patched/R-admin.html#Installing-packages
So it cannot access the repository index (real problem), then concludes that the package ‘remotes’ package is not available for my R version. Which is BS, since it was not able to read the index at the first place. So I tried a simple HTTP call in the same R shell.
curlGetHeaders("https://example.com")
and got this error:
Error in curlGetHeaders("https://example.com") : libcurl error code 77:
unable to access SSL/TLS CA certificates
So it cannot find the CA certificates in /usr/share/ca-certificates.
Since the R installed by APT has none of these problems. The compiled R does not search the right places. Even if I omit the --sysconfdir=/etc/R build option and copy or symlink the /etc/R directory under the prefix, so it will be at /opt/R/4.1.2/etc. It will still not find its config files.
The greater problem that I do not even no know how to specify the /usr/share so it may find the certificates. The rsharedir build options (the -- also missing in the makefile) will not do, because it should point to /usr/share/R/ not /usr/share, which would be a bad practice anyway.
I also tried all of this it with the 3.6.3 R version and got the same results.
Questions: How can I make the compiled R installations to find the system-wide or any config files and the certificates.
Update 1
I ran the build script on a Ubuntu server which I do not manage with the same Ansible code. On both of them R successfully finds the certificates. So the problem is not with the build script but the system state.
Update 2
I created a simple R script (install-r-package.R) which install a package:
install.packages("renv", repos="https://cran.wu.ac.at/")
then I executed it with Rscript and traced which file do they open on both the correct and erroneous hosts:
strace -o strace.log -e trace=open,openat,close,read,write,connect,accept ./Rscript install-r-package.R
It turned out that on the problematic system R does not even try to open the certificate files.
The relevant trace snippet on the correct system:
connect(5, {sa_family=AF_INET, sin_port=htons(443), sin_addr=inet_addr("137.208.57.37")}, 16) = -1 EINPROGRESS (Operation now in progress)
openat(AT_FDCWD, "/etc/ssl/certs/ca-certificates.crt", O_RDONLY) = 6
read(6, "-----BEGIN CERTIFICATE-----\nMIIH"..., 200704) = 200704
read(6, "--\n", 4096) = 3
read(6, "", 4096) = 0
close(6) = 0
on the problematic system:
connect(5, {sa_family=AF_INET, sin_port=htons(443), sin_addr=inet_addr("137.208.57.37")}, 16) = -1 EINPROGRESS (Operation now in progress)
openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/nss/libsoftokn3.so", O_RDONLY|O_CLOEXEC) = 6
read(6, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220A\0\0\0\0\0\0"..., 832) = 832
close(6) = 0
In both cases R connects to the mirror (137.208.57.37) after that on the correct system it reads the ca-certificates.crt certificate file and many other .crt files after that. However the erroneous system jump this step altogether.
CodePudding user response：
Finally I found the solution:
Since both system has the arch and OS. I cross copied the R compiled installations between them. The R which was compiled on the problematic system, but was run on the correct one gave the warnings below after the calling of the install.packages("renv", repos="https://cran.wu.ac.at/")
Warning: unable to access index for repository https://cran.wu.ac.at/src/contrib:
internet routines cannot be loaded
Warning messages:
1: In download.file(url, destfile = f, quiet = TRUE) :
unable to load shared object '/opt/R/4.1.2/lib/R/modules//internet.so':
libcurl-nss.so.4: cannot open shared object file: No such file or directory
2: package ‘remotes’ is not available for this version of R
A version of this package for your version of R might be available elsewhere,
see the ideas at
https://cran.r-project.org/doc/manuals/r-patched/R-admin.html#Installing-packages
If I do the reverse then the installation works.
The libcurl-nss.so.4: cannot open shared object file: No such file or directory line gave me the clue that different libcurl4 flavors was used as build dependecies. I checked which dev dependecies were installed on the systems and libcurl4-nss-dev 7.58.0-2ubuntu3 were installed on the problematic system and libcurl4-gnutls-dev 7.58.0-2ubuntu3.16 on the correct system.
So I purged libcurl4-gnutls-dev from the problematic system:
apt purge libcurl4-nss-dev -y
and installed libcurl4-gnutls-dev:
aptitude install libcurl4-gnutls-dev
I used aptitude, because I had to downgrade libcurl3-gnutls 7.58.0-2ubuntu3.16 (now) -> 7.58.0-2ubuntu3 (bionic) which is a dependency of libcurl4-gnutls-dev, then I run a make clean in the R-4.1.2 source directory. Finally I re-run the build script from the question, and got a well working R, which can read the certificates, hence can reach the HTTPS using package mirrors.