Installing compiler on Mac M1 Monterey for Rcpp and other tools-CodePudding

I'm trying to use packages that require Rcpp in R on my M1 Mac, which I was never able to get up and running after purchasing this computer. I updated it to Monterey in the hope that this would fix some installation issues but it hasn't. I tried running the Rcpp check from this page but I get the following error:

> Rcpp::sourceCpp("~/github/helloworld.cpp")

ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0'
ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib'
ld: library not found for -lgfortran
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [sourceCpp_4.so] Error 1
clang   -arch arm64 -std=gnu  14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include   -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/RcppArmadillo/include" -I"/Users/afredston/github" -I/opt/R/arm64/include   -fPIC  -falign-functions=64 -Wall -g -O2  -c helloworld.cpp -o helloworld.o
clang   -arch arm64 -std=gnu  14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o sourceCpp_4.so helloworld.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
Error in Rcpp::sourceCpp("~/github/helloworld.cpp") : 
  Error 1 occurred building shared library.

I get that it can't "find" gfortran. I installed this release of gfortran for Monterey. When I type which gfortran into Terminal, it returns /opt/homebrew/bin/gfortran. (Maybe this version of gfortran requires Xcode tools that are too new—it says something about 13.2 and when I run clang --version it says 13.0—but I don't see another release of gfortran for Monterey?)

I also appended /opt/homebrew/bin: to PATH in R so it looks like this now:

> Sys.getenv("PATH")

[1] "/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Library/TeX/texbin:/Applications/RStudio.app/Contents/MacOS/postback"

Other things I checked:

Xcode command line tools is installed (which clang returns /usr/bin/clang).
Files ~/.R/Makevars and ~/.Renviron don't exist.

Here's my session info:

R version 4.1.1 (2021-08-10)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS Monterey 12.1

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_4.1.1           tools_4.1.1              RcppArmadillo_0.10.7.5.0
[4] Rcpp_1.0.7

UPDATE: After trying fixes from @Mikael, it still fails, but on a slightly different error:


> Rcpp::sourceCpp("~/github/helloworld.cpp")
ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0'
ld: library not found for -lemutls_w
clang-13: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [sourceCpp_2.so] Error 1
/opt/homebrew/opt/llvm/bin/clang   -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -target arm64-apple-macos12 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include   -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/RcppArmadillo/include" -I"/Users/afredston/github" -I/opt/homebrew/opt/llvm/include -I/opt/R/arm64/include   -fPIC  -g -O2 -Wall -pedantic -c helloworld.cpp -o helloworld.o
/opt/homebrew/opt/llvm/bin/clang   -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -target arm64-apple-macos12 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/homebrew/opt/llvm/lib -L/opt/R/arm64/lib -o sourceCpp_2.so helloworld.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
Error in Rcpp::sourceCpp("~/github/helloworld.cpp") : 
  Error 1 occurred building shared library.

CodePudding user response：

There are some obstacles on ARM-based Macs (and macOS in general) to compiling R packages containing C/C code from their sources, and likewise to using Rcpp. The main issues and their fixes are documented in the R-admin manual, but they are a bit scattered.

Your proximal problem is perhaps easier to solve than the problem you haven't mentioned, which is that the clang toolchain provided with Apple's Command Line Tools doesn't support OpenMP. Not having OpenMP support is an issue if you want to compile a C/C program that makes use of multithreading. I'll discuss your proximal problem first, then provide a complete set of instructions that should address everything at once, so that you can obtain a fully enabled toolchain.

Look at `make` variables, not `PATH`

When R (and Rcpp, etc.) compiles C/C /Fortran programs, it determines what compilers, compiler flags, preprocessor flags, etc. to use from make variables set in Makefiles. The correct way to specify libraries needed to link Fortran code is to set the make variable FLIBS in a Makefile that R knows about. System-level default values of all make variables used by R are stored in $(R_HOME)/etc/Makeconf. You can query them with R CMD config, like so:

$ R CMD config FLIBS --no-user-files

A user can override any Makeconf setting by creating and modifying $(HOME)/.R/Makevars. Since you do not have a Makevars, R is finding something like

FLIBS=-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm

in Makeconf, because that is how CRAN configured the build of R that you have installed. This configuration is intended for Big Sur, not Monterey. Your compilation error should go away if you set

FLIBS=-L$(F_DIR)/lib/gcc/aarch64-apple-darwin21/12.0.0 -L$(F_DIR)/lib -lgfortran -lemutls_w -lm

in Makevars, replacing $(F_DIR) with the path to your gfortran installation. (I am assuming that you did, in fact, install gfortran from here.)

I should emphasize here that the environment variable PATH doesn't really affect how R compiles C/C /Fortran programs apart from telling R where to find make. That is, the paths printed by

$ which clang clang   gfortran

aren't necessarily the paths to the C/C /Fortran compilers used by R.

Instructions for obtaining a working toolchain on ARM-based Macs

For the benefit of anyone else reading this, I'm going to assume that you are starting from nothing. Feel free to skip steps you've already taken, though you might find a fresh start helpful.

Download an R binary from CRAN here and install. Be sure to select the binary built for Apple silicon.
Run
```
$ xcode-select --install
```
in Terminal to download and install the latest version of Xcode, which includes Apple's Command Line Tools. You can also download Xcode from your browser here. Earlier versions of Xcode are available here, but I would start with the latest. (There might be a good reason to install the exact version of Xcode than CRAN used to build your R binary, and that version might not be the newest. Not sure.)
Install the LLVM clang toolchain with Homebrew. Unlike Apple's clang, it supports OpenMP.
```
$ brew update
$ brew install llvm 
```
It should unpack into /opt/homebrew/opt.
Download a gfortran binary built for your macOS version and architecture, then install. This step differs by macOS version:

Monterey: Download the disk image file hosted here and open to install. The installer will unpack the binary into /usr/local; you should move the installation into /opt/R/arm64.
```
$ sudo mkdir -p /opt/R/arm64/bin
$ sudo mv /usr/local/gfortran /opt/R/arm64
$ sudo mv /usr/local/bin/gfortran /opt/R/arm64/bin
$ sudo ln -sfn /opt/R/arm64/gfortran/bin/gfortran /opt/R/arm64/bin/gfortran
```
Big Sur: Download the tarball hosted here and unpack directly into /opt/R/arm64.
```
$ sudo mkdir -p /opt/R/arm64
$ sudo tar xvf path/to/tarball -C /opt/R/arm64
```

Add the following lines to $(HOME)/.R/Makevars, creating the file if necessary.

LLVM_DIR=/opt/homebrew/opt/llvm
LIBS_DIR=/opt/R/arm64
F_DIR=$(LIBS_DIR)/gfortran
SDK_DIR=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk

CC=$(LLVM_DIR)/bin/clang -isysroot $(SDK_DIR) -target arm64-apple-macos12
CXX=$(LLVM_DIR)/bin/clang   -isysroot $(SDK_DIR) -target arm64-apple-macos12
FC=$(F_DIR)/bin/gfortran -mtune=native

CFLAGS=-falign-functions=8 -g -O2 -Wall -pedantic -Wno-implicit-function-declaration
CXXFLAGS=-g -O2 -Wall -pedantic
FFLAGS=-g -O2 -Wall -pedantic

SHLIB_OPENMP_CFLAGS=-fopenmp
SHLIB_OPENMP_CXXFLAGS=-fopenmp
SHLIB_OPENMP_FFLAGS=-fopenmp

CPPFLAGS=-I$(LLVM_DIR)/include -I$(LIBS_DIR)/include
LDFLAGS=-L$(LLVM_DIR)/lib -L$(LIBS_DIR)/lib
FLIBS=-L$(F_DIR)/lib/gcc/aarch64-apple-darwin21/12.0.0 -L$(F_DIR)/lib -lgfortran -lemutls_w -lm

CC, CXX, and FLIBS need to be tweaked on Big Sur:

CC=$(LLVM_DIR)/bin/clang -isysroot $(SDK_DIR) -target arm64-apple-macos11
CXX=$(LLVM_DIR)/bin/clang   -isysroot $(SDK_DIR) -target arm64-apple-macos11
FLIBS=-L$(F_DIR)/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L$(F_DIR)/lib -lgfortran -lemutls_w -lm

Run R and test that you can compile a program with OpenMP support. For example:

if (!requireNamespace("Rcpp", quietly = TRUE)) {
  install.packages("Rcpp")
}
if (!requireNamespace("RcppArmadillo", quietly = TRUE)) {
  install.packages("RcppArmadillo")
}
Rcpp::sourceCpp(code = '
#include <RcppArmadillo.h>
#ifdef _OPENMP
# include <omp.h>
#endif

// [[Rcpp::depends(RcppArmadillo)]]
// [[Rcpp::plugins(openmp)]]
// [[Rcpp::export]]
void omp_test()
{
#ifdef _OPENMP
    Rprintf("OpenMP threads available: %d\\n", omp_get_max_threads());
#else
    Rprintf("OpenMP not supported\\n");
#endif
}
')
omp_test()

OpenMP threads available: 8

If this fails to compile, or if it compiles without error but you get the message saying that OpenMP is not supported, then something is wrong. Let me know if either happens.

CodePudding user response：

I resolved this issue by adding a path to the homebrew installation of gfortran to my ~/.R/Makevars following these instructions: https://pat-s.me/transitioning-from-x86-to-arm64-on-macos-experiences-of-an-r-user/#gfortran

Look at make variables, not PATH

Instructions for obtaining a working toolchain on ARM-based Macs

Look at `make` variables, not `PATH`