About r : Error-when-running-Rcpp-in-linux-batch-mode

I am trying to run a very simple model with rstan in batch mode and have some errors. I do not have any errors when I try to run the same code not in batch mode.

The community from STAN Forums helped me to understand that the issue is that Rcpp instead of STAN is not working properly on the server of my university. We reached this understanding from a test trying to run a very simple Rcpp code in batch mode and not in batch mode. The Rcpp code is:

### Libraries
library(Rcpp)

Rcpp::sourceCpp(code='
  #include <Rcpp.h>

  // [[Rcpp::export]]
  int fibonacci(const int x) {
    if (x == 0) return(0);
    if (x == 1) return(1);
    return (fibonacci(x - 1)) + fibonacci(x - 2);
  }',
                verbose = TRUE,
                rebuild = TRUE
)

The results are below.

Not in batch mode:

Generated R functions
-------------------------------------------------------

`.sourceCpp_1_DLLInfo` <- dyn.load('/tmp/RtmpDNL6of/sourceCpp-x86_64-pc-linux-gnu-1.0.7/sourcecpp_178b21d435d91/sourceCpp_4.so')

fibonacci <- Rcpp:::sourceCppFunction(function(x) {}, FALSE, `.sourceCpp_1_DLLInfo`, 'sourceCpp_1_fibonacci')

rm(`.sourceCpp_1_DLLInfo`)

Building shared library
--------------------------------------------------------

DIR: /tmp/RtmpDNL6of/sourceCpp-x86_64-pc-linux-gnu-1.0.7/sourcecpp_178b21d435d91

/hpc/apps/R/4.1.2/lib64/R/bin/R CMD SHLIB --preclean -o 'sourceCpp_4.so' 'file178b25628f5ea.cpp'
g++ -std=gnu++14 -I"/hpc/apps/R/4.1.2/lib64/R/include" -DNDEBUG   -I"/hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include" -I"/tmp/RtmpDNL6of/sourceCpp-x86_64-pc-linux-gnu-1.0.7" -I/hpc/apps/zlib/1.2.8/include -I/hpc/apps/pcre/8.38/include -I/hpc/apps/bzip2/1.0.6/include -I/hpc/apps/curl/7.54.1/include -I/hpc/apps/cairo/1.14.12/include   -fpic  -g -O2  -c file178b25628f5ea.cpp -o file178b25628f5ea.o
g++ -std=gnu++14 -shared -L/hpc/apps/R/4.1.2/lib64/R/lib -L/hpc/apps/zlib/1.2.8/lib -L/hpc/apps/pcre/8.38/lib -L/hpc/apps/bzip2/1.0.6/lib -L/hpc/apps/curl/7.54.1/lib -L/hpc/apps/cairo/1.14.12/lib -o sourceCpp_4.so file178b25628f5ea.o -L/hpc/apps/R/4.1.2/lib64/R/lib -lR

In batch mode:

Generated R functions 
-------------------------------------------------------

`.sourceCpp_1_DLLInfo` <- dyn.load('/localscratch/250948.1.ragatkolab.q/RtmpDlK5Sy/sourceCpp-x86_64-pc-linux-gnu-1.0.7/sourcecpp_142bd68d660f1/sourceCpp_2.so')

fibonacci <- Rcpp:::sourceCppFunction(function(x) {}, FALSE, `.sourceCpp_1_DLLInfo`, 'sourceCpp_1_fibonacci')

rm(`.sourceCpp_1_DLLInfo`)

Building shared library
--------------------------------------------------------

DIR: /localscratch/250948.1.mylab.q/RtmpDlK5Sy/sourceCpp-x86_64-pc-linux-gnu-1.0.7/sourcecpp_142bd68d660f1

/hpc/apps/R/4.1.2/lib64/R/bin/R CMD SHLIB -o 'sourceCpp_2.so' 'file142bd64ebe902.cpp' 
g++ -std=gnu++14 -I"/hpc/apps/R/4.1.2/lib64/R/include" -DNDEBUG   -I"/hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include" -I"/localscratch/250948.1.ragatkolab.q/RtmpDlK5Sy/sourceCpp-x86_64-pc-linux-gnu-1.0.7" -I/hpc/apps/zlib/1.2.8/include -I/hpc/apps/pcre/8.38/include -I/hpc/apps/bzip2/1.0.6/include -I/hpc/apps/curl/7.54.1/include -I/hpc/apps/cairo/1.14.12/include   -fpic  -g -O2  -c file142bd64ebe902.cpp -o file142bd64ebe902.o
In file included from /hpc/apps/gcc/9.1.0/include/c++/9.1.0/x86_64-pc-linux-gnu/bits/c++config.h:524,
                 from /hpc/apps/gcc/9.1.0/include/c++/9.1.0/cmath:41,
                 from /hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include/Rcpp/platform/compiler.h:100,
                 from /hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include/Rcpp/r/headers.h:66,
                 from /hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include/RcppCommon.h:30,
                 from /hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include/Rcpp.h:27,
                 from file142bd64ebe902.cpp:2:
/hpc/apps/gcc/9.1.0/include/c++/9.1.0/x86_64-pc-linux-gnu/bits/os_defines.h:39:10: fatal error: features.h: No such file or directory
   39 | #include <features.h>
      |          ^~~~~~~~~~~~
compilation terminated.
make: *** [file142bd64ebe902.o] Error 1
Error in Rcpp::sourceCpp(code = "\n  #include <Rcpp.h>\n\n  // [[Rcpp::export]]\n  int fibonacci(const int x) {\n    if (x == 0) return(0);\n    if (x == 1) return(1);\n    return (fibonacci(x - 1)) + fibonacci(x - 2);\n  }",  : 
  Error 1 occurred building shared library.

WARNING: The tools required to build C++ code for R were not found.

Please install GNU development tools including a C++ compiler.

Execution halted

Then, they asked me to modify my makevars file – which I did using the R-package usethis::edit_r_makevars() – adding the lines:

CXXFLAGS += -I"/hpc/apps/gcc/9.1.0/include/c++/9.1.0/parallel/"
CXX14FLAGS += -I"/hpc/apps/gcc/9.1.0/include/c++/9.1.0/parallel/"

Once I did this modification (I hope I did it correctly), my error message change to:

Generated extern "C" functions 
--------------------------------------------------------


#include <Rcpp.h>
#ifdef RCPP_USE_GLOBAL_ROSTREAM
Rcpp::Rostream<true>&  Rcpp::Rcout = Rcpp::Rcpp_cout_get();
Rcpp::Rostream<false>& Rcpp::Rcerr = Rcpp::Rcpp_cerr_get();
#endif

// fibonacci
int fibonacci(const int x);
RcppExport SEXP sourceCpp_1_fibonacci(SEXP xSEXP) {
BEGIN_RCPP
    Rcpp::RObject rcpp_result_gen;
    Rcpp::RNGScope rcpp_rngScope_gen;
    Rcpp::traits::input_parameter< const int >::type x(xSEXP);
    rcpp_result_gen = Rcpp::wrap(fibonacci(x));
    return rcpp_result_gen;
END_RCPP
}

Generated R functions 
-------------------------------------------------------

`.sourceCpp_1_DLLInfo` <- dyn.load('/localscratch/251578.1.mylablab.q/RtmpCHyEAd/sourceCpp-x86_64-pc-linux-gnu-1.0.7/sourcecpp_1e50364a03315/sourceCpp_2.so')

fibonacci <- Rcpp:::sourceCppFunction(function(x) {}, FALSE, `.sourceCpp_1_DLLInfo`, 'sourceCpp_1_fibonacci')

rm(`.sourceCpp_1_DLLInfo`)

Building shared library
--------------------------------------------------------

DIR: /localscratch/251578.1.mylablab.q/RtmpCHyEAd/sourceCpp-x86_64-pc-linux-gnu-1.0.7/sourcecpp_1e50364a03315

/hpc/apps/R/4.1.2/lib64/R/bin/R CMD SHLIB -o 'sourceCpp_2.so' 'file1e50325e7686.cpp' 
g++ -std=gnu++14 -I"/hpc/apps/R/4.1.2/lib64/R/include" -DNDEBUG   -I"/hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include" -I"/localscratch/251578.1.mylablab.q/RtmpCHyEAd/sourceCpp-x86_64-pc-linux-gnu-1.0.7" -I/hpc/apps/zlib/1.2.8/include -I/hpc/apps/pcre/8.38/include -I/hpc/apps/bzip2/1.0.6/include -I/hpc/apps/curl/7.54.1/include -I/hpc/apps/cairo/1.14.12/include   -fpic  -g -O2  -I"/hpc/apps/gcc/9.1.0/include/c++/9.1.0/parallel/" -c file1e50325e7686.cpp -o file1e50325e7686.o
In file included from /hpc/apps/gcc/11.1.0/include/c++/11.1.0/x86_64-pc-linux-gnu/bits/c++config.h:571,
                 from /hpc/apps/gcc/11.1.0/include/c++/11.1.0/cmath:41,
                 from /hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include/Rcpp/platform/compiler.h:100,
                 from /hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include/Rcpp/r/headers.h:66,
                 from /hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include/RcppCommon.h:30,
                 from /hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include/Rcpp.h:27,
                 from file1e50325e7686.cpp:2:
/hpc/apps/gcc/11.1.0/include/c++/11.1.0/x86_64-pc-linux-gnu/bits/os_defines.h:44:19: error: missing binary operator before token "("
   44 | #if __GLIBC_PREREQ(2,15) && defined(_GNU_SOURCE)
      |                   ^
/hpc/apps/gcc/11.1.0/include/c++/11.1.0/x86_64-pc-linux-gnu/bits/os_defines.h:52:19: error: missing binary operator before token "("
   52 | #if __GLIBC_PREREQ(2, 27)
      |                   ^
In file included from /hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include/Rcpp/platform/compiler.h:100,
                 from /hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include/Rcpp/r/headers.h:66,
                 from /hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include/RcppCommon.h:30,
                 from /hpc/home/user/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include/Rcpp.h:27,
                 from file1e50325e7686.cpp:2:
/hpc/apps/gcc/11.1.0/include/c++/11.1.0/cmath:45:15: fatal error: math.h: No such file or directory
   45 | #include_next <math.h>
      |               ^~~~~~~~
compilation terminated.
make: *** [file1e50325e7686.o] Error 1
Error in Rcpp::sourceCpp(code = "\n  #include <Rcpp.h>\n\n  // [[Rcpp::export]]\n  int fibonacci(const int x) {\n    if (x == 0) return(0);\n    if (x == 1) return(1);\n    return (fibonacci(x - 1)) + fibonacci(x - 2);\n  }",  : 
  Error 1 occurred building shared library.

WARNING: The tools required to build C++ code for R were not found.

Please install GNU development tools including a C++ compiler.

Execution halted

My knowledge in Linux is limited and I am afraid that my question is out of scope for STAN Forums. Any thoughts how to proceed?

I fear your problem is still local. On the computing you use, someone may have set up R in way that makes it differ between interactive R use, and what you call batch use.

But that is non-standard, and at your end. In general, it just works. Using a minimally modified version of your program (which I still find confusing: it loads doParallel, registers cores but … triggers to parallel code — anyway) it works just fine:

Using Rscript

[email protected]:~/git/stackoverflow/70470842(master)$ Rscript answer.R  
Loading required package: foreach        
Loading required package: iterators      
Loading required package: parallel       
Fib(10) is  55                           
[email protected]:~/git/stackoverflow/70470842(master)$ 

Using R in batch mode

[email protected]:~/git/stackoverflow/70470842(master)$ R -s -f answer.R 
Loading required package: foreach     
Loading required package: iterators   
Loading required package: parallel    
Fib(10) is  55                        
[email protected]:~/git/stackoverflow/70470842(master)$  

Using our littler frontend

[email protected]:~/git/stackoverflow/70470842(master)$ r answer.R     
Loading required package: foreach               
Loading required package: iterators             
Loading required package: utils                 
Loading required package: parallel              
Fib(10) is  55                                  
[email protected]:~/git/stackoverflow/70470842(master)$   

Modified source code

library(doParallel)             # Library        

ncores <- 5                     # Simulation parameters
n_sim <- 5                  
registerDoParallel(ncores)  

Rcpp::sourceCpp(code='  
  #include <Rcpp.h>     
  // [[Rcpp::export]]            
  int fibonacci(const int x) {   
    if (x == 0) return(0);       
    if (x == 1) return(1);       
    return (fibonacci(x - 1)) + fibonacci(x - 2); 
  }'          
  )           
                                                                                                                                                                                                                   
cat("Fib(10) is ", fibonacci(10), "\n")  

For future reference, I interacted with IT team from my institution after feedback received here and STAN Forums, and they finally found a solution. In their words:

“Compute nodes aren’t normally used for development. So certain development packages are not installed by default. We installed “glibc-devel” and “glibc-headers” on the compute nodes.

So, it seems that when run interactively the job was running on the submit node (which had these libraries installed) and when run from the scheduler it was run on the compute nodes (which did not have the library installed). The reason for the discrepancy being that the libraries in question have traditionally been only used during development and compilation. This use posed a unique use case.”