Rbowtie fails to compile using spack with "error: reference to Byte is ambiguous"
1
0
Entering edit mode
@pariksheet-nanda-10892
Last seen 11 days ago
University of Connecticut

Edit: The fault happens with gcc 7.4.0 that the bioconductor build server uses, in addition to gcc 9.1.0

There appears to be a C++ namespace collision between bowtie's bundled dependency seqan::Byte against zlib's Byte:

> BiocManager::install("Rbowtie")                                                                                                                                            
Bioconductor version 3.9 (BiocManager 1.30.4), R 3.6.0 (2019-04-26)                                                                                                                            
Installing package(s) 'Rbowtie'
---snip--
* installing *source* package ‘Rbowtie’ ...
** using staged installation
** libs
** arch -
g++ -O3 -m64 -DCOMPILER_OPTIONS="\"-O3 -m64  -Wl,--hash-style=both -DPOPCNT_CAPABILITY -g -O2   -g -O2  \""  -Wl,--hash-style=both -DPOPCNT_CAPABILITY -g -O2   -g -O2    \
                -fno-strict-aliasing -DBOWTIE_VERSION="\"`cat VERSION`\"" -DBUILD_HOST="\"`hostname`\"" -DBUILD_TIME="\"`date`\"" -DCOMPILER_VERSION="\"`g++ -v 2>&1 | tail -1`\"" -D_LARGEFIL\
E_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE  -DPREFETCH_LOCALITY=2 -DBOWTIE_MM -DBOWTIE_SHARED_MEM -DNDEBUG -Wall -Wno-unused-parameter -Wno-reorder -Wno-unused-local-typedefs \
                 -isystem ./SeqAn-1.1 -I third_party -I third_party \
                -o bowtie-build-s ebwt_build.cpp \
                ccnt_lut.cpp ref_read.cpp alphabet.cpp shmem.cpp edit.cpp ebwt.cpp tinythread.cpp  bowtie_build_main.cpp \
                -L/usr/local/lib64  -lz -lpthread
In file included from /home/pan14001/spack/opt/spack/linux-rhel6-x86_64/gcc-9.1.0/zlib-1.2.11-dxygq2yfaklrcc7jc6l4rcdhtovxam4h/include/zlib.h:34,
                 from pat.h:6,
                 from sequence_io.h:12,
                 from multikey_qsort.h:8,
                 from diff_sample.h:13,
                 from blockwise_sa.h:19,
                 from ebwt.h:27,
                 from ebwt_build.cpp:11:
/home/pan14001/spack/opt/spack/linux-rhel6-x86_64/gcc-9.1.0/zlib-1.2.11-dxygq2yfaklrcc7jc6l4rcdhtovxam4h/include/zconf.h:400: error: reference to ‘Byte’ is ambiguous
/home/pan14001/spack/opt/spack/linux-rhel6-x86_64/gcc-9.1.0/zlib-1.2.11-dxygq2yfaklrcc7jc6l4rcdhtovxam4h/include/zconf.h:391: error: candidates are: typedef unsigned char Byte
./SeqAn-1.1/seqan/basic/basic_alphabet_interface.h:62: error:                 typedef unsigned char seqan::Byte
---snip---

Thosee Byte typedefs are defined in bowtie's SeqAn-1.1/seqan/basic/basic_generated_forwards.h and zlib's include/zconf.h respectively.

I'm guessing this is related to bug# 76: https://github.com/BenLangmead/bowtie/issues/76

That bug was reported in 2017. Given that upstream has ignored the issue for 2 years, maybe Rbowtie should add a flag to use the older C++ standard per the workaround described in that bug report?

Steps to reproduce

I compiled gcc 9.1.0 and R 3.6.0 on my university cluster using spack.

# Clone spack to your home directory or wherever you prefer.
cd
git clone https://github.com/spack/spack
source ~/spack/share/spack/setup-env.sh
# RHEL6 provides an ancient gcc version 4.4.7, so bootstrap a more recent compiler.
spack compiler find
spack install gcc@9.1.0
# The above install command can randomly fail; lately, package downloads can fail with:
#     Error: FetchError: All fetchers failed for ...
# If you run into download failures, you can stupidly workaround it with a loop like:
#     while [[ $? -ne 0 ]]; do spack install gcc@9.1.0; done
# That saves time because there are a fair number of packages to install per `spack spec gcc@9.1.0`
spack find -p gcc@9.1.0 | tail -1 | awk '{print $2}' | xargs spack compiler find
==> Added 1 new compiler to /home/pan14001/.spack/linux/compilers.yaml
    gcc@9.1.0
==> Compilers are defined in the following files: 
    /home/pan14001/.spack/linux/compilers.yaml
spack install r@3.6.0
# Clear other environmental modules from your PATH, etc
module purge
spack load -r r@3.6.0
R -q --no-save <<EOF
install.packages("BiocManager")
BiocManager::install("Rbowtie")
EOF 

sessionInfo

> sessionInfo()
R version 3.6.0 (2019-04-26)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Red Hat Enterprise Linux Server release 6.7 (Santiago)

Matrix products: default
BLAS:   /home/pan14001/spack/opt/spack/linux-rhel6-x86_64/gcc-9.1.0/r-3.6.0-ofbi5xuqekgairbpsjkaui4jh6kfdzbp/rlib/R/lib/libRblas.so
LAPACK: /home/pan14001/spack/opt/spack/linux-rhel6-x86_64/gcc-9.1.0/r-3.6.0-ofbi5xuqekgairbpsjkaui4jh6kfdzbp/rlib/R/lib/libRlapack.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] BiocManager_1.30.4 compiler_3.6.0     tools_3.6.0       
> 

System information

# GCC version
gcc --version | head -1
gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-17)
# Spack git version
git -C ~/spack/ for-each-ref --sort=-committerdate refs/heads/
4c465d260235090127e876ea493fee7de9372412 commit refs/heads/develop
# Operating system
./neofetch --backend off
pan14001@cn01
-------------
OS: Red Hat Enterprise Linux Server release 6.7 (Santiago) x86_64
Host: ProLiant SL390s G7
Kernel: 2.6.32-642.13.1.el6.x86_64
Uptime: 155 days, 21 hours
Packages: 1626 (rpm)
Shell: bash 4.1.2
Terminal: /dev/pts/34
CPU: Intel Xeon X5650 (12) @ 2.666GHz
GPU: AMD ATI ES1000
Memory: 18964MiB / 48253MiB
# Machine name
hostname -A
cn01 login.hpc.uconn.edu ib-cn01.engr.uconn.edu 
rbowtie software error • 667 views
ADD COMMENT
0
Entering edit mode

Edit: I fixed the md5-* injections by adding newlines between code blocks.

I'm not sure why the post has all the md5-* nonsense that wasn't in the preview >_<

ADD REPLY
0
Entering edit mode

Fault persists using the same compiler and flags as the Bioconductor malbec2 build server.

I found Bioconductor's report for the build server and Rbowtie installation:

  1. Build server compiler and compiler flags: https://bioconductor.org/checkResults/3.9/bioc-LATEST/malbec2-NodeInfo.html
  2. Rbowtie https://bioconductor.org/checkResults/3.9/bioc-LATEST/Rbowtie/malbec2-install.html

I tried using the same compiler gcc 7.4.0 as the build server, but the fault persists:

* installing *source* package ‘Rbowtie’ ...
** using staged installation
** libs
** arch -
g++ -O3 -m64 -DCOMPILER_OPTIONS="\"-O3 -m64  -Wl,--hash-style=both -DPOPCNT_CAPABILITY -g -O2   -g -O2  \""  -Wl,--hash-style=both -DPOPCNT_CAPABI\
LITY -g -O2   -g -O2    \
                -fno-strict-aliasing -DBOWTIE_VERSION="\"`cat VERSION`\"" -DBUILD_HOST="\"`hostname`\"" -DBUILD_TIME="\"`date`\"" -DCOMPILER_VERSI\
ON="\"`g++ -v 2>&1 | tail -1`\"" -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE  -DPREFETCH_LOCALITY=2 -DBOWTIE_MM -DBOWTIE_SHARED_MEM -\
DNDEBUG -Wall -Wno-unused-parameter -Wno-reorder -Wno-unused-local-typedefs \
                 -isystem ./SeqAn-1.1 -I third_party -I third_party \
                -o bowtie-build-s ebwt_build.cpp \
                ccnt_lut.cpp ref_read.cpp alphabet.cpp shmem.cpp edit.cpp ebwt.cpp tinythread.cpp  bowtie_build_main.cpp \
                -L/usr/local/lib64  -lz -lpthread
In file included from /home/pan14001/spack/opt/spack/linux-rhel6-x86_64/gcc-7.4.0/zlib-1.2.11-6w55r3jcve53es7i2iiamckkw5u4gb6a/include/zlib.h:34,
                 from pat.h:6,
                 from sequence_io.h:12,
                 from multikey_qsort.h:8,
                 from diff_sample.h:13,
                 from blockwise_sa.h:19,
                 from ebwt.h:27,
                 from ebwt_build.cpp:11:
/home/pan14001/spack/opt/spack/linux-rhel6-x86_64/gcc-7.4.0/zlib-1.2.11-6w55r3jcve53es7i2iiamckkw5u4gb6a/include/zconf.h:400: error: reference to \
‘Byte’ is ambiguous
/home/pan14001/spack/opt/spack/linux-rhel6-x86_64/gcc-7.4.0/zlib-1.2.11-6w55r3jcve53es7i2iiamckkw5u4gb6a/include/zconf.h:391: error: candidates ar\
e: typedef unsigned char Byte
./SeqAn-1.1/seqan/basic/basic_alphabet_interface.h:62: error:                 typedef unsigned char seqan::Byte

Like the build server, my C++ flags default to gnu+11:

[pan14001@cn01 ~]$ spack load r %gcc@7.4.0
[pan14001@cn01 ~]$ R CMD config --all | grep ^C | sed -E 's#(/home/.+/)(g..)#\2#'
CC = gcc
CFLAGS = -g -O2
CPICFLAGS = -fpic
CPP = gcc -E
CPPFLAGS = -I/usr/local/include
CXX = g++ -std=gnu++11
CXXCPP = g++ -std=gnu++11 -E
CXXFLAGS = -g -O2
CXXPICFLAGS = -fpic
CXX98 = g++
CXX98STD = -std=gnu++98
CXX98FLAGS = -g -O2
CXX98PICFLAGS = -fpic
CXX11 = g++
CXX11STD = -std=gnu++11
CXX11FLAGS = -g -O2
CXX11PICFLAGS = -fpic
CXX14 = g++
CXX14STD = -std=gnu++14
CXX14FLAGS = -g -O2
CXX14PICFLAGS = -fpic
CXX17 = g++
CXX17STD = -std=gnu++17
CXX17FLAGS = -g -O2
CXX17PICFLAGSS =
[pan14001@cn01 ~]$
ADD REPLY
0
Entering edit mode

Dear Pariksheet

Thanks for the report. I will try to reproduce this on our local environment and let you know when I find a fix.

Best, Michael

On 25 May 2019, at 16:18, Pariksheet Nanda [bioc] <noreply@bioconductor.org<a rel="nofollow" href="mailto:noreply@bioconductor.org">noreply@bioconductor.org> wrote:

Activity on a post you are following on support.bioconductor.orghttps://support.bioconductor.org/

User Pariksheet Nandahttps://support.bioconductor.org/u/10892/ wrote Comment: Rbowtie fails to compile with gcc 9.1.0 with error: reference to Byte is ambiguoushttps://support.bioconductor.org/p/121329/#121378:

Fault persists using the same compiler and flags as the Bioconductor malbec2 build server.

I found Bioconductor's report for the build server and Rbowtie installation:

  1. Build server compiler and compiler flags: https://bioconductor.org/checkResults/3.9/bioc-LATEST/malbec2-NodeInfo.html
  2. Rbowtie https://bioconductor.org/checkResults/3.9/bioc-LATEST/Rbowtie/malbec2-install.html

I tried using the same compiler gcc 7.4.0 as the build server, but the fault persists:

  • installing source package ‘Rbowtie’ ... * using staged installation * libs ** arch - g++ -O3 -m64 -DCOMPILEROPTIONS="\"-O3 -m64 -Wl,--hash-style=both -DPOPCNTCAPABILITY -g -O2 -g -O2 \"" -Wl,--hash-style=both -DPOPCNTCAPABI\ LITY -g -O2 -g -O2 \ -fno-strict-aliasing -DBOWTIEVERSION="\"cat VERSION\"" -DBUILDHOST="\"hostname\"" -DBUILDTIME="\"date\"" -DCOMPILERVERSI\ ON="\"g++ -v 2>&1 | tail -1\"" -DLARGEFILESOURCE -DFILEOFFSETBITS=64 -DGNUSOURCE -DPREFETCHLOCALITY=2 -DBOWTIEMM -DBOWTIESHAREDMEM -\ DNDEBUG -Wall -Wno-unused-parameter -Wno-reorder -Wno-unused-local-typedefs \ -isystem ./SeqAn-1.1 -I thirdparty -I thirdparty \ -o bowtie-build-s ebwtbuild.cpp \ ccntlut.cpp refread.cpp alphabet.cpp shmem.cpp edit.cpp ebwt.cpp tinythread.cpp bowtiebuildmain.cpp \ -L/usr/local/lib64 -lz -lpthread In file included from /home/pan14001/spack/opt/spack/linux-rhel6-x8664/gcc-7.4.0/zlib-1.2.11-6w55r3jcve53es7i2iiamckkw5u4gb6a/include/zlib.h:34, from pat.h:6, from sequenceio.h:12, from multikeyqsort.h:8, from diffsample.h:13, from blockwisesa.h:19, from ebwt.h:27, from ebwtbuild.cpp:11: /home/pan14001/spack/opt/spack/linux-rhel6-x8664/gcc-7.4.0/zlib-1.2.11-6w55r3jcve53es7i2iiamckkw5u4gb6a/include/zconf.h:400: error: reference to \ ‘Byte’ is ambiguous /home/pan14001/spack/opt/spack/linux-rhel6-x8664/gcc-7.4.0/zlib-1.2.11-6w55r3jcve53es7i2iiamckkw5u4gb6a/include/zconf.h:391: error: candidates ar\ e: typedef unsigned char Byte ./SeqAn-1.1/seqan/basic/basicalphabet_interface.h:62: error: typedef unsigned char seqan::Byte

Like the build server, my C++ flags default to gnu+11:

[pan14001@cn01 ~]$ spack load r %gcc@7.4.0 [pan14001@cn01 ~]$ R CMD config --all | grep ^C | sed -E 's#(/home/.+/)(g..)#\2#' CC = gcc CFLAGS = -g -O2 CPICFLAGS = -fpic CPP = gcc -E CPPFLAGS = -I/usr/local/include CXX = g++ -std=gnu++11 CXXCPP = g++ -std=gnu++11 -E CXXFLAGS = -g -O2 CXXPICFLAGS = -fpic CXX98 = g++ CXX98STD = -std=gnu++98 CXX98FLAGS = -g -O2 CXX98PICFLAGS = -fpic CXX11 = g++ CXX11STD = -std=gnu++11 CXX11FLAGS = -g -O2 CXX11PICFLAGS = -fpic CXX14 = g++ CXX14STD = -std=gnu++14 CXX14FLAGS = -g -O2 CXX14PICFLAGS = -fpic CXX17 = g++ CXX17STD = -std=gnu++17 CXX17FLAGS = -g -O2 CXX17PICFLAGSS = [pan14001@cn01 ~]$

ADD REPLY
0
Entering edit mode

​​Dear Pariksheet

Thanks again for reporting.

I have not been able to reproduce the issue on our local environment (CentOS Linux release 7.6.1810 (Core), gcc 9.1.0, binutils 2.32, zlib 1.2.11, R 3.6.0, BioC 3.9).

Compile flags seem to be mostly default and the same as yours (especially CXX, which may be relevant here). I do switch on more warnings by using a ~/.R/Makevars with:

CFLAGS = -g -O2 -Wall -pedantic CXXFLAGS = -g -O2 -Wall -pedantic CC = gcc -std=gnu99 CFLAGS = -g -O2 -Wall -pedantic CPICFLAGS = -fpic CPP = gcc -std=gnu99 -E CPPFLAGS = -I/usr/local/include CXX = g++ -std=gnu++11 CXXCPP = g++ -std=gnu++11 -E CXXFLAGS = -g -O2 -Wall -pedantic CXXPICFLAGS = -fpic CXX98 = g++ CXX98STD = -std=gnu++98 CXX98FLAGS = -g -O2 CXX98PICFLAGS = -fpic CXX11 = g++ CXX11STD = -std=gnu++11 CXX11FLAGS = -g -O2 CXX11PICFLAGS = -fpic CXX14 = CXX14STD = CXX14FLAGS = CXX14PICFLAGS = CXX17 = CXX17STD = CXX17FLAGS = CXX17PICFLAGSS =

As you mentioned, compilation also fails with gcc 7.4.0 on your side. However, it does work successfully on our local system with different versions of gcc (9.1.0 and 7.3.1), on the BioC build systems (both release and devel, gcc 7.4.0 and 7.3.0, respectively), and also on travis with gcc 4.8.4.

Without being able to reproduce the issue on my side, I don't know how to help. It seems possible that there is something specific to your environment that is causing an issue, and that it is not directly related to gcc 9.1.0, since compilation also fails for gcc 7.4.0.

I am happy to look further if you have any more ideas.

Best,

Michael

ADD REPLY
0
Entering edit mode
@michael-stadler-5887
Last seen 4 weeks ago
Switzerland

Dear Pariksheet

I have in the meantime updated the sources of bowtie within Rbowtie to version 1.2.3. This version includes several fixes to allow compilation of bowtie with new toolchains, including gcc 8, gcc 9 and intel. The updates are available from BioC-devel (BioC version 3.11, Rbowtie 1.27.1).

Maybe it would be worth trying again if you are now able to compile it.

Best wishes, Michael

ADD COMMENT

Login before adding your answer.

Traffic: 279 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6