Compiling with LDB Toolchain (Recommended)

This guide is about how to compile Doris using the LDB Toolchain. This method serves as a supplement to the Docker compilation approach to help the developers and users without a Docker environment. The recommended LDB Toolchain version is 0.17, which includes clang-16 and gcc-11.

tip

LDB Toolchain is fully known as Linux Distribution Based Toolchain Generator. It helps compile modern C++ projects on almost all Linux distributions.

Special thanks to Amos Bird for the contribution.

Prepare the compilation environment:

This method applies to most Linux distributions (CentOS, Ubuntu, etc.).

1. Download ldb_toolchain_gen.sh

Download the latest ldb_toolchain_gen.sh from here. This script is used to generate ldb toolchain.

tip

For more information, please visit https://github.com/amosbird/ldb_toolchain_gen

2. Execute the following command to generate ldb toolchain.

sh ldb_toolchain_gen.sh /path/to/ldb_toolchain/

/path/to/ldb_toolchain/ is the installation directory for the toolchain. After successful execution, the following directory structure will be generated under /path/to/ldb_toolchain/:

├── bin
├── include
├── lib
├── share
├── test
└── usr

3. Download and install other compilation components

• Download Java8 and install it to /path/to/java.

  For versions later than 3.0 (inclusive), or the master branch, please use Java 17.

• Download Apache Maven 3.6.3 and install it to /path/to/maven.

• Download Node v12.13.0 and install it to /path/to/node.

• Different Linux distributions may include different default components. Therefore, you may need to install some additional components. The following takes CentOS 6 as an example. Similar steps may apply to other distributions:

install required system packages
sudo yum install -y byacc patch automake libtool make which file ncurses-devel gettext-devel unzip bzip2 zip util-linux wget git python2

install autoconf-2.69
wget http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz && \
    tar zxf autoconf-2.69.tar.gz && \
    cd autoconf-2.69 && \
    ./configure && \
    make && \
    make install

install bison-3.0.4
wget http://ftp.gnu.org/gnu/bison/bison-3.0.4.tar.gz && \
    tar xzf bison-3.0.4.tar.gz && \
    cd bison-3.0.4 && \
    ./configure && \
    make && \
    make install

4. Download Doris source code

git clone https://github.com/apache/doris.git

After downloading, navigate to the Doris source code directory, create a custom_env.sh file, and set the PATH environment variable as follows:

export JAVA_HOME=/path/to/java/
export PATH=$JAVA_HOME/bin:$PATH
export PATH=/path/to/maven/bin:$PATH
export PATH=/path/to/node/bin:$PATH
export PATH=/path/to/ldb_toolchain/bin:$PATH

Compile Doris

tip

The first step of compiling Doris source code is to first download third-party libraries and compile them. You can refer to the following instructions to download precompiled versions of the third-party libraries.

1. Enter the Doris source code directory and execute the following command to check if the compilation machine supports the AVX2 instruction set.

$ cat /proc/cpuinfo | grep avx2

2. Execute compilation.

# By default, it builds AVX2 version.
$ sh build.sh

# If you need the no AVX2 version, add USE_AVX2=0.
$ USE_AVX2=0 sh build.sh

# To compile a debug version of BE, add BUILD_TYPE=Debug.
$ BUILD_TYPE=Debug sh build.sh

This script first compiles the third-party libraries and then the Doris components (FE, BE, MS). The compilation output can be found in the output/ directory. MS stands for Meta Service, which a module of Doris in the compute-storage decoupled mode. For more information about MS, refer to this doc.

Pre-compile third-party libraries

The build.sh script compiles third-party libraries first. You can download the pre-compiled third-party libraries directly from the following link:

https://github.com/apache/doris-thirdparty/releases

The pre-compiled libraries for Linux and MacOS are provided. If they match your compilation and runtime environment, you can download them and use them directly.

After downloading, extract the files to obtain an installed/ directory. Copy this directory to the thirdparty/directory, and then run build.sh to proceed with the compilation.