Cross compiling Rust

I have not written in a long time so let me give it another try. I have been spending the last couple of weeks on a network service in Rust. The binary is deployed on a low resource aarch64 machine and as most linux users do, I use amd64 for development. Building Rust projects on low resource machines is painfully slow so naturally, we need to set up a cross compilation environment for Rust.

Unlike most tools in the Rust ecosystem, cross-compilation was not as well documented. This post describes the key knobs I had to tweak to get it working.

C toolchain

Moderately complex network services using TLS end up depending on a few -sys crates, so we first need a good cross compiler. We could have used the distro-provided one but I did not want to deal with mismatching kernel or libc versions on the target machine.

Thanks to the neat work of crosstool-ng, we first build a cross-toolchain with host=x86_64-build_pc-linux-gnu and target=aarch64-unknown-linux-gnu with the appropriate kernel and glibc(xx) versions.

Configuring Rust

We set the following environment variables to the appropriate binaries we just built. Here, TOOLS_DIR is typically $HOME/x-tools/$TRIPLE and $TRIPLE is the target triple.

export CC="$TOOLS_DIR/bin/$TRIPLE-gcc"
export CPP="$TOOLS_DIR/bin/$TRIPLE-gcc -E"
export CXX="$TOOLS_DIR/bin/$TRIPLE-g++"
export LD="$TOOLS_DIR/bin/$TRIPLE-ld"
export AR="$TOOLS_DIR/bin/$TRIPLE-ar"
export RANLIB="$TOOLS_DIR/bin/$TRIPLE-ranlib"

For practical reasons, C++ binaries built with clang link with the GCC libstdc++ and not to LLVM libc++. Since bindgen uses clang for parsing C sources, we also need to instruct bindgen to use the right sysroot with clang.

export BINDGEN_EXTRA_CLANG_ARGS="--sysroot=$TOOLS_DIR/$TRIPLE/sysroot"

We also set the linker and archiver once again for cargo.

export CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_AR="$AR"
export CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER="$CC"

I almost missed this and ended up with linker errors about linking two objects with mismatching architectures. While the rest of the environment variables are automatically picked up by cargo, AR and LINKER need to be set explicitly. ^[1]

Use CRATE_CC_NO_DEFAULTS=1 to further tweak the compilation flags if necessary.

That's it - a nice and simple cross compilation setup with cargo build --target aarch64-unknown-linux-gnu.

$ file target/aarch64-unknown-linux-gnu/release/myservice
target/aarch64-unknown-linux-gnu/release/myservice: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 5.0.19, not stripped

$ objdump -p target/aarch64-unknown-linux-gnu/release/myservice
<...>
Dynamic Section:
  NEEDED               libgcc_s.so.1
  NEEDED               libm.so.6
  NEEDED               libc.so.6
<...>