Skip to content

Run on a Bare-Metal Platform

IREE supports CPU model execution on a bare-metal platform. That is, a platform without operating system support, and the executable is built with the machine-specific linker script and/or the board support package (BSP).

Bare-metal deployment typically uses IREE's LLVM compiler target much like the CPU - Dylib configuration, but using a limited subset of IREE's CPU HAL driver at runtime to load and execute compiled programs.

Prerequisites

Out-of-tree bare-metal platform tools and source code for the system should be ready, such as

  • Compilation toolchain
  • Platform linker script
  • Firmware libraries

Please follow the instructions to retrieve the IREE compiler.

Compile the model for bare-metal

The model can be compiled with the following command from the IREE compiler build directory

iree/tools/iree-translate \
    -iree-mlir-to-vm-bytecode-module \
    -iree-stream-partitioning-favor=min-peak-memory \
    -iree-hal-target-backends=dylib-llvm-aot \
    -iree-llvm-target-triple=x86_64-pc-linux-elf \
    -iree-llvm-debug-symbols=false \
    iree/samples/models/simple_abs.mlir \
    -o /tmp/simple_abs_dylib.vmfb

In which

  • -iree-stream-partitioning-favor=min-peak-memory: Optimize for minimum peak memory usage at the cost of concurrency - include when targeting single-threaded execution to reduce memory consumption.
  • iree-hal-target-backends=dylib-llvm-aot: Build the model for the dynamic library CPU HAL driver
  • iree-llvm-target-triple: Use the <arch>-pc-linux-elf LLVM target triple so the artifact has a fixed ABI to be rendered by the elf_module library
  • iree-llvm-debug-symbols=false: To reduce the artifact size

See generate.sh for example command-line instructions of some common architectures

You can replace the MLIR file with the other MLIR model files, following the instructions

Compiling the bare-metal model for static-library support

See the static_library demo sample for an example and instructions on running a model with IREE's static_library_loader.

By default, the demo targets the host machine when compiling. To produce a bare-metal compatible model, run iree-translate as in the previous example and add the additional -iree-llvm-static-library-output-path= flag to specify the static library destination. This will produce a .h\.o file to link directly into the target application.

Build bare-metal runtime from the source

A few CMake options and macros should be set to build a subset of IREE runtime libraries compatible with the bare-metal platform. We assume there's no multi-thread control nor system library support in the bare-metal system. The model execution is in a single-thread synchronous fashion.

Set CMake options

  • set(IREE_BUILD_COMPILER OFF): Build IREE runtime only
  • set(CMAKE_SYSTEM_NAME Generic): Tell CMake to skip targeting a specific operating system
  • set(IREE_BINDINGS_TFLITE OFF): Disable the TFLite binding support
  • set(IREE_ENABLE_THREADING OFF): Disable multi-thread library support
  • set(IREE_HAL_DRIVER_DEFAULTS OFF): Disable HAL drivers by default, then enable the synchronous HAL drivers with set(IREE_HAL_DRIVER_VMVX_SYNC ON) and set(IREE_HAL_DRIVER_DYLIB_SYNC ON)
  • set(IREE_BUILD_TESTS OFF): Disable tests until IREE supports running them on bare-metal platforms
  • set(IREE_BUILD_SAMPLES ON): Build simple_embedding example

Todo

Clean the list up after #6353 is fixed.

Also, set the toolchain-specific cmake file to match the tool path, target architecture, target abi, linker script, system library path, etc.

Define IREE macros

  • -DIREE_PLATFORM_GENERIC: Let IREE to build the runtime library without targeting a specific platform
  • -DIREE_SYNCHRONIZATION_DISABLE_UNSAFE=1: Disable thread synchronization support
  • -DIREE_FILE_IO_ENABLE=0: Disable file I/O
  • -DIREE_TIME_NOW_FN: A function to return the system time. For the bare-metal system, it can be set as -DIREE_TIME_NOW_FN=\"\{ return 0;\}\" as there's no asynchronous wait handling

Examples of how to setup the CMakeLists.txt and .cmake file:

Bare-metal execution example

See simple_embedding for generic platform to see how to use the IREE runtime library to build/run the IREE model for the bare-metal target.

Back to top