环境介绍

本文基于以下环境编写：

• Ubuntu 24.04
• PPU SDK v2.0.0
• CUDA 兼容 SDK 12.9

编译过程

首先将 llama.cpp 代码仓库克隆到你喜欢的目录，例如我这里是 ~/work/llama.cpp/src。

git clone https://github.com/ggml-org/llama.cpp.git -b v0.4.0 --depth 1 ~/work/llama.cpp/src

保存 patch 文件 ~/work/llama.cpp/ppu.patch：

diff --git a/ggml/src/ggml-cuda/topk-moe.cu b/ggml/src/ggml-cuda/topk-moe.cu
index 5af6cc2..4341f34 100644
--- a/ggml/src/ggml-cuda/topk-moe.cu
+++ b/ggml/src/ggml-cuda/topk-moe.cu
@@ -5,6 +5,10 @@
 #include 
<cmath>
 #include 
<initializer_list>

+#if defined(USE_HGGC)
+#define __isnanf(x) isnan(x)
+#endif
+
 // Kernel config struct - passed by value to CUDA kernel
 struct topk_moe_config {
     bool use_sigmoid;

创建目录 /opt/llama.cpp：

mkdir -p /opt/llama.cpp

保存编译脚本 ~/work/llama.cpp/build.sh：

#!/usr/bin/env bash

ROOT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
cd $ROOT_DIR/src

git apply ../ppu.patch

CUDA_ROOT="${CUDA_HOME:-/usr/local/PPU_SDK/CUDA_SDK}"
NVCC="${CUDA_ROOT}/bin/nvcc"

CUDA_ARCH="$(
    "${NVCC}" --list-gpu-arch |
    while IFS= read -r arch; do
        case "${arch}" in
            compute_[0-9]*) printf '%s\n' "${arch#compute_}" ;;
        esac
    done |
    sort -V |
    tail -n 1
)"

echo "CUDA architecture: ${CUDA_ARCH}"

cmake -S . -B build-ppu -G Ninja \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_INSTALL_PREFIX="/opt/llama.cpp" \
    -DCMAKE_CUDA_COMPILER="${NVCC}" \
    -DCUDAToolkit_ROOT="${CUDA_ROOT}" \
    -DCMAKE_CUDA_ARCHITECTURES="${CUDA_ARCH}" \
    -DCMAKE_INSTALL_RPATH="\$ORIGIN/../lib;\$ORIGIN;${CUDA_ROOT}/lib64;/usr/local/PPU_SDK/lib" \
    -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON \
    -DGGML_CUDA=ON \
    -DLLAMA_BUILD_TESTS=OFF \
    -DLLAMA_BUILD_EXAMPLES=OFF \
    -DLLAMA_BUILD_TOOLS=ON \
    -DLLAMA_BUILD_SERVER=ON \
    -DLLAMA_BUILD_APP=ON

cmake --build build-ppu --config Release --parallel 48
cmake --install build-ppu --config Release

编译完成后会将可执行安装到 /opt/llama.cpp，将如下内容追加至 /etc/profile 末尾：

export LLAMA_CPP_HOME=/opt/llama.cpp
export PATH=$LLAMA_CPP_HOME/bin:$PATH

重新连接会话即可使用。