BCS Blog

COBOL, z/OS performance measurement, and systems programming updates.

Run llama.cpp

By Arch Brooks on July 10, 2026 in Programming

Back to Blog ← Back

Permalink

If you built llama.cpp from source, you start it by running one of the executables in the build/bin directory.

1. Load the oneAPI environment (for SYCL builds)

My current Gentoo already have the variables loaded.  Do not run /opt/intel/oneapi/setvars.sh
source /opt/intel/oneapi/setvars.sh

2. Change to the llama.cpp directory

cd ~/Applications/llama.cpp

3. Run a model

For interactive chat:

./build/bin/llama-cli \
    -m ~/models/qwen3-8b.gguf \
    -i

./build/bin/llama-cli \
    -m ~/models/gemma-3-4b-it.gguf \
    -i

4. Start the OpenAI-compatible server

./build/bin/llama-server \
    -m ~/models/qwen3-8b.gguf \
    --host 0.0.0.0 \
    --port 8080

You can then access it at:

http://localhost:8080

5. Run with GPU acceleration (SYCL)

If you compiled with:

cmake -DGGML_SYCL=ON ...

then simply running llama-cli or llama-server uses the SYCL backend automatically if an Intel GPU is available.

To verify, you should see output similar to:

SYCL0 : Intel(R) Arc(TM) Graphics

6. Check which executables are available

ls build/bin

Typical output:

llama-bench
llama-cli
llama-convert
llama-gguf-split
llama-perplexity
llama-quantize
llama-run
llama-server
llama-simple

7. If `llama.cpp` is in your `PATH`

You can run it directly:

llama-cli -m ~/models/qwen3-8b.gguf -i

llama-server -m ~/models/qwen3-8b.gguf

8. Verify your build

./build/bin/llama-cli --version

./build/bin/llama-bench

Since we've already discussed your Gentoo setup with Intel oneAPI and an Intel Arc GPU, your SYCL-enabled build should automatically use GPU acceleration when the oneAPI environment is loaded and the model is compatible.

Comments

No comments yet.

Title *

Slug

Excerpt

Content *

HTML is supported.

SEO Meta Data

Meta Title

Meta Description