BCS Blog

COBOL, z/OS performance measurement, and systems programming updates.

re route hf models

By Arch Brooks on July 26, 2026 in Artificial Intelligence

Back to Blog ← Back

Permalink

Recent versions of llama.cpp use the standard Hugging Face cache when you use -hf. By default that's ~/.cache/huggingface/hub, but you can redirect it.

To store all downloaded Hugging Face models in ~/models:

mkdir -p ~/models

export HF_HUB_CACHE=$HOME/models
export HUGGINGFACE_HUB_CACHE=$HOME/models
export LLAMA_CACHE=$HOME/models

Then run:

llama-cli -hf ggml-org/gemma-3-1b-it-GGUF:Q4_K_M

The model will be downloaded into ~/models instead of ~/.cache/huggingface/hub.

To make this permanent on Gentoo, add the following to your ~/.bashrc:

export HF_HUB_CACHE="$HOME/models"
export HUGGINGFACE_HUB_CACHE="$HOME/models"
export LLAMA_CACHE="$HOME/models"

Reload it:

source ~/.bashrc

Verify:

echo $HF_HUB_CACHE
echo $LLAMA_CACHE

You should see:

/home/archman/models
/home/archman/models

Alternatively, if you prefer to keep your models as normal .gguf files in one directory instead of using the Hugging Face cache layout, download them once and run them directly:

llama-cli -m ~/models/gemma-3-1b-it-Q4_K_M.gguf

This is the approach many people use when maintaining a local model library.

Comments

No comments yet.

Title *

Slug

Excerpt

Content *

HTML is supported.

SEO Meta Data

Meta Title

Meta Description