---
name: modelos-locales-huggingface-llama-cpp
description: Guia tecnica para buscar, seleccionar y ejecutar modelos GGUF de HuggingFace en local usando llama.cpp, con soporte para CPU, Mac Metal, CUDA y ROCm. Incluye seleccion de cuantizacion, arranque de servidor compatible con OpenAI y conversion desde pesos Transformers.
license: Apache-2.0
metadata:
  id: c44b906b
  slug: modelos-locales-huggingface-llama-cpp
  titulo: "Modelos Locales HuggingFace con llama.cpp"
  servicio: IA-Ingenieria-MLOps
  categoria_recurso: IA-Generativa
  tipo: referencia
  nivel: avanzado
  idioma: es
  idioma_original: en
  acceso: gratis
  precio_eur: 0
  plataformas: [HuggingFace Hub, llama.cpp, macOS, Linux, Windows, CUDA, ROCm]
  dependencias: [llama.cpp, hf CLI, Python (conversion opcional)]
  licencia: { spdx: Apache-2.0, redistribuible: true, uso_comercial: true }
  fuente:
    repo: huggingface/skills
    url: https://github.com/huggingface/skills/tree/main/skills/huggingface-local-models
    commit: c68f1b0
    autor: huggingface
    nombre_original: huggingface-local-models
    duplicados_en: []
  seguridad: { veredicto: seguro, riesgo: bajo, escaneado: "2026-06-14", motor: "grep-estatico+auditor-llm" }
  ficha:
    que_hace: "Permite desplegar modelos LLM open-source en local con un endpoint compatible con la API de OpenAI."
    como_lo_hace: "Busca repos GGUF en el Hub, selecciona la cuantizacion optima segun hardware y lanza el modelo con llama-cli o llama-server."
  content_hash: "c44b906b72360170bceb31f7c31e0db2ec566b83642315b3191b04e56803ff23"
  version: 1.0.0
---

# Hugging Face Local Models

Search the Hugging Face Hub for llama.cpp-compatible GGUF repos, choose the right quant, and launch the model with `llama-cli` or `llama-server`.

## Default Workflow

1. Search the Hub with `apps=llama.cpp`.
2. Open `https://huggingface.co/<repo>?local-app=llama.cpp`.
3. Prefer the exact HF local-app snippet and quant recommendation when it is visible.
4. Confirm exact `.gguf` filenames with `https://huggingface.co/api/models/<repo>/tree/main?recursive=true`.
5. Launch with `llama-cli -hf <repo>:<QUANT>` or `llama-server -hf <repo>:<QUANT>`.
6. Fall back to `--hf-repo` plus `--hf-file` when the repo uses custom file naming.
7. Convert from Transformers weights only if the repo does not already expose GGUF files.

## Quick Start

### Install llama.cpp

```bash
brew install llama.cpp
winget install llama.cpp
```

```bash
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
make
```

### Authenticate for gated repos

```bash
hf auth login
```

### Search the Hub

```text
https://huggingface.co/models?apps=llama.cpp&sort=trending
https://huggingface.co/models?search=Qwen3.6&apps=llama.cpp&sort=trending
https://huggingface.co/models?search=<term>&apps=llama.cpp&num_parameters=min:0,max:24B&sort=trending
```

### Run directly from the Hub

```bash
llama-cli -hf unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_M
llama-server -hf unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_M
```

### Run an exact GGUF file

```bash
llama-server \
    --hf-repo unsloth/Qwen3.6-35B-A3B-GGUF \
    --hf-file Qwen3.6-35B-A3B-UD-Q4_K_M.gguf \
    -c 4096
```

### Convert only when no GGUF is available

```bash
hf download <repo-without-gguf> --local-dir ./model-src
python convert_hf_to_gguf.py ./model-src \
    --outfile model-f16.gguf \
    --outtype f16
llama-quantize model-f16.gguf model-q4_k_m.gguf Q4_K_M
```

### Smoke test a local server

```bash
llama-server -hf unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_M
```

```bash
curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer no-key" \
  -d '{
    "messages": [
      {"role": "user", "content": "Write a limerick about exception handling"}
    ]
  }'
```

## Quant Choice

- Prefer the exact quant that HF marks as compatible on the `?local-app=llama.cpp` page.
- Keep repo-native labels such as `UD-Q4_K_M` instead of normalizing them.
- Default to `Q4_K_M` unless the repo page or hardware profile suggests otherwise.
- Prefer `Q5_K_M` or `Q6_K` for code or technical workloads when memory allows.
- Consider `Q3_K_M`, `Q4_K_S`, or repo-specific `IQ` / `UD-*` variants for tighter RAM or VRAM budgets.
- Treat `mmproj-*.gguf` files as projector weights, not the main checkpoint.

## Load References

- Read [hub-discovery.md](references/hub-discovery.md) for URL-first workflows, model search, tree API extraction, and command reconstruction.
- Read [quantization.md](references/quantization.md) for format tables, model scaling, quality tradeoffs, and `imatrix`.
- Read [hardware.md](references/hardware.md) for Metal, CUDA, ROCm, or CPU build and acceleration details.

## Resources

- llama.cpp: `https://github.com/ggml-org/llama.cpp`
- Hugging Face GGUF + llama.cpp docs: `https://huggingface.co/docs/hub/gguf-llamacpp`
- Hugging Face Local Apps docs: `https://huggingface.co/docs/hub/main/local-apps`
- Hugging Face Local Agents docs: `https://huggingface.co/docs/hub/agents-local`
- GGUF converter Space: `https://huggingface.co/spaces/ggml-org/gguf-my-repo`