# Phase C2: Llama.cpp server preparation for Phase 4–5

**Decision**: Llama.cpp server deployment postponed from F2 to Phase 4–5 (see Phase C1 decision: Ollama-only for F2).

## Purpose

This document documents the llama.cpp server configuration and dependency chain required for deployment after F2 completion. The server runs on port 8080 and is not required for F2 Gate 1/2 verification.

## Configuration

### `cb26-lab-start.sh` — Lab startup script

The startup script is the primary mechanism for launching the llama.cpp server in the development environment.

- **Port**: 8080
- **Model loading**: Requires a `.gguf` model file path
- **GPU layers**: `n_gpu_layers=0` (CPU-only for F2 Phase 4–5 preview)
- **Context**: `n_ctx=2048`
- **Threads**: `n_threads=4`

### Required environment variables

| Variable | Default | Description |
|---|---|---|
| `LLAMA_CPP_BASE_URL` | `http://127.0.0.1:8080` | Server base URL |
| `LLAMA_CPP_TIMEOUT` | `600` | Request timeout in seconds |
| `LLAMA_CPP_MODEL_PATH` | *required* | Path to `.gguf` model file |

### `llama_cpp_client.py` — Python client wrapper

- Uses `httpx.Client` with `base_url=self.config.base_url`
- Timeout configurable via `LlamaCppConfig.timeout` (default 600s)
- Supports `verify_ssl` flag (default True)
- Client initialized from `ModelGateway` config (`fallback_providers: ["ollama", "llama.cpp"]`)

### Dependency chain

1. **Python package**: `llama-cpp-python` (with `gguf` support)
2. **Model file**: `.gguf` format model (e.g., `phi-2.gguf`, `mistral-7b.Q4_gguf.gguf`)
3. **Server binary**: `llama-server` (or `llama.cpp` CLI)
4. **Startup**: `cb26-lab-start.sh` launches server, registers provider in `ModelGateway`

### Known limitations (Phase 4–5)

- **No GPU support** in F2 preview: `n_gpu_layers=0`
- **Embedding pipeline**: Remains Ollama-only (`127.0.0.1:11434`, 1024-dim vectors)
- **llama.cpp chat completions**: Available as fallback after Ollama exhaustion
- **Profiling required**: Token throughput and latency measurements before Phase 4 production deployment

### Integration points

- `ModelGateway._register_builtin_providers()` registers `llama.cpp` as a free provider with:
  - `api_base="http://127.0.0.1:8080"`
  - `available_models=["llama.cpp"]`
  - `local_availability=True`
- `_resolve_route()` checks `prefer_local` / `config.prefer_local` first
- `_fallback_chat()` switches to llama.cpp when Ollama falls exhausted
- Gate 1/2 tests: **Pass with Ollama-only**, llama.cpp not required

### Open questions for owner

1. **Model selection**: Which `.gguf` model(s) to provision for Phase 4–5?
2. **Hardware**: GPU vs CPU-only deployment target?
3. **Port conflict**: `llama.cpp` port 8080 vs other services — coordination needed?
4. **Production deployment**: Docker vs bare-metal for Phase 4–5?

## References

- `src/cyberbrain26/model_layer/llama_cpp_client.py`: Client implementation
- `cb26-lab-start.sh`: Lab startup script (root of project)
- `src/cyberbrain26/control_plane/model_gateway/__init__.py:141-166`: Provider registration
- Phase C1 decision: Ollama-only for F2 — Proof: 5 Ollama models available, embedding verified