# Ornith vLLM — ia-lab GPU 2

This installation is isolated from llama.cpp (GPU 0) and SGLang (GPU 1).

- Physical GPU: `GPU-5f32faf3-97b9-895e-0a9e-ffa0f51bca3f` (RTX 3090).
- Environment: `/home/lab/venvs/vllm`.
- Official source: `ornith-ai/Ornith-1.5-9B`, revision `489cb97981b8654bcfcf30ce1f94ed1b62e07b53`.
- Read-only original checkpoint: `/home/lab/models/Ornith-1.5-9B-original`.
- API: `http://127.0.0.1:8140/v1`, model `ornith-vllm`.
- Quantization: bitsandbytes in-flight NF4 4-bit. It quantizes supported linear weights when loading; this is **not** a saved quantized checkpoint. Some weights remain BF16.
- Initial text-only configuration: 8192 context, maximum 2 concurrent sequences, BF16 activations, eager execution, 80% GPU memory budget. No QLoRA or training.

## Controls

```sh
bash /home/lab/bench/ornith/vllm/service.sh start
bash /home/lab/bench/ornith/vllm/service.sh status
bash /home/lab/bench/ornith/vllm/service.sh stop
tail -f /home/lab/bench/ornith/vllm/server.log
python3 /home/lab/bench/ornith/vllm/smoke.py
```

The transient user service survives SSH disconnects, but is not enabled on reboot. It binds localhost only; use an SSH tunnel for remote access.

`environment-before.txt` and `environment.txt` record package versions. `setup.log`, `server.log`, and `smoke.json` are evidence artifacts; only a successful smoke run confirms serving.

## Verified 2026-09-18

vLLM 0.29.0, PyTorch 2.13.0+cu130, Transformers 5.17.0, bitsandbytes 0.50.2, vllm-bnb-plugin 0.0.3. `pip check` passed. The masked environment exposes exactly one RTX 3090. Model loading used 7.55 GiB; additional allocated memory is KV cache/runtime, not full-precision weights.

Health and model-list endpoints returned HTTP 200. Real completions returned `VLLM_OK`, arithmetic answer `42`, and valid-looking Python `suma(a,b)` code. The last two were concurrent requests. These are functional smoke tests, not a quality benchmark or a fair speed comparison. No inference requests remain from this smoke run.

Official references: https://docs.vllm.ai/en/latest/features/quantization/bnb/ and https://github.com/vllm-project/vllm-bnb-plugin .
