🪨 Granite Switch 4.1 8B — Mellea IVR Demo (ZeroGPU)
ibm-granite/granite-switch-4.1-8b-preview
is a single 8B checkpoint with embedded LoRA adapters. This demo drives it
with Mellea's HuggingFace backend:
- Pick adapters, optionally state requirements, and submit a prompt.
- With requirement-check selected, Mellea runs a validate-and-resample loop. The model answers your prompt without seeing the requirement; the embedded requirement-check aLoRA then judges each draft against it, and the model resamples until a draft passes or the attempt budget runs out. Each draft streams into its own bubble, followed by the checker's verdict in a separate bubble.
- uncertainty and guardian-core then judge the final answer, each in its own bubble. Use Clear to start over.
Under each response a ⚡ note reports that generation's KV-cache hit rate: adapter and retry turns reuse the cached conversation prefix (actual measured reuse — aLoRA adapters only apply after their activation token, so the base KV stays valid). The first draft of every interaction is a cold start because ZeroGPU releases the GPU between interactions.
The adapters are activated by control tokens spliced in by the model's chat template — no separate adapter weights are loaded. Judged turns are always greedy; only the drafts use your temperature.
Switch Configuration
requirement-check: Runs a validate-and-resample loop: the model answers without seeing your requirement, the requirement-check aLoRA judges each draft against it, and the model resamples until a draft passes (or the attempt budget runs out).
uncertainty: After the final answer, scores how certain the model is about it (0–1).
guardian-core: After the final answer, screens it for harm and reports a risk score (0–1).