Heretic 12B — Uncensored Gemma 4 Running Locally on Ollama

Someone fine-tuned Gemma 4 12B to remove the guardrails. This is Heretic, an uncensored model you can run entirely on your own machine via Ollama — no API, no cloud, no data leaving your system. The focus here is the engineering: getting a tricky GGUF to load and run locally.

The setup (and the gotchas)

• GGUF from HuggingFace (Q4_K_M, 7.4GB) by igorls
• Build the Modelfile with the correct Gemma 4 chat template — extracted straight from the GGUF metadata (turn tokens, thinking-channel suppression, stop sequences; three lines won't cut it)
• ollama create fails validation on Gemma 4's mixed quantization → fix: build the model manifest manually (the blob is already copied; bypass the broken quantize check)
• ⚠️ GPU mode crashes in llama 130.6 (stack buffer overrun in the CUDA kernel from Gemma 4's per-layer scale tensors) → workaround: CPU mode via the API options (~4–5 tok/s; slow but works)

Model → https://huggingface.co/igorls/gemma-4-12B-it-heretic-GGUF · Modelfile + scripts → https://github.com/PromptEngineer48

Answers in the video are blurred — the point is the local setup and the failure-fixes, not the content.

Full walkthrough in the video above. Subscribe for more local-LLM engineering.