I Ran Google's Gemma 4 12B on an RTX 4060 Laptop — Here's What Happened

Google dropped Gemma 4 — a full open-source family, Apache 2.0. The 12B runs on a consumer RTX 4060 (8GB). I downloaded the GGUF, ran it, and here's what it actually does.

The family + setup

Five sizes: 2B and 4B (phones/laptops), 12B, 26B, 31B. The 12B is special — the smallest that supports text, images, and audio, with a 256K context window, commercially free.

Setup is one download: Q4_K_M GGUF from Unsloth (6.6GB), launch llama-server -m gemma-4-12b-it-Q4_K_M.gguf -ngl 99 --port 8080. Loads in ~45s, using 7.9 of 8GB VRAM — tight, but it runs.

What I tested

Reasoning (bat & ball — shows its thinking chain)
Code (palindrome checker — all 4 tests pass)
JSON structured extraction (58 tokens, 2.8s)
Multilingual (Japanese, Arabic, Spanish)
Honesty — asked about a fake study, it refused to hallucinate
Speed: 21.5 tok/s average on the 4060

Notes: Ollama parses the E4B but errors on the 12B GGUF; vision needs the mmproj companion file.

GGUF → https://huggingface.co/unsloth/gemma-4-12b-it-GGUF

Full first look + verdict in the video above. Subscribe for more local-LLM tests.