June 3, 2026
I Ran NVIDIA's New Vision Model Locally — LocateAnything-3B on RTX 4060 8GB
NVIDIA released LocateAnything-3B — a 3B vision-language model that can find anything you describe in an image. I ran it fully locally on an RTX 4060 laptop (8GB) in 4-bit NF4 — no Ollama, no GGUF, just transformers + bitsandbytes.
Architecture
MoonViT (visual encoder) + Qwen2.5-3B + Parallel Box Decoding — predicts bounding boxes in parallel instead of token by token, which is why it's fast.
Five jobs, one model
- Object detection — list categories, it boxes them all (27 hits on a fruit scene)
- Phrase grounding — natural language → exact region
- OCR — every text region in box format
- Pointing — returns an XY coordinate
- GUI grounding — finds UI elements (e.g. the address bar)
Honest trade-offs
No GGUF/Ollama path (custom architecture, trust_remote_code), 4-bit required, slow mode only, and an NVIDIA non-commercial license (research/academic only).
Model → https://huggingface.co/nvidia/LocateAnything-3B · Setup: pip install transformers==4.57.1 bitsandbytes accelerate opencv-python pillow
All five tasks demoed in the video above. Subscribe for more local vision-AI.