Best Local Coding LLM for 16GB VRAM: gpt-oss, Qwen3.8, Gemma 4
For a 16GB VRAM GPU, start with gpt-oss-20b. Try Qwen3.8-27B only as a 3-bit file with a q8_0 cache, and use Gemma 4 12B when you need long context.
On this page

As of October 22, 2026, the best local coding LLM for a 16GB VRAM GPU is still gpt-oss-20b as the safe default: it is the only candidate whose maker says it runs within 16GB of memory, and its 12.11 GB MXFP4 file plus a 64K context comes to about 12.8 GiB by calculation. The picks around it changed since July. Qwen3.8-27B (released August 5) replaced Qwen3.6 35B A3B as the first Qwen to try, but only as a 3-bit file such as UD-Q3_K_XL with a quantized cache. Gemma 4 12B replaced Gemma 4 E4B as the fit-first model on a 16GB card, with room for 128K context. Devstral Small 2 is the agentic-coding specialist, but only at short context on 16GB.
Two settings decide more than the model name:
- Pick an exact file, not a family.
qwen3.8:27bin Ollama is an 18GB Q4_K_M package and does not fit a 16GB card; the 13.15 GBUD-Q3_K_XLGGUF does. - Raise the context. Ollama starts GPUs with less than 24 GiB of VRAM at a 4K context, while its own docs recommend at least 64,000 tokens for coding tools.
GLM-5.3-Flash, Qwen3.8-Flash-Next and Qwen3-Coder-Next are not 16GB models at all: the first two ship hundreds of gigabytes of weights, and Qwen3-Coder-Next is 48.4 GB even at Q4_K_M.
Quick Answer: Pick the Route First
This board shows the July 2026 picks (Qwen3.6 low-bit, Gemma 4 E4B); the table below has the current ones.
Pick by what you need the model to hold, then pick the file. Sizes are Hugging Face file sizes (GB = 10^9 bytes); the totals are weights plus KV cache in GiB (2^30 bytes), calculated from each model's config, before runtime buffers and whatever your desktop already uses on the card.
| Your situation | Model and exact file | Weights | Estimated total | Main caveat |
|---|---|---|---|---|
| Safe default, first install | gpt-oss-20b, MXFP4 GGUF | 12.11 GB file (11.28 GiB) | ≈12.0 GiB at 32K, ≈12.8 GiB at 64K | Smaller than the 24–27B models; no vendor coding scores comparable to Qwen's |
| Strongest Qwen that fits | Qwen3.8-27B, UD-Q3_K_XL GGUF (Unsloth) | 13.15 GB file (12.24 GiB) | ≈13.3 GiB at 32K, ≈14.4 GiB at 64K, both with a q8_0 cache | 3-bit community quant; Qwen's scores were measured at full precision |
| Long context with room to spare | Gemma 4 12B, Google's QAT Q4_0 GGUF | 6.98 GB file (6.50 GiB) | ≈7.8 GiB at 64K, ≈8.8 GiB at 128K | Coding scores are below the 27B class |
| Agentic coding specialist | Devstral Small 2 24B, Q3_K_M GGUF | 11.47 GB file (10.69 GiB) | ≈13.4 GiB at 32K with a q8_0 cache | Heavy KV cache; 64K does not fit |
| 8GB card, laptop, 16GB Mac | Gemma 4 E4B or Gemma 4 12B QAT Q4_0 | 5.15 GB / 6.98 GB files | see the 16GB RAM section | Mac budget is about 11.8 GiB, not 16 |
| Repo-scale agent loops | More VRAM or hosted coding | — | — | 16GB runs out of context before it runs out of model |
If you want one answer: install gpt-oss-20b, set the context to 64K, and run the smoke test further down. If it fails on quality and your card still shows headroom, try Qwen3.8-27B UD-Q3_K_XL. If you need a whole module in context, use Gemma 4 12B.
What Changed Since July: Qwen3.8-27B, Gemma 4 12B, Devstral Small 2
If you saw a July 2026 list built around Qwen3.6 35B A3B, Gemma 4 E4B and older code models, here is what each pick maps to now.
| July pick | Current pick | Why it changed |
|---|---|---|
gpt-oss-20b (safe default) | gpt-oss-20b, unchanged | OpenAI has not released a newer small gpt-oss model; it is still the only candidate with a first-party "within 16GB of memory" statement |
| Qwen3.6 35B A3B, low-bit or offload | Qwen3.8-27B in a 3-bit or IQ4 file | Released August 5, 2026; Qwen's own table puts it ahead of Qwen3.6-27B on every coding row. Qwen3.6 35B A3B stays as the offload option for machines with lots of system RAM |
| Gemma 4 E4B (fit-first fallback) | Gemma 4 12B | The 12B size appeared on May 23, 2026. Google aims it at consumer GPUs, and it scores 72.0% vs 52.0% for E4B on LiveCodeBench v6. E4B stays for 8GB cards and laptops |
| Qwen2.5-Coder, DeepSeek Coder, Qwen3-Coder-30B-A3B | Devstral Small 2 24B, or Qwen3.6-27B's qwen3.6:27b-coding package on bigger cards | Devstral Small 2 is built for agentic coding and reports 68.0% on SWE-bench Verified. The older coder families still run but are no longer the strongest open options |
Other open models with 3-bit or QAT files that land near the 16GB line: GLM-4.7-Flash (UD-Q3_K_XL 13.78 GB, Q3_K_M 14.61 GB), Gemma 4 26B A4B (Google's QAT Q4_0 file 14.44 GB, plus a 1.19 GB vision projector), and IBM Granite 4.2 30B (Q3_K_M 14.13 GB). They leave less room for context than the picks above, so they are worth trying only after those.
Evidence Boundary: Official, Runtime, Community
Each claim on this page belongs to one of four kinds of source, and they carry different weight.
First-party model cards. The OpenAI gpt-oss-20b model card on Hugging Face describes a 21B-parameter model with 3.6B active parameters, MoE weights post-trained in MXFP4, and says it runs "within 16GB of memory." The Gemma 4 12B model card says the 12B, 26B A4B and 31B sizes target consumer GPUs and workstations, while E2B and E4B target mobile and edge devices. The Qwen3.8-27B model card positions the model for coding and long-horizon agentic tasks but makes no 16GB claim. The Devstral Small 2 model card says it is "light enough to run on a single RTX 4090 or a Mac with 32GB RAM", which is a 24GB card, not a 16GB one.
Vendor benchmark scores. Qwen's coding scores for Qwen3.8-27B were measured at full precision with a 256K context window. A 3-bit file at 32K–64K context is a different setup, so the score is a reason to try the model, not a promise about your build. Google's Gemma scores come from LiveCodeBench and Codeforces, not repo-level agent tasks.
Quantized files and runtime packages. The 3-bit and IQ4 Qwen and Devstral files come from Unsloth, a community quantizer, not from the model makers. Gemma's QAT Q4_0 files are Google's own. Ollama package sizes (gpt-oss:20b 14GB, qwen3.8:27b 18GB, gemma4:12b 7.7–8.0GB) are download sizes listed on the Ollama library page, not measured VRAM peaks.
Local observations. Only two facts here come from a real machine, a 16 GiB Apple M4 Mac with llama.cpp build 11429: the command-line flags in the commands below exist in that build, and llama.cpp reports a 12,124 MiB GPU budget on it. No model was downloaded or run, so nothing here is a speed, quality or fit measurement. Every memory total in the tables is arithmetic from the model's config.json and the file size, and says so.
Forum threads and benchmark posts still show what people are trying. Treat a "fits in 16GB" claim as useful only when it names the exact file, the runtime, the context length and the KV cache type.
What 16GB VRAM Actually Buys You
VRAM is the dedicated memory on your GPU. It is not system RAM, and on a Mac it is a slice of unified memory, which is a different budget covered in the next section. On a 16GB card, three things compete for that memory:
| Memory pressure | What it means for coding | How to estimate it |
|---|---|---|
| Model weights | The quantized file loaded onto the GPU | Roughly the GGUF file size; a vision projector (mmproj) adds up to about 1 GB if you load it |
| KV cache | Memory that holds the prompt, the files you pasted and the model's answer | Bytes per token from the model config × context length |
| Runtime and offload | Compute buffers, the desktop, and anything moved to system RAM | Not in the totals below; offload avoids out-of-memory errors but slows each token |
The KV cache is where 16GB surprises people, because it grows with every token of context and differs a lot between models:
| Model | KV cache per token (F16) | At 32K | At 64K |
|---|---|---|---|
| Gemma 4 12B | 16 KiB on 8 global layers, plus a fixed ≈320 MiB sliding-window cache | ≈0.8 GiB | ≈1.3 GiB |
| Qwen3.6 35B A3B | 20 KiB | 0.63 GiB | 1.25 GiB |
| gpt-oss-20b | 24 KiB on 12 full-attention layers (sliding layers add ≈3 MiB) | 0.75 GiB | 1.5 GiB |
| Qwen3.8-27B | 64 KiB on 16 full-attention layers | 2 GiB (1.06 GiB as q8_0) | 4 GiB (2.13 GiB as q8_0) |
| Devstral Small 2 24B | 160 KiB on all 40 layers | 5 GiB (≈2.65 GiB as q8_0) | 10 GiB |
These come from each config (layers × KV heads × head size × 2 for K and V × 2 bytes). A q8_0 cache takes about 0.53× the F16 size. Qwen3.8-27B also keeps a fixed recurrent state for its 48 linear-attention layers, which these numbers do not count.
Put together, this is what each candidate file needs at common coding contexts:
| File | Weights | + 32K context | + 64K context | On a 16GB card |
|---|---|---|---|---|
| Gemma 4 12B QAT Q4_0 | 6.50 GiB | ≈7.3 GiB | ≈7.8 GiB (≈8.8 GiB at 128K) | Fits with lots of room |
| gpt-oss-20b MXFP4 | 11.28 GiB | ≈12.0 GiB | ≈12.8 GiB | Fits |
| Qwen3.8-27B UD-IQ3_XXS | 10.18 GiB | ≈11.2 GiB (q8_0) | ≈12.3 GiB (q8_0) | Fits; the lowest-quality Qwen3.8 file here |
| Qwen3.8-27B UD-Q3_K_XL | 12.24 GiB | ≈13.3 GiB (q8_0) | ≈14.4 GiB (q8_0) | Fits with a q8_0 cache |
| Qwen3.8-27B UD-IQ4_XS | 13.27 GiB | ≈14.3 GiB (q8_0) | ≈15.4 GiB (q8_0) | 32K only |
| Qwen3.8-27B UD-Q4_K_M | 15.33 GiB | — | — | Weights alone fill the card |
| Qwen3.6 35B A3B UD-IQ3_XXS | 12.30 GiB | ≈12.9 GiB | ≈13.6 GiB (≈14.8 GiB at 128K) | Fits, but it squeezes a 35B model into 3 bits |
| Devstral Small 2 Q3_K_M | 10.69 GiB | ≈13.4 GiB (q8_0) | ≈16.0 GiB (q8_0) | 32K only |
| Devstral Small 2 Q4_K_M | 13.35 GiB | ≈18.4 GiB (F16) | — | Does not fit at 32K; ≈14.6 GiB even at 8K |
Read "fits" as "the arithmetic leaves room," not as a tested result. The closer a total gets to 16, the less room remains for compute buffers and your display, so anything in the 15s is a gamble.

The bottom row of this chart still names the July 2026 picks; the per-model numbers above replace it.
16GB RAM or a 16GB Mac: What Fits in the 12,124 MiB Metal Budget
If you have 16GB of system RAM rather than a 16GB graphics card, the tables above do not apply directly.
On a 16 GiB Apple M4 Mac, llama-cli --list-devices reports the GPU as MTL0: Apple M4 (12124 MiB, 12123 MiB free). That is about 11.8 GiB that llama.cpp can put on the GPU by default; macOS keeps the rest. That is the default limit; raising macOS's GPU wired-memory limit changes it, and the numbers below do not assume that. Against the default budget:
- Gemma 4 12B QAT Q4_0 (6.50 GiB) fits with about 4 GiB left at 64K context. This is the model to start with on a 16GB Mac.
- Gemma 4 E4B QAT Q4_0 (5.15 GB file, 128K context) is lighter still, with more room for other apps.
- gpt-oss-20b MXFP4 (11.28 GiB) leaves about 0.56 GiB, which is less than its 32K cache alone (0.75 GiB). It is a 16GB-VRAM model, not a 16GB-Mac model.
- Qwen3.8-27B is out. Even its smallest 3-bit file,
UD-IQ3_XXS, leaves only about 0.6 GiB after a 32K q8_0 cache, before the recurrent state and compute buffers. That is too thin to plan around.
These are budget arithmetic; no model was run on that Mac, so speed is unknown.
On a Windows or Linux PC with 16GB of RAM and no large GPU, the model runs on the CPU and shares memory with the operating system, your editor and the browser. Stay with Gemma 4 E4B or Gemma 4 12B in their QAT Q4_0 files, and expect much slower answers than on a GPU.
Route 1: Use gpt-oss-20b as the Safe Default
Use gpt-oss-20b first. It is not the strongest coder on paper, but it is the one model here whose maker states it runs within 16GB of memory, its MXFP4 format is OpenAI's own rather than a community 3-bit squeeze, and its KV cache is small: 64K context adds only about 1.5 GiB.
Start here if:
- you have a 16GB card such as an RTX 4060 Ti 16GB, RTX 5060 Ti 16GB, RTX 5070 Ti, RTX 4070 Ti Super, or RTX 4080/5080
- you want local, private help for focused tasks: explaining code, writing a function, reviewing a diff
- you want a baseline before spending an evening tuning Qwen3.8-27B or Devstral
With Ollama, set the context before you judge the model. Ollama starts at 4K on cards with less than 24 GiB of VRAM, which is too short for a coding tool to see more than a file or two:
OLLAMA_CONTEXT_LENGTH=64000 ollama serveThen, in a second terminal:
ollama pull gpt-oss:20b
ollama run gpt-oss:20b
ollama psThe Ollama package is listed at 14GB with a 128K window. In ollama ps, the PROCESSOR column should read 100% GPU. A CPU/GPU split means part of the model spilled into system RAM, and every token gets slower.
With llama.cpp, download gpt-oss-20b-MXFP4.gguf from the ggml-org gpt-oss-20b GGUF repository and point at the file:
llama-server -m ./gpt-oss-20b-MXFP4.gguf -c 65536 -ngl all -fa on-c sets the context, -ngl all keeps every layer on the GPU, and -fa on turns on flash attention.
Then give it a real task, not a riddle:
You are helping with this repository slice.
Explain what this function does, identify one safe refactor,
show the patch, and name the test that should be updated.The catch is the capability ceiling. Qwen3.8-27B and Devstral Small 2 are larger and built for agentic coding, so gpt-oss-20b can lose to them on deeper multi-file work. Treat it as the baseline you compare against, not a permanent winner.
Route 2: Try Qwen3.6 35B A3B Only as an Ambitious Route
The first Qwen to try on 16GB is now Qwen3.8-27B, not Qwen3.6 35B A3B. Qwen3.8-27B is a dense 27B model released August 5, 2026, with 262K native context and an Apache-2.0 license. Qwen's model card scores it ahead of Qwen3.6-27B on every coding benchmark it lists:
| Benchmark (Qwen's table) | Qwen3.8-27B | Qwen3.6-27B |
|---|---|---|
| Terminal Bench 2.1 | 73.0 | 63.4 |
| SWE-bench Pro | 61.7 | 53.5 |
| LiveCodeBench v6 | 90.3 | 83.9 |
| QwenSWEBench | 79.0 | 49.3 |
Those were run at full precision with a 256K window. On a 16GB card you will run a 3-bit file at 32K–64K, so expect some of that lead to shrink. How much is not documented.
Which Qwen3.8-27B file fits a 16GB card
The default Ollama tag qwen3.8:27b is a Q4_K_M package listed at 18GB, and the matching Unsloth UD-Q4_K_M GGUF is 16.46 GB. Neither fits. The files that do, from the Unsloth Qwen3.8-27B GGUF repository:
UD-Q3_K_XL, 13.15 GB: the balanced choice; about 13.3 GiB at 32K and 14.4 GiB at 64K with a q8_0 cache.UD-IQ4_XS, 14.25 GB: higher quality, but stick to 32K (about 14.3 GiB).UD-IQ3_XXS, 10.93 GB: room for 64K at about 12.3 GiB, at the cost of quality.
Skip the 0.93 GB vision projector unless you need image input.
With llama.cpp, name the quant explicitly. Its help text says -hf defaults to Q4_K_M when you leave the quant off, which is exactly the file that does not fit:
llama-server -hf unsloth/Qwen3.8-27B-GGUF:UD-Q3_K_XL --no-mmproj -c 32768 -ngl all -fa on -ctk q8_0 -ctv q8_0-ctk q8_0 -ctv q8_0 stores the KV cache at 8 bits, which roughly halves it. Raise -c to 65536 only after the 32K run stays fully on the GPU.
With Ollama, pull the same file straight from Hugging Face:
ollama run hf.co/unsloth/Qwen3.8-27B-GGUF:UD-Q3_K_XLWith an F16 cache, the 32K arithmetic is about 14.2 GiB, and 64K would need about 16.2 GiB. So in Ollama, keep Qwen3.8-27B at 32K (OLLAMA_CONTEXT_LENGTH=32768), or use llama.cpp's q8_0 cache for 64K.
Qwen3.6-27B and Ollama's qwen3.6:27b-coding package (19GB) sit in the same size class. Qwen's own table puts 3.8-27B ahead, so pick 3.8 unless a tool you use is pinned to 3.6. For 32GB cards and two-GPU setups, the Qwen3.8-27B VRAM guide has the full budgets up to 262K.
Where Qwen3.6 35B A3B still fits
Qwen3.6 35B A3B (April 15, 2026) is a mixture-of-experts model: 35B total parameters, 3B active per token. The "A3B" describes compute per token, not memory, so all 35B parameters still need a place to live. Its UD-Q4_K_M GGUF is 22.13 GB and Ollama's qwen3.6:35b is 23–24GB, so neither fits on a 16GB card alone.
It still has two uses:
- Low-bit on the GPU.
UD-IQ3_XXS(13.21 GB) fits at 64K for about 13.6 GiB, or at the 128K that Qwen's card recommends for thinking mode for about 14.8 GiB. That is a heavy squeeze for a 35B model. - Experts in system RAM. If your PC has plenty of system RAM, keep the 4-bit file and move expert weights off the card with llama.cpp's
--n-cpu-moe N, which keeps the MoE weights of the first N of the model's 40 layers on the CPU:
llama-server -hf unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_M --no-mmproj -c 131072 -ngl all -fa on --n-cpu-moe 20The 20 is only a starting value, not a tested setting. Raise N if the model does not load, and lower it while the GPU still has free memory. Expect it to be slower than a model that sits fully on the GPU.
Choose this when you want Qwen's long thinking context and have the system RAM for it. Otherwise Qwen3.8-27B UD-Q3_K_XL is the simpler Qwen on 16GB. The older Qwen3-30B-A3B and Qwen3-Coder-30B-A3B packages are compared in the Qwen3-30B-A3B version guide.
Route 3: Keep Gemma 4 E4B and Smaller Coders Ready
On a 16GB card, this lane is now led by Gemma 4 12B, and Gemma 4 E4B moves down to 8GB cards and laptops.
Gemma 4 12B (11.95B parameters, 256K context, Apache-2.0) is the size Google aims at consumer GPUs. Google's own QAT Q4_0 file is 6.98 GB, and it was quantized during training rather than afterwards. By calculation it needs about 7.8 GiB at 64K and about 8.8 GiB at 128K, so it is the only strong candidate that can hold a large slice of a repository on 16GB. In Google's benchmark table, it clearly beats E4B on code:
| Google's table | Gemma 4 12B | Gemma 4 E4B |
|---|---|---|
| LiveCodeBench v6 | 72.0% | 52.0% |
| Codeforces ELO | 1659 | 940 |
Neither number measures multi-file agent work, so run the smoke test before you trust it with patches.
ollama pull gemma4:12b
ollama run gemma4:12bThe gemma4:12b package is listed at 7.7–8.0GB with a 256K window. Ollama will still start it at 4K on a 16GB card, so set OLLAMA_CONTEXT_LENGTH as in Route 1. You can go well past 64K with this model.
Use Gemma 4 12B when:
- the task needs many files in context at once, such as reviewing a module or tracing a bug across layers
- Qwen3.8-27B loads but runs too slowly or forgets earlier files
- you share the GPU with other work, or run on a 16GB Mac
Keep Gemma 4 E4B (gemma4:e4b, 6.6–9.5GB; QAT Q4_0 file 5.15 GB, 128K context) for 8GB cards, laptops and quick single-file questions. IBM's Granite 4.2 8B (Q4_K_M GGUF 5.35 GB) is another small option released August 25, 2026.
The older code families still run: DeepSeek Coder has 1B–33B versions, and Qwen2.5-Coder has several sizes. Newer models have overtaken them, so reach for them only if a tool you use is tuned for them.
The practical rule still holds: a smaller model that edits correctly in 12 seconds beats a larger one that needs two minutes, drops context, or fails halfway through a patch.
Devstral Small 2 on 16GB VRAM: Agentic Coder, Short Context
Devstral Small 2 (24B, Apache-2.0, 256K context) is Mistral's model built for agentic coding tools, and Mistral reports 68.0% on SWE-bench Verified. That makes it the natural "best agentic model for 16GB VRAM" candidate, with one large catch: its KV cache costs 160 KiB per token, more than six times gpt-oss-20b's.
From the Unsloth Devstral Small 2 GGUF repository:
Q4_K_M(14.33 GB, the same as Ollama's 15GBdevstral-small-2:24b) plus a 32K F16 cache comes to about 18.4 GiB. It does not fit.IQ4_XS(12.78 GB) fits with about 16K of F16 cache, at about 14.4 GiB.Q3_K_M(11.47 GB) fits 32K with a q8_0 cache, at about 13.4 GiB.
llama-server -hf unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:Q3_K_M --no-mmproj -c 32768 -ngl all -fa on -ctk q8_0 -ctv q8_0Devstral cannot reach the 64K that Ollama recommends for coding tools on 16GB, and Mistral itself names a 24GB RTX 4090 or a 32GB Mac as the target hardware. Compared with gpt-oss-20b, you trade context length for a model trained for agent loops. If your agent sessions stay short, try Devstral. If they read many files, stay with gpt-oss-20b or Gemma 4 12B, or move to a 24GB card.
GLM-5.3-Flash, Qwen3.8-Flash-Next, Qwen3-Coder-Next: Not 16GB Models
Some newer names sound small but are not:
| Model | Size | What it means on 16GB |
|---|---|---|
| GLM-5.3-Flash | Official FP8 weights about 305.8 GiB | Use the API; see the GLM-5.3-Flash guide |
| Qwen3.8-Flash-Next | 125B sparse MoE (6B active); BF16 repo about 360 GB | Multi-GPU or a large-memory workstation; see Qwen3.8 Flash-Next vs 27B |
| Qwen3-Coder-Next | Q4_K_M GGUF is four files totaling 48.4 GB | Needs about three times a 16GB card even at 4 bits |
"Flash" and "6B active" describe speed and compute per token, not memory. Every parameter still has to be stored somewhere. For a 16GB card, the Qwen3.8 model to use is the 27B.
Runtime Path: Ollama, LM Studio, llama.cpp, or a Coding Wrapper
The runtime changes the answer as much as the model, because it decides the file, the context length and whether anything spills to system RAM.
| Runtime | Best use | First check | Watch out for |
|---|---|---|---|
| Ollama | Fastest baseline with one command | ollama ps PROCESSOR column; context set with OLLAMA_CONTEXT_LENGTH | 4K default context below 24 GiB of VRAM; default tags may be Q4_K_M files that do not fit |
| LM Studio | GUI model browsing and a local server | The exact quant file you select, and its context setting | Picking the family name instead of the 3-bit or QAT file |
| llama.cpp | Full control of file, context, cache type and MoE offload | -hf user/repo:QUANT, -c, -ctk/-ctv, -ngl | -hf without a quant picks Q4_K_M |
| Coding agent or IDE plugin | Daily use inside your editor | Which local endpoint it calls and how much context it sends | The tool can send more context than the server allows, and the model then silently loses files |
The Ollama 4K trap. Ollama sets the default context by VRAM: 4K below 24 GiB, 32K for 24–48 GiB, 256K at 48 GiB and above. Its docs recommend at least 64,000 tokens for agents and coding tools. A 16GB card therefore starts every model at 4K, which is a common reason a local coding agent "forgets" the file it read a minute ago. Set OLLAMA_CONTEXT_LENGTH=64000 (or the context slider in the Ollama app), then confirm with ollama ps that the model still shows 100% GPU.
For a file Ollama does not ship, such as a 3-bit Qwen3.8-27B, ollama run hf.co/{user}/{repo}:{quant} pulls it from Hugging Face. The quant name has to match the file name in the repository, for example UD-Q3_K_XL.
For llama.cpp, every flag in the commands above (-c, -ngl, -fa, -ctk, -ctv, --n-cpu-moe, -hf, --no-mmproj, -m) appears in llama-server --help for build 11429. If your build is much older, compare against its own --help first.
If the model loads but the agent still fails on real tasks, the cause is usually quantization, context, or the agent harness, and the local coding agent diagnosis guide separates those three.
Smoke Test: Prove the Model Works on Your Code

Step 5 in this chart still says Qwen3.6 low-bit from July 2026; today that step is Qwen3.8-27B UD-Q3_K_XL.
Do not make a local coding model your daily driver because it loads. Make it pass a small coding workflow on your own repository.
- Start with
gpt-oss-20bat 64K context. - Check
ollama ps(or the llama.cpp startup log) and confirm the model is fully on the GPU. - Load one real repo slice: one file, one nearby test, and a short task description.
- Ask for an explanation, a patch, and the test that should change.
- Watch VRAM, latency, and whether the model keeps the relevant context.
- Repeat with a larger slice that uses more of the context.
- Only if the safe default fails on quality, repeat the same test with Qwen3.8-27B
UD-Q3_K_XLat 32K with a q8_0 cache, or Devstral Small 2Q3_K_Mfor agent tasks. - If you need more files in context than those allow, repeat with Gemma 4 12B at 128K.
- Keep the model that edits correctly, explains the trade-off, and stays fast enough that you would actually use it.
Use the same prompt for every model:
Given the files below, refactor one function without changing behavior.
Explain the tradeoff, show the patch, and name the test that should be updated.
If the context is insufficient, say exactly what file or symbol you need next.Pass means four things:
- latency is tolerable enough that you would actually use it
- no out-of-memory error and no spill to system RAM on the first real task
- the model keeps the function, requirement and test straight
- the patch is small, reviewable and tied to the request
A failure does not mean the model is bad. It means this combination of model, file, context and machine is the wrong fit for that job.
Stop Rules: When 16GB Is the Wrong Constraint
Leave the 16GB local setup when the job needs more context than the card can hold. The usual failure is not "the model is dumb" but "the model cannot see enough of the project without slowing down or spilling."
| Symptom | Likely cause | Better choice |
|---|---|---|
| Model loads but every answer crawls | Part of the model spilled to system RAM, or MoE experts are offloaded | A file that fits fully (see the fit table), or more VRAM |
| Agent forgets files it just read | Context still at Ollama's 4K default, or the tool sends more than the server allows | Set 64K; if the model cannot fit 64K, switch to gpt-oss-20b or Gemma 4 12B |
| Out-of-memory error after raising context | KV cache outgrew the card (Devstral at 64K, Qwen3.8 IQ4_XS at 64K) | q8_0 cache, a smaller file, or a lower context |
| Good on snippets, bad on repo tasks | Context packing is the bottleneck | Narrow the task, or use a hosted coding agent |
| Tuning Qwen3.8 or Qwen3.6 takes longer than the work | The experiment has become the project | Return to gpt-oss-20b or Gemma 4 12B |
A 24GB card is the next comfort tier. By the same arithmetic, Qwen3.8-27B UD-Q4_K_M (15.33 GiB) plus a 64K F16 cache (4 GiB) comes to about 19.3 GiB, and Ollama starts 24–48 GiB cards at 32K instead of 4K. A 32GB card or two GPUs open up Qwen3.8-27B's long context. If you are still choosing hardware, the best 16GB GPU for local LLM coding guide compares the cards.
Hosted coding becomes the rational choice when the job is "get reliable multi-file edits" rather than "run locally at all costs." The guide to when local coding agents beat Claude and Codex works through that cost and quality trade.
FAQ
What is the best local coding LLM for 16GB VRAM?
As of October 22, 2026, start with gpt-oss-20b (MXFP4 GGUF, 12.11 GB, about 12.8 GiB at 64K context by calculation). For the strongest Qwen, try Qwen3.8-27B in the 13.15 GB UD-Q3_K_XL file with a q8_0 cache. For the longest context, use Gemma 4 12B in Google's 6.98 GB QAT Q4_0 file.
Can Qwen3.6 35B A3B run on 16GB VRAM?
Only in a 3-bit file or with experts offloaded. Its UD-Q4_K_M GGUF is 22.13 GB and Ollama's qwen3.6:35b is 23–24GB, so neither fits on the card alone. UD-IQ3_XXS (13.21 GB) fits at 64K for about 13.6 GiB by calculation. Alternatively, keep the 4-bit file and move experts to system RAM with llama.cpp's --n-cpu-moe, which is slower. Qwen3.8-27B has since replaced it as the first Qwen to try.
Is gpt-oss-20b good enough for coding?
It is the right baseline for 16GB because it fits with a 64K context and its memory claim comes from OpenAI. It handles focused edits, explanations and short repo slices well enough to judge with the smoke test. For deeper agentic work, Qwen3.8-27B and Devstral Small 2 are built for it, but on 16GB they need 3-bit files and shorter context.
Is Gemma 4 E4B a coding model?
It is a general model that can code, and Google targets it at phones and edge devices. Its LiveCodeBench v6 score in Google's table is 52.0%, against 72.0% for Gemma 4 12B. On a 16GB GPU use the 12B; keep E4B for 8GB cards, laptops and quick single-file questions.
Should I use Ollama or LM Studio?
Use Ollama for a fast command-line baseline and LM Studio for a GUI. Either way, choose the exact quant file and set the context yourself: Ollama starts 16GB cards at 4K context, and its default qwen3.8:27b tag is an 18GB file that does not fit. Use llama.cpp when you need a q8_0 KV cache or MoE expert offload.
Is 16GB VRAM enough for an RTX 4060 Ti or 5060 Ti?
Yes, for a useful local coding helper. gpt-oss-20b at 64K, Gemma 4 12B at 128K, and Qwen3.8-27B UD-Q3_K_XL at 32K all fit by calculation. It is not enough for comfortable long-context agent loops with 24–27B models. Those want a 24GB card.
What if I only have 8GB VRAM?
Use Gemma 4 E4B (QAT Q4_0 file 5.15 GB) or Granite 4.2 8B (Q4_K_M 5.35 GB). Gemma 4 12B's 6.98 GB file is already tight on 8GB once you add context. The 16GB picks (gpt-oss-20b, Qwen3.8-27B, Devstral) do not fit.
Is 24GB VRAM a better target?
Yes, if you want 4-bit 27B models with real context. Qwen3.8-27B UD-Q4_K_M plus a 64K F16 cache is about 19.3 GiB by calculation, Devstral Small 2 Q4_K_M at 32K is about 18.4 GiB, and Ollama starts 24GB cards at 32K context instead of 4K. You still need to check the file size, context and cache type.
How much context should I use on 16GB?
Aim for 64K, the minimum Ollama recommends for coding tools, when the model allows it: gpt-oss-20b (≈12.8 GiB) and Gemma 4 12B (≈7.8 GiB) do. Qwen3.8-27B reaches 64K only with a q8_0 cache in the UD-Q3_K_XL file. Devstral Small 2 stays at 16K–32K. Never leave Ollama at its 4K default for coding.
When should I stop using local coding LLMs?
Stop when memory tuning becomes the main job, when tasks need more files than 64K holds, or when offload makes each answer too slow to use. At that point, move to a smaller local model, a 24GB card, or hosted coding.
Can Qwen3.8 27B run on 16GB VRAM?
Yes, in a 3-bit or IQ4 file, not in the default 4-bit one. UD-Q3_K_XL (13.15 GB) comes to about 13.3 GiB at 32K and 14.4 GiB at 64K with a q8_0 cache, by calculation. Ollama's qwen3.8:27b (18GB) and the UD-Q4_K_M GGUF (16.46 GB) do not fit. Qwen's benchmark scores were measured at full precision, so test the 3-bit build on your own code.
What is the best local LLM for 16GB RAM or a 16GB Mac?
Gemma 4 12B in Google's QAT Q4_0 file. On a 16 GiB M4 Mac, llama.cpp reports a 12,124 MiB GPU budget; Gemma 4 12B needs about 7.8 GiB at 64K by calculation, while gpt-oss-20b's 11.28 GiB of weights leaves almost nothing for context. On a PC with 16GB of RAM and no large GPU, use Gemma 4 E4B or 12B on the CPU.
Can I run GLM-5.3-Flash locally on 16GB?
No. Its official FP8 weights are about 305.8 GiB, so on 16GB it is an API model. The same goes for Qwen3.8-Flash-Next (about 360 GB in BF16) and Qwen3-Coder-Next (48.4 GB even at Q4_K_M).
What is the best Ollama coding model for 16GB VRAM?
gpt-oss:20b with OLLAMA_CONTEXT_LENGTH=64000, or gemma4:12b if you need more context. For Qwen3.8-27B, skip the 18GB default tag and run ollama run hf.co/unsloth/Qwen3.8-27B-GGUF:UD-Q3_K_XL at 32K.
Sources9
External pages this guide links to, in the order they appear. Last updated Oct 22, 2026.
Sources9
External pages this guide links to, in the order they appear. Last updated Oct 22, 2026.
- 1.OpenAI gpt-oss-20b model card on Hugging Facehuggingface.co/openai/gpt-oss-20b
- 2.Gemma 4 12B model cardhuggingface.co/google/gemma-4-12B-it
- 3.Qwen3.8-27B model cardhuggingface.co/Qwen/Qwen3.8-27B
- 4.Devstral Small 2 model cardhuggingface.co/mistralai/Devstral-Small-2-24B-Instruct-2512
- 5.ggml-org gpt-oss-20b GGUF repositoryhuggingface.co/ggml-org/gpt-oss-20b-GGUF
- 6.Unsloth Qwen3.8-27B GGUF repositoryhuggingface.co/unsloth/Qwen3.8-27B-GGUF
- 7.DeepSeek Coderdeepseekcoder.github.io
- 8.Qwen2.5-Coderqwenlm.github.io/blog/qwen2.5-coder-family
- 9.Unsloth Devstral Small 2 GGUF repositoryhuggingface.co/unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF





