Skip to main content

Best Local Coding LLM for 16GB VRAM: gpt-oss, Qwen3.8, Gemma 4

For a 16GB VRAM GPU, start with gpt-oss-20b. Try Qwen3.8-27B only as a 3-bit file with a q8_0 cache, and use Gemma 4 12B when you need long context.

LaoZhang AI TeamPublishedUpdated 25 min read
On this page
July 2026 route board for 16GB VRAM local coding LLMs showing gpt-oss-20b as the safe default, Qwen3.6 low-bit, Gemma 4 E4B, and a stop rule

As of October 22, 2026, the best local coding LLM for a 16GB VRAM GPU is still gpt-oss-20b as the safe default: it is the only candidate whose maker says it runs within 16GB of memory, and its 12.11 GB MXFP4 file plus a 64K context comes to about 12.8 GiB by calculation. The picks around it changed since July. Qwen3.8-27B (released August 5) replaced Qwen3.6 35B A3B as the first Qwen to try, but only as a 3-bit file such as UD-Q3_K_XL with a quantized cache. Gemma 4 12B replaced Gemma 4 E4B as the fit-first model on a 16GB card, with room for 128K context. Devstral Small 2 is the agentic-coding specialist, but only at short context on 16GB.

Two settings decide more than the model name:

  • Pick an exact file, not a family. qwen3.8:27b in Ollama is an 18GB Q4_K_M package and does not fit a 16GB card; the 13.15 GB UD-Q3_K_XL GGUF does.
  • Raise the context. Ollama starts GPUs with less than 24 GiB of VRAM at a 4K context, while its own docs recommend at least 64,000 tokens for coding tools.

GLM-5.3-Flash, Qwen3.8-Flash-Next and Qwen3-Coder-Next are not 16GB models at all: the first two ship hundreds of gigabytes of weights, and Qwen3-Coder-Next is 48.4 GB even at Q4_K_M.

Quick Answer: Pick the Route First

This board shows the July 2026 picks (Qwen3.6 low-bit, Gemma 4 E4B); the table below has the current ones.

Pick by what you need the model to hold, then pick the file. Sizes are Hugging Face file sizes (GB = 10^9 bytes); the totals are weights plus KV cache in GiB (2^30 bytes), calculated from each model's config, before runtime buffers and whatever your desktop already uses on the card.

Your situationModel and exact fileWeightsEstimated totalMain caveat
Safe default, first installgpt-oss-20b, MXFP4 GGUF12.11 GB file (11.28 GiB)≈12.0 GiB at 32K, ≈12.8 GiB at 64KSmaller than the 24–27B models; no vendor coding scores comparable to Qwen's
Strongest Qwen that fitsQwen3.8-27B, UD-Q3_K_XL GGUF (Unsloth)13.15 GB file (12.24 GiB)≈13.3 GiB at 32K, ≈14.4 GiB at 64K, both with a q8_0 cache3-bit community quant; Qwen's scores were measured at full precision
Long context with room to spareGemma 4 12B, Google's QAT Q4_0 GGUF6.98 GB file (6.50 GiB)≈7.8 GiB at 64K, ≈8.8 GiB at 128KCoding scores are below the 27B class
Agentic coding specialistDevstral Small 2 24B, Q3_K_M GGUF11.47 GB file (10.69 GiB)≈13.4 GiB at 32K with a q8_0 cacheHeavy KV cache; 64K does not fit
8GB card, laptop, 16GB MacGemma 4 E4B or Gemma 4 12B QAT Q4_05.15 GB / 6.98 GB filessee the 16GB RAM sectionMac budget is about 11.8 GiB, not 16
Repo-scale agent loopsMore VRAM or hosted coding——16GB runs out of context before it runs out of model

If you want one answer: install gpt-oss-20b, set the context to 64K, and run the smoke test further down. If it fails on quality and your card still shows headroom, try Qwen3.8-27B UD-Q3_K_XL. If you need a whole module in context, use Gemma 4 12B.

What Changed Since July: Qwen3.8-27B, Gemma 4 12B, Devstral Small 2

If you saw a July 2026 list built around Qwen3.6 35B A3B, Gemma 4 E4B and older code models, here is what each pick maps to now.

July pickCurrent pickWhy it changed
gpt-oss-20b (safe default)gpt-oss-20b, unchangedOpenAI has not released a newer small gpt-oss model; it is still the only candidate with a first-party "within 16GB of memory" statement
Qwen3.6 35B A3B, low-bit or offloadQwen3.8-27B in a 3-bit or IQ4 fileReleased August 5, 2026; Qwen's own table puts it ahead of Qwen3.6-27B on every coding row. Qwen3.6 35B A3B stays as the offload option for machines with lots of system RAM
Gemma 4 E4B (fit-first fallback)Gemma 4 12BThe 12B size appeared on May 23, 2026. Google aims it at consumer GPUs, and it scores 72.0% vs 52.0% for E4B on LiveCodeBench v6. E4B stays for 8GB cards and laptops
Qwen2.5-Coder, DeepSeek Coder, Qwen3-Coder-30B-A3BDevstral Small 2 24B, or Qwen3.6-27B's qwen3.6:27b-coding package on bigger cardsDevstral Small 2 is built for agentic coding and reports 68.0% on SWE-bench Verified. The older coder families still run but are no longer the strongest open options

Other open models with 3-bit or QAT files that land near the 16GB line: GLM-4.7-Flash (UD-Q3_K_XL 13.78 GB, Q3_K_M 14.61 GB), Gemma 4 26B A4B (Google's QAT Q4_0 file 14.44 GB, plus a 1.19 GB vision projector), and IBM Granite 4.2 30B (Q3_K_M 14.13 GB). They leave less room for context than the picks above, so they are worth trying only after those.

Evidence Boundary: Official, Runtime, Community

Each claim on this page belongs to one of four kinds of source, and they carry different weight.

First-party model cards. The OpenAI gpt-oss-20b model card on Hugging Face describes a 21B-parameter model with 3.6B active parameters, MoE weights post-trained in MXFP4, and says it runs "within 16GB of memory." The Gemma 4 12B model card says the 12B, 26B A4B and 31B sizes target consumer GPUs and workstations, while E2B and E4B target mobile and edge devices. The Qwen3.8-27B model card positions the model for coding and long-horizon agentic tasks but makes no 16GB claim. The Devstral Small 2 model card says it is "light enough to run on a single RTX 4090 or a Mac with 32GB RAM", which is a 24GB card, not a 16GB one.

Vendor benchmark scores. Qwen's coding scores for Qwen3.8-27B were measured at full precision with a 256K context window. A 3-bit file at 32K–64K context is a different setup, so the score is a reason to try the model, not a promise about your build. Google's Gemma scores come from LiveCodeBench and Codeforces, not repo-level agent tasks.

Quantized files and runtime packages. The 3-bit and IQ4 Qwen and Devstral files come from Unsloth, a community quantizer, not from the model makers. Gemma's QAT Q4_0 files are Google's own. Ollama package sizes (gpt-oss:20b 14GB, qwen3.8:27b 18GB, gemma4:12b 7.7–8.0GB) are download sizes listed on the Ollama library page, not measured VRAM peaks.

Local observations. Only two facts here come from a real machine, a 16 GiB Apple M4 Mac with llama.cpp build 11429: the command-line flags in the commands below exist in that build, and llama.cpp reports a 12,124 MiB GPU budget on it. No model was downloaded or run, so nothing here is a speed, quality or fit measurement. Every memory total in the tables is arithmetic from the model's config.json and the file size, and says so.

Forum threads and benchmark posts still show what people are trying. Treat a "fits in 16GB" claim as useful only when it names the exact file, the runtime, the context length and the KV cache type.

What 16GB VRAM Actually Buys You

VRAM is the dedicated memory on your GPU. It is not system RAM, and on a Mac it is a slice of unified memory, which is a different budget covered in the next section. On a 16GB card, three things compete for that memory:

Memory pressureWhat it means for codingHow to estimate it
Model weightsThe quantized file loaded onto the GPURoughly the GGUF file size; a vision projector (mmproj) adds up to about 1 GB if you load it
KV cacheMemory that holds the prompt, the files you pasted and the model's answerBytes per token from the model config × context length
Runtime and offloadCompute buffers, the desktop, and anything moved to system RAMNot in the totals below; offload avoids out-of-memory errors but slows each token

The KV cache is where 16GB surprises people, because it grows with every token of context and differs a lot between models:

ModelKV cache per token (F16)At 32KAt 64K
Gemma 4 12B16 KiB on 8 global layers, plus a fixed ≈320 MiB sliding-window cache≈0.8 GiB≈1.3 GiB
Qwen3.6 35B A3B20 KiB0.63 GiB1.25 GiB
gpt-oss-20b24 KiB on 12 full-attention layers (sliding layers add ≈3 MiB)0.75 GiB1.5 GiB
Qwen3.8-27B64 KiB on 16 full-attention layers2 GiB (1.06 GiB as q8_0)4 GiB (2.13 GiB as q8_0)
Devstral Small 2 24B160 KiB on all 40 layers5 GiB (≈2.65 GiB as q8_0)10 GiB

These come from each config (layers × KV heads × head size × 2 for K and V × 2 bytes). A q8_0 cache takes about 0.53× the F16 size. Qwen3.8-27B also keeps a fixed recurrent state for its 48 linear-attention layers, which these numbers do not count.

Put together, this is what each candidate file needs at common coding contexts:

FileWeights+ 32K context+ 64K contextOn a 16GB card
Gemma 4 12B QAT Q4_06.50 GiB≈7.3 GiB≈7.8 GiB (≈8.8 GiB at 128K)Fits with lots of room
gpt-oss-20b MXFP411.28 GiB≈12.0 GiB≈12.8 GiBFits
Qwen3.8-27B UD-IQ3_XXS10.18 GiB≈11.2 GiB (q8_0)≈12.3 GiB (q8_0)Fits; the lowest-quality Qwen3.8 file here
Qwen3.8-27B UD-Q3_K_XL12.24 GiB≈13.3 GiB (q8_0)≈14.4 GiB (q8_0)Fits with a q8_0 cache
Qwen3.8-27B UD-IQ4_XS13.27 GiB≈14.3 GiB (q8_0)≈15.4 GiB (q8_0)32K only
Qwen3.8-27B UD-Q4_K_M15.33 GiB——Weights alone fill the card
Qwen3.6 35B A3B UD-IQ3_XXS12.30 GiB≈12.9 GiB≈13.6 GiB (≈14.8 GiB at 128K)Fits, but it squeezes a 35B model into 3 bits
Devstral Small 2 Q3_K_M10.69 GiB≈13.4 GiB (q8_0)≈16.0 GiB (q8_0)32K only
Devstral Small 2 Q4_K_M13.35 GiB≈18.4 GiB (F16)—Does not fit at 32K; ≈14.6 GiB even at 8K

Read "fits" as "the arithmetic leaves room," not as a tested result. The closer a total gets to 16, the less room remains for compute buffers and your display, so anything in the 15s is a gamble.

Memory and context budget for 16GB VRAM local coding LLMs

The bottom row of this chart still names the July 2026 picks; the per-model numbers above replace it.

16GB RAM or a 16GB Mac: What Fits in the 12,124 MiB Metal Budget

If you have 16GB of system RAM rather than a 16GB graphics card, the tables above do not apply directly.

On a 16 GiB Apple M4 Mac, llama-cli --list-devices reports the GPU as MTL0: Apple M4 (12124 MiB, 12123 MiB free). That is about 11.8 GiB that llama.cpp can put on the GPU by default; macOS keeps the rest. That is the default limit; raising macOS's GPU wired-memory limit changes it, and the numbers below do not assume that. Against the default budget:

  • Gemma 4 12B QAT Q4_0 (6.50 GiB) fits with about 4 GiB left at 64K context. This is the model to start with on a 16GB Mac.
  • Gemma 4 E4B QAT Q4_0 (5.15 GB file, 128K context) is lighter still, with more room for other apps.
  • gpt-oss-20b MXFP4 (11.28 GiB) leaves about 0.56 GiB, which is less than its 32K cache alone (0.75 GiB). It is a 16GB-VRAM model, not a 16GB-Mac model.
  • Qwen3.8-27B is out. Even its smallest 3-bit file, UD-IQ3_XXS, leaves only about 0.6 GiB after a 32K q8_0 cache, before the recurrent state and compute buffers. That is too thin to plan around.

These are budget arithmetic; no model was run on that Mac, so speed is unknown.

On a Windows or Linux PC with 16GB of RAM and no large GPU, the model runs on the CPU and shares memory with the operating system, your editor and the browser. Stay with Gemma 4 E4B or Gemma 4 12B in their QAT Q4_0 files, and expect much slower answers than on a GPU.

Route 1: Use gpt-oss-20b as the Safe Default

Use gpt-oss-20b first. It is not the strongest coder on paper, but it is the one model here whose maker states it runs within 16GB of memory, its MXFP4 format is OpenAI's own rather than a community 3-bit squeeze, and its KV cache is small: 64K context adds only about 1.5 GiB.

Start here if:

  • you have a 16GB card such as an RTX 4060 Ti 16GB, RTX 5060 Ti 16GB, RTX 5070 Ti, RTX 4070 Ti Super, or RTX 4080/5080
  • you want local, private help for focused tasks: explaining code, writing a function, reviewing a diff
  • you want a baseline before spending an evening tuning Qwen3.8-27B or Devstral

With Ollama, set the context before you judge the model. Ollama starts at 4K on cards with less than 24 GiB of VRAM, which is too short for a coding tool to see more than a file or two:

bash
OLLAMA_CONTEXT_LENGTH=64000 ollama serve

Then, in a second terminal:

bash
ollama pull gpt-oss:20b
ollama run gpt-oss:20b
ollama ps

The Ollama package is listed at 14GB with a 128K window. In ollama ps, the PROCESSOR column should read 100% GPU. A CPU/GPU split means part of the model spilled into system RAM, and every token gets slower.

With llama.cpp, download gpt-oss-20b-MXFP4.gguf from the ggml-org gpt-oss-20b GGUF repository and point at the file:

bash
llama-server -m ./gpt-oss-20b-MXFP4.gguf -c 65536 -ngl all -fa on

-c sets the context, -ngl all keeps every layer on the GPU, and -fa on turns on flash attention.

Then give it a real task, not a riddle:

You are helping with this repository slice.
Explain what this function does, identify one safe refactor,
show the patch, and name the test that should be updated.

The catch is the capability ceiling. Qwen3.8-27B and Devstral Small 2 are larger and built for agentic coding, so gpt-oss-20b can lose to them on deeper multi-file work. Treat it as the baseline you compare against, not a permanent winner.

Route 2: Try Qwen3.6 35B A3B Only as an Ambitious Route

The first Qwen to try on 16GB is now Qwen3.8-27B, not Qwen3.6 35B A3B. Qwen3.8-27B is a dense 27B model released August 5, 2026, with 262K native context and an Apache-2.0 license. Qwen's model card scores it ahead of Qwen3.6-27B on every coding benchmark it lists:

Benchmark (Qwen's table)Qwen3.8-27BQwen3.6-27B
Terminal Bench 2.173.063.4
SWE-bench Pro61.753.5
LiveCodeBench v690.383.9
QwenSWEBench79.049.3

Those were run at full precision with a 256K window. On a 16GB card you will run a 3-bit file at 32K–64K, so expect some of that lead to shrink. How much is not documented.

Which Qwen3.8-27B file fits a 16GB card

The default Ollama tag qwen3.8:27b is a Q4_K_M package listed at 18GB, and the matching Unsloth UD-Q4_K_M GGUF is 16.46 GB. Neither fits. The files that do, from the Unsloth Qwen3.8-27B GGUF repository:

  • UD-Q3_K_XL, 13.15 GB: the balanced choice; about 13.3 GiB at 32K and 14.4 GiB at 64K with a q8_0 cache.
  • UD-IQ4_XS, 14.25 GB: higher quality, but stick to 32K (about 14.3 GiB).
  • UD-IQ3_XXS, 10.93 GB: room for 64K at about 12.3 GiB, at the cost of quality.

Skip the 0.93 GB vision projector unless you need image input.

With llama.cpp, name the quant explicitly. Its help text says -hf defaults to Q4_K_M when you leave the quant off, which is exactly the file that does not fit:

bash
llama-server -hf unsloth/Qwen3.8-27B-GGUF:UD-Q3_K_XL --no-mmproj -c 32768 -ngl all -fa on -ctk q8_0 -ctv q8_0

-ctk q8_0 -ctv q8_0 stores the KV cache at 8 bits, which roughly halves it. Raise -c to 65536 only after the 32K run stays fully on the GPU.

With Ollama, pull the same file straight from Hugging Face:

bash
ollama run hf.co/unsloth/Qwen3.8-27B-GGUF:UD-Q3_K_XL

With an F16 cache, the 32K arithmetic is about 14.2 GiB, and 64K would need about 16.2 GiB. So in Ollama, keep Qwen3.8-27B at 32K (OLLAMA_CONTEXT_LENGTH=32768), or use llama.cpp's q8_0 cache for 64K.

Qwen3.6-27B and Ollama's qwen3.6:27b-coding package (19GB) sit in the same size class. Qwen's own table puts 3.8-27B ahead, so pick 3.8 unless a tool you use is pinned to 3.6. For 32GB cards and two-GPU setups, the Qwen3.8-27B VRAM guide has the full budgets up to 262K.

Where Qwen3.6 35B A3B still fits

Qwen3.6 35B A3B (April 15, 2026) is a mixture-of-experts model: 35B total parameters, 3B active per token. The "A3B" describes compute per token, not memory, so all 35B parameters still need a place to live. Its UD-Q4_K_M GGUF is 22.13 GB and Ollama's qwen3.6:35b is 23–24GB, so neither fits on a 16GB card alone.

It still has two uses:

  • Low-bit on the GPU. UD-IQ3_XXS (13.21 GB) fits at 64K for about 13.6 GiB, or at the 128K that Qwen's card recommends for thinking mode for about 14.8 GiB. That is a heavy squeeze for a 35B model.
  • Experts in system RAM. If your PC has plenty of system RAM, keep the 4-bit file and move expert weights off the card with llama.cpp's --n-cpu-moe N, which keeps the MoE weights of the first N of the model's 40 layers on the CPU:
bash
llama-server -hf unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_M --no-mmproj -c 131072 -ngl all -fa on --n-cpu-moe 20

The 20 is only a starting value, not a tested setting. Raise N if the model does not load, and lower it while the GPU still has free memory. Expect it to be slower than a model that sits fully on the GPU.

Choose this when you want Qwen's long thinking context and have the system RAM for it. Otherwise Qwen3.8-27B UD-Q3_K_XL is the simpler Qwen on 16GB. The older Qwen3-30B-A3B and Qwen3-Coder-30B-A3B packages are compared in the Qwen3-30B-A3B version guide.

Route 3: Keep Gemma 4 E4B and Smaller Coders Ready

On a 16GB card, this lane is now led by Gemma 4 12B, and Gemma 4 E4B moves down to 8GB cards and laptops.

Gemma 4 12B (11.95B parameters, 256K context, Apache-2.0) is the size Google aims at consumer GPUs. Google's own QAT Q4_0 file is 6.98 GB, and it was quantized during training rather than afterwards. By calculation it needs about 7.8 GiB at 64K and about 8.8 GiB at 128K, so it is the only strong candidate that can hold a large slice of a repository on 16GB. In Google's benchmark table, it clearly beats E4B on code:

Google's tableGemma 4 12BGemma 4 E4B
LiveCodeBench v672.0%52.0%
Codeforces ELO1659940

Neither number measures multi-file agent work, so run the smoke test before you trust it with patches.

bash
ollama pull gemma4:12b
ollama run gemma4:12b

The gemma4:12b package is listed at 7.7–8.0GB with a 256K window. Ollama will still start it at 4K on a 16GB card, so set OLLAMA_CONTEXT_LENGTH as in Route 1. You can go well past 64K with this model.

Use Gemma 4 12B when:

  • the task needs many files in context at once, such as reviewing a module or tracing a bug across layers
  • Qwen3.8-27B loads but runs too slowly or forgets earlier files
  • you share the GPU with other work, or run on a 16GB Mac

Keep Gemma 4 E4B (gemma4:e4b, 6.6–9.5GB; QAT Q4_0 file 5.15 GB, 128K context) for 8GB cards, laptops and quick single-file questions. IBM's Granite 4.2 8B (Q4_K_M GGUF 5.35 GB) is another small option released August 25, 2026.

The older code families still run: DeepSeek Coder has 1B–33B versions, and Qwen2.5-Coder has several sizes. Newer models have overtaken them, so reach for them only if a tool you use is tuned for them.

The practical rule still holds: a smaller model that edits correctly in 12 seconds beats a larger one that needs two minutes, drops context, or fails halfway through a patch.

Devstral Small 2 on 16GB VRAM: Agentic Coder, Short Context

Devstral Small 2 (24B, Apache-2.0, 256K context) is Mistral's model built for agentic coding tools, and Mistral reports 68.0% on SWE-bench Verified. That makes it the natural "best agentic model for 16GB VRAM" candidate, with one large catch: its KV cache costs 160 KiB per token, more than six times gpt-oss-20b's.

From the Unsloth Devstral Small 2 GGUF repository:

  • Q4_K_M (14.33 GB, the same as Ollama's 15GB devstral-small-2:24b) plus a 32K F16 cache comes to about 18.4 GiB. It does not fit.
  • IQ4_XS (12.78 GB) fits with about 16K of F16 cache, at about 14.4 GiB.
  • Q3_K_M (11.47 GB) fits 32K with a q8_0 cache, at about 13.4 GiB.
bash
llama-server -hf unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:Q3_K_M --no-mmproj -c 32768 -ngl all -fa on -ctk q8_0 -ctv q8_0

Devstral cannot reach the 64K that Ollama recommends for coding tools on 16GB, and Mistral itself names a 24GB RTX 4090 or a 32GB Mac as the target hardware. Compared with gpt-oss-20b, you trade context length for a model trained for agent loops. If your agent sessions stay short, try Devstral. If they read many files, stay with gpt-oss-20b or Gemma 4 12B, or move to a 24GB card.

GLM-5.3-Flash, Qwen3.8-Flash-Next, Qwen3-Coder-Next: Not 16GB Models

Some newer names sound small but are not:

ModelSizeWhat it means on 16GB
GLM-5.3-FlashOfficial FP8 weights about 305.8 GiBUse the API; see the GLM-5.3-Flash guide
Qwen3.8-Flash-Next125B sparse MoE (6B active); BF16 repo about 360 GBMulti-GPU or a large-memory workstation; see Qwen3.8 Flash-Next vs 27B
Qwen3-Coder-NextQ4_K_M GGUF is four files totaling 48.4 GBNeeds about three times a 16GB card even at 4 bits

"Flash" and "6B active" describe speed and compute per token, not memory. Every parameter still has to be stored somewhere. For a 16GB card, the Qwen3.8 model to use is the 27B.

Runtime Path: Ollama, LM Studio, llama.cpp, or a Coding Wrapper

The runtime changes the answer as much as the model, because it decides the file, the context length and whether anything spills to system RAM.

RuntimeBest useFirst checkWatch out for
OllamaFastest baseline with one commandollama ps PROCESSOR column; context set with OLLAMA_CONTEXT_LENGTH4K default context below 24 GiB of VRAM; default tags may be Q4_K_M files that do not fit
LM StudioGUI model browsing and a local serverThe exact quant file you select, and its context settingPicking the family name instead of the 3-bit or QAT file
llama.cppFull control of file, context, cache type and MoE offload-hf user/repo:QUANT, -c, -ctk/-ctv, -ngl-hf without a quant picks Q4_K_M
Coding agent or IDE pluginDaily use inside your editorWhich local endpoint it calls and how much context it sendsThe tool can send more context than the server allows, and the model then silently loses files

The Ollama 4K trap. Ollama sets the default context by VRAM: 4K below 24 GiB, 32K for 24–48 GiB, 256K at 48 GiB and above. Its docs recommend at least 64,000 tokens for agents and coding tools. A 16GB card therefore starts every model at 4K, which is a common reason a local coding agent "forgets" the file it read a minute ago. Set OLLAMA_CONTEXT_LENGTH=64000 (or the context slider in the Ollama app), then confirm with ollama ps that the model still shows 100% GPU.

For a file Ollama does not ship, such as a 3-bit Qwen3.8-27B, ollama run hf.co/{user}/{repo}:{quant} pulls it from Hugging Face. The quant name has to match the file name in the repository, for example UD-Q3_K_XL.

For llama.cpp, every flag in the commands above (-c, -ngl, -fa, -ctk, -ctv, --n-cpu-moe, -hf, --no-mmproj, -m) appears in llama-server --help for build 11429. If your build is much older, compare against its own --help first.

If the model loads but the agent still fails on real tasks, the cause is usually quantization, context, or the agent harness, and the local coding agent diagnosis guide separates those three.

Smoke Test: Prove the Model Works on Your Code

Smoke-test workflow for local coding LLMs on 16GB VRAM

Step 5 in this chart still says Qwen3.6 low-bit from July 2026; today that step is Qwen3.8-27B UD-Q3_K_XL.

Do not make a local coding model your daily driver because it loads. Make it pass a small coding workflow on your own repository.

  1. Start with gpt-oss-20b at 64K context.
  2. Check ollama ps (or the llama.cpp startup log) and confirm the model is fully on the GPU.
  3. Load one real repo slice: one file, one nearby test, and a short task description.
  4. Ask for an explanation, a patch, and the test that should change.
  5. Watch VRAM, latency, and whether the model keeps the relevant context.
  6. Repeat with a larger slice that uses more of the context.
  7. Only if the safe default fails on quality, repeat the same test with Qwen3.8-27B UD-Q3_K_XL at 32K with a q8_0 cache, or Devstral Small 2 Q3_K_M for agent tasks.
  8. If you need more files in context than those allow, repeat with Gemma 4 12B at 128K.
  9. Keep the model that edits correctly, explains the trade-off, and stays fast enough that you would actually use it.

Use the same prompt for every model:

Given the files below, refactor one function without changing behavior.
Explain the tradeoff, show the patch, and name the test that should be updated.
If the context is insufficient, say exactly what file or symbol you need next.

Pass means four things:

  • latency is tolerable enough that you would actually use it
  • no out-of-memory error and no spill to system RAM on the first real task
  • the model keeps the function, requirement and test straight
  • the patch is small, reviewable and tied to the request

A failure does not mean the model is bad. It means this combination of model, file, context and machine is the wrong fit for that job.

Stop Rules: When 16GB Is the Wrong Constraint

Leave the 16GB local setup when the job needs more context than the card can hold. The usual failure is not "the model is dumb" but "the model cannot see enough of the project without slowing down or spilling."

SymptomLikely causeBetter choice
Model loads but every answer crawlsPart of the model spilled to system RAM, or MoE experts are offloadedA file that fits fully (see the fit table), or more VRAM
Agent forgets files it just readContext still at Ollama's 4K default, or the tool sends more than the server allowsSet 64K; if the model cannot fit 64K, switch to gpt-oss-20b or Gemma 4 12B
Out-of-memory error after raising contextKV cache outgrew the card (Devstral at 64K, Qwen3.8 IQ4_XS at 64K)q8_0 cache, a smaller file, or a lower context
Good on snippets, bad on repo tasksContext packing is the bottleneckNarrow the task, or use a hosted coding agent
Tuning Qwen3.8 or Qwen3.6 takes longer than the workThe experiment has become the projectReturn to gpt-oss-20b or Gemma 4 12B

A 24GB card is the next comfort tier. By the same arithmetic, Qwen3.8-27B UD-Q4_K_M (15.33 GiB) plus a 64K F16 cache (4 GiB) comes to about 19.3 GiB, and Ollama starts 24–48 GiB cards at 32K instead of 4K. A 32GB card or two GPUs open up Qwen3.8-27B's long context. If you are still choosing hardware, the best 16GB GPU for local LLM coding guide compares the cards.

Hosted coding becomes the rational choice when the job is "get reliable multi-file edits" rather than "run locally at all costs." The guide to when local coding agents beat Claude and Codex works through that cost and quality trade.

FAQ

What is the best local coding LLM for 16GB VRAM?

As of October 22, 2026, start with gpt-oss-20b (MXFP4 GGUF, 12.11 GB, about 12.8 GiB at 64K context by calculation). For the strongest Qwen, try Qwen3.8-27B in the 13.15 GB UD-Q3_K_XL file with a q8_0 cache. For the longest context, use Gemma 4 12B in Google's 6.98 GB QAT Q4_0 file.

Can Qwen3.6 35B A3B run on 16GB VRAM?

Only in a 3-bit file or with experts offloaded. Its UD-Q4_K_M GGUF is 22.13 GB and Ollama's qwen3.6:35b is 23–24GB, so neither fits on the card alone. UD-IQ3_XXS (13.21 GB) fits at 64K for about 13.6 GiB by calculation. Alternatively, keep the 4-bit file and move experts to system RAM with llama.cpp's --n-cpu-moe, which is slower. Qwen3.8-27B has since replaced it as the first Qwen to try.

Is gpt-oss-20b good enough for coding?

It is the right baseline for 16GB because it fits with a 64K context and its memory claim comes from OpenAI. It handles focused edits, explanations and short repo slices well enough to judge with the smoke test. For deeper agentic work, Qwen3.8-27B and Devstral Small 2 are built for it, but on 16GB they need 3-bit files and shorter context.

Is Gemma 4 E4B a coding model?

It is a general model that can code, and Google targets it at phones and edge devices. Its LiveCodeBench v6 score in Google's table is 52.0%, against 72.0% for Gemma 4 12B. On a 16GB GPU use the 12B; keep E4B for 8GB cards, laptops and quick single-file questions.

Should I use Ollama or LM Studio?

Use Ollama for a fast command-line baseline and LM Studio for a GUI. Either way, choose the exact quant file and set the context yourself: Ollama starts 16GB cards at 4K context, and its default qwen3.8:27b tag is an 18GB file that does not fit. Use llama.cpp when you need a q8_0 KV cache or MoE expert offload.

Is 16GB VRAM enough for an RTX 4060 Ti or 5060 Ti?

Yes, for a useful local coding helper. gpt-oss-20b at 64K, Gemma 4 12B at 128K, and Qwen3.8-27B UD-Q3_K_XL at 32K all fit by calculation. It is not enough for comfortable long-context agent loops with 24–27B models. Those want a 24GB card.

What if I only have 8GB VRAM?

Use Gemma 4 E4B (QAT Q4_0 file 5.15 GB) or Granite 4.2 8B (Q4_K_M 5.35 GB). Gemma 4 12B's 6.98 GB file is already tight on 8GB once you add context. The 16GB picks (gpt-oss-20b, Qwen3.8-27B, Devstral) do not fit.

Is 24GB VRAM a better target?

Yes, if you want 4-bit 27B models with real context. Qwen3.8-27B UD-Q4_K_M plus a 64K F16 cache is about 19.3 GiB by calculation, Devstral Small 2 Q4_K_M at 32K is about 18.4 GiB, and Ollama starts 24GB cards at 32K context instead of 4K. You still need to check the file size, context and cache type.

How much context should I use on 16GB?

Aim for 64K, the minimum Ollama recommends for coding tools, when the model allows it: gpt-oss-20b (≈12.8 GiB) and Gemma 4 12B (≈7.8 GiB) do. Qwen3.8-27B reaches 64K only with a q8_0 cache in the UD-Q3_K_XL file. Devstral Small 2 stays at 16K–32K. Never leave Ollama at its 4K default for coding.

When should I stop using local coding LLMs?

Stop when memory tuning becomes the main job, when tasks need more files than 64K holds, or when offload makes each answer too slow to use. At that point, move to a smaller local model, a 24GB card, or hosted coding.

Can Qwen3.8 27B run on 16GB VRAM?

Yes, in a 3-bit or IQ4 file, not in the default 4-bit one. UD-Q3_K_XL (13.15 GB) comes to about 13.3 GiB at 32K and 14.4 GiB at 64K with a q8_0 cache, by calculation. Ollama's qwen3.8:27b (18GB) and the UD-Q4_K_M GGUF (16.46 GB) do not fit. Qwen's benchmark scores were measured at full precision, so test the 3-bit build on your own code.

What is the best local LLM for 16GB RAM or a 16GB Mac?

Gemma 4 12B in Google's QAT Q4_0 file. On a 16 GiB M4 Mac, llama.cpp reports a 12,124 MiB GPU budget; Gemma 4 12B needs about 7.8 GiB at 64K by calculation, while gpt-oss-20b's 11.28 GiB of weights leaves almost nothing for context. On a PC with 16GB of RAM and no large GPU, use Gemma 4 E4B or 12B on the CPU.

Can I run GLM-5.3-Flash locally on 16GB?

No. Its official FP8 weights are about 305.8 GiB, so on 16GB it is an API model. The same goes for Qwen3.8-Flash-Next (about 360 GB in BF16) and Qwen3-Coder-Next (48.4 GB even at Q4_K_M).

What is the best Ollama coding model for 16GB VRAM?

gpt-oss:20b with OLLAMA_CONTEXT_LENGTH=64000, or gemma4:12b if you need more context. For Qwen3.8-27B, skip the 18GB default tag and run ollama run hf.co/unsloth/Qwen3.8-27B-GGUF:UD-Q3_K_XL at 32K.

Sources9

External pages this guide links to, in the order they appear. Last updated Oct 22, 2026.

  1. 1.OpenAI gpt-oss-20b model card on Hugging Facehuggingface.co/openai/gpt-oss-20b
  2. 2.Gemma 4 12B model cardhuggingface.co/google/gemma-4-12B-it
  3. 3.Qwen3.8-27B model cardhuggingface.co/Qwen/Qwen3.8-27B
  4. 4.Devstral Small 2 model cardhuggingface.co/mistralai/Devstral-Small-2-24B-Instruct-2512
  5. 5.ggml-org gpt-oss-20b GGUF repositoryhuggingface.co/ggml-org/gpt-oss-20b-GGUF
  6. 6.Unsloth Qwen3.8-27B GGUF repositoryhuggingface.co/unsloth/Qwen3.8-27B-GGUF
  7. 7.DeepSeek Coderdeepseekcoder.github.io
  8. 8.Qwen2.5-Coderqwenlm.github.io/blog/qwen2.5-coder-family
  9. 9.Unsloth Devstral Small 2 GGUF repositoryhuggingface.co/unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF
Connect OpenClaw to local or hosted inference and verify replies and tool use
Developer Tools & Agents

OpenClaw LLM Setup: Connect Local Models, Cloud Providers, and Fallbacks

Run OpenClaw onboarding on the Gateway host, connect the provider or local server, then select an exact model from its catalog. Verify both a short reply and a harmless tool task. Configured defaults can use fallbacks; a manually pinned chat model remains strict.

14 min
Choosing a single RTX 5090 or dual GPUs for Qwen3.8-27B based on precision, context, and concurrency.
Developer Tools & Agents

Qwen3.8-27B VRAM Guide: RTX 5090, Dual GPUs, and 262K Context

A 32GB RTX 5090 can run quantized Qwen3.8-27B, but fitting 262,144 tokens depends on the weights, KV cache, runtime, and workload. Use these memory budgets and configuration checks to choose a practical single- or dual-GPU setup.

12 min
Qwen3-30B-A3B model choices for a local setup
Model Comparisons

Qwen3-30B-A3B: Which Version Should You Run Locally?

Use Instruct-2507 for everyday instructions, Thinking-2507 for reasoning-focused work, Coder for code, or the original when you need switchable thinking. An explicit model tag and a short context window make the first local check easier to interpret.

11 min
Local deployment decision board comparing Qwen3.8 Flash-Next with Qwen3.8-27B
Model Comparisons

Qwen3.8 Flash-Next vs 27B: Which Model Should You Run Locally?

Start with Qwen3.8-27B when memory, a simpler serving setup, and Apache-2.0 matter most. Test Flash-Next if you can host its much larger weights and want to evaluate its reported advantages on coding and agent tasks.

8 min