DeepSeek V4 Flash (mixed 2/4-bit)
antirez's mixed-precision community GGUF of DeepSeek V4 Flash 0731 — the July 2026 re-post-trained 284B-parameter Mixture-of-Experts (~13B active per token) with 1M-token context and Compressed Sparse Attention. Layers 37-42 keep Q4_K routed experts while the remaining expert layers drop to IQ2_XXS gate/up + Q2_K down, landing at ~98 GB. This is the quality/size sweet spot antirez recommends for 128 GB MacBooks: noticeably better than the straight 2-bit build without the ~156 GB footprint of the full Q4_K one. Produced for antirez's `ds4` inference engine; whether it loads on stock llama depends on its DeepSeek-V4 support.
At a glance
- Parameters
- 284B
- Quantization
- IQ2_XXS/Q4_K
- Context window
- 1000K tokens
- Approx. size
- 97.6 GB
- Engines
- llama.cpp
- License
- MIT (open)
- Version
- v1.0.0 , released 2026-08-01
- Family
- deepseek
Tuning
Defaults a gezel client applies out of the box, from real evaluation runs against this exact quantization.
- sampling.maxTokens
16384- reasoning.thinkingBudget
4096- reasoning.enableThinking
true- profiles
- thinking-general, thinking-coding, thinking-precise, instruct, creative
Model behaviors
Client-side behavior modules the catalog enables for this model (reasoning-tag handling, fabrication detection, prompt shaping).
- reasoning.strip-think-tags
- prompt.private-reasoning-guidance
- prompt.tool-cookbook-condensed
- prompt.meester-build-prelude
- mcp.compact-tool-schemas
- fabrication.detect-past-tense-no-tools
- turn.preamble-folding
Sources
- llama.cpp (GGUF)
- antirez/deepseek-v4-gguf ·
DeepSeek-V4-Flash-Layers37-42Q4KExperts-OtherExpertLayersIQ2XXSGateUp-Q2KDown-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-fixed-0731.gguf· IQ2_XXS/Q4_K - Upstream
- https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash
- Manifest
- View on GitHub
Tags
- deepseek
- reasoning
- agentic
- moe
- tools
- long-context
- large
- mixed-precision
- iq2
- q4