Nemotron 3.5 Lightning (30B-A3B, Q4)
NVIDIA's Nemotron 3.5 Lightning at 30B parameters — text-only, with native reasoning, native tool calling, and a 256K context window. Capable-tier on-device option: about 25GB of memory on most machines, about 18GB on Apple Silicon.
At a glance
- Parameters
- 30B
- Quantization
- Q4_K_M
- Context window
- 262K tokens
- Approx. size
- 25.4 GB
- Engines
- llama.cpp, MLX
- License
- OpenMDW-1.1 (open)
- Version
- v1.0.1 , released 2026-08-15
- Family
- nemotron
Tuning
Defaults a gezel client applies out of the box, from real evaluation runs against this exact quantization.
- sampling.temperature
1- sampling.topP
0.95- sampling.maxTokens
16384- reasoning.thinkingBudget
8192- reasoning.enableThinking
true- profiles
- thinking-general, thinking-coding, thinking-precise, instruct, creative
Model behaviors
Client-side behavior modules the catalog enables for this model (reasoning-tag handling, fabrication detection, prompt shaping).
- reasoning.strip-think-tags
- prompt.private-reasoning-guidance
- prompt.tool-cookbook-condensed
- mcp.compact-tool-schemas
- fabrication.detect-past-tense-no-tools
- prompt.retrieval-first
- prompt.workspace-gestalt
- turn.preamble-folding
- turn.ramble-detection
- tools.mlx-grammar
Sources
- llama.cpp (GGUF)
- ggml-org/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF ·
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Q4_K_M.gguf· Q4_K_M - MLX
- mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-4bit
- Upstream
- https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16
- Manifest
- View on GitHub
Tags
- nvidia
- reasoning
- agentic
- coding
- tools
- long-context
- moe
- hybrid
- mamba
- open-weights
- apple-silicon