NVIDIA launches Nemotron 3.5 Super VL and 24GB Bolt. What can you run locally?

A larger agent model and a smaller deployment footprint

NVIDIA released Nemotron 3.5 Super VL and Nemotron 3.5 Bolt 74B-A9B 24GB on 15 October 2026. A third model, Bolt E4B 4GB, is scheduled for 23 October. The first two are available now; the smaller model belongs in the forthcoming column. Nemotron 3.5 Super VL and Bolt launch announcement.

The practical question is which model delivers useful work on the hardware available. Super VL adds visual understanding to a larger agent model. Bolt compresses that model family for a much smaller memory budget. That creates a choice between capability, capacity and response time, rather than a single winner for every user.

Main image: official NVIDIA launch artwork. This briefing analyses published evidence; the newsroom has not run these models on a local laptop.

Compare the models before choosing hardware

Published specifications, checked 15 October 2026. Parameter counts describe the model; they are not memory measurements.

Model / release	Parameters, total / active	Format and context	Status / deployment
Super VL BF16	120B / 12B	BF16; up to 1M tokens	Available. NVIDIA lists 4× H100 or 2× GB200.
Super VL NVFP4	120B / 12B	Mixed precision; up to 1M tokens	Available. Listed options include one DGX Spark or H200, two H100s, or one GB200.
Bolt 74B-A9B 24GB	74B / 9B	IQ2-based mixed precision; up to 256K tokens	Available. Packed model is 24GB; match the runtime and hardware recipe.
Bolt E4B 4GB	Not established here	4GB model announced	Scheduled for 23 October; no verified release checkpoint assessed here.

Specification sources: Super VL BF16 model card, Super VL NVFP4 model card and deployment recipes, Bolt 74B-A9B 24GB model card and Nemotron 3.5 Super VL and Bolt launch announcement. Super VL and Bolt accept text, images and video and generate text, according to their model cards. Both cards identify OpenMDW-1.1 as the governing licence. Review the actual terms for the intended deployment.

Total and active parameters answer different questions. A mixture-of-experts model selects only some of its experts for a token. That can reduce the computation needed for that token, while the other experts still need storage or an offloading strategy. Reading “12B active” as “a 12B model to load” would produce the wrong memory plan.

Independent scores show progress and the remaining gap

Use the explorer’s tabs to compare different questions. Capability uses independent scores. Local speed uses NVIDIA’s laptop measurements. Memory budget is an editable illustration, not a compatibility result. How Bolt works steps through compression, capability recovery, quantisation and speculative decoding. Short transitions show what changes; reduced-motion preferences are respected.

Artificial Analysis reports 26.5 for Super VL and 21.4 for Bolt. Its selected open-weight comparators include Inkling Small at 25.7, Nemotron 3 Ultra at 23, K2 Horizon at 31 and MiMo-V2.6-Pro at 46. Independent Nemotron 3.5 launch evaluation.

The chart makes two relationships visible: the new Super VL exceeds these US comparators, while the broader selected open-weight field contains higher-scoring models. It cannot establish a universal best model. The comparison also does not hold model size or deployment hardware constant.

The Intelligence Index v4.3.2 combines ten evaluations, with category weights of 30% agents, 20% coding, 20% scientific reasoning and 30% general capability. Artificial Analysis describes it as primarily text-based and English-language; visual and multilingual evaluations are separate. Intelligence Index v4.3.2 methodology.

A near-lossless headline can hide task-specific regressions

NVIDIA’s Super VL BF16 versus NVFP4 measurements. Changes are percentage points, NVFP4 minus BF16. These are vendor results from its published model-card suite, not the independent Intelligence Index.

Evaluation	BF16	NVFP4	Change, pp
GPQA Diamond	89.52	89.90	+0.38
MMLongBench-Doc	62.82	62.82	0.00
OCRBench v2, English	68.12	68.34	+0.22
Terminal-Bench 2.1	66.70	60.80	−5.90
WebVoyager, offline	77.72	66.31	−11.41
DeepSWE, offline	24.65	18.20	−6.45

Source: Super VL NVFP4 model card, benchmark table. NVIDIA says the BF16/NVFP4 comparison holds the non-model evaluation settings constant. The table illustrates why a retained aggregate score cannot certify every workflow.

For example, 77.72 − 66.31 = 11.41 percentage points on the listed offline WebVoyager result. Calling that an 11.41% relative fall would use the wrong denominator; the relative fall is about 14.7%. An OCR-heavy task and a browsing agent can therefore respond differently to the same deployment format.

What the five-times-faster comparison actually measured

Artificial Analysis measured Super VL NVFP4 at approximately 630 output tokens per second, against 124 for Inkling Small, using 100,000-token prompts. Super VL’s endpoint was a pre-release DeepInfra service. Independent Nemotron 3.5 launch evaluation.

630 ÷ 124 is approximately 5.08. That supports a roughly fivefold output-speed comparison under the reported conditions. It does not predict the speed of a home laptop, a different endpoint or a complete agent workflow.

Keep three measurements separate: time to first token, generation speed after output starts, and time until the task is correctly completed. An agent may spend substantial time reading tool responses, executing code or recovering from mistakes. Faster token generation can help without producing the same multiple of end-to-end improvement.

NVIDIA’s laptop table shows why the trade-off matters

NVIDIA measurements on a 128GB Surface Laptop Ultra with RTX Spark and llama.cpp. These are vendor results, not newsroom or independent laptop tests.

Model and decoding method	Coding output, tokens/s ↑	Agent trajectory replay, seconds ↓
Bolt 74B-A9B IQ2_XXS + LiLiCorr	83.99	600.80
Qwen3.6 35B-A3B UD-Q4_K_M + MTP	96.63	453.97
Gemma 4 26B-A4B UD-Q4_K_M + MTP	100.56	480.68
Qwen3.8 27B UD-Q4_K_M + DFlash	28.26	1043.28

Source: Bolt compression, quantisation and local inference measurements. Output speed uses AIPerf SPEED-Bench-Coding. Replay duration uses AA-AgentPerf Local. Lower replay duration is better; replay is not a new measurement of whether each model independently solved the tasks.

Bolt is faster than the dense Qwen3.8 27B configuration in this table, but slower than the two smaller MoE comparators. The buying argument therefore needs both capability and speed. “Fastest local model” would be a stronger claim than these measurements support. Different quantisation formats and speculative methods are part of the tested configurations, so the table is a deployment comparison, not an isolated architecture experiment.

Artificial Analysis’ own repository confirms that AgentPerf Local measures serving speed without measuring output quality. Its default replay uses an exact output-length policy and a full-context requirement; reduced-context runs are flagged as non-comparable. The NVIDIA release does not disclose every replay flag, so this briefing does not assume a particular tool-time mode or recipe commit. AA-AgentPerf Local benchmark measurement rules.

Why a 24GB model still needs a memory budget

The Bolt card calls this a 24GB packed model. Its context window is shared by prompts, visual inputs, conversation history, tool results and the generated response; deployment recipes may lower context to accommodate memory and concurrency. Bolt 74B-A9B 24GB model card.

An initial planning calculation is weight bytes ≈ total parameters × bits per weight ÷ 8. A hypothetical uniform two-bit 74-billion-parameter model would occupy 18.5 decimal GB. This is an idealised calculation, not the size of Bolt: real checkpoints include formats, scales and components at other precisions.

NVIDIA’s research describes compressing the parent model and using quantisation-aware distillation. Its expert weights use IQ2_XXS at 2.0625 bits per element, with other components at different precisions. Bolt compression, quantisation and local inference measurements.

Budget the complete session: weights + drafter, if used + attention caches and recurrent state + visual processing + temporary buffers + memory reserved for the operating system and applications. A hybrid Mamba/attention model needs an architecture-specific calculation; a standard all-transformer cache estimate is insufficient.

For Super VL NVFP4, NVIDIA publishes DGX Spark deployment instructions and local-library links. Its opening Spark recipe uses one concurrent sequence and a LiLiCorr drafter. Those details matter when translating “runs locally” into a service for several users. Super VL NVFP4 model card and deployment recipes.

How the models gain capability and how inference becomes faster

The research release describes a hybrid of sparse experts, attention and Mamba recurrent layers. Visual inputs pass through a vision encoder and connector into the language backbone. These mechanisms address different costs: sparse routing limits the experts used for each token; recurrent state changes how some sequence history is represented; quantisation reduces the precision of selected tensors. None of these removes the need to budget the complete model and runtime.

For Super, NVIDIA describes an MTP warm-up, distillation from Nemotron 3 Ultra Base, continued pretraining, long-context work and software-engineering mid-training. Post-training combines supervised examples, reinforcement learning and teacher guidance. A recovery stage follows the shorter-context engineering stage because its long-context result had deteriorated. That is a useful reminder that improving one capability can damage another. Super and Lightning architecture and training research release.

The inference-side distinction is equally important. MTP supplies a native drafter. LiLiCorr is a separate speculative approach recommended for some local configurations. Candidates still need target-model verification. Prefix reuse and hybrid-state caching reduce repeated work across agent turns. The developer article also describes Dynamo results on 16 B200 GPUs; that server experiment should never be presented as a laptop result. Super VL technical launch explanation.

A simple speculative-decoding illustration is effective tokens/s ≈ accepted tokens per verification step ÷ seconds per step, with drafting and scheduling overhead included. If four tokens are accepted in a 0.05-second complete step, the arithmetic is 80 tokens/s. This is an invented example explaining the mechanism, not a measurement of either Nemotron model. Poor acceptance or expensive drafting can erase the benefit.

The model does not continually improve its weights merely because a user chats with it. A deployed agent can accumulate files, retrieved context or explicit memory. Updating the underlying model requires an intentional training or adaptation process. Context accumulation, training and speculative decoding are separate mechanisms.

Software support needs an exact checkpoint and runtime

Deployment paths disclosed by NVIDIA. A named runtime is not a guarantee that every version, device, quantisation and modality combination works.

Path	Evidence in the release	What to verify
Super VL BF16	Multi-GPU reference deployment	Weights, runtime version, tensor parallelism and full memory footprint.
Super VL NVFP4	DGX Spark and data-centre recipes; local-library links	GPU architecture, kernels, cache precision, drafter and supported modalities.
Bolt packed / GGUF	llama.cpp on RTX Spark; vLLM on DGX Spark in NVIDIA research	Exact build, model file and drafter pairing. Do not interchange GGUF and other formats blindly.
Ollama / LM Studio	Named in the NVIDIA announcement	Confirm current import and architecture support for the selected checkpoint. No newsroom compatibility test performed.
NIM / Dynamo	Production packaging and serving optimisation	Commercial support terms, validated configuration, workload and infrastructure.

Sources: the linked model cards, Bolt research release and Super VL technical launch explanation. Support for an OpenAI-compatible API describes the request interface; it does not establish identical tool-call behaviour or image/video handling across runtimes.

For an actual trial, pin the model revision and runtime build. Start without optional acceleration, confirm a short text request, then test vision and structured tool calls. Add speculative decoding and caching one at a time. Otherwise a faster but incorrect parser can be mistaken for a better model.

The cost question is useful work per pound

A downloadable model is not a free operating service. Local cost includes the computer, memory capacity, electricity, setup time, maintenance and work lost to incorrect answers. API cost includes input, output, reasoning and cache treatment under the chosen provider’s tariff. No single current UK API price or universal local token cost is established here.

A transparent local estimate is cost per successful task = (hardware allocation + energy + operations + service fees) ÷ correctly completed tasks. The denominator matters. A model that generates cheap tokens but needs frequent human repair may cost more per useful result.

For an explicitly hypothetical energy example, 0.25kW × 8 hours × £0.25/kWh = £0.50 per day, or £15 over 30 such days. The power and tariff are selected assumptions, not measurements or a quoted electricity rate. Idle time, the rest of the computer and cooling can change the total.

For a separate hardware-allocation example, £3,000 ÷ 36 months = £83.33 per month before energy and operations. At 10,000 correct tasks monthly, that allocation contributes about 0.83p each; at 100, it contributes about 83p. These are scenarios, not a quoted system price, resale estimate or investment recommendation.

Compare the same prompt distribution, concurrency, response-time requirement and correctness threshold. A hosted endpoint avoids some local administration; a local system can be attractive for repeatable utilisation, offline operation or sensitive data flows. The balance changes with workload and staffing, not just the model’s nominal size.

Licensing, documentation gaps and operational limits

OpenMDW-1.1 grants broad rights in the supplied model materials without a licence fee, subject to its terms. Distribution requires retaining the agreement and applicable notices. The licence imposes no obligations on model outputs, but it includes termination provisions and an “as is” disclaimer. It does not establish that every input, training artefact or output is free of third-party rights. OpenMDW-1.1 licence text.

What the reviewed evidence establishes, and what still needs checking.

Question	Supported conclusion	Remaining gap
Can the model see?	Cards describe image/video input and text output.	Your own documents, charts, language mix and video settings need testing.
Does 24GB guarantee a 24GB GPU?	The name describes a packed-model target.	Measure all runtime allocations and supported hardware together.
Is the smaller precision equivalent?	Some vendor tasks retain scores; some regress.	Representative task-level validation, not an aggregate promise.
Is the training budget fully reconciled?	NVIDIA discloses multiple stages and data accounts.	Counts differ across release documents; no independent training-cost audit here.
Does local mean isolated?	Weights can be served on local hardware.	Tools, telemetry, retrieval and remote APIs can still create outbound flows.
Are supported languages equally strong?	The cards describe multilingual support.	Do not infer equal quality from a language list or an English-heavy index.
Is E4B available?	Launch announcement schedules 23 October.	A released checkpoint and independent results were not verified.

Documentation-gap sources: Super and Lightning architecture and training research release, Super VL technical launch explanation and Bolt model card, training methodology and dataset overview. The discrepancies do not by themselves show that benchmark results are invalid; they limit what can be concluded about training accounting.

For an agent that can change files or call tools, deployment needs controls around the model: restricted tool permissions, validation of structured outputs, a human decision point for consequential actions and logs that make failures diagnosable. Model openness provides access; it does not provide those controls automatically.

What small labs and desktop users should test next

Assessment: the useful market shift is the possibility of running a capable visual and tool-using agent within a smaller local footprint. Local operation can give a lab more control over its files and inference spending. Whether it is economical depends on the real hardware purchase, energy use, maintenance and the amount of useful work completed.

A local model can still send information outside the machine through a browser, search tool or remote API. Check the agent’s tools and destinations before treating local inference as a private workflow.

What would change this assessment? Reproducible measurements on ordinary desktop configurations, a measured total memory footprint at useful context lengths, and successful task completion at a disclosed cost. The current evidence supports a promising local deployment option, with configuration limits that deserve as much attention as the headline score.