Muse Spark Open: 533 GB at 4-bit, about 50 GB read per token, and what Meta's launch leaves unstated

The 4-bit weights are 533 GB, and the 4 does less work than it sounds

Meta released Muse Spark Open on 15 October 2026. The public file listing for its two Hugging Face repositories, read on 18 October, gives the size of the weight files exactly: 1,644,333,393,864 bytes in 65 shards for the BF16 checkpoint, and 533,011,023,472 bytes in 127 shards for the NVFP4 checkpoint. That is 1.644 TB and 533 GB in decimal units, a ratio of 3.08 to 1 (derived), not 4 to 1. Averaged over every parameter, the NVFP4 files hold 5.19 bits per parameter (derived: 533,010,718,720 bytes times 8, divided by 822,036,529,664 parameters in the file headers). The quantisation file in the repository excludes the attention blocks, the output layer and the experts of the first and last layers from the 4-bit format, and Meta's developer blog says the same thing in five words: "Only the experts are quantized."

That is a property of the files, and it matters because a decoded token has to read weights from memory. From the model's configuration file, the 42 billion parameters Meta says are active per token imply roughly 49 GB read per token from the NVFP4 files and 84 GB from the BF16 files (both derived, with a range explained below). So the 3.08 times saving in storage becomes a 1.70 times saving in the bytes each token reads. None of the launch material states what that means for the bytes each token reads, and none of it is false: the storage figure is right, and the 4-bit label describes the experts.

What the arithmetic does not establish is speed, accuracy or safety. Every decode figure in this briefing is a bandwidth ceiling, an upper bound computed from vendor memory specifications, not a benchmark. Hugging Face's launch post says the 4-bit checkpoint loses "negligible" accuracy; no independent comparison of the two checkpoints was found, and Meta's model card says quantised inference "may show minor quality differences in edge cases". No independent safety evaluation of the model was found either. The sections below say what was read, what each number rests on, and what the launch leaves out.

What a token reads: 49 GB, not 21 GB

The configuration file gives the structure. There are 62 layers, each with 256 experts, of which 8 are routed per token. The expert size is set by two fields, a hidden size of 4,096 and an intermediate size of 4,096, with a SiLU gated design, which gives 3 x 4,096 x 4,096 = 50,331,648 parameters per expert (derived, the three matrices of a gated SiLU design). That figure is consistent with the NVFP4 file headers to the last digit, not just with the configuration: the 773,094,113,280 packed 4-bit values divide exactly into 60 layers of 256 experts at that size, and the quantisation file excludes layers 0 and 61. The 8 routed experts in each of the 62 layers hold 24,964,497,408 parameters (derived).

Meta's figure of about 42 billion active parameters then implies about 17.0 billion non-expert parameters read on every token (derived: 42 billion minus 24.96 billion). In the NVFP4 files those non-expert weights, which include all of the attention, are BF16. Each 4-bit value also carries one 8-bit scale per 16 values, so an FP4 parameter costs 0.5625 bytes (the headers confirm one scale per 16 values). Adding the routed FP4 experts (13.59 GB), the routed BF16 experts of layers 0 and 61 (1.61 GB) and the BF16 non-expert weights (34.07 GB) gives 49.27 GB per token (derived). Seventy-two per cent of those bytes are still BF16 (derived). The naive reading, 42 billion parameters at 4 bits, is 21.0 GB, and the real figure is 2.35 times that.

Bytes stored and bytes read per decoded token, from the public file listing, config.json and the quantisation file. The read figures are derived from Meta's stated 42B active parameters.

Format	Stored (file listing)	Read per token (derived)
BF16	1,644 GB	about 84 GB (42B x 2 bytes), up to 89 GB on the upper reading
NVFP4	533 GB	about 49 GB, up to 55 GB on the upper reading
Naive: 42B at 4 bits	not a real file	21 GB, which the files do not deliver

The upper reading counts every non-expert parameter except the input embedding and the 1.8 billion parameter perception encoder as read on each token, which gives 44.7 billion active parameters rather than 42 billion. Which is right is not stated: Meta gives "about 42B" and no breakdown. The two readings give 49.3 and 54.6 GB for NVFP4, so the conclusion does not move: the active read is roughly 50 GB, not the 21 GB a reader would compute from the headline. Two smaller discrepancies sit beside it. The file headers count 822.04 billion parameters against the "about 816B" in every launch document (a 6.04 billion, 0.74 per cent difference; which tensors the 816B excludes is not stated), and the Hugging Face post's "about 1.5 TB" for BF16 is 1.4955 TiB, so its TB is a binary unit while every other figure here is decimal.

Bandwidth sets the ceiling, and the ceilings are upper bounds

Decoding one token for one user reads the active weights once, so the decode rate cannot exceed memory bandwidth divided by bytes read per token. Memory and bandwidth below are as printed on NVIDIA's and Apple's own pages, read on 18 October. Both companies sell this hardware, and the pages are marketing specifications, not measurements of this model.

Decode ceilings for the NVFP4 checkpoint (derived: aggregate bandwidth divided by 49.27 GB, plus the cache read at 1,048,576 tokens). Memory and bandwidth from the vendor pages named in the first column.

Hardware (vendor page)	Memory and bandwidth as printed	Ceiling, short context	Ceiling at 1,048,576 tokens
8 x NVIDIA H100 SXM	640 GB; 26.8 TB/s (80 GB, 3.35 TB/s each)	544 tokens/s	226 tokens/s
8 x NVIDIA RTX PRO 6000 Blackwell Server	768 GB; 12.8 TB/s (96 GB, 1,597 GB/s each)	259 tokens/s	108 tokens/s
2 x Mac Studio, M5 Ultra, 512 GB	1,024 GB; 2.4 TB/s (1.2 TB/s each)	49 tokens/s	20 tokens/s
8 x NVIDIA H200	1,128 GB; 38.4 TB/s (141 GB, 4.8 TB/s each)	779 tokens/s	324 tokens/s
NVIDIA DGX B200 (8 GPUs)	1,440 GB; 64 TB/s as printed	1,299 tokens/s	541 tokens/s
NVIDIA GB200 NVL72 rack	13.4 TB; 576 TB/s as printed	11,690 tokens/s	4,866 tokens/s

Capacity is a separate question. The 533 GB of NVFP4 weights fit on all six, but the 1,644 GB of BF16 weights fit on none of the first five: the DGX B200 holds 1,440 GB. Hugging Face's BF16 recipe uses four nodes of eight H200 (4,512 GB, 153.6 TB/s, a ceiling of 1,829 tokens/s, derived), and Meta's developer guide lists eight B300 for BF16 and four for NVFP4; NVIDIA's DGX B300 page prints 2.1 TB of memory for eight GPUs but no memory bandwidth, so no ceiling is computed for that recipe. The rack-scale systems hold both formats, and a GB300 NVL72 rack, with NVIDIA's "up to 576 TB/s", has the same ceiling as the GB200 rack. Meta's model card says the model "requires a GPU with at least 700GB of VRAM for NVFP4 and 2TB for BF16". The files leave 167 GB and 356 GB of headroom against those figures (derived), 31 and 22 per cent.

What the arithmetic leaves out is most of what a production system does. It ignores batching: with 64 concurrent streams and, as an assumption, independent uniform routing (a trained router does not behave that way), 87 per cent of the experts in each layer are touched on each step, an eight-H200 step reads about 457 GB, and each stream falls from 779 to about 84 tokens/s while the node serves about 5,400 tokens/s in aggregate (all derived). It also ignores prefix caching, interconnect time, expert placement and speculative decoding. The only measured figure in the pages read is Artificial Analysis's median output speed across five hosted providers, 140.3 tokens/s, on hardware it does not name. NVIDIA's developer blog reports "up to 600 tokens per second per user" on four GB300 GPUs with speculative decoding, a vendor claim for a regime the ceiling does not model; the weights-only ceiling for four such GPUs is 649 tokens/s (derived from the rack figures divided by 72). Meta's launch post says two Apple M5 Ultra Mac Studios can run the model on a desktop. Neither it nor the cookbook prints a speed for that setup; the ceiling for it is 49 tokens/s at short context.

The cache: the post's 266 GB and what the configuration lists

The Hugging Face post says each token adds 2 x 16 heads x 64 dimensions x 62 layers = 126,976 values to the cache, "about 250 KB per token and about 266 GB for a full 1M-token context" in BF16. The arithmetic is right, and it treats all 62 layers as full attention. The configuration file does not: its layer types list 16 full-attention layers (every fourth, at indices 1, 5, 9 and so on to 61) and 46 sliding-attention layers with a window of 2,048 tokens. If a serving stack frees entries that fall outside the window, the cache at 1,048,576 tokens is 69.1 GB, 3.85 times smaller (derived). That condition was not tested here. Meta's developer blog says one drafter "stays cheap at long context by using the sliding-window attention in Muse Spark Open", which is consistent with it and is not a measurement of cache size. Meta's TensorRT-LLM recipe also uses an FP8 cache, which halves the figures again.

Cache per sequence in BF16 unless stated, derived from the post's formula and from config.json (16 full-attention layers, 46 windowed layers of 2,048 tokens, 16 KV heads, 64 dimensions).

Basis	131,072 tokens	1,048,576 tokens
The post: all 62 layers full attention (248 KiB per token)	33.3 GB	266.3 GB
config.json: 16 full plus 46 windowed layers (64 KiB per token on the full layers, plus 386 MB fixed)	9.0 GB	69.1 GB
config.json with an FP8 cache, as in Meta's TensorRT-LLM recipe	4.5 GB	34.6 GB

At long context the cache, not the weights, can dominate the read. At 1,048,576 tokens the full-attention layers' cache is 69.1 GB per sequence, more than the 49.3 GB of weights each token reads, so the ceiling for eight H200 falls from 779 to 324 tokens/s (derived). The headroom after the NVFP4 weights on eight H200 holds 8.6 such sequences by the configuration's arithmetic, or 2.2 by the post's, before activations and runtime overhead. The diagram draws the weights, the two cache figures and the memory of the hardware on one scale.

Where the Hugging Face launch post and Meta's own files or vendor pages differ, read 18 October 2026.

Item	The post says	The files or vendor page say
BF16 size	"about 1.5 TB"	1.4955 TiB, or 1.644 TB decimal; Meta's docs say "About 1.65 TB"
Cache at 1M tokens	about 266 GB	69.1 GB if 46 of 62 layers are windowed, as config.json lists (derived)
RTX PRO 6000 memory	98 GB	NVIDIA's page: 96 GB GDDR7
Reasoning effort integers	16, 64, 256, 512	Meta's chat template: low 8, medium 32, high 256, xhigh 512
Total parameters	about 816B	822.04B tensor elements in the file headers (derived)

Pick a machine, a weights format and a context length in the chart below, or let it play once. It redraws numbers already in this briefing and adds no new reporting: every figure it adds, such as a ceiling at a context length the tables do not show, is this site's own arithmetic and is marked derived, and the ceilings are upper bounds, not benchmarks.

What the launch states, and what it leaves out

A launch is a set of claims by the people who made the model, repeated by the people who host it. The table separates what the pages read state from what they do not. Meta is the licensor, the author of the comparison chart and the methodology, the runner of its own safety tests and, per its methodology, of several competitors' tests, and the seller of hosted Muse models through its Model API.

Stated and not stated for Muse Spark Open, from Meta's launch post, developer blog and docs, evaluation methodology, the model card on Meta's repository page, the Hugging Face post and NVIDIA's blog, read 18 October 2026.

Item	Stated	Not stated
Size	816B total, 42B active (Meta, Hugging Face, Artificial Analysis)	which tensors the 816B excludes; the split of the 42B between experts and attention
Training data	categories: public sources, crawled from the internet, acquired from third parties, derived from Meta's products and services, synthetic (model card)	sources, volumes, dates, filtering rules or a list
Training compute and cost	NVIDIA says it was trained on GB300 NVL72 systems	GPU count, hours, tokens, cost, energy
Evaluation	a benchmark chart and an 8-page methodology PDF dated October 2026	the full evaluation report (Meta, 15 October: "available soon"); run logs; confidence intervals
Safety	four areas tested; refusal rates; "met the thresholds our framework requires for release"	an independent evaluator's report; the thresholds; results after fine-tuning
Integrity	repository revisions are listed on the Hub	published hashes or signatures: the public listing masks the hash fields, and none was found elsewhere
Licence	the name, the text and a usage policy	a definition of "European region"; termination; governing law (not in the text read)
Speed	vendor claims, for example NVIDIA's 600 tokens per second per user with speculative decoding	an independent throughput measurement on stated hardware

The "not stated" column is mostly ordinary for open-weight releases, and it is the column a deployer's risk register depends on. The UK's voluntary Code of Practice for the Cyber Security of AI says developers "should" document "the sources of training data" and "shall release cryptographic hashes for model components"; on the two items read, the training-data sources are not stated, and the hash fields in the public listing are masked.

"Open": what the label covers

Meta's launch post is headed "The Most Capable US Open-Weight Model to Date" and says the weights are released "under the Muse Model License with limited commercial terms, which allow many developers to use the model for free". The Hub metadata says "license: other" with the name "muse-model-license". Open weights are a fact about the files. They are not a statement about the licence, the data or the training code.

The repositories hold the weights, the configuration, the tokenizer, the chat template, the processor configuration, a licence and a usage policy. They do not hold the pre-training data, the training run's record or the evaluation harness. Meta's developer blog points to PyTorch's TorchTitan for pre-training and fine-tuning recipes, outside the repositories. Artificial Analysis, an independent index, scores openness on a published scale with a maximum raw score of 18. It gives Muse Spark Open 27.8 out of 100 (5 of 18, derived): 4 for availability (weights access scores 3, so the licence scores 1, which AA's scale defines as commercial use with attribution required; derived), 1 for methodology, and 0 for both pre-training and post-training data. On the same page AA scores Nemotron 3 Ultra 83.3 and MiMo-V2.6-Pro 52.8. The scale is AA's and is one view of openness, but it shows the label covers less than the word suggests.

The Muse Model License (version Muse Spark Open, 15 October 2026) and its Exhibit A usage policy, from the licence text published in the Hub metadata and the byte-identical LICENSE.md and USE_POLICY.md.

Clause	What the text says	Question for a UK deployer
Grant	free of charge to any person, to use, copy, modify (including fine-tune), create derivative works, distill, merge, publish, distribute, sublicense and sell	Does the grant reach your group and your suppliers?
Attribution (clause 2)	"Built with Muse Spark Open" in the interface of each product or service that uses the model or any Derivative; Derivatives include any model trained, distilled or improved "using the Software or any outputs or results thereof"	Where does it appear, and does a model trained on its outputs carry it?
Model as a Service (clause 3)	if you or affiliates give third parties API access to models and gross revenue exceeds US$20 million in any consecutive 12 months, notify Meta and sign a separate agreement first; internal use, research and powering your own product features are outside it	Is group revenue over the line, and are you reselling access?
As is (clause 5)	no warranty; no liability for Derivatives; the licensee is "solely responsible for the safety, deployment, and use of its Derivatives"	Who owns the risk when you fine-tune?
Usage policy: security-relevant lines	no creating malicious code or malware (1.7); no circumventing usage restrictions or safety measures (1.8); no operating critical infrastructure (2.4); no military, nuclear or espionage use (2.1); not intended for under 18s	The text read has no carve-out for authorised security testing: ask counsel before a red team or malware analysis
Contracting party	Meta Platforms Ireland Limited if you are in "the European region", Meta Platforms, Inc. outside it	"European region" is not defined in the text, so whether a UK organisation counts is not stated

None of the text read contains a termination clause, a governing-law clause or a right for Meta to change the usage policy. Those absences are findings about the text, not a legal opinion; a lawyer should read the full file before the form is submitted. Access is gated by a form that records the acceptance, and the licence's own condition 1 requires the licence and the usage policy to travel with every copy and Derivative.

Independent measurements, with the cost beside each

Three days after release, two independent indexes list the model. Artificial Analysis gives it 38.9 on its Intelligence Index v4.3.2 (shown as 39), 10th of 120 among the open-weights models AA compares it with, at $0.84 per task, on a run that generated about 110 million output tokens, roughly 70 per cent of them reasoning tokens by AA's raw counts (derived). Vals AI's Vals Index (version 2.1, updated 16 October) gives it 46.61 per cent, rank 38 of the 49 models listed, at $4.08 per test. CWE-bench does not list it; its closest entry, Muse Spark 1.2, is a different model and is not used here. The model's list price on OpenRouter, listed from 15 October at 17:57 UTC, is $1.25 per million input tokens and $4.25 per million output tokens.

Independent and vendor measures for Muse Spark Open with the cost beside each, read 18 October 2026. Who ran each is in the first column.

Measure (who ran it)	Muse Spark Open	Cost beside it
Intelligence Index v4.3.2 (Artificial Analysis, its own harness)	38.9; 10th of 120 open-weights models in its comparison class	$0.84 per task; $1.25 in and $4.25 out per million tokens
Vals Index v2.1 (Vals AI, page updated 16 October)	46.61 per cent; rank 38 of 49 listed	$4.08 per test; 6.48 billion input and 55.52 million output tokens
CWE-bench (Collinear AI)	not listed	not applicable
Meta's chart, Intelligence Index row (Artificial Analysis scores sourced on 7 October, 8 days before release)	39; fourth of the seven models with a number	none shown
Terminal-Bench 2.1 in Meta's chart (AA, per Meta)	76.4	none shown
Terminal-Bench 4.0 in AA's index and, via OpenRouter, Vals	8.1 (AA) and 6.6 (Vals) per cent	a different benchmark version; Vals says 4.0 is "substantially harder"

Two things follow. The narrow claim in the headline holds on AA's data. Muse Spark Open is the highest-scoring open-weights model from a US creator it lists, and the next US entry, Beam 1.0, is 8.5 points lower at 30.3. But nine open-weights entries from other creators score higher, and the Meta chart omits the highest, MiMo-V2.6-Pro, which AA lists at 46.3 for $0.13 per task under an MIT licence. Meta's methodology says it compared against "a global set of leading open weight models" and does not say how the set was chosen. An earlier briefing covers that model and the provenance question attached to it. The second point is Meta's own hosted model: AA scores Muse Spark 1.3 (Max) at 48.1 and Vals at 58.16 per cent, against 38.9 and 46.61 for the open model, at the same OpenRouter list price of $1.25 and $4.25 per million tokens.

Open-weights models in Artificial Analysis's data on 18 October 2026: Intelligence Index v4.3.2 score, cost per task and licence as AA lists them. Models in Meta's chart, plus MiMo-V2.6-Pro.

Model	Index; cost per task	Licence as AA lists it
MiMo-V2.6-Pro (not in Meta's chart)	46.3; $0.13	MIT
GLM-5.3 (Max)	44.8; $2.01	GLM-5.3 License
Kimi K3 (Max)	43.6; $2.00	Kimi K3 License
Qwen3.8 2.4T A95B	39.9; $2.16	Qwen3.8-Max License
Muse Spark Open (xhigh)	38.9; $0.84	Muse Model License
Mistral Large 4 Preview	38.4; $1.13	not listed
Beam 1.0	30.3; $0.57	Apache 2.0
Nemotron 3 Ultra 550B A55B	22.9; $0.58	OpenMDW

Cost per task depends on reasoning and on context. Meta's developer docs say "Thinking cannot be turned off", the default effort is xhigh, and the chart in Meta's methodology runs it at xhigh. In AA's own split, input tokens, including cache reads and writes, are $0.67 of the $0.84 per task and output is $0.17 (derived from AA's data), because agentic tasks resend long contexts. A deployer who self-hosts pays for that as prefill time and cache memory, which the ceilings above do not model. Meta's methodology also says its evaluation set-up is "not specifically tuned for third-party models" and that it reports "best-effort evaluations" of them. Where Meta quotes AA's index, its chart agrees with AA's page today to the rounded point for six of seven models; Inkling reads 26 in the chart and 25.0 on AA's page (AA may have re-run it since 7 October; not stated).

Safety evidence, and what open weights change

Meta's model card reports tests in four areas. On cyber misuse it says the model "refused 99.2% of jailbreaks on our internal benchmark and 94.8%" across three public benchmarks, and that in the same setup Kimi K3 refused 75.5 and 70.2 per cent. The tests used adaptive attackers "with no system-level safeguards in place". Meta's methodology calls its StrongREJECT v2 test "an internal benchmark" of 202 behaviours and says the results for all models except two "are from internal runs", so on that table Meta ran the competitors' models. The FORTRESS figures come from Scale AI's leaderboard. No independent evaluator's report on the model was found.

The card also says the evaluations "characterized the released checkpoint under the tested conditions", that "Downstream modifications could degrade safety", and that the results should not be assumed to apply to modified models. Meta's launch post says a full evaluation report is "available soon"; the research blog index read on 18 October did not list one. The UK AI Security Institute's blog index, read the same day, lists no post on Muse Spark Open. Meta's August account of a third-party evaluation of Muse Spark 1.1, in which a misconfiguration let a pre-release model reach a real website, is covered in an earlier briefing. It matters here because the card describes this model's cyber tests as run without the production safeguards.

What open weights change, in one plain sentence: the weights cannot be withdrawn once copied, and Meta's usage rules bind only through the licence. Meta's own Scaling Framework update of 2 October says "Someone holding the weights can resample to get around a refusal, prefill outputs, or fine-tune refusal behavior away". The UK AI Security Institute wrote on 17 July, in its analysis of open-weight models, that remaining techniques such as refusal training "are often easily reversible with access to the weights", and in August 2025 that "Once a system is released with open weights, it cannot be rolled back." On 18 October the Hub's search returned 32 repositories whose names contain Muse-Spark-Open: 2 are Meta's and 30 come from other publishers, three days after release, among them quantised forks and draft models for speculative decoding. One name contains the word "Abliterated"; it was not downloaded or inspected, so what it is is not stated here.

UK context, as read from the primary pages

The AI Security Institute has published nothing found on Muse Spark Open. For the comparable class it published, on 17 July 2026, How Far Behind the Frontier are Leading Open Weight Models on Cyber?, finding that GLM-5.2 and DeepSeek V4-Pro performed like closed frontier models released 4 to 7 months earlier, and that its evaluations of recent open-weight models "were largely unimpeded by safeguards". A joint assessment with the US CAISI on 23 July found Kimi K3's safeguards "did not prevent it from attempting cyber exploit development". Muse Spark Open's cyber capability has no AISI figure in what was found, so the card's refusal rates are the only cyber safety evidence, and they are Meta's.

The DSIT Code of Practice for the Cyber Security of AI, published 31 January 2025 and voluntary, treats anyone "creating or adapting" a model as a Developer, open-source models included, and anyone deploying one as a System Operator. It adds that "Some provisions for Developers in the Code are less applicable to AI systems involving open-source models." Four provisions bear on this launch. Provision 3.1.2 asks Developers to manage risk from "superfluous functionalities", with a multi-modal model used for one modality as its example: this model takes text, images, audio and video. Provision 7.3 says Developers and System Operators "shall re-run evaluations on released models that they intend on using". Provision 8.1.2 says Developers "shall release cryptographic hashes for model components"; the hash fields in the public listing are masked and none was found elsewhere. Provision 12.1 says System Operators "shall log system and user actions". A UK organisation that fine-tunes the weights is adapting a model, which on this briefing's reading of the definition makes it a Developer too, and the gaps upstream limit what it can document and pass on.

The NCSC's interim advice on agentic AI, 20 August 2026, applies because Meta and Hugging Face describe the model as trained for agentic coding. It warns that safety controls built into models can be bypassed and are not sufficient alone, and its network maturity model puts "no external network access, with the model hosted locally inside the network sandbox" at its highest level, level 4. Self-hosted open weights can reach that level and a hosted-only model cannot, which is a real advantage of the release, and it comes with the duty to run the runtime, the sandbox and the logs yourself.

On data protection, a self-hosted model keeps prompts inside the organisation. Meta names seven hosted providers: Baseten, Fireworks AI, Groq, Modal, Nebius, OpenRouter and Together AI. Where each processes UK personal data is not stated in the pages read. The ICO's international transfers guide, last updated 15 January 2026, sets a three step test that includes whether the receiving organisation is "a separate legal entity", and counts making information "accessible" as a transfer. The ICO's October report on AI developers is in another briefing. For the licence, whether a UK organisation contracts with Meta Platforms Ireland Limited or Meta Platforms, Inc. depends on a phrase the text does not define.

What to do, in order

The question the launch leaves open

The weights are 533 GB, the files sit behind an access form, and the public listing masks the hash of every file; whether the repository shows the hashes after approval was not tested. So the thing a change record points at is a revision and a size, and not yet a hash Meta has published. If Meta stopped hosting those files tomorrow, which copy would your change record point to, and who checked it against what?

Key facts