P.K. SHARMA

Cyber security intelligence, AI governance, practitioner analysis

Microsoft says MAI Cyber 2 Flash scores 92.3% on CyberGym at under 2% of GPT-6 Astra's cost; 92 tasks excluded

Microsoft AI's page of 17 November 2026 says MAI Cyber 2 Flash scored 92.3% against GPT-6 Astra's 91.5% on CyberGym-E2E at under 2% of the cost per task, after excluding 92 of 905 overlapping tasks. On the 813 left the gap is 6 tasks (derived); tokens per task are not stated.

By · LinkedIn · · 22 min read

92 of 905 tasks overlapped Microsoft's training sources, and the whole gap is 6 tasks

Microsoft AI's page of 17 November 2026, "MAI Cyber 2 Flash: Leading security. Lower cost", says that on CyberGym-E2E "MAI Cyber 2 (92.3%) matches GPT 6 Astra (91.5%) at less than 2% of the cost per task". Its benchmark note says that 92 of the benchmark's 905 tasks "are based on the same open source projects and vulnerability reports as some of our training data", so Microsoft excluded those 92 "for every model" and reports the remaining 813. The overlap is 10.2% of the benchmark (derived: 92 / 905). On 813 tasks, 92.3% is 750 tasks and 91.5% is 744 tasks (derived: 750 / 813 = 92.25%, which prints as 92.3; 744 / 813 = 91.51%, which prints as 91.5; no other whole number of tasks prints those percentages, since 749 and 751 print 92.1 and 92.4 and 743 and 745 print 91.4 and 91.6). The whole headline gap is therefore 6 tasks (derived: 750 less 744).

Microsoft found and disclosed the overlap itself, which is to its credit as a disclosure. The price claim has its own arithmetic. The page's footnote prices MAI Cyber 2 Flash at $0.10, $0.01 and $0.50 per million input, cached input and output tokens, and GPT-6 Astra on Azure OpenAI Global at $10, $1 and $50. Each Microsoft price is exactly 1% of the Azure price on all three lines (derived: 0.10 / 10, 0.01 / 1 and 0.50 / 50 each equal 0.01). The page nevertheless says "less than 2% of the cost per task". The difference is how many tokens each model used on each task, and the page does not print that number.

What that does not establish. It does not establish that MAI Cyber 2 Flash is better or worse than any model. It does not establish who else could reproduce the result: the page names no API, no weights and no outside evaluator. It does not establish how the 92 excluded tasks would have scored, what CyberGym-E2E's 905 tasks are, or how many tokens either model spent. This site reports the page, works out the arithmetic and lists the gaps. Every figure it worked out is labelled "derived".

Whose account this is. Microsoft made the model, sells the security products it is going into, ran the benchmark itself and chose the comparators. Microsoft also sells Azure OpenAI, which supplies the GPT-6 Astra list price it compares against. The page is Microsoft's claim, and Microsoft's disclosure of the overlap is part of the claim. An earlier brief read another Microsoft account of its own AI security work in the same way.

What Microsoft's page says, in its own order

The page opens with the problem (security teams need AI to find vulnerabilities, recommend fixes and investigate threats "around the clock", and the cost adds up) and calls MAI Cyber 2 Flash "our latest specialist security model" that draws on "decades of proprietary security data and expertise". It says the model "delivers leading performance on real-world security tasks at a fraction of the cost of larger models" and is coming to Microsoft's security products.

Its figures are then these, each with a numbered footnote under the heading "About these results". CyberGym-E2E, described only as "a benchmark measuring real-world vulnerability discovery": 92.3% for MAI Cyber 2 Flash and 91.5% for GPT-6 Astra, with Flash "using half the reasoning budget". WebExploitBench: 40.4% against 38.4% for GPT-5.6 Cyber "at the same reasoning budget". Cost per task, "measured on CyberGym-E2E from all input, cached input and output tokens, priced at list price", which "reflects offline evaluation cost, not customer spend". A customer, Insight Enterprises, reports a 20% token efficiency advantage. A one million token context window, "four times larger than MAI Cyber 1.1 Flash" (so 250,000 tokens for that model, derived: 1,000,000 / 4; the saved page for it prints no window). And "early evaluations through Microsoft Perception" showing performance "comparable to leading models" on investigation agents and security detection authoring, with no figures.

Artificial Analysis lists a one million token window for both GPT-6 Astra and GPT-6.2 Astra, so the window is the same size as the comparator's. Microsoft's Ignite announcement, published the same day, says the model is "priced 86% lower on output tokens than MAI Cyber 1.1 Flash"; against the $0.50 output price that implies about $3.57 per million for the older model (derived: 0.50 / (1 - 0.86), subject to rounding of the 86%). No saved page prints the older model's price. The same announcement lists MAI Cyber 2 Flash first among "seven new world class models".

The availability paragraph is one sentence: MAI Cyber 2 Flash "is coming exclusively to Microsoft security products, including Codename MDASH, where it will automatically replace the existing model in the MAI profile for eligible customers starting mid-December". The page prints no API, no price list for customers, no weights and no model card. Its price figures are list prices for an offline evaluation.

From 905 tasks to a six task gap

The diagram puts every count on one zero based axis of tasks. The page prints 905, 92 and 813 and the two percentages; the 750 and 744 are this site's task counts from the percentages. The thin amber band on the two pass bars is the gap, and the zoom below the chart opens it up.

Drawn from the figures on Microsoft AI's page of 17 November 2026. The scale comes from this site's scale code. 750, 744, 63, 69, 6 and 10.2% are derived.

The counts, where each comes from and the arithmetic. Figures the page does not print are labelled derived.

CountValueSource and arithmetic
Tasks in CyberGym-E2E905Page, benchmark integrity note
Excluded, for every model92, 10.2%Page. Derived share: 92 / 905 = 10.17%
Scored813Page. Check: 905 less 92 = 813
MAI Cyber 2 Flash, 92.3%750 tasks (derived)750 / 813 = 92.25%. 749 prints 92.1 and 751 prints 92.4, so only 750 prints 92.3
GPT-6 Astra, 91.5%744 tasks (derived)744 / 813 = 91.51%. 743 prints 91.4 and 745 prints 91.6, so only 744 prints 91.5
Gap in tasks6 (derived)750 less 744 = 6. In points: 6 / 813 = 0.74. The printed percentages give 0.8 (92.3 less 91.5)
Not passed, MAI Cyber 2 Flash63 (derived)813 less 750
Not passed, GPT-6 Astra69 (derived)813 less 744
WebExploitBench40.4% against 38.4%Page. Gap 2.0 points. The task count is not printed, so no task counts can be worked out

The page's verb is "matches", and the arithmetic fits it. Six tasks out of 813 is 0.74 points. For scale, the standard error of a single pass rate near 92% on 813 tasks is about 0.94 points (derived: the square root of 0.9225 x 0.0775 / 813). That is an illustration of how small the gap is, not a significance test: both models ran the same tasks, which makes a paired difference less noisy than two independent rates, and the page prints no paired results, no repeat runs and no confidence interval, so no test can be run from what it prints.

These counts rest on one assumption: that each of the 813 tasks was scored once as passed or not passed. The page says "task success" and does not say how it is scored, whether tasks were run more than once, or whether partial credit exists. If scoring is not one pass or fail per task, the 750 and 744 are not task counts, and the 6 is not a count of tasks. The percentages themselves are what the page prints.

The price gap is 1% per token; the page says under 2% per task

The footnote's prices are per million tokens at list price. Set side by side, MAI Cyber 2 Flash costs exactly 1% of the GPT-6 Astra price on every line. The diagram plots them on a log axis, with dots rather than bars because length on a log axis does not mean anything, and adds GPT-6.2 Astra at the prices Artificial Analysis lists.

Drawn from the cost note on Microsoft AI's page and from Artificial Analysis's pages for GPT-6 Astra and GPT-6.2 Astra. Ratios and the cached price for GPT-6.2 Astra are derived.

List prices per million tokens. The first two figures are printed in Microsoft's cost note; the last two columns are from Artificial Analysis's pages. Ratios are derived.

LineMicrosoft's note: MAI Cyber 2 Flash / GPT-6 Astra (Azure OpenAI Global)Flash as a share of GPT-6 Astra, then of GPT-6.2 Astra (Artificial Analysis: $8, $0.40 cached derived, $40)
Input$0.10 / $101%, then 1.25%
Cached input$0.01 / $11%, then 2.5%
Output$0.50 / $501%, then 1.25%

Why per token is not per task. Cost per task is price times tokens. With a uniform 1% price ratio, the cost ratio equals 1% times the ratio of tokens used, valued at the Azure prices (derived). 1% therefore means both models used the same number of tokens on average. "Less than 2%" is an upper bound: it holds if Flash used anything below twice GPT-6 Astra's tokens, and it would also hold if Flash used fewer. The footnote's "half the reasoning budget" is not a token count: the page does not define the budget and does not say half of what (the natural reading is half of GPT-6 Astra's, which it does not state). No token count, no dollar figure per task and no run to run spread is printed, so the page's own number cannot be checked from the page.

The framing also changes between Microsoft's own pages. The MAI Cyber 2 Flash page says "less than 2% of the cost per task" and adds that the cost is an offline evaluation cost, "not customer spend". The Ignite announcement says the model "matches GPT-6 Astra's quality at less than 2% of the cost per task" and then, in its closing summary, "frontier cyber defense at 2% of the cost". Neither of those lines carries the offline note. This site reports the differences and does not rank them.

The comparator's price. The page prices GPT-6 Astra on "Azure OpenAI Global", not OpenAI's own API. Artificial Analysis lists GPT-6 Astra at $10 in and $50 out, with a 90% cache discount, which gives $1 for cached input (derived: 10 x (1 - 0.90)). So the two sources agree on all three lines. What each is differs: Microsoft's figures are the Azure OpenAI Global list prices it names. Artificial Analysis's page prints one input and output pair for the model and lists its providers as Amazon Bedrock and Microsoft Azure OpenAI. OpenAI is not listed as a provider for GPT-6 Astra there, so the saved pages do not show whether $10 and $50 is OpenAI's own API price.

Who the page compares with, and who it does not

The diagram sorts the page's comparisons into figures printed, claims with no figure or no comparison, and what is not stated.

Drawn from Microsoft AI's page of 17 November 2026 and Artificial Analysis's page for GPT-6.2 Astra. Red cards are not stated on the saved pages.

The comparators as the page names them, and what the other saved pages add.

ComparatorWhat Microsoft's page saysWhat the other saved pages print
GPT-6 AstraCyberGym-E2E comparator: 91.5%, priced on Azure OpenAI GlobalArtificial Analysis: released September 2026, "This model is deprecated" on its tracking, Intelligence Index 57, $6.24 per Index task, measured as GPT-6 Astra (Max). The Microsoft page does not say which setting GPT-6 Astra ran at
GPT-5.6 CyberWebExploitBench comparator: 38.4% at the same reasoning budget. The body also says Flash outperforms it on CyberGym-E2ENo figure for GPT-5.6 Cyber on CyberGym-E2E is printed. Not on the saved Artificial Analysis pages
GPT-6 CyberNot compared. "Currently available only to select customers"; results "in the coming weeks"Not on the saved pages
GPT-6.2 AstraNot mentioned on the saved Microsoft pagesArtificial Analysis: released November 2026, $8 in and $40 out, Intelligence Index 62, $4.00 per Index task, provider OpenAI
GPT 5.4, GPT 5.3 Codex, GPT 5.4 MiniFootnote 4: the models Insight Enterprises compared against. The body says "GPT Profile"Not on the saved pages

Artificial Analysis's page for GPT-6 Astra says "OpenAI has launched a newer release, GPT-6.2 Astra", and its page for that model says "Released November 2026". The saved Microsoft pages do not mention GPT-6.2 Astra. On Artificial Analysis's own Intelligence Index, GPT-6.2 Astra scores 62 against 57 at $4.00 per Index task against $6.24, which is 35.9% lower (derived: 1 - 4.00 / 6.24). That index is nine evaluations, including Terminal-Bench 4.0 and others, and none of them is CyberGym-E2E, so those figures say nothing about CyberGym-E2E. They show only that GPT-6 Astra, the model Microsoft chose to compare against on 17 November, has a newer sibling that Artificial Analysis says was released in November 2026, the month of Microsoft's page, at a lower list price (derived: $8 and $40 against $10 and $50). Whether Microsoft ran GPT-6.2 Astra on CyberGym-E2E is not stated.

Two comparator gaps are the page's own. The body says the model "outperforms GPT 5.6 Cyber on CyberGym-E2E and WebExploitBench", but the footnotes print only the WebExploitBench pair, so the CyberGym-E2E claim against that model has no number on the page. And the footnote for GPT-6 Cyber explains that evaluating it on "agentic, tool-using benchmarks requires additional safety safeguards that are still being put in place". That is a stated reason for the absence, with a promise of results, and no date.

Labels doing the work of evidence

Several words on the page carry more than the figures behind them. "Leading" is the headline's word, and "leading performance" is the body's, while the only comparative verb for the headline comparison is "matches". "Real-world" describes the benchmark and the tasks ("real-world vulnerability discovery", "real-world security tasks"); the page does not say what the 905 tasks are, who maintains the benchmark, or how success is judged, and this site did not read the benchmark's own pages. "Specialist" and "Flash" describe a model whose size, training data and training method are not stated; the page says only that it draws on proprietary security data. The Ignite announcement calls the same model part of "seven new world class models" and speaks of "frontier intelligence at a fraction of the cost". None of these words is a measurement, and each reads as one.

The page also uses two names. The body says "MAI Cyber 2 (92.3%)"; the title, the heading and the footnotes say "MAI Cyber 2 Flash". The page does not say whether the two are one model. Flash is the name this brief uses for every figure, because the footnotes attach the 92.3% to MAI Cyber 2 Flash.

The claims, the footnotes and what is not stated

This table is the centre of the brief: each claim, where Microsoft's page prints it and what the footnote says, and what is stated and not stated.

Claims on Microsoft AI's page of 17 November 2026, the footnote each rests on, and what is stated and not stated. Footnote numbers follow the markers in the saved page's text.

ClaimWhere printed and what the footnote saysStated, and not stated
92.3% against 91.5% on CyberGym-E2EBody and footnote 1: "task success", Flash "using half the reasoning budget"Stated: both percentages. Not stated: how success is scored, repeats, half of what
Less than 2% of the cost per taskBody and footnote 3: all input, cached input and output tokens, list price, offline evaluation cost, not customer spendStated: the six prices. Not stated: tokens per task, cost per task in dollars
Outperforms GPT 5.6 Cyber on both benchmarksBody, and footnote 2 for WebExploitBench: 40.4% against 38.4%, same reasoning budgetStated: the WebExploitBench pair. Not stated: the CyberGym-E2E figure, the WebExploitBench task count and definition
813 tasks after excluding 92 of 905Footnote 5, "Benchmark integrity": overlap with training data, reference solutions never used, excluded for every modelStated: the counts and the rule. Not stated: which 92 tasks, scores with and without them, the effect on either model
20% more token efficient (Insight Enterprises)Body: against "GPT Profile". Footnote 4: against GPT 5.4, GPT 5.3 Codex and GPT 5.4 Mini, "customer-reported"Stated: the figure and the customer. Not stated: which MAI Cyber model, which tasks, how token efficiency is measured
One million token window, four times 1.1 FlashBody onlyStated: the window. Not stated: 1.1 Flash's window (250,000, derived), accuracy over long inputs
Comparable to leading models on investigation agents and detection authoringBody: "Early evaluations through Microsoft Perception"Not stated: which models, any figure, who ran it, what Microsoft Perception is
Coming to Microsoft security products, replacing the existing model from mid-DecemberBody, "Get started": "eligible customers", "automatically replace"Not stated: who is eligible, the date, which model is replaced, opt out, pinning, rollback, an evaluation window
GPT-6 Cyber not comparedFootnote 6: select customers only, safeguards "still being put in place"Stated: the reason and "the coming weeks". Not stated: a date
Training data and sizeBody: "proprietary security data and expertise"Not stated: the data, the size, the method, the date of the cut off
Independent resultsNone on the page or among the saved pagesNot stated: any third party run of MAI Cyber 2 Flash
False positives and detection qualityNot on the pageNot stated: any rate, on any benchmark or customer data

What the overlap note establishes and what it leaves open

The note is Microsoft's own disclosure. It says four things. Microsoft found the overlap "while reviewing our CyberGym-E2E results". 92 of the 905 tasks "are based on the same open source projects and vulnerability reports as some of our training data". "The benchmark's reference solutions were never used in training." And "to be conservative" Microsoft "excluded these 92 tasks for every model" and reports 813.

That is the right kind of step: a disclosed overlap, handled by excluding the tasks for every model rather than only for one model. What it establishes is the count and the rule. What it leaves open is a list. The note says the reference solutions were not used. It also says the 92 tasks share their source projects and reports with some of Microsoft's training data, and the sentence about solutions does not speak to that: it speaks to solutions, not to the projects. It does not say how the 92 were found, whether any other tasks share sources with Microsoft's training data without having been found, or what GPT-6 Astra's training data contains. It does not print the score of either model on the 92, or on all 905. Excluding the tasks for every model is symmetrical in its rule, and whether it is symmetrical in its effect is not stated. This is a description of what the note says and does not say, and carries no accusation.

The size of the exclusion is the part a reader can weigh. 92 tasks is 10.2% of the benchmark and about 15 times the 6 task gap (derived: 92 / 6 = 15.3). That comparison shows only the relative sizes. The page does not say the exclusion changed the outcome. A reader who wants the figure on all 905 tasks, or on the 92, has to ask for it.

Availability: no outside party can run it, and from mid-December it replaces the existing model for eligible customers

The page's only availability statement is the "coming exclusively to Microsoft security products" sentence. It names no API, no marketplace listing and no weights, so no outside party can run CyberGym-E2E or WebExploitBench on the model from what the saved pages say. Microsoft's MAI Licensed Weights page is about taking a Microsoft model into a customer's own Azure environment, and says it is "coming soon through our Frontier Company engagements". Its customer examples cite MAI Code 2 Flash and Code 2, and the page never names a Cyber model, so this brief does not say MAI Cyber 2 Flash is part of it.

The change in how the model reaches customers is visible between two Microsoft pages. On 27 October Microsoft told existing MDASH customers to "select the MAI Cyber 1.1 Flash profile when starting a new scan", and said the model was available "to verified customers", with usage policies and monitoring. On 17 November the new model "will automatically replace the existing model in the MAI profile for eligible customers". The saved page does not name the model it replaces, does not say whether the 1.1 Flash profile stays selectable, and mentions no opt out, no pinning, no rollback and no evaluation window. What happens to a customer's detection and investigation outputs when the model changes is not stated.

Microsoft's two pages also give different shares of its own product: the 27 October page says MAI Cyber "now accounts for almost 50% of all MDASH scans" and that "nearly 9 in 10 subsequent scans" use the MAI profile after a first scan with it; the Ignite announcement says it "accounts for almost 40% of scans in MDASH" and that 90% of scans that started with MAI "continued to use it". The pages do not say why the shares differ, and this site does not resolve it.

Where the pages differ

Eleven points where Microsoft's pages, or Microsoft's and Artificial Analysis's, differ or leave a gap. This site resolves none of them.

Differences and gaps between the saved pages, as read on 17 November 2026.

PointOne page printsThe other page prints, or what is missing
Model nameBody: "MAI Cyber 2 (92.3%)"Title and footnote: "MAI Cyber 2 Flash". The page does not say whether they are one model
Cost framingBody: "less than 2% of the cost per task"Footnote: prices 1% per token on all three lines, "half the reasoning budget", no token counts. Ignite closing line: "at 2% of the cost"
Reasoning budgetCyberGym-E2E: "half the reasoning budget"WebExploitBench: "the same reasoning budget". Body: GPT 5.6 Cyber compared "at the same reasoning budget" on both benchmarks
GPT-5.6 Cyber on CyberGym-E2EBody: Flash "outperforms" itFootnotes print only the WebExploitBench pair
Customer comparatorBody: Insight Enterprises, "GPT Profile"Footnote 4: GPT 5.4, GPT 5.3 Codex, GPT 5.4 Mini. Not the model in the headline
GPT-6 Astra's priceMicrosoft: $10 / $1 / $50 on "Azure OpenAI Global"Artificial Analysis: $10 and $50, 90% cache discount ($1 cached, derived), providers Bedrock and Azure OpenAI, not OpenAI
GPT-6 Astra's standingMicrosoft: the CyberGym-E2E comparator, no date for the runArtificial Analysis: released September 2026, "deprecated" on its tracking, GPT-6.2 Astra released November 2026
Share of MDASH scans27 October: "almost 50%", nearly 9 in 10 later scans17 November Ignite: "almost 40%", 90% of scans that started with MAI
How customers get it27 October: select the profile for a new scan17 November: "automatically replace the existing model in the MAI profile" from mid-December
Overlap and solutionsNote: 92 tasks share projects and reports with training data"Reference solutions were never used in training" speaks to solutions, not to the source projects
WebExploitBenchTwo percentagesNo task count and no definition, so no task counts can be derived

What to do, in order

What Microsoft says: MAI Cyber 2 Flash is coming to its security products, including Codename MDASH, and will automatically replace the existing model in the MAI profile for eligible customers starting mid-December. The page lists no customer actions. What follows is this site's suggestions.

Take this with you

This site's suggestions, in order

  • Microsoft security customers: ask your account team now which model version runs your detections and investigations after mid-December, whether you can pin the old version or opt out, and for evaluation results on your own tenants and data. Keep a record of detection outputs before and after the switch.
  • Treat a vendor benchmark as a claim. Ask for the task list, the tokens per task, the pass criteria and an independent run, and look for a stated exclusion like this one.
  • Compare cost per completed task on your own work, not per token. The 1% per token price gap on this page says nothing about tokens per task.
  • Do not compare across generations. GPT-6 Astra has been followed by GPT-6.2 Astra on Artificial Analysis's pages. Ask which comparator, which setting and which date.
  • Check how a vendor handles contamination. The overlap here was disclosed by Microsoft and handled by excluding tasks for every model, which is the right kind of step. Record what remains unknown: which tasks, the scores with and without them, and whether other tasks share sources.
  • Read a customer's efficiency figure against its own comparator. The 20% here is against older GPT models, not the model in the headline.

The question the page leaves open

Microsoft ran the benchmark, chose the comparator, set the reasoning budget, priced the tokens, and then found the overlap itself. It deserves credit for the last step. The page prints two percentages that imply a six task gap on 813 tasks (derived), a cost ratio without its tokens, and a replacement month without an opt out. When a vendor's own benchmark is the only measurement and the model replaces your current one automatically, who checked that it is better for your tenant?

Sources

  1. PrimaryMAI Cyber 2 Flash: Leading security. Lower cost, 17 November 2026: the claims, the About these results notes, the benchmark integrity note and the availability sentence. Read in full from a saved copyMicrosoft AIaccessed 2026-11-17
  2. PrimarySeven new world class models at scalable price-performance, 17 November 2026: the same claim restated, the 86% output price statement and the MDASH shareMicrosoft AIaccessed 2026-11-17
  3. PrimaryMAI Licensed Weights: Frontier performance you control, 17 November 2026: weights in a customer's own Azure environment, coming soon; names no Cyber modelMicrosoft AIaccessed 2026-11-17
  4. PrimaryMAI Cyber 1.1 Flash: Sharper findings and fewer tokens inside MDASH, 27 October 2026: the previous model, profile selection and the share of MDASH scansMicrosoft AIaccessed 2026-11-17
  5. Reported byGPT-6 Astra model page: list price, cache discount, Intelligence Index, cost per Index task, providers, deprecation notice. An independent measurement, read from a saved copyArtificial Analysisaccessed 2026-11-17
  6. Reported byGPT-6.2 Astra model page: release month, list price, cache discount, Intelligence Index, cost per Index task, provider. An independent measurement, read from a saved copyArtificial Analysisaccessed 2026-11-17

Share this briefing

Know someone who owns this problem? Send it to them.

Related briefings

The briefing, in your inbox

Practitioner analysis of cyber and AI security news. No vendor noise.

How often

Every new briefing in one email, at 7am, or at 7am, 12:30pm and 6pm. Nothing is sent when nothing is new. Unsubscribe any time.