P.K. SHARMA

Cyber security intelligence, AI governance, practitioner analysis

Fireworks AI says an unnamed cloud provider's incident widened its breach; update came 34 days after detection

Fireworks AI says a third-party cloud provider's 23 October incident let an unauthorized party reach its development machines on 24 October, adding customer data impact. The update came 34 days (derived) after the first detection on 6 October; the provider is not named.

By · LinkedIn · · 22 min read

34 days from first detection to the update, and a list of what was accessed that grew from one category to six

Fireworks AI's bulletin "Update on October 2026 security incident" is marked "Last updated November 9, 2026" and prints no time. Its first bulletin says Fireworks identified the activity on 6 October, at 9am PDT. From 6 October to 9 November is 34 days (derived). The first bulletin went out at 10pm PDT on 9 October, 31 days before the update (derived), and the update followed the 24 October activity it describes by 16 days (derived). In the first bulletin the thing Fireworks said was captured was one category (derived count): "environment variables, including API keys and tokens stored as secrets". It also said it had "not detected evidence of unauthorized access to customer models, datasets, training data, personal information, or inference traffic". The update lists six categories (derived count: secrets, account information, billing information, resource metadata, datasets and incomplete model weights) and says that for certain customers "a subset of incomplete model weights and datasets were accessed". Two of the five things the first bulletin said it had not detected, customer models and datasets, now appear on the accessed side.

What that does not establish is most of the story. The update does not name the third-party cloud provider, and does not say what that provider's incident was. It prints no number of customers or accounts: the words are "a limited set of accounts". It does not say which customers had datasets or weights accessed, what "incomplete" means for a model, whether any weights were copied out, or how the actor first got hold of the internal credentials in October. It does not say whether the 6 October and 24 October activity is one campaign, though it frames both under one bulletin and refers to "the threat actor" in the singular. It gives no notification dates for the new impact and no regulator filing, and it names no actor. All of this is Fireworks's own account as read from 10:44 UTC on Tuesday 10 November 2026; the page carries the status "Active monitoring" and says "Our investigation is ongoing and may reveal additional information."

A note on interest, once: Fireworks sells a hosted inference and training platform and its bulletins are its own account of an incident on its own systems; the update describes Palo Alto Networks Unit 42 as a cybersecurity firm helping with forensics and prints none of its findings; and Google Threat Intelligence Group and Mandiant, whose post the update cites, sell security services too (the post carries a "Contact Mandiant" offer). The sources here are Fireworks's two bulletins and its bulletins index, the Google and Mandiant post, and one secondary, BleepingComputer, which this site used to check how the story was reported and not as the source of any figure.

What the first bulletin said, as it stood on 10 October

The incident. "On October 6, Fireworks identified a security incident involving unauthorized use of internal Fireworks credentials." The bulletin says an unauthorized third party "accessed environment variables used in evaluation jobs in a limited number of customer accounts". It does not say how the internal credentials came to be used by that party. Its later section says Fireworks believes "environment variables, including API keys and tokens stored as secrets, were captured in certain affected accounts".

Containment and notice. "We began containment the same day and have directly notified every identified affected customer." The timeline, in PDT, is in Technical details: notification began at 7am on 7 October, employee API keys were rotated at 4pm on 9 October, identified affected customers were notified at 6pm that day and the bulletin was published at 10pm (05:00 UTC on 10 October, derived). On 10 October Fireworks revoked more user API keys and rotated service account tokens after a credential review that it said was continuing. The bulletin names no firm and no agency: "We're working with a cybersecurity expert firm to assist with the incident."

The reassurance. The bulletin says Fireworks "does not store prompts or outputs by default, as our platform is ZDR", and says the review of calls made with the exposed credentials was still under way: "our review of other potential access remains ongoing". Read with the update, the lesson is in the tense. The first bulletin reports what had been detected at that time, and the update, 31 days later (derived), reports more. The first bulletin also said Fireworks was reviewing how secrets reach evaluation jobs; the update does not say what came of that review.

What the update adds, and the list that grew

The new paragraph. "A third-party cloud provider that hosted Fireworks R&D machines experienced a security incident on October 23." Then: "the unauthorized third party used it to reach Fireworks development machines and then run an unauthorized workload that led to additional customer data impact". Fireworks says "We detected this activity on the same day", and that "As of October 25, we have not detected further evidence of successful unauthorized access by the threat actor on the Fireworks platform customers use." Two details deserve a careful reading, and both are inference. The unauthorized party is said to have used the provider's incident to reach Fireworks machines, which on the update's wording places the route through a supplier, though Fireworks does not say how. And the 25 October statement is scoped to the platform customers use: it does not speak of the development machines, so it is not a statement that the machines were clean.

The categories. The update's "Data involved" section opens "The affected internal application provided access to the following categories of information:" and lists three: resource metadata, billing information, and model weights and datasets. For billing it says "Payment card numbers are not stored by Fireworks and were not impacted." Separately, the opening paragraph says Fireworks "discovered impacts to account and billing information" and, for a limited set of accounts, impact to secrets, datasets and incomplete models. The table below sets each version beside the other, copied from the pages.

Drawn by this site from Fireworks's first bulletin and its update, read 10 November 2026. The statements are Fireworks's own.

What each version of the Fireworks bulletin says about each kind of data, copied or closely quoted from the pages read 10 November 2026

Kind of dataFirst bulletin (9 Oct, updated 10 Oct)Update (9 Nov)
Secrets and environment variablesCaptured: "environment variables, including API keys and tokens stored as secrets" in certain affected accounts"only Fireworks stored secrets (e.g. used in evaluation jobs) were known to have been accessed" at the time of the first bulletin; now impact to secrets for a limited set of accounts
Account informationNot listed"impacts to account and billing information"
Billing informationNot listed"billing and shipping addresses, invoice and payment history, account balance, and credit usage"; card numbers not stored and not impacted
Resource metadataNot listed"Fireworks resource identifiers, display names, descriptions, creation time, etc."
DatasetsNo evidence detected of unauthorized accessAccessed, a subset, for certain customers
Model weightsNo evidence detected of unauthorized access to customer models"a subset of incomplete model weights" accessed, for certain customers
Training dataNo evidence detected of unauthorized accessNot named as accessed. Training SDK is described as ZDR, "no training data stored at rest"
Personal informationNo evidence detected of unauthorized accessNeither repeated nor withdrawn. Billing and shipping addresses are listed, and are not labelled personal information
Inference traffic, prompts, outputsNo evidence detected; "Fireworks does not store prompts or outputs by default""We have not detected any impact to Inference, Training SDK and Nexus", all ZDR by default, "based on our investigation to date"

What Fireworks says it has done. The update lists four heads of hardening with Unit 42: access and credential lockdown (credentials and service accounts revoked and replaced, unnecessary elevated privileges removed), network isolation, data protection, and forensic and audit integrity, meaning secured and enhanced logging and monitoring. It also says "we've disabled self-serve dataset uploads and downloads used by Batch Inference and dataset-based training while we investigate this incident". A customer who relies on dataset upload for batch inference or dataset-based training therefore may find that path closed as of this reading, and the update says it will give further updates as appropriate. Unlike the first bulletin, the update has no section on what affected customers should do. It says "We are notifying customers whose data was impacted, and providing details about the impact and recommended response actions." The actions are in the notification email, which this site cannot see.

The clock: the dates Fireworks gives, and the ones nobody gives

The house rule is that the access window, detection, containment, notification and publication are separate dates. Fireworks gives detection and containment for the first activity (6 October), notification of customers (7 and 9 October) and publication (9 October). For the second it gives the provider's incident date (23 October), the date of the reach into Fireworks machines (24 October), detection the same day, and a status date (25 October). It gives no start date for the first access, no notification dates for the new impact, and no date for when the update's other findings were made: the update says only "Since then, we discovered".

Drawn by this site from Fireworks's two bulletins and the Google post, read 10 November 2026. Day counts are derived.

The dates, who stated each, and the intervals this site counted (derived by script from the stated dates; Fireworks's first bulletin times are PDT, UTC minus 7)

Date (2026)What it isSource, and interval
6 October, 9am PDTFireworks identified the activity and began containmentFirst bulletin. 34 days before the update (derived)
7 October, 7am PDTStarted notifying affected customersFirst bulletin. 1 day after identification (derived)
9 October, 4pm to 10pm PDTEmployee API keys rotated; identified affected customers notified at 6pm; bulletin published at 10pmFirst bulletin. 3 days after 6 October (derived); 31 days before the update (derived)
10 October, 11:16 PDTLast update of the first bulletin, by its own stamp; user API keys revoked and service account tokens rotatedFirst bulletin and the bulletins index. 13 days before 23 October (derived)
23 OctoberA third-party cloud provider that hosted Fireworks R&D machines had a security incidentUpdate. 17 days after 6 October (derived)
24 OctoberThe unauthorized third party reached Fireworks development machines and ran a workload; detected the same dayUpdate. 1 day after the provider's incident (derived); 16 days before the update (derived)
25 October"As of October 25" no further successful unauthorized access detected on the platform customers useUpdate. A status date, not a containment date. 15 days before the update (derived)
2 NovemberGoogle Threat Intelligence Group and Mandiant publish an LLMjacking post; the post does not mention FireworksGoogle post, page data 2 November 2026. 7 days before the update (derived)
9 NovemberFireworks updates its bulletin; no time printedUpdate and bulletins index. 34 days after 6 October (derived)
Not statedWhen the provider's incident began and ended; when affected customers were told of the new impact; when the other findings were made; any regulator filingThe update is silent on each

Read the table from the bottom up. The update says detection was the same day as the 24 October reach, which is a strong statement about the second activity. By the pages' own stamps the public bulletins then carry nothing about it until 9 November, 16 days later (derived). Customers may have been told directly in between; the pages do not say. One more reading, labelled as inference: the update frames the 23 October incident as the provider's own, which leaves open whether the provider had told Fireworks, or Fireworks found it first. The page does not say.

Careful wording: "provided access to", "limited", "incomplete" and "evidence indicates"

Several phrases in the update do the work of a finding, and each is worth reading slowly. "Provided access to." The data section says the affected internal application "provided access to the following categories of information". Only the third item, model weights and datasets, goes on to say "were accessed". For resource metadata and billing information the page says what the application could reach, and the opening paragraph says Fireworks "discovered impacts to account and billing information". That is an impact, not a count of records read. The update does not say whether every item on the list was read, copied or both, so this site does not.

"Limited" and "incomplete." Fireworks says "a limited set of accounts" and "a subset of incomplete model weights and datasets". No number, no fraction and no definition of an incomplete model is given. A partly trained checkpoint and a fine tuned adapter are different things to a customer, and this site does not guess which is meant. What is established is that, for certain customers, Fireworks says datasets and incomplete weights were accessed. Where those datasets and weights sat, and whether the customers had uploaded them through the dataset path now disabled, is not stated.

"Zero data retention." The update says "We have not detected any impact to Inference, Training SDK and Nexus, which are all zero data retention (ZDR) by default." The label describes a default retention setting. As inference, it is not a control over the other places data lives, and the update itself says datasets and incomplete models were accessed, so some customer data was somewhere other than the ZDR products. The sentence is also hedged by its own next line, "These findings are based on our investigation to date." A customer who used the dataset uploads and downloads for Batch Inference or dataset based training, the path the update says it has disabled, should read the notification email, not the label.

"The evidence indicates links." The update says "The evidence indicates links to a cybercriminal threat actor group, which is financially motivated", aimed at "selling tokens from unauthorized access to Large Language Models (LLMs) to illicit routing gateways", known as "LLMJacking". That is Fireworks's assessment, made with a firm it names (Unit 42), and the evidence is not published: no indicator, no group name and no tie to a specific incident are printed. It also does not say which of the two sets of activity the sentence covers. A link is not an identification, and an intent to resell access is a statement about what the actor wants, not about what Fireworks found it did. The update does not say that any customer was billed for use they did not make.

What the Google and Mandiant post says about LLMjacking, and what it does not say about Fireworks

The update cites a post by Google Threat Intelligence Group and Mandiant, "The Rise of LLMJacking", which its page data dates 2 November 2026, 7 days before the update (derived). Read in full, it does not mention Fireworks. Its subject is other incidents: it says Mandiant "responded to multiple incidents in 2026 in which threat actors hijacked enterprise access to Large Language Models (LLMs)", and that across individual incidents it saw charges "exceeding $100,000 USD in a single day, with cumulative charges at some organizations reaching $1 million USD before access was revoked".

It describes a division of labour: actors who gain initial access, and operators of downstream resale. In its words, "LLM Mixers function as illicit routing gateways that proxy, aggregate, and resell unauthorized LLM access to underground subscribers." That matches the phrase Fireworks uses, illicit routing gateways, and Fireworks cites the post for the technique and the motive. The post says "the dominant initial access vector has been exposed or stolen cloud identity tokens" in the investigations it covers. Whether anything of that applies to the Fireworks activity is not something either page says: the Fireworks update does not describe how the first credentials were obtained, and the Google post does not describe Fireworks.

The post's defender advice is aimed at cloud and model accounts in general and is useful to a Fireworks customer even though it was not written about Fireworks. It tells readers to set spending limits, "Utilize spending caps to set budgets and avoid over-spending.", to "Restrict and segment API keys by specific model endpoints, environments (dev/prod), and usage limits", and to enforce phishing resistant multi factor authentication across cloud identity providers. Some of its detection lines are in Technical details. They name AWS, Azure and Google Cloud operations, and a Fireworks customer should not assume that those are the logs available for Fireworks keys.

What Fireworks states, and what it does not

Stated by Fireworks in the update or first bulletin (read 10 November 2026), and what is not stated. Derived items are marked

QuestionStatedNot stated
Who hosted the machines"A third-party cloud provider that hosted Fireworks R&D machines"The provider's name, and what its incident was
What the actor reachedFireworks development machines, on 24 October, then an unauthorized workloadWhat the workload did, how long it ran, what it read
Data involvedResource metadata; billing information; for certain customers a subset of incomplete model weights and datasets; secrets for a limited set of accountsHow many records or accounts; what incomplete means; whether anything was copied out or used
Customers"a limited set of accounts"; affected customers are being notifiedA number, any name, notification dates for the new impact
Card dataCard numbers are not stored by Fireworks and were not impactedWhether any other payment detail was reachable beyond the listed billing data
The first accessUnauthorized use of internal Fireworks credentials, detected 6 OctoberHow the credentials were obtained
One campaign or twoOne bulletin, one threat actor referred to, a section titled Further activityWhether the provider's incident and the October credential use are linked
AttributionEvidence indicates links to a financially motivated group; the term is LLMJackingThe group's name, the evidence, which activity it covers
AuthoritiesLaw enforcement notified (first bulletin silent on this)Which agency, when, and any regulator filing
ContainmentCredentials revoked and replaced, network isolation, logging secured, dataset upload and download disabledWhen each step was completed; whether the access is closed, beyond "As of October 25"
ProductsNo impact detected to Inference, Training SDK and Nexus, ZDR by defaultMore than "based on our investigation to date"

The negative column is long on purpose. Fireworks's update is specific about the shape of the second event and silent on its size, and for a customer deciding what to rotate, the size is the missing part. Every cell in the right hand column is a question a customer can put to Fireworks by replying to the notification email, and the update gives security-notice@fireworks.ai as the contact.

Where the reporting differs from the primary

This site read one news report, BleepingComputer's, dated 10 November 2026 by its page data (02:49:58 at UTC minus five hours, which is 07:49 UTC, derived). It says the new breach was disclosed in "Fireworks noted in a Monday advisory", and Monday was 9 November (derived). Where it goes further than the Fireworks pages, this article follows the Fireworks pages.

Differences between the BleepingComputer report and the Fireworks pages, as read 10 November 2026

TopicBleepingComputer saysFireworks pages say
When the LLMJacking link was madeQuotes the link and adds "Fireworks said at the time", placing it in the early October incidentThe first bulletin has no mention of LLMJacking, law enforcement or Unit 42; the sentence is in the 9 November update
23 October"Fireworks discovered another security incident on October 23"The provider's incident was on 23 October; Fireworks says it detected the activity on 24 October, the same day as the reach
Which data"In the new breach, the threat actors accessed resource metadata", billing information and incomplete weights and datasetsThe application "provided access to" the categories; the opening paragraph also lists secrets and account information and does not tie each item to one event
One incident or two"a new security breach" and "another security incident"One bulletin titled October 2026 security incident, with a section called Further activity
CustomersNames no customerNames no customer
ContactNot givenThe first bulletin gave security at fireworks.ai; the update gives security-notice at fireworks.ai

On customers: no Fireworks page this site read names a customer, and the BleepingComputer report names none. This site read no other press coverage, and states no customer as affected.

What to do, in order

Fireworks's own steps are in the first bulletin, for customers who received a notice, and they are about the October credential exposure. The update prints none, and says notified customers get response actions by email. They are reproduced first. This site's suggestions follow, labelled as such.

Take this with you

Fireworks's steps, as printed in the first bulletin (10 October)

  • Replace revoked Fireworks API keys if you still need them, then update the applications that used them.
  • Rotate third-party credentials identified in your notice, with the providers that issued them. Removing a stored secret alone does not invalidate the underlying credential.
  • Review data and activity associated with exposed third-party credentials, including for unauthorized reads, downloads or changes, even if you find nothing suspicious.
  • Check for unrecognized resources or activity: API keys, users, evaluators, training jobs and datasets. Send Fireworks the names or IDs of anything you do not recognize.
  • If you have not received an affected-customer notice, Fireworks says no incident-specific action is currently required; contact it if you observe suspicious activity or need to confirm your account's status.

Take this with you

This site's suggestions, not Fireworks's advice

  • Read any Fireworks notice in full and keep a dated copy. Reply to it with the questions in the right hand column of the table above that apply to your account.
  • Rotate every Fireworks API key and every secret you stored on the platform, whether or not you were told you were affected, and rotate the third party credentials behind those secrets at their issuers. Fireworks says the actor aims to sell access to models; as inference, that makes keys and quotas likely targets.
  • Check billing and usage on Fireworks and on every provider whose key you stored there for spikes you do not recognise. The update does not say any customer was charged for use they did not make; the Google post describes charges that reached over 100,000 US dollars in a day at other organisations.
  • Set spending caps and alerts, and scope each API key to the model, environment and usage it needs, as the Google post advises.
  • List the datasets and fine tuned weights stored on any hosted AI platform, decide whether each needs to be there, and note that Fireworks has disabled dataset uploads and downloads for batch inference and dataset based training for now.
  • Ask each AI platform you use which third party clouds host its development and research machines, and whether a supplier's incident would reach customer data.
  • Keep a record of which of your data sets and keys sit where, so that the next vendor bulletin that widens can be answered in an hour and not in a week.

Technical details

What the primary prints. Neither Fireworks bulletin prints an indicator of compromise: no address, domain, hash, file name, key prefix, tool name, session pattern, query or detection rule, and no logs. So there is nothing to defang or to load into a blocklist from the Fireworks pages, and this site has not invented any. What the pages do print is a timeline, one product restriction, and contact addresses. They are reproduced below with what to do with them.

The first bulletin's timeline. This is the table from the first bulletin, copied from the page text, in PDT. A defender uses it to line up their own logs: this site suggests reviewing any use of a Fireworks API key or stored secret from 6 October onward that you cannot explain, and the first bulletin says the exposed credentials covered environment variables in evaluation jobs. The separator bar is added by this site; the dates and text are Fireworks's.

Text
Oct 6 9am, 2026 | Fireworks identified the activity and began containment.
Oct 7 7am, 2026 | Started notifying affected customers
Oct 9 4pm, 2026 | Rotated Fireworks employee API keys
Oct 9 6pm, 2026 | Notified identified affected customers
Oct 9 10pm, 2026 | Published bulletin
Oct 10, 2026 | Revoked additional Fireworks user API keys and rotated service-account tokens identified in a completed credential review. Further credential review and rotation are continuing.
Fireworks first bulletin, incident findings and response timeline, all dates in PDT, as read on 10 November 2026. The bar between date and text is added by this site.

The restriction the update announces. This is a change to how the platform works for customers, as Fireworks printed it. A team that uses dataset upload for batch inference or dataset based training should expect that path to be closed while the investigation runs, plan around it, and ask Fireworks when it will return.

Text
we've disabled self-serve dataset uploads and downloads used by Batch Inference and dataset-based training while we investigate this incident. Training SDK remains available with ZDR, i.e. no training data stored at rest, and Fireworks Lab-based training operates in an isolated environment with additional network and storage protection.
Fireworks update, What we have done, as read on 10 November 2026.

Hunting lines from the Google and Mandiant post. These are not Fireworks indicators. They are operation names the Google post says to monitor for LLM hijacking on AWS, Azure and Google Cloud, and a line about validation requests, copied from the post's guidance. They apply to a cloud account in which a model service might be enabled or invoked, including one holding a stolen key. A defender checks that the logs behind them are switched on (the post notes that Google Cloud inference data access logs are off by default), then searches for them. The labels in the block are this site's grouping. The two network indicators the post publishes concern other incidents, and nothing links them to Fireworks, so they are not repeated here.

Text
AWS CloudTrail, events from bedrock.amazonaws.com
  direct execution of the model: Converse, InvokeModel, and InvokeModelWithResponseStream
  error spikes that can indicate spraying: ThrottlingException, ValidationException, or ResourceNotFoundException
  logging changes, high severity if not from infrastructure as code: PutModelInvocationLoggingConfiguration and DeleteModelInvocationLoggingConfiguration
Azure, Microsoft.CognitiveServices namespace
  unauthorized creation of new instances: Microsoft.CognitiveServices/accounts/deployments/write
GCP, Cloud Audit Logging
  enablement of the platform services: google.api.serviceusage.v1.ServiceUsage.EnableService
  direct execution of the model: google.cloud.aiplatform.v1.PredictionService.GenerateContent
Any provider, gateway and API access logs
  validation sweeps: anomalous, low-volume requests (such as max_tokens=1 or zero-temperature pings)
Google Threat Intelligence Group and Mandiant post on LLMjacking, 2 November 2026 by its page data, detection and monitoring sections, as read on 10 November 2026. Group labels are added by this site.

The question the update leaves open

Fireworks has said, in two bulletins 31 days apart (derived), what it had detected at the time of each. The second one adds a supplier the first never mentioned and widens the list from one category of data to six (derived). The question that exposes the gap is the one the update never answers: if the first bulletin was right about what had been detected on 10 October, and the update is right about what has been detected since, then what is the date after which a customer can treat a Fireworks statement about the data it holds for them as complete, and who tells them when that date has arrived?

Key facts

Sources

  1. PrimaryUpdate on October 2026 security incident, last updated 9 November 2026: the further activity, data involved, attribution sentence and hardening. Read in full. Fireworks sells an inference and training platformFireworks AIaccessed 2026-11-10
  2. PrimaryFireworks October 2026 security incident, last updated 10 October 2026, 11:16 PDT: the first bulletin, its timeline and customer steps. Read in fullFireworks AIaccessed 2026-11-10
  3. PrimarySecurity bulletins index: lists the two bulletins with their last updated stampsFireworks AIaccessed 2026-11-10
  4. PrimaryThe Rise of LLMJacking, Google Threat Intelligence Group and Mandiant, 2 November 2026 by page data: the post the update cites. Read in full; it does not mention Fireworks. Mandiant sells security servicesGoogle Cloudaccessed 2026-11-10
  5. Reported byFireworks AI discloses new breach after cloud provider hack, 10 November 2026 by page data: a secondary used to compare how the update was reported, and for no figureBleepingComputeraccessed 2026-11-10

Share this briefing

Know someone who owns this problem? Send it to them.

Related briefings

The briefing, in your inbox

Practitioner analysis of cyber and AI security news. No vendor noise.

How often

Every new briefing in one email, at 7am, or at 7am, 12:30pm and 6pm. Nothing is sent when nothing is new. Unsubscribe any time.