A BGP hijack delivered a malicious Virtualizor update, and the TLS certificate on it was genuine

Between 20:57 UTC on 28 August and 06:10 UTC on 30 August, a network called NexonHost announced a block of address space it did not own. The block, 162.55.80.0/24, belongs to Hetzner and holds the servers Softaculous uses to distribute updates.

For about thirty-three hours, in two waves separated by an eleven hour lull, update traffic from Virtualizor installations went somewhere else. Some of those installations checked for updates during the window and received a malicious package. The indicator of compromise Virtualizor published is a systemd unit at /etc/systemd/system/java-jre-update.service.

Nothing inside Virtualizor was breached. No developer account was phished, no build server was accessed, no signing key was stolen. The attacker changed where the internet thought a set of IP addresses lived.

Three checks, and what each one actually proved

Three things were meant to make that update trustworthy

The middle row is the one worth sitting with.

The attacker obtained a genuinely valid TLS certificate from Let's Encrypt during the window. Not a forged one, not a stolen one, not one from a compromised CA. Let's Encrypt validates domain control by reaching out to the domain and checking who answers. That validation request was routed through the same hijack, so the answer came back from the attacker, and the certificate was issued correctly according to every rule the CA operates by.

Affected connections produced no warning, because there was nothing to warn about.

That is the part most write-ups skip, and it inverts the intuition almost everybody carries about TLS. A certificate does not prove you reached the organisation named on it. It proves you reached whoever could answer for that name from where the certificate authority was standing, at the moment it asked.

The check that was not there

Virtualizor's advisory is unusually direct about why the package landed: "product update clients did not yet cryptographically verify update packages."

That is the whole story in one sentence. Two checks were upstream of the attacker and passed honestly, and the one check the attacker could not have influenced did not exist.

Package signing is decided at build time, on a key the attacker never touched, and verified on the host after the download. Routing has no opinion on it. It is the only one of the three that would have caught this, and it is the one that had not been implemented.

To their credit, Virtualizor say they will implement code signing, report the fraudulent certificate to Let's Encrypt, and move infrastructure. Version 3.2.9.9 shipped on 1 September with a Security Analyzer tool in the admin panel.

The figure to be careful with

What is established, and what is one party's account

Claim	Source	Status
28 Aug 20:57 UTC to 30 Aug 06:10 UTC, two waves, an 11 hour lull	Virtualizor advisory	Vendor statement
AS62390 announced 162.55.80.0/24, spoofing AS24940 in the path, transit via AS6204	Virtualizor advisory	Vendor statement, and checkable against public routing archives
A valid Let’s Encrypt certificate was issued during the window	Virtualizor advisory	Vendor statement
Update clients did not cryptographically verify packages	Virtualizor advisory	Vendor statement, and the root cause they name themselves
IoC: /etc/systemd/system/java-jre-update.service	Virtualizor advisory	Vendor statement
“A handful of servers” received the package	Virtualizor advisory	Vendor characterisation. They also say they cannot produce a definitive affected list
5 of 34 hypervisors root compromised	One hosting provider’s own account	Single source. Not a vendor figure, and not independently confirmed

The last row is circulating widely without its attribution. It may well be accurate. It is one operator describing their own estate, and it is being read as a sample rate for everybody, which it is not.

The vendor figure is "a handful of servers", and it is paired with an admission that matters more than the number: Virtualizor say they cannot produce a definitive list of affected installations from the attacker's logs. That is why every operator has to check rather than wait to be told.

What to do if you run Virtualizor

The position

The uncomfortable part of this incident is how little of it was anybody's mistake in the ordinary sense.

BGP accepted a more specific prefix, which is what BGP is specified to do. Let's Encrypt issued a certificate to a requester who demonstrated control of the domain, which is what it is specified to do. The update client downloaded from the right hostname over a valid TLS connection and installed what it received, which is what it was written to do. Each layer behaved correctly and the composition was wrong, because two of the three checks were answered by the same party the attacker was pretending to be.

The general lesson is not about hosting panels. Any product that ships updates over the public internet and verifies them only with TLS has the same shape, and the number of those is not small. The question to ask of your own suppliers is narrow and answerable: is the update signed with a key, and does the client verify the signature before installing. Everything else in the chain is decided by routing, and routing is not something your vendor controls.