Chainguard CEO Dan Lorenc just posted his account on LinkedIn of why Chainguard shipped fixed PHP builds before the embargo on this week's two critical remote code execution bugs lifted. He says reports of exploits in the wild drove the decision. Nils Adermann, a PHP Foundation board member, replied on the post that those reports weren't true.

The early builds are one of the main complaints in the PHP Foundation's account of the release. It says Chainguard published images with the fix at 07:41 UTC on October 21, more than six hours before the coordinated release, and that this "disregarded weeks of work by many parties toward a coordinated disclosure." If you haven't updated yet, start with our article on the two vulnerabilities.

Lorenc's Account

Lorenc opens by praising the people on the other side of the decision:

First of all, the PHP Foundation and maintainers did an amazing job here in a high stress situation. I'm sorry that my decision led to even more stress for them.

Here's his timeline, as he tells it:

  • Friday, October 16: One of the original researchers told Chainguard something was coming in PHP. That night, Chainguard found the same issue on its own, using its own tooling and a frontier AI model with a basic prompt to find bad vulnerabilities in PHP.
  • Saturday, October 17: Chainguard got the full details, confirmed some of its findings matched the original report, and helped test the fixes.
  • Monday and Tuesday: News started to leak. Lorenc says that by Tuesday night, several other security vendors in the disclosure group had confirmed seeing exploits in the wild.
  • Tuesday night: After a video call with several other researchers, Lorenc decided to ship the fixed builds and says he immediately notified the group.

He says the decision came down to three things: how little effort it took Chainguard to find the bug and build an exploit, how much was already public about the coming fixes, and the evidence of active exploitation. He says the coordinated disclosure guidelines allow releasing early once there's evidence of exploits in the wild, and that Chainguard made the builds public for anyone, not just its customers.

Lorenc says Chainguard is working through hundreds of disclosures like this one every day and has been for the last three months. He links to a Chainguard blog post on its approach and closes by calling this "the new normal for the industry."

The Dispute Over Exploits in the Wild

Nils Adermann, the co-creator of Composer, replied on the post. He says he wrote the PHP Foundation's statement and that Lorenc misrepresents what happened.

His main point is about the exploit reports. He calls the claim that vendors confirmed exploits in the wild false:

There were no confirmed exploits. All that was reported on the Slack channel you are referring to, was people probing, trying to exploit.

He adds that at the time of the release, there was still no report of a successful exploit. That matches the Foundation's post, which says the security team "did not identify any functional exploits at the time of release/public disclosure."

Adermann also says the embargo was reviewed again and again as new information came in. He calls Chainguard's move a decision "taken unilaterally without any communication with the PHP team." That conflicts with Lorenc's statement that he notified the group right away. Neither post says exactly who was told, or when.

My Take

I think breaking the embargo was a bad call, and the reasons in Lorenc's post don't change that.

An embargo exists so that distributions, hosts, and vendors can all have fixed builds ready the moment the details go public. According to Adermann, close to 100 vendors were working to this one. The Foundation says distributors were still packaging fixes. Every one of them held back so that nobody would be left exposed while someone else had a head start. Chainguard decided, without the PHP team, that its read of the risk outweighed everyone else's, and everyone else had to scramble.

The reason he gives doesn't add up either. The exploit reports he leaned on were, according to the person who wrote the Foundation's statement, people trying and failing. The security team says it found no working exploits at release. If reports of probing are enough to end an embargo early, there's no such thing as an embargo, because someone is always probing.

The coordinated release was also only hours away, and it had already been moved up a day because of the leak. Shipping at 07:41 UTC instead of 14:00 bought about six hours. The cost of those six hours landed on the maintainers and distributors who'd spent weeks getting this right.

"We're all on one team here," Lorenc writes. On a team, one member doesn't get to change the plan without the people who own it and tell them after.