U.S. Investigators Are Reportedly Looking Into Codecov Breach

U.S. federal investigators are purportedly looking into a security breach at Codecov, a platform used to test software code with more 29,000 customers worldwide, Reuters reported on Saturday. The company has confirmed the breach and stated that it went undetected for months.

According to Reuters, the breach has affected an unknown number of the company’s customers, which include Atlassian, Proctor & Gamble, GoDaddy, and the Washington Post. A security update on the incident written by CEO Jerrod Engelberg published this week did not specify the number of customers affected, either. Gizmodo reached out to Codecov to confirm whether there was a federal probe into the incident, but the company said it did not have any other additional comments besides the Engelberg’s statement on its website.

In the security update, Engelberg explained that the threat actor gained unauthorized access to the company’s Bash Uploader script and modified it, allowing them to potentially access any credentials, tokens, or keys stored in customers’ continuous integration environments as well as any services, datastores, or application code that could be accessed with those credentials, tokens, or keys. The accessed data was then sent to a third-party server outside Codecov.

The company’s Bash Uploader is also used in three related uploaders, Codecov-actions uploader for Github, the Codecov CircleCl Orb, and the Codecov Bitrise Step. All of these were affected as well.

Codecov said it had addressed the vulnerability and that it was safe to use its systems and services. It has not been able to determine who carried out the breach.

“The actor gained access because of an error in Codecov’s Docker image creation process that allowed the actor to extract the credential required to modify our Bash Uploader script,” Engelberg said. “Immediately upon becoming aware of the issue, Codecov secured and remediated the affected script and began investigating any potential impact on users.”

The company added that it had engaged a third-party forensic firm to help it analyze the impact on its users. It also said it had reported the incident to law enforcement authorities and was cooperating with them.

After carrying out an investigation into the incident, the company determined that the threat actor had made periodic alterations of its Bash Uploader script beginning on Jan. 31 of this year. Codecov learned about the breach on April 1 when a customer detected and reported a discrepancy on the Bash Uploader.

Codecov said it emailed affected users on April 15 to the email on file from Github, Gitlab, and Bitbucket and also enabled a notification banner for affected users after they log into Codecov. The company said that customers who use a self-hosted version of Codecov are unlikely affected.

“We strongly recommend affected users immediately re-roll all of their credentials, tokens, or keys located in the environment variables in their CI processes that used one of Codecov’s Bash Uploaders,” Engelberg said.

Reuters pointed out that the incident is being compared to the massive SolarWinds hack, which the U.S. government is attributing to Russia’s Foreign Intelligence Service, because of the possible effects on various organizations and because of the amount of time the attack went undetected. Importantly, the scope of Codecov breach is still unclear.

Codecov stated that it’s taken a number of steps to address security, including rotating all relevant internal credentials, setting up monitoring and auditing tools to make sure that threat actors can’t modify the Bash Uploader again, and working with the hosting provider of the third-party server to ensure it was properly decommissioned, among other actions.

“Codecov maintains a variety of information security policies, procedures, practices, and controls. We continually monitor our network and systems for unusual activity, but Codecov, like any other company, is not immune to this type of event,” Engelberg stated. “We regret any inconvenience this may cause and are committed to minimizing any potential impact on you, our users and customers.”