Software supply chain alert: ‘7 million’ cleartext access tokens in Travis CI logs

Travis CI logs are trivially easy to access—all 770 million of them. And researchers have found lots of sensitive data in a sample.

It’s not the first time we’ve heard these warnings about Travis CI, but the scale of the problem revealed this week is frightening.

The biggest issue is the number of GitHub access tokens. In today’s Secure Software Blogwatch, we fear more software supply chain attacks.

Your humble blogwatcher curated these bloggy bits for your entertainment. Not to mention: 512 floppy drives, etc.

[ Get key takeaways from a survey of 300+ security professionals on software security. Plus: Download the report: Flying Blind: Firms Struggle to Detect Software Supply Chain Attacks ]
 

Rotate your tokens!

What’s the craic? Tara Seals reports—“Exposed Travis CI API Leaves All Free-Tier Users Open to Attack”:

“Sensitive data”
A security flaw in the Travis CI API has left tens of thousands of developers' user tokens and other sensitive information exposed to attack. … The issue was first reported as far back as 2015, but the vulnerability in the API can still be exploited.
…
The Travis CI API is commonly used by developers to test apps. … During their research the analysts were able to access more than 770 million cleartext logs, chock-full of the kind of sensitive data that threat actors could leverage … to launch attacks laterally across the cloud.

 
And Ionut Ilascu adds—“Thousands of GitHub, AWS, Docker tokens exposed in Travis CI logs”:

“Seems to be a recurrent problem”
For a second time in less than a year, the Travis CI platform … has exposed user data containing authentication tokens that could give access to developers’ accounts on GitHub, Amazon Web Services, and Docker Hub. … The researchers found … sensitive strings in the form of tokens, secrets, and various credentials associated with cloud services.
…
Exposing user logs seems to be a recurrent problem for Travis CI, as reports about this type of risk have been published in 2015, 2019 and in 2021.

 
Who are the researchers this time around? Yakir Kadkoda, Ilay Goldman, Assaf Morag and Ofek Itach at Team Nautilus:

“We recommend rotating your keys immediately”
These days [CI/CD is] a major part of modern development and cloud native application pipelines. … These environments usually store many secrets such as access tokens to automatically reach other parts in the cloud or pipeline.
…
We can easily apply an enumeration script to fetch all … available logs. … We randomly sampled … 8 million requests (about 1%) and after we cleaned the data we ended up with about 73,000 tokens, secrets, and various credentials.
…
We disclosed our findings to Travis CI, which responded that this issue is "by design." … All Travis CI free tier users are potentially exposed, so we recommend rotating your keys immediately. … Almost all of … the respective service providers … were alarmed and quickly responded. Some initiated a wide key rotation.
…
Travis CI slowed down the velocity of API calls [but] this was not enough. … There’s no doubt that Travis CI makes an effort to obfuscate secrets and tokens [but still] in 42% of the cases we can get a valid log.

 
What’s the supply-chain angle? Fancy Internet Person explains:

There's a huge downstream problem: … If your software depends, directly or indirectly, on a project that can be hijacked through this, then you're also exposed.
 
For example, you could build software with a 3rd party ORM that uses a json library which had its GitHub credentials exposed. Now a malicious user can push bad code into the json library, which ultimately ends up in your software when you update dependencies in a reasonable fashion.

 
What can we learn? Scott Gerlach:

Leading teams know securing the pipeline ranks as a first step in software delivery. … Here’s a great opportunity for security teams to build bridges with DevOps teams and act as consultants for secure pipelines and infrastructure.

 
And that includes the CD pipeline, as reiella knows:

Minimum permissions can help some, but the bit to acknowledge is the use-case for those credentials. In a lot of them, it is a delivery pipeline, which means the minimum would include 'pushing to production.'

 
Trust a CISO such as Al Berg to lay it on the line:

Very bad news, with wide reaching implications for users of many open source projects. Also raises questions about risks of using “free” (as in beer) products in places where mistakes can lead to breaches and liability.

 
But is it entirely fair to blame Travis CI? sqlrob thinks not:

The root cause of this vulnerability [is] simple, and best practices would have eliminated it entirely. Don't log sensitive information—ever.

 
Meanwhile, Virtual Horus sounds slightly sarcastic:

Ha-ha … you fell victim to one of the classic blunders: … “Never check your secrets into your source repo.” … Only slightly less well known is this: “Never log your secrets when your supply chain is on the line.”

 
And Finally:

A fantastically daft idea, now taken to its illogical conclusion

Previously in And Finally

You have been reading Secure Software Blogwatch by Richi Jennings. Richi curates the best bloggy bits, finest forums, and weirdest websites … so you don’t have to. Hate mail may be directed to @RiCHi or ssbw@richi.uk. Ask your doctor before reading. Your mileage may vary. E&OE. 

Keep learning

• Download the free report: Flying Blind: Firms Struggle to Detect Software Supply Chain Attacks.
• See interactive sample reports to help your team stop software supply chain attacks and improve your security approach.
• Find out about how to secure your CI/CD workflows.

Image sauce: Hugo Jehanne (via Unsplash; leveled and cropped)