Today, code is everywhere, especially in the cloud world. This is a no-brainer if you develop applications, but code can be found in other places too. Several people manage their infrastructure using code and tools like Terraform. Platform teams manage deployment tools and orchestration systems (e.g. Kubernetes) using code like YAML. Even in security, people use code like Rego to manage policies, and code like YARA to detect threats. Code sits in repositories and flows through pipelines, which can be attacked. In fact, it is being attacked widely. And that brings us to today's topic - namely the software supply chain, and its associated threats and vulnerabilities.
What is the Software Supply Chain?
Software applications are generally made up of source code that you write, plus third-party code in the form of software dependencies. A build process combines different sources and gives you packaged software, which can then be run by your organisation or consumed by others as dependencies, and eventually delivered to end users as products and services. This process is effectively the software supply chain, and is broader than the traditional software development life cycle (SDLC) in that it includes the entire ecosystem of software components and dependencies involved, not just first-party software.
As we've seen repeatedly in the past few years (e.g. SolarWinds, Codecov, Log4J, and others), attacks can occur at each link of the software supply chain, and are increasingly disruptive and costly to the unsuspecting victims. Let's examine each link of this critical chain and consider where things can go wrong.
Software Supply Chain Threats and Vulnerabilities
The following figure was published by the OpenSSF SLSA (short for Supply chain Levels for Software Artifacts) project, which is a specification for describing and improving supply chain security. SLSA primarily focuses on supply chain integrity, with a secondary focus on availability. I'll cover SLSA in a future post, but let's review their handy representation of supply chain threats and vulnerabilities here.
Let's consider the end-to-end supply chain, starting from a developer commit.
- [A] Submit unauthorised change: Keeping secure development practices (and developer mistakes) aside for a moment, the first threat to the supply chain is the submission of unauthorised code to the source repository. Of course, you could argue that the former leads to significantly more risks, but for this item, we'll stick to unauthorised changes only. As an example, a contractor with legitimate access at SushiSwap pushed a malicious commit to the repository and siphoned cryptocurrency.
- [B] Compromise source repo: While managed source repositories like GitHub and GitLab are on the upswing now, a lot of code still resides in private repositories, which are just self-hosted Git or SVN servers, where a host compromise can easily lead to unauthorised access. Even with managed repositories, not everyone changes the defaults, which tend to be more open than secure. The 3CX VOIP client is a recent case study, where attackers compromised their GitHub repository and injected malicious code, which was then packaged and delivered to unsuspecting 3CX customers.
- [C] Build from modified source: In this case, the attacker builds from a version of the source code other than the official source control repository. This could happen if the build was executed from unofficial forks, branches, tags, build steps or parameters, or from code modified after checkouts. In the case of Webmin, a version was released with a backdoor after the development build server was exploited.
- [D] Use compromised dependency: A fairly common threat, we've seen this most recently with the Log4j brouhaha, but it has also happened several times before (e.g. OpenSSL, Spring, Jenkins, and more). As the reliance on open source software grows, so does the risk of compromised dependencies, irrespective of whether they are build or runtime.
- [E] Compromise build process: If you are even remotely attached to security or information technology, you cannot have missed the SolarWinds attack in 2021, which left even large organisations like Microsoft and Federal Government agencies reeling from the impact. SolarWinds had their build system compromised and malicious code injected into their widely used Orion IT monitoring and management system, resulting in a not-so-friendly gift of malicious backdoors and sensitive data leakage for their customers.
- [F] Upload modified package: At CodeCov, an attacker used leaked credentials to upload a malicious artifact to their cloud storage bucket, from which their users downloaded files without suspecting anything amiss. This could have also happened if the artifacts were built with untrusted CI/CD tools.
- [G] Compromise package repo: Just like the source repository, the package (or artifact) repository could also get compromised. We saw glimpses of this threat when researchers ran mirrors for several popular package repositories; while they didn't really do anything malicious, a rogue maintainer or an attacker could easily have done so.
- [H] Use compromised package: Finally, an attacker could skip all this and simply upload a rogue package. Or they could do something more niche, like dependency/namespace confusion or typosquatting - just ask browserify. In their case, the attacker uploaded a malicious npm package with a similar name as the original.
So yes, the software supply chain has issues, but how do we fix them? Especially, in the decentralised, open-source world. Well, there is no silver bullet but, we can make a dent if we all get together, raise awareness of the challenges, adopt open standards, and implement incremental changes in the way we currently develop and deploy software. More on this in upcoming posts, stay tuned! Until then, if you are interested in open source security, check out this post.