Over the last two years, the software supply chain has become one of the most pressing and relevant security issues for organisations. In 2022, the number and severity of the attacks increased with no signs of slowing down. The majority of these attacks fell into one of three categories: package manager attacks, developer compromise, or large software vendor breaches. In this post, we will explore each of these and how you can defend against them.
Package managers have been of increasing interest for hackers over the past few years with attacks against the packages fundamental to so much of our software. Not only this, but there has been an upward trend of attacks against the managers themselves. The managers such as PyPI, Maven and the most popular npm’s track record of security leaves a lot to be desired. Research like dependency confusion showed how they could be leveraged to attack some of the largest organisations in the world.
ReversingLabs, a company focused on threat hunting and supply chain security have released a report, specifically looking at attacks against package management software. They have seen a continual increase of known vulnerabilities in this software, up nearly 300% in the last four years. This includes attacks where malicious packages could masquerade as reputable ones, and attacks against additional security measures such as MFA. See aqua’s articles on the subject:
- Package Planting: Are You [Unknowingly] Maintaining Poisoned Packages?
- New npm Flaws Let Attackers Better Target Packages for Account Takeover
Npm in particular seems to be a target for attackers, given it is by far the largest package manager with over 1.3 million packages. Nearly 7000 packages in npm were identified as malicious in 2022, up from nearly 5000 in 2021. These malicious packages often exploit human error to be installed via typosquatting. This extremely simple technique has shown that supply chain attacks are not exclusive to advanced hacker groups, but also bored teenagers as well.
What can be done to reduce the risk of package managers? The responsibility lies both with organisations and the repositories themselves.
A double-edged sword with package managers is the lack of vetting. Anyone can create a package, which means we can draw on the amazing pool of developers from around the world. But this also means threat actors can create and distribute their malicious code. None of the top five package managers review their code, but the sheer size of the managers means that going to a review system would be complex, expensive and slow. This article by duncanlock has done the math, Supply Chain Attacks & Package Managers – a Solution?
If Debian (the largest vetted repository) needs 240 maintainers, something on the scale of PyPi (only the third largest repository) would need nearly 1000. This opens the door for actors to target unvetted package managers with the assurance they are unlikely to be caught.
An alternative may be for an organisation to vet their own libraries and take them offline. This will combat legitimate packages turned malicious, or any accidental imports that could lead to typosquatting attacks. As with anything, there are some drawbacks. Development time may be slowed, and the vetting will cost money. A reasonable first step would be to use private repositories and then look to vet the libraries from most to least critical to your development.
For certain software products in your inventory, it can be difficult to know their secure development practices and you may be greatly increasing your risk of compromise via a third party. To combat this, some organisations will provide a software bill of materials. This should be a list of all software components used in a software product, making it easier to understand your attack surface and manage it accordingly.
One of the most vulnerable elements of the supply chain is developers. As we saw with typosquatting, human error is often the entry point for attacks. Some developers also implement little to no security best practice, understandably because they create and maintain packages in their spare time as a hobby. There also may only be a single developer hosting hundreds of packages that have millions of downloads. This single point of failure quickly becomes a prime target in supply chain attacks.
Developer bad practice led to numerous compromised libraries in 2022. One such example affected three million users after a developer let a domain hosting a library expire and a deleted username was quickly taken by an attacker and then was allowed to make changes to the library. Both of these attacks were unofficially attributed to a university student, perfectly illustrating that supply chain attacks are no longer exclusive to nation state groups (as with SolarWinds).
Sometimes, the developers themselves may turn malicious. There have been recent incidents of disgruntled developers disrupting the supply chain. A “hacktivist” developer recently corrupted two projects they owned: faker.js and color.js. These had 2.5 million and 22.4 million weekly downloads respectively. This example perfectly outlines the inherent risk with using and relying on third-party code.
The responsibility does not entirely fall on the developers — the package managers and code repositories also share it. A lot of these attacks may have been prevented by using MFA, shockingly something that wasn’t enforced for “critical” developer accounts. However, the likes of Github have begun to implement security decisions on behalf of the developers like enforcing MFA for important developer accounts.
Preventing developer-centric issues will involve the package managers and repositories enforcing more security measures on behalf of the developers. While it may not be possible to vet each package, ones that are integral to thousands of software projects should be reviewed for malicious code or significant changes to the functionality.
Large Software Vendors
The exploitation of SolarWinds in 2020 by the Russian government first brought supply chain security onto centre stage. Hundreds of organisations were compromised in what is considered one of the most successful attacks in history. Since then, the supply chain attacks have gotten creative against package managers and developers. But the level of access and the outreach of targeting the likes of SolarWinds remains extremely popular, especially with advanced hacking groups.
Most recently we have seen the Okta and Twilio breaches. Okta (a supplier of identity and access management) were breached by the group Lapsus$ and over 350 of their corporate customers were affected as part of the breach. The breach itself was caused by a third-party service provider (supply chain-ception?), showing how nested the supply chain has become. Twilio’s breach affected its authentication service authy, which subsequently affected 125 Twilio customers. The breach was part of a phishing attack 0ktapus, named due targeting Okta customers via phishing attacks.
Back in 2021 the Log4Shell vulnerability had the cybersecurity industry scrambling. Due to the immense effort of so many people, mass exploitation was avoided. However, the aftershocks of the vulnerability were felt in 2022, no more so than in the exploitation of VMware Horizon. This product enables virtual desktops and applications and is used by thousands of large organisations. Attackers leveraged the exploit in Horizon to target Microsoft, the UK’s NHS, CrowdStrike and more.
These more complex and advanced attacks are important to prevent for many organisations, as the software can form an integral part of a company’s business. Instead, companies can take steps to slow down the attackers and mitigate any meaningful malicious actions from occurring.
We need a paradigm shift from standard perimeter-based networks, where everything internal is considered a trusted asset to more of a Zero Trust architecture. This is a long arduous process, but many security steps aid the process such as network segmentation, endpoint detection and gaining a full understanding of the attack surface. This shift is a topic I’m passionate about, and I had the chance to contribute to CyberWire Pro on a related incident: the implications of the recent MOVEit exploitation for software supply chain security.
“…There is not a single responsible party for the supply chain, it’s down to the vendors, the repositories, the software consumers and the developers. The second half of 2023 should be when we see meaningful progress by all parties involved to control the supply chain and ensure it can be used in a secure way.”Read the full quote here.
To reduce the likelihood of any one third party being compromised, an organisation should attempt to minimise the attack surface and reliance on the supply chain. It is becoming common practice to hand off security to a vendor but no one is going to care more about your security and your data than you. The more control you have in your systems, the better placed you are to identify threats in the supply chain and also any other parts of your systems.
2022 brought an increase of supply chain attacks across the board, with many of these attacks being by less sophisticated actors at much broader scales. The supply chain looks more like a web and it’s more difficult than ever to fully understand what risks you bring in through your third-party vendors.
But there is hope. Organisations are now seeing the risks of the supply chain clearly and beginning to take action. There is not a single responsible party for the supply chain — it’s down to the vendors, the repositories, the software consumers and the developers. 2023 should be the year we see meaningful progress by all parties involved to control the supply chain and ensure it can be used in a secure way.
NetSPI’s team of offensive security specialists can help! Our Attack Surface Management solution gives security teams insight into their attack surface, and helps monitor changes, validate vulnerabilities, and prioritize remediation efforts.
See Attack Surface Management in action with this on-demand demo.