An engineer begins her Monday, sitting down at her desk with a cup of coffee. She's feeling productive, inspired, and ready to write some good code. She opens her GitHub inbox to find 42 notifications on her projects, like this one:
⚠️ CRITICAL VULNERABILITY: Upgrade garbodep from 1.1.12 to 1.1.13
"What's garbodep? I don't even directly depend on that, why am I getting notified for this?" And
so, her day begins.
Backstory
garbodep, of course, is a very popular (fake) sub-dependency of a sub-dependency of the HTTP
server that this engineer's projects uses that is used to represent a version number as a “real”
SemVer string. This library has a subtle vulnerability that allows remote code execution, so now all
projects that used this dependency have to upgrade their dependency that uses this library. Not only
that, each application has to go through automated tests, compile, and get redeployed to whatever
production environment it was running in previously. Our engineer, after spending a few hours
resolving these vulnerabilities, takes lunch without getting any real work done.
Status Quo
Above, we see a possibly hyperbolic example of what every engineer deals with in the cloud-native
development world. This is just a part of developing software with open source libraries, there are
vulnerabilities and we have to resolve them. However, in our last livestream, we looked at this from
a completely new perspective - removing these non-functional requirements (NFRs) from our code
entirely. When we separate our pure logic from the libraries that are required to make our
applications work (e.g. garbodep deep in a NFR) we no longer have the maintenance burden on each
and every application.
A core tenet of wasmCloud is to remove boilerplate code and vulnerabilities from applications with this separation. The code you write to implement your idea, application, or business requirement is in terms of pure logic and abstract interfaces that represent a capability. Applications written in this way don't compile in the open source libraries that write files to disk, or execute commands, or send network requests; the entire attack surface is abstracted and made part of the platform itself.
Impact
There are some famous examples of the consequences of bundling application logic with NFRs. The Log4shell vulnerability in Log4j brought 40% of businesses worldwide to a standstill. Even in a language focused on safety and security, Rust, there are vulnerabilities in libraries like Hyper that sit underneath popular libraries and make headlines.
Hyper sits at the core of a ton of Rust frameworks. The Hyper vulnerability takes an incoming HTTP body and converts it into an array of bytes. Theoretically, someone can send a large content length header or a huge body length with the aim to crash the microservice. The thing is, as the developer, you probably didn't know it happened and, if you're running 20, 200 or 20,000 microservices, all running the same code, you start to see the scale of the issue.
According to a recent IBM study, the average cost to an organization of a breach like Log4shell, is $4.62 million dollars, and they take months to resolve. Vulnerabilities lie within the Non-Functional Requirements of an application - the policies that define how an application performs, rather than its core function. It makes sense, then, that finding a way to remove Non-Functional Requirements from the development process should dramatically reduce the risk of inherited vulnerabilities.
Real World Example
Recently JFrog released an article titled “Watch out for DoS when using Rust's popular hyper package.” Many Rust developers who interact with HTTP in some form end up using hyper, but through a sub-dependency instead of depending on the library directly. The article cites Axum, Salvo and conduit-hyper as a few examples of this. The actual vulnerability here is that crates that directly depend on hyper have to be careful to check the Content-Length header before allocating a memory buffer for an HTTP payload, otherwise you may allocate an arbitrary amount of memory. Standard for an HTTP library, but possible to miss in their documentation.
There are two big problems here. First is that if you have hyper anywhere in your dependency chain, you have to rebuild, retest, and redeploy your application. Second is that hyper is much more common as a sub-dependency, so you may not even realize you need to do this! Without tools like Dependabot or Snyk that automatically scan libraries, you could go weeks without resolving this issue.
During the steam I illustrated the difference between the status quo and what development looks like with wasmCloud with a simple example. I showed the friction in rebuilding a container to resolve the vulnerability. Then, with wasmCloud, illustrated how you don't need to rebuild or redeploy any piece of your application that depends on HTTP functionality; the platform engineer simply updates the HTTP server and all applications that depend on it can continue functioning as they were with no downtime. Check it out:
Conclusion
This stream and blog highlight a major pain that we take for granted in modern software engineering. The process of patching vulnerabilities has severe impacts on developer productivity, developer motivation, development cost, and organizational reputation. We want to fix this problem at the source, decoupling vulnerable open source libraries from business logic rather than trying to add more band-aids on top of the problem.
Have thoughts on this post? Come chat with us on Discord!