There are several new standardization efforts happening within the WebAssembly (Wasm) space, including what we believe to be a new way to write software applications. By way of describing this new model, I would like to dive into some of the history of Wasm as a way to describe where we are heading.
This article was originally published in InfoWorld as part of the New Tech Forum.
The design for WebAssembly (Wasm) began in 2015, years before officially becoming the 4th language of the web in 2019. While many technologists are familiar with Wasm as a browser technology, Wasm itself does not depend on JavaScript or web APIs.
Wasm's precursor, asm.js, rose to prominence in 2013. A subset of JavaScript that is highly
optimizable for browsers and that can act as a compilation target for languages like C and C++. One
of my all time favorite talks, “The Birth & Death of JavaScript” covers a fictional future inspired
by asm.js. Note the similarities between the future we eventually pave with Wasm and Gary's
predictions by
watching his talk from
PyCon 2014.
I often call asm.js the greatest hack of all time (in the most loving way). Who would have thought
a high-level scripting language could be A")" a compilation target and B")" so incredibly fast? In
2012, I ported several C++ libraries to asm.js and felt that I had unlocked the secret code to a
portable universe.
The technology proved several things. First, there is a need and desire to bring other languages to the web, but devs didn't want to stop there. The types of applications being ported were computationally and graphically demanding, from data visualization tools (like the components I worked on at SAS), to games built with Unity and Unreal Engine (UE3 was ported in 4 days).
That's why when the W3C WebAssembly community group and the corresponding WebAssembly design repository was created in 2015, browser vendors like Mozilla, Google, Microsoft, and Apple were among the first contributors to the effort and the first to see a tangible opportunity. The design called for a language with a binary format that could be used as a portable compilation target, optimized for size to enable fast downloads, and support for streaming compilation that allows the downloaded module to have near instant instantiation. Most importantly, these modules must facilitate sandboxed [execution environments] as any arbitrary code run in browsers must.
Why Wasm beyond the browser?
Much of what was learned in browser-side deployments gave clues to the many ways in which Wasm could fulfill its potential beyond the browser. The “cloud” makes up a heterogeneous set of compute architectures, operating systems, and system constraints in a world-wide network of machinery.
Consider some of Wasm's key properties as a portable compilation target that is fast, sandboxed, and easily distributed thanks to the compact binary format. These properties of Wasm make it the perfect distributable unit of compute for the cloud. Additionally, companies want to build applications for different environments, but don't want to have to refactor every time. Wasm removes these barriers.
When I first learned of asm.js, I was on the hunt for a solution for how to take our existing
Flash application to HTML5. This ActionScript/Flex app was a rewritten version of its Java
counterpart which was a port of earlier versions of the same business logic written in C. This story
might seem wild to you if you haven't worked in large enterprises, but you can find this type of
porting between each epoch of computing in every organization that is lucky enough to survive the
test of time.
Wasm allows total portability to any Wasm-compliant runtime including browsers, runtimes that are purpose-built for FaaS, or something designed to run on tiny architectures for IoT. In the web, Wasm modules are able to use JavaScript “glue” code to access web APIs like WebGL, networking, and devices to do things beyond pure computation. At the end of the day, a Wasm program really only operates on numeric values, or said differently, a Wasm module is a bunch of i32s in a trenchcoat. In order to do interesting things, a Wasm module needs to be able to call functions from the host runtime.
WASI: the frontier of server-side Wasm
Around the same time that WebAssembly 1.0 became a recommended web standard, a new subgroup within the W3C WebAssembly working group was created for exploring a systems level interface for WebAssembly known as WASI (WebAssembly Systems Interface). The group has been working on creating a set of standardized interfaces ever since.
WASI exists to make WebAssembly work well anywhere, not just within the browser, but the key defining feature of WASI is its capability-based security model. Capability-based security has been around since the 60's (Dennis & Van Horn, 1966), but Dan Gohman architected a new take on this by building on ideas from Cloud ABI.
Understandably, this technology soon attracted the attention of companies interested in running Wasm outside the web. Companies like Fastly, Intel, Red Hat, and Mozilla saw a chance to put Wasm to work in the cloud, on devices, and at the edge. Those companies were the founding members of the Bytecode Alliance (BA), a non-profit organization for building secure software foundations for Wasm using standards like WASI. Many other organizations soon joined the BA, including major players across the software industry and it is now standing at over 30 members and growing!
Over the last couple of years, we've made massive progress towards building the necessary tooling to run Wasm in cloud native applications. The community learned a great deal from these early experiences, and this influenced us to create a new standard which we now call the Component Model. The Component Model is at a lower-level than WASI, it works well with WASI but is not dependent upon WASI. This is the result of us envisioning a software ecosystem that is not just based around a portable unit of compute, but something entirely new with composable, interoperable, and platform virtualizable WebAssembly modules. Let's break that down.
- Composability: Allow for modular code reuse in a language-independent way.
- Platform Virtualization: The ability to layer in the platform-specific pieces that a component needs to run in a given environment. An earlier proposal for the feature for platform virtualization and composability was called module-linking.
- Interoperability: With composable and virtualizable components, we need a way to exchange information between components. We started with a proposal called interface-types, but this too is now a key feature of the component model proposal.
This is the story of how the Component Model started to take shape. Each one of the previous proposals have now been refined into this overarching standard. We expect to see the next major stable iteration of WASI Standards and the Component Model later this year.
WASI Standards - where are we now?
Take a look at the video below which takes you through a plotted history of Wasm, WASI and the development of the Component Model.
Key Milestones Box Out
2019: Pure compile target, adding early interfaces that connect syscalls to the host. In many
respects, it appeared that we were heading towards a WebAssembly version of POSIX. We were able to
write a really simple CLI and run it with Wasmtime on a desktop or in a serverless function.
2020: WASI APIs are focused on the sorts of things any CLI program might need, e.g. system
clock, or a file system. Fastly's Lucet Wasm runtime merged with Wasmtime (a BA project).
2021: Of course, all of these elements continue to evolve and improve. In 2021, we start to see
the development of new use case-specific WASI interfaces, e.g. wasi-nn for when hardware
acceleration is needed for inferencing. This is also the year when
Luke Wagner began defining the Component Model.
2022: We
start to see new higher-level APIs for making Wasm modules run well in cloud environments where
capabilities like working with a key-value store or a pub/sub messaging services are needed.
Finally, after a lot of work, WASI sockets were introduced.
2023: We're working towards
stability milestones for the component model and WASI standards.
Looking at the journey from 2019, you can start to see how standards have diversified as use cases have proliferated and users start to pick and choose what they do and don't need. From one ubiquitous block to a growing suite of smaller building blocks designed to operate with one another within a flexible framework: the Component Model.
What is the Component Model?
In this model, developers can pick and choose pieces of their application, implemented in different languages, as different value propositions. As developers begin creating Wasm component libraries, other developers can treat them as the world's largest crate of 'LEGO'.
In a full-circle moment, we believe new innovation will come when the Component Model starts to influence the way we write web applications. This makes sense when considering the web is one of those cool but constrained environments, with very impatient users - a breeding ground for fresh experimentation.
For example, I expect components to make designing a language-neutral plugin system for a web application even easier. If there's a piece needed for a language runtime like python, multiple components that leverage that language runtime could use it. Compare this to today's world, where we only have Wasm modules (not components) and these are typically built with all of its compile-time dependencies baked into a single binary. If a large application were to support third-party plugins, then likely each Wasm plugin will have duplicate dependencies leading to size and memory bloat and slower downloads.
With a future system of Wasm components for the web, where a single application may have their choice of components written in any language, an application will only need to download exactly what it needs and interact with components like any other ES module with an import. In this world, the best component will rise to the top. The best could mean the fastest or the cleanest API, but most importantly its defining characteristic will not be the source language. May the best component win!
Building a stable technical foundation for WASI and Components
A huge part of making WASI standards and the Component Model real is the role the BA plays in creating a usable technical framework: the SDKs, tooling, and core components; all built in a consistent and secure way, and accessible by all as examples of best practice.
Equally, the role of the Technical Steering Committee (TSC) of the BA will be critical in providing technical governance and support for every BA project. We work alongside the folks designing the best possible set of standards in the W3C WebAssembly and WASI working groups, which means we collaborate with them to make sure everything works in practice. The W3C WebAssembly and WASI groups are focused on the finalization of these standards, and the BA is focused on making them consumable within the community as quickly as possible to establish an active feedback loop.
Another important part of the BA's charter is to enable language and environment interoperability. The BA provides tooling for generating language bindings for many different languages, but an aspect of how to reach language interoperability nirvana is that we need registries and package managers in various language ecosystems to interop with Wasm components. That is why we are working on designing a registry protocol (called warg) as part of SIG-Registries that enables any registry that implements the protocol to publish, consume, store, and share Wasm components.
Perhaps the most critical piece of any Wasm software stack is the Wasm runtime, and the BA hosts two! Wasmtime is written in Rust and is often the test bed for experimenting with new WASI and WebAssembly proposals. The WebAssembly Micro-Runtime (WAMR) is written in C and supports many architectures including tiny ones like ESP32. Both of these runtimes act as great reference implementations of a Wasm runtime. The SDKs and tools for building a Wasm module are in-line with Wasm standards, so any standards compliant Wasm runtime (including those not hosted by the BA) can build on these software foundations.
Given all of the new and exciting standards evolving through WASI and the Component Model and the implementations from the Bytecode Alliance, I expect 2023 to be an exciting year!