Introduction

A reference type (aka externref or anyref) is an opaque reference made available to a WASM module by the host environment. Such references cannot be forged in the WASM code and can be associated with arbitrary host data, thus making them a good alternative to ad-hoc handles (e.g., numeric ones). References cannot be stored in WASM linear memory; they are confined to the stack and tables with externref elements.

Rust does not support reference types natively; there is no way to produce an import / export that has externref as an argument or a return type. wasm-bindgen patches WASM if externrefs are enabled. This library strives to accomplish the same goal for generic low-level WASM ABIs (wasm-bindgen is specialized for browser hosts).

`externref` use cases

Since externrefs are completely opaque from the module perspective, the only way to use them is to send an externref back to the host as an argument of an imported function. (Depending on the function semantics, the call may or may not consume the externref and may or may not modify the underlying data; this is not reflected by the WASM function signature.) An externref cannot be dereferenced by the module, thus, the module cannot directly access or modify the data behind the reference. Indeed, the module cannot even be sure which kind of data is being referenced.

It may seem that this limits externref utility significantly, but externrefs can still be useful, e.g. to model capability-based security tokens or resource handles in the host environment. Another potential use case is encapsulating complex data that would be impractical to transfer across the WASM API boundary (especially if the data shape may evolve over time), and/or if interactions with data must be restricted from the module side.

Usage

externref is available as a library and a CLI app. These two are complementary: the library is used on the WASM (guest) side, and the CLI app can be used for post-processing of the generated WASM modules. (Such post-processing is mandatory, but it can be performed with the library as well.)

Using Library

Add this to your Crate.toml:

[dependencies]
externref = "0.3.0-beta.1"

See the library docs for detailed description of its API.

General workflow

The basic approach is as follows:

Use Resources as arguments / return results for imported and/or exported functions in a WASM module in place of externrefs . Reference args (including mutable references) and the Option<_> wrapper are supported as well.
Add the #[externref] proc macro on the imported / exported functions.
Post-process the generated WASM module with the processor. This can be possible

Resources support primitive downcasting and upcasting with Resource<()> signalling a generic resource. Downcasting is unchecked; it is up to the Resource users to define a way to check the resource kind dynamically if necessary. One possible approach for this is defining a WASM import fn(&Resource<()>) -> Kind, where Kind is the encoded kind of the supplied resource, such as i32.

Note

Resource is essentially a smart pointer. Correspondingly, its Eq and Hash trait implementations treat it as such (i.e., two resources are equal if they point to the same externref).

Note

Resource implements the RAII pattern, in which it releases the associated externref on drop. Correspondingly, Resource does not implement Clone / Copy. To clone resources, you may want to wrap it in a Rc / Arc, or to use copyable resources.

Basic example

The code sample below demonstrates the basic usage of the #[externref] proc macro with 2 resource types.

#![allow(unused)]
fn main() {
use externref::{externref, Resource};

// Two marker types for different resources.
pub struct Arena(());
pub struct Bytes(());

#[cfg(target_arch = "wasm32")]
#[externref]
#[link(wasm_import_module = "arena")]
extern "C" {
    // This import will have signature `(externref, i32) -> externref`
    // on host.
    fn alloc(arena: &Resource<Arena>, size: usize) 
        -> Option<Resource<Bytes>>;
}

// Fallback for non-WASM targets.
#[cfg(not(target_arch = "wasm32"))]
unsafe fn alloc(_: &Resource<Arena>, _: usize) 
    -> Option<Resource<Bytes>> { None }

// This export will have signature `(externref) -> ()` on host.
#[externref]
#[unsafe(export_name = "test_export")]
pub extern "C" fn test_export(arena: &Resource<Arena>) {
    let bytes = unsafe { alloc(arena, 42) }.expect("cannot allocate");
    // Do something with `bytes`...
}
}

Copyable resources

ResourceCopy is a variation of Resource that implements Clone / Copy. As a trade-off, it does not implement any resource management.

Imports below allow to construct a copyable resource (ResourceCopy<Bytes>) from an ordinary one.

#![allow(unused)]
fn main() {
type MessageCopy = ResourceCopy<Bytes>;

#[externref::externref]
#[link(wasm_import_module = "test")]
unsafe extern "C" {
    #[resource]
    pub(crate) fn send_message_copy(
        sender: &Resource<Sender>,
        message_ptr: *const u8,
        message_len: usize,
    ) -> MessageCopy;

    /// This is valid because `Resource<..>` is guaranteed to have `usize` representation,
    /// so the host essentially receives the index into the `externrefs` table.
    pub(crate) fn inspect_message(#[resource = false] bytes: MessageCopy);
}
}

Then, these resources can be used in the app similarly to shared references (&_).

#![allow(unused)]
fn main() {
#[externref]
pub extern "C" fn test_export_with_copies(sender: Resource<Sender>) {
    let str = "test";
    let message = unsafe { imports::send_message_copy(&sender, str.as_ptr(), str.len()) };
    let other_message = unsafe { imports::send_message(&sender, str.as_ptr(), str.len()) };
    // `ResourceCopy` can be created by leaking a `Resource<_>`.
    let other_message = other_message.leak();
    let message_copy = message;

    // `Resource`s implement pointer comparison semantics.
    assert!(message == message);
    assert!(message_copy == message);
    assert!(message != other_message);

    let messages: HashSet<_> =
        [message, other_message, message, message_copy, other_message].into();
    assert_eq!(messages.len(), 2);
    for message in messages {
        unsafe {
            imports::inspect_message(message);
        }
    }
}
}

Important

For some apps, it’s acceptable for the externref table to grow unboundedly (e.g., if the target module is executed for a short duration and then the execution env is discarded). In other cases, the table cleanup must be implemented by the caller, e.g. by hooking to the Drop implementation for the ResourceCopy wrapper, which may pass ResourceCopy (i.e., the index in the externref table) to the host for cleanup.

externref CLI

externref CLI app allows transforming WASM modules that use externref shims to use real externref types. This is a mandatory step to get the module to work correctly.

Usage

The executable provides the same functionality as the WASM [processor] from the externref crate. See its docs and the output of externref --help for a detailed description of available options.

Warning

The processor should run before WASM optimization tools such as wasm-opt from binaryen.

The terminal capture below demonstrates transforming a test WASM module. The capture includes the tracing output, which was switched on by setting the RUST_LOG env variable. Tracing info includes each transformed function and some other information that could be useful for debugging.

Output with tracing

Installation options

Use a pre-built binary for popular targets (x86_64 for Linux / macOS / Windows and AArch64 for macOS) from the master branch.
Use a pre-built binary for popular targets from GitHub Releases.
Use the app Docker image.
Build from sources using Rust / cargo.

Downloads

Important

The binaries are updated on each push to the git repo branch. Hence, they may contain more bugs than the release binaries mentioned above.

Platform	Architecture	Download link
Linux	x86_64, GNU	Download
macOS	x86_64	Download
macOS	arm64	Download
Windows	x86_64	Download

Building from Sources

To build the CLI app from sources, run:

cargo install --locked externref-cli
# This will install `externref` executable, which can be checked
# as follows:
externref --help

This requires a Rust toolchain locally installed.

Minimum supported Rust version

The crate supports the latest stable Rust version. It may support previous stable Rust versions, but this is not guaranteed.

Crate feature: `tracing`

By default, tracing is enabled via the tracing crate feature. You can disable the feature manually by adding a --no-default-features arg to the installation command. Tracing is performed with the externref::* targets, mostly on the DEBUG and INFO levels. Tracing events are output to the stderr using the standard subscriber; its filtering can be configured using the RUST_LOG env variable (e.g., RUST_LOG=externref=debug).

Docker Image

As a lower-cost alternative to the local installation, you may install and use the CLI app from the GitHub Container registry. To run the app in a Docker container, use a command like

docker run -i --rm ghcr.io/slowli/externref:main - \
  < module.wasm \
  > processed-module.wasm

Here, - is the argument to the CLI app instructing to read the input module from the stdin. To output tracing information, set the RUST_LOG env variable in the container, e.g. using docker run --env RUST_LOG=debug ....

Keyboard shortcuts

externref Documentation