Vange-rs: WebAssembly in Rust

Vangers — is one of the most technological and celebrated game of its time (1998). The game still living in today. Thanks to great community, the game has received many improvements: HD, 60 FPS, new multiplayer modes, etc.

Vange-rs — is one of the most interesting project in Vangers universe. It is written in Rust and bringing real 3D view of worlds (based on wgpu graphics library).

wgpu — is a cross-platform, safe, pure-rust graphics api. It runs natively on Vulkan, Metal, D3D12, D3D11, and OpenGLES; and on top of WebGPU on wasm.

Vange-rs demonstrates full power of wgpu library. You can read explanation of all render techniques that can be applied to Vangers in wiki page. Even just driving in real 3D is looking amazing. Let’s port it to WebAssembly!

I hope we can do it with ease, because Rust is a modern language that supports WebAssembly out of the box, but that’s just not how it comes off.

WebAssembly in Rust

Rust documentation tells us about three different platforms with WebAssembly support (Tier 2: rustc “guaranteed to build”).

• wasm32-unknown-emscripten
• wasm32-unknown-unknown
• wasm32-unknown-wasi

wasm32 — architecture, unknown — vendor, emscripten | unknown | wasi — operation system. For all platforms guaranteed support for standard library (fs, time functions, etc.).

wasm32-unknown-emscripten

Emscripten, is oldest and advanced toolkit for compilation C/C++ into WebAssembly. It supports almost all posix functions, multiple virtual file systems and OpenGL ES3 out of the box. In theory it should compile project into WebAssembly without changing the source code. In fact, this is not entirely true.

To compile project with emscripten you need to add flag — target wasm32-unknown-emscripten to cargo.

Rust will compile the source code, and on linking stage it will use emcc to produce WebAssembly. Unfortunately, rustc have hardcoded emcc flag that causes link time error when some of symbols is not defined ( ERROR_ON_UNDEFINED_SYMBOLS=1). Because of that event hello-world project can’t compile with wasm32-unknown-emscripten.

It’s not possible to fix event with RUSTFLAGS, because this flags from rustc are always appended to the end of emcc command. Even if we can solve problem with undefined symbols (like said in link above), we can’t remove ASSERTIONS=1 flag. So, we can’t produce WebAssembly without debug code!

Luckily, emscripten can change this behavior with rarely used environment variable:

EMMAKEN_CFLAGS="-s ERROR_ON_UNDEFINED_SYMBOLS=0 --no-entry -s ASSERTIONS=0"

We use this workaround to compile WebAssembly, but this will not be enough. Most of crates that have WebAssembly support, doing it with help of special wrappers such as web-sys and stdweb. The wrap browser API, like WebGL into Rust types. web-sys is based on wasm-bindgen and so it not support emscripten.

The wasm-bindgen project is designed to target the wasm32-unknown-unknown target in Rust. This target is a “bare bones” target for Rust which emits WebAssembly as output.

You will get an error if you try to run web-sys based project compiled with wasm32-unknown-emscripten:

cannot call wasm-bindgen imported functions on non-wasm targets

wasm-bindgen requires changing produced WebAssemlby file for correct work. It generates placeholder functions that should be replaced with actual implementation during the build. Here is what one of the authors writes about possible emscripten:

AFAIK emscripten wants its own JS shims and all that, and having two systems of managing shims won’t mix well.

…

wasm-bindgen doesn’t support the emscripten wasm target at this time. I haven’t ever tested it myself so I don’t know how far off we would be from getting it to work, but it’s expected that emscripten doesn’t work at the moment.

wasm-bindgen is not actually compatible with emscripten, this also means that all crates based on it will not work too (web-sys, etc.). At the moment, these are generally all libraries. Since stdweb is actually dead:

• winit (crate for managing windows and input) drops support of stdweb
• rust does not produce correct WebAssembly during stdweb build starting from nightly-2019–11–25. Reported 2 years ago!

I tried using stdweb in conjunction with winit, and the rust error (link above) prevents me from doing this. Here’s another reason why vange-rs won’t work with emscripten:

• wgpu doesn’t support emscripten yet
• glow doesn’t support emscripten yet (already fixed)
• ron didn’t compile (already fixed)

One thing I figured out is the rust community is trying to use emscripten in the wrong way. The whole point of emscripten is that it provides an identical environment as a regular *nix system, and therefore you do not need to use it in conjunction with wrappers like web-sys or stdweb.

For example, linux OpenGL ES3 backend of wgpu uses khronos-egl and glow. When compilation to WebAssembly occurs, khronos-egl is disabled and the web-sys and glow wrappers are used instead. But, for emscripten it is not necessary to do this, since it emulates a complete system with OpenGL ES3. Thus, you just need to use the same code as for the native platform. By this way we can easily add emscripten support to both wgpu и в glow.

Finally, we can compile vange-rs with emscripten. Had to completely abandon the winit library, since we use emscripten, we don’t need to create a window. In the browser case window is a just canvas that we declared in index.html. We need only to pass WebGL context to wgpu. The library has a good enough level of abstraction to accept a stub as a window handle.

struct WebWindow {} 
unsafe impl HasRawWindowHandle for WebWindow { 
   fn raw_window_handle(&self) -> RawWindowHandle {
        RawWindowHandle::Web(WebHandle::empty()) 
   } 
} 
// ... 
let window = WebWindow {}; 
let surface = unsafe { instance.create_surface(&window) };

wgpu will consider that the window has been created, although in fact it does not exist, there is only a WebGL context (OpenGL ES3), which will return through the native krhonos-egl. By removing winit, we lost control from the keyboard, since the library did it for us, but we can easily add a couple of handlers to return it. So it really works, but with big caveats.

Unfortunately, I’ve seen in many discussions that rust is abandoning wasm32-unknown-emscripten in favor of wasm32-unknown-unknown. I think this is not correct view, since emscripten has a lot of its advantages that may never appear in the wasm32-unknown-unknown, for example, asyncify support, 3 types of file systems to choose from, a network stack, etc.

wasm32-unknown-unknown

All the community attention is focused on the web-sys and wasm-bindgen. Well, should they work out of the box? Unfortunatelly, no…

Everything is compiling without any problems; just pass the flag — target wasm32-unknown-unknown to compile it.

Now we can run it in the browser, but:

panic: operation not supported on this platform

File system functions is not supported.

Obviously, the game cannot start without the files. Let me remind you that in emscripten this is solved quite simply: by default, the memory file system is used. Memory fs implements the standard functions for working with files like on native platform. You can just preload files into it before starting. This is a completely logical solution and, as it seems, a similar approach could be applied in Rust (using a virtual file system), then this code would work:

File::create(file).expect(&format!("Unable to create {}", file))
    .write(include_bytes!("../file"))
    .expect(&format!("Unable to write in {}", file));

We do not have files in the file system, but we could create them: upload via wasm-bidngen or simply by inserting a section into the WebAssembly data section.

However, this does not work, because the file system implementation in the case of unknown is simply missing. Not only the filesystem, there is also no implementation for threading or time related functions. Even env_logger won’t work out of the box because it tries to get the current time and this feature is not implemented.

So we have two options here: either rewrite vange-rs by completely abandoning the libraries and code that work with the filesystem, or fix rustc compiler by adding support for the in-memory filesystem.

Obviously, we choose the second one. It was not easy, especially considering that I only read selected chapters from the rust book. But it worked in the end!

Funny case, I unsuccessfully tried for several days to build the rust compiler using the command:

./x.py build library/std --target wasm32-unknown-unknown

The command worked, but I could not use this compiler, because of the error:

error[E0463]: can’t find crate for `core`

It turned out that you need to pass 2 architectures at the same time (x86_64-unknown-linux-gnu, wasm32-unknown-unknown):

You need to use — target x86_64-unknown-linux-gnu — target wasm32-unknown-unknown”. If you don’t include the host in — target you can’t build build scripts and proc macros that need to be compiled for the host.

By doing this, I finally got a working compiler with a virtual filesystem. In my repository, you can download the compiled version for x86_64-unknown-linux-gnu. Now we can simply register it with rustup and apply it to the project:

rustup toolchain link memfs memfs/stage1 
cd <project> rustup override set memfs

After all, we can build project as usual with cargo. It is surprising, but winit is also unable to catch keyboard events due to an error, so same as in emscripten case we need to do it by ourselves.

Finally we have a working version of vange-rs for the browser!

wasm32-unknown-wasi

Since wasm-bindgen does not support anything other than unknown platform, we have all the same problems as with emscripten. Moreover, wasi is targeted for node.js, not for browser. Therefore, I did not experiment with him.

Сonclusions

Rust is probably too young and support for WebAssembly is experimental. Despite the large amount of effort required, both compilation options (emscripten, unknown) work!

Which one is easier is up to you, for unknown you will have to build a compiler, and in the case of emscripten you will have to abandon most of the libraries. But the result in both cases will delight you, because nothing is impossible.

Many thanks to the Vangers community and Dzmitry (@kvark) — the author of the vange-rs and wgpu projects, for the very opportunity to ride mechas in 3D and help in fixing WebGL errors.

Links

• vange-rs browser version
• source code (emscripten, unknown)
• rust-memfs
• wgpu

UPD. Part 2 (vange-rs in original game)

Originally published at https://habr.com (Russian) on December 10, 2021.