Support precompiled bytecode and clone()

[lum1n0us]

In order to enable WAMR JIT in Envoy, we shall enable two features below. The core idea of those is to minimize start-up time by reducing loading phase.

• Precompiled Bytecode. It is stored in a custom section in a .wasm file. When a configuration of envoy.yml enables it, Envoy will load contents from a .wasm(exclude precompiled section) and the precompiled section, and pass them to WasmVm with load(). The precompiled content was injected into the .wam offline. Envoy and WasmVm will not write anything back to the .wasm file.

Wasm Vm	Support precompiled bytecode
v8	Y
wasmtime	N
wavm	Y

• Clone() a thread-local VM. There are three level of Cloneable.
  • NotCloneable. Vms can't be cloned and should be create from scratch. Envoy will store binary content, call Vm's load(), link() and startVm() to load, validate, instantiate, and execute entrance.
  • CompiledBytecode. Vms can be cloned with compiled bytecode, Envoy will not keep binary content, skip load(), and just call link() and startVm()`
  • InstantiatedModule. Vms can be cloned from an instantiated module. Envoy will not keep binary content, skip all load(), link() and startVm().

Wasm Vm	Cloneable
v8	CompiledBytecode
wasmtime	CompiledBytecode
wavm	InstantiatedModule

Reference

envoyproxy/envoy#19404 (comment)

[PiotrSikora]

Regarding precompiled bytecode - I believe wamrc can already be used for that. It's true that WAVM and V8 (via custom wee8_compile tool) append those to the Custom Section of Wasm modules, but I wouldn't be opposed to adding option in Proxy-Wasm C++ Host and Envoy to simply consume precompiled Wasm module from a separate file instead, we need to do that for Wasmtime anyway.