Webassembly is a technology allowing binary instruction format to be run on the browser. We put this to the test using video scopes written in C++ and translated to Webassembly.
To run the website, you must have node and python3 installed.
The webserver can be started by going to the webapp/ directory and running
python3 host.py
Next, go to the server/ directory and ensure the dependencies for the backend server are installed by running
npm install
Run the backend server with
node server.js
Navigate to localhost:8080 to view the site.
Compilation into webassembly requires LLVM 11+ to be installed, and LLVM must be built with webassembly enabled.
To check if llvm version is suitable, type llc --version
Compilation from Halide code to Webassembly requires a 2 step complation process (AOT compilation) which is handled by the gn build.
Enter the project directory and generate build files with
gn gen out/{OS}
Next, run the build files using
ninja -C out/{OS}
The webassembly file will be generated in out/{OS}/lib.
If there is an error about a missing libclang_rt.builtins-wasm32.a, the file should be copied over from zmo/extern into the correct directory.
Compilation of angular requires node and npm to be installed. After these are installed, navigate to the angular-video-player/ directory and install the dependencies using
npm install
Navigate to the angular video player base directory and run the following:
ng serve
Navigate to the angular video player base directory and run the following:
ng build
The new files can then be served from a webserver (note that zmo.wasm must be located in the assets/ directory of the output).
The web frontend files are in the angular-video-player/ directory and is built into static production files in the output directory. The algorithm files are all located in the src/ directory and is built into a webassembly file in the output directory.
Adding a new generator is required to create a new Halide pipeline for a scope.
- Add a new class in src/generator/scope.cpp and register the class name with HALIDE_REGISTER_GENERATOR at the bottom of the file. You must also specify the generator name here.
- Edit BUILD.gn andd add the generator name of the scope in the "scopes" list.
- Edit src/scope.h and add a new constant to the enum at the top of the header file.
- Edit src/scope.cpp and add a new header for the generator using the generator name. Then, add the new constant in the select_scope using the enum constant.
- Edit src/halide_zmo.cpp and use the enums from scope.h to select which generator you want to use.
The new generator simply needs to be called from the frontend.
- In the loadWasm() function in app.component.ts, add the new scope and its function call.
- Navigate to
in your chrome browser.
- Click on
Load Unpackedand select the chrome-extension folder in the directory. - Navigate to youtube or vimeo to see the extension in use.
The following function calls are implemented in the zmo.wasm scope library.
- Functions prefixed with cpp is implemented in C++, otherwise they are implemeneted using Halide.
- data_in is the input image. This array must be made through malloc in Javascript
- data_out is where the output image will be. This array must be made through malloc in Javascript
- width is the width of the input image
- height is the height of the input image
void cpp_lumascope(char data_in[], char data_out[], int width, int height)
void cpp_color_lumascope(char data_in[], char data_out[], int width, int height)
void cpp_rgb_parade(char data_in[], char data_out[], int width, int height)
void cpp_vectorscope(char data_in[], char data_out[], int width, int height)
void cpp_color_vectorscope(char data_in[], char data_out[], int width, int height)
void lumascope(unsigned char data[], unsigned char data2[],int width, int height)
void clumascope(unsigned char data[], unsigned char data2[],int width, int height)
void rgbparade(unsigned char data[], unsigned char data2[], int width, int height)
void vectorscope(unsigned char data[], unsigned char data2[], int width, int height)
void cvectorscope(unsigned char data[], unsigned char data2[], int width, int height)