This app is focused on using litmus tests to showcase the allowed behaviors on Vulkan memory model on Android devices. This work is originated from WebGPU Memory Model Testing.
This app is required to be run on devices with 8.0+ and GPU that supports Vulkan 1.1.
Visit this link to download the latest and stable version of this app's APK file.
Visit this link to see a table of collected data using this app on various Android devices' GPU.
Once you first launch the app, make sure to accept giving permission to let the app to read and write files in the storage. This permission is required since the app needs to use shader files to run litmus tests and write test output to a text file.
By clicking the navigation drawer on the top-left side of the screen, you will be able to see the full list of litmus tests like the screenshot below.
Similiar to WebGPU Memory Model Testing page, this app provides four categories of litmus tests.
- Weak Memory Tests: These test the semantics of memory when two threads access multiple memory locations concurrently, specifically whether hardware is allowed to re-order cetain combinations of reads and writes to different memory locations.
- Coherence Tests: These test the semantics of memory when a single memory location is accessed concurrently, ensuring coherency is respected.
- Atomicity: This test checks that an atomic read-modify-write instruction is, indeed, atomic.
- Barrier Tests: These test the implementation of Vulkan's fence synchronization primitives, ensuring memory is properly synchronized with respect to the barrier.
Each test page contains a brief description of the program under test, as well as pseudocode showing the instructions executed by each thread. The psuedocode also shows where each thread executes relative to the other and calls out the behavior of interest.
Each test page provides two mode for testing: explorer and tuning.
In explorer mode, you will be able to manually change parameters to induce interesting behaviors. For more information on each test parameters, visit the WebGPU Memory Model Testing page. There will be parameter presets that you will be able to quickly load "Default" or "Stress" parameters. There also will be shader option where you can select shader variant. Not selecting any shader option will run test with default shader. For more information on shader variants, visit the WebGPU Memory Model Testing page and click on variant option in test page to see the difference on the shader variants. Finally, you can click "Start" button to start running the test.
While the test is running, you will be able to see progress of the current running test below the "Tuning" button.
After the test has completed running, you will be able to see the result by clicking the "Result" button.
In tuning mode, you can choose a number of configuration to run and a number of iterations to run for. Each configuration generates a random parameter set and run the test. You can also set a random seed to generate same parameters among different devices. (Empty random seed will result in random parameters) You can also specify the range of number and size of workgroups to produce the random parameter. This was to prevent spending time running tests with number and size of workgroups that will produce all sequential behaviors and no interleaved and weak behaviors. The tuning mode allows users to generate data showing what parameter combinations are most effective at revealing weak behaivors or even bugs. Finally, you can click "Start" button to start running the test.
While test is running, you will be able to see progress of the current running test below the "Tuning" button.
After test has completed running, you will be able to see the result by clicking the "Result" button.
If you want to run multiple tests with certain parameters at once, you can visit the "Running Multiple Tests" page by opening up the navigation drawer. Here, you will be able to select which tests that you want to run, and a test type.("Explorer" or "Tuning") Once you have finished making selections, clicking "Start" will start running selected tests. Similar to other testing mode, you will be able to see the test progress under the "Start" button. Once all the test has been completed, the result will be visible under the "Start" button.
After using the "Running Multiple Tests" function, you can send the result to the developer by clicking the "Send Result" button. The current version of this feature is temporary, so it can be changed in the future After clicking the button, there will be a pop-up of other applications like "Gmail" that you can select. Currently, "Gmail" is the most preferable application that you can use the send results to the developer. Other email applications may not work as intented.
Some Android devices may run into issue with sending result feature due to Google Play being disabled, unable to open "Gmail" application through another app, and etc. If this happens, there is an alternative way for you to retrieve the result file. Download X-plore or any application that lets you look into the devices' storage. After downloading X-plore, head over to "/storage/emulated/0/Android/data/com.example.litmusandroid/files/DCIM/" where you will find a json file called "litmustest_multitest_explorer_result.json" (result file for explorer) or "litmustest_multitest_tuning_result.json" (result file for tuning). You can transfer this file to somewhere else like Google Drive then email to mingun0108@gmail.com with the file attached to send the result file to the developer.
- Navigate to "Running Multiple Tests" page in the app.
- Select 6 of the weak memory tests. (Message Passing, Store, Read, Load Buffer, Store Buffer, and 2+2 Write)
- Click "Tuning" as Test Type.
- Set tuning test parameters as following screenshot:
- Click "Start" button to start the test.
- Once the test has finished running, click "Send Result" to send the result to the developer.
Thank you so much for helping us with collecting the results!