Skip to content
This repository has been archived by the owner on Nov 2, 2021. It is now read-only.


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Hasal Performance Test


Master Linux Status Master Windows Status Updates Python 3


Dev Linux Status Dev Windows Status Updates Python 3

"Hasal" /ha'sɑlu/ this word is came from Indigenous Taiwanese "Bunun", and it means hail. Why we choose this word is because we expect our testing could like the hail, fast and weighty. And through our testing could bring more improvement on our performance or quality.

A Framework for testing web performance between different browser

Automatical Installation

If you do not wish to install Hasal automatically, please proceed to next section "Manual Installation". For those who wants to install Hasal, please be notified that a clean environment or

  • Ubuntu
  • Mac OS
  • Windows 7 Launch a command line console with administrator privilege.
  • Windows 8 or above Launch a powershell console with administrator privilege.
Set-ExecutionPolicy Unrestricted

Please noted that Hasal in Windows systems can be run from command line only.

Manual Installation

  • Install Firefox
  • Install SikluliX (, place the runsikulix and other installation files in hasal/thirdparty. When sikulix setup diaglog come out, please select the Pack 1 and make sure scripting language: Python is checked.
  • Install video recording codes and libs (windows/mac:ffmpeg or ubuntu:avconv)
  • Install video recording main program (windows/mac:ffmpeg or ubuntu:avconv)
  • Install opencv
  • Download the client certificate from here ( -- note: needs a account ATM), place all certificates in your hasal working dir.
  • Run

For Ubuntu:

apt-get install virtualenv python-dev
virtualenv ~/.hasalenv            # or "make clean dev-env"
source ~/.hasalenv/bin/activate

pip install selenium

sudo apt-get install wget libav-tools ffmpeg libavc1394-0 libavformat-extra-53 libavfilter2 libavutil-extra-51 mencoder libavahi-common-data xsel xclip
follow this link[] to install opencv or:

sudo apt-get install build-essential cmake git pkg-config
sudo apt-get install libjpeg8-dev libtiff4-dev libjasper-dev libpng12-dev
sudo apt-get install libjpeg8-dev libtiff5-dev libjasper-dev libpng12-dev
sudo apt-get install libgtk2.0-dev
sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
sudo apt-get install libatlas-base-dev gfortran
sudo pip install virtualenv virtualenvwrapper
sudo apt-get install pip
sudo apt-get install python-pip
sudo apt-get install python2.7-dev
pip install numpy
git clone
cd opencv
git checkout 3.1.0
cd ..
git clone
cd opencv_contrib/
git checkout 3.1.0
cd ../opencv

mkdir build
cd build
make -j4
sudo make install
sudo ldconfig
ln -s /usr/local/lib/python2.7/site-packages/ virtualenv_path_you_create/lib/python2.7/site-packages/

python install

For Mac OS:

virtualenv PATH_TO_YOUR_VENV            # or "make clean dev-env"
source PATH_TO_YOUR_VENV/bin/activate

brew install ffmpeg --with-fdk-aac --with-ffplay --with-freetype --with-frei0r --with-libass --with-libvo-aacenc --with-libvorbis --with-libvpx --with-opencore-amr --with-openjpeg --with-opus --with-rtmpdump --with-schroedinger --with-speex --with-theora --with-tools
brew install libav
brew tap homebrew/science
brew install opencv
ln -s /usr/local//Cellar/opencv/2.4.13/lib/python2.7/site-packages/ PATH_TO_YOUR_VENV/lib/python2.7/site-packages/

python install

OR manual build opencv2

virtualenv PATH_TO_YOUR_VENV           # or "make clean dev-env"
source PATH_TO_YOUR_VENV/bin/activate

download the opencv2 package, compile and install
download the opencv2 package here :
  unzip the package 
  cmake the folder unzipped 
  make install
  ln -s /usr/local/lib/python2.7/site-packages/ PATH_TO_YOUR_VENV/lib/python2.7/site-packages/

python install



  • Trigger the framework: python

Usage [--exec-config=<str>] [--firefox-config=<str>] [--index-config=<str>] [--online-config=<str>] [--global-config=<str>] (-h | --help)


  -h --help | Show this screen.
  --exec-config=<str> | Specify the test execution config file; max-run, max-retry, advance, keep-browser etc. settings can be controlled in here. [default: configs/exec/default.json]
  --firefox-config=<str> | Specify the test Firefox config file; [default: configs/firefox/default.json]
  --index-config=<str> | Specify the index config file; you can specify which index you want to generate here. [default: configs/index/runtimeDctGenerator.json]
  --online-config=<str> | Specify the online config file; you can specify if you want to enable online data submission and other related settings here. [default: configs/online/default.json]
  --global-config=<str> | Specify the global config file; you can modify the output fn and status fn here. [default: configs/global/default.json]

Suite file template

  • regression test case format: test_script_path

    Example: tests.regression.gdoc.test_firefox_gdoc_read_basic_txt_1

  • pilot test case format: test_sikuli_script_path

    Example: tests/pilot/facebook/test_firefox_facebook_load_homepage.sikuli/

About generator

They are all inside configs/index/ directory. Generator is all about what data do we want to generate. We use runtimeDctGenerator.json as the default generator.

Generator What can we get?
runtimeDctGenerator The total running time and related statistics
speedIndexDctGenerator Get the SI/PSI for your test cases
inputLatencyDctGenerator The input latency time for specific action (2 images)
inputLatencyAnimationDctGenerator The input latency time for specific action (1 images)
frameThroughputDctGenerator Get the frame throughput time for specific action

Please be remind that most cases with ail in its naming are using inputLatencyAnimationDctGenerator and some of them are using inputLatencyDctGenerator. You may set it with --index-config= mentioned previously for

Basic Debugging

What if the Hasal framework doesn't work?

  1. Try to make the resolution of your pc to 1920*1080
  2. Try to set about:blank as your browser start-up page
  3. Remember to activate your virtualenv for all the things we set up for you
  4. Try to update your code and rerun bootstrap. For windows, you might want to recover the environment variable PATH from the backup file generated after you ran the bootstrap.bat

VM Template (out-of-date)

You can download the VM tempalte for Hasal framework environment from vagrant.

  • vagrant init shako/hasal
  • vagrant up --provider virtualbox
  • Default user name and password : hasal/hasal