Skip to content
Yuriy Kashnikov edited this page Nov 30, 2017 · 22 revisions

We processed SVG 1.1 Second Edition Test Suite with Macaw, parsed and serialized into new SVG files, then converted these new SVG files and reference SVG into PNGs and compared them with ImageMagick and RMSE metric. To convert from SVG we used Google Chrome in both cases and defined it to be a reference/baseline renderer.

Test PASSES only if Diff value is less than 6%.

Macaw 0.9.0 Results

Data for Parse -> Render in Macaw -> Save Image -> Compare with reference cycle:

Name Percentage (%) Absolute Value
Total amount of tests 100% 521
Passed 11% 56
Failed 89% 465
Failed to parse 29% 151

Data for Parse -> Serialize -> Render SVG in Chrome -> Compare with reference cycle:

Name Percentage (%) Absolute Value
Total amount of tests 100% 521
Passed 16% 82
Failed 84% 439
Failed to parse 29% 150

How To Run SVG 1.1 Test Harness

First you will need to checkout task/test-svg-coverage branch from Macaw main repository.

Install prerequisites:

  • ImageMagick
brew update && brew install imagemagick
  • Python 2.7
brew install python
  • Selenium and ChromeDriver
brew install chromedriver
pip install selenium

Running Test Harness

  1. Go to Macaw/MacawTests/scripts and do sanity check to make sure that everything is OK:
./sanity-check.sh

Fix errors if there are any. Script will also try to fix possible problems, e.g. create missing folders.

There are two ways to calculate coverage:

  1. For Macaw Rendering
  2. For Macaw Serialization (Google Chrome Rendering)

Results data is always stored in data.csv file with the following format:

color-prof-01-f-manual,18213.2,0.277915
color-prop-01-b-manual,14368.2,0.219245

To Run Test Suite with Macaw Rendering

  1. Open Macaw project in Xcode and run execute MacawSVGTestSuiteWithViews.
  2. Run diff-all.sh script in Macaw/MacawTests/scripts folder:
cd Macaw/MacawTests/scripts
./diff-all.sh

To Run Test Suite with Chrome Rendering

Open Macaw project in Xcode and run execute MacawSVGTestSuite. TODO: Finish script to automate this step.