PLDI 2020 AEC, Paper# 471
NOTE: In the rest of this document, we refer to our tool as
In our submission, we called it
Albatross for anonymity.
Goals of the artifact
In our paper, we evaluated the following about Szalinski (
End-to-End: we ran Szalinski on the flat CSG outputs of a mesh decompiler (Reincarnate). The results are in
Scalability: we evaluated Szalinski on a large dataset of models scraped from a popular online repository (Thingiverse). The results are in
Figure 14(first three box plots).
Sensitivity: we evaluated the usefulness of Szalinski's two main features: CAD rewrites and Inverse Transformations. The results are in
Figure 14(last two box plots).
In support of these results, this artifact reproduces
Table 2 and
Figure 14. In addition, it also generates the output programs in
Figure 15 that are
used in the case studies.
This document contains the following parts:
How to run Szalinski
- Reproducing Table 2 (takes < 5 minutes)
- Reproducing Figure 14 (takes approx. 1.5 hour)
- Reproducing Figure 15 (takes < 5 minutes)
- How to set up Szalinski on a different machine (this is also how we set up the VM)
- Description of the code and how to modify it
Notes and remarks
We provide the artifact as a virtual machine image. To open it you need virtual box version
6.1.2, which can be downloaded here.
Specs of the machine where we ran the VM: Intel i7-8700K (12 threads @ 4.7GHz), 32GiB RAM
- Please download the
.ovafile here and open it with Virtual Box by going to
File -> import applianceand giving the path to the
.ovafile and clicking on
continue. In the next window that pops up, click on
Import. It should take a few minutes to import.
NOTE: When you import the
.ova file, we recommend that you
increase the CPU count as much as you can afford.
Similarly, when running
make, we recommend adding
make -jN where
N is the number of CPUs you allocated to the VM.
Next, please open the virtual machine image in virtual box by clicking on the green
Login is automatic, but in case needed, the password is:
The terminal should be open at startup. The project repository is already cloned. Navigate to the
szalinskidirectory. All the required packages are already installed and Szalinki is already compiled for you, ready to be run.
To allow a quick verification of our artifact, we provided pre-generated data and results in the VM. You can therefore skip the
makecommands in the instructions and directly view the results (see below on how to do that).
As a next step, you can verify that the results are indeed generated from the data we provided. To do so, delete the results (
out/fig14.pdf) and run the
makecommands as explained below.
To run the tool yourself entirely from scratch, first delete the entire
outdirectory and follow the instructions below.
Running the tools
Reproducing Table 2
Navigate to the directory that contains the
This should take about 3 minutes.
This will reproduce
Table 2 from the paper.
To view the content of the table, type
cat out/aec-table2/table2.csv | column -t -s, and compare the numbers
Table 2 in the paper.
- We have significantly improved Szalinski since the PLDI deadline. As a result, for several case studies, the numbers in the last three columns of the table are lower (hence better in this case) than what is reported in the paper.
- We suspect that different versions of OpenSCAD use
different triangulation algorithms for compiling to mesh.
The version supported by Ubuntu 19.10 (the VM) is different from the
version we used during the deadline because we ran it on a MacOS.
Due to this, the numbers in the
#Tricolumn may vary in this artifact.
Reproducing Figure 14
We have included in the repo the 2,127 examples from Thingiverse that
we evaluated on in the paper.
The remainder of the 12,939 scraped from Thingiverse were either
malformed or used features not supported by Szalinski.
The script (
scripts/scrape-thingiverse.sh) scrapes models under the
customizable category, from the first 500 pages.
NOTE: Running this part takes about an hour.
We recommend first reproducing
Figure 15 and
Table 2, both of which take much less time.
Navigate to the directory that contains the
Makefile and type
make out/fig14.pdf. Open the generated file in a pdf viewer and
Figure 14 in the paper.
Reproducing Figure 15 programs
Navigate to the directory that contains the
make aec-fig15. This should take less than a minute.
Then look in the
out/aec-fig15 directory. The
optimized programs generated by Szalinski are in the files with extensions
normal.csexp.opt. There should be 6 such files. Open them and compare the
content with the programs listed in
Figure 15 of the paper.
- The programs in the paper are sugared and represented more compactly for space.
MapIfound in the artifact results corresponds to
Tabulatein the paper.
- When comparing the results generated by the artifact
to the programs in
Figure 15of the paper, it is most important to check that the high-level structure in terms of
MapIsynthesized by the artifact matches that reported in the paper.
Section 6 of our paper describes Szalinski's validation process.
We use OpenSCAD to compile CSG programs to meshes and use CGAL to compute
the Hausdorff distance between two meshes.
To validate the programs in
make out/aec-fig15/hausdorff. This should terminate in less than 3
minutes. It should show you the names of the 6 examples in
Figure 15 and the
corresponding Hausdorff distances which are close to zero.
We have also validated all our other results reported in the paper. However, our experience indicates that OpenSCAD's compilation step is often very slow. Therefore, the other commands mentioned in the instruction for reproducing the results do not perform validation by default.
You can validate any example from our evaluation by typing:
make out/dir_name/example_name.normal.diff, where
dir_name can be
example_name is the name of whatever example you choose.
Then open the generated
.diff file and check
that the Hausdorff distance is within some epsilon of 0.
NOTE: For many example, CGAL crashes or is slow at computing the Hausdorff distance.
For these, we recommend a manual validation if you are interested.
In order to validate an example, type the following:
make out/dir_name/example_name.diff.scad. You can open the generated
file in OpenSCAD (already installed in the VM). In OpenSCAD, click on the
Render button (the second button from the right) in the toolbar.
You should either see nothing rendered or some residual thin walls that are
artifacts of rounding error prevalent in OpenSCAD.
Here we provide instructions on how to start using Szalinski including installation and changing the rules and features of the Caddy language.
Following are the steps for setting up Szalinski from scratch on a different machine that runs Ubuntu 19.10.
Install rust. Type
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | shin the terminal and follow the subsequent instructions. The version we used is
https://www.rust-lang.org/tools/installfor more information.
Make sure you configure your current shell by typing:
source $HOME/.cargo/env(the Rust installation will prompt you to do this).
Install make by typing:
sudo apt-get install make
Install g++ by typing:
sudo apt-get install g++
Install jq by typing:
sudo apt-get install jq
Install CGAL by typing
sudo apt-get install libcgal-dev
Install OpenSCAD by typing
sudo apt-get install openscad
Install git by typing
sudo apt install git
Install pip by typing
sudo apt install python3-pipand then install
pip3 install numpyand
pip3 install matplotlib
We have made a github release for the PLDI AEC from where you can get the source.
Navigate to the project directory where the
Makefileis and run the tool as described above.
Changing Caddy and modifying the rules
The Caddy language is defined in
srcdirectory. A simple feature you can add is support for a new primitive or new transformations. You can also change the costs of various language constructs. The definition of the
costfunction starts at line
As we described in the paper, to verify the correctness of Szalinski, we evaluate Caddy programs to flat Core Caddy and pretty print to CSG. This code is in
permute.rscontains code that solves for first and second degree polynomials in Cartesian and Spherical coordinates, and performs partitioning and permutations of lists.
The rewrites rules are in
rules.rs. Syntactic rewrites are written using the
rw!macro. Each rewrite has a name, a left hand side, and a right hand side. You can add / remove rules to see how that affects the final Caddy output of Szalinski. For example, if you comment out the rules for inverse transformations, they will not be propagated and eliminated, and therefore the quality of Szalinski's output will not be as good.