Dev notes

Random useful notes for developing quil itself.

Release process

1. Run lein do check, cljfmt check and verify that there are no reflection warnings and that everything is formatted.
2. Run "automated" tests: lein test. If tests through open GL error and cannot be executed all in single run - run them using sets: lein test :set-0, lein test :set-1, etc. Watch out for exceptions in console. If you want to run tests one by one manually proceeding to the next test - use MANUAL=true lein test.
3. Run manual Clojure tests: lein test :manual.
4. Run manual ClojureScript tests: lein with-profile cljs-testing do cljsbuild once tests, ring server. Run both "automated" and manual tests. In Chrome, Firefox and ideally IE.
5. Update version in project.clj.
6. Update RELEASE-NOTES.md, filling all changes which went to the release.
7. Update Quil version in README.md.
8. Change version, commit, create git tag with new version.
9. Upload to clojars.
10. Update quil.info website.
11. Update templates.
12. Make an announcement on clj-processing and possibly clojure google groups.
13. Update Quil version in third-party sites/pages. Check list below.

Announcement template

Subject:
[ANN] Quil $VERSION Release
Body:
Happy to announce Quil 2.2.5 release.
Quil is a Clojure/ClojureScript library for creating interactive drawings and animations.

The release available on clojars: https://clojars.org/quil. List of changes:

• Change 1
• Change 2

Documentation on http://quil.info has been updated as well.

Happy hacking!
$YOUR_NAME

Places to update Quil version

• https://github.com/quil/quil/blob/master/README.md
• https://github.com/quil/quil/wiki/Runnable-jar
• https://github.com/quil/quil-examples/blob/master/project.clj
• https://github.com/nbeloglazov/blog-projects
• https://github.com/nbeloglazov/nbeloglazov.github.io

How to update quil when new version of Processing is released

Unfortunately Processing doesn't use maven so we can't just update versions in project.clj and we have to upload Processing jars to clojars ourselves.

Uploading Processing jars

1 - Download new version of Processing, let it be 1.2.3.
2 - Create temp folder and copy 3 Processing jars there:

• cp core/library/core.jar temp/processing-core-1.2.3.jar
• cp modes/java/libraries/pdf/library/pdf.jar temp/processing-pdf-1.2.3.jar
• cp modes/java/libraries/dxf/library/dxf.jar temp/processing-dxf-1.2.3.jar
• cp modes/java/libraries/svg/library/svg.jar temp/processing-svg-1.2.3.jar

3 - Create temp/pom.xml like this:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>quil</groupId>
  <artifactId>processing-core</artifactId>
  <packaging>jar</packaging>
  <version>1.2.3</version>

  <licenses>
    <license>
      <name>GNU Lesser General Public License</name>
      <url>https://www.gnu.org/licenses/lgpl.html</url>
    </license>
  </licenses>
</project>

Check that <version>1.2.3</version> is correct version of Processing.
4 - Upload all jars one by one to clojars using following command:

lein deploy clojars quil/processing-core 1.2.3 processing-core-1.2.3.jar pom.xml

Be sure you modified <artifactId> entry in pom.xml before uploading pdf.jar and dxf.jar. More on pushing manually to clojars: https://github.com/ato/clojars-web/wiki/Pushing#ssh
5 - Check if jars were uploaded successfully:

• http://clojars.org/quil/processing-core
• http://clojars.org/quil/processing-pdf
• http://clojars.org/quil/processing-dxf
• http://clojars.org/quil/processing-svg

6 - Modify project.clj to use new jars:

:dependencies [...
               [quil/processing-core "1.2.3"]
               [quil/processing-pdf "1.2.3"]
               [quil/processing-dxf "1.2.3"]
               ...]

Updating jogl and gluegen-rt jars

It is very likely that new version of Processing will use new jogl and gluegen-rt libs. Jogl/gluegen-rt guys use maven so in theory we could just use their maven libs. But no... Unfortunately uberjars built from this libs won't work correctly because of native libs loading. There is a special page describing how to create jars containing native libs in it: https://jogamp.org/wiki/index.php/JogAmp_JAR_File_Handling. We decided to go with Fat-Jar option. Fat jar is a single jar that contains all native libs using following structure (example):

fat.jar
  natives
    linux-amd64
      lib1.so
      lib2.so 
    linux-i586
      lib1.so
      lib2.so
    windows-amd64
      lib1.dll
      lib2.dll

Undoubtedly it increases size of jar. But in case of uberjar it is not so mmm... crucial because whole clojure library is included in uberjar which is pretty heavy. One always can remove unnecessary native libraries from uberjar afterwards. So we want to create our own Fat-Jar for jogl and gluegen-rt libraries and upload it to clojars just like we did with Processing jars. Here are what we should do:

1 - Figure out what versions of jogl and gluegen-rt are used in processing. Go to core/library/ and check gluegen-rt.jar/meta-inf/manifest.mf, it should contain version used. Let it be 4.5.6 for example.

2 - Use following script to build Fat-Jar: https://gist.github.com/nbeloglazov/11332612. Run following commands which will create 2 jars:

(fat-jar 'org.jogamp.jogl/jogl-all-main "4.5.6" "jogl-all-fat")
(fat-jar 'org.jogamp.gluegen-rt/gluegen-rt-main "4.5.6" "gluegen-rt-fat")

Basically script does following: download all dependencies of provided artifacts (first and second arguments of fat-jar) and pack them to single jar (third argument), putting them to appropriate subfolder (e.g. natives/linux-amd64) depending on classifier of dependency.

3-6 - Repeat steps 3-6 of "Uploading Processing jars" part.

Congratulations! Finally you can have fun fixing all the issues caused by updating to the latest Processing!

Automated image tests

Quil has tests that run all snippets, take screenshots and compare them with expected goldens which are stored in repo. It supports both clj and cljs. During test run a screenshot of the sketch is taken and compared with reference image. If difference exceeds threshold then snippet considered failed and snippet-name-difference.png is stored in the folder where screenshots are kept.

Ensure that you have imagemagick and geckodriver installed before running tests. Imagemagick required for both clj and cljs while geckodriver is needed only for cljs.

To see all differences for failed snippets try:

$IMAGEVIEWER path/to/screenshots/folder/*difference*

where $IMAGEVIEWER is your image viewer. For example on ubuntu it can be eog.

Running clj

In clj simply run regular tests like lein test :set-0, lein test :set-1, etc. It will automatically compare images and print to console names of failed tests if diff exceeds thresold.

Running cljs

Before running cljs test you need to compile cljs snippets and start test page server:

lein with-profile cljs-testing do cljsbuild once tests, ring server

After that you can run the test that using webdriver goes through all snippets, takes screenshots and compares them:

lein test :manual :only quil.snippet/all-cljs-snippets-produce-expected-output

Updating screenshots

To update screenshots (save existing screenshots as foo-expected.png) - prepend UPDATE_SCREENSHOTS=true to the lein command:

UPDATE_SCREENSHOTS=true lein test :set-0

Works for both clj and cljs.