-
Notifications
You must be signed in to change notification settings - Fork 164
Dev notes
Random useful notes for developing quil itself.
- Run
lein do check, cljfmt check
and verify that there are no reflection warnings and that everything is formatted. - Run "automated" tests:
lein test
. Watch out for exceptions in console. If you want to run tests one by one manually proceeding to the next test - useMANUAL=true lein test
. - Run manual Clojure tests:
lein test :manual
. - Run manual ClojureScript tests:
lein with-profile cljs-testing do cljsbuild once tests, ring server
. Run both "automated" and manual tests. In Chrome, Firefox and ideally IE. - Update version in
project.clj
. - Update
RELEASE-NOTES.md
, filling all changes which went to the release. - Update Quil version in
README.md
. - Change version, commit, create git tag with new version.
- Upload to clojars.
- Update quil.info website.
- Update templates.
- Make an announcement on clj-processing and possibly clojure google groups.
- Update Quil version in third-party sites/pages. Check list below.
Subject:
[ANN] Quil $VERSION Release
Body:
Happy to announce Quil 2.2.5 release.
Quil is a Clojure/ClojureScript library for creating interactive drawings and animations.
The release available on clojars: https://clojars.org/quil. List of changes:
- Change 1
- Change 2
Documentation on http://quil.info has been updated as well.
Happy hacking!
$YOUR_NAME
- https://github.com/quil/quil/blob/master/README.md
- https://github.com/quil/quil/wiki/Runnable-jar
- https://github.com/quil/quil-examples/blob/master/project.clj
- https://github.com/nbeloglazov/blog-projects
- https://github.com/nbeloglazov/nbeloglazov.github.io
Unfortunately Processing doesn't use maven so we can't just update versions in project.clj
and we have to upload Processing jars to clojars ourselves.
1 - Download new version of Processing, let it be 1.2.3
.
2 - Create temp folder and copy 3 Processing jars there:
cp core/library/core.jar temp/processing-core-1.2.3.jar
cp modes/java/libraries/pdf/library/pdf.jar temp/processing-pdf-1.2.3.jar
cp modes/java/libraries/dxf/library/dxf.jar temp/processing-dxf-1.2.3.jar
cp modes/java/libraries/svg/library/svg.jar temp/processing-svg-1.2.3.jar
3 - Create temp/pom.xml
like this:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>quil</groupId>
<artifactId>processing-core</artifactId>
<packaging>jar</packaging>
<version>1.2.3</version>
<licenses>
<license>
<name>GNU Lesser General Public License</name>
<url>https://www.gnu.org/licenses/lgpl.html</url>
</license>
</licenses>
</project>
Check that <version>1.2.3</version>
is correct version of Processing.
4 - Upload all jars one by one to clojars using following command:
lein deploy clojars quil/processing-core 1.2.3 processing-core-1.2.3.jar pom.xml
Be sure you modified <artifactId>
entry in pom.xml
before uploading pdf.jar
and dxf.jar
.
More on pushing manually to clojars: https://github.com/ato/clojars-web/wiki/Pushing#ssh
5 - Check if jars were uploaded successfully:
- http://clojars.org/quil/processing-core
- http://clojars.org/quil/processing-pdf
- http://clojars.org/quil/processing-dxf
- http://clojars.org/quil/processing-svg
6 - Modify project.clj
to use new jars:
:dependencies [...
[quil/processing-core "1.2.3"]
[quil/processing-pdf "1.2.3"]
[quil/processing-dxf "1.2.3"]
...]
It is very likely that new version of Processing will use new jogl and gluegen-rt libs. Jogl/gluegen-rt guys use maven so in theory we could just use their maven libs. But no... Unfortunately uberjars built from this libs won't work correctly because of native libs loading. There is a special page describing how to create jars containing native libs in it: https://jogamp.org/wiki/index.php/JogAmp_JAR_File_Handling. We decided to go with Fat-Jar option. Fat jar is a single jar that contains all native libs using following structure (example):
fat.jar
natives
linux-amd64
lib1.so
lib2.so
linux-i586
lib1.so
lib2.so
windows-amd64
lib1.dll
lib2.dll
Undoubtedly it increases size of jar. But in case of uberjar it is not so mmm... crucial because whole clojure library is included in uberjar which is pretty heavy. One always can remove unnecessary native libraries from uberjar afterwards. So we want to create our own Fat-Jar for jogl and gluegen-rt libraries and upload it to clojars just like we did with Processing jars. Here are what we should do:
1 - Figure out what versions of jogl and gluegen-rt are used in processing. Go to core/library/
and check gluegen-rt.jar/meta-inf/manifest.mf
, it should contain version used. Let it be 4.5.6
for example.
2 - Use following script to build Fat-Jar: https://gist.github.com/nbeloglazov/11332612. Run following commands which will create 2 jars:
(fat-jar 'org.jogamp.jogl/jogl-all-main "4.5.6" "jogl-all-fat")
(fat-jar 'org.jogamp.gluegen-rt/gluegen-rt-main "4.5.6" "gluegen-rt-fat")
Basically script does following: download all dependencies of provided artifacts (first and second arguments of fat-jar
) and pack them to single jar (third argument), putting them to appropriate subfolder (e.g. natives/linux-amd64
) depending on classifier of dependency.
3-6 - Repeat steps 3-6 of "Uploading Processing jars" part.
Congratulations! Finally you can have fun fixing all the issues caused by updating to the latest Processing!
Quil has tests that run each snippets, takes a screenshot and compares with expected output image. The expected output is stored in the repo under the dev-resources/snippet-snapshots
directory. It supports both clj and cljs. During test run a screenshot of the sketch is taken and compared with reference image. If difference exceeds threshold then snippet considered failed and snippet-name-difference.png
is stored in the folder where screenshots are kept.
Ensure that you have imagemagick
, geckodriver
, and chromedriver
installed before running tests. Imagemagick required for both clj and cljs while geckodriver
or chromedriver
is needed only for cljs. Currently CLJS tests are using chromedriver
to account for a screenshot size issue with geckodriver
.
To see all differences for failed snippets try:
$IMAGEVIEWER path/to/screenshots/folder/*difference*
where $IMAGEVIEWER is your image viewer. For example on ubuntu it can be eog
.
In clj simply run regular tests like lein test
. It will automatically compare images and print to console names of failed tests if diff exceeds threshold. An example snapshot difference failure would look like:
lein test :only quil.snippet/quil_snippets_output_save-frame
FAIL in (quil_snippets_output_save-frame) (test_util.clj:73)
Image differences in "save-frame", see: dev-resources/snippet-snapshots/clj/normal/save-frame-difference.png
dev-resources/snippet-snapshots/clj/normal/save-frame-actual.png PNG 500x500 500x500+0+0 8-bit sRGB 5870B 0.000u 0:00.000
dev-resources/snippet-snapshots/clj/normal/save-frame-expected.png PNG 500x500 500x500+0+0 8-bit sRGB 3929B 0.000u 0:00.000
expected: (<= result threshold)
actual: (not (<= 0.0974062 0.03))
The first line shows how to re-run just that test (possibly with the MANUAL=true
environment variable to interact with the sketch while it's running). The "Image differences" line reports the diff image to inspect to see why it failed. The next two lines are output from identify
which helps verify the sizes are equivalent. Finally we see can the see the threshold for comparison that was exceeded.
This can be adjusted on a per snippet basis using the :accepted-diff-threshold
key. Snippets can also skip the snapshot test entirely by setting :skip-image-diff?
to true. Image diffs can be unreliable so feel free to either increase the threshold or skip the test entirely if it's not deterministic, or hard to keep it deterministic across different machines or architectures.
Before running cljs test you need to compile cljs snippets and start test page server:
lein with-profile cljs-testing do cljsbuild once tests, ring server
After that you can run the test that using webdriver goes through all snippets, takes screenshots and compares them:
lein test :manual :only quil.snippet/all-cljs-snippets-produce-expected-output
To update screenshots (save existing screenshots as foo-expected.png
) - prepend UPDATE_SCREENSHOTS=true
to the lein
command:
# all CLJ tests
UPDATE_SCREENSHOTS=true lein test
# or to just update on test, in this example the output.save-frame test
UPDATE_SCREENSHOTS=true lein test :only quil.snippet/quil_snippets_output_save-frame
# cljs tests
UPDATE SCREENSHOTS=true lein test :manual :only quil.snippet/all-cljs-snippets-produce-expected-output