A hexagonal circle packing implementation for canvasing/sampling large geographical areas (specifically optimized for Google/Instagram/Twitter APIs)
Here's the blog post that explains the project and process behind it.
$ git clone email@example.com:shayanjm/hexpacker.git
$ lein repl
$ lein uberjar $ java -jar hexpacker-0.1.1-standalone.jar [args]
There are 3 main components for dealing with distributed computing with hexpacker: Generator, Worker, and Transformer.
hexpacker.gen/maingenerates the lat/long pairs for a given coverage area, chunked into batches of 1000 pairs.
hexpacker.work/mainreads in one such batch and executes (currently google) queries using the lat/long pairs and returns a set of the clean data. Requires an environmental variable
WORKERto be set to an integer value which denotes the ID of the worker processing the data (depending on how many machines you want to distribute to). This is set up in
hexpacker.transform/mainreads in the accumulated result sets, transforms them, and writes them as csv.
Compilation & local test example (sub-radius of 15m.)
$ lein uberjar $ java -cp hexpacker-0.1.1-standalone.jar clojure.main -m hexpacker.gen | WORKER=1 java -cp hexpacker-0.1.1-standalone.jar clojure.main -m hexpacker.work 15 | java -cp hexpacker-0.1.1-standalone.jar clojure.main -m hexpacker.transform
Calling hexpacker from your code
Since this repo is an implementation (as opposed to a library), you will have to load/include the appropriate namespaces manually.
hexpacker.stitchcontains all of the code pertaining to circle packing/coordinate generation. Incanter is also included in this namespace in case you want to run some tests via plotting.
hexpacker.mercatorcontains the mercator projection definitions (keep in mind that there are some artifacts in the file for niceness when plotting with incanter)
hexpacker.haversinecontains the haversine (and reverse-haversine) implementations.
hexpacker.coreis the actual 'implementation' for data gathering. You can use it as an example to build out your own cool projects.
Realistically, you should only need to load the
hexpacker.stitch/pack-geo-circle function if you just want to generate circle packings.