Convert output formats to use Data.Text.Lazy and SVG combinator library #56

Merged
merged 8 commits into from Sep 9, 2012

Projects

None yet

2 participants

Contributor

While poking around with the internals of ImplicitCAD in order to develop a package that adds GHCLive support*, I noticed that all of the functions that convert objects to various output formats directly concatenate strings. In order to reduce memory consumption and accelerate the output process, I converted everything to either use Data.Text.Lazy builders or (in the case of SVG) special combinator libraries.

The largest impact these commits have is the introduction of a few new dependencies:

  • text >= 0.11.1.0. As your convention for dependencies seems be not specifying package versions, I omitted this requirement from implicit.cabal - however, it won't compile with anything less than 0.11.1.0.
  • blaze-svg, which in turn requires blaze-markup >= 0.5
  • mtl, in order to provide the Reader monad.

Otherwise, there are no public-facing API changes at this time - although starting a discussion about refactoring the output API seems smart.

Note that I couldn't find particularly good test cases for openscad output - especially for objects that invoke the ExtrudeRM codepath. I suspect this module is correct, but some more verification wouldn't hurt.

* This may just be the killer application for GHCLive - it's a really cool combination!

matthewSorensen added some commits Aug 23, 2012
@matthewSorensen matthewSorensen Switch to using Data.Text.Lazy instead of String for output.
Note that all of the existing formatting functions still build strings,
so this commit introduces no performance gains. However, I plan on rewriting
all of them using the relevant builder modules, so all of the output
code paths should get much faster.

(Commit 1/n of my quest to add efficient GHCLive support for ImplicitCAD.)
d6164e9
@matthewSorensen matthewSorensen Rewrite the SVG output code to use blaze-svg.
Presumably, this has tremendous positive performance implications,
but I haven't benchmarked it.

(Commit 2/n of my quest to add efficient GHCLive support for ImplicitCAD.)
f906115
@matthewSorensen matthewSorensen Rewrite laser cutter output to use Data.Text.Lazy.Builder.
Output is line-for-line identical to the previous version. Note that
this does introduce a lower bound for base, as <> is a somewhat recent
introduction.

(Commit 3/n of my quest to add efficient GHCLive support for ImplicitCAD.)
ed3e934
@matthewSorensen matthewSorensen Update STL output. ef3f51e
@matthewSorensen matthewSorensen Create a new module Graphics.Implicit.Export.TextBuilderUtils to mana…
…ge exports.

Also switch all of the functions for showing integers and floats to
the native Data.Text.Lazy.Builder.* functions.

(Commit 5/n of my quest to add efficient GHCLive support for ImplicitCAD.)
e2c786b
@matthewSorensen matthewSorensen Re-implement the SCAD output module.
Note that there's a really nice pretty printer inside that module
just waiting for the Reader's context to change from ℝ to (ℝ,Int) and a
few changes to call.

Otherwise, this is the same story as all of the other rewrites - no
more string concatenation and slightly less awkward parameter passing.

Note that this does introduce a dependency on mtl, which may not be
popular. This would also work with transformers, so...

(Commit 6/6 of my quest to add efficient GHCLive support for ImplicitCAD.)
c7dfb60
@matthewSorensen matthewSorensen Define the <> syn for mappend so as to build with base < 4.5. 916bd78
Owner
colah commented Aug 26, 2012

Hey Matthew!

It's really exciting that you are interested in ImplicitCAD. More people contributing would be awesome! (You may wish to check out hacking.md)

On the surface, switching to Data.Text.Lazy seems like an awesome idea. I've tried using a few different String alternatives (namely some of the byestrings), since we seem to be loosing a significant amount of time on generating the text of different files (according to some profiling that I don't entirely trust).

Unfortunately, I don't really understand lazy performance well enough to predict if things will actually speed up ImplicitCAD. So I rely on benchmarking code to determine if a change genuinely has a positive impact on performance (and I've tried and had to reject dozens of things I expected to improve performance but didn't).

To test your code, I used the following object:

$res = 1;
union() {
  cube(40);
  translate([30,30,30]) cube(40);
}

I ran the old code 10 times (with one thread, since multi-threading makes performance harder to track). It took the following amounts of time:

3.440s
3.377s
3.356s
3.399s
3.507s
3.493s
3.348s
3.415s
3.459s
3.442s

I then ran your code:

3.694s
3.788s
3.776s
3.795s
3.787s
3.779s
3.739s
3.759s
3.599s
3.649s

My guess is that it has something to do with the triangles not being lazily generated (this is a result of parallelism having different sections of the list generated simultaneously).

That said, your code for generating SVGs is a bit faster. So that's definitely something to incorporate.

I can't merge this code right now, but if you can get it to the point where it is more performant than the present version, I'd be thrilled to accept it.

Owner
colah commented Aug 26, 2012

That said, your code for generating SVGs is a bit faster

It occurs to me that parallelism is presently not being used in 2D objects right now... This would reinforce the notion that it has to do with the laziness of the triangle mesh.

@matthewSorensen matthewSorensen Switch to a much larger buffer size (16 K chars, vs 128).
This pretty much solves the performance issues vs. the original
string implementation.
71dc7e4
Contributor

It turns out that the main performance issue was that I was using a ridiculously tiny buffer size when converting Builders to Text. The last commit remedies this - here's a benchmark demonstrating that, and the typical results of running it. The end result is that both implementations are very close in terms of performance, although the text version tends to be slightly faster. My main interest in making these changes was not performance, however - rather, fully evaluated Text uses significantly less (~2/5 as much) memory as Strings.

I was also considering implementing support for binary stl files - is this a desirable addition?

@colah colah merged commit 3562552 into colah:master Sep 9, 2012
Owner
colah commented Sep 9, 2012

Binary STLs would be appreciated. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment