encodeByNameSansHeader: like encodeByName but don't write the header. Us...

[alang9]

...e it only for ordering

It seems difficult to encode a vector of "named" values which is too big to fit into memory without this.

[tibbe]

I don't understand what you mean by:

It seems difficult to encode a vector of "named" values which is too big to fit into memory without this.

Writing the header seems to take negligible amount of memory compared to holding the actual data in memory. Could you please clarify.

I'm wary about adding any more top-level encode methods, as we will end up with a combinatorial explosion. If this makes sense then we should add it as a EncodeOptions field I think.

[alang9]

Sorry I wasn't very clear. Basically I mean that I might want to append to a csv file which already has its header written at the top.

I agree with making this an EncodeOptions field instead. I'll submit a patch if you think this makes sense.

[tibbe]

If we made the encode family of functions use constant space (i.e. by having them take a [a] instead of a Vector a) would that solve your memory issue?

[alang9]

Yes that would probably work.

[tibbe]

Just out of curiosity, how much data are you trying to write?

[alang9]

About 5 million rows by 100 columns. I guess I would be able to fit all of it in memory, bit it still seems like something you should be able to do without using that much memory.

[tibbe]

I went with the constant space encoding route instead. Fixed in #42.