Performance of vroom_format

[matmu]

I want to convert a data frame to a delimited string with vroom::vroom_format (vroom_1.5.5). The example below vroom_format takes around 4 seconds to convert a data.frame with 35k rows and 400 columns of double values (~240MB) into a string. Is this a time one would expect? Given that the throughput of vroom is 1.23 GB/sec it feels quite slow.

library(vroom)

df = data.frame(replicate(400, runif(35000, min=0, max=100)))

system.time({
  res = vroom_format(df)
})

[jimhester]

vroom_format() is primarily intended for testing and debugging, the implementation uses only a single thread.

[matmu]

readr_2.0.2 uses vroom_format in format_delim (here)

[jimhester]

And readr::format_delim() as well is primarily used for testing and debugging.

[matmu]

@jimhester thanks for your feedback. I am confused now. I am implementing a REST API with plumber and the function plumber::serialize_tsv wraps readr::format_tsv as serialization function. I didn't expect that this is all based on a testing and debugging function. Do you know of any performant function for this purpose that is not only meant for testing and debugging?

[jimhester]

Thank you for the usage example, I refactored the vroom_format() code to use most of the same machinery as vroom_write(), so it is now multi-threaded.

As a result on my machine your example now takes a little less than a second to run, where before it was a little more than 3 seconds.

[matmu]

Cool, that's awesome! Thanks a lot for your prompt help. Will check it out later today.