Skip to content
This repository has been archived by the owner on Nov 16, 2023. It is now read-only.

StructBuilder #13

Merged
merged 24 commits into from
Oct 27, 2020
Merged

StructBuilder #13

merged 24 commits into from
Oct 27, 2020

Conversation

pelletier
Copy link
Contributor

@pelletier pelletier commented Oct 6, 2020

Fixes #8

The goal is to be able to read rows from Parquet files into Go values.

In addition, we eventually would like the following properties:

  • Easy-to-use API to read rows.
  • Predicate push-down (only read the minimum values to decide whether to read a whole row) RowReader predicates #10
  • Projection (only read the minimal amount of columns) RowReader projection #9
  • Schema stays a representation of a Parquet schema (to reuse in the writer).
BenchmarkReflectReadRow
BenchmarkReflectReadRow-8            	      38	  32238412 ns/op	20581119 B/op	  338369 allocs/op
BenchmarkReflectReadRowParquetGo
BenchmarkReflectReadRowParquetGo-8   	       5	 220576040 ns/op	93017072 B/op	 1473595 allocs/op

@pelletier pelletier marked this pull request as ready for review October 7, 2020 03:36
@pelletier pelletier mentioned this pull request Oct 9, 2020
@pelletier pelletier linked an issue Oct 9, 2020 that may be closed by this pull request
@pelletier pelletier mentioned this pull request Oct 9, 2020
@pelletier pelletier mentioned this pull request Oct 11, 2020
@pelletier pelletier merged commit 1e38054 into main Oct 27, 2020
@pelletier pelletier deleted the structbuilder branch October 27, 2020 14:59
fpetkovski pushed a commit to fpetkovski/parquet-go that referenced this pull request Jul 17, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
1 participant