feat: SelectObjectContentRequest #236
Conversation
…essage, headers, and payload parsing, Added new functions to parse EventStream messages
SelectObjectContents struct and add XML serialization helpers
options for input_serialization, output_serialization, and scan_range
improve code organization
|
ping @bernardd |
modules to improve readability and consistency.
performance.
select object contents test
| describe "integration test" do | ||
| setup [:start_bypass] | ||
|
|
||
| test "stream SelectObjectContent results", %{bypass: bypass} do |
There was a problem hiding this comment.
ex-aws/ex_aws#1012 has to be merged for this test to pass
… ExAws.SParsers.EventStream to buffer streams
|
This is great work, would love to see this merged. |
| def to_xml(value) when is_atom(value), do: Atom.to_string(value) |> String.upcase() | ||
| def to_xml(value) when is_binary(value), do: value | ||
| def to_xml(value) when is_integer(value), do: Integer.to_string(value) | ||
| def to_xml(value) when is_float(value), do: Float.to_string(value) |
There was a problem hiding this comment.
It's probably best not to roll our own XML encoder here - it misses some basic stuff like escaping of control characters. Maybe add something like https://github.com/joshnuss/xml_builder as an optional dependency to be used here?
|
Thanks very much for your work on this and the corresponding PR over in ex_aws_s3, @avinayak. Please also accept my apologies for not getting to it sooner. It's been quite a busy end of the year over here. |
This PR Adds support for SelectObjectContentRequest in S3. This let's users query contents of S3 Objects if the yare of type CSV, JSON or Parquet.
Explanation for some design choices:
Rangeheader for this request. So, we cannot use multiple http requests to stream this.@bernardd Can you please check this?