You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
Added Arrow schema utilities for safely converting and validating column types from arrow::Schema objects:
as_r_schema(): Converts an Arrow schema to a named character vector of equivalent R types (e.g., "int32" → "integer"). Errors on unsupported types.
arrow_schema_to_string(): Extracts the raw Arrow type strings for field in a schema.
is_supported_arrow_type(): Returns a named logical vector indicating which schema fields have supported types.
validate_arrow_schema(): Validates that all field types in an Arrow schema are supported. Throws a helpful error otherwise.
Added arrow_to_r_datatypes, a named character vector defining the mapping of safe and portable Arrow types to their R equivalents.
Added r_schema argument to create_timeseries_schema() and create_oracle_output_schema() functions to enable returning the schema as a vector of R data types instead of an arrow::Schema object (#95)
Added output_type_id_datatype argument to create_oracle_output_schema() and connect_target_oracle_output() functions to allow users to explicitly specify the data type of the output_type_id column in the schema. This ensuring compatibility with create_hub_schema() and connect_hub() (#95).
(Internal) Refactored target data schema and connection tests to use embedded example hubs and reusable schema fixtures, improving reliability and making tests independent of dataset size and ordering.
Added utilities for working with hive-partitioned data file paths:
extract_hive_partitions() for extracting key value pairs from paths to hive-partitioned data files.
is_hive_partitioned_path() for checking if a path is hive-partitioned.
create_oracle_output_schema() and create_timeseries_schema() now define a schema for hive-partitions whose data types are defined in the tasks.json config (#89).