Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Truncate IPC record batch #2040

Merged
merged 11 commits into from
Jul 14, 2022
Merged

Truncate IPC record batch #2040

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
7d5cfd9
Truncate IPC record batch
viirya Jul 11, 2022
bd63562
Fix clippy
viirya Jul 11, 2022
8da6f94
Fix clippy
viirya Jul 11, 2022
52c9273
Check deserilized record batch
viirya Jul 11, 2022
ffb2ff2
For review
viirya Jul 11, 2022
4c3fb6b
Truncate DictionaryArray. Fix null buffer truncation. Add null value …
viirya Jul 12, 2022
6f477d0
Use BufferBuilder
viirya Jul 12, 2022
a9bd443
Add option to IpcWriteOptions
viirya Jul 12, 2022
311ff3f
For review
viirya Jul 14, 2022
2dcf751
Remove truncate option. Revise test for review.
viirya Jul 14, 2022
50c4eb7
Trigger Build
viirya Jul 14, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions arrow/src/array/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,7 @@ pub(crate) use self::data::layout;
pub use self::data::ArrayData;
pub use self::data::ArrayDataBuilder;
pub use self::data::ArrayDataRef;
pub(crate) use self::data::BufferSpec;

pub use self::array_binary::BinaryArray;
pub use self::array_binary::FixedSizeBinaryArray;
Expand Down
15 changes: 15 additions & 0 deletions arrow/src/datatypes/datatype.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -721,6 +721,21 @@ impl DataType {
)
}

/// Returns true if this type is temporal: (Date*, Time*, Duration, or Interval).
pub fn is_temporal(t: &DataType) -> bool {
use DataType::*;
matches!(
t,
Date32
| Date64
| Timestamp(_, _)
| Time32(_)
| Time64(_)
| Duration(_)
| Interval(_)
)
}

/// Returns true if this type is valid as a dictionary key
/// (e.g. [`super::ArrowDictionaryKeyType`]
pub fn is_dictionary_key_type(t: &DataType) -> bool {
Expand Down