Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doctest for GenericListArray. #474

Merged
merged 8 commits into from
Jun 23, 2021
Merged

Doctest for GenericListArray. #474

merged 8 commits into from
Jun 23, 2021

Conversation

novemberkilo
Copy link
Contributor

Which issue does this PR close?

Re #301

What changes are included in this PR?

Doctest for GenericListArray

Are there any user-facing changes?

No

@novemberkilo
Copy link
Contributor Author

@alamb I can't tell if GenericListArray is something that folks would build directly so am not sure if I have chosen a good example or not. I've added a test as well. I could add another example based on a builder but wanted to please get your thoughts on this first. Thanks.

@codecov-commenter
Copy link

codecov-commenter commented Jun 20, 2021

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.65%. Comparing base (6e2f684) to head (9147c9a).
Report is 2894 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #474      +/-   ##
==========================================
- Coverage   82.65%   82.65%   -0.01%     
==========================================
  Files         165      165              
  Lines       45524    45524              
==========================================
- Hits        37628    37627       -1     
- Misses       7896     7897       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @novemberkilo -- I think users of Arrow would typically use ListArray or LargeListArray (which are both typedefs / implemented in terms of GenericListArray)

Maybe adding some documentation to GenericListArray hinting / directing users to ListArray and LargeListArray would be helpful (as this distinction / convention is not obvious in my opinion)

@novemberkilo
Copy link
Contributor Author

Thanks @alamb - I will do so and add examples for ListArray and LargeListArray

@github-actions github-actions bot added the arrow Changes to the arrow crate label Jun 22, 2021
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @novemberkilo -- I have a small suggestion, but otherwise this is looking great.

Thank you

@@ -50,6 +50,9 @@ impl OffsetSizeTrait for i64 {
}
}

/// Generic struct for a primitive Array
///
/// Instead of using `GenericListArray` directly, consider using `ListArray` or `LargeListArray`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// Instead of using `GenericListArray` directly, consider using `ListArray` or `LargeListArray`
/// Instead of using `GenericListArray` directly, consider using [`ListArray`] or [`LargeListArray`]

If you use [`ListArray`] instead, the docs will include a hyperlink to the class

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fwiw, I do not think this recommendation is fair. GenericListArray allows to write code generic over the Offset, which is more generic than only supporting ListArray or LArgeListArray.

The same applies for Utf8Array<T>, PrimitiveArray<T>, etc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jorgecarleitao - I'm a little confused though. My initial commit attempted to show an example of this 4c5b2b4 which admittedly was the equivalent of the example for ListArray

Would you perhaps suggest different wording for this comment, or perhaps could you point me in the direction of a better example that we could use for GenericListArray ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am sorry I was not very explicit. :(

The example is great, I would just omit the

Instead of using GenericListArray directly, consider using [ListArray] or [LargeListArray]`

which suggests that using ListArray is better than using GenericListArray, when imo we do not have enough information here to make this recommendation. In general using GenericListArray is better because it allows to write things like

fn some_f<O: ListOffset>(array: &GenericListArray<O>)

which makes it generic over both "ListArray" and "LargeListArray". Most code in arrow and DataFusion uses this to reduce code duplication while supporting both.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks - fwiw I was trying to tackle the specific problem of showing by an example, how to construct a GenericListArray

Per #474 (review) I removed the example for GenericListArray in favour of examples for ListArray and LargeListArray. The intent was not to suggest that using ListArray is better or worse than using GenericListArray

In any case, also removing the comment will result in no documentation or examples for GenericListArray

Is that the final state we want? Or should we have some documentation or hints for usage for GenericListArray

Hope that all makes sense. Thanks.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is valuable if the comment on GenericListArray pointed (likely) novice users towards ListArray and LargeListArray as their existence may not be obvious at this stage

How about making the wording less judgemental such as

For non generic lists, you may wish to consider using [`ListArray`] or [`LargeListArray`]`

arrow/src/array/array_list.rs Outdated Show resolved Hide resolved
arrow/src/array/array_list.rs Outdated Show resolved Hide resolved
novemberkilo and others added 4 commits June 22, 2021 20:54
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
@alamb
Copy link
Contributor

alamb commented Jun 23, 2021

Thank @novemberkilo !

@alamb alamb merged commit 6e47f3c into apache:master Jun 23, 2021
alamb added a commit that referenced this pull request Jun 23, 2021
* Doctest for GenericListArray.

* Fix linter errors.

* fixup! Fix linter errors.

* fixup! fixup! Fix linter errors.

* Update arrow/src/array/array_list.rs

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* Update arrow/src/array/array_list.rs

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* Update arrow/src/array/array_list.rs

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* fixup! Update arrow/src/array/array_list.rs

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
alamb added a commit that referenced this pull request Jun 23, 2021
* Doctest for GenericListArray.

* Fix linter errors.

* fixup! Fix linter errors.

* fixup! fixup! Fix linter errors.

* Update arrow/src/array/array_list.rs

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* Update arrow/src/array/array_list.rs

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* Update arrow/src/array/array_list.rs

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* fixup! Update arrow/src/array/array_list.rs

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

Co-authored-by: Navin <navin@novemberkilo.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants