Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-13055: [Doc] Create canonical extension types document #14167

Merged
merged 2 commits into from Sep 20, 2022

Conversation

pitrou
Copy link
Member

@pitrou pitrou commented Sep 19, 2022

@pitrou
Copy link
Member Author

pitrou commented Sep 19, 2022

@wjones127 Would you like to review this?

@github-actions
Copy link

Copy link
Member

@lidavidm lidavidm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

A few things:

  • Weston had some comments in the vote thread, will those be integrated?

    This is maybe implied but I would add that
    modification of extension types must also require a vote and should be
    backwards compatible. Furthermore, extension types (particularly
    those with extensive parameterization/serialization should discuss how
    future additions would be made. For example, if serialized via JSON
    than readers should tolerate (and ignore) unexpected keys.

  • Maybe add 'canonical extension type' to Glossary.rst as well?

Copy link
Member

@wjones127 wjones127 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks pretty good. Once we have our first canonical type, hopefully we will have a model proposal on the ML we can link to. It would be nice, for example, to show what kind of detail the semantics should be described in.

docs/source/format/CanonicalExtensions.rst Show resolved Hide resolved
2) Its parameters, if any, *must* be described in the proposal.

3) Its serialization *must* be described in the proposal and should
not require unduly work or unusual software dependencies (for example, a
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean by "unduly work"? Is that in terms of readability to reviewers? Or in terms of computation time?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant in terms of implementation effort.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can make this more explicit, though ideally we should not deviate too much from what has been voted on on the ML.

@pitrou pitrou force-pushed the ARROW-13055-canonical-extension-types branch from 4578e16 to 6c60780 Compare September 20, 2022 08:45
@pitrou
Copy link
Member Author

pitrou commented Sep 20, 2022

@lidavidm @wjones127 I believe I addressed your comments, could you take another look?

@pitrou pitrou merged commit 2629f20 into apache:master Sep 20, 2022
@pitrou pitrou deleted the ARROW-13055-canonical-extension-types branch September 20, 2022 12:50
@ursabot
Copy link

ursabot commented Sep 20, 2022

Benchmark runs are scheduled for baseline = 2577ac1 and contender = 2629f20. 2629f20 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed ⬇️0.48% ⬆️0.0%] test-mac-arm
[Failed ⬇️0.0% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.14% ⬆️0.04%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] 2629f208 ec2-t3-xlarge-us-east-2
[Finished] 2629f208 test-mac-arm
[Failed] 2629f208 ursa-i9-9960x
[Finished] 2629f208 ursa-thinkcentre-m75q
[Finished] 2577ac1a ec2-t3-xlarge-us-east-2
[Failed] 2577ac1a test-mac-arm
[Failed] 2577ac1a ursa-i9-9960x
[Finished] 2577ac1a ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

zagto pushed a commit to zagto/arrow that referenced this pull request Oct 7, 2022
…14167)

Vote result at https://lists.apache.org/thread/sxd5fhc42hb6svs79t3fd79gkqj83pfh

Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
fatemehp pushed a commit to fatemehp/arrow that referenced this pull request Oct 17, 2022
…14167)

Vote result at https://lists.apache.org/thread/sxd5fhc42hb6svs79t3fd79gkqj83pfh

Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants