Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-81: [Format] Augment dictionary encoding metadata to accommodate additional use cases #297

Closed
wants to merge 1 commit into from

Conversation

wesm
Copy link
Member

@wesm wesm commented Jan 22, 2017

cc @julienledem @nongli @jacques-n. I am hoping to close the loop on our discussion in https://issues.apache.org/jira/browse/ARROW-81. In my applications, I need the flexibility to transmit:

  • Dictionaries encoded in signed integers smaller than int32. For example, with 10 dictionary values, we may send int8 indices
  • Indicator that the dictionary is ordered

These features are needed for Python and R support, and in general for statistical computing applications.

Change-Id: I07a001ed0614db8340addbc8f45f5a3af5d5cbed
@nongli
Copy link
Contributor

nongli commented Jan 23, 2017

lgtm

Copy link
Member

@xhochy xhochy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM

@wesm
Copy link
Member Author

wesm commented Jan 23, 2017

Merging -- I'm open to discussing this further, perhaps after the 0.2 release

@asfgit asfgit closed this in 085c875 Jan 23, 2017
@wesm wesm deleted the ARROW-81 branch January 23, 2017 14:14
@julienledem
Copy link
Member

LGTM

wesm pushed a commit to wesm/arrow that referenced this pull request Sep 2, 2018
…Schema

This is a follow up fix for [PARQUET-918](apache/parquet-cpp#295), do I need to create another jira for this?

Looks like some .idea files are included by accident. It looks no harm. Do I need to revert them?@wesm

cc @wesm @itaiin  for reviewing

Author: Xianjin YE <advancedxy@gmail.com>

Closes apache#297 from advancedxy/master and squashes the following commits:

e606d9d [Xianjin YE] Add .idea/ to .gitignore and make style check happy.
1adb192 [Xianjin YE] Add API doc for FromParquetSchema(parquet_schema, column_indices, out)
8de263b [Xianjin YE] Keep ordering in column indices when converting Parquet Schema to Arrow Schema
wesm pushed a commit to wesm/arrow that referenced this pull request Sep 4, 2018
…Schema

This is a follow up fix for [PARQUET-918](apache/parquet-cpp#295), do I need to create another jira for this?

Looks like some .idea files are included by accident. It looks no harm. Do I need to revert them?@wesm

cc @wesm @itaiin  for reviewing

Author: Xianjin YE <advancedxy@gmail.com>

Closes apache#297 from advancedxy/master and squashes the following commits:

e606d9d [Xianjin YE] Add .idea/ to .gitignore and make style check happy.
1adb192 [Xianjin YE] Add API doc for FromParquetSchema(parquet_schema, column_indices, out)
8de263b [Xianjin YE] Keep ordering in column indices when converting Parquet Schema to Arrow Schema

Change-Id: I97447368cf803e5782c00abae4423bee967b7552
wesm pushed a commit to wesm/arrow that referenced this pull request Sep 6, 2018
…Schema

This is a follow up fix for [PARQUET-918](apache/parquet-cpp#295), do I need to create another jira for this?

Looks like some .idea files are included by accident. It looks no harm. Do I need to revert them?@wesm

cc @wesm @itaiin  for reviewing

Author: Xianjin YE <advancedxy@gmail.com>

Closes apache#297 from advancedxy/master and squashes the following commits:

e606d9d [Xianjin YE] Add .idea/ to .gitignore and make style check happy.
1adb192 [Xianjin YE] Add API doc for FromParquetSchema(parquet_schema, column_indices, out)
8de263b [Xianjin YE] Keep ordering in column indices when converting Parquet Schema to Arrow Schema

Change-Id: I97447368cf803e5782c00abae4423bee967b7552
wesm pushed a commit to wesm/arrow that referenced this pull request Sep 7, 2018
…Schema

This is a follow up fix for [PARQUET-918](apache/parquet-cpp#295), do I need to create another jira for this?

Looks like some .idea files are included by accident. It looks no harm. Do I need to revert them?@wesm

cc @wesm @itaiin  for reviewing

Author: Xianjin YE <advancedxy@gmail.com>

Closes apache#297 from advancedxy/master and squashes the following commits:

e606d9d [Xianjin YE] Add .idea/ to .gitignore and make style check happy.
1adb192 [Xianjin YE] Add API doc for FromParquetSchema(parquet_schema, column_indices, out)
8de263b [Xianjin YE] Keep ordering in column indices when converting Parquet Schema to Arrow Schema

Change-Id: I97447368cf803e5782c00abae4423bee967b7552
wesm pushed a commit to wesm/arrow that referenced this pull request Sep 8, 2018
…Schema

This is a follow up fix for [PARQUET-918](apache/parquet-cpp#295), do I need to create another jira for this?

Looks like some .idea files are included by accident. It looks no harm. Do I need to revert them?@wesm

cc @wesm @itaiin  for reviewing

Author: Xianjin YE <advancedxy@gmail.com>

Closes apache#297 from advancedxy/master and squashes the following commits:

e606d9d [Xianjin YE] Add .idea/ to .gitignore and make style check happy.
1adb192 [Xianjin YE] Add API doc for FromParquetSchema(parquet_schema, column_indices, out)
8de263b [Xianjin YE] Keep ordering in column indices when converting Parquet Schema to Arrow Schema

Change-Id: I97447368cf803e5782c00abae4423bee967b7552
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants