Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI][GLib][Parquet] GParquetColumnChunkMetadata comparing failed #35266

Closed
mapleFU opened this issue Apr 21, 2023 · 6 comments · Fixed by #35278 or #35286
Closed

[CI][GLib][Parquet] GParquetColumnChunkMetadata comparing failed #35266

mapleFU opened this issue Apr 21, 2023 · 6 comments · Fixed by #35278 or #35286
Assignees
@kou
Labels
Component: Continuous Integration Component: GLib Component: Parquet Component: Ruby Type: bug
Milestone
13.0.0

Comments

@mapleFU
Copy link
Member

mapleFU commented Apr 21, 2023

Describe the bug, including details regarding any error messages, version, and platform.

===============================================================================
Failure: test: #==(TestParquetColumnChunkMetadata):
        @metadata == other_metadata
        |         |  |
        |         |  #<Parquet::ColumnChunkMetadata:0x7fec99845598 ptr=0x600000f6be20>
        |         false
        #<Parquet::ColumnChunkMetadata:0x7fec9984f638 ptr=0x600000eba320>
/Users/runner/work/arrow/arrow/c_glib/test/parquet/test-column-chunk-metadata.rb:50:in `block in <class:TestParquetColumnChunkMetadata>'
     47:   test("#==") do
     48:     reader = Parquet::ArrowFileReader.new(@file.path)
     49:     other_metadata = reader.metadata.get_row_group(0).get_column_chunk(0)
  => 50:     assert do
     51:       @metadata == other_metadata
     52:     end
     53:   end
===============================================================================
.......................O
===============================================================================
Omission: parquet::RowGroupMetaData::Equals() isn't stable. [test: #==(TestParquetRowGroupMetadata)]
/Users/runner/work/arrow/arrow/c_glib/test/parquet/test-row-group-metadata.rb:48:in `block in <class:TestParquetRowGroupMetadata>'
===============================================================================

Eg:

  1. https://github.com/apache/arrow/actions/runs/4763103224/jobs/8466124309?pr=35242
  2. https://github.com/apache/arrow/actions/runs/4762190574/jobs/8464152443?pr=35262

Component(s)

Continuous Integration, Parquet, Ruby
@mapleFU mapleFU changed the title [Ruby][Parquet] ParquetColumnChunkMetadata comparing failed [CI][Ruby][Parquet] ParquetColumnChunkMetadata comparing failed Apr 21, 2023
@mapleFU mapleFU mentioned this issue Apr 21, 2023
Merged
@kou
Copy link
Member

kou commented Apr 21, 2023

It seems that this is happen since we merge #35229.

@kou
Copy link
Member

kou commented Apr 21, 2023

It's not reproduced on my local M1 macOS...

@kou kou changed the title [CI][Ruby][Parquet] ParquetColumnChunkMetadata comparing failed [CI][GLib][Parquet] GParquetColumnChunkMetadata comparing failed Apr 21, 2023
kou added a commit to kou/arrow that referenced this issue Apr 21, 2023
@kou
apacheGH-35266: [CI][GLib][Parquet] Omit gparquet_column_chunk_metada…
b2f7138 
…ta_equal() test

Because it's not stable on macOS.

We omit the test for now like apacheGH-20207.
@github-actions github-actions bot mentioned this issue Apr 21, 2023
Merged
kou added a commit that referenced this issue Apr 22, 2023
@kou
GH-35266: [CI][GLib][Parquet] Omit gparquet_column_chunk_metadata_equ…
080a74c 
…al() test (#35278)

### Rationale for this change

Because it's not stable on macOS.

### What changes are included in this PR?

We omit the test for now like GH-20207.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

No.
* Closes: #35266

Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@kou kou added this to the 13.0.0 milestone Apr 22, 2023
@mapleFU
Copy link
Member Author

mapleFU commented Apr 22, 2023

@kou Thanks for fast fixing this. Sorry for making that failed, however, I'm not familiar with Ruby, I think I just change some comments in the code in #35278 . If you find something wrong, please tell me, or print the difference of these two metadatas.

@kou
Copy link
Member

kou commented Apr 22, 2023

OK. Could you implement parquet::ColumnChunkMetaData::ToString()? Then we can dump metadata contents easily.

@mapleFU
Copy link
Member Author

mapleFU commented Apr 22, 2023

Okay, will add one today

@mapleFU
Copy link
Member Author

mapleFU commented Apr 22, 2023

#35281 add one naive implemention

kou added a commit to kou/arrow that referenced this issue Apr 22, 2023
@kou
apacheGH-35266: [GLib][Parquet] Fix a GC bug that parent metadata ref…
78436c4 
…erence is missing in sub metadata

`GParquetColumnChunkMetadata` must not be GC-ed while the parent
`GParquetRowGroupMetadata` is alive.

`GParquetRowGroupMetadata` must not be GC-ed while the parent
`GParquetFileMetadata` is alive.
@kou kou reopened this Apr 22, 2023
@github-actions github-actions bot mentioned this issue Apr 22, 2023
Merged
kou added a commit that referenced this issue Apr 23, 2023
@kou
GH-35266: [GLib][Parquet] Fix a GC bug that parent metadata reference…
f4bd43d 
… is missing in sub metadata (#35286)

### Rationale for this change

`GParquetColumnChunkMetadata` must not be GC-ed while the parent `GParquetRowGroupMetadata` is alive.

`GParquetRowGroupMetadata` must not be GC-ed while the parent `GParquetFileMetadata` is alive.

### What changes are included in this PR?

Add missing parent metadata references to sub metadata.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

Yes.
* Closes: #35266

Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
liujiacheng777 pushed a commit to LoongArch-Python/arrow that referenced this issue May 11, 2023
@kou @liujiacheng777
apacheGH-35266: [CI][GLib][Parquet] Omit gparquet_column_chunk_metada…
549da3e 
…ta_equal() test (apache#35278)

### Rationale for this change

Because it's not stable on macOS.

### What changes are included in this PR?

We omit the test for now like apacheGH-20207.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

No.
* Closes: apache#35266

Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
liujiacheng777 pushed a commit to LoongArch-Python/arrow that referenced this issue May 11, 2023
@kou @liujiacheng777
apacheGH-35266: [GLib][Parquet] Fix a GC bug that parent metadata ref…
a04b9a2 
…erence is missing in sub metadata (apache#35286)

### Rationale for this change

`GParquetColumnChunkMetadata` must not be GC-ed while the parent `GParquetRowGroupMetadata` is alive.

`GParquetRowGroupMetadata` must not be GC-ed while the parent `GParquetFileMetadata` is alive.

### What changes are included in this PR?

Add missing parent metadata references to sub metadata.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

Yes.
* Closes: apache#35266

Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
ArgusLi pushed a commit to Bit-Quill/arrow that referenced this issue May 15, 2023
@kou @ArgusLi
apacheGH-35266: [CI][GLib][Parquet] Omit gparquet_column_chunk_metada…
60b1044 
…ta_equal() test (apache#35278)

### Rationale for this change

Because it's not stable on macOS.

### What changes are included in this PR?

We omit the test for now like apacheGH-20207.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

No.
* Closes: apache#35266

Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
ArgusLi pushed a commit to Bit-Quill/arrow that referenced this issue May 15, 2023
@kou @ArgusLi
apacheGH-35266: [GLib][Parquet] Fix a GC bug that parent metadata ref…
4002d70 
…erence is missing in sub metadata (apache#35286)

### Rationale for this change

`GParquetColumnChunkMetadata` must not be GC-ed while the parent `GParquetRowGroupMetadata` is alive.

`GParquetRowGroupMetadata` must not be GC-ed while the parent `GParquetFileMetadata` is alive.

### What changes are included in this PR?

Add missing parent metadata references to sub metadata.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

Yes.
* Closes: apache#35266

Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
rtpsw pushed a commit to rtpsw/arrow that referenced this issue May 16, 2023
@kou @rtpsw
apacheGH-35266: [CI][GLib][Parquet] Omit gparquet_column_chunk_metada…
9a2d7bd 
…ta_equal() test (apache#35278)

### Rationale for this change

Because it's not stable on macOS.

### What changes are included in this PR?

We omit the test for now like apacheGH-20207.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

No.
* Closes: apache#35266

Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
rtpsw pushed a commit to rtpsw/arrow that referenced this issue May 16, 2023
@kou @rtpsw
apacheGH-35266: [GLib][Parquet] Fix a GC bug that parent metadata ref…
2063b44 
…erence is missing in sub metadata (apache#35286)

### Rationale for this change

`GParquetColumnChunkMetadata` must not be GC-ed while the parent `GParquetRowGroupMetadata` is alive.

`GParquetRowGroupMetadata` must not be GC-ed while the parent `GParquetFileMetadata` is alive.

### What changes are included in this PR?

Add missing parent metadata references to sub metadata.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

Yes.
* Closes: apache#35266

Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kou kou

Labels
Component: Continuous Integration Component: GLib Component: Parquet Component: Ruby Type: bug
Projects
None yet
Milestone
13.0.0
2 participants
@kou @mapleFU