Skip to content

Move openCurrentWriter to base class#3492

Closed
zhaomin1423 wants to merge 1 commit intoapache:masterfrom
zhaomin1423:fileWriter
Closed

Move openCurrentWriter to base class#3492
zhaomin1423 wants to merge 1 commit intoapache:masterfrom
zhaomin1423:fileWriter

Conversation

@zhaomin1423
Copy link
Member

@zhaomin1423 zhaomin1423 commented Nov 7, 2021

The openCurrentWriter is related to the method of write, so I think that it is more appropriate to move it here.

@github-actions github-actions bot added the core label Nov 7, 2021
super(fileFactory, io, targetFileSizeInBytes, spec, partition);
this.writerFactory = writerFactory;
this.dataFiles = Lists.newArrayList();
openCurrentWriter();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

openCurrentWriter calls newWriter, which is implemented by subclasses. In this case, it uses writerFactory. Calling openCurrentWriter in the parent class will cause newWriter to be called before setting writerFactory.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I ignored it. Thanks for pointing out my mistake.

@rdblue
Copy link
Contributor

rdblue commented Nov 7, 2021

I ran CI to demonstrate the issue with this change. Here's the test failure:

org.apache.iceberg.spark.source.TestSparkRollingFileWriters > testRollingPositionDeleteWriterNoRecords[FileFormat=AVRO, Partitioned=false] FAILED
    java.lang.NullPointerException
        at org.apache.iceberg.io.RollingPositionDeleteWriter.newWriter(RollingPositionDeleteWriter.java:54)
        at org.apache.iceberg.io.RollingPositionDeleteWriter.newWriter(RollingPositionDeleteWriter.java:36)
        at org.apache.iceberg.io.RollingFileWriter.openCurrentWriter(RollingFileWriter.java:105)
        at org.apache.iceberg.io.RollingFileWriter.<init>(RollingFileWriter.java:55)
        at org.apache.iceberg.io.RollingPositionDeleteWriter.<init>(RollingPositionDeleteWriter.java:46)

@rdblue rdblue closed this Nov 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants