Skip to content

Checkstyle: prevent using unshaded Guava classes in Arrow and AssertJ#3463

Merged
jackye1995 merged 2 commits intoapache:masterfrom
jackye1995:checkstyle-guava
Nov 4, 2021
Merged

Checkstyle: prevent using unshaded Guava classes in Arrow and AssertJ#3463
jackye1995 merged 2 commits intoapache:masterfrom
jackye1995:checkstyle-guava

Conversation

@jackye1995
Copy link
Contributor

add new checkstyle rules for unshaded Guava classes for recently introduced libraries. @kbendick

Copy link
Contributor

@kbendick kbendick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few updates needed in the checkstyle descriptions and comments, but this is a good catch.

Is there anything in org.apache.arrow.util we might ever need?

I think it's safe to add this and then it can be reconsidered or made more explicit to the Preconditions check otherwise.

I know that my Intellij autoimports Preconditions from all sorts of places and it's easy to miss.

<message key="import.illegal" value="Use org.apache.iceberg.relocated.* classes from bundled-guava module instead."/>
</module>
<module name="IllegalImport">
<property name="id" value="BanUnrelocatedGuavaClasses"/>
Copy link
Contributor

@kbendick kbendick Nov 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: The value here looks like it should maybe be BanUnrelocatedAssertjUtilClasses?

If we're looking to only ban the guava ones, maybe illegalClasses needs to be more specifc to org.apache.arrow.util.Preconditions?

EDIT: I checked and there's pretty much only Preconditions and other things available in our bundled-guava. Might need a unique id but otherwise this is fine.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was mostly following this:

<module name="IllegalImport">
            <property name="id" value="BanUnrelocatedGuavaClasses"/>
            <property name="illegalPkgs" value="com.google.common"/>
            <message key="import.illegal" value="Use org.apache.iceberg.relocated.* classes from bundled-guava module instead."/>
        </module>

I am actually not sure if the ID has to be unique, let me check

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see, so that's a user-defined name, then let me update it

Copy link
Contributor

@kbendick kbendick Nov 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked myself when I was verifying this (not 100% used to updating these files), and they do not have to be unique.

It's just used in the output it seems, like this example where I added an import.

> Task :iceberg-arrow:checkstyleMain FAILED
[ant:checkstyle] [ERROR] /iceberg/arrow/src/main/java/org/apache/iceberg/arrow/ArrowAllocation.java:23:1:
Use org.apache.iceberg.relocated.* classes from bundled-guava module instead.
[BanUnrelocatedGuavaClasses]

@kbendick
Copy link
Contributor

kbendick commented Nov 3, 2021

A few updates needed in the checkstyle descriptions and comments, but this is a good catch.

Is there anything in org.apache.arrow.util we might ever need?

I think it's safe to add this and then it can be reconsidered or made more explicit to the Preconditions check otherwise.

I know that my Intellij autoimports Preconditions from all sorts of places and it's easy to miss.

I went and checked the arrow library, and the only things I found in org.apache.arrow.util were Preconditions and something called AutoCloseables and something called Collections2 (a test class). Doubt we'll need those and we can update this if we do.

EDIT: I also went looking at org.assertj.core.util, and everything was copied from Guava or elsewhere. I think this is safe to do as well and we can also update it need be.

@kbendick
Copy link
Contributor

kbendick commented Nov 3, 2021

cc @nastra who often updates the check style file, but I think this is safe and will help accidentally importing Preconditions and other Guava / google commons things from these libraries that also seem to have forked.

Copy link
Contributor

@kbendick kbendick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked this out locally and run ./gradlew check and everything looked good. 👍

I also added an illegal import and checkstyle failed. Thanks @jackye1995.

Copy link
Contributor

@nastra nastra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changes make sense, what about preventing usage of org.apache.parquet.Preconditions? I believe this is also being used unintentionally at a few places

@jackye1995
Copy link
Contributor Author

changes make sense, what about preventing usage of org.apache.parquet.Preconditions? I believe this is also being used unintentionally at a few places

sounds good, I also want to block the use of org.apache.commons.io.IOUtils in favor of Guava ByteStreams, let me put up another PR for these 2 to avoid too many changes in a single PR.

I will go ahead to merge this one, thanks for the reviews!

@jackye1995 jackye1995 merged commit 1847e65 into apache:master Nov 4, 2021
KnightChess pushed a commit to KnightChess/iceberg that referenced this pull request Nov 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants