Skip to content

Conversation

breskeby
Copy link
Contributor

@breskeby breskeby commented Nov 16, 2020

This PR splits the precommit checks into internal and external as we do not want to enforce Elasticsearch build specific internal checks being enforced on external plugin authors.

On the way also ported some of the touched classes from groovy to java and introduced the general notion of an InternalPlugin.

Fixes #63697

@breskeby breskeby self-assigned this Nov 16, 2020
@breskeby breskeby added :Delivery/Build Build or test infrastructure Team:Delivery Meta label for Delivery team v7.11.0 v8.0.0 labels Nov 16, 2020
@breskeby breskeby force-pushed the seperate-internal-precommit branch from 81198e1 to 670ee71 Compare November 17, 2020 10:13
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removing JarHellPrecommitPlugin and ThirdPartyAuditPrecommitPlugin here

Copy link
Contributor Author

@breskeby breskeby Nov 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only need this for internal builds

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the actual change in this class except porting it to java

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we removed jarhell and thirdpartyaudit from InternalPrecommitTasks what then applies those? Shouldn't we always apply PrecommitTasks, not just as part of this else condition?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PrecommitTasks are applied within InternalPrecommitTasks

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

address package change

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we only wanna test the testing convention plugin here

@breskeby
Copy link
Contributor Author

I made a general comment related to this on #63697 (comment) regarding backwards compatibility and deprecation cycles. This PR as is is a breaking change for likely most external plugin authors out there and we might want to reconsider how to approach this and be a bit more conservative in introducing this change.

@breskeby breskeby requested review from jakelandis and mark-vieira and removed request for jakelandis November 17, 2020 12:12
@breskeby breskeby marked this pull request as ready for review November 17, 2020 13:49
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-delivery (Team:Delivery)

Copy link
Contributor

@mark-vieira mark-vieira left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we migrated a few Groovy files to Java but didn't move them to the java source set. We ideally don't want to rely on joint compilations sicne it's incredibly slow. If moving to java isn't possible because of all the dependencies that'd have to move over too that's fine, let's just keep as Groovy for now rather than rely on Groovy joint-compilation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be true?

Copy link
Contributor Author

@breskeby breskeby Nov 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

true, fixed

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would comment above but GitHub sucks. We should similarly wrap all the isModule stuff above in this check. For external projects it will always be a plugin, not a module, so we shouldn't accidentally treat it as such if an external author incidentally uses conflicting project naming conventions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer we just stick with the slightly more verbose streams api here than rely on internal Gradle util classes. Or introduce our own util method.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now kept this to be groovy so reverted this change

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we removed jarhell and thirdpartyaudit from InternalPrecommitTasks what then applies those? Shouldn't we always apply PrecommitTasks, not just as part of this else condition?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we go ahead and port this to Java too?

Copy link
Contributor Author

@breskeby breskeby Nov 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unfortunately we can't at the moment. Whats blocking porting most groovy classes here to java is the AntTask class and classes inheriting this one. It makes use of groovys ant builder. I'm sure we can rewrite the LicenseHeadersTask task in a way to not rely on ant under the hood and then move most of the precommit stuff over to plain java. But not as part of this PR

@breskeby breskeby force-pushed the seperate-internal-precommit branch from 3421ece to ece5df5 Compare November 18, 2020 16:18
@breskeby breskeby merged commit 46d2681 into elastic:master Nov 19, 2020
@breskeby breskeby deleted the seperate-internal-precommit branch November 19, 2020 08:45
breskeby added a commit to breskeby/elasticsearch that referenced this pull request Nov 23, 2020
* Split precommit plugins into internal and external
* Introduce interface for InternalPlugins
* avoid joint java groovy compilation
* Move isModule handling into internal only
breskeby added a commit that referenced this pull request Nov 24, 2020
* Split precommit plugins into internal and external
* Introduce interface for InternalPlugins
* avoid joint java groovy compilation
* Move isModule handling into internal only
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Delivery/Build Build or test infrastructure Team:Delivery Meta label for Delivery team v7.11.0 v8.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Expose only a subset of precommit tasks to 3rd party developers
4 participants