CLI, maven, gradle: filter extensions by categories #17765

TomasHofman · 2021-06-08T11:08:20Z

add ability to list extensions by category,
add command to list categories,
extensions ordered alphabetically by artifactId,
consistent maven property names (-Dformat).

devtools/cli/src/main/java/io/quarkus/cli/common/ListFormatOptions.java

TomasHofman · 2021-06-08T11:25:03Z

devtools/cli/src/main/java/io/quarkus/cli/common/CategoryListFormatOptions.java

+import picocli.CommandLine;
+
+public class CategoryListFormatOptions {
+    @CommandLine.Option(names = { "--name" }, order = 1, description = "Display category name only. (default)")


I was thinking if --name format switch should rather be --id, both for extensions and categories (and the same for maven and gradle). Cause it doesn't really show verbose names but artifactIds / category ids. It feels bit confusing to me, plus I don't know what to write in the description - "--name displays category id". :)

@aloubyansky ? I don't have an issue with switching to --id

devtools/cli/src/main/java/io/quarkus/cli/build/GradleRunner.java

devtools/cli/src/main/java/io/quarkus/cli/common/ListFormatOptions.java

devtools/cli/src/main/java/io/quarkus/cli/ProjectExtensions.java

devtools/cli/src/main/java/io/quarkus/cli/ProjectExtensionsCategories.java

...rojects/tools/devtools-common/src/main/java/io/quarkus/devtools/commands/ListCategories.java

...common/src/main/java/io/quarkus/devtools/commands/handlers/ListCategoriesCommandHandler.java

...common/src/main/java/io/quarkus/devtools/commands/handlers/ListExtensionsCommandHandler.java

TomasHofman · 2021-06-09T18:36:48Z

@ebullient thanks for great review! I didn't manage to find more time today, I will get to it again on Monday.

ebullient · 2021-06-17T01:03:02Z

#17966 will intersect with this one, but I don't think they should disagree violently. They just touch some of the same files. Should be straight-forward to rebase on it alongside the other requested changes.

quarkus-bot · 2021-06-17T18:22:42Z

This workflow status is outdated as a new workflow run has been triggered.

🚫 This workflow run has been cancelled.

Failing Jobs - Building `1a63f09`

⚠️ Artifacts of the workflow run were not available thus the report misses some details.

Status	Name	Step	Test failures	Logs	Raw logs
⌛	Initial JDK 11 Build	`Build`	⚠️ Check →	Logs	Raw logs

TomasHofman · 2021-06-17T18:33:19Z

I'm not sure if got your intentions regarding pushing the hints from handlers to gradle and maven plugins, so I pushed it as a separate commit. Going to squash later on if we agree on it.

...common/src/main/java/io/quarkus/devtools/commands/handlers/ListExtensionsCommandHandler.java

ebullient

A few more comments, but way better. Make sure you rebase to pull in #17966. Note that the "list extensions for the current project" is important wording.. it isn't just generally existing categories, it has to do with whether or not the extensions being listed are relative to the platform used by the project, or based on some other factor (no project, or explicit stream/platform specified on command line). #17966 makes that meaning more obvious (I think).

...rojects/tools/devtools-common/src/main/java/io/quarkus/devtools/commands/ListCategories.java

...common/src/main/java/io/quarkus/devtools/commands/handlers/ListCategoriesCommandHandler.java

...common/src/main/java/io/quarkus/devtools/commands/handlers/ListExtensionsCommandHandler.java

quarkus-bot · 2021-06-17T21:18:41Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `839d503`

Status	Name	Step	Test failures	Logs	Raw logs
✔️	JVM Tests - JDK 11
✖	JVM Tests - JDK 11 Windows	`Build`	Test failures	Logs	Raw logs
✔️	JVM Tests - JDK 16

Full information is available in the Build summary check run.

Test Failures

⚙️ JVM Tests - JDK 11 Windows #

📦 extensions/vertx-http/deployment

✖ io.quarkus.vertx.http.testrunner.QuarkusTestTypeTestCase.testQuarkusTestMode line 36 - More details - Source on GitHub

✖ io.quarkus.vertx.http.testrunner.TestChangeTrackingWhenStartFailsTestCase.testChangeTrackingOnStartupFailure line 37 - More details - Source on GitHub

✖ io.quarkus.vertx.http.testrunner.TestRunnerSmokeTestCase.checkTestsAreRun line 40 - More details - Source on GitHub

✖ io.quarkus.vertx.http.testrunner.UnitTestTypeTestCase.testUnitMode line 36 - More details - Source on GitHub

✖ io.quarkus.vertx.http.testrunner.brokenonly.TestBrokenOnlyTestCase.testBrokenOnlyMode line 39 - More details - Source on GitHub

✖ io.quarkus.vertx.http.testrunner.includes.ExcludePatternTestCase.checkTestsAreRun line 40 - More details - Source on GitHub

✖ io.quarkus.vertx.http.testrunner.includes.IncludePatternTestCase.checkTestsAreRun line 40 - More details - Source on GitHub

✖ io.quarkus.vertx.http.testrunner.params.TestParameterizedTestCase.testParameterizedTests line 38 - More details - Source on GitHub

✖ io.quarkus.vertx.http.testrunner.tags.ExcludeTagsTestCase.checkTestsAreRun line 39 - More details - Source on GitHub

✖ io.quarkus.vertx.http.testrunner.tags.IncludeTagsTestCase.checkTestsAreRun line 39 - More details - Source on GitHub

quarkus-bot · 2021-06-21T08:47:19Z

This workflow status is outdated as a new workflow run has been triggered.

🚫 This workflow run has been cancelled.

Failing Jobs - Building `6031a05`

⚠️ Artifacts of the workflow run were not available thus the report misses some details.

Status	Name	Step	Test failures	Logs	Raw logs
⌛	Gradle Tests - JDK 11	`Build`	⚠️ Check →	Logs	Raw logs
⌛	Gradle Tests - JDK 11 Windows	`Build`	⚠️ Check →	Logs	Raw logs
⌛	JVM Tests - JDK 11	`Build`	⚠️ Check →	Logs	Raw logs
⌛	JVM Tests - JDK 11 Windows	`Build`	⚠️ Check →	Logs	Raw logs
⌛	JVM Tests - JDK 16	`Build`	⚠️ Check →	Logs	Raw logs
⌛	Maven Tests - JDK 11 Windows	`Build`	⚠️ Check →	Logs	Raw logs
⌛	MicroProfile TCKs Tests	`Verify`	⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Amazon	`Build`	⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Cache		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Data1	`Build`	⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Data3	`Build`	⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Data4	`Build`	⚠️ Check →	Logs	Raw logs
⌛	Native Tests - HTTP		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Messaging1		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Messaging2		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Misc1		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Misc2		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Misc3		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Misc4		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Security1		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Security2		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Security3		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Spring		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Windows - hibernate-validator		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - gRPC		⚠️ Check →	Logs	Raw logs

TomasHofman · 2021-06-21T09:22:44Z

BTW moving all hints away from handlers to the CLI module doesn't work that well for the GradleRunner. This runner actually executes the gradle listSomething command to print the list, which means that the output looks like:

$ quarkus ext ls -i
// lots of gradle log output
// list of extensions
// another gradle log output
// hints...

i.e. the hints are separated from the extensions list by plenty of gradle logging - they are hard to find and it doesn't looks sensible...

For this reason I will try to make the CLI not to print any hints if gradle runner is used, and delegate that to the gradle task instead. It means there will be couple more conditions in gradle task, and the decoupling from CLI won't be perfect, but IMO it can't stay this way.

quarkus-bot · 2021-06-21T10:06:31Z

This workflow status is outdated as a new workflow run has been triggered.

🚫 This workflow run has been cancelled.

Failing Jobs - Building `b87d50a`

⚠️ Artifacts of the workflow run were not available thus the report misses some details.

Status	Name	Step	Test failures	Logs	Raw logs
⌛	Gradle Tests - JDK 11 Windows	`Build`	⚠️ Check →	Logs	Raw logs
⌛	JVM Tests - JDK 11	`Build`	⚠️ Check →	Logs	Raw logs
⌛	JVM Tests - JDK 11 Windows	`Build`	⚠️ Check →	Logs	Raw logs
⌛	JVM Tests - JDK 16	`Build`	⚠️ Check →	Logs	Raw logs
⌛	Maven Tests - JDK 11 Windows	`Build`	⚠️ Check →	Logs	Raw logs
⌛	MicroProfile TCKs Tests	`Verify`	⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Amazon		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Cache		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - HTTP		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Messaging1		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Messaging2		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Misc1		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Misc2		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Misc3		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Misc4		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Security1		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Security2		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Security3		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Spring		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - Windows - hibernate-validator		⚠️ Check →	Logs	Raw logs
⌛	Native Tests - gRPC		⚠️ Check →	Logs	Raw logs

TomasHofman · 2021-06-21T10:13:19Z

I think most should be done. From my side I need to clarify:

1. The position of hints - which put before or after the extension / category list. Currently:

Current Quarkus extensions installable/installed/available: 

// the list

To get more information, append `--full` to your command line.

To list only extensions from specific category, append `--category "categoryId"` to your command line.

Add an extension to your project by adding the dependency to your build.gradle or use `quarkus extension add "artifactId"`

and

Available Quarkus extension categories: 

// the list

To get more information, append `--full` to your command line.

To list extensions in given category, use:
`quarkus extensions list --installable --category "categoryId"``

2. I didn't find how to obtain the executable name or alias which was used to run the CLI command, so I still use quarkus in the hints. Do we already know what command name will be used in documentation etc? (I thought there are ideas like qs vs quarkus.)

ebullient · 2021-06-21T19:39:03Z

The command as defined by the CLI is quarkus [1]

I don't think it is right to have to go down into the bowels of the gradle plugin to fix a message about cli options (same for the maven plugin, or for the jbang plugin when it eventually supports listing things like this).

As an option, instead of immediately returning at [2], we could store the result.
If the operation to list extensions was successful (and the right combination of options was selected), emit hints about how to get more detailed information (or, if want hints first, emit them before calling the runner)
Then return the result.

[1]

quarkus/devtools/cli/src/main/java/io/quarkus/cli/QuarkusCli.java

Line 26 in 6d738a1

    
           @CommandLine.Command(name = "quarkus", versionProvider = Version.class, subcommandsRepeatable = false, mixinStandardHelpOptions = true, subcommands = {

[2]

quarkus/devtools/cli/src/main/java/io/quarkus/cli/ProjectExtensionsList.java

Line 59 in 6d738a1

return listPlatformExtensions();

TomasHofman · 2021-06-22T09:27:54Z

As an option, instead of immediately returning at [2], we could store the result.
If the operation to list extensions was successful (and the right combination of options was selected), emit hints about how to get more detailed information (or, if want hints first, emit them before calling the runner)
Then return the result.

That's what I already do... Simply calling the gradle runner you get the extensions list surrounded by some gradle noise from both the beginning and the end.

But actually the gradle noise and the end is not as long as the noise at the beggining of the runner ouput. I was still thinking of having some hints at the beginning yesterday, but having hints at the end only looks OK (and I would prefer that anyway).

Example:

$ quarkus ext ls -i -c core

[gradle noise at the beginning...]

Current Quarkus extensions installable: 

quarkus-consul-config                             
quarkus-logging-gelf                              
quarkus-logging-json                              
quarkus-logging-sentry                            
:listExtensions (Thread[Execution worker for ':',5,main]) completed. Took 0.085 secs.

BUILD SUCCESSFUL in 1s
1 actionable task: 1 executed

To get more information, append `--full` to your command line.

Add an extension to your project by adding the dependency to your build.gradle or use `quarkus extension add "artifactId"`

TomasHofman · 2021-06-22T09:29:46Z

I reverted the last part - hints are now printed from CLI / gradle runner / maven runner depending on which tool was executed by a user.

quarkus-bot · 2021-06-22T09:43:30Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `e55fc3c`

Status	Name	Step	Test failures	Logs	Raw logs
✖	Initial JDK 11 Build	`Build`	⚠️ Check →	Logs	Raw logs

ebullient

One nit, and then squash to one commit, and we're good to go!

Thanks for sticking with this and working through feedback, I appreciate it! =)

devtools/cli/src/main/java/io/quarkus/cli/ProjectExtensionsList.java

quarkus-bot · 2021-06-22T14:08:54Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `a534b9d`

Status	Name	Step	Test failures	Logs	Raw logs
✖	JVM Tests - JDK 11	`Build`	Test failures	Logs	Raw logs
✖	JVM Tests - JDK 11 Windows	`Build`	Test failures	Logs	Raw logs
✖	JVM Tests - JDK 16	`Build`	Test failures	Logs	Raw logs

Full information is available in the Build summary check run.

Test Failures

⚙️ JVM Tests - JDK 11 #

📦 integration-tests/scala

✖ io.quarkus.scala.maven.it.ScalaDevModeIT.testThatTheApplicationIsReloadedOnScalaChange line 23 - More details - Source on GitHub

✖ io.quarkus.scala.maven.it.ScalaRemoteDevModeIT.testThatTheApplicationIsReloadedOnScalaChange line 25 - More details - Source on GitHub

⚙️ JVM Tests - JDK 11 Windows #

📦 integration-tests/scala

✖ io.quarkus.scala.maven.it.ScalaDevModeIT.testThatTheApplicationIsReloadedOnScalaChange line 23 - More details - Source on GitHub

✖ io.quarkus.scala.maven.it.ScalaRemoteDevModeIT.testThatTheApplicationIsReloadedOnScalaChange line 25 - More details - Source on GitHub

⚙️ JVM Tests - JDK 16 #

📦 integration-tests/scala

✖ io.quarkus.scala.maven.it.ScalaDevModeIT.testThatTheApplicationIsReloadedOnScalaChange line 23 - More details - Source on GitHub

✖ io.quarkus.scala.maven.it.ScalaRemoteDevModeIT.testThatTheApplicationIsReloadedOnScalaChange line 25 - More details - Source on GitHub

* add ability to list extensions by category, * add command to list categories, * extensions ordered alphabetically by artifactId, * consistent maven property names (-Dformat rather than -Dquarkus.extensions.format), * --name switch renamed to --id (--name still works though)

TomasHofman · 2021-06-23T05:22:48Z

@ebullient thanks for your time reviewing this! Commits are squashed, but let me know should notice anything more...

quarkus-bot · 2021-06-23T08:20:36Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `feb2c6d`

Status	Name	Step	Test failures	Logs	Raw logs
⌛	MicroProfile TCKs Tests	`Verify`	⚠️ Check →	Logs	Raw logs

quarkus-bot · 2021-06-23T16:01:15Z

Failing Jobs - Building `1a1131d`

Status	Name	Step	Test failures	Logs	Raw logs
✖	Native Tests - Data3	`Build`	⚠️ Check →	Logs	Raw logs

gsmet · 2021-07-05T15:31:13Z

@ebullient please do not merge PRs with merge commits they are a pain to cherry-pick...

ebullient · 2021-07-05T20:48:19Z

@ebullient please do not merge PRs with merge commits they are a pain to cherry-pick...

sorry..

TomasHofman mentioned this pull request Jun 8, 2021

CLI: list --installable should be sorted #16154

Closed

TomasHofman commented Jun 8, 2021

View reviewed changes

devtools/cli/src/main/java/io/quarkus/cli/common/ListFormatOptions.java Show resolved Hide resolved

TomasHofman commented Jun 8, 2021

View reviewed changes

ebullient requested changes Jun 8, 2021

View reviewed changes

ebullient mentioned this pull request Jun 8, 2021

showing massive list of extensions does not scale - can we improve ? #16275

Closed

TomasHofman force-pushed the ext-ls-output branch 2 times, most recently from 1a63f09 to 839d503 Compare June 17, 2021 18:22

TomasHofman commented Jun 17, 2021

View reviewed changes

...common/src/main/java/io/quarkus/devtools/commands/handlers/ListExtensionsCommandHandler.java Outdated Show resolved Hide resolved

ebullient requested changes Jun 17, 2021

View reviewed changes

TomasHofman force-pushed the ext-ls-output branch from 6031a05 to b87d50a Compare June 21, 2021 08:44

TomasHofman force-pushed the ext-ls-output branch from 43348a3 to e55fc3c Compare June 22, 2021 09:27

TomasHofman force-pushed the ext-ls-output branch from e55fc3c to a534b9d Compare June 22, 2021 10:54

ebullient reviewed Jun 22, 2021

View reviewed changes

devtools/cli/src/main/java/io/quarkus/cli/ProjectExtensionsList.java Show resolved Hide resolved

ebullient mentioned this pull request Jun 22, 2021

CLI: Extensions list enhancements #18062

Closed

TomasHofman force-pushed the ext-ls-output branch from a534b9d to feb2c6d Compare June 23, 2021 05:08

ebullient approved these changes Jun 23, 2021

View reviewed changes

Merge branch 'main' into ext-ls-output

1a1131d

ebullient merged commit bb209ae into quarkusio:main Jun 23, 2021

quarkus-bot bot added this to the 2.1 - main milestone Jun 23, 2021

gsmet added the triage/backport? label Jul 5, 2021

gsmet modified the milestones: 2.1 - main, 2.0.1.Final Jul 5, 2021

gsmet removed the triage/backport? label Jul 5, 2021

CLI, maven, gradle: filter extensions by categories #17765

CLI, maven, gradle: filter extensions by categories #17765

Conversation

TomasHofman commented Jun 8, 2021 • edited Loading

TomasHofman Jun 8, 2021 • edited Loading

Choose a reason for hiding this comment

ebullient Jun 8, 2021

Choose a reason for hiding this comment

TomasHofman commented Jun 9, 2021

ebullient commented Jun 17, 2021

quarkus-bot bot commented Jun 17, 2021 • edited Loading

Failing Jobs - Building 1a63f09

TomasHofman commented Jun 17, 2021

ebullient left a comment • edited Loading

Choose a reason for hiding this comment

quarkus-bot bot commented Jun 17, 2021 • edited Loading

Failing Jobs - Building 839d503

Test Failures

⚙️ JVM Tests - JDK 11 Windows #

📦 extensions/vertx-http/deployment

quarkus-bot bot commented Jun 21, 2021 • edited Loading

Failing Jobs - Building 6031a05

TomasHofman commented Jun 21, 2021 • edited Loading

quarkus-bot bot commented Jun 21, 2021 • edited Loading

Failing Jobs - Building b87d50a

TomasHofman commented Jun 21, 2021 • edited Loading

ebullient commented Jun 21, 2021 • edited Loading

TomasHofman commented Jun 22, 2021

TomasHofman commented Jun 22, 2021

quarkus-bot bot commented Jun 22, 2021 • edited Loading

Failing Jobs - Building e55fc3c

ebullient left a comment

Choose a reason for hiding this comment

quarkus-bot bot commented Jun 22, 2021 • edited Loading

Failing Jobs - Building a534b9d

Test Failures

⚙️ JVM Tests - JDK 11 #

📦 integration-tests/scala

⚙️ JVM Tests - JDK 11 Windows #

📦 integration-tests/scala

⚙️ JVM Tests - JDK 16 #

📦 integration-tests/scala

TomasHofman commented Jun 23, 2021

quarkus-bot bot commented Jun 23, 2021 • edited Loading

Failing Jobs - Building feb2c6d

quarkus-bot bot commented Jun 23, 2021

Failing Jobs - Building 1a1131d

gsmet commented Jul 5, 2021

ebullient commented Jul 5, 2021

TomasHofman commented Jun 8, 2021 •

edited

Loading

TomasHofman Jun 8, 2021 •

edited

Loading

quarkus-bot bot commented Jun 17, 2021 •

edited

Loading

Failing Jobs - Building `1a63f09`

ebullient left a comment •

edited

Loading

quarkus-bot bot commented Jun 17, 2021 •

edited

Loading

Failing Jobs - Building `839d503`

quarkus-bot bot commented Jun 21, 2021 •

edited

Loading

Failing Jobs - Building `6031a05`

TomasHofman commented Jun 21, 2021 •

edited

Loading

quarkus-bot bot commented Jun 21, 2021 •

edited

Loading

Failing Jobs - Building `b87d50a`

TomasHofman commented Jun 21, 2021 •

edited

Loading

ebullient commented Jun 21, 2021 •

edited

Loading

quarkus-bot bot commented Jun 22, 2021 •

edited

Loading

Failing Jobs - Building `e55fc3c`

quarkus-bot bot commented Jun 22, 2021 •

edited

Loading

Failing Jobs - Building `a534b9d`

quarkus-bot bot commented Jun 23, 2021 •

edited

Loading

Failing Jobs - Building `feb2c6d`

Failing Jobs - Building `1a1131d`