Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Design specification for improved Console output #1063

Merged
merged 9 commits into from
Jan 25, 2017

Conversation

eriwen
Copy link
Contributor

@eriwen eriwen commented Dec 23, 2016

This is a first pass at a Console implementation that displays background work and parallel work to the user.

* User extensibility

### User Goals for Console
In priority order, the user wants to answer the following questions:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd think different users have different priorities. Personally, I'd be mostly interested in "what Gradle is going now" as top priority.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly that those 2 priorities are actually high on users' lists and we're addressing them is more important than the ordering. How would you change the look of the Console if these goals were reordered?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's important to indicate a priority in the introduction. I would have just said that these are the goals with out a specific order. The priority of work items is already laid out by the stories below.

* Is my build successful?
* What command did I execute?

## Story: Display parallel work in-progress independently by operation
Copy link
Contributor

@bmuschko bmuschko Dec 23, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I understand this story lays the ground work for customizing individual portions of the new console display layout. I'd try to reflect that in the title. Follow up stories describe the exact look & feel + behavior. Is this story meant to be a general description without the actual implementation? If we do implement something in this story how will the individual portions of RichConsole look like?

Show incomplete ProgressOperations on separate lines, up to a specified maximum number.

### Implementation
A `RichConsole` has
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You use these class names? Could we spell out what they actually are, what their purpose is and where they live? Without trying to look it up I am assuming they are existing Gradle core classes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are yet-to-be created interfaces for the most part, intended to be an attempt at domain modeling.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add the interfaces in this story including their proposed methods to get a better idea. That would outline their concrete responsibilities.


The `ProgressLabel` is intended to give the user a glance-able indicator that tells
them whether the build will be finished soon or not.
See ["Display build progress as a progress bar through colorized output"](TODO) below.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing link

* What command did I execute?

## Story: Display parallel work in-progress independently by operation
Show incomplete ProgressOperations on separate lines, up to a specified maximum number.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A simple mock up would help a lot with visualizing the end result.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See our internal projects board for a demo. I have other demos pending.

* Renders well on default macOS Terminal, default Windows Console, Cygwin, and common Linux Terminals
* Console display degrades gracefully given BuildOperations that are out-of-order (e.g. complete event received before start)

### Open issues
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have appropriate fallback logic that kicks in if the console doesn't support the feature? What behavior + look & feel does it fall back to?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, currently we have logic that disables the AnsiConsole if we detect that the attached console doesn't support color or is "dumb". You can get a feel for what this'd look like if you use TERM=dumb ./gradlew task

* Is my build successful?
* What command did I execute?

## Story: Display parallel work in-progress independently by operation
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How will the functionality be introduce to the user? Is it an opt-in feature? Can be turned off/on.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. I'll answer this in the spec.

[terminfo](https://en.wikipedia.org/wiki/Terminfo)-based Console handling instead of using ANSI.

### Test Coverage
* A terminal size with fewer than max parallel operations shows (rows - 2) operations in parallel
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the expected behavior for that test case?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should clarify the statement. The key part is "shows (rows - 2) operations in parallel"

### Test Coverage
* A terminal size with fewer than max parallel operations shows (rows - 2) operations in parallel
* Operation status lines are trimmed at (cols - 1)
* `System.in` and SystemConsole I/O happens on the mainArea, which is unaffected by other areas
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean we'll support this use case (see corresponding JIRA + GitHub issues)? I think that should be a separate story.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My hope is that we can implement this in such a way as to avoid the existing bug thereby not requiring "separate" work as opposed to a naive implementation that still breaks + a story to fix it.

* Operation status lines are trimmed at (cols - 1)
* `System.in` and SystemConsole I/O happens on the mainArea, which is unaffected by other areas
* Renders well on default macOS Terminal, default Windows Console, Cygwin, and common Linux Terminals
* Console display degrades gracefully given BuildOperations that are out-of-order (e.g. complete event received before start)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is meant by "degrade"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will change to "gracefully handles"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be baked into the BuildOperation contract. Consumers should never receive complete events before start events, and producers should have no way to generate a complete event before a start event.


### Open issues
* How is `--continuous` build displayed? During execution? While waiting? (as we allow processes to stay running)
* When things fail, will the new CLI look like a regular failure or something else?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can get to error messages/stack traces/warnings that occurred during the build but have been overwritten?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They shall not be overwritten because they'll be written with log level WARN or higher and thus appear in the "main" TextArea.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not clear to me what happens if you get a lot of these message. From my what I understand the text are will show "what's currently going on the build" + warning and error messages. What happens if it fills up? How do we visually separate these two types of output?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find it helpful to imagine it like this: The output looks exactly the same as it does today (with the "status" at the bottom) except that:

  • That status line is many lines instead of 1
  • Lifecycle logs are no longer output

We could visually separate these things by adding an empty line above the "status" text area if necessary.

* Console display degrades gracefully given BuildOperations that are out-of-order (e.g. complete event received before start)

### Open issues
* How is `--continuous` build displayed? During execution? While waiting? (as we allow processes to stay running)
Copy link
Contributor

@bmuschko bmuschko Dec 23, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add some sort of indicator (like an icon or text with colorized background) to the text area that indicates that we are running a continuous build?

### User-visible Changes
We can visually represent complete, in-progress, and un-started tasks of build using colorized output:

`#####green#####>##yellow##>#####black#####> 40% Building`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How's the percentage calculated? Based on the number of tasks as we have it right now? Will there be any more intelligence built-in? Should say what's out of scope.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We currently already calculate and display this percentage in the Status Bar. IIUC, it's the % of tasks complete from the calculated task graph at configuration time. We won't change this behavior IMO until we have sophisticated/historical timing data.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, that should be explicitly spelled out in the spec.


Intended to give the user a very fast way of telling whether the build will be finished soon or not.
The work-in-progress (yellow) section shows how much of the build would be complete if all
Operations displayed in the progressArea completed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Colorized output needs explanation and documentation. Maybe that is already an anti-pattern as it should be self-explanatory?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please follow up with me on Slack about this. I'm confused by this question.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think what @bmuschko means is that once you start having different colors in the progress bar, these colors need to be explained to the user or they need to have an immediately apparent meaning. Looking at progress bars in other programs I've rarely if ever seen a multi-color bar. It's usually a single solid color filling out an empty/neutral space.

support them
* ASCII-based, unstyled alternatives:
```
[## ] 6% Building
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 for this option

Each`RegionType` is associated with a color. This allows us to show failures in arbitrary locations
as needed with `--continue`

### Test Coverage
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the output behavior if run from CI or and IDE?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neither CI or IDEs attach to a Console, so their behavior will not change at all except for additional information they'll see from increased BuildOperations.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the output would look as it looks right now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, exactly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Depends whether the CI or IDE are using LongRunningOperation.setColorOutput(true) when they run Gradle via the tooling API. We should test this in some form.

On ANSI terminals, we can use empty spaces with different background colors for the "bars" and
A red background shall be used for failed tasks.

### Implementation
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs to explain changes to TAPI

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hope there aren't any changes to TAPI necessary, just handling of additional types of BuildOperations. That has yet to be speced

* Logs should be streamed with plain output and not throttled when not attached to a Console
* The TAPI is unaffected except for additional BuildOperations now published
* Logs should be streamed with plain output when attached Terminal lacks of color or cursor support
* Renders well on default macOS Terminal, default Windows Console, Cygwin, PowerShell and common Linux Terminals
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will require extensive test fixtures and testing infrastructure e.g. we can't even test MACOS behavior via CI.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rendering itself will be difficult to test as we don't know what font is used for the console which significantly affects display. Asciinema inspires me for this purpose, though. Can we use that somehow?

«==========================» 100% BUILD SUCCESS
```

## Story: Display intra-operation progress
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should pick individual use cases and break them into different stories. I think an easy one would be "resolving dependencies". What's the behavior if executed in parallel?

When we know in advance the number of things to be processed, we display progress in [Complete / All] format.
Provide APIs to optionally add # complete and # of all items.

For example:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am missing a story that I consider a quick win. Render the completion of different phases in Gradle's lifecycle. Done initializing, done configuring, done executing. And then within these phases render individual events.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's proposed below, though I disagree. I don't think it'd be quick and IMO I'm not yet convinced the value would be high. I'm happy to be convinced, though. Perhaps you can modify one of my demos (linked to internal issue) to show me what you're thinking.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we would just have to register a Listener and then render that we reached a certain stage. I think it would give a good indication of "as a user i want to know when the actual of execution of tasks really starts", "if configuration takes longer than I expected it to take - maybe there's something wrong my code".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you watch the demo closely, we already show "Initializing, Configuring, and Executing" on the progress bar. They're just fast and are replaced with Executing quickly. Is that sufficient?

### Open issues
* Could this concept allow customization of ProgressLine format, providing even greater flexibility?

## Story: Gradle-provided slick themes
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd move this story before the previous one. We should ship with an existing theme before we make it customizable for the end user.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll respond to this in the design doc and with a demo, but there will be an ASCII based existing theme. However, this is a blocker for release IMO.

### Open issues
* Color choices we would use if we could

## Story: (Optional) Illustrate build phases independently
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my comment above. I don't think this should be optional and should be moved before customizing things by the end user.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm looking forward to a concrete suggestion about what this would look like.

* Operation status lines are trimmed at (cols - 1)
* `System.in` and SystemConsole I/O happens on the mainArea, which is unaffected by other areas
* Renders well on default macOS Terminal, default Windows Console, Cygwin, and common Linux Terminals
* Console display degrades gracefully given BuildOperations that are out-of-order (e.g. complete event received before start)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be baked into the BuildOperation contract. Consumers should never receive complete events before start events, and producers should have no way to generate a complete event before a start event.

and errors.

`--quiet`, `--info`, and `--debug` log level flags affect the output visible in the main TextArea,
However, the default will change so that `LIFECYCLE` logs are no longer displayed.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something that is not entirely clear is how this relates to what we show in the main area now. I read this as saying that the main area will show exactly the same content as it does now, minus LIFECYCLE log messages.

It would also be useful to make clearer what happens when the build process is not attached to a terminal, in general.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would think about including some context when we log an important message, as often the warning/error messages don't include enough context for a human to know which build operation (such as a task) the message relates to. This is particularly important when running in parallel, and logging things like compile error messages.

Currently the lifecycle logging serves this purpose (not perfectly), so if we remove it, we should add something else to help the user make sense of the error and logging messages.

Each`RegionType` is associated with a color. This allows us to show failures in arbitrary locations
as needed with `--continue`

### Test Coverage
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Depends whether the CI or IDE are using LongRunningOperation.setColorOutput(true) when they run Gradle via the tooling API. We should test this in some form.

Copy link
Member

@eskatos eskatos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This spec adds a lot of system properties for UI customization purpose. I'm not sure we want to go down that route. Another extension mechanism could be used, e.g. by letting users provide custom renderers/themes as a jar/class/plugin/extension. I don't know how it could work in details though.

org.gradle.console.progressbar.inprogress.suffix=""
org.gradle.console.progressbar.unstarted.suffix=""
org.gradle.console.progressbar.completechar=" "
org.gradle.console.progressbar.completechar="#"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

org.gradle.console.progressbar.completechar is present twice with a different value

eriwen added a commit that referenced this pull request Jan 5, 2017
- Provide more detail about parallel WIP is displayed and
implemented
- Add stories improve logging to give more context

Issue: #1063
@eriwen
Copy link
Contributor Author

eriwen commented Jan 5, 2017

@eskatos I investigated some ways to customize the Console output through provided classes, and I believe it would add significant implementation complexity and usage complexity. I removed 3 properties in the latest revision. If you think that is still problematic, can you explain why further?

Copy link
Contributor

@oehme oehme left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't wait to actually see this in action :)

One thing the spec doesn't tackle at the moment is how nested operations are displayed. A worker might be executing the compileMainCpp task, which in turn is broken down into header compilation and object file compilation, which in turn could be broken down into per-file operations. What would the status line for that worker show? If we only show the lower-most operation, we'd get incomprehensible flickering. If we only show the top-most one we loose information in case something does actually take a long time. Maybe we could use some nested structure using > characters? Executing compileMainCpp > Compiling Headers > Compiling Foo.h.

## Primary Goals
* Improve perceived performance by making user aware of background work
* Modern feel
* User extensibility
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to challenge this goal. What is the impact of being able to change the color theme/progress bar characters? What big problem does this solve or what exciting new development does it enable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the difference between plain output and rich output — Given the current spec, I believe that if we want to achieve the latter (and I think we do) it is very little additional effort to allow users to do so. Node and Python tools have achieved a sense of modernism by including pretty (colorized, non-ASCII) output in their tools, and I think we should do the same.

Here the reasons I think that could be beneficial:

  • It gives users a new vehicle for contribution for Gradle. I believe someone out there has some crazy awesome ideas for the console UI
  • It gives users an easy way for colorized output to match the rest of their Terminal.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm on board with rich output, but doubt the value of configurability, at least as a main goal. I think it adds significant overhead to implementation, testing and documentation. I'd prefer us focusing our energy on a great default that is in line with our brand before we tackle configurability.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're absolutely right — it is not a main goal and we'll consider it an optional last milestone.

* What is Gradle doing right now?
* Is my build successful?
* What command did I execute?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Are there any other interesting outcomes, except success/failure? E.g. build scan
  • How much work did Gradle actually do vs. how much did Gradle save me (e.g. up-to-date, from-cache)?
  • How well is the build using the system resources?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the first point should be added. I hesitate to consider the latter two. I personally get the sense that users really do not care at all about how Gradle achieve it's result. If they do, that's what build scans are for!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is valuable to have a quick overview how much work was really done. We've seen on several occasions that users aren't even aware that their build is wrongly out of date because the many log messages make that hard to see. Now we are removing those log messages, so there is even less context to notice a problem. Just having a single percentage (work avoidance rate or something) could help raise awareness. I realize build scans show all this and much more, but consider this: Many users aren't aware that their build has a problem and just assume that a 10s turnaround is normal, so the idea of creating a scan doesn't even occur to them. This leads to the wrong sentiment of "Gradle is slow", when really they would just need to fix their build.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 for what Stefan said. This is also related to something we want to show for task output caching: https://github.com/gradle/task-output-cache/issues/302

## Story: Display parallel work in-progress independently by operation
Show incomplete ProgressOperations on separate lines, up to a specified maximum number.

This is opt-in initially by specifying the `org.gradle.console.parallel=true` Gradle property.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reason for making this opt-in or configurable at all?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case I'm considering milestone releases where the console itself isn't usable for a given project — allowing a user to turn it off. This is a good question, though, I'd really, really like to not need this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay I think we should call this out. For the 4.0 release this should be on by default, maybe with an opt out. And at some point we'll want to remove the code for the old output structure.

The user wants to answer the following questions:

* When will my build finish?
* Soon (should I keep watching it) or later ("I'll check back")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would Gradle know when it will finish?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gradle doesn't yet know when it'll finish, but it does know if it's working on the last task. That is something we're not surfacing to the user and should.

with an optional worker ID to provide information about "where" an operation is running
so that a mapping of worker to index of operation status line can be maintained.

A `OperationStatusBarFormatter` can be used to customize the intra-line display of each
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the use case for making this configurable? Why not use the build operation's display name in all cases for consistency?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We currently have a DefaultStatusBarFormatter that adds the >s — this is very similar to that and doesn't have any user facing implications. In later stories, this might become configurable, but it's optional.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay I stumbled over the "can be used to customize". That sounds like a user api

`org.gradle.console.theme=powerline` and substitute `PowerlineProgressBarRegion`
instead of the `DefaultProgressBarRegion`.

## Story: (Optional) Customizable StatusBarFormat
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know I'm repeating myself now :) I don't think this is important and would rather have a consistent experience.

### Open issues
* How is intra-operation progress ([55 / 1234] or spinner) affected?

## Story: (Optional) Indicate Gradle's continued progress in absence of intra-operation updates
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this one should not be optional, as it very much affects perceived performance. I'd prioritize this over any kind of customizability.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple notes about this. I think you're almost certainly right that we need this. The thing that sucks is that everything that looks polished isn't compatible with nearly all terminal environments (console size, color, font, font-size, etc). I'll think harder about how this might look, but so far I've only come up with ugly solutions.

Support detection and use of UCS-2 (default supported by Windows) or UTF-8 characters could further polish this if we choose.

This would allow us to use characters like ᐅ or ► supported by a majority of monospace fonts.
More granular width for progress bars: ' ', '▏', '▎', '▎', '▍', '▍', '▌', '▌', '▋', '▋', '▊', '▊', '▉', '▉', '█'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we don't know what portion of a build a task represents, I don't think we'll need different slice sizes. I'd just use a pretty small increment by default and draw as many of them as we need to visualize the completed percentage.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The vast majority of terminal fonts are monospace and so take up the same horizontal space regardless of the character, therefore we cannot draw many small increments — we must do N "wide" bars and then a "thinner" one.

On that note, we don't have control over the fonts used. Some may not have implementations for these characters and may display as placeholders (like some kind of boxed [x]).

This is one major reason I advocate for themes. Everyone's terminal environment is different. We cannot hope to develop a single "pretty" theme that works for everyone. I believe we must give users more options (even better, create their own)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would assume (and that may be wrong) we can develop one that looks great for the 80% case. I'd worry about theming later/if we actually get requests.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's see what we can come up with. I think it's going to be really tough to get one that looks great for 80% of users. I want something that is viral because it's so awesome — that's likely not achievable.

supports advanced features.

## Story: (Optional) "Fade out" operation result
Tasks completed continue to be displayed for 1 more "tick" of the clock with their resolution status,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is slow enough so I can actually read it, it'll hide what is actually being worked on right now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm... we can probably remove this story. It would be cool but probably not practical.

* Color choices we would use

## Story: (Optional) Illustrate build phases independently
Similar to "Display build progress as a progress bar through colorized output" above, but having clearly separated output for each build phase.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prioritize this over any customization. It improves several things:

  • Configuration time is usually a very small part of the build, so it would only be a single tick on the progress bar. Seeing a whole bar fill up quickly instead is much more engaging.
  • We don't even know how many tasks we'll execute before we calculate the task graph. By starting over we can size the increments correctly

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the confusion here. The current demo actually fills up the whole bar for each phase separately, it's just very fast. That is, we do already start over.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this is about adding a phase label next to it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that should be in the Progress Bar story. If not, I'll add it.

@corneil
Copy link

corneil commented Jan 17, 2017

One thing that would be helpful is the ability to log detail to a file like with log4j or such and configuring log level for that file then a CI build can be fairly quiet and you can dig in the log file if needed. One of my customers has a project that builds more than 300 sub projects and Jenkins cannot handle the slave pushing that much detail if the build is logging debug level.

So being able to send the output to a file as well with control over which classes or tasks have which loglevels will help a lot.

@oehme
Copy link
Contributor

oehme commented Jan 17, 2017

@corneil I think adjusting Gradle's logging is out of scope for this story. It is about making Gradle more interactive when attached to a "smart" terminal. The use case you mentioned is definitely valid, but I think we should have a separate issue for that.

@eriwen
Copy link
Contributor Author

eriwen commented Jan 17, 2017

@corneil We hear you and lots of others loud and clear on the logging front. We have some unpublished design specs to help address this. Expect to read some more about it later this year.

* Additions needed to `org.gradle.api.Logger` and `StyledTextOutput` for use in build scripts

## Story: Allow user to include additional info in log messages
Allow users to specify whether timestamp, log level, and/or category are logged with each message.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

though I'm not sure this is strictly necessary.

100% a solution to the use case is necessary before we remove the task names. There's no way we should ask our users to detangle the error output of 8 different broken JavaCompile tasks where is all jumbled together and interleaved.

The proposed solution (Allow users to specify whether timestamp, log level, and/or category are logged with each message) doesn't really solve the problem.

Instead, I would do something like group the log output of a build operation (however that came to be) together and include either a header or a prefix. I would do this as a built-in capability and not something the task author or build author or build operator has to take action to enable.

# Milestone 1 - Show parallel WIP and summary

## Story: Display parallel work in-progress independently by operation
Show incomplete ProgressOperations on separate lines, up to a specified maximum number.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BuildOperations, not ProgressOperations.


`TaskExecutionLogger` no longer sets logging headers. This prevents logging of task names
in main area. To compensate for the user's loss of context, [WARN and ERROR logs can be
configured to give more context](#story-log-messages-can-be-configured-to-give-more-context),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not configured, this should just be baked in and require no action by the task author or build author or build operator.

I would do this by generalizing what the task exection logger does and apply it to all BuildOperations that generate output.

```

### Implementation
`DefaultGradleLauncherFactory` registers a listener `BuildProgressBarListener` (similar to `BuildProgressFilter`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would instead have the task graph executor generate an event attached to the build operation that represents task execution, to indicate how many tasks will be executed. I wouldn't couple DefaultGradleLauncherFactory in to this.

In other words, move to towards the situation where the build operation event stream describes everything relevant to the visualisation of build execution. The thing that schedules and executes a sequence of build operations can, once it knows, inform consumers that care how many build operations it will be running.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have similar constructs that could generate events for initialization and project configuration?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, they're all build operations already. That's the point: don't add another ad hoc mechanism, instead add a general mechanism and reuse it for configuration, test execution, native compilation, dependency resolution, etc. It's the same amount of effort to add the general mechanism as it is to add the ad hoc mechanism.

### Open Issues
* How is buildSrc rendered?
* How are composite builds rendered?
* How are GradleBuild tasks rendered?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For these, the answer is: same as everything else. Build operation events are generated for these builds.

```

### Implementation
`BuildProgressBarListener` implements a TestExecutionListener (TBD) and
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should instead replace TestListenerInternal with build operation events.

@adammurdoch
Copy link
Member

+1 to very low priority for customisation. I would do it through an API that plugins can mess with, so I can easily package my customisations (which will sometimes include logic) and share them with others.

- Add domain modeling information where possible
- Split WIP on progress bar to separate story
- Flesh out implementation details for some stories

Issue: gradle-private#510
- Provide more detail about parallel WIP is displayed and
implemented
- Add stories improve logging to give more context

Issue: #1063
- Introduce 4 deliverable milestones
- Removed stories that will almost certainly not be handled
- Add stories for some BuildOperation improvements

Issue: gradle-private#510
@eriwen eriwen merged commit e30f912 into master Jan 25, 2017
@eriwen eriwen deleted the ew-improved-console-spec branch January 25, 2017 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants