New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

custom compile progress via env var template #6070

Open
wants to merge 15 commits into
base: master
from

Conversation

Projects
None yet
6 participants
@matthiaskrgr
Contributor

matthiaskrgr commented Sep 21, 2018

Allows to customize the compile progress via env var.
This was inspired by NINJA_STATUS environmental variable:
https://ninja-build.org/manual.html#_environment_variables

I had to add some workarounds to make sure the CARGO_STATUS var does not mess with the git fetch/crate download progress indicators.
Right now this should only change the output of "cargo build/check"

Docs from the readme:

CARGO_STATUS — If this is set, cargo will read it as compile progress template.
The default format is [%b] %f/%t: %n which will look like this:
Building [=====> ] 115/145: libssh2-sys(build), thread_l...
The following parameters are supported:
* %b progress bar =======>
* %s number of started jobs
* %f number of finished jobs
* %t number of total jobs of the build
* %u number or remaining jobs to start
* %r number of active (currently running) jobs
* %p progress percentage with decimals
* %P progress percentage without decimals
* %% plain % character (can be used for %P%% => 14%)
* %n (truncated) list of names of running jobs
* %e elapsed time in seconds
* %E elapsed time in seconds (human readable)
* %o overall time per job
* %O overall time per job (human readable)
* %c jobs per second

matthiaskrgr added some commits Sep 13, 2018

compile-progress: add initial support for customizable compile progre…
…ss formatting

You can pass CARGO_STATUS env var to cargo to change the formatting.

%b is the progress bar
%s is the number of done jobs
%t is the number of total jobs
%p is progress percentage
%n is a list of names of running jobs

The default formatting is "[%b] %s/%t%n"

You can pass alternatives like this:
	CARGO_STATUS="%s of %t, running: %n, [%b]" cargo build
compile-progress: work around customizable compile-progress interferi…
…ng with download progress.

When downloading crates, use preset, non-costumizable compile-progress formats.
compile-progress: change %s to %f, add %s and %u
change %s (number of started jobs) to %f (number of finished jobs) since that's what it actually is
add new %s which represents the actual number of started jobs
add %u representing the number of remaining jobs to start
@rust-highfive

This comment has been minimized.

Show comment
Hide comment
@rust-highfive

rust-highfive Sep 21, 2018

r? @alexcrichton

(rust_highfive has picked a reviewer for you, use r? to override)

rust-highfive commented Sep 21, 2018

r? @alexcrichton

(rust_highfive has picked a reviewer for you, use r? to override)

Show outdated Hide outdated src/doc/src/reference/environment-variables.md Outdated
Show outdated Hide outdated src/cargo/util/progress.rs Outdated

@matthiaskrgr matthiaskrgr changed the title from [WIP] custom compile progress via env var template to custom compile progress via env var template Sep 21, 2018

compile-progress: make sure that %% is evaluated last
In a pattern like %%s, we would previously evaluate %% first and then have a bare s char
	%%s => %s
Evaluate %% last so that
	%%s => %2
@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Sep 24, 2018

Member

Thanks for the PR! In general though Cargo's output isn't customizable to this degree and we try to avoid mini-DSLs wherever possible in Cargo. I would personally expect a feature like this to likely require an RFC rather than being accepted through a PR. Others on @rust-lang/cargo may feel different though.

Member

alexcrichton commented Sep 24, 2018

Thanks for the PR! In general though Cargo's output isn't customizable to this degree and we try to avoid mini-DSLs wherever possible in Cargo. I would personally expect a feature like this to likely require an RFC rather than being accepted through a PR. Others on @rust-lang/cargo may feel different though.

@joshtriplett

This comment has been minimized.

Show comment
Hide comment
@joshtriplett

joshtriplett Sep 24, 2018

Member

@alexcrichton I agree. If there's information not being provided via the build log, we should provide it. And if there's a different format we need when stdout isn't a TTY, we should provide that too. But let's not make it customizable.

Member

joshtriplett commented Sep 24, 2018

@alexcrichton I agree. If there's information not being provided via the build log, we should provide it. And if there's a different format we need when stdout isn't a TTY, we should provide that too. But let's not make it customizable.

@matthiaskrgr

This comment has been minimized.

Show comment
Hide comment
@matthiaskrgr

matthiaskrgr Sep 26, 2018

Contributor

mini-DSLs

what does DSL stand for?

When using ninja, I find the number of currently running jobs and the current build time very useful to know, but I didn't want to change the default progress of cargo which is why I made it customisable.

Contributor

matthiaskrgr commented Sep 26, 2018

mini-DSLs

what does DSL stand for?

When using ninja, I find the number of currently running jobs and the current build time very useful to know, but I didn't want to change the default progress of cargo which is why I made it customisable.

@dwijnand

This comment has been minimized.

Show comment
Hide comment
@dwijnand

dwijnand Sep 26, 2018

Member

what does DSL stand for?

Domain-specific language. (https://en.wikipedia.org/wiki/Domain-specific_language)

Member

dwijnand commented Sep 26, 2018

what does DSL stand for?

Domain-specific language. (https://en.wikipedia.org/wiki/Domain-specific_language)

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Sep 26, 2018

Member

I think it's fine to tweak and update Cargo's default output, and it's likely beneficial to everyone to have a better default output rather than allowing one-off customization.

Member

alexcrichton commented Sep 26, 2018

I think it's fine to tweak and update Cargo's default output, and it's likely beneficial to everyone to have a better default output rather than allowing one-off customization.

compile progress: add time-realted options: %e %E %o %O %c
	%e elapsed time in seconds
	%E elapsed time (human readable)
	%o overall time per job
	%O overall time per job (human readable)
	%c jobs done per second
@joshtriplett

This comment has been minimized.

Show comment
Hide comment
@joshtriplett

joshtriplett Sep 26, 2018

Member

We discussed this in the @rust-lang/cargo meeting, and just as discussed here in this issue, this degree of output customization is not something we're looking to support. We'd prefer to make the output as usable as possible for various use cases, and if people have extremely specialized needs, they may need the full flexibility of build plans and custom build processes.

Given that:

@rfcbot fcp close

Member

joshtriplett commented Sep 26, 2018

We discussed this in the @rust-lang/cargo meeting, and just as discussed here in this issue, this degree of output customization is not something we're looking to support. We'd prefer to make the output as usable as possible for various use cases, and if people have extremely specialized needs, they may need the full flexibility of build plans and custom build processes.

Given that:

@rfcbot fcp close

@joshtriplett

This comment has been minimized.

Show comment
Hide comment
@joshtriplett

joshtriplett Sep 26, 2018

Member

(oops, accidentally hit "close and comment")

Member

joshtriplett commented Sep 26, 2018

(oops, accidentally hit "close and comment")

@joshtriplett

This comment has been minimized.

Show comment
Hide comment
@joshtriplett

joshtriplett Sep 26, 2018

Member

@rfcbot fcp close

Member

joshtriplett commented Sep 26, 2018

@rfcbot fcp close

@rfcbot

This comment has been minimized.

Show comment
Hide comment
@rfcbot

rfcbot Sep 26, 2018

Team member @joshtriplett has proposed to close this. The next step is review by the rest of the tagged teams:

No concerns currently listed.

Once a majority of reviewers approve (and none object), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

rfcbot commented Sep 26, 2018

Team member @joshtriplett has proposed to close this. The next step is review by the rest of the tagged teams:

No concerns currently listed.

Once a majority of reviewers approve (and none object), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

@matthiaskrgr

This comment has been minimized.

Show comment
Hide comment
@matthiaskrgr

matthiaskrgr Sep 26, 2018

Contributor

Well ok.

In the latest commit I added a bunch of time-related formatters.

So far the NINJA_STATUS template that has served me well is CARGO_STATUS="[%P%%/%f/%r/%t/%e]".
This would look like this: Building [ 12%/50/4/402/22] ([percentage/done/active/total/seconds].
The current cargo default is [%b] %f/%t: %n ([progress bar] done/total: job_names)

If we are going to display additional information (without customizale formatting), I would vote for something like
"%P%% [%b] %f/%r/%t (%E): %n" as new default:
Building 29% [==========> ] 17/4/58 (14.19s): termcolor, ansi_term, natord, glob
(github is truncating away some whitespaces unfortunately..)
It shows the progress percentage, bar, and afterwards the number of finished jobs, number of currently running/active jobs (this is easier to see than counting the job names (which might get truncated anyway)) and how long the build has been running so far in human readable format, completed by the names of current jobs.

I would like to know what others would want to see or if they are happy with the current compile progress layout. :)

Contributor

matthiaskrgr commented Sep 26, 2018

Well ok.

In the latest commit I added a bunch of time-related formatters.

So far the NINJA_STATUS template that has served me well is CARGO_STATUS="[%P%%/%f/%r/%t/%e]".
This would look like this: Building [ 12%/50/4/402/22] ([percentage/done/active/total/seconds].
The current cargo default is [%b] %f/%t: %n ([progress bar] done/total: job_names)

If we are going to display additional information (without customizale formatting), I would vote for something like
"%P%% [%b] %f/%r/%t (%E): %n" as new default:
Building 29% [==========> ] 17/4/58 (14.19s): termcolor, ansi_term, natord, glob
(github is truncating away some whitespaces unfortunately..)
It shows the progress percentage, bar, and afterwards the number of finished jobs, number of currently running/active jobs (this is easier to see than counting the job names (which might get truncated anyway)) and how long the build has been running so far in human readable format, completed by the names of current jobs.

I would like to know what others would want to see or if they are happy with the current compile progress layout. :)

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Sep 27, 2018

Member

That looks reasonable to me, but the 17/4/58 I feel is a bit confusing in that it's sort of just three numbers where it's not clear what to do with it. The 29% I feel may also be redundant with the number of crates to build and total crates?

Member

alexcrichton commented Sep 27, 2018

That looks reasonable to me, but the 17/4/58 I feel is a bit confusing in that it's sort of just three numbers where it's not clear what to do with it. The 29% I feel may also be redundant with the number of crates to build and total crates?

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Oct 12, 2018

Member

@matthiaskrgr have you had a chance to game this out and see what it would look like to change Cargo's default progress bar?

Member

alexcrichton commented Oct 12, 2018

@matthiaskrgr have you had a chance to game this out and see what it would look like to change Cargo's default progress bar?

@matthiaskrgr

This comment has been minimized.

Show comment
Hide comment
@matthiaskrgr

matthiaskrgr Oct 14, 2018

Contributor

I'll hopefully get to this early in the upcomming week.

Contributor

matthiaskrgr commented Oct 14, 2018

I'll hopefully get to this early in the upcomming week.

@matthiaskrgr

This comment has been minimized.

Show comment
Hide comment
@matthiaskrgr

matthiaskrgr Oct 17, 2018

Contributor

That looks reasonable to me, but the 17/4/58 I feel is a bit confusing in that it's sort of just three numbers where it's not clear what to do with it. The 29% I feel may also be redundant with the number of crates to build and total crates?

So you would opt for just the current progress bar plus the build time?
( CARGO_STATUS="[%b] %f/%t (%E): %n")

have you had a chance to game this out and see what it would look like to change Cargo's default progress bar?

Hm, I'm not sure if I understand "what it would look like"...
On the branch we can try everything out and once we decided for a new(?) format, I can remove all the unnecessary (:() code that made it customizable and hardcode/update to the new format.

Contributor

matthiaskrgr commented Oct 17, 2018

That looks reasonable to me, but the 17/4/58 I feel is a bit confusing in that it's sort of just three numbers where it's not clear what to do with it. The 29% I feel may also be redundant with the number of crates to build and total crates?

So you would opt for just the current progress bar plus the build time?
( CARGO_STATUS="[%b] %f/%t (%E): %n")

have you had a chance to game this out and see what it would look like to change Cargo's default progress bar?

Hm, I'm not sure if I understand "what it would look like"...
On the branch we can try everything out and once we decided for a new(?) format, I can remove all the unnecessary (:() code that made it customizable and hardcode/update to the new format.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment