Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use git for detecting a dirty worktree. #1319

Merged
merged 5 commits into from
May 4, 2018
Merged

Use git for detecting a dirty worktree. #1319

merged 5 commits into from
May 4, 2018

Conversation

pgavlin
Copy link
Member

@pgavlin pgavlin commented May 4, 2018

The go-git implementation of git status is outrageously expensive,
as it performs a hash-based comparision of the working tree against the
committed state with no caching. In some example runs, this takes
upwards of 15 seconds. Because this is on the startup path for updates,
this results in a rather poor user experience.

These changes replace the go-git implementation with a call to git status --porcelain -z, which only writes data to stdout if the working
tree is dirty.

The `go-git` implementation of `git status` is outrageously expensive,
as it performs a hash-based comparision of the working tree against the
committed state with no caching. In some example runs, this takes
upwards of 15 seconds. Because this is on the startup path for updates,
this results in a rather poor user experience.

These changes replace the `go-git` implementation with a call to `git
status --porcelain -z`, which only writes data to stdout if the working
tree is dirty.
*w = true
}
return len(d), nil
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fun. Would not have thought of this approach :-).

cmd/util.go Outdated
if err != nil {
return m, errors.Wrapf(err, gitErrCtx, "getting worktree status")
// nolint: gas
gitStatusCmd := exec.Command(gitBin, "status", "--porcelain", "-z")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My one worry is that it is possible there will be non-zero environments where pulumi may be used where git is not available.

Do we have to hard fail on git not being installed? Is there a reasonable default value (and perhaps a warning?)

/cc @chrsmith

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could fall back to the go-git implementation. We could also simply not set the property in order to indicate a lack of information.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also simply not set the property in order to indicate a lack of information.

FWIW, I prefer this approach for all update metadata.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We try hard not to require any data about a stack in the service. So simply not including the data won't cause any problems.

That having been said, the more data we can push into the service, the more linking and cross referencing we can do. So as a general principle we should try to have a fallback where appropriate.

Though it seems pretty reasonable to not include git data if git isn't on your PATH.

cmd/util.go Outdated
if ee, ok := err.(*exec.ExitError); ok {
ee.Stderr = stderr.Bytes()
}
return m, errors.Wrapf(err, gitErrCtx, fmt.Sprintf("invoking git status"))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is fmt.Sprintf needed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope. I’ll remove that.

@lukehoban
Copy link
Member

BTW - does this effectively fix #1150 and/or #1206 ?

@pgavlin
Copy link
Member Author

pgavlin commented May 4, 2018

BTW - does this effectively fix #1150 and/or #1206 ?

I don't think that it fixes #1206, but it certainly helps. I suspect that fixing #1206 will also require us to be smarter about how we calculate hashes for asset archives.

@pgavlin
Copy link
Member Author

pgavlin commented May 4, 2018

(it will certainly help substantially with #1150, and may even fix that issue)

cmd/util.go Outdated
@@ -355,6 +357,15 @@ func (cf *colorFlag) Colorization() colors.Colorization {
return cf.value
}

type anyWriter bool
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: A comment here would be nice, as non-level-9 Go masters may not recognize what you are up to.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do.

@pgavlin
Copy link
Member Author

pgavlin commented May 4, 2018

@chrsmith @ellismg @joeduffy as part of these changes I've changed things s.t. each piece of git-related metadata is only filled out if available. PTAL

cmd/util.go Outdated
// Commit at HEAD
head, err := repo.Head()
if err != nil {
glog.Warningf("getting HEAD commit")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Include err in the Warningf statement. Here and on line 431.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, good catch! Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants