Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gitmanager refactor #1216

Closed
wants to merge 7 commits into from
Closed

Gitmanager refactor #1216

wants to merge 7 commits into from

Conversation

quartata
Copy link
Member

@quartata quartata commented Nov 9, 2017

This does two things to help improve Smokey's Git workflow:

  • Adds a pre-commit hook that checks to see if we're on deploy, and refuse to commit if so. This is to prevent people from accidentally committing to deploy and creating nasty local changes.
  • Simplify the blacklisting process and avoid checking out or modifying the local master branch (in case it's not up to date or has local changes):
    • Fetch origin/master from Github
    • Check it out into a detached HEAD
    • Make blacklist commit
    • If the user has code privileges, push directly to master from detached HEAD
    • Else, checkout a new temporary branch and make a PR.

Note that at no point do we commit directly to the local master or even check it out. This also eliminates any merging, which would run into issues if the changes couldn't be fast-forwarded.

@quartata
Copy link
Member Author

quartata commented Nov 9, 2017

This also eliminates the notorious "HEAD isn't at the tip of origin's master branch" error.

@tripleee
Copy link
Member

tripleee commented Nov 9, 2017

I'm here but I'm waiting for the revamp which avoids creating a branch for privileged users.

https://chat.stackexchange.com/transcript/message/41039077#41039077

ws.py Outdated
import os.path

if not os.path.islink(".git/hooks/pre-commit"):
os.symlink("hooks/pre-commit", ".git/hooks/pre-commit")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about copying hooks/* into .git/hooks/ on each launch? This keeps the scripts up-to-date without the issues surrounding symlinks.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally I thought that would be more inconvenient than a symlink. At any rate, I found a better way: you can set a custom hook directory from git config.

teward
teward previously requested changes Nov 9, 2017
Copy link
Member

@teward teward left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to review things. Get on it.

git.checkout("-b", branch)

# Clear HEAD just in case
git.reset("HEAD")
Copy link
Member

@teward teward Nov 9, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be kept somewhere. We need to make sure we're actually on HEAD and not in a detached or reverted state when we start working on data in the code at all. Which is incredibly important.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In your test case in CHQ room, you detailed why this is important - you clobber remote commits that may be just as important. Therein lies the core problem and why we left this here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -148,13 +130,13 @@ def add_to_blacklist(cls, **kwargs):
"-m", u"Auto {0} of {1} by {2} --autopull".format(op, item, username))

if code_permissions:
git.checkout("master")
git.merge(branch)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we're letting GH handle all of the commits then? We had this here because if you have code privs you should already push this but also locally merge it. As you refactor it, this will prevent in-place activation of changed things if one has code perms.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also was built to push to master directly, hence why we checked out master to begin with. your changes will fubar this process.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a valid point, however I'm inclined to prefer that we wait for CI.

@AWegnerGitHub
Copy link
Member

Pulling this out of an embedded comment, discussion about this in CHQ: https://chat.stackexchange.com/rooms/11540/conversation/pr-1216

Copy link
Member

@tripleee tripleee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I too participated in the Charcoal discussion here:

https://chat.stackexchange.com/rooms/11540/conversation/review-discussion-for-pr1216

In summary, the rough consensus is that we need to avoid losing commits to the local master, or be thoroughly convinced that there are no scenarios where such commits can contain useful work.

I like the general thinking here, and would love to see a more versatile and robust Git manager. But I think our first task needs to be refactoring the current code to something we can create a test suite for, and only then start experimenting with different concurrency models. I am quite sure we will uncover additional failure scenarios in the current code which we are not well aware of.

@quartata
Copy link
Member Author

quartata commented Nov 9, 2017

As we're currently discussing in chat, I think there might be a bit of a misunderstanding here. Nothing about this will clobber remote commits -- the only window of opportunity for a remote commit to cause a conflict is between the fetch and the push which is a few seconds at max, and if it did occur the push would simply fail (Git only tries to fast forward commits when pushing, it'll give up if it needs to merge them.)

As for local commits, they're all still there -- they just won't be pushed with the blacklist (by design). Presumably they were created by a human, so a human should resolve and push them.

@quartata
Copy link
Member Author

quartata commented Nov 10, 2017

I should add that the finally clause makes it so that it always switches back to deploy and deletes any temporary branch made no matter what, if say the push did fail. (I could probably add in a stash just in case deploy has uncommitted (human) changes, but this would only be to avoid an "you have uncommitted changes" error)

@quartata
Copy link
Member Author

quartata commented Nov 11, 2017

I've set up another repository with a clean Git tree with my modifications, and confirmed that blacklisting (at least calling gitmanager.GitManager.add_to_blacklist directly) works: https://github.com/quartata/smokey-git-test. Feel free to experiment.

@quartata
Copy link
Member Author

Pull requests also work, and no local deploy or master changes were harmed: quartata/smokey-git-test#1

@coveralls
Copy link

Coverage Status

Coverage remained the same at 65.087% when pulling e973b7a on quartata:gitmanager into 836dd08 on Charcoal-SE:master.

@teward teward dismissed their stale review April 12, 2018 17:58

Obsolete Review

@ArtOfCode- ArtOfCode- added the status: deferred 6-8 longer weeks. label Jul 8, 2018
@ArtOfCode- ArtOfCode- closed this Jul 8, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status: deferred 6-8 longer weeks.
Development

Successfully merging this pull request may close these issues.

None yet

8 participants