Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: clone repository before getting working dir #3867

Merged
merged 11 commits into from Dec 12, 2023

Conversation

ghaiszaher
Copy link
Contributor

@ghaiszaher ghaiszaher commented Oct 16, 2023

what

  • Fixes Planning a project fails when using new disable autoplan label #3855
  • As described in the issue, when running atlantis plan on specific projects, the repository is not cloned first
  • This is reproducible when the first command you run in a PR is atlantis plan -d <dir> or atlantis plan -p <project> in either one of the following situations:
    • ATLANTIS_DISABLE_AUTOPLAN = true is used
    • ATLANTIS_DISABLE_AUTOPLAN_LABEL = "no-autoplan" is used and the pr is labeled with no-autoplan
    • A draft PR
    • A PR that was opened while atlantis/webhook is not working
  • Solution: clone the PR earlier

why

  • It is necessary to make sure the plan command clones the repo at all times and shouldn't expect it to be already cloned (e.g. if the local directory had to be cleared for some reason)

tests

I have done the tests by running the service locally and using a webhook in a private GitHub project:

  • sudo rm -r .atlantis
  • Restart atlantis
  • Comment atlantis plan -d my-dir
Before After
image image

references

@ghaiszaher ghaiszaher marked this pull request as ready for review October 16, 2023 19:49
@ghaiszaher ghaiszaher requested a review from a team as a code owner October 16, 2023 19:49
@github-actions github-actions bot added the go Pull requests that update Go code label Oct 16, 2023
@nicolst
Copy link

nicolst commented Oct 17, 2023

I don't know all the inner workings of Atlantis very well, but the comment above the lock for buildAllCommandsByCfg suggests that the workspace should be locked before cloning?

// Need to lock the workspace we're about to clone to.
workspace := DefaultWorkspace
unlockFn, err := p.WorkingDirLocker.TryLock(ctx.Pull.BaseRepo.FullName, ctx.Pull.Num, workspace, DefaultRepoRelDir)
if err != nil {

Is there any downside to moving the TryLock up as well?

@ghaiszaher
Copy link
Contributor Author

I don't know all the inner workings of Atlantis very well, but the comment above the lock for buildAllCommandsByCfg suggests that the workspace should be locked before cloning?

// Need to lock the workspace we're about to clone to.
workspace := DefaultWorkspace
unlockFn, err := p.WorkingDirLocker.TryLock(ctx.Pull.BaseRepo.FullName, ctx.Pull.Num, workspace, DefaultRepoRelDir)
if err != nil {

Is there any downside to moving the TryLock up as well?

I missed this, thank you. I don't see any downsides: ab78b2b. I'll test it again and report if I see any issues.

@ghaiszaher
Copy link
Contributor Author

I'll test it again and report if I see any issues.

No issues

@nicolst
Copy link

nicolst commented Oct 17, 2023

Tested creating a PR with a no-autoplan label now, seems to work! Thanks for the quick fix!

@nitrocode
Copy link
Member

Nice work fixing this. Thank you for contributing the pr.

Would it be possible to add a golang test for this as well to ensure this feature doesn't break in the future?

@jamengual jamengual added the waiting-on-review Waiting for a review from a maintainer label Oct 20, 2023
defer unlockFn()

ctx.Log.Debug("cloning repository")
_, _, err = p.WorkingDir.Clone(ctx.HeadRepo, ctx.Pull, DefaultWorkspace)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's necessary to clone DefaultWorkspace at this point and not workspace

Comment on lines +596 to +601
if DefaultWorkspace != workspace {
ctx.Log.Debug("cloning repository with workspace %s", workspace)
_, _, err = p.WorkingDir.Clone(ctx.HeadRepo, ctx.Pull, workspace)
if err != nil {
return pcc, err
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This whole block is unnecessary because this call happens later anyway, but I kept it just in case it's needed for cases that I didn't notice

@ghaiszaher
Copy link
Contributor Author

Nice work fixing this. Thank you for contributing the pr.

Would it be possible to add a golang test for this as well to ensure this feature doesn't break in the future?

@nitrocode I adjusted the e2e tests here: 9f4d3e1
I added 2 test cases with disabled auto-plan and with no pre-workflow webhooks (because pre-workflow webhooks are already cloning the default workspace).

Please also check my comments in server/events/project_command_builder.go

Copy link
Member

@nitrocode nitrocode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ghaiszaher . Approved.

Ill defer to a follow up review before merging.

@jamengual jamengual added this to the v0.26.0 milestone Nov 15, 2023
@GenPage GenPage modified the milestones: v0.26.0, v0.27.0 Dec 11, 2023
@GenPage GenPage merged commit 379efd2 into runatlantis:main Dec 12, 2023
24 checks passed
ijames-gc pushed a commit to gocardless/atlantis that referenced this pull request Feb 13, 2024
* fix: clone repository before getting working dir

* unrelated change

* move TryLock up

* fix workspace and add e2e tests

* fix description

---------

Co-authored-by: PePe Amengual <jose.amengual@gmail.com>
ijames-gc pushed a commit to gocardless/atlantis that referenced this pull request Feb 13, 2024
* fix: clone repository before getting working dir

* unrelated change

* move TryLock up

* fix workspace and add e2e tests

* fix description

---------

Co-authored-by: PePe Amengual <jose.amengual@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
go Pull requests that update Go code waiting-on-review Waiting for a review from a maintainer
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Planning a project fails when using new disable autoplan label
5 participants