Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need instructions for migrating from hashicorp/github to integrations/github #652

Closed
docwhat opened this issue Jan 8, 2021 · 19 comments
Closed
Labels
Authentication Provider Status: Stale Used by stalebot to clean house Type: Breaking change Used to note any change that requires a major version bump Type: Documentation Improvements or additions to documentation

Comments

@docwhat
Copy link
Contributor

docwhat commented Jan 8, 2021

Now that GitHub has taken over ownership of the github provider we need some instructions for upgrading.

I tried the obvious:

  1. update required_providers:
    1. change source attribute to integrations/github.
    2. change version to 4.2.0 (from 4.1.0 in my case)
  2. terraform state replace-provider registry.terraform.io/hashicorp/github registry.terraform.io/integrations/github
  3. terraform init
    • This shows it installing both hashicorp 4.1.0 and integrations 4.2.0
  4. terraform plan

Unfortunately, it then seems to hang forever.

TRACE shows that it is trying to access github.com (it should be using our enterprise server) without specifying the org using the hashicorp 4.1.0 provider:

[DEBUG] plugin.terraform-provider-github_v4.1.0_x4: GET /orgs//teams/sre-team HTTP/1.1
[DEBUG] plugin.terraform-provider-github_v4.1.0_x4: Host: api.github.com

There is exactly one GET using the 4.2 provider:

[DEBUG] plugin.terraform-provider-github_v4.2.0: GET /api/v3/orgs/the-org-on-our-ghe HTTP/1.1
[DEBUG] plugin.terraform-provider-github_v4.2.0: Host: github.example.com

No matter what I do, terraform insists on using the official hashicorp provider.

$ terraform -version
Terraform v0.14.4
+ provider registry.terraform.io/hashicorp/github v4.1.0
+ provider registry.terraform.io/integrations/github v4.2.0
@frbayart
Copy link
Contributor

frbayart commented Jan 8, 2021

On my side I make:

  1. edit all .TF files (plan and modules too)
  2. terraform state replace-provider kensu.io/kensu/github integrations/github
  3. terraform init
  4. terraform plan

And it works; at the beginning I didn't update my modules so I got some confusion....
(kensu.io/kensu/github is fork of hashicorp/github with last features suggested on the repo)

@jcudit
Copy link
Contributor

jcudit commented Jan 9, 2021

🤔 maybe we document this as a breaking change?
I can get the CHANGELOG updated if it helps boost visibility of the state replace-provider procedure above.

@frbayart
Copy link
Contributor

frbayart commented Jan 9, 2021

Also @docwhat if the problem persists, it's probably useful to check terraform providers

You have details per plan and modules, output is like:

Providers required by configuration:
.
├── provider[registry.terraform.io/integrations/github] 4.2.0
├── provider[registry.terraform.io/hashicorp/vault] ~> 2.11
├── module.dam-mock-module-webui
│   └── provider[registry.terraform.io/integrations/github] 4.2.0
|── module...
 
...

Providers required by state:

    provider[registry.terraform.io/hashicorp/vault]

    provider[registry.terraform.io/integrations/github]

@jcudit jcudit added Type: Breaking change Used to note any change that requires a major version bump Type: Documentation Improvements or additions to documentation Provider labels Jan 14, 2021
@aliculPix4D
Copy link

aliculPix4D commented Jan 20, 2021

@docwhat I am also seeing the same issue as you do and it seems that, at least for me, it was due to the bug described here: #655 (comment)
where both values for organization and token in provider block were ignored.

provider "github" {
  organization = "..."
  token        = var.github_token
}

by setting the GITHUB_TOKEN ,GITHUB_ORGANIZATION environmental variables, the upgrade was successful and only integrations/github (4.3.0) can be used.

@jcudit
Copy link
Contributor

jcudit commented Feb 3, 2021

terraform {
  required_providers {
    github = {
      source = "integrations/github"
    }
  }
}

Adding the required_providers configuration block may unblock others experiencing this.

@MRostanski
Copy link

@jcudit no, unfortunately it does not. I have:

terraform {
  required_providers {
    github = {
      source  = "integrations/github"
      version = "~> 4.4.0"
    }
    random = {
      source  = "hashicorp/random"
      version = "~> 3.1.0"
    }
    tls = {
      source  = "hashicorp/tls"
      version = "~> 3.1.0"
    }
  }
}

and still both get installed:

Initializing provider plugins...

  • Finding hashicorp/random versions matching "~> 3.1.0"...
  • Finding hashicorp/tls versions matching "~> 3.1.0"...
  • Finding latest version of hashicorp/github...
  • Finding integrations/github versions matching "~> 4.4.0"...
  • Installing hashicorp/random v3.1.0...
  • Installed hashicorp/random v3.1.0 (signed by HashiCorp)
  • Installing hashicorp/tls v3.1.0...
  • Installed hashicorp/tls v3.1.0 (signed by HashiCorp)
  • Installing hashicorp/github v4.4.0...
  • Installed hashicorp/github v4.4.0 (signed by HashiCorp)
  • Installing integrations/github v4.4.0...
  • Installed integrations/github v4.4.0 (signed by a HashiCorp partner, key ID 38027F80D7FD5FB2)

Partner and community providers are signed by their developers.
If you'd like to know more about provider signing, you can read about it here:
https://www.terraform.io/docs/cli/plugins/signing.html

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Warning: Additional provider information from registry

The remote registry returned warnings for
registry.terraform.io/hashicorp/github:

  • For users on Terraform 0.13 or greater, this provider has moved to
    integrations/github. Please update your source in required_providers.

What is going on in terraform providers:

Providers required by configuration:
.
├── provider[registry.terraform.io/integrations/github] ~> 4.2.0
├── provider[registry.terraform.io/hashicorp/random] ~> 3.1.0
├── provider[registry.terraform.io/hashicorp/tls] ~> 3.1.0
├── module.github_repository
│   └── provider[registry.terraform.io/hashicorp/github]
├── module.naming_convention
│   └── provider[registry.terraform.io/hashicorp/random]
└── module.ssh_key
    └── provider[registry.terraform.io/hashicorp/tls]

@rmc47
Copy link

rmc47 commented Feb 25, 2021

@MRostanski it looks like you're hitting the same thing we just did: if you use modules, each module needs to declare its own required_providers block with the source of the github module pointing to integrations/github instead of hashicorp/github.

Without that, you end up with the mix of the two providers...

@ops-hummus
Copy link

The workaround is to put this snippet with the main.tf and also inside the github module directory.
terraform { required_providers { github = { source = "integrations/github" version = "4.13.0" } } }

lets assume we have main.tf and subdirectory containing github.tf. The main file uses a module that uses the github resource. Now the problem begins. Terraform download both integrations/github and hashicorp/github, but if you put the required_providers in both places, with the main and within the subdirectory, it stops the problem.
Hope I was clear.

@aliculPix4D
Copy link

aliculPix4D commented Aug 23, 2021

The workaround is to put this snippet with the main.tf and also inside the github module directory.

Another simplification that worked for me is to move the following section from main.tf to i.e tf-modules/github/main.tf

provider "github" {
  token = var.github_token
  owner = "somename"
}

and just send the token to the module as a variable:

module "github" {
  source = "../tf-modules/github"

  github_token  = var.github_token
}

this results in terraform providers command to output only
github
:

This should prevent terraform to download hashicorp/github provider.
Note that you also need to replace the provider in remote state file using terraform state replace-provider

@TheJackson
Copy link

@rmc47: @MRostanski it looks like you're hitting the same thing we just did: if you use modules, each module needs to declare its own required_providers block with the source of the github module pointing to integrations/github instead of hashicorp/github. Without that, you end up with the mix of the two providers...

This is the one, thanks for this workaround!

@larstobi
Copy link

larstobi commented Aug 2, 2022

I have already run terraform state replace-provider -- registry.terraform.io/hashicorp/github registry.terraform.io/integrations/github so there are no references to hashicorp/github in the state.

When using github resource inside modules, then this becomes quite confusing. As an experiment, I put nothing github under required_providers in the top level, nor inside the modules. When I then run terraform init, it wil pick a mix of hashicorp/github and integrations/github, but both with the same version (v4.28.0). In addition it prints a message from the Registry:

Warning: Additional provider information from registry

The remote registry returned warnings for registry.terraform.io/hashicorp/github:

  • For users on Terraform 0.13 or greater, this provider has moved to integrations/github. Please update your source in required_providers.

Now, if I put integrations/github in required_providers in the top level terraform only, and nothing in the modules, then it still downloads a mix of the two provider names, and prints the above warning from the registry.

It initialized successfully, but the provider config in the top level isn't inherited to the module, so it can't work.

Then, if I integrations/github and provider config only into the modules and not in the top level, then it only downloads integrations/github and there is no mention of hashicorp/github. It initialized successfully and the provider config is correctly set.

So, I guess this problem is caused by both hashicorp/github and integrations/github using the same name github, and that if I specify no provider, it will by default pick hashicorp/github and in addition integrations/github if it's referenced in the state file. If it would always go for integrations/github instead, then maybe the whole problem goes away?

@ahmadnassri
Copy link

I can confirm @larstobi findings on my end.

@usmonster
Copy link
Contributor

So, I guess this problem is caused by both hashicorp/github and integrations/github using the same name github, and that if I specify no provider, it will by default pick hashicorp/github and in addition integrations/github if it's referenced in the state file.

Can this be considered a bug, then, and not just about documentation?

@kfcampbell
Copy link
Member

I'm not sure how I would go about troubleshooting that at the provider level. I wonder if that's something specific to the Terraform platform or tooling.

ChrisBAshton added a commit to alphagov/govuk-aws that referenced this issue Jan 18, 2023
Plan was giving warning:

```
The remote registry returned warnings for registry.terraform.io/-/fastly:
- For users on Terraform 0.13 or greater, this provider has moved to
fastly/fastly. Please update your source in required_providers.
```

Need to define `required_providers` in multiple places, as per
integrations/terraform-provider-github#652 (comment)
damianx1337 pushed a commit to damianx1337/k8s that referenced this issue Feb 23, 2023
@jackyou84
Copy link

I meet same issue lately.
I try to following steps:

a. change providers from module

required_providers {
    github = {
      source  = "integrations/github"
      version = "~> 5.17"
    }
  }

in local just refer providers:

provider "github" {
  owner = var.github_organization
  token = var.github_token
}

b. rm -rf .terraform .terraform.lock.hcl
c. terraform init
it still install hashicorp/github by default.
try to relpace state
terraform state replace-provider registry.terraform.io/hashicorp/github registry.terraform.io/integrations/github
and run step b and c. but getting same result.

finally, I add following block in local tf. and run step b and c.

terraform {
  required_providers {
    github = {
      source = "integrations/github"
    }
  }
}

and only integrations/github is used. and tf plan also smoothly.

@jackyou84
Copy link

so seems to switch providers from old one. we must update both module and local tf provider.

@mado-m
Copy link

mado-m commented Sep 15, 2023

I have also solved this problem by using replace-provider.

terraform state replace-provider registry.terraform.io/hashicorp/github registry.terraform.io/integrations/github

Copy link

👋 Hey Friends, this issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Please add the Status: Pinned label if you feel that this issue needs to remain open/active. Thank you for your contributions and help in keeping things tidy!

@github-actions github-actions bot added the Status: Stale Used by stalebot to clean house label Jun 12, 2024
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Jun 19, 2024
@larstobi
Copy link

I have also solved this problem by using replace-provider.

terraform state replace-provider registry.terraform.io/hashicorp/github registry.terraform.io/integrations/github

This doesn't in fact solve the issue completely. See my above findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Authentication Provider Status: Stale Used by stalebot to clean house Type: Breaking change Used to note any change that requires a major version bump Type: Documentation Improvements or additions to documentation
Projects
None yet
Development

No branches or pull requests