Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Properly allow arguments id or path_with_namespace for project data source #806

Merged

Conversation

timofurrer
Copy link
Member

@timofurrer timofurrer commented Jan 27, 2022

The problem with the gitlab_project data source was that if the id is a path with namespace and the reading of
the data source is delayed until apply, e.g. seeing something like
this:

  # module.main.data.gitlab_project.some_project will be read during apply
  # (config refers to values not yet known)
 <= data "gitlab_project" "some_project"  {
      + archived                         = (known after apply)
      + default_branch                   = (known after apply)
      + description                      = (known after apply)
      + http_url_to_repo                 = (known after apply)
      + id                               = "timo.furrer/some-project"
      + issues_enabled                   = (known after apply)
      + lfs_enabled                      = (known after apply)
      + merge_requests_enabled           = (known after apply)
      + name                             = (known after apply)
      + namespace_id                     = (known after apply)
      + path                             = (known after apply)
      + path_with_namespace              = (known after apply)
      + pipelines_enabled                = (known after apply)
      + push_rules                       = (known after apply)
      + remove_source_branch_after_merge = (known after apply)
      + request_access_enabled           = (known after apply)
      + runners_token                    = (known after apply)
      + snippets_enabled                 = (known after apply)
      + ssh_url_to_repo                  = (known after apply)
      + visibility_level                 = (known after apply)
      + web_url                          = (known after apply)
      + wiki_enabled                     = (known after apply)

The id is the path_with_namespace, which ends up like that in the
state (as the string path). However, the data source in fact stores
the actually project id (which it gets from the API) in the state durig
apply
(see

d.SetId(fmt.Sprintf("%d", found.ID))
)
which leads to an inconsistent final plan in dependent resources as
shown here:

╷
│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for
│ module.main.gitlab_repository_file.some_file
│ to include new values learned so far during apply, provider
│ "registry.terraform.io/timofurrer/gitlab-repository-files" produced an
│ invalid new value for .project: was
│ cty.StringVal("timo.furrer/some-project"), but now
│ cty.StringVal("30716").
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵

The root cause for this is that the data source always stores the
actual project id in the state no matter if a path_with_namespace was
actually given. The id attribute is NOT marked as being Computed,
but it sometimes is (when the path_with_namespace is given).

We cannot mark the id attribute as Required and Computed, because
this is not allowed by terraform (makes sense :D). Therefore, I've used
the same pattern as in other data sources (althought I'd argue that the
handling is not quite consistent here across them :( ).

The implementation here should be (please verify) backwards compatible.
However, people can still run into the same issue as before when
specifying the path_with_namespace in the id argument, but can
reasolve it by switching to the path_with_namespace argument ;)

For long term, and when we have a major release I suggest to break the
interface in these data sources to make it more consistent.

@timofurrer timofurrer self-assigned this Jan 27, 2022
@timofurrer timofurrer added the bug label Jan 27, 2022
@timofurrer timofurrer added this to the 3.9.0 milestone Jan 27, 2022
Copy link
Collaborator

@armsnyder armsnyder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice writeup. Relates to #553 that we are very inconsistent about project vs id vs project_id vs full_path all being used to mean a project name/id. 😭 Some of this we may be able to fix with state migrations. The rest will have to be a v4 initiative. Anyway this looks backward compatible to me 👍

gitlab/data_source_gitlab_project.go Outdated Show resolved Hide resolved
…ta source

The problem with the `gitlab_project` data source was that if
the id is a path with namespace and the reading of the
data source is delayed until apply, e.g. seeing something like
this:

```
  # module.main.data.gitlab_project.some_project will be read during apply
  # (config refers to values not yet known)
 <= data "gitlab_project" "some_project"  {
      + archived                         = (known after apply)
      + default_branch                   = (known after apply)
      + description                      = (known after apply)
      + http_url_to_repo                 = (known after apply)
      + id                               = "timo.furrer/some-project"
      + issues_enabled                   = (known after apply)
      + lfs_enabled                      = (known after apply)
      + merge_requests_enabled           = (known after apply)
      + name                             = (known after apply)
      + namespace_id                     = (known after apply)
      + path                             = (known after apply)
      + path_with_namespace              = (known after apply)
      + pipelines_enabled                = (known after apply)
      + push_rules                       = (known after apply)
      + remove_source_branch_after_merge = (known after apply)
      + request_access_enabled           = (known after apply)
      + runners_token                    = (known after apply)
      + snippets_enabled                 = (known after apply)
      + ssh_url_to_repo                  = (known after apply)
      + visibility_level                 = (known after apply)
      + web_url                          = (known after apply)
      + wiki_enabled                     = (known after apply)
```

The `id` is the `path_with_namespace`, which ends up like that in the
state (as the string path). However, the data source in fact stores
the actually project id (which it gets from the API) in the state durig
apply
(see https://github.com/gitlabhq/terraform-provider-gitlab/blob/8bc5606d786d73cb9e1947d0f9bc941c2e4f61b0/gitlab/data_source_gitlab_project.go#L173)
which leads to an inconsistent final plan in dependent resources as
shown here:

```
╷
│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for
│ module.main.gitlab_repository_file.some_file
│ to include new values learned so far during apply, provider
│ "registry.terraform.io/timofurrer/gitlab-repository-files" produced an
│ invalid new value for .project: was
│ cty.StringVal("timo.furrer/some-project"), but now
│ cty.StringVal("30716").
│
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
```

The root cause for this is that the data source always stores the
actual project id in the state no matter if a `path_with_namespace` was
actually given. The `id` attribute is NOT marked as being `Computed`,
but it sometimes is (when the `path_with_namespace` is given).

We cannot mark the `id` attribute as `Required` and `Computed`, because
this is not allowed by terraform (makes sense :D). Therefore, I've used
the same pattern as in other data sources (althought I'd argue that the
handling is not quite consistent here across them :( ).

The implementation here should be (please verify) backwards compatible.
However, people can still run into the same issue as before when
specifying the `path_with_namespace` in the `id` argument, but can
reasolve it by switching to the `path_with_namespace` argument ;)

For long term, and when we have a major release I suggest to break the
interface in these data sources to make it more consistent.
Copy link
Collaborator

@armsnyder armsnyder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@armsnyder armsnyder merged commit fc84cc3 into gitlabhq:master Jan 27, 2022
@github-actions
Copy link

github-actions bot commented Feb 4, 2022

This functionality has been released in 3.9.0 of the Terraform GitLab Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue. Thank you!

@github-actions github-actions bot locked and limited conversation to collaborators Nov 9, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
2 participants