Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maintain exclusion list for provider versions with no required assets #54

Open
kislerdm opened this issue Nov 28, 2023 · 1 comment
Open

Comments

@kislerdm
Copy link
Contributor

kislerdm commented Nov 28, 2023

Problem

Historical releases of some providers don't include required assets, but we keep attempting to bump their versions. It causes wasted throughput.

For example:

{"time":"2023-11-28T21:22:51.131338327Z","level":"WARN","msg":"asset not found","type":"provider","provider":{"namespace":"jaczkal","name":"linux"},"github":{"url":"https://github.com/jaczkal/terraform-provider-linux/releases/download/0.7.1/terraform-provider-linux_0.7.1_SHA256SUMS"}}
{"time":"2023-11-28T21:22:51.131378842Z","level":"WARN","msg":"checksums not found in release, skipping...","type":"provider","provider":{"namespace":"jaczkal","name":"linux"},"release":"0.7.1"}
  • 3lvia/confluent-schema-registry is missing required assets for several tags
{"time":"2023-11-28T21:21:44.573684971Z","level":"WARN","msg":"asset not found","type":"provider","provider":{"namespace":"3lvia","name":"confluent-schema-registry"},"github":{"url":"https://github.com/3lvia/terraform-provider-confluent-schema-registry/releases/download/v0.4.0/terraform-provider-confluent-schema-registry_0.4.0_SHA256SUMS"}}
{"time":"2023-11-28T21:21:44.573770479Z","level":"WARN","msg":"checksums not found in release, skipping...","type":"provider","provider":{"namespace":"3lvia","name":"confluent-schema-registry"},"release":"v0.4.0"}
{"time":"2023-11-28T21:21:44.596027617Z","level":"WARN","msg":"asset not found","type":"provider","provider":{"namespace":"3lvia","name":"confluent-schema-registry"},"github":{"url":"https://github.com/3lvia/terraform-provider-confluent-schema-registry/releases/download/v0.5.0/terraform-provider-confluent-schema-registry_0.5.0_SHA256SUMS"}}
{"time":"2023-11-28T21:21:44.596083Z","level":"WARN","msg":"checksums not found in release, skipping...","type":"provider","provider":{"namespace":"3lvia","name":"confluent-schema-registry"},"release":"v0.5.0"}
{"time":"2023-11-28T21:21:44.6579[22](https://github.com/opentofu/registry-stable/actions/runs/7024681594/job/19113935619#step:4:23)635Z","level":"WARN","msg":"asset not found","type":"provider","provider":{"namespace":"3lvia","name":"confluent-schema-registry"},"github":{"url":"https://github.com/3lvia/terraform-provider-confluent-schema-registry/releases/download/v0.6.1/terraform-provider-confluent-schema-registry_0.6.1_SHA256SUMS"}}
{"time":"20[23](https://github.com/opentofu/registry-stable/actions/runs/7024681594/job/19113935619#step:4:24)-11-28T21:21:44.658007562Z","level":"WARN","msg":"checksums not found in release, skipping...","type":"provider","provider":{"namespace":"3lvia","name":"confluent-schema-registry"},"release":"v0.6.1"}
{"time":"2023-11-28T21:21:44.698064282Z","level":"WARN","msg":"asset not found","type":"provider","provider":{"namespace":"3lvia","name":"confluent-schema-registry"},"github":{"url":"https://github.com/3lvia/terraform-provider-confluent-schema-registry/releases/download/v0.2.0/terraform-provider-confluent-schema-registry_0.2.0_SHA256SUMS"}}
{"time":"2023-11-28T21:21:44.698111851Z","level":"WARN","msg":"checksums not found in release, skipping...","type":"provider","provider":{"namespace":"3lvia","name":"confluent-schema-registry"},"release":"v0.2.0"}
{"time":"2023-11-28T21:21:44.712340356Z","level":"WARN","msg":"asset not found","type":"provider","provider":{"namespace":"3lvia","name":"confluent-schema-registry"},"github":{"url":"https://github.com/3lvia/terraform-provider-confluent-schema-registry/releases/download/v0.6.0/terraform-provider-confluent-schema-registry_0.6.0_SHA256SUMS"}}
{"time":"2023-11-28T21:21:44.71[24](https://github.com/opentofu/registry-stable/actions/runs/7024681594/job/19113935619#step:4:25)12861Z","level":"WARN","msg":"checksums not found in release, skipping...","type":"provider","provider":{"namespace":"3lvia","name":"confluent-schema-registry"},"release":"v0.6.0"}
{"time":"2023-11-28T21:21:44.718326067Z","level":"WARN","msg":"asset not found","type":"provider","provider":{"namespace":"3lvia","name":"confluent-schema-registry"},"github":{"url":"https://github.com/3lvia/terraform-provider-confluent-schema-registry/releases/download/0.3.0/terraform-provider-confluent-schema-registry_0.3.0_SHA[25](https://github.com/opentofu/registry-stable/actions/runs/7024681594/job/19113935619#step:4:26)6SUMS"}}
{"time":"2023-11-28T21:21:44.718373175Z","level":"WARN","msg":"checksums not found in release, skipping...","type":"provider","provider":{"namespace":"3lvia","name":"confluent-schema-registry"},"release":"0.3.0"}
{"time":"2023-11-28T21:21:44.777948074Z","level":"WARN","msg":"asset not found","type":"provider","provider":{"namespace":"3lvia","name":"confluent-schema-registry"},"github":{"url":"https://github.com/3lvia/terraform-provider-confluent-schema-registry/releases/download/v0.1.0/terraform-provider-confluent-schema-registry_0.1.0_SHA256SUMS"}}
{"time":"2023-11-28T21:21:44.777991084Z","level":"WARN","msg":"checksums not found in release, skipping...","type":"provider","provider":{"namespace":"3lvia","name":"confluent-schema-registry"},"release":"v0.1.0"}
{"time":"2023-11-28T21:21:44.807403206Z","level":"WARN","msg":"asset not found","type":"provider","provider":{"namespace":"3lvia","name":"confluent-schema-registry"},"github":{"url":"https://github.com/3lvia/terraform-provider-confluent-schema-registry/releases/download/v0.7.0/terraform-provider-confluent-schema-registry_0.7.0_SHA256SUMS"}}
{"time":"2023-11-28T21:21:44.807443Z","level":"WARN","msg":"checksums not found in release, skipping...","type":"provider","provider":{"namespace":"3lvia","name":"confluent-schema-registry"},"release":"v0.7.0"}
{"time":"2023-11-28T21:21:44.8396[27](https://github.com/opentofu/registry-stable/actions/runs/7024681594/job/19113935619#step:4:28)776Z","level":"WARN","msg":"asset not found","type":"provider","provider":{"namespace":"3lvia","name":"confluent-schema-registry"},"github":{"url":"https://github.com/3lvia/terraform-provider-confluent-schema-registry/releases/download/v0.3.2/terraform-provider-confluent-schema-registry_0.3.2_SHA256SUMS"}}
{"time":"2023-11-[28](https://github.com/opentofu/registry-stable/actions/runs/7024681594/job/19113935619#step:4:29)T21:21:44.839680694Z","level":"WARN","msg":"checksums not found in release, skipping...","type":"provider","provider":{"namespace":"3lvia","name":"confluent-schema-registry"},"release":"v0.3.2"}
{"time":"2023-11-28T21:21:44.874419639Z","level":"WARN","msg":"asset not found","type":"provider","provider":{"namespace":"3lvia","name":"confluent-schema-registry"},"github":{"url":"https://github.com/3lvia/terraform-provider-confluent-schema-registry/releases/download/v0.8.1/terraform-provider-confluent-schema-registry_0.8.1_SHA256SUMS"}}
{"time":"2023-11-28T21:21:44.874456878Z","level":"WARN","msg":"checksums not found in release, skipping...","type":"provider","provider":{"namespace":"3lvia","name":"confluent-schema-registry"},"release":"v0.8.1"}
{"time":"2023-11-28T21:21:44.912244969Z","level":"WARN","msg":"asset not found","type":"provider","provider":{"namespace":"3lvia","name":"confluent-schema-registry"},"github":{"url":"https://github.com/3lvia/terraform-provider-confluent-schema-registry/releases/download/v0.8.0/terraform-provider-confluent-schema-registry_0.8.0_SHA256SUMS"}}
{"time":"2023-11-28T21:21:44.912[29](https://github.com/opentofu/registry-stable/actions/runs/7024681594/job/19113935619#step:4:30)0673Z","level":"WARN","msg":"checksums not found in release, skipping...","type":"provider","provider":{"namespace":"3lvia","name":"confluent-schema-registry"},"release":"v0.8.0"}
{"time":"2023-11-28T21:21:44.9[31](https://github.com/opentofu/registry-stable/actions/runs/7024681594/job/19113935619#step:4:32)776664Z","level":"WARN","msg":"asset not found","type":"provider","provider":{"namespace":"3lvia","name":"confluent-schema-registry"},"github":{"url":"https://github.com/3lvia/terraform-provider-confluent-schema-registry/releases/download/v0.3.1/terraform-provider-confluent-schema-registry_0.3.1_SHA256SUMS"}}
{"time":"2023-11-28T21:21:44.931861792Z","level":"WARN","msg":"checksums not found in release, skipping...","type":"provider","provider":{"namespace":"3lvia","name":"confluent-schema-registry"},"release":"v0.3.1"}
{"time":"2023-11-28T21:21:44.9429050[41](https://github.com/opentofu/registry-stable/actions/runs/7024681594/job/19113935619#step:4:42)Z","level":"WARN","msg":"asset not found","type":"provider","provider":{"namespace":"3lvia","name":"confluent-schema-registry"},"github":{"url":"https://github.com/3lvia/terraform-provider-confluent-schema-registry/releases/download/v0.3.3/terraform-provider-confluent-schema-registry_0.3.3_SHA256SUMS"}}
{"time":"2023-11-28T21:21:44.9[42](https://github.com/opentofu/registry-stable/actions/runs/7024681594/job/19113935619#step:4:43)941018Z","level":"WARN","msg":"checksums not found in release, skipping...","type":"provider","provider":{"namespace":"3lvia","name":"confluent-schema-registry"},"release":"v0.3.3"}

Proposed Solution

  1. Update the provider version json files by removing tags with missing assets;
  2. Maintain a separate exclusion list which will be used to skip tags w/o assets.

Rationale

Assets are missing in 14% cases, hence addressing it will give a non-marginal throughput/execution time improvement:

providers total versions total versions w/o assets
3584 34492 4884
@kislerdm kislerdm changed the title Maintain exclusion list for provider versions with no requires assets Maintain exclusion list for provider versions with no required assets Nov 28, 2023
@Yantrio Yantrio added this to the Stage 2 Stretch Goals milestone Nov 29, 2023
@cam72cam
Copy link
Contributor

We do need to be careful as a release could be created for a tag hours or days after the tag is initially pushed. We would either want to re-check every day or only exclude tags older than a week (for example).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants