Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chain of Trust errors on signing tasks #2388

Closed
ahal opened this issue Feb 25, 2023 · 10 comments · Fixed by #2389
Closed

Chain of Trust errors on signing tasks #2388

ahal opened this issue Feb 25, 2023 · 10 comments · Fixed by #2389

Comments

@ahal
Copy link
Member

ahal commented Feb 25, 2023

RyanVM noticed some chain of trust errors:
https://firefox-ci-tc.services.mozilla.com/tasks/groups/CrIshxNpQPGHuAMsnsOjEA

This is due to a key rotation we did last week and the fact that the cached docker-image task was run on a worker that had the old key. To fix it, we'll need to cause a rebuild.

Unfortunately glean is using a non-standard index route which is causing the add-new-jobs action to fail, and it is also using an older version of Taskgraph that is missing the rebuild-cached-tasks action. Both of these things should be simple to fix.

ahal added a commit to ahal/glean that referenced this issue Feb 25, 2023
This is the route that various actions (like `add-new-jobs`) in
Taskgraph expect to exist. Alternatively we could have updated Taskgraph
to support both formats, but having a single standard is preferable.

Issue: mozilla#2388
ahal added a commit to ahal/glean that referenced this issue Feb 25, 2023
This is mainly to make use of the `rebuild-cached-tasks` action which
will help us resolve some Chain of Trust failures.

Issue: mozilla#2388
ahal added a commit to ahal/glean that referenced this issue Feb 25, 2023
This is mainly to make use of the `rebuild-cached-tasks` action which
will help us resolve some Chain of Trust failures.

Issue: mozilla#2388
@Dexterp37
Copy link
Contributor

@ahal is there anything needed from us, here?

@ahal
Copy link
Member Author

ahal commented Feb 27, 2023

@Dexterp37 Hi, yes.. It looks like there are some gradle failures that are likely caused by my PR:
https://firefox-ci-tc.services.mozilla.com/tasks/EV1q2hJdSa62PBlo46kbyQ/runs/0/logs/live/public/logs/live.log

I think by using the newer decision task image, it caused gradle to upgrade and fail due to incompatibilities. Unfortunately this newer image is needed to upgrade Taskgraph. Do you think these would be easy to resolve (I'm not familiar with gradle)?

@JohanLorenzo have you encountered gradle errors like this while updating any of the mobile repos to the newer Decision image?

@ahal
Copy link
Member Author

ahal commented Feb 27, 2023

Also if we just need the signing tasks fixed ASAP, it should be sufficient to land the first commit in that PR which doesn't touch the Decision task image.

@ahal
Copy link
Member Author

ahal commented Feb 27, 2023

Hm, I think my earlier diagnosis was wrong.

Either that failure is simply an intermittent that is unrelated to my PR, or re-running the build-docker-image-linux cached task somehow caused a newer version of gradle to be used (though afaict, that image doesn't install gradle at all).

@ahal
Copy link
Member Author

ahal commented Feb 27, 2023

Ok, looks like it was just an unrelated intermittent after all, went green on a re-run. I'll still need help getting the PR reviewed and landed though, as I don't have collaborator permissions here.

Dexterp37 pushed a commit that referenced this issue Feb 27, 2023
* Add a Decision task index route to appease Taskgraph actions

This is the route that various actions (like `add-new-jobs`) in
Taskgraph expect to exist. Alternatively we could have updated Taskgraph
to support both formats, but having a single standard is preferable.

Issue: #2388

* Upgrade taskgraph to version 4.1.1

This is mainly to make use of the `rebuild-cached-tasks` action which
will help us resolve some Chain of Trust failures.

Issue: #2388
@rvandermeulen
Copy link
Contributor

@travis79 You'll need to respin the 52.3.0 release (or create a new 52.3.1 one) for this issue too.

@ahal
Copy link
Member Author

ahal commented Feb 27, 2023

But not quite yet, I'm fixing the scope error that got hit on the push to main.

@ahal
Copy link
Member Author

ahal commented Feb 27, 2023

Scopes should be fixed now.

@travis79
Copy link
Member

There was a crash report related to 52.3.0 from Firefox iOS. I'm going to try and get a fix for that and then I'll cut a 52.3.1

@ahal
Copy link
Member Author

ahal commented Feb 27, 2023

Sounds good, we'll need the build-docker-image-linux task to run on the main branch before the chain of trust error is fixed. But sounds like your crash fix will take care of that. If it doesn't run, please ping me and I'll trigger it manually.

ahal added a commit to ahal/glean that referenced this issue Mar 1, 2023
This fixes a couple places I neglected to convert when upgrading
Taskgraph.

Issue: mozilla#2388
chutten pushed a commit that referenced this issue Mar 1, 2023
This fixes a couple places I neglected to convert when upgrading
Taskgraph.

Issue: #2388
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants