Skip to content

feat: signinscript - recover from notarization connection error#714

Merged
hneiva merged 4 commits into
mozilla-releng:masterfrom
hneiva:hneiva/rcodesign-retry
Apr 27, 2023
Merged

feat: signinscript - recover from notarization connection error#714
hneiva merged 4 commits into
mozilla-releng:masterfrom
hneiva:hneiva/rcodesign-retry

Conversation

@hneiva

@hneiva hneiva commented Apr 21, 2023

Copy link
Copy Markdown
Contributor

No description provided.

@hneiva hneiva force-pushed the hneiva/rcodesign-retry branch from 0197b73 to 72c18e3 Compare April 22, 2023 02:50
Comment thread signingscript/src/signingscript/sign.py Outdated
Comment thread signingscript/src/signingscript/sign.py Outdated

@jcristau jcristau left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with Ben's comments about keeping the flow the same with/without retries.

Comment thread signingscript/src/signingscript/sign.py Outdated
Comment thread signingscript/src/signingscript/sign.py Outdated
await utils.execute_subprocess(command)

# Similar implementation to utils.execute_subprocess, but handling some errors:
message = 'Running "{}"'.format(" ".join(command))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use a f-string?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

f-string in this case makes it harder to copy-pasta into the terminal :P

@hneiva

hneiva commented Apr 25, 2023

Copy link
Copy Markdown
Contributor Author

Updated the code so it's more straight forward, and separates the rcodesign interface into it's own file.

Still missing tests, I'll add that next, but code is functional.

@hneiva hneiva requested review from bhearsum and jcristau April 26, 2023 00:32
@hneiva

hneiva commented Apr 26, 2023

Copy link
Copy Markdown
Contributor Author

Fixed and added unit tests. Ready for full review 🚀

@hneiva hneiva requested a review from a team April 26, 2023 18:48
if stderr:
# Unfortunately a lot of outputs from rcodesign come out to stderr
log.warn(stderr)
output_lines.append(stderr)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @jcristau's earlier comments about mixing stdout and stderr (this is just another form of that). Presumably you know whether it's stdout or stderr that has the string you're looking for, so you should be able to keep these separate?

@hneiva hneiva Apr 26, 2023

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jcristau comment was regarding sending stderr=asyncio.subprocess.STDOUT to asyncio.create_subprocess_exec - which wouldn't allow us to collect them at all.

Regardless, as stated on the comment above the log.warn, rcodesigndoesn't only output errors to stderr (there's good data in it). Therefore separating them has the opposite desired effect.
Let me know if the inline comment isn't enough.

args=[path],
attempts=ATTEMPTS,
retry_exceptions=RCodesignError,
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure we need to call all of these? The rcodesign docs seems to suggest that your first function should be waiting and stapling all by itself. It says: "Or to wait and automatically staple the file if notarization was successful:"

rcodesign notary-submit \
  --api-key-path ~/.appstoreconnect/key.json \
  --staple \
  path/to/file/to/notarize

It seems like the first call being retried ought to do everything we need (at the cost of resubmitting entirely on any failure).

I think a better way to approach this may be:

  1. Submit without --staple, so that notarize returns right away. This should be retried because the submission may fail
  2. Run notary_wait with retries (obviously the polling may fail for various reasons).
  3. Run staple. There shouldn't be any need to retry here.

What do you think? Am I overlooking something?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

staple does reach out to the Apple servers to collect the ticket, therefore I see the benefit of having a retry. But I can change it if you still think it's a good idea.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, good to know. I think the overarching comment still remains though: the first command here (notary-submit --staple) already does all of: submission, polling, stapling. If we're already retrying that command, what is the purpose of running notary_wait and staple separately?

@hneiva hneiva Apr 27, 2023

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The way I changed it "wait and staple" is optional. In this case, it just returns the submission id, then we have the wait, and then stapling.

# function:
async def rcodesign_notarize(app_path, creds_path, staple=False):
.......
# usage:
    submission_id = await retry_async(
        func=rcodesign_notarize,
        args=(path, creds_path), # <--- no staple

So, the first command only submits and returns the submission id.

I think you missed the change in the notary-submit code 🙃

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh right, sorry. I somehow glossed over the fact that --staple was optional, heh. Alright, proceed!

@hneiva hneiva merged commit 2e8a08e into mozilla-releng:master Apr 27, 2023
@hneiva hneiva deleted the hneiva/rcodesign-retry branch April 27, 2023 16:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants