Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[action][download_dsyms] use filter argument to /builds to find specific build #19670

Merged
merged 6 commits into from
Dec 3, 2021

Conversation

tmm1
Copy link
Contributor

@tmm1 tmm1 commented Dec 3, 2021

fixes #19669
cc @joshdholtz

@google-cla google-cla bot added the cla: yes label Dec 3, 2021
@tmm1 tmm1 marked this pull request as draft December 3, 2021 02:16
@tmm1
Copy link
Contributor Author

tmm1 commented Dec 3, 2021

Unfortunately this did not work. I think only version can be passed in, not build number? The former is not specific enough to be useful in filtering.

@tmm1
Copy link
Contributor Author

tmm1 commented Dec 3, 2021

The former is not specific enough to be useful in filtering.

In my case I have 500 builds under v2020 and another 500 under v2021. So using filter["preReleaseVersion.version"] = "2021" does help in that it cuts down the number of requests and time by half. But its still almost a minute to download all 500 builds just to notice that the one we want isn't in there yet.

Anyway I updated this PR so its functional now, but would be nice to find further performance improvements here. If the API let us also specify preReleaseVersion.buildNumber or similar that would be ideal. But I didn't see anything like that on https://developer.apple.com/documentation/appstoreconnectapi/list_builds

@joshdholtz joshdholtz self-requested a review December 3, 2021 02:33
@tmm1 tmm1 marked this pull request as ready for review December 3, 2021 02:40
@tmm1
Copy link
Contributor Author

tmm1 commented Dec 3, 2021

Alright I managed to get this working. Now the same download_dsyms invocation takes less than one second for me, vs almost two minutes before.

@joshdholtz
Copy link
Member

@tmm1 I think what you want is filter[version] 🤔 build.version is the build number

build.appStoreVersion and build. preReleaseVersion are the version numbers

Example of using filter[version] 👇

Screen Shot 2021-12-02 at 8 41 10 PM

Sooo... you could filter by preReleaseVersion.version (for the version) and version (for the build number) to really narrow it down 🤷‍♂️ Thoughts?

@tmm1
Copy link
Contributor Author

tmm1 commented Dec 3, 2021

Sooo... you could filter by preReleaseVersion.version (for the version) and version (for the build number) to really narrow it down 🤷‍♂️ Thoughts?

You're right, that totally worked and is super fast. Updated this PR accordingly!

@tmm1
Copy link
Contributor Author

tmm1 commented Dec 3, 2021

Before:

+------+-----------------------+-------------+
|              fastlane summary              |
+------+-----------------------+-------------+
| Step | Action                | Time (in s) |
+------+-----------------------+-------------+
| 1    | opt_out_usage         | 0           |
| 2    | gym                   | 184         |
| 3    | gym                   | 205         |
| 4    | pilot                 | 94          |
| 5    | pilot                 | 73          |
| 6    | download_dsyms        | 108         |
| 7    | download_dsyms        | 99          |
| 8    | download_dsyms        | 97          |
| 9    | download_dsyms        | 103         |
| 10   | download_dsyms        | 103         |
| 11   | download_dsyms        | 102         |
| 12   | download_dsyms        | 99          |
| 13   | download_dsyms        | 109         |
| 14   | sentry_upload_dif     | 90          |
| 15   | download_dsyms        | 98          |
| 16   | download_dsyms        | 94          |
| 17   | download_dsyms        | 103         |
| 18   | sentry_upload_dif     | 16          |
| 19   | clean_build_artifacts | 0           |
+------+-----------------------+-------------+

After:

+------+-----------------------+-------------+
|              fastlane summary              |
+------+-----------------------+-------------+
| Step | Action                | Time (in s) |
+------+-----------------------+-------------+
| 1    | opt_out_usage         | 0           |
| 2    | gym                   | 178         |
| 3    | gym                   | 186         |
| 4    | pilot                 | 87          |
| 5    | pilot                 | 78          |
| 6    | download_dsyms        | 5           |
| 7    | download_dsyms        | 4           |
| 8    | download_dsyms        | 3           |
| 9    | download_dsyms        | 5           |
| 10   | download_dsyms        | 5           |
| 11   | download_dsyms        | 4           |
| 12   | download_dsyms        | 3           |
| 13   | download_dsyms        | 3           |
| 14   | download_dsyms        | 3           |
| 15   | download_dsyms        | 4           |
| 16   | download_dsyms        | 3           |
| 17   | download_dsyms        | 2           |
| 18   | download_dsyms        | 2           |
| 19   | download_dsyms        | 4           |
| 20   | download_dsyms        | 3           |
| 21   | download_dsyms        | 6           |
| 22   | download_dsyms        | 5           |
| 23   | download_dsyms        | 3           |
| 24   | download_dsyms        | 4           |
| 25   | download_dsyms        | 4           |
| 26   | download_dsyms        | 5           |
| 27   | download_dsyms        | 3           |
| 28   | download_dsyms        | 3           |
| 29   | download_dsyms        | 3           |
| 30   | download_dsyms        | 3           |
| 31   | download_dsyms        | 5           |
| 32   | sentry_upload_dif     | 91          |
| 33   | download_dsyms        | 4           |
| 34   | download_dsyms        | 4           |
| 35   | download_dsyms        | 4           |
| 36   | download_dsyms        | 3           |
| 37   | download_dsyms        | 4           |
| 38   | download_dsyms        | 3           |
| 39   | download_dsyms        | 4           |
| 40   | download_dsyms        | 3           |
| 41   | download_dsyms        | 4           |
| 42   | download_dsyms        | 3           |
| 43   | download_dsyms        | 4           |
| 44   | sentry_upload_dif     | 14          |
| 45   | clean_build_artifacts | 0           |
+------+-----------------------+-------------+

Copy link
Member

@joshdholtz joshdholtz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! I added one commit to fix the tests but this is 🔥 Thanks you so much ❤️

@@ -101,7 +101,7 @@ def self.run(params)

if after_uploaded_date && after_uploaded_date >= uploaded_date
UI.verbose("Upload date #{after_uploaded_date} not reached: #{uploaded_date}")
break
next
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using next here doesn't really make sense to me. I tried to reason about it in my commit message in dac7592

Since we know the results are sorted by uploadedDate, I think it should be safe to break out of the loop when we're past the expected date already.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tmm1 Yeah yeah, that does seem okay but I don't fully trust they will be returned in the right order 😬 I trust the official ASC API but we also use the proxy endpoint for Apple ID auth which is the one I don't really trust. So I think I'd feel safer here doing a next just to make sure all the correct ones are found. The slowest part was the API request so this shouldn't increase things too much 🤞

I hope my insane uneasiness makes sense 😛

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough ⛵

@joshdholtz joshdholtz merged commit 77be2c7 into fastlane:master Dec 3, 2021
@fastlane-bot
Copy link

Hey @tmm1 👋

Thank you for your contribution to fastlane and congrats on getting this pull request merged 🎉
The code change now lives in the master branch, however it wasn't released to RubyGems yet.
We usually ship about once a week, and your PR will be included in the next one.

Please let us know if this change requires an immediate release by adding a comment here 👍
We'll notify you once we shipped a new release with your changes 🚀

Copy link

@fastlane-bot fastlane-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Congratulations! 🎉 This was released as part of fastlane 2.199.0 🚀

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

download_dsyms loops over all builds, taking several minutes
3 participants