Skip to content

Fix screenshot generation#24918

Merged
kean merged 1 commit intotrunkfrom
task/screenshot-generation
Oct 7, 2025
Merged

Fix screenshot generation#24918
kean merged 1 commit intotrunkfrom
task/screenshot-generation

Conversation

@kean
Copy link
Copy Markdown
Contributor

@kean kean commented Oct 6, 2025

Changes

Note: this is an approximate list.

  • Fix an issue with status bar override not working
  • Fix compilation errors in the screenshot generation targets
  • Update the targets to the current Fastlane version
  • Optimize the login by using the new auto-login flow
  • Update tests to rely only accessibility identifier so they work for any locale
  • Enable "New Stats" in screenshot generation
  • Fix an issue with FeatureFlagOverrideStore incompatible with launch arguments (String only)
  • Disable animations for SwiftUI views (Stats)
  • Update MockStatsService to give a recently boost to the recent data so the numbers appear to be growing
  • Update it to run non-headless so you could verify what's going on. It doesn't seem to affect the execution time.

Testing

  • Run rake dependencies
  • Enable mocks (rake mocks)
  • Run bundle exec fastlane jetpack_screenshots (screenshots for WordPress)

The screenshots will be under ./fastlane/jetpack_screenshots

Notes

I ran out of time, but in the future it would be ideal to:

  • Add a shortcut to reset the state of the app before each flow without relaunching it. In the current tests, I had to find a good order of steps to make it easier to undo them)
  • Change the locales at runtime without relaunching the tests (it would make it ~20x faster)
  • Rely less on WireMock as we are not interested in testing the flows end-to-end. We just want to take the screenshot quickly and make it easy to maintain

Screenshots (taken by the script)

iPad Pro 11-inch (M4)-1-light-Stats iPad Pro 11-inch (M4)-2-light-Gutenberg iPad Pro 11-inch (M4)-3-light-Reader iPad Pro 11-inch (M4)-4-light-Notifications iPhone 17 Pro Max-1-light-Stats iPhone 17 Pro Max-2-light-Gutenberg iPhone 17 Pro Max-3-light-Reader iPhone 17 Pro Max-4-light-Notifications iPhone 17-1-light-Stats iPhone 17-2-light-Gutenberg iPhone 17-3-light-Reader iPhone 17-4-light-Notifications

@kean kean added this to the 26.5 milestone Oct 6, 2025
@kean kean added the Tooling Build, Release, and Validation Tools label Oct 6, 2025
'iPad Pro (12.9-inch) (5th generation)' # 12.9in - 2732x2048 @ 264 ppi
'iPhone 17 Pro Max', # 6.5in - 2688x1242 @ 458 ppi
'iPhone 17',
'iPad Pro 11-inch (M4)'
Copy link
Copy Markdown
Collaborator

@dangermattic dangermattic Oct 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 Layout/TrailingWhitespace: Trailing whitespace detected.

@dangermattic
Copy link
Copy Markdown
Collaborator

dangermattic commented Oct 6, 2025

1 Warning
⚠️ This PR is larger than 500 lines of changes. Please consider splitting it into smaller PRs for easier and faster reviews.

Generated by 🚫 Danger

let month = calendar.component(.month, from: date)
let year = calendar.component(.year, from: date)

let components: DateComponents = {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The entire MockStatsService is vibe-coded – do not judge too harshly.

case let value as Bool:
return value
case let value as String:
return (value as NSString).boolValue
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The way we set these in UI tests is with the launch arguments that support only strings [String]:

"-ff-override-New Stats", "true",

There was a recent regression in overriddenValue where it was changed to support only Bool value – incompatible with string launch arguments.

actions.append(AnonymousAction(title: "create-mock-post") {
let post = blog.createDraftPost()
post.postTitle = "Welcome to Gutenberg"
post.content = """
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I relied on WireMock in Reader, but then switched to mocks in code for screenshots. For this purpose, we are not interested in testing the flows end-to-end, so there is no reason to mock the JSONs.

.openNotification(withSubstring: "commented on")
}
notificationList.thenTakeScreenshot(5, named: "Notifications")
// // Get a screenshot of the editor with keyboard (iPad only)
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't updated the screenshot Generation for the WordPress app yet. I only focused on Jetpack.

@wpmobilebot
Copy link
Copy Markdown
Contributor

wpmobilebot commented Oct 6, 2025

App Icon📲 You can test the changes from this Pull Request in WordPress by scanning the QR code below to install the corresponding build.
App NameWordPress
ConfigurationRelease-Alpha
Build Number29393
VersionPR #24918
Bundle IDorg.wordpress.alpha
Commitea7dbdd
Installation URL0mv9roh5h8a2g
Automatticians: You can use our internal self-serve MC tool to give yourself access to those builds if needed.

@wpmobilebot
Copy link
Copy Markdown
Contributor

wpmobilebot commented Oct 6, 2025

App Icon📲 You can test the changes from this Pull Request in Jetpack by scanning the QR code below to install the corresponding build.
App NameJetpack
ConfigurationRelease-Alpha
Build Number29393
VersionPR #24918
Bundle IDcom.jetpack.alpha
Commitea7dbdd
Installation URL0k7m18jvttpq8
Automatticians: You can use our internal self-serve MC tool to give yourself access to those builds if needed.

@kean kean marked this pull request as ready for review October 6, 2025 19:58
@kean kean requested a review from a team as a code owner October 6, 2025 19:58
@kean kean force-pushed the task/screenshot-generation branch from 85de1db to ea7dbdd Compare October 6, 2025 20:00
@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud Bot commented Oct 6, 2025

@kean kean requested a review from jkmassel October 6, 2025 20:55

func disableDelays() {
delaysDisabled = true
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: I don't think it's worth the trouble to call this function from the app. The MockStatsService can check for launch arguments and things internally to show data without delay.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Yeah, it's a bit of indirection. I'll try and see if maybe UITestConfigurator could be moved to one of the frameworks.

@kean kean added this pull request to the merge queue Oct 7, 2025
Merged via the queue into trunk with commit a95b395 Oct 7, 2025
28 of 32 checks passed
@kean kean deleted the task/screenshot-generation branch October 7, 2025 17:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Tooling Build, Release, and Validation Tools

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants