Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add lightbox tests covering theme.json configurations, including edge cases #60150

Closed
wants to merge 17 commits into from

Conversation

artemiomorales
Copy link
Contributor

@artemiomorales artemiomorales commented Mar 25, 2024

What?

Since the UI for the lightbox has now been updated in #57608 with extensive edge cases covered via #59890 (comment), this PR reenables and updates the old lightbox tests while adding many new ones.

Why?

We want to have test coverage on the lightbox to prevent regressions.

How?

  • It adds a test theme to sandbox all of the lightbox maneuvering, then toggles the various theme.json configurations via test plugins to reach various scenarios, testing both the UI and the frontend.
  • It ensures lightbox settings in theme.json can be overridden via the global styles.
  • For tests in the Site Editor, it adds blocks as needed then cleans them up.
  • Since this PR is already big, it skips over various related tests, such as testing the lightbox markup and keyboard navigation, that I will handle in a separate PR.

Testing Instructions

Run the tests locally with Docker running, using the command npm run test:e2e:playwright -- image.spec.js --debug

Copy link

github-actions bot commented Mar 25, 2024

Size Change: +18.6 kB (+1%)

Total Size: 1.73 MB

Filename Size Change
build/block-editor/content-rtl.css 4.46 kB +30 B (+1%)
build/block-editor/content.css 4.46 kB +30 B (+1%)
build/block-editor/index.min.js 254 kB +1.2 kB (0%)
build/block-editor/style.css 15.6 kB -1 B (0%)
build/block-library/blocks/navigation/view.min.js 1.02 kB +3 B (0%)
build/block-library/blocks/social-links/editor-rtl.css 676 B -2 B (0%)
build/block-library/blocks/social-links/editor.css 675 B -3 B (0%)
build/block-library/editor-rtl.css 12.4 kB -2 B (0%)
build/block-library/editor.css 12.4 kB -2 B (0%)
build/block-library/index.min.js 218 kB +247 B (0%)
build/blocks/index.min.js 51.9 kB +83 B (0%)
build/commands/index.min.js 15.6 kB +8 B (0%)
build/commands/style-rtl.css 956 B +21 B (+2%)
build/commands/style.css 953 B +23 B (+2%)
build/components/index.min.js 222 kB -1.17 kB (-1%)
build/components/style-rtl.css 11.8 kB +1 B (0%)
build/components/style.css 11.8 kB -1 B (0%)
build/edit-post/classic-rtl.css 578 B +20 B (+4%)
build/edit-post/classic.css 579 B +21 B (+4%)
build/edit-post/index.min.js 23.7 kB -298 B (-1%)
build/edit-post/style-rtl.css 5.54 kB -23 B (0%)
build/edit-post/style.css 5.53 kB -23 B (0%)
build/edit-site/index.min.js 232 kB +15.2 kB (+7%) 🔍
build/edit-site/style-rtl.css 15.1 kB -6 B (0%)
build/edit-site/style.css 15.1 kB -10 B (0%)
build/edit-widgets/index.min.js 17.4 kB +11 B (0%)
build/editor/index.min.js 67.9 kB +2.84 kB (+4%)
build/editor/style-rtl.css 5.49 kB +53 B (+1%)
build/editor/style.css 5.49 kB +53 B (+1%)
build/format-library/index.min.js 8.11 kB +7 B (0%)
build/interactivity/navigation.min.js 1.15 kB +3 B (0%)
build/patterns/index.min.js 5.91 kB +177 B (+3%)
build/preferences-persistence/index.min.js 2.06 kB +6 B (0%)
build/preferences/index.min.js 2.83 kB +17 B (+1%)
build/router/index.min.js 1.88 kB +90 B (+5%) 🔍
build/url/index.min.js 3.74 kB +14 B (0%)
ℹ️ View Unchanged
Filename Size
build/a11y/index.min.js 955 B
build/annotations/index.min.js 2.69 kB
build/api-fetch/index.min.js 2.32 kB
build/autop/index.min.js 2.1 kB
build/blob/index.min.js 578 B
build/block-directory/index.min.js 7.26 kB
build/block-directory/style-rtl.css 1.03 kB
build/block-directory/style.css 1.03 kB
build/block-editor/default-editor-styles-rtl.css 394 B
build/block-editor/default-editor-styles.css 394 B
build/block-editor/style-rtl.css 15.6 kB
build/block-library/blocks/archives/editor-rtl.css 61 B
build/block-library/blocks/archives/editor.css 60 B
build/block-library/blocks/archives/style-rtl.css 90 B
build/block-library/blocks/archives/style.css 90 B
build/block-library/blocks/audio/editor-rtl.css 150 B
build/block-library/blocks/audio/editor.css 150 B
build/block-library/blocks/audio/style-rtl.css 122 B
build/block-library/blocks/audio/style.css 122 B
build/block-library/blocks/audio/theme-rtl.css 133 B
build/block-library/blocks/audio/theme.css 133 B
build/block-library/blocks/avatar/editor-rtl.css 116 B
build/block-library/blocks/avatar/editor.css 116 B
build/block-library/blocks/avatar/style-rtl.css 104 B
build/block-library/blocks/avatar/style.css 104 B
build/block-library/blocks/block/editor-rtl.css 305 B
build/block-library/blocks/block/editor.css 305 B
build/block-library/blocks/button/editor-rtl.css 415 B
build/block-library/blocks/button/editor.css 414 B
build/block-library/blocks/button/style-rtl.css 627 B
build/block-library/blocks/button/style.css 626 B
build/block-library/blocks/buttons/editor-rtl.css 337 B
build/block-library/blocks/buttons/editor.css 337 B
build/block-library/blocks/buttons/style-rtl.css 332 B
build/block-library/blocks/buttons/style.css 332 B
build/block-library/blocks/calendar/style-rtl.css 239 B
build/block-library/blocks/calendar/style.css 239 B
build/block-library/blocks/categories/editor-rtl.css 113 B
build/block-library/blocks/categories/editor.css 112 B
build/block-library/blocks/categories/style-rtl.css 124 B
build/block-library/blocks/categories/style.css 124 B
build/block-library/blocks/code/editor-rtl.css 53 B
build/block-library/blocks/code/editor.css 53 B
build/block-library/blocks/code/style-rtl.css 121 B
build/block-library/blocks/code/style.css 121 B
build/block-library/blocks/code/theme-rtl.css 124 B
build/block-library/blocks/code/theme.css 124 B
build/block-library/blocks/columns/editor-rtl.css 108 B
build/block-library/blocks/columns/editor.css 108 B
build/block-library/blocks/columns/style-rtl.css 421 B
build/block-library/blocks/columns/style.css 421 B
build/block-library/blocks/comment-author-avatar/editor-rtl.css 125 B
build/block-library/blocks/comment-author-avatar/editor.css 125 B
build/block-library/blocks/comment-content/style-rtl.css 92 B
build/block-library/blocks/comment-content/style.css 92 B
build/block-library/blocks/comment-template/style-rtl.css 199 B
build/block-library/blocks/comment-template/style.css 198 B
build/block-library/blocks/comments-pagination-numbers/editor-rtl.css 123 B
build/block-library/blocks/comments-pagination-numbers/editor.css 121 B
build/block-library/blocks/comments-pagination/editor-rtl.css 222 B
build/block-library/blocks/comments-pagination/editor.css 209 B
build/block-library/blocks/comments-pagination/style-rtl.css 235 B
build/block-library/blocks/comments-pagination/style.css 231 B
build/block-library/blocks/comments-title/editor-rtl.css 75 B
build/block-library/blocks/comments-title/editor.css 75 B
build/block-library/blocks/comments/editor-rtl.css 840 B
build/block-library/blocks/comments/editor.css 839 B
build/block-library/blocks/comments/style-rtl.css 637 B
build/block-library/blocks/comments/style.css 636 B
build/block-library/blocks/cover/editor-rtl.css 647 B
build/block-library/blocks/cover/editor.css 650 B
build/block-library/blocks/cover/style-rtl.css 1.7 kB
build/block-library/blocks/cover/style.css 1.69 kB
build/block-library/blocks/details/editor-rtl.css 65 B
build/block-library/blocks/details/editor.css 65 B
build/block-library/blocks/details/style-rtl.css 98 B
build/block-library/blocks/details/style.css 98 B
build/block-library/blocks/embed/editor-rtl.css 322 B
build/block-library/blocks/embed/editor.css 322 B
build/block-library/blocks/embed/style-rtl.css 410 B
build/block-library/blocks/embed/style.css 410 B
build/block-library/blocks/embed/theme-rtl.css 133 B
build/block-library/blocks/embed/theme.css 133 B
build/block-library/blocks/file/editor-rtl.css 326 B
build/block-library/blocks/file/editor.css 327 B
build/block-library/blocks/file/style-rtl.css 280 B
build/block-library/blocks/file/style.css 281 B
build/block-library/blocks/file/view.min.js 324 B
build/block-library/blocks/footnotes/style-rtl.css 201 B
build/block-library/blocks/footnotes/style.css 199 B
build/block-library/blocks/form-input/editor-rtl.css 227 B
build/block-library/blocks/form-input/editor.css 227 B
build/block-library/blocks/form-input/style-rtl.css 343 B
build/block-library/blocks/form-input/style.css 343 B
build/block-library/blocks/form-submission-notification/editor-rtl.css 340 B
build/block-library/blocks/form-submission-notification/editor.css 340 B
build/block-library/blocks/form-submit-button/style-rtl.css 69 B
build/block-library/blocks/form-submit-button/style.css 69 B
build/block-library/blocks/form/view.min.js 471 B
build/block-library/blocks/freeform/editor-rtl.css 2.61 kB
build/block-library/blocks/freeform/editor.css 2.61 kB
build/block-library/blocks/gallery/editor-rtl.css 947 B
build/block-library/blocks/gallery/editor.css 952 B
build/block-library/blocks/gallery/style-rtl.css 1.72 kB
build/block-library/blocks/gallery/style.css 1.72 kB
build/block-library/blocks/gallery/theme-rtl.css 108 B
build/block-library/blocks/gallery/theme.css 108 B
build/block-library/blocks/group/editor-rtl.css 647 B
build/block-library/blocks/group/editor.css 647 B
build/block-library/blocks/group/style-rtl.css 57 B
build/block-library/blocks/group/style.css 57 B
build/block-library/blocks/group/theme-rtl.css 78 B
build/block-library/blocks/group/theme.css 78 B
build/block-library/blocks/heading/style-rtl.css 189 B
build/block-library/blocks/heading/style.css 189 B
build/block-library/blocks/html/editor-rtl.css 336 B
build/block-library/blocks/html/editor.css 337 B
build/block-library/blocks/image/editor-rtl.css 878 B
build/block-library/blocks/image/editor.css 878 B
build/block-library/blocks/image/style-rtl.css 1.6 kB
build/block-library/blocks/image/style.css 1.59 kB
build/block-library/blocks/image/theme-rtl.css 133 B
build/block-library/blocks/image/theme.css 133 B
build/block-library/blocks/image/view.min.js 1.54 kB
build/block-library/blocks/latest-comments/style-rtl.css 357 B
build/block-library/blocks/latest-comments/style.css 357 B
build/block-library/blocks/latest-posts/editor-rtl.css 213 B
build/block-library/blocks/latest-posts/editor.css 212 B
build/block-library/blocks/latest-posts/style-rtl.css 478 B
build/block-library/blocks/latest-posts/style.css 478 B
build/block-library/blocks/list/style-rtl.css 88 B
build/block-library/blocks/list/style.css 88 B
build/block-library/blocks/media-text/editor-rtl.css 306 B
build/block-library/blocks/media-text/editor.css 305 B
build/block-library/blocks/media-text/style-rtl.css 505 B
build/block-library/blocks/media-text/style.css 503 B
build/block-library/blocks/more/editor-rtl.css 431 B
build/block-library/blocks/more/editor.css 431 B
build/block-library/blocks/navigation-link/editor-rtl.css 668 B
build/block-library/blocks/navigation-link/editor.css 669 B
build/block-library/blocks/navigation-link/style-rtl.css 259 B
build/block-library/blocks/navigation-link/style.css 257 B
build/block-library/blocks/navigation-submenu/editor-rtl.css 296 B
build/block-library/blocks/navigation-submenu/editor.css 295 B
build/block-library/blocks/navigation/editor-rtl.css 2.26 kB
build/block-library/blocks/navigation/editor.css 2.26 kB
build/block-library/blocks/navigation/style-rtl.css 2.26 kB
build/block-library/blocks/navigation/style.css 2.25 kB
build/block-library/blocks/nextpage/editor-rtl.css 395 B
build/block-library/blocks/nextpage/editor.css 395 B
build/block-library/blocks/page-list/editor-rtl.css 377 B
build/block-library/blocks/page-list/editor.css 377 B
build/block-library/blocks/page-list/style-rtl.css 175 B
build/block-library/blocks/page-list/style.css 175 B
build/block-library/blocks/paragraph/editor-rtl.css 235 B
build/block-library/blocks/paragraph/editor.css 235 B
build/block-library/blocks/paragraph/style-rtl.css 335 B
build/block-library/blocks/paragraph/style.css 335 B
build/block-library/blocks/post-author/style-rtl.css 175 B
build/block-library/blocks/post-author/style.css 176 B
build/block-library/blocks/post-comments-form/editor-rtl.css 96 B
build/block-library/blocks/post-comments-form/editor.css 96 B
build/block-library/blocks/post-comments-form/style-rtl.css 508 B
build/block-library/blocks/post-comments-form/style.css 508 B
build/block-library/blocks/post-content/editor-rtl.css 74 B
build/block-library/blocks/post-content/editor.css 74 B
build/block-library/blocks/post-date/style-rtl.css 61 B
build/block-library/blocks/post-date/style.css 61 B
build/block-library/blocks/post-excerpt/editor-rtl.css 71 B
build/block-library/blocks/post-excerpt/editor.css 71 B
build/block-library/blocks/post-excerpt/style-rtl.css 141 B
build/block-library/blocks/post-excerpt/style.css 141 B
build/block-library/blocks/post-featured-image/editor-rtl.css 666 B
build/block-library/blocks/post-featured-image/editor.css 662 B
build/block-library/blocks/post-featured-image/style-rtl.css 342 B
build/block-library/blocks/post-featured-image/style.css 342 B
build/block-library/blocks/post-navigation-link/style-rtl.css 215 B
build/block-library/blocks/post-navigation-link/style.css 214 B
build/block-library/blocks/post-template/editor-rtl.css 99 B
build/block-library/blocks/post-template/editor.css 98 B
build/block-library/blocks/post-template/style-rtl.css 409 B
build/block-library/blocks/post-template/style.css 408 B
build/block-library/blocks/post-terms/style-rtl.css 96 B
build/block-library/blocks/post-terms/style.css 96 B
build/block-library/blocks/post-time-to-read/style-rtl.css 69 B
build/block-library/blocks/post-time-to-read/style.css 69 B
build/block-library/blocks/post-title/style-rtl.css 100 B
build/block-library/blocks/post-title/style.css 100 B
build/block-library/blocks/preformatted/style-rtl.css 125 B
build/block-library/blocks/preformatted/style.css 125 B
build/block-library/blocks/pullquote/editor-rtl.css 135 B
build/block-library/blocks/pullquote/editor.css 135 B
build/block-library/blocks/pullquote/style-rtl.css 354 B
build/block-library/blocks/pullquote/style.css 354 B
build/block-library/blocks/pullquote/theme-rtl.css 174 B
build/block-library/blocks/pullquote/theme.css 174 B
build/block-library/blocks/query-pagination-numbers/editor-rtl.css 122 B
build/block-library/blocks/query-pagination-numbers/editor.css 121 B
build/block-library/blocks/query-pagination/editor-rtl.css 221 B
build/block-library/blocks/query-pagination/editor.css 211 B
build/block-library/blocks/query-pagination/style-rtl.css 288 B
build/block-library/blocks/query-pagination/style.css 284 B
build/block-library/blocks/query-title/style-rtl.css 63 B
build/block-library/blocks/query-title/style.css 63 B
build/block-library/blocks/query/editor-rtl.css 486 B
build/block-library/blocks/query/editor.css 486 B
build/block-library/blocks/query/view.min.js 958 B
build/block-library/blocks/quote/style-rtl.css 237 B
build/block-library/blocks/quote/style.css 237 B
build/block-library/blocks/quote/theme-rtl.css 233 B
build/block-library/blocks/quote/theme.css 235 B
build/block-library/blocks/read-more/style-rtl.css 140 B
build/block-library/blocks/read-more/style.css 140 B
build/block-library/blocks/rss/editor-rtl.css 149 B
build/block-library/blocks/rss/editor.css 149 B
build/block-library/blocks/rss/style-rtl.css 289 B
build/block-library/blocks/rss/style.css 288 B
build/block-library/blocks/search/editor-rtl.css 184 B
build/block-library/blocks/search/editor.css 184 B
build/block-library/blocks/search/style-rtl.css 629 B
build/block-library/blocks/search/style.css 628 B
build/block-library/blocks/search/theme-rtl.css 114 B
build/block-library/blocks/search/theme.css 114 B
build/block-library/blocks/search/view.min.js 478 B
build/block-library/blocks/separator/editor-rtl.css 146 B
build/block-library/blocks/separator/editor.css 146 B
build/block-library/blocks/separator/style-rtl.css 229 B
build/block-library/blocks/separator/style.css 229 B
build/block-library/blocks/separator/theme-rtl.css 194 B
build/block-library/blocks/separator/theme.css 194 B
build/block-library/blocks/shortcode/editor-rtl.css 323 B
build/block-library/blocks/shortcode/editor.css 323 B
build/block-library/blocks/site-logo/editor-rtl.css 754 B
build/block-library/blocks/site-logo/editor.css 754 B
build/block-library/blocks/site-logo/style-rtl.css 204 B
build/block-library/blocks/site-logo/style.css 204 B
build/block-library/blocks/site-tagline/editor-rtl.css 86 B
build/block-library/blocks/site-tagline/editor.css 86 B
build/block-library/blocks/site-title/editor-rtl.css 116 B
build/block-library/blocks/site-title/editor.css 116 B
build/block-library/blocks/site-title/style-rtl.css 57 B
build/block-library/blocks/site-title/style.css 57 B
build/block-library/blocks/social-link/editor-rtl.css 184 B
build/block-library/blocks/social-link/editor.css 184 B
build/block-library/blocks/social-links/style-rtl.css 1.48 kB
build/block-library/blocks/social-links/style.css 1.48 kB
build/block-library/blocks/spacer/editor-rtl.css 350 B
build/block-library/blocks/spacer/editor.css 350 B
build/block-library/blocks/spacer/style-rtl.css 48 B
build/block-library/blocks/spacer/style.css 48 B
build/block-library/blocks/table/editor-rtl.css 395 B
build/block-library/blocks/table/editor.css 395 B
build/block-library/blocks/table/style-rtl.css 639 B
build/block-library/blocks/table/style.css 639 B
build/block-library/blocks/table/theme-rtl.css 152 B
build/block-library/blocks/table/theme.css 152 B
build/block-library/blocks/tag-cloud/style-rtl.css 251 B
build/block-library/blocks/tag-cloud/style.css 253 B
build/block-library/blocks/template-part/editor-rtl.css 431 B
build/block-library/blocks/template-part/editor.css 431 B
build/block-library/blocks/template-part/theme-rtl.css 107 B
build/block-library/blocks/template-part/theme.css 107 B
build/block-library/blocks/term-description/style-rtl.css 111 B
build/block-library/blocks/term-description/style.css 111 B
build/block-library/blocks/text-columns/editor-rtl.css 95 B
build/block-library/blocks/text-columns/editor.css 95 B
build/block-library/blocks/text-columns/style-rtl.css 166 B
build/block-library/blocks/text-columns/style.css 166 B
build/block-library/blocks/verse/style-rtl.css 99 B
build/block-library/blocks/verse/style.css 99 B
build/block-library/blocks/video/editor-rtl.css 552 B
build/block-library/blocks/video/editor.css 555 B
build/block-library/blocks/video/style-rtl.css 185 B
build/block-library/blocks/video/style.css 185 B
build/block-library/blocks/video/theme-rtl.css 133 B
build/block-library/blocks/video/theme.css 133 B
build/block-library/classic-rtl.css 179 B
build/block-library/classic.css 179 B
build/block-library/common-rtl.css 1.11 kB
build/block-library/common.css 1.11 kB
build/block-library/editor-elements-rtl.css 75 B
build/block-library/editor-elements.css 75 B
build/block-library/elements-rtl.css 54 B
build/block-library/elements.css 54 B
build/block-library/reset-rtl.css 472 B
build/block-library/reset.css 472 B
build/block-library/style-rtl.css 14.8 kB
build/block-library/style.css 14.8 kB
build/block-library/theme-rtl.css 707 B
build/block-library/theme.css 713 B
build/block-serialization-default-parser/index.min.js 1.12 kB
build/block-serialization-spec-parser/index.min.js 2.87 kB
build/compose/index.min.js 12.6 kB
build/core-commands/index.min.js 2.77 kB
build/core-data/index.min.js 72.9 kB
build/customize-widgets/index.min.js 11.2 kB
build/customize-widgets/style-rtl.css 1.36 kB
build/customize-widgets/style.css 1.36 kB
build/data-controls/index.min.js 640 B
build/data/index.min.js 8.98 kB
build/date/index.min.js 17.9 kB
build/deprecated/index.min.js 451 B
build/dom-ready/index.min.js 324 B
build/dom/index.min.js 4.65 kB
build/edit-widgets/style-rtl.css 4.15 kB
build/edit-widgets/style.css 4.15 kB
build/element/index.min.js 4.83 kB
build/escape-html/index.min.js 537 B
build/format-library/style-rtl.css 492 B
build/format-library/style.css 490 B
build/hooks/index.min.js 1.55 kB
build/html-entities/index.min.js 448 B
build/i18n/index.min.js 3.58 kB
build/interactivity/file.min.js 447 B
build/interactivity/image.min.js 1.67 kB
build/interactivity/index.min.js 13 kB
build/interactivity/query.min.js 740 B
build/interactivity/router.min.js 1.36 kB
build/interactivity/search.min.js 618 B
build/is-shallow-equal/index.min.js 527 B
build/keyboard-shortcuts/index.min.js 1.74 kB
build/keycodes/index.min.js 1.46 kB
build/list-reusable-blocks/index.min.js 2.11 kB
build/list-reusable-blocks/style-rtl.css 851 B
build/list-reusable-blocks/style.css 849 B
build/media-utils/index.min.js 2.92 kB
build/modules/importmap-polyfill.min.js 12.2 kB
build/notices/index.min.js 948 B
build/nux/index.min.js 2 kB
build/nux/style-rtl.css 747 B
build/nux/style.css 742 B
build/patterns/style-rtl.css 553 B
build/patterns/style.css 552 B
build/plugins/index.min.js 1.8 kB
build/preferences/style-rtl.css 710 B
build/preferences/style.css 712 B
build/primitives/index.min.js 975 B
build/priority-queue/index.min.js 1.52 kB
build/private-apis/index.min.js 1 kB
build/react-i18n/index.min.js 623 B
build/react-refresh-entry/index.min.js 9.47 kB
build/react-refresh-runtime/index.min.js 6.78 kB
build/redux-routine/index.min.js 2.7 kB
build/reusable-blocks/index.min.js 2.73 kB
build/reusable-blocks/style-rtl.css 256 B
build/reusable-blocks/style.css 256 B
build/rich-text/index.min.js 10.5 kB
build/server-side-render/index.min.js 1.96 kB
build/shortcode/index.min.js 1.39 kB
build/style-engine/index.min.js 2.03 kB
build/token-list/index.min.js 582 B
build/vendors/inert-polyfill.min.js 2.48 kB
build/vendors/react-dom.min.js 41.7 kB
build/vendors/react.min.js 4.02 kB
build/viewport/index.min.js 957 B
build/warning/index.min.js 249 B
build/widgets/index.min.js 7.22 kB
build/widgets/style-rtl.css 1.17 kB
build/widgets/style.css 1.17 kB
build/wordcount/index.min.js 1.02 kB

compressed-size-action

@gziolo gziolo added [Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. [Block] Image Affects the Image Block labels Mar 25, 2024
@gziolo
Copy link
Member

gziolo commented Mar 28, 2024

Nice job covering the scenarios with configuring the lightbox using the theme.json and the block settings.

It adds four test themes containing different lightbox configurations, and toggles them on and off to reach desired edge cases.

It would be great to explore an alternative approach so don’t have to create 4 new test themes to only validate the settings. It doesn’t seem like a scalable approach, given how many settings are there in the theme.json and all the possible permutations. One alternative approach I was thinking about would be using WP filters. There are hooks that allow overriding settings in theme.json that might fit better there, but you would still need to activate them somehow, so that would probably still mean 4 plugins: https://make.wordpress.org/core/2022/10/10/filters-for-theme-json-data/. A good side effect would be that those hooks would get some basic e2e test coverage.

Copy link

Flaky tests detected in 683318f.
Some tests passed with failed attempts. The failures may not be related to this commit but are still reported for visibility. See the documentation for more information.

🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/8482711228
📝 Reported issues:

@artemiomorales
Copy link
Contributor Author

It would be great to explore an alternative approach so don’t have to create 4 new test themes to only validate the settings. It doesn’t seem like a scalable approach, given how many settings are there in the theme.json and all the possible permutations. One alternative approach I was thinking about would be using WP filters. There are hooks that allow overriding settings in theme.json that might fit better there, but you would still need to activate them somehow, so that would probably still mean 4 plugins: https://make.wordpress.org/core/2022/10/10/filters-for-theme-json-data/. A good side effect would be that those hooks would get some basic e2e test coverage.

@gziolo Thanks for the suggestion! I removed the extra themes and moved the theme.json settings to plugins instead.

@artemiomorales artemiomorales marked this pull request as ready for review March 30, 2024 00:13
Copy link

github-actions bot commented Mar 30, 2024

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: artemiomorales <artemiosans@git.wordpress.org>
Co-authored-by: gziolo <gziolo@git.wordpress.org>
Co-authored-by: Mamaduka <mamaduka@git.wordpress.org>
Co-authored-by: kevin940726 <kevin940726@git.wordpress.org>
Co-authored-by: SantosGuillamot <santosguillamot@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@artemiomorales
Copy link
Contributor Author

There is some flakiness in these tests that I'm still hoping to address, but I do think this is ready for reviewing.

@SantosGuillamot
Copy link
Contributor

I don't fully understand why we need a new theme for these tests. If we use plugins with filters, can't we use emptytheme for example?

@SantosGuillamot
Copy link
Contributor

so that would probably still mean 4 plugins

In order to avoid having 4 plugins maybe it could have only 1 plugin that registers two site settings (with register_setting?): one for allowEditing and other for enabled.

The plugin could use just one filter that can read from those settings and those settings can be updated in the tests by using the util updateSiteSettings: example.

Could something like that work?

@artemiomorales
Copy link
Contributor Author

I don't fully understand why we need a new theme for these tests. If we use plugins with filters, can't we use emptytheme for example?

I initially included the new theme to avoid conflicting with other tests in case changes to the site templates somehow persisted — the cleanup code is a bit flaky when run locally. But you're right that we can do without this. Will continue looking at the cleanup code and remove the extra theme in a future push.

In order to avoid having 4 plugins maybe it could have only 1 plugin that registers two site settings (with register_setting?): one for allowEditing and other for enabled.

The plugin could use just one filter that can read from those settings and those settings can be updated in the tests by using the util updateSiteSettings: example.

Could something like that work?

Thanks for the suggestion! I'm actually thinking to simplify these tests to tackle just one theme.json configuration, so we wouldn't need all of these plugins anymore, at least to start. That being said, I'll look into this suggestion further as work on these tests continues.

@artemiomorales
Copy link
Contributor Author

This PR is very big and I hadn't been able to resolve the flakiness. It's been a few weeks now and looking at it with fresh eyes, I'd like to extract pieces of this PR to work on and review separately, which should be easier on me and reviewers, so will close this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Block] Image Affects the Image Block [Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants