QA Coverage

[phyllis-sy-wu]

This is a plan to automate the manual QA flow.

QA step → test mapping

For a more user-friendly view, please see this google sheet.

QA Step for Apps	A 🟢	B 🔵	C 🟣	D 🟠	E 🟡	F ⚪️
Create app	🟢	🔵	🟣	🟠	🟡	⚪️
Add different extensions			🟣
Generate admin_action extension				🟠
Generate theme_app_extension					🟡
Generate discount extension						⚪️
Generate flow_action extension			🟣
Generate 3 other random extensions			🟣
Dev Console		🔵	🟣	🟠	🟡	⚪️
Dev server starts		🔵	🟣	🟠	🟡	⚪️
Open the shop and see the dev console		🔵	🟣	🟠	🟡	⚪️
App shows as connected (green connected icon)		🔵	🟣	🟠	🟡	⚪️
Admin action				🟠
Confirm/define products				🟠
Open link on the admin-action extension ➔ Navigate to product admin page and open the action modal				🟠
Hot reload admin action file (src/ActionExtension.js)				🟠
Add another random extension, see in Dev Console			🟣
Verify the newly added extension shows in Dev Console			🟣
GraphQL		🔵
Open GraphiQL		🔵
Test GraphQL query in GraphiQL		🔵
GraphQL via app execute		🔵
Theme app extension					🟡
Click "Setup theme app extension" link					🟡
Add section, choose app and Save					🟡
Open theme app extension local preview					🟡
Hot reload liquid file (blocks/star_rating.liquid)					🟡
Press q to stop dev		🔵	🟣	🟠	🟡	⚪️
Dev Console: shows disconnected		🔵	🟣	🟠	🟡	⚪️
shopify app dev clean		🔵	🟣	🟠	🟡	⚪️
Dev preview hidden		🔵	🟣	🟠	🟡	⚪️
Test function extension						⚪️
Move to the function directory						⚪️
Function extension: build						⚪️
Function extension: run with test input						⚪️
Return to root of the app	🟢	🔵				⚪️
Deploy app	🟢	🔵
List versions	🟢
Config link (new app)	🟢
Deploy to new app	🟢

Test breakdown

Test	Sub-issue	File	Scope	Status
A	#7369	Extend app-deploy.spec.ts	init → deploy → versions list → config link (new app) → deploy to new app	Unblocked
B	#7370	Extend app-dev-basic.spec.ts	init → dev → Dev Console green → g → GraphiQL query { shop { name } } → result → execute → q → disconnected → clean → hidden → deploy	Unblocked
C	#7371	New app-ext-basic.spec.ts	init → generate 4 (flow_action + 3 from SAFE_FOR_C_EXTRAS) → dev → Console lists all 4 → generate 5th mid-dev via CLI → Console updates → q/clean	Blocked on 401
D	#7373	New app-ext-admin-action.spec.ts	init → generate admin_action → dev → Console → product action modal → edit src/ActionExtension.js → hot reload → q/clean	Blocked on 401
E	#7374	New app-ext-theme-app.spec.ts	init → generate theme_app_extension → dev → setup theme link → add section + save → preview → edit blocks/star_rating.liquid → hot reload → q/clean	Blocked on 401
F	#7375	New app-ext-function.spec.ts	init → generate discount → function build → function run with test input → return to root	Blocked on 401

Hydrogen QA is tracked separately in #7376.