Skip to content

v0.20.0-alpha1

Pre-release
Pre-release

Choose a tag to compare

View all tags
@rtibbles rtibbles released this 12 Aug 22:21
· 1540 commits to develop since this release
v0.20.0-alpha1
428d58d
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

QA team review guidance

Part 1 — "happy path"

Basic functionality

1. Learner -- from course assignment through finishing a unit

Before the course starts:

  • As the learner, open the newly assigned course from the home page — confirm you land on a welcome page showing the course thumbnail, title, and a description
  • Confirm the welcome page lists every unit, with each unit's lessons and its pre-test/post-test shown
  • Confirm the main action button reads "Start course" and is disabled, since no pre-test is open yet

Taking the pre-test:

  • As the coach, start the first unit's pre-test
  • As the learner, confirm the "Start course" button is now enabled — click it and confirm you land directly in the pre-test, not other course content
  • Answer all questions and submit — confirm submission succeeds
  • After submitting, confirm you see a "Pre-test completed" screen telling you that you'll continue once the coach closes the test

Working through the unit's lessons:

  • As the coach, close the pre-test
  • As the learner, confirm the unit's first lesson and its resources are now unlocked and clickable in the side panel
  • Open the side panel — confirm your current resource is visually marked as such (e.g. highlighted/bordered), resources you haven't reached yet look locked and can't be clicked, and resources you can reach but haven't started show a distinct "not started" look, different from ones that are in progress or completed
  • Open a video resource — confirm it plays normally, the same as it would outside a course
  • Finish the video and confirm the resource updates to a "completed" state
  • Confirm the bottom Prev/Next bar shows your position (e.g. "resource 1 of 3") and clicking "Next" moves you to the next resource
  • Open an article or PDF resource — confirm it displays normally and behaves the same as it would outside a course
  • Open an exercise resource — confirm it behaves the same as a standalone exercise (questions, feedback, mastery tracking)
  • On any resource you haven't finished, use the "mark as complete" checkmark control next to it in the side panel — confirm it immediately marks that resource done and lets you move on
  • Finish every resource in the first lesson — confirm you move into the next lesson's first resource, with no interstitial screen in between
  • Finish the very last resource in the unit — confirm you see a "Unit completed" screen explaining you can still go back and review resources until the next unit opens
  • From that screen, confirm you can go back (Prev) and re-open already-completed resources, but there's nothing further to advance to yet

Taking the post-test and finishing the unit:

  • As the coach, start the unit's post-test
  • As the learner, confirm you can now enter the post-test the same way you entered the pre-test
  • Answer all questions and submit — confirm submission succeeds
  • Confirm you see a "Post-test completed" screen telling you that you'll continue once the coach closes the test
  • As the coach, close the post-test (and, if there's a next unit, start its pre-test)
  • As the learner, confirm a "next unit" prompt becomes available and takes you into the next unit — or, if this was the last unit, confirm the welcome page now shows the course as finished, with the main button disabled

Related issues: #13437, #14063, #14064, #14104, #14105, #14213, #14134, #15178

2. Coach happy path: assigning and running a course

Assigning the course:

  • As the coach, go to the Courses page — confirm the list of available courses loads
  • Select a course to assign — confirm a side panel opens showing a condensed preview: units, lessons, and pre/post-tests, with resource counts
  • The "course preview" should allow navigation through all folders, down to the resource preview level, by clicking on the content cards or by using the breadcrumbs to return. No content should be "selectable" (in the way you can select quiz questions, for example) -- it is preview only
  • Choose recipients (e.g. the whole class) and confirm the assignment — confirm it completes without error and the course now appears in the coach's Courses list
  • As a learner in that class, confirm the course now appears on their home page

Running the first unit:

  • Open the course summary/unit view for the newly assigned course — confirm the first unit shows a control to start its pre-test
  • Start the pre-test — confirm a confirmation dialog appears before it actually starts, and once confirmed, the control now reads something like "End pre-test"
  • While the pre-test is open, have one learner (not all) complete it, then confirm the summary page's "learners completed" count reflects that — specifically, that it changes as learners actually finish rather than showing a fixed number regardless of real progress (this has been a hardcoded placeholder in at least one past build — worth confirming it's live data now)
  • End the pre-test — confirm a confirmation dialog appears (mentioning that learners who haven't finished will be marked incomplete), and once confirmed, the control now allows starting the post-test
  • While learners work through the unit's lessons, confirm the course summary/unit view shows their per-lesson, per-resource completion status, and that it updates as learners make progress without needing a manual page refresh
  • Once ready, start the post-test — confirm the same confirmation-dialog behavior as the pre-test
  • End the post-test — confirm the unit now shows as complete, and (if there's a next unit) the next unit's pre-test can be started

Checking reports:

  • Open the unit's Learning Objectives report tab — confirm it shows results based on the pre-test and post-test the class just took
  • Open the Learners tab — confirm it lists the class's learners, each with a performance indicator (e.g. "support needed" vs. "on track") based on the tests just taken
  • Open a learner's detail side panel — confirm it shows that individual learner's pre-test/post-test results

Related issues: #13435, #13436, #13682, #14099, #14100, #14101

QA Guidance - areas with previous bugs/complications

This list was put together with Claude as a first round of places to confirm the functionality of, most of which had at least some bugs/multiple review rounds and testing (and some on the fly decisions) in the implementing PRs. Some of this is a bit redundant with the above, but it more of expecting some edge cases and what we think might go wrong that deserves confirmation. I defer to QA team if you would like to copy paste all of this into a separate document and reorganize in any way that is more helpful for your organization and process!

1. Test gating (pre-tests and post-tests)

What this is: While learners work through a course, a coach controls when each unit's pre-test and post-test are open. Learners should only be able to enter a test while it's open, and shouldn't be able to reach a unit's lessons until that unit's pre-test is done.

Setup: A coach account and a learner account already assigned to the same course — use two browser sessions
(or one regular + one incognito window) so you can act as both at once.

Starting a unit's pre-test:

  • As the learner, open a newly assigned course before any pre-test has been started — confirm the "Start" button is disabled
  • As the coach, start the first unit's pre-test — then, as the learner, reload the page and confirm the "Start" button is now enabled (whether it updates without reloading is tested separately below)
  • As the learner, click "Start" — confirm you land directly in the pre-test, not other course content

Locked lessons and tests:

  • Paste a locked lesson's link directly into the address bar — confirm you're redirected back to your current valid position in the course (not shown the lesson content, not left on a blank/broken page)
  • After the coach ends the pre-test, reload the learner's page and confirm the unit's lessons become unlocked
  • Confirm the post-test stays locked until the coach explicitly starts it, even after all lessons in the unit are complete

While a test is open:

  • Confirm "Previous"/"Next" only move between the test's own questions
  • Submit the test, then try two ways to get back into it — the browser's back button, and a direct link to the test's own URL — confirm neither lets you re-enter it or change your answers

Coach changes test state while the learner's course is already open:

  • As the learner, open the course and leave the tab open without navigating anywhere else
  • As the coach, start a pre-test for that unit
  • Back in the learner's already-open tab, without reloading the page, try to start the test — confirm it becomes available on its own, with no manual reload needed
  • Repeat for the coach ending a test while the learner's tab is already open — confirm the learner loses access on their own, with no manual reload needed
  • If anything is unclear, inconsistent, or doesn't match the pass criteria above — record a screencast and share on Slack, tagging Marcella and Richard for further discussion

Moving between units:

  • Complete a unit's post-test — confirm the next unit's pre-test becomes available under the same locked/unlocked rules as unit one
  • Paste a direct link to a resource in a future, not-yet-reached unit — confirm you're redirected back to your current valid position in the course, rather than shown the content

Related issues: #14213, #14134, #13436, #14041

2. Learning objective scoring in reports

What this is: Coaches can see how learners are doing against each unit's learning objectives, based on pre-test and post-test results — a distribution chart per learning objective in the report, and per-learner scores in side panels.

Setup: A completed test where at least one question is tagged to more than one learning objective. (these should be updated after Richard sorted out a previous bug)

Checklist:

  • Open the unit's Learning Objectives report — confirm the shared question's result shows under every learning objective it's tagged to, not just the first one listed
  • Confirm each of those learning objectives' per-learner side panel shows this learner's correct/incorrect result for that same question
  • Confirm each learning objective's "out of X questions" count matches the total number of questions associated with that, including ones shared with other learning objectives
  • With a learner who has completed only the pre-test: confirm the report shows pre-test results and clearly marks the post-test as not yet done (not blank, not scored as zero or incorrect)
  • With a learner who has completed only the post-test: confirm the same in reverse — post-test results shown, pre-test clearly marked as not done
  • With a learner who has completed both: confirm both sets of results are shown and clearly distinguishable from each other
  • With a learner who has completed neither: confirm the report shows a clear "no data yet" state, not a blank space or an error

Related issues: #14470, #13682, #14167, #14169

3. Syncing courses on learner-only devices

What this is: On learner-only devices, assigning a course should download its content in a prioritized order, and clean it up if the course is unassigned.

Setup: A learner-only device setup and enrolled in a class

Checklist:

  • Assign a course — confirm content download begins, visible in the device's task/download manager
  • Right after assigning, confirm the learner's device is syncing, that the cousre appears in the learner's home page and that the active test becomes available (if it is active/started) it should become available before the entire course resources have finish downloading
  • Unassign the course — confirm its downloaded content is removed from the device (check device storage or the content list)
  • Update the source channel (e.g. publish a new version) after a course has already been assigned and downloaded — confirm the device re-downloads the update and the learner sees the change (e.g. an updated title or resource), without the course needing to be removed and reassigned

Related issues: #14282, #13970 #15157

4. Content deleted or missing while a course is assigned

What this is: A course, or a resource inside it, may be removed from the library after being assigned to a class. This is an edge case for now -- we are just confirming there is some error management.

Setup: A course assigned to a class and in progress, then the course is deleted from the device the assignment still exists.

Checklist:

  • Delete a resource that belongs to an assigned, in-progress course — confirm a clear "missing" indicator appears for it, rather than it being listed as if nothing changed (note, we will probably have to figure out a QA option for this that might need a bit of coordination, so okay to wait on this for now)
  • Delete the entire course or channel while a class is actively assigned and mid-course — confirm it's still listed correctly (not vanished, not duplicated) in the device's content management view, clearly marked as removed or unavailable, and that opening other, unrelated courses or content still works normally afterward

Related issues: #14142, #14138, #14135

Full list of PRs since 0.20.0-alpha0

What's Changed

Dependencies

Full Changelog: v0.20.0-alpha0...v0.20.0-alpha1

Contributors

bjester, rtibbles, and 5 other contributors
Assets 9
Loading