Skip to content

docs: Updated evidence & artifact linking on validation reports#1343

Merged
validbeck merged 38 commits into
mainfrom
beck/sc-15648/documentation-modular-document-checker
May 22, 2026
Merged

docs: Updated evidence & artifact linking on validation reports#1343
validbeck merged 38 commits into
mainfrom
beck/sc-15648/documentation-modular-document-checker

Conversation

@validbeck
Copy link
Copy Markdown
Collaborator

@validbeck validbeck commented May 22, 2026

Pull Request Description

What and why?

sc-15648

Doc updates for the ability to pull from several document evidence and minor artifacts UI adjustment.

How to test

Click on the live previews:

Branch change summary

Excludes include partials, notebooks, images, zips, and generated site output.

Guides

OLD NEW
Map and assess evidence Map and assess evidence
Assess compliance Assess compliance
Add and manage artifacts Add and manage artifacts (Review all the content in the sections: Refine validation report, Manage artifacts, Submit report for approval)

Training

OLD NEW
Running Data Quality Tests Running Data Quality Tests (Click thru to the next slide.)
Developing Challenger Models Developing Potential Challengers (Click thru to the next slide.)
Finalizing Validation Reports Finalizing Validation Reports (Review the slides under the sections: Refine validation report, Manage artifacts, Submit report for approval)

What needs special review?

n/a

Dependencies, breaking changes, and deployment notes

Related updates in the Validator Fundamentals series of notebooks updated in validmind/validmind-library#518.

Release notes

Covered by:

Checklist

  • What and why
  • Screenshots or videos (Frontend)
  • How to test
  • What needs special review
  • Dependencies, breaking changes, and deployment notes
  • Labels applied
  • PR linked to Shortcut
  • Unit tests added (Backend)
  • Tested locally
  • Documentation updated (if required)
  • Environment variable additions/changes documented (if required)

@validbeck validbeck self-assigned this May 22, 2026
@validbeck validbeck added the internal Not to be externalized in the release notes label May 22, 2026
@validbeck validbeck requested review from jamadriz and nrichers May 22, 2026 02:49
Copy link
Copy Markdown

@jamadriz jamadriz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Image

@github-actions
Copy link
Copy Markdown
Contributor

PR Summary

This PR refines many of the tutorial and documentation files by updating the step-by-step instructions for linking evidence, mapping evidence to guidelines, adding and managing artifacts, and assigning compliance assessments. Key changes include:

  • Improved guidance for validator and developer evidence workflows: Steps have been renumbered and clarified to reflect updated UI terminology—e.g., replacing links to the separate validator evidence file with inline instructions, clarifying which panel to expand, and updating the button labels (e.g., changing from “Link Evidence to Report” to explicitly using the new Link Evidence button and selecting the appropriate evidence type).

  • Revised instructions for artifact management: The steps for adding validation issues (artifacts) have been updated to make the workflow more intuitive. The text now clearly specifies when to click the Add Artifact or Suggest Validation Issues buttons and how to confirm that the artifact is correctly linked.

  • Updates in compliance assessment instructions: The process for mapping and assessing evidence (AI-assisted) has been adjusted. The revised text now instructs users to click on the appropriate section (ex: 2.2.1. Data Quality or 2.2.2. Model Performance) and guides them through expanding evidence panels and approving generated suggestions.

  • Notebook metadata consistency: Several notebook files have been updated to include explicit settings for execution count and outputs (using null and empty output arrays), ensuring consistency across examples.

Overall, these changes enhance clarity, improve workflow coherence in the training materials and user guides, and ensure that the documentation remains in sync with the latest UI and backend updates.

Test Suggestions

  • Perform an end-to-end walkthrough using a sample model to verify that evidence mapping, linking, and artifact submission work as described in the updated instructions.
  • Test both developer and validator evidence workflows to ensure that the correct evidence is mapped and that the guidance text correctly reflects the UI options.
  • Run the modified notebooks to confirm that the execution metadata (execution_count and outputs) no longer cause errors and display as expected.
  • Conduct user acceptance testing on the updated training materials to verify that instructions are clear and actions trigger the correct functionality.

@validbeck
Copy link
Copy Markdown
Collaborator Author

@jamadriz Thanks for the review, addressed:

Not issues

1.

I'm going to make the executive decision as the human writer to keep the "Link evidence to reports" top-level header, as that is a parent topic that covers the two steps of linking artifacts: mapping and then assessing.

2.

This has been addressed in validmind/validmind-library#518, and the changes already pulled in — notebooks are not updated from the docs but from the validmind-library repo. Not sure why you're still seeing the issue, is your branch stale?

Screenshot 2026-05-22 at 10 21 41 AM

Fixed

4.

Updated in commit #43b7bb9

  • Only tests logged in content can be selected
  • Developer Evidence: Selecting content blocks and test results
Screenshot 2026-05-22 at 10 20 35 AM

Already addressed, not sure why the agent is confused

  • UI lists where each evidence item comes from —

    developer evidence validator evidence
    Screenshot 2026-05-22 at 10 12 38 AM Screenshot 2026-05-22 at 10 12 47 AM

Not relevant

  • PRT document-type filtering — This isn't at the document/evidence picker level, this is managed under Settings by the admin that sets up the document types and document type defaults etc. and is already covered under Manage document types

5.

  • Typo in quickstart-validator.qmd fixed in commit #9909ae2
  • Work with content blocks has the updated screenshot, just a minor update if you wanted to confirm (I didn't include in the PR comment b/c it was minor, just forgot to add the list item for referencing document sections in a previous PR alongside the updated content bar capture).
  • _assign-compliance-assessments.qmdExecutive decision to defer the screenshot; it's been outdated for a while in terms of styling but the gist of it is the same. We are trying to minimize the "low-level" edits that take way too much time esp. as now there is just me doing docs.

@validbeck validbeck requested a review from jamadriz May 22, 2026 17:23
@github-actions
Copy link
Copy Markdown
Contributor

Validate docs site

✓ INFO: A live preview of the docs site is available — Open the preview

@github-actions
Copy link
Copy Markdown
Contributor

Execute training notebooks for PRs

✓ INFO: Live previews are available —

@github-actions
Copy link
Copy Markdown
Contributor

Lighthouse check results

⚠️ WARN: Average accessibility score is 0.87 (required: >0.9) — Check the workflow run

Show Lighthouse scores

Folder depth level checked: 0

Commit SHA: 28c6ca9

Modify the workflow to check a different depth:

  • 0: Top-level navigation only — /index.html, /guide/guides.html, ...
  • 1: All first-level subdirectories — /guide/.html, /developer/.html, ...
  • 2: All second-level subdirectories — /guide/attestation/*.html, ...
Page Accessibility Performance Best Practices SEO
/developer/validmind-library.html 0.85 0.47 1.00 0.82
/get-started/get-started.html 0.85 0.70 1.00 0.73
/guide/guides.html 0.85 0.69 1.00 0.82
/index.html 0.93 0.68 1.00 0.82
/releases/all-releases.html 0.86 0.68 1.00 0.73
/support/support.html 0.91 0.69 1.00 0.82
/training/training.html 0.85 0.63 0.96 0.73

@validbeck validbeck merged commit f4ea95c into main May 22, 2026
8 checks passed
@validbeck validbeck deleted the beck/sc-15648/documentation-modular-document-checker branch May 22, 2026 20:18
@validbeck validbeck mentioned this pull request May 26, 2026
11 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

internal Not to be externalized in the release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants