docs: Add Databricks integration documentation#1301
Conversation
Add documentation for the new Databricks integration that enables linking ValidMind inventory records to Databricks Unity Catalog resources including models, datasets, and agents. - Add Databricks to supported connections in configure-connections.qmd - Add Databricks to model inventory integrations in managing-integrations.qmd - Add step-by-step connection and linking examples in integrations-examples.qmd Closes sc-14813
|
Pull requests must include at least one of the required labels: |
- databricks host → workspace url - client id → sql warehouse id - client secret → personal access token
- client id: Update to describe SQL Warehouse ID - client secret: Update to describe PAT token secret
|
@nateshim thank you for reviewing! I've addressed your comments, could you approve, please? |
|
Reminder for Nik: @nateshim has a very nice demo video that shows how to run a databricks notebook in validmind/validmind-library#495. Might be good to borrow a bit from that for the docs. |
Add a new section explaining how to run validation notebooks against Databricks-hosted data, with a link to the quickstart notebook and a mermaid diagram showing the data flow between platforms.
Lighthouse check resultsShow Lighthouse scoresFolder depth level checked: 0 Commit SHA: 1a58d73 Modify the workflow to check a different depth:
|
PR SummaryThis PR updates the integration documentation and examples by enhancing the sidebar structure and adding detailed integration guides. In the sidebar configuration file, the single reference to the integrations examples page has been replaced by a structured file reference that includes multiple subpages. These subpages offer step-by-step instructions for setting up and linking various integrations such as Jira, ServiceNow, AWS SageMaker, AWS Bedrock, GitLab, and Databricks, as well as using webhooks with workflows. Key functional changes include:
Overall, these changes are aimed at improving the clarity, usability, and navigability of the integration documentation, while also providing enriched examples for practical setup scenarios. Test Suggestions
|
Validate docs site✓ INFO: A live preview of the docs site is available — Open the preview |
* docs: Add Databricks Unity Catalog integration documentation Add documentation for the new Databricks integration that enables linking ValidMind inventory records to Databricks Unity Catalog resources including models, datasets, and agents. - Add Databricks to supported connections in configure-connections.qmd - Add Databricks to model inventory integrations in managing-integrations.qmd - Add step-by-step connection and linking examples in integrations-examples.qmd Closes sc-14813 * Add placeholder columns to integrations grid layout * Fix Databricks connection field names to match UI - databricks host → workspace url - client id → sql warehouse id - client secret → personal access token * Fix Databricks link model steps to match UI * docs: Fix Databricks connection field descriptions - client id: Update to describe SQL Warehouse ID - client secret: Update to describe PAT token secret * docs: Add developer workflow for running validations on Databricks data Add a new section explaining how to run validation notebooks against Databricks-hosted data, with a link to the quickstart notebook and a mermaid diagram showing the data flow between platforms. * Split examples up into separate files. * File rename * Remove terminology ambiguity * Edits * Add missing headings for steps * Add configuration screenshot
* docs: Map and assess evidence (#1299) * docs: Add autogenerate validation reports documentation Document the AI-powered features for validators: - Auto-Link Evidence for single/bulk guideline evidence linking - Risk Assessment Generation for single/bulk assessments - Understanding the structured AI-generated assessment format Closes sc-14812. * fix: Rename to map-and-assess-evidence with accurate UI terminology - Rename file from autogenerate-validation-reports.qmd to map-and-assess-evidence.qmd - Update title to "Map and assess evidence" - Fix UI element names to match actual product (Map Evidence, Assess Evidence) - Describe Evidence Assessment output structure accurately - Add relevance threshold slider documentation * Add screenshots for Map Evidence and Assess Evidence panels - Add map-evidence-panel.png showing evidence type toggles and relevance threshold - Add assess-evidence-panel.png showing compliance assessment option - Display both in column-margin divs alongside their respective sections * fix: Add Approve All / Reject All workflow for Map and Assess Evidence - Update Map Evidence steps to describe approve/reject workflow - Update Assess Evidence steps to describe approve/reject workflow - Clarify that editing is available after approving assessments * Add individual approve/reject, relevance analysis, and reassess options - Add individual Approve/Reject buttons for mapped evidence - Add See Relevance Analysis option with relevance score explanation - Add individual Approve/Reject buttons for evidence assessments - Add Reassess Evidence option to regenerate assessments - Add new "Understand mapped evidence" section explaining relevance scores and analysis * Distinguish report-level vs section-level approve/reject workflows - Report overview: Approve All / Reject All for entire report - Section-level: Individual Approve / Reject per evidence item or assessment - Clarify the two-level review structure for both Map Evidence and Assess Evidence * Consolidate intro into benefits-focused overview section Replace detailed "Understand mapped evidence" and "Understand evidence assessments" sections with a concise "How do evidence mapping and assessment work?" section that explains the benefits for new users. * Format overview section as definition list * Remove What's next section * Remove unused evidence-assessment-example.png * Add What's next section with listing to assess-compliance Links to the manual compliance assessment workflow for users who prefer traditional evidence linking and assessment selection. * Document Remap Evidence button in report overview * Replace 'report overview' with visible UI terminology Changed 'From the report overview' to 'For the entire report' and removed references to non-existent 'Overview page' label. * Simplify navigation: click Validation link directly * Address review comments from Fernanda * Fix link warning * docs: Add Databricks integration documentation (#1301) (#1316) * docs: Add Databricks Unity Catalog integration documentation Add documentation for the new Databricks integration that enables linking ValidMind inventory records to Databricks Unity Catalog resources including models, datasets, and agents. - Add Databricks to supported connections in configure-connections.qmd - Add Databricks to model inventory integrations in managing-integrations.qmd - Add step-by-step connection and linking examples in integrations-examples.qmd Closes sc-14813 * Add placeholder columns to integrations grid layout * Fix Databricks connection field names to match UI - databricks host → workspace url - client id → sql warehouse id - client secret → personal access token * Fix Databricks link model steps to match UI * docs: Fix Databricks connection field descriptions - client id: Update to describe SQL Warehouse ID - client secret: Update to describe PAT token secret * docs: Add developer workflow for running validations on Databricks data Add a new section explaining how to run validation notebooks against Databricks-hosted data, with a link to the quickstart notebook and a mermaid diagram showing the data flow between platforms. * Split examples up into separate files. * File rename * Remove terminology ambiguity * Edits * Add missing headings for steps * Add configuration screenshot --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
What and why?
This PR:
How to test
Try the live preview:
What needs special review?
These docs mention linking to models, datasets, and agents in Databricks Unity Catalog, but I only see Link Model in the UI. Is Dataset/Agent linking planned for this release, or should I update the docs to clarify that only model linking is currently supported?
I figured out these steps by inspecting the code changes and testing in the UI where I could, but I can't actually set up a real Databricks connection. If you have some credentials for testing, I'd be grateful (or please review the steps here carefully!).
Dependencies, breaking changes, and deployment notes
Depends on validmind/validmind-library#495 for the Databricks notebook link to work.
Release notes
Added documentation for configuring the Databricks Unity Catalog integration, including connection setup and linking models.
Checklist