Skip to content

[DOCS-1254] Update docs for Run Comparer#1232

Merged
mdlinville merged 9 commits into
mainfrom
DOCS-1254
Apr 15, 2025
Merged

[DOCS-1254] Update docs for Run Comparer#1232
mdlinville merged 9 commits into
mainfrom
DOCS-1254

Conversation

@mdlinville
Copy link
Copy Markdown
Contributor

@mdlinville mdlinville commented Apr 2, 2025

DOCS-1254 Update docs for Run Comparer

  • Give more details about choosing what to compare
  • Give more details about what Diff Only is about
  • You can now filter values by key
  • You can now copy any value
  • You can now adjust the display density
  • Remove the outdated image

Ready for technical review.

Preview: https://docs-1254.docodile.pages.dev/guides/app/features/panels/run-comparer/

@mdlinville mdlinville requested a review from a team as a code owner April 2, 2025 22:33
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 2, 2025

Images automagically compressed by Calibre's image-actions

Compression reduced images by 36.8%, saving 12.36 KB.

Filename Before After Improvement Visual comparison
assets/images/general/run-comparer.png 33.60 KB 21.24 KB -36.8% View diff

437 images did not require optimisation.

@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages Bot commented Apr 2, 2025

Deploying docs with  Cloudflare Pages  Cloudflare Pages

Latest commit: 8cce94f
Status: ✅  Deploy successful!
Preview URL: https://ebe52bf8.docodile.pages.dev
Branch Preview URL: https://docs-1254.docodile.pages.dev

View logs

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 2, 2025

Images automagically compressed by Calibre's image-actions

Compression reduced images by 36.8%, saving 12.36 KB.

Filename Before After Improvement Visual comparison
assets/images/general/run-comparer.png 33.60 KB 21.24 KB -36.8% View diff

437 images did not require optimisation.

@ericakdiaz
Copy link
Copy Markdown

updated content looks good to me! i clicked on the preview link though, and saw the old gif, and couldn't see the new image - is that expected? i haven't used this docs preview before 😅
image

@mdlinville
Copy link
Copy Markdown
Contributor Author

updated content looks good to me! i clicked on the preview link though, and saw the old gif, and couldn't see the new image - is that expected? i haven't used this docs preview before 😅 image

I believe I may have fixed this now. What is happening is that in the preview, it is using docs.wandb.ai as the base URL for the image, instead of / inside the PR. (this would not look broken if the image already existed in production) We have noticed this before but haven't fixed it. Fingers crossed.

@mdlinville mdlinville requested a review from ericakdiaz April 2, 2025 23:08
Copy link
Copy Markdown

@ericakdiaz ericakdiaz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good now! thanks for fixing :)

@mdlinville
Copy link
Copy Markdown
Contributor Author

mdlinville commented Apr 2, 2025

@johndmulhausen Can you help figure out why the image still is not working in the preview? It is pointing to docs.wandb.ai instead of https://docs-1254.docodile.pages.dev/images/general/run-comparer.png (which does work). It just means that the PR preview doesn't show the actual image. I tried updating the baseURL, which you suggested last time we ran into this, but doesn't seem to be working here.

- Update Hugo config to preview images in PRs correctly
- To copy any configuration or metric's value, hover your mouse over the value, then click the copy button. The entire value is copied, even if it is too long to display on the screen.

{{% alert %}}
Run Comparer does not differentiate runs by type. This means that it is possible to compare runs that are not really comparable, like comparing an image run to an audio run. Search, filter, group, or sort the list of runs to limit it to the runs you want to analyze.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Runs don't have "types". You can, however, specify a "job_type" when you create a run. See "job_type" in https://docs.wandb.ai/ref/python/init/

{{% /alert %}}

Toggle the **diff only** option to hide rows where the values are the same across runs.​​
This image compares three image runs with **Diff only** enabled, and filters for configurations that match the string `run`.
Copy link
Copy Markdown
Contributor

@ngrayluna ngrayluna Apr 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should show this image, example. I think this confuses run names with filtering runs by what they do. For a couple of reasons:

  • Run names are normally used for describing the step in the ML. E.g. "training-image
  • Normally you wouldn't create runs with the same name. At a minimum, you'd append the run.id to differentiate between runs.
  • You can sort runs based on the "job_type" specified when you create a run.

Comment thread hugo.yaml
@@ -1,4 +1,4 @@
baseURL: https://docs.wandb.ai
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please move this to a separate PR for review.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was in this PR to work around the broken image in the PR preview. If I were not removing the image based on your feedback, it would make sense to keep it together.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to #1244

Comment thread scripts/production.sh Outdated
hugo mod get -u
rm -rf public
hugo
hugo -b $CF_PAGES_URL
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please move this to a separate PR for review.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto here, I'll make that change since I am also removing the image here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to #1244

Comment thread content/guides/models/app/features/panels/run-comparer.md Outdated
@mdlinville mdlinville requested a review from ngrayluna April 9, 2025 18:02
Co-authored-by: Noah Luna <15202580+ngrayluna@users.noreply.github.com>
@mdlinville mdlinville enabled auto-merge (squash) April 11, 2025 22:59
Comment thread content/guides/models/app/features/panels/run-comparer.md Outdated
@mdlinville mdlinville merged commit 7025ec0 into main Apr 15, 2025
@mdlinville mdlinville deleted the DOCS-1254 branch April 15, 2025 20:29
mdlinville added a commit that referenced this pull request Apr 18, 2025
Moved from #1232 

This change updates production.sh to call hugo
differently in Cloudflare environments than otherwise.
- In Cloudflare, the variable $CF_PAGES_URL tracks the
  base URL of a PR build or the production deploy and needs
  to be part of the hugo command.
- Locally, this variable is not set so don't try to use it.

To help with debugging, log the environment we are building in.

## Tests
- [x] Local build works, image shows up correctly
- [x] PR build works, image shows up correctly
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants