Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add table showing artifacts disk space consumption to workflow view #7152

Merged
merged 5 commits into from
Jun 29, 2023

Conversation

MichaelBuessemeyer
Copy link
Contributor

@MichaelBuessemeyer MichaelBuessemeyer commented Jun 16, 2023

This PR adds a new table to the vx workflow view that enables users to identify the artifacts consuming the most disk space. The table is wrapped in a modal and can be shown by clicking the "Show Disk Usage of Artifacts" option in workflow view under the menu next to the "Collapse all button" in the header on the right side of the view.

URL of deployed dev instance (used for testing):

  • https://___.webknossos.xyz

Steps to test:

  • I suggest testing locally, as vx needs to be active on the wk instance.
  • Enable voxelytics in the application.conf
  • execute WK_URL=http://localhost:9000 WK_TOKEN=secretSampleUserToken vx voxelytics/connect/test/configs/segem_test_2d.yaml --cpu in the working directory of vx and wait for the command to finish
  • Go to http://localhost:9000/workflows and open the workflow.
  • In the "..." menu-button select the option "Show Disk Usage of Artifacts"
  • A modal should open up, that displays all artifacts with names, task name, artifact size, inode count, and path in table. Each column of the table should be sortable.
  • The table should also work with meta tasks

Issues:


(Please delete unneeded items, merge only when none are left open)

@MichaelBuessemeyer MichaelBuessemeyer marked this pull request as ready for review June 16, 2023 14:05
@MichaelBuessemeyer
Copy link
Contributor Author

I'd say this pr is ready for the first review (despite the open TODO).

@MichaelBuessemeyer
Copy link
Contributor Author

I honestly don't know who should review these changes. @philippotto in case you want/know someone else that could review the changes better, please reassign the review

Copy link
Member

@philippotto philippotto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! only the see my one comment :)

Comment on lines 41 to 42
artifactSize: formatCountToDataAmountUnit(artifact.fileSize),
fileSize: artifact.fileSize,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't this the same? the first entry is formatted and the second is not?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right: one is the formatted value and one is the unformatted raw value.
The unformatted raw value is used to be able to sort the artifact entries properly by their size.
The formatted value is simply the value that is displayed in the table.

I looked at the docs that list the render prop for the columns prop of a table. I updated the artifact entries to only have the file size and to format them in the render function.
This way is much better as the filesize is not contained in every entry twice. 👍

Thanks for mentioning this 🙏

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great 👍 did you push this already?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I got interrupted a little. It is pushed now :)

@MichaelBuessemeyer
Copy link
Contributor Author

@philippotto Do you think I should test the artifact size table with meta tasks? If yes, I somehow need a way to get a workflow with a meta task into webknossos.
Running the vx tests with the WK_URL and WK_TOKEN being set did not make the run workflows accessible to wk :/

@philippotto
Copy link
Member

@philippotto Do you think I should test the artifact size table with meta tasks? If yes, I somehow need a way to get a workflow with a meta task into webknossos. Running the vx tests with the WK_URL and WK_TOKEN being set did not make the run workflows accessible to wk :/

If it's too tedious to do the testing locally, I'm also fine with merging the PR and then directly having a look in production, since the chances are rather good that it just works ™️ :)

Copy link
Member

@philippotto philippotto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@MichaelBuessemeyer
Copy link
Contributor Author

MichaelBuessemeyer commented Jun 29, 2023

If it's too tedious to do the testing locally, I'm also fine with merging the PR and then directly having a look in production, since the chances are rather good that it just works tm :)

Ok, let's just :shipit: ™️ (after the lunch break I'll deploy the newer version)

@MichaelBuessemeyer MichaelBuessemeyer enabled auto-merge (squash) June 29, 2023 10:06
@MichaelBuessemeyer MichaelBuessemeyer merged commit ab9bf11 into master Jun 29, 2023
2 checks passed
@MichaelBuessemeyer MichaelBuessemeyer deleted the add-vx-artifacts-disk-usage-list branch June 29, 2023 10:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow to identify large artifacts in voxelytics reports
2 participants