Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve performance when content validation is disabled #178

Closed
jordanpadams opened this issue Jan 30, 2020 · 0 comments · Fixed by #181
Closed

Improve performance when content validation is disabled #178

jordanpadams opened this issue Jan 30, 2020 · 0 comments · Fixed by #181
Assignees
@hhlee445
Labels
bug Something isn't working
Milestone
PDS.10 (ends 2020...

Comments

@jordanpadams
Copy link
Member

jordanpadams commented Jan 30, 2020 •
edited
Loading

Using some test data from https://pds-geosciences.wustl.edu/lunar/urn-nasa-pds-kaguya_grs_spectra/ (already uploaded to internal server), it looks like validate is still affected by the size of products even when content validation is disabled.

For instance, if I execute validate against kgrs_calibrated_spectra_per1.xml with content validation disabled, it takes ~30 seconds to execute. but if I move kgrs_calibrated_spectra_per1.csv out of the way, and just touch a kgrs_calibrated_spectra_per1.csv file so its there, it executes in 3 seconds.
@jordanpadams jordanpadams added enhancement New feature or request high labels Jan 30, 2020
@jordanpadams jordanpadams added this to the PDS.10 (ends 2020-02-12) milestone Jan 30, 2020
@jordanpadams jordanpadams added bug Something isn't working and removed enhancement New feature or request labels Jan 30, 2020
jordanpadams added a commit that referenced this issue Jan 30, 2020
@jordanpadams
Remove md5 generation from validation when not possible
0e12810 
Original implementation generated MD5 checksum for all products, even if there was no checksum validation
possible in the validation run (e.g. no checksum in label and no checksum manifest provided).

Resolves #178
@jordanpadams jordanpadams mentioned this issue Jan 30, 2020
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hhlee445 hhlee445

Labels
bug Something isn't working
Projects
None yet
Milestone
PDS.10 (ends 2020-02-12)
Development

Successfully merging a pull request may close this issue.

Fixes multi-threading and MD5 checksum generation bugs NASA-PDS/validate
2 participants
@jordanpadams @hhlee445