Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

validate inexplicably writes to validate_stack_traces.log #344

Closed
rchenatjpl opened this issue May 15, 2021 · 7 comments · Fixed by #348 or #350
Closed

validate inexplicably writes to validate_stack_traces.log #344

rchenatjpl opened this issue May 15, 2021 · 7 comments · Fixed by #348 or #350
Assignees
Labels
bug Something isn't working i&t.issue s.low

Comments

@rchenatjpl
Copy link

validate passes the attached bundle but writes something to validate_stack_traces.log, which is worrisome. I assume the bundle has no bugs, but it would be nice to be sure. I guess this issue has a good chance of being a wontfix, but really, don't you want to clean up these exceptions?

% validate -t uranus_occ_u36_maunakea_380cm
PDS Validate Tool Report
Configuration:
Version 2.0.4
Date 2021-05-15T10:19:52Z
Parameters:
Targets [file:/Users/rchen/Desktop/uranus_occ_u36_maunakea_380cm/]
Severity Level WARNING
Recurse Directories true
File Filters Used [*.xml, *.XML]
Data Content Validation on
Product Level Validation on
Max Errors 100000
Registered Contexts File /Users/rchen/PDS4tools/validate/resources/registered_context_products.json
Product Level Validation Results
PASS: file:/Users/rchen/Desktop/uranus_occ_u36_maunakea_380cm/bundle.xml
1 product validation(s) completed
PASS: file:/Users/rchen/Desktop/uranus_occ_u36_maunakea_380cm/data/collection_data.xml
2 product validation(s) completed
PASS: file:/Users/rchen/Desktop/uranus_occ_u36_maunakea_380cm/data/u36_maunakea_380cm_2200nm_ring_beta_ingress_sqw.xml
3 product validation(s) completed
Summary:
0 error(s)
0 warning(s)
Product Validation Summary:
3 product(s) passed
0 product(s) failed
0 product(s) skipped
Referential Integrity Check Summary:
0 check(s) passed
0 check(s) failed
0 check(s) skipped
End of Report
Completed execution in 5319 ms

uranus_occ_u36_maunakea_380cm.zip
validate_stack_traces.log

@qchaupds
Copy link
Contributor

The issue was this line:

TableReader tmpReader = new TableReader(tableObjects.get(0), dataFile, false, false);

For some labels, there are no tableObjects so putting in a check of the size before accessing it resulted in no stack file printing.

qchaupds pushed a commit that referenced this issue May 19, 2021
…e checking for integer fields for DSV file

1. Remove double quotes before checking for valid pattern in FieldValueValidator.java
2. Add check for tableObjects size before accessing it to avoid index exception in TableDataContentValidationRule
3. Add new tests for github344 and github345 to validate.feature file

Refs:

#344 validate inexplicably writes to validate_stack_traces.log
#345 validate incorrectly flags integers bounded by "" in a .csv
qchaupds pushed a commit that referenced this issue May 24, 2021
…orrect file_size tags in two test labels

1. Add new label test artifact astro_sample_data_t.xml to src/test/resources/github345/ directory
2. Remove Utility.removeQuotes() from FieldValueValidator.java since the quotes have been removed before already and also the function was remove empty spaces which is not desirable
3. Add new test to src/test/resources/features/validate.feature file
4. Correct the file_size tag from 132 to 164 to the actual value in src/test/resources/github09/val9b.xml
5. Correct the file_size tag from 101 to 64 to the actual value in src/test/resources/github137/delimited_table_bad.xml

Refs:

#344 validate inexplicably writes to validate_stack_traces.log
#345 validate incorrectly flags integers bounded by "" in a .csv
jordanpadams pushed a commit that referenced this issue May 24, 2021
…orrect file_size tags in two test labels

1. Add new label test artifact astro_sample_data_t.xml to src/test/resources/github345/ directory
2. Remove Utility.removeQuotes() from FieldValueValidator.java since the quotes have been removed before already and also the function was remove empty spaces which is not desirable
3. Add new test to src/test/resources/features/validate.feature file
4. Correct the file_size tag from 132 to 164 to the actual value in src/test/resources/github09/val9b.xml
5. Correct the file_size tag from 101 to 64 to the actual value in src/test/resources/github137/delimited_table_bad.xml

Refs:

#344 validate inexplicably writes to validate_stack_traces.log
#345 validate incorrectly flags integers bounded by "" in a .csv
@msbentley
Copy link

HI @jordanpadams et al. Just to confirm, under what conditions should we expect a stack trace log? With 2.0.7 we are still seeing this in cases where we would not expect it...

@jordanpadams
Copy link
Member

@msbentley you should only see it when there is a fatal error if it is still appearing at other times, we will do a full scrub of the software to remove times when it is created unexpectedly

@msbentley
Copy link

OK, thanks @jordanpadams - let me know if you need some examples!

@qchaupds
Copy link
Contributor

qchaupds commented Aug 2, 2021

@msbentley @jordanpadams Yes, having an example would be nice to see where the code is writing to a stack trace log when it shouldn't.
I am combing through the code right now but haven't been able to pin-point what or where the trace log shouldn't be writing.

@msbentley
Copy link

msbentley commented Aug 3, 2021

This may be related to #379 but one other case I see is when the data file cannot be found (e.g. it is missing, or the filename in the label is wrong), e.g.

ERROR  [error.table.bad_file_read]   table 1: Error occurred while trying to read table: /home/mbentley/Dropbox/work/bepi/archive/test_products/minimal_test_product.csv (No such file or directory)

This generates validate_stack_traces.log and validate_stack_traces.log.temp

@qchaupds
Copy link
Contributor

qchaupds commented Aug 3, 2021

@msbentley Do you have an example of test data that when you run and validate is successful but still writes to stack trace log file. It would help me to know where in the code that is happening. As of right now, I cannot find it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment