Validate gives incorrect records mismatch WARNING for interleaved data objects

[radiosci]

Validate issues a warning of the form:

table 2: Number of records read is not equal to the defined number of records in the label (expected 1097, got 1105)

when multiple Table_Character objects are interleaved with shorter (identifier, single record) tables and/or Headers. The difference in the record count is the accumulated number of records in the interleaving tables and/or Headers.

To reproduce - first example

1. Unzip the attached i070135a.{xml,dlf} pair
2. Type "validate 1070135a.xml"

Expected behavior
There is a file header (20 records @ 82 bytes = 1640 bytes)
Followed by a "1-Way" header (4 * 82 = 328 bytes)
Followed by a "1-Way" data table (1073 * 82 = 87986 bytes)
Followed by a one record dummy line (1 * 82 = 82 bytes)
Followed by a "2-Way" header (4 * 82 = 328 bytes)
Followed by a "2-Way" data table (1097 * 82 = 89954 bytes)
Followed by a one record dummy line (1 * 82 = 82 bytes)
Followed by two trailer records (2 * 82 = 164 bytes)

**Actual behavior"
PASS: file:/Users/rsimpson/Data/DOCUMENT_BUNDLE/WORKING/DSN_0159-SCIENCE/i070135a.xml
Begin Content Validation: file:/Users/rsimpson/Data/DOCUMENT_BUNDLE/WORKING/DSN_0159-SCIENCE/i070135a.dlf
WARNING [error.table.records_mismatch] table 2: Number of records read is not equal to the defined number of records in the label (expected 1097, got 1105).
End Content Validation: file:/Users/rsimpson/Data/DOCUMENT_BUNDLE/WORKING/DSN_0159-SCIENCE/i070135a.dlf
1 product validation(s) completed

The "2-Way" data table has 1097 records; the three following records make 1100, not the 1105 claimed by Validate. In fact, the 1097 in the data table plus the "1-Way" Header and the "2-Way" header in the middle of the file make 1105. The three actual trailing records are not included in the count.

To reproduce - second example

1. Unzip the attached GRV_JUGR_2016001_2016292_10_V01.{xml,tab} pair
2. Type "validate GRV_JUGR_2016001_2016292_10_V01.xml"

Expected behavior
This is a repeating structure; for each day there is an "identifier" record, followed by 4 records with header information, followed by approximately 49 records of meteorological data, followed by a record filled with ASCII blanks. All records are 60 bytes. The meteorological data are collected approximately every 30 minutes from midnight to midnight; but sometimes the cadence is off and there are either more or fewer than 49 measurements (the range is 29-50, with most tables having 49). Most days will contribute 55 records (3300 bytes) to the file; there are 292 days in this file. The data table for the last day is nominal (49 meteorological records). Only the 292 blank records between days are not specified in the label; there should be, at most 1 unexplained record at the end of the last table.

Actual behavior
Begin Content Validation: file:/Users/rsimpson/Data/JUNO/ANCILLARY_LYLE_200522/WEA/GRV_JUGR_2016001_2016292_10_V01.tab
WARNING [error.table.records_mismatch] table 584: Number of records read is not equal to the defined number of records in the label (expected 49, got 1509).
End Content Validation: file:/Users/rsimpson/Data/JUNO/ANCILLARY_LYLE_200522/WEA/GRV_JUGR_2016001_2016292_10_V01.tab

The difference between 1509 and 49 is 292*5. That is, the records "found" but not expected are the identifier and header records for all of the 292 days scattered through the file. The blank record which might have been "found" is not included in the count.

** Version of Software Used**
Validate 1.23.1

Test Data / Additional context
Add test data or any other context about the problem here.

i070135a.xml.gz
i070135a.dlf.gz
GRV_JUGR_2016001_2016292_10_V01.tab.gz
GRV_JUGR_2016001_2016292_10_V01.xml.gz

Screenshots
None

Desktop (please complete the following information):
Mac OSX 10.15.4
Safari 13.1

Applicable requirements
Sorry, don't understand what you want here.

[msbentley]

I imagine this is similar to #233?

[radiosci]

I agree.   But #234 involves multiple tables interleaved with other data objects, whereas #233 seemed to be about a single array + table pair.  I was hoping my examples would shed more light on the problem, especially since one of the examples did not produce a warning about unspecified records that were there. On Friday, May 29, 2020, 12:02:06 AM PDT, Mark S. Bentley <notifications@github.com> wrote: I imagine this is similar to #233? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.