HPCC-21418 Don't output a line for empty file parts when despraying JSON file

[AttilaVamos]

Avoid to create partition for empty file parts (contain
only headers and/or part separators) and prevent to store
them into the despray target file.

Extend spray_test_json.ecl to create logical file with empty
headers and json content, despray then spray it back and compare
the sprayed file with the original one.

Re-targeted version of #12143

Signed-off-by: Attila Vamos attila.vamos@gmail.com

Type of change:

• This change is a bug fix (non-breaking change which fixes an issue).
• This change is a new feature (non-breaking change which adds functionality).
• This change improves the code (refactor or other change that does not change the functionality)
• This change fixes warnings (the fix does not alter the functionality or the generated code)
• This change is a breaking change (fix or feature that will cause existing behavior to change).
• This change alters the query API (existing queries will have to be recompiled)

Checklist:

• My code follows the code style of this project.
  • My code does not create any new warnings from compiler, build system, or lint.
• The commit message is properly formatted and free of typos.
  • The commit message title makes sense in a changelog, by itself.
  • The commit is signed.
• My change requires a change to the documentation.
  • I have updated the documentation accordingly, or...
  • I have created a JIRA ticket to update the documentation.
  • Any new interfaces or exported functions are appropriately commented.
• I have read the CONTRIBUTORS document.
• The change has been fully tested:
  • I have added tests to cover my changes.
  • All new and existing tests passed.
  • I have checked that this change does not introduce memory leaks.
  • I have used Valgrind or similar tools to check for potential issues.
• I have given due consideration to all of the following potential concerns:
  • Scalability
  • Performance
  • Security
  • Thread-safety
  • Premature optimization
  • Existing deployed queries will not be broken
  • This change fixes the problem, not just the symptom
  • The target branch of this pull request is appropriate for such a change.
• There are no similar instances of the same problem that should be addressed
  • I have addressed them here
  • I have raised JIRA issues to address them separately
• This is a user interface / front-end modification
  • I have tested my changes in multiple modern browsers
  • The component(s) render as expected

Testing:

Tested manually and locally executed spray/despray test cases from Regression Suite.

[hpcc-jirabot]

https://track.hpccsystems.com/browse/HPCC-21418
Jira not updated (pull request already registered)

[AttilaVamos]

@richardkchapman Re-targeted, but the relevant regression test (spray_test_json.ecl) will be failed based on "make install" and directory mismatch (fixed on 7.x)

[HPCCSmoketest]

Automated Smoketest: ❌
OS: centos 7.4.1708 (Linux 3.10.0-327.28.3.el7.x86_64)
Sha: de08d2b
Build: success
Build: success
HPCC Start: OK

Unit tests result:

Test	total	passed	failed	errors	timeout
unittest	97	97	0	0	0
wutoolTest(Dali)	19	19	0	0	0
wutoolTest(Cassandra)	19	19	0	0	0

Regression test result:

phase	total	pass	fail
setup (hthor)	11	11	0
setup (thor)	11	11	0
setup (roxie)	11	11	0
test (hthor)	736	730	6
test (thor)	634	631	3
test (roxie)	769	764	5

Errors:

• test
  • hthor
    • 143. despray, 557. spray_header_test (isTerminated=false), 558. spray_header_test (isTerminated=true), 560. spray_test (sprayFixed=false, sprayEmpty=false), 561. spray_test (sprayFixed=false, sprayEmpty=true), 559. spray_test (sprayFixed=true),
  • thor
    • 523. spray_test_json (isSmallFile=false), 521. spray_test_json (isSmallFile=true, isUnBallanced=false), 522. spray_test_json (isSmallFile=true, isUnBallanced=true),
  • roxie
    • 140. despray, 449. persist_replicate, 599. spray_test_json (isSmallFile=false), 597. spray_test_json (isSmallFile=true, isUnBallanced=false), 598. spray_test_json (isSmallFile=true, isUnBallanced=true),

HPCC Stop: OK
Time stats:

Prep time	Build time	Package time	Install time	Start time	Test time	Stop time	Summary
39 sec (00:00:39)	143 sec (00:02:23)	0 sec (00:00:00)	2 sec (00:00:02)	40 sec (00:00:40)	1231 sec (00:20:31)	39 sec (00:00:39)	1494 sec (00:24:54)