Allow gaps in segment redefines

[yruslan]

Background

When a file has, say, 30 segments, but we want to extract just 3 of them, we can write the code like this:

val df = spark
  .read
  .format("cobol")
  .option("copybook", "s3://my_bucket/copybook.cpy")
  .option("record_format", "F")
  .option("segment_field", "SEGMENT-ID")
  .option("redefine-segment-id-map:1", "SEG1=>VALUE1")
  .option("redefine-segment-id-map:10", "SEG10=>VALUE10")
  .option("redefine-segment-id-map:15", "SEG15=>VALUE15")
  .load("s3://my_bucket/my_data")

But because these 3 segments are not contiguous redefines of the base segment, you can get the error:

The 'SEG10' field is specified to be a segment redefine. However, it is not in the same group of REDEFINE fields

Feature

Allow gaps in segment redefines.

Example [Optional]

--

Proposed Solution [Optional]

Allow gaps as long as they redefine the same base field.