spark: Disable CLL for LogicalRDD plans

[kchledowski]

One-line summary for changelog:

Disable Column-Level Lineage extraction for Spark LogicalRDD plans to prevent incorrect lineage caused by lost schema and transformation context.

Meaningful description

Column-Level Lineage generated from Spark LogicalRDD plans is currently
unreliable. Although OpenLineage can still identify the underlying data
source associated with an RDD, all transformation context is lost —
both:

• transformations performed before converting the DataFrame to an RDD,
• and transformations performed on the RDD itself.

When Spark constructs a LogicalRDD, it contains only the final row
schema, without any expression trees, renames, derived columns, or
projections that produced it. As a result, OpenLineage cannot determine
whether the output schema corresponds directly to the source schema or
has been altered by transformations.

This leads to the situation where:

• lineage might be correct if no schema‑changing transformations
  occurred,
• lineage might be incorrect if transformations happened before or
  during RDD processing,
• and there is no way for OpenLineage to distinguish these cases.

Therefore, any column-level lineage produced from a LogicalRDD is
inherently unverifiable and may be misleading.

Example illustrating the problem

// Source table: (a, b)
val df = spark.table("source_table")

// Derive new column 'c'
val transformed = df.select(
  $"a",
  $"b",
  concat($"a", $"b").as("c")
)

// Convert to RDD and perform another transformation on the RDD
val rdd = transformed.rdd.map { row =>
  // Example RDD-level transformation, e.g. uppercasing column c
  Row(
    row.getString(0),
    row.getString(1),
    row.getString(2).toUpperCase
  )
}

val finalDf = spark.createDataFrame(rdd, transformed.schema)

finalDf.write.saveAsTable("result_table")

Expected lineage:

result_table.c → source_table.a, source_table.b

Current lineage with LogicalRDD:

result_table.c → source_table.c

To avoid producing wrong lineage — which is worse than incomplete lineage — this PR disables Column-Level Lineage generation when Spark produces a LogicalRDD. This ensures correctness and avoids misleading downstream consumers.