Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spark: add nested fields to SchemaDatasetFieldsFacet #2689

Conversation

dolfinus
Copy link
Contributor

@dolfinus dolfinus commented May 11, 2024

Problem

Solution

One-line summary:

Add nested Spark Dataframe fields support to SchemaDatasetFieldsFacet. Also include field comment as description.

Checklist

  • You've signed-off your work
  • Your pull request title follows our guidelines
  • Your changes are accompanied by tests (if relevant)
  • Your change contains a small diff and is self-contained
  • You've updated any relevant documentation (if relevant)
  • Your comment includes a one-liner for the changelog about the specific purpose of the change (not required for changes to tests, docs, or CI config)
  • You've versioned the core OpenLineage model or facets according to SchemaVer (if relevant)
  • You've added a header to source files (if relevant)

SPDX-License-Identifier: Apache-2.0
Copyright 2018-2023 contributors to the OpenLineage project

@dolfinus dolfinus force-pushed the feature/spark-nested-fields-facet branch 4 times, most recently from fa93994 to 79938a3 Compare May 11, 2024 15:40
Signed-off-by: Martynov Maxim <martinov_m_s_@mail.ru>
@dolfinus dolfinus force-pushed the feature/spark-nested-fields-facet branch from 79938a3 to 211879b Compare May 11, 2024 15:55
@dolfinus dolfinus marked this pull request as ready for review May 11, 2024 16:15
@pawel-big-lebowski pawel-big-lebowski merged commit d0e8013 into OpenLineage:main May 13, 2024
33 checks passed
@dolfinus dolfinus deleted the feature/spark-nested-fields-facet branch May 13, 2024 08:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants