Skip to content

[docs] Remove OpenLineage Cross Provider dependency documentation#63836

Closed
TylerLubeck wants to merge 3 commits intoapache:mainfrom
TylerLubeck:openlineage-docs
Closed

[docs] Remove OpenLineage Cross Provider dependency documentation#63836
TylerLubeck wants to merge 3 commits intoapache:mainfrom
TylerLubeck:openlineage-docs

Conversation

@TylerLubeck
Copy link

Best I can tell, this is out of date and unused:

  • The extras don't exist anymore
  • The "extra" dependencies are now full-on requirements

Was generative AI tooling used to co-author this PR?

Nope

Best I can tell, this is out of date and unused:
- The extras don't exist anymore
- The "extra" dependencies are now full-on requirements
@mobuchowski
Copy link
Contributor

I think this is autogenerated from something. @kacpermuda you might know about this

@TylerLubeck
Copy link
Author

I went spelunking and think I found where it comes from, but would love some guidance if the most recent commit doesn't do it

@kacpermuda
Copy link
Collaborator

Not sure where it comes from, but I'd guess it is auto generated from provider.yml or something similar. But given that these two libs are required by OL, it makes sense to remove them from optional. @mobuchowski maybe run full tests on this PR and see it if any check breaks (auto generation tries to modify file), if not , we should be fine

@TylerLubeck
Copy link
Author

I found it - it's auto generated by breeze. Breeze parses all of the imports in all of the python files in the provider, then generates a list of ones coming from other providers.

It then uses that list to generate a pyproject.toml and an index.rst (among some others)

  • For pyproject.toml, it will generate an extra entry for each provider not found in the explicit requirements
  • For index.rst, it does not check the explicit requirements before generating the text I attempted to remove here.

tl;dr this looks to be a bug in breeze and I opened #63891. Dunno how y'all want to proceed from here - I can chase this over to Breeze and fix the bug there and then follow up with question about regenerating docs?

@kacpermuda
Copy link
Collaborator

Yes, if the files is auto-generate across all providers, let's get a consensus on the breeze behavior and then apply the fix there, so it regenerates all providers. We can close this PR then, imho.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants