Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Add SAPBW source and Prefect tasks #1053

Merged
merged 22 commits into from
Sep 30, 2024
Merged

✨ Add SAPBW source and Prefect tasks #1053

merged 22 commits into from
Sep 30, 2024

Conversation

gwieloch
Copy link
Contributor

Summary

SAP BW connector migration from viadot 1 to viadot 2. Also added some utility functions into viadot utils.

Importance

Required by the migration project.

Checklist

This PR:

  • follows the guidelines laid out in CONTRIBUTING.md
  • links relevant issue(s)
  • adds/updates tests (if appropriate)
  • adds/updates docstrings (if appropriate)
  • adds an entry in CHANGELOG.md

@trymzet
Copy link
Contributor

trymzet commented Sep 25, 2024

@gwieloch it seems the checks are failing, can you make sure they pass first, then request PR review? Otherwise I would need to check the same code twice.

Also, pls rebase as well.

As for the tests failing due to pyrfc not being installed, you need to escape this import in the source, tasks, and tests, same as in the SAP RFC source:
https://github.com/dyvenia/viadot/blob/2.0/src/viadot/sources/sap_rfc.py#L19-L24
https://github.com/dyvenia/viadot/blob/2.0/tests/unit/test_sap_rfc.py#L6-L9
https://github.com/dyvenia/viadot/blob/2.0/src/viadot/orchestration/prefect/tasks/sap_rfc.py#L11-L12

For more info, see the "optional dependencies" sections in the developer guide docs https://viadot.docs.dyvenia.com/developer_guide/

Copy link
Contributor

@trymzet trymzet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checks not passing

@trymzet trymzet changed the title Sapbw 2.0 ✨ Add Sapbw source and Prefect tasks Sep 27, 2024
Copy link
Contributor

@trymzet trymzet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added my comments. Some unit tests should also be added.

src/viadot/sources/__init__.py Outdated Show resolved Hide resolved
src/viadot/sources/__init__.py Outdated Show resolved Hide resolved
src/viadot/orchestration/prefect/flows/sap_bw_to_adls.py Outdated Show resolved Hide resolved
src/viadot/orchestration/prefect/flows/sap_bw_to_adls.py Outdated Show resolved Hide resolved
src/viadot/sources/sap_bw.py Outdated Show resolved Hide resolved
src/viadot/sources/sap_bw.py Outdated Show resolved Hide resolved
src/viadot/sources/sap_bw.py Outdated Show resolved Hide resolved
src/viadot/sources/sap_bw.py Outdated Show resolved Hide resolved
tests/integration/test_sap_bw.py Outdated Show resolved Hide resolved
@angelika233 angelika233 changed the title ✨ Add Sapbw source and Prefect tasks ✨ Add SAPBW source and Prefect tasks Sep 30, 2024
@trymzet trymzet mentioned this pull request Sep 30, 2024
3 tasks
@trymzet trymzet merged commit 16cae13 into 2.0 Sep 30, 2024
3 checks passed
@trymzet trymzet deleted the sapbw_2.0 branch September 30, 2024 08:55
@trymzet trymzet added the 2.0 viadot 2.0 label Oct 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.0 viadot 2.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants