Skip to content

Add example DAG for ERP data migration pipeline (ETL workflow)#66774

Open
venkatamandavilli-code wants to merge 3 commits into
apache:mainfrom
venkatamandavilli-code:main
Open

Add example DAG for ERP data migration pipeline (ETL workflow)#66774
venkatamandavilli-code wants to merge 3 commits into
apache:mainfrom
venkatamandavilli-code:main

Conversation

@venkatamandavilli-code
Copy link
Copy Markdown

Summary

This PR adds an example DAG demonstrating a typical ERP data migration workflow using Apache Airflow.

Details

The DAG models a structured ETL pipeline commonly used in enterprise ERP transformations:

  • Data extraction from legacy systems
  • Data validation (completeness and accuracy checks)
  • Data transformation to target schema
  • Data loading into target system
  • Post-load reconciliation

Use Case

This example is intended to help users understand how Airflow can be used to orchestrate complex enterprise data migration workflows, especially in ERP environments.

Notes

This is a simplified illustrative example based on real-world enterprise migration patterns.

@boring-cyborg
Copy link
Copy Markdown

boring-cyborg Bot commented May 12, 2026

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

@venkatamandavilli-code
Copy link
Copy Markdown
Author

Thank you for reviewing my contribution.

Please let me know if any modifications or improvements are required. I would be happy to update the PR accordingly.

Copy link
Copy Markdown
Author

@venkatamandavilli-code venkatamandavilli-code left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the example DAG with additional documentation and function-level descriptions to clarify that this is an illustrative ERP migration workflow example.

@potiuk
Copy link
Copy Markdown
Member

potiuk commented May 19, 2026

@venkatamandavilli-code A few things need addressing before review — see our Pull Request quality criteria.

  • Pre-commit / static checks — Failing: Basic tests / Static checks: basic checks only. See docs.

Note: Your branch is 146 commits behind main. Please rebase and push again to get up-to-date CI results.

No rush.


Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you.


Drafted-by: Claude Code (Opus 4.7); reviewed by @potiuk before posting

@venkatamandavilli-code
Copy link
Copy Markdown
Author

Thanks for the guidance. I’ll update the branch and review the static check failure before pushing the correction.

Appreciate the feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants