Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a test Airflow DAG from housing.py #275

Closed
marov opened this issue Oct 12, 2021 · 1 comment · Fixed by #320
Closed

Create a test Airflow DAG from housing.py #275

marov opened this issue Oct 12, 2021 · 1 comment · Fixed by #320

Comments

@marov
Copy link
Contributor

marov commented Oct 12, 2021

As a first step towards #236 we will create the cli command that does the following:

  • Extract an executable snippet from housing.py using slicing and save as a py file
  • Add Airflow specific code to that file
  • Make sure Airflow test passes
@marov marov self-assigned this Oct 12, 2021
@yifanwu
Copy link
Contributor

yifanwu commented Oct 12, 2021

For the slice we can start with the following "internal APIs" currently used in cli.py

artifact = db.get_artifact_by_name(slice)
sliced_code = get_program_slice(graph, [cast(LineaID, artifact.id)])

Also we should be concrete about what the user experience would be. We can have something like the following, with 'p value' being the example from housing.py.

# loading the slice from db
lineapy --airflow 'p value'
# loading the slice only from the runtime
lineapy tests/housing.py --airflow 'p value'

I'm leaning towards supporting the first case lineapy --airflow 'p value' and decoupling the execution from the workflow deployment, and it's supported by the python code snippet I included earlier. We should add tests though to check for the db retrieval (per @saulshanabrook's hygiene check!)

@saulshanabrook saulshanabrook added this to In Progress in Issue Triage Oct 14, 2021
Issue Triage automation moved this from In Progress to Done Oct 25, 2021
@dorx dorx unassigned marov Feb 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Development

Successfully merging a pull request may close this issue.

2 participants