fix: #1387 persist include dir after running pytest

[zach-nicoll-wcq]

Description

After running astro dev pytest, the coverage reports created with the include dir are not being persisted outside of the test container.

This is a problem, because the coverage reports cannot then be used to update PR comments etc. with the report results. Essentially, coverage reports are useless.

In the current state, this statement is false:

If your test generates artifacts, such as code coverage reports, you can output the artifacts to the include folder of your Astro project so they can be accessed after the test has finished.

This PR makes the above statement true, copying the include folder on the host machine after the Pytest run completes.

🎟 Issue(s)

#1387

🧪 Functional Testing

1. astro dev init
2. Add pytest-cov==4.1.0 to requirements.txt (or any other pytest-cov version)
3. astro dev pytest --args "--cov --cov-report=xml:include/coverage.xml --cov-report=html:include/html"
4. Observe that the include dir is populated once the tests succeed

📸 Screenshots

📋 Checklist

• Rebased from the main (or release if patching) branch (before testing)
• Ran make test before taking out of draft
• Ran make lint before taking out of draft
• Added/updated applicable tests
• Tested against Astro-API (if necessary).
• Tested against Houston-API and Astronomer (if necessary).
• Communicated to/tagged owners of respective clients potentially impacted by these changes.
• Updated any related documentation

[sunkickr]

did you try astro dev pytest --args "--cov --cov-report xml:include/coverage.xml"?

[sunkickr]

The way volumes is handled by the pytest command changed which must have broke the functionality described in docs. we can add the cp command if needed. The code within if htmlReport is used by the astro dev upgrade-test command to create an html report using pytest-html. That code would need to stay

[zach-nicoll-wcq]

did you try astro dev pytest --args "--cov --cov-report xml:include/coverage.xml"?

Yes, if you see my functional testing steps, this is one of them to verify the functionality. Alas, it does not work in the current state.

The way volumes is handled by the pytest command changed which must have broke the functionality described in docs. we can add the cp command if needed. The code within if htmlReport is used by the astro dev upgrade-test command to create an html report using pytest-html. That code would need to stay

I see - to simplify the process, would it be easier to just put the dag-test-report.html in the include dir, and then perform a copy? That way everything can be done at once.

[sunkickr]

@zach-nicoll-wcq we copy the test report into a specific testHomeDirectory which is separate from include. it makes sense to me to continue to only copy the html-report into testHomeDirectory for the upgrade-test command and add this more general copy of the include folder every time the pytest function is called

[zach-nicoll-wcq]

@zach-nicoll-wcq we copy the test report into a specific testHomeDirectory which is separate from include. it makes sense to me to continue to only copy the html-report into testHomeDirectory for the upgrade-test command and add this more general copy of the include folder every time the pytest function is called

@sunkickr roger that - I have amended PR to perform the copy after the html report logic is run.

I also noticed that the container shutdown logic was repeated, so I refactored the error handling at the end of the Pytest function to log the docker errors, and only remove the docker container in a single place.

[zach-nicoll-wcq]

@sunkickr @neel-astro @kushalmalani any movement on this? Would love to start using this functionality in my own projects.

[SebOpo]

if docErr != nil {

[zach-nicoll-wcq]

@SebOpo have amended :)

[zach-nicoll-wcq]

@sunkickr @neel-astro @kushalmalani @SebOpo

Any movement? This feels like a very easy win for the project, and will align the functionality to what people expect from the docs...

[sunkickr]

Worried about is verifying that other command that use the function still work:

• astro dev pytest without using the --args flag
• astro dev upgrade-test command

I'll just verify before release