Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tests that run example python scripts #2559

Merged
merged 19 commits into from Jun 17, 2019

Conversation

@mtreinish
Copy link
Member

commented Jun 3, 2019

Summary

This commit adds a new test module that will execute all the example
scripts to verify they still work as we make changes. However, a number
of example scripts rely on using the ibmq service which is an optional
component now. To make filtering the example scripts between those which
don't need ibmq from those that need ibmq this also moves all the ibmq
examples into a self contained subdir in examples/python. Those examples
which require ibmq are run conditionally as online_tests. For example
scripts which were running on both local simulators and ibmq these are
split into 2 files one for just simulation and the other in ibmq that
uses the ibmq backends.

Details and comments

Fixes #970
Fixes #2549

@mtreinish

This comment has been minimized.

Copy link
Member Author

commented Jun 3, 2019

This is going to be blocked by #2549 because the teleport example fails. We can add the fix for that here, or fix it independently and just rebase this after that lands.

@ajavadia

This comment has been minimized.

Copy link
Member

commented Jun 7, 2019

I would say that we should not introduce a dependency on ddt here.

For one, the ddt docs say that the package is only tested for Python 2.7 and 3.3.

But more importantly, I would say that if the standard library provides a functionality, it is strongly preferred over 3rd party libraries. Since Python 3.4, unittest.subTest was introduced, which seems to achieve most of what is intended here (i.e. test each example).

@mtreinish

This comment has been minimized.

Copy link
Member Author

commented Jun 7, 2019

I would say that we should not introduce a dependency on ddt here.

For one, the ddt docs say that the package is only tested for Python 2.7 and 3.3.

But more importantly, I would say that if the standard library provides a functionality, it is strongly preferred over 3rd party libraries. Since Python 3.4, unittest.subTest was introduced, which seems to achieve most of what is intended here (i.e. test each example).

The docs are out of date they run on 2.7, 3.5, and 3.4 for ci: https://travis-ci.org/datadriventests/ddt/builds/497695081?utm_source=github_status&utm_medium=notification but it's not like the library is particularly complex and it doesn't have a lot of api churn that will likely cause an issues on newer versions of unittest in stdlib: https://github.com/datadriventests/ddt/blob/master/ddt.py

For this use case subtest would work easily enough, where ddt is more useful is in examples like what @1ucian0 is doing in #2563

@ajavadia

This comment has been minimized.

Copy link
Member

commented Jun 7, 2019

So if subTest does the job here, can we use it? Then I'd block #2563 and #2519 until reaching a consensus on #2589.

mtreinish and others added some commits Jun 3, 2019

Add tests that run example python scripts
This commit adds a new test module that will execute all the example
scripts to verify they still work as we make changes. However, a number
of example scripts rely on using the ibmq service which is an optional
component now. To make filtering the example scripts between those which
don't need ibmq from those that need ibmq this also moves all the ibmq
examples into a self contained subdir in examples/python. Those examples
which require ibmq are run conditionally as online_tests. For example
scripts which were running on both local simulators and ibmq these are
split into 2 files one for just simulation and the other in ibmq that
uses the ibmq backends.

Fixes #970
Add skips for windows/appveyor
The text drawer isn't working correctly on windows/appveyor. Until that
unrelated issue is fixed it will block this. So this commit adds a skip
on windows envs until issue #2616 is fixed.

@mtreinish mtreinish force-pushed the mtreinish:examples-as-tests branch from baeae77 to ab57bb3 Jun 11, 2019

mtreinish added some commits Jun 11, 2019

Show resolved Hide resolved examples/python/ghz.py Outdated
@kdk

kdk approved these changes Jun 17, 2019

@mtreinish mtreinish merged commit f5dd2e2 into Qiskit:master Jun 17, 2019

3 checks passed

Travis CI - Pull Request Build Passed
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
license/cla Contributor License Agreement is signed.
Details

@mtreinish mtreinish deleted the mtreinish:examples-as-tests branch Jun 17, 2019

ajavadia added a commit that referenced this pull request Jul 11, 2019

introduce ddt, test already-mapped circuits in the transpiler (#2563)
* fix

* cleaning up

* lint

* preset_passmanagers/level3.py

* test

* requirements-dev.txt from #2559

* docstring

* dsc

* docstring

* case are dicts

* simpler approach

* docstring

* avoid double zip

* lint

* add name

* add name

* name

* decorator

* lint

* lint

* no METATEST

* lint

* revert changes to preset pass managers

* docstring

* small consistency tweak

* style
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.