Skip to content

Python: Port test model to API graphs #9368

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
May 30, 2022
Merged

Python: Port test model to API graphs #9368

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
a5dc4f4
Python: Expand test-filter tests
RasmusWL May 30, 2022
a8b4b6a
Python: Move test-modeling to API-graphs
RasmusWL May 30, 2022
08e64ea
Python: Remove contrived test-case example
RasmusWL May 30, 2022
420dea0
Python: Fix example TestCase
RasmusWL May 30, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 7 additions & 6 deletions python/ql/lib/semmle/python/filters/Tests.qll
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,15 @@
import python
private import semmle.python.ApiGraphs

abstract class TestScope extends Scope { }

// don't extend Class directly to avoid ambiguous method warnings
class UnitTestClass extends TestScope {
class UnitTestClass extends TestScope, Class {
UnitTestClass() {
exists(ClassValue cls | this = cls.getScope() |
cls.getABaseType+() = Module::named("unittest").attr(_)
or
cls.getABaseType+().getName().toLowerCase() = "testcase"
exists(API::Node testCaseClass, string testCaseString |
testCaseString.matches("%TestCase") and
testCaseClass = any(API::Node mod).getMember(testCaseString)
|
this.getParent() = testCaseClass.getASubclass*().getAnImmediateUse().asExpr()
)
}
}
Expand Down
13 changes: 10 additions & 3 deletions python/ql/test/library-tests/filters/tests/Filter.expected
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,13 @@
| test.py:4:1:4:23 | Class MyTest |
| test.py:6:5:6:21 | Function test_1 |
| test.py:9:5:9:21 | Function test_2 |
Comment on lines -1 to -3
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the fact that these are no longer matched intended? If so, then maybe we should just get rid of test.py entirely?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah I noted in a8b4b6a:

Notice that although we loose the contrived examples in test.py, we do
gain support for real-world test-case construction, which seems worth
the tradeoff.

I think I'll delete that file 👍

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah! I didn't see that commit message. Thanks!

| test_foo.py:3:1:3:15 | Function test_foo |
| unittest_test.py:3:1:3:33 | Class FooTest |
| unittest_test.py:4:5:4:25 | Function test_valid |
| unittest_test.py:7:1:7:49 | Class FunctionFooTest |
| unittest_test.py:8:5:8:25 | Function test_valid |
| unittest_test.py:11:1:11:50 | Class AsyncTest |
| unittest_test.py:12:11:12:31 | Function test_valid |
| unittest_test.py:17:1:17:45 | Class MyDjangoUnitTest |
| unittest_test.py:18:5:18:25 | Function test_valid |
| unittest_test.py:23:1:23:56 | Class MyFlaskUnitTest |
| unittest_test.py:24:5:24:25 | Function test_valid |
| unittest_test.py:29:1:29:59 | Class MyTornadoUnitTest |
| unittest_test.py:30:5:30:25 | Function test_valid |
10 changes: 0 additions & 10 deletions python/ql/test/library-tests/filters/tests/test.py
View file
Open in desktop

This file was deleted.

26 changes: 26 additions & 0 deletions python/ql/test/library-tests/filters/tests/unittest_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,29 @@
class FooTest(unittest.TestCase):
def test_valid(self):
pass

class FunctionFooTest(unittest.FunctionTestCase):
def test_valid(self):
pass

class AsyncTest(unittest.IsolatedAsyncioTestCase):
async def test_valid(self):
pass

# django -- see https://docs.djangoproject.com/en/4.0/topics/testing/overview/
import django.test
class MyDjangoUnitTest(django.test.TestCase):
def test_valid(self):
pass

# flask -- see https://pythonhosted.org/Flask-Testing/
import flask_testing
class MyFlaskUnitTest(flask_testing.LiveServerTestCase):
def test_valid(self):
pass

# tornado -- see https://www.tornadoweb.org/en/stable/testing.html#tornado.testing.AsyncHTTPTestCase
import tornado.testing
class MyTornadoUnitTest(tornado.testing.AsyncHTTPTestCase):
def test_valid(self):
pass
4 changes: 2 additions & 2 deletions python/ql/test/query-tests/Statements/asserts/assert.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,8 +61,8 @@ def ok_assert_false(x):
if x:
assert 0==1, "Ok"

class TestCase:
pass
from unittest import TestCase


class MyTest(TestCase):
def test_ok_assert_in_test(self, x):
Expand Down