Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to get SQLAlchemy sinks to work #826

Closed
cyounkins opened this issue Apr 16, 2024 · 4 comments
Closed

Unable to get SQLAlchemy sinks to work #826

cyounkins opened this issue Apr 16, 2024 · 4 comments

Comments

@cyounkins
Copy link

Pysa Bug

Pre-submission checklist
[x] I've checked the list of common issues and mine does not appear

Bug description
I've been unable to get pysa to work for a minimal sqlalchemy project. I have set it up using the sqlalchemy types from the pyre-check repo. I have set up two flows that should be detected - one to SQLAlchemy's execute, and my_sink. my_sink works as expected, SQLAlchemy does not.

Reproduction steps
Try my repo: https://github.com/cyounkins/pysa-testing/tree/63395fc6b3ba826b1ebb9dec1c55abcadf4c7622

$ docker-compose build pyre && docker-compose run -it pyre /bin/bash
# pyre analyze

Expected behavior
Both flows should be detected.

Logs
Please run your reproduction steps with --noninteractive (eg. pyre --noninteractive analyze) and paste the output here:

# pyre --noninteractive analyze
2024-04-16 18:41:29,714 [PID 559] INFO No binary specified, looking for `pyre.bin` in PATH
2024-04-16 18:41:29,717 [PID 559] INFO Pyre binary is located at `/usr/local/bin/pyre.bin`
2024-04-16 18:41:29,719 [PID 559] INFO Could not determine the number of Pyre workers from configuration. Auto-set the value to 9.
2024-04-16 18:41:29,721 [PID 559] INFO No typeshed specified, looking for it...
2024-04-16 18:41:29,722 [PID 559] INFO Found: `/usr/local/lib/pyre_check/typeshed`
2024-04-16 18:41:29,726 [PID 559] INFO Writing arguments into /tmp/pyre_arguments_in7fld1z.json...
2024-04-16 18:41:29,727 [PID 559] DEBUG Arguments:
{
  "source_paths": {
    "kind": "simple",
    "paths": [
      "/usr/src/app"
    ]
  },
  "search_paths": [
    "/usr/local/lib/python3.10/site-packages$dataclasses_json",
    "/usr/local/lib/python3.10/site-packages$pip",
    "/usr/local/lib/python3.10/site-packages$packaging",
    "/usr/local/lib/python3.10/site-packages$libcst",
    "/usr/local/lib/python3.10/site-packages$testslide",
    "/usr/local/lib/python3.10/site-packages$click",
    "/usr/local/lib/python3.10/site-packages$sqlalchemy-stubs",
    "/usr/local/lib/python3.10/site-packages$marshmallow",
    "/usr/local/lib/python3.10/site-packages$typeguard",
    "/usr/local/lib/pyre_check/typeshed/stdlib",
    "/usr/local/lib/pyre_check/typeshed/stubs/D3DShot",
    "/usr/local/lib/pyre_check/typeshed/stubs/DateTimeRange",
    "/usr/local/lib/pyre_check/typeshed/stubs/Deprecated",
    "/usr/local/lib/pyre_check/typeshed/stubs/ExifRead",
    "/usr/local/lib/pyre_check/typeshed/stubs/Flask-Cors",
    "/usr/local/lib/pyre_check/typeshed/stubs/Flask-Migrate",
    "/usr/local/lib/pyre_check/typeshed/stubs/Flask-SQLAlchemy",
    "/usr/local/lib/pyre_check/typeshed/stubs/JACK-Client",
    "/usr/local/lib/pyre_check/typeshed/stubs/Markdown",
    "/usr/local/lib/pyre_check/typeshed/stubs/Pillow",
    "/usr/local/lib/pyre_check/typeshed/stubs/PyAutoGUI",
    "/usr/local/lib/pyre_check/typeshed/stubs/PyMySQL",
    "/usr/local/lib/pyre_check/typeshed/stubs/PyScreeze",
    "/usr/local/lib/pyre_check/typeshed/stubs/PyYAML",
    "/usr/local/lib/pyre_check/typeshed/stubs/Pygments",
    "/usr/local/lib/pyre_check/typeshed/stubs/SQLAlchemy",
    "/usr/local/lib/pyre_check/typeshed/stubs/Send2Trash",
    "/usr/local/lib/pyre_check/typeshed/stubs/aiofiles",
    "/usr/local/lib/pyre_check/typeshed/stubs/annoy",
    "/usr/local/lib/pyre_check/typeshed/stubs/appdirs",
    "/usr/local/lib/pyre_check/typeshed/stubs/aws-xray-sdk",
    "/usr/local/lib/pyre_check/typeshed/stubs/babel",
    "/usr/local/lib/pyre_check/typeshed/stubs/backports.ssl_match_hostname",
    "/usr/local/lib/pyre_check/typeshed/stubs/beautifulsoup4",
    "/usr/local/lib/pyre_check/typeshed/stubs/bleach",
    "/usr/local/lib/pyre_check/typeshed/stubs/boto",
    "/usr/local/lib/pyre_check/typeshed/stubs/braintree",
    "/usr/local/lib/pyre_check/typeshed/stubs/cachetools",
    "/usr/local/lib/pyre_check/typeshed/stubs/caldav",
    "/usr/local/lib/pyre_check/typeshed/stubs/cffi",
    "/usr/local/lib/pyre_check/typeshed/stubs/chardet",
    "/usr/local/lib/pyre_check/typeshed/stubs/chevron",
    "/usr/local/lib/pyre_check/typeshed/stubs/click-spinner",
    "/usr/local/lib/pyre_check/typeshed/stubs/colorama",
    "/usr/local/lib/pyre_check/typeshed/stubs/commonmark",
    "/usr/local/lib/pyre_check/typeshed/stubs/console-menu",
    "/usr/local/lib/pyre_check/typeshed/stubs/contextvars",
    "/usr/local/lib/pyre_check/typeshed/stubs/croniter",
    "/usr/local/lib/pyre_check/typeshed/stubs/dateparser",
    "/usr/local/lib/pyre_check/typeshed/stubs/decorator",
    "/usr/local/lib/pyre_check/typeshed/stubs/dj-database-url",
    "/usr/local/lib/pyre_check/typeshed/stubs/dockerfile-parse",
    "/usr/local/lib/pyre_check/typeshed/stubs/docopt",
    "/usr/local/lib/pyre_check/typeshed/stubs/docutils",
    "/usr/local/lib/pyre_check/typeshed/stubs/editdistance",
    "/usr/local/lib/pyre_check/typeshed/stubs/emoji",
    "/usr/local/lib/pyre_check/typeshed/stubs/entrypoints",
    "/usr/local/lib/pyre_check/typeshed/stubs/first",
    "/usr/local/lib/pyre_check/typeshed/stubs/flake8-2020",
    "/usr/local/lib/pyre_check/typeshed/stubs/flake8-bugbear",
    "/usr/local/lib/pyre_check/typeshed/stubs/flake8-builtins",
    "/usr/local/lib/pyre_check/typeshed/stubs/flake8-docstrings",
    "/usr/local/lib/pyre_check/typeshed/stubs/flake8-plugin-utils",
    "/usr/local/lib/pyre_check/typeshed/stubs/flake8-rst-docstrings",
    "/usr/local/lib/pyre_check/typeshed/stubs/flake8-simplify",
    "/usr/local/lib/pyre_check/typeshed/stubs/flake8-typing-imports",
    "/usr/local/lib/pyre_check/typeshed/stubs/fpdf2",
    "/usr/local/lib/pyre_check/typeshed/stubs/gdb",
    "/usr/local/lib/pyre_check/typeshed/stubs/google-cloud-ndb",
    "/usr/local/lib/pyre_check/typeshed/stubs/hdbcli",
    "/usr/local/lib/pyre_check/typeshed/stubs/html5lib",
    "/usr/local/lib/pyre_check/typeshed/stubs/httplib2",
    "/usr/local/lib/pyre_check/typeshed/stubs/humanfriendly",
    "/usr/local/lib/pyre_check/typeshed/stubs/ibm-db",
    "/usr/local/lib/pyre_check/typeshed/stubs/influxdb-client",
    "/usr/local/lib/pyre_check/typeshed/stubs/invoke",
    "/usr/local/lib/pyre_check/typeshed/stubs/jmespath",
    "/usr/local/lib/pyre_check/typeshed/stubs/jsonschema",
    "/usr/local/lib/pyre_check/typeshed/stubs/keyboard",
    "/usr/local/lib/pyre_check/typeshed/stubs/ldap3",
    "/usr/local/lib/pyre_check/typeshed/stubs/mock",
    "/usr/local/lib/pyre_check/typeshed/stubs/mypy-extensions",
    "/usr/local/lib/pyre_check/typeshed/stubs/mysqlclient",
    "/usr/local/lib/pyre_check/typeshed/stubs/netaddr",
    "/usr/local/lib/pyre_check/typeshed/stubs/oauthlib",
    "/usr/local/lib/pyre_check/typeshed/stubs/openpyxl",
    "/usr/local/lib/pyre_check/typeshed/stubs/opentracing",
    "/usr/local/lib/pyre_check/typeshed/stubs/paho-mqtt",
    "/usr/local/lib/pyre_check/typeshed/stubs/paramiko",
    "/usr/local/lib/pyre_check/typeshed/stubs/parsimonious",
    "/usr/local/lib/pyre_check/typeshed/stubs/passlib",
    "/usr/local/lib/pyre_check/typeshed/stubs/passpy",
    "/usr/local/lib/pyre_check/typeshed/stubs/peewee",
    "/usr/local/lib/pyre_check/typeshed/stubs/pep8-naming",
    "/usr/local/lib/pyre_check/typeshed/stubs/pika",
    "/usr/local/lib/pyre_check/typeshed/stubs/playsound",
    "/usr/local/lib/pyre_check/typeshed/stubs/polib",
    "/usr/local/lib/pyre_check/typeshed/stubs/prettytable",
    "/usr/local/lib/pyre_check/typeshed/stubs/protobuf",
    "/usr/local/lib/pyre_check/typeshed/stubs/psutil",
    "/usr/local/lib/pyre_check/typeshed/stubs/psycopg2",
    "/usr/local/lib/pyre_check/typeshed/stubs/pyOpenSSL",
    "/usr/local/lib/pyre_check/typeshed/stubs/pyRFC3339",
    "/usr/local/lib/pyre_check/typeshed/stubs/pyasn1",
    "/usr/local/lib/pyre_check/typeshed/stubs/pyaudio",
    "/usr/local/lib/pyre_check/typeshed/stubs/pycocotools",
    "/usr/local/lib/pyre_check/typeshed/stubs/pycurl",
    "/usr/local/lib/pyre_check/typeshed/stubs/pyfarmhash",
    "/usr/local/lib/pyre_check/typeshed/stubs/pyflakes",
    "/usr/local/lib/pyre_check/typeshed/stubs/pyinstaller",
    "/usr/local/lib/pyre_check/typeshed/stubs/pynput",
    "/usr/local/lib/pyre_check/typeshed/stubs/pyserial",
    "/usr/local/lib/pyre_check/typeshed/stubs/pysftp",
    "/usr/local/lib/pyre_check/typeshed/stubs/pytest-lazy-fixture",
    "/usr/local/lib/pyre_check/typeshed/stubs/python-crontab",
    "/usr/local/lib/pyre_check/typeshed/stubs/python-datemath",
    "/usr/local/lib/pyre_check/typeshed/stubs/python-dateutil",
    "/usr/local/lib/pyre_check/typeshed/stubs/python-gflags",
    "/usr/local/lib/pyre_check/typeshed/stubs/python-jose",
    "/usr/local/lib/pyre_check/typeshed/stubs/python-nmap",
    "/usr/local/lib/pyre_check/typeshed/stubs/python-slugify",
    "/usr/local/lib/pyre_check/typeshed/stubs/python-xlib",
    "/usr/local/lib/pyre_check/typeshed/stubs/pytz",
    "/usr/local/lib/pyre_check/typeshed/stubs/pyvmomi",
    "/usr/local/lib/pyre_check/typeshed/stubs/pywin32",
    "/usr/local/lib/pyre_check/typeshed/stubs/redis",
    "/usr/local/lib/pyre_check/typeshed/stubs/regex",
    "/usr/local/lib/pyre_check/typeshed/stubs/requests",
    "/usr/local/lib/pyre_check/typeshed/stubs/retry",
    "/usr/local/lib/pyre_check/typeshed/stubs/setuptools",
    "/usr/local/lib/pyre_check/typeshed/stubs/simplejson",
    "/usr/local/lib/pyre_check/typeshed/stubs/singledispatch",
    "/usr/local/lib/pyre_check/typeshed/stubs/six",
    "/usr/local/lib/pyre_check/typeshed/stubs/slumber",
    "/usr/local/lib/pyre_check/typeshed/stubs/stdlib-list",
    "/usr/local/lib/pyre_check/typeshed/stubs/stripe",
    "/usr/local/lib/pyre_check/typeshed/stubs/tabulate",
    "/usr/local/lib/pyre_check/typeshed/stubs/tensorflow",
    "/usr/local/lib/pyre_check/typeshed/stubs/termcolor",
    "/usr/local/lib/pyre_check/typeshed/stubs/toml",
    "/usr/local/lib/pyre_check/typeshed/stubs/toposort",
    "/usr/local/lib/pyre_check/typeshed/stubs/tqdm",
    "/usr/local/lib/pyre_check/typeshed/stubs/tree-sitter",
    "/usr/local/lib/pyre_check/typeshed/stubs/tree-sitter-languages",
    "/usr/local/lib/pyre_check/typeshed/stubs/ttkthemes",
    "/usr/local/lib/pyre_check/typeshed/stubs/typed-ast",
    "/usr/local/lib/pyre_check/typeshed/stubs/tzlocal",
    "/usr/local/lib/pyre_check/typeshed/stubs/ujson",
    "/usr/local/lib/pyre_check/typeshed/stubs/untangle",
    "/usr/local/lib/pyre_check/typeshed/stubs/urllib3",
    "/usr/local/lib/pyre_check/typeshed/stubs/vobject",
    "/usr/local/lib/pyre_check/typeshed/stubs/waitress",
    "/usr/local/lib/pyre_check/typeshed/stubs/whatthepatch",
    "/usr/local/lib/pyre_check/typeshed/stubs/xmltodict",
    "/usr/local/lib/pyre_check/typeshed/stubs/xxhash",
    "/usr/local/lib/pyre_check/typeshed/stubs/zstd",
    "/usr/local/lib/pyre_check/typeshed/stubs/zxcvbn"
  ],
  "excludes": [],
  "checked_directory_allowlist": [
    "/usr/src/app"
  ],
  "checked_directory_blocklist": [],
  "extensions": [],
  "log_path": "/usr/src/app/.pyre",
  "global_root": "/usr/src/app",
  "debug": false,
  "python_version": {
    "major": 3,
    "minor": 10,
    "micro": 14
  },
  "shared_memory": {},
  "parallel": true,
  "number_of_workers": 9,
  "inline_decorators": false,
  "no_verify": false,
  "verify_dsl": false,
  "verify_taint_config_only": false,
  "strict": false,
  "taint_model_paths": [
    "/usr/src/app/stubs"
  ],
  "use_cache": false,
  "build_cache_only": false,
  "check_invariants": false,
  "limit_entrypoints": false,
  "compact_ocaml_heap": false,
  "saved_state": {
    "watchman_root": null,
    "project_name": null,
    "cache_critical_files": []
  }
}
2024-04-16 18:41:30,750 [PID 559] INFO  Initializing shared memory (heap_size: 8589934592, dep_table_pow: 27, hash_table_pow: 26)
2024-04-16 18:41:30,760 [PID 559] INFO  Verifying taint configuration.
2024-04-16 18:41:30,762 [PID 559] PERFORMANCE  Verified taint configuration: 0.050s
2024-04-16 18:41:30,773 [PID 559] INFO  Verifying model syntax.
2024-04-16 18:41:30,775 [PID 559] INFO  Finding taint models in `/usr/src/app/stubs`.
2024-04-16 18:41:30,778 [PID 559] PERFORMANCE  Verified model syntax: 0.013s
2024-04-16 18:41:30,780 [PID 559] INFO  Parsing taint models for decorator modes...
2024-04-16 18:41:30,783 [PID 559] INFO  Finding taint models in `/usr/src/app/stubs`.
2024-04-16 18:41:30,783 [PID 559] PERFORMANCE  Parsed taint models for decorator modes: 0.004s
2024-04-16 18:41:30,784 [PID 559] INFO  Starting type checking...
2024-04-16 18:41:30,784 [PID 559] INFO  Creating environment...
2024-04-16 18:41:30,784 [PID 559] INFO  Building module tracker...
2024-04-16 18:41:30,785 [PID 559] PERFORMANCE  Module tracker built: 0.177s
2024-04-16 18:41:30,786 [PID 559] PERFORMANCE  Full environment built: 0.296s
2024-04-16 18:41:30,786 [PID 559] INFO  Found 4420 modules
2024-04-16 18:41:30,786 [PID 559] INFO  Collecting all definitions...
2024-04-16 18:41:33,752 [PID 559] PERFORMANCE  Collected definitions (defines: 79227): 2.834s
2024-04-16 18:41:33,769 [PID 559] INFO  Checking 79227 functions...
2024-04-16 18:41:36,762 [PID 559] INFO  Processed 4402 of 79227 functions
2024-04-16 18:41:39,779 [PID 559] INFO  Processed 8804 of 79227 functions
2024-04-16 18:41:40,777 [PID 559] INFO  Processed 13206 of 79227 functions
2024-04-16 18:41:40,781 [PID 559] INFO  Processed 17608 of 79227 functions
2024-04-16 18:41:40,789 [PID 559] INFO  Processed 22010 of 79227 functions
2024-04-16 18:41:41,779 [PID 559] INFO  Processed 26412 of 79227 functions
2024-04-16 18:41:41,783 [PID 559] INFO  Processed 30814 of 79227 functions
2024-04-16 18:41:41,787 [PID 559] INFO  Processed 35216 of 79227 functions
2024-04-16 18:41:42,780 [PID 559] INFO  Processed 39618 of 79227 functions
2024-04-16 18:41:43,781 [PID 559] INFO  Processed 44020 of 79227 functions
2024-04-16 18:41:45,784 [PID 559] INFO  Processed 48422 of 79227 functions
2024-04-16 18:41:46,787 [PID 559] INFO  Processed 52824 of 79227 functions
2024-04-16 18:41:46,789 [PID 559] INFO  Processed 57226 of 79227 functions
2024-04-16 18:41:47,788 [PID 559] INFO  Processed 61619 of 79227 functions
2024-04-16 18:41:47,789 [PID 559] INFO  Processed 66021 of 79227 functions
2024-04-16 18:41:47,790 [PID 559] INFO  Processed 70423 of 79227 functions
2024-04-16 18:41:48,790 [PID 559] INFO  Processed 74825 of 79227 functions
2024-04-16 18:41:48,791 [PID 559] INFO  Processed 79227 of 79227 functions
2024-04-16 18:41:48,792 [PID 559] PERFORMANCE  Check_TypeCheck: 14.984s
2024-04-16 18:41:48,793 [PID 559] MEMORY  Shared memory size post-typecheck (size: 190)
2024-04-16 18:41:48,793 [PID 559] INFO  Computing class hierarchy graph...
2024-04-16 18:41:49,794 [PID 559] PERFORMANCE  Computed class hierarchy graph: 0.769s
2024-04-16 18:41:49,800 [PID 559] INFO  Computing class intervals...
2024-04-16 18:41:49,816 [PID 559] PERFORMANCE  Computed class intervals: 0.160s
2024-04-16 18:41:49,831 [PID 559] INFO  Fetching initial callables to analyze...
2024-04-16 18:41:50,795 [PID 559] PERFORMANCE  Fetched initial callables to analyze (definitions: 13493, internals: 4, stubs: 50160): 1.150s
2024-04-16 18:41:50,795 [PID 559] INFO  Parsing taint models...
2024-04-16 18:41:51,795 [PID 559] INFO  Finding taint models in `/usr/src/app/stubs`.
2024-04-16 18:41:51,796 [PID 559] PERFORMANCE  Parsed taint models (models: 8, queries: 0): 0.840s
2024-04-16 18:41:51,796 [PID 559] INFO  Computing inferred models...
2024-04-16 18:41:52,797 [PID 559] PERFORMANCE  Computed inferred models (models: 768): 0.889s
2024-04-16 18:41:52,818 [PID 559] INFO  Computing overrides...
2024-04-16 18:41:53,803 [PID 559] WARNING  `google.protobuf.message.Message.ClearField` has 57 overrides, this might slow down the analysis considerably.
2024-04-16 18:41:53,804 [PID 559] WARNING  `google.protobuf.message.Message.__init__` has 58 overrides, this might slow down the analysis considerably.
2024-04-16 18:41:53,805 [PID 559] WARNING  `libcst._nodes.base.CSTNode._codegen_impl` has 102 overrides, this might slow down the analysis considerably.
2024-04-16 18:41:53,805 [PID 559] WARNING  `libcst._nodes.base.CSTNode._visit_and_replace_children` has 119 overrides, this might slow down the analysis considerably.
2024-04-16 18:41:53,805 [PID 559] WARNING  `object.__eq__` has 530 overrides, this might slow down the analysis considerably.
2024-04-16 18:41:53,806 [PID 559] WARNING  `object.__hash__` has 115 overrides, this might slow down the analysis considerably.
2024-04-16 18:41:53,806 [PID 559] WARNING  `object.__init__` has 1927 overrides, this might slow down the analysis considerably.
2024-04-16 18:41:53,807 [PID 559] WARNING  `object.__ne__` has 370 overrides, this might slow down the analysis considerably.
2024-04-16 18:41:53,807 [PID 559] WARNING  `object.__repr__` has 176 overrides, this might slow down the analysis considerably.
2024-04-16 18:41:53,807 [PID 559] WARNING  `object.__setattr__` has 52 overrides, this might slow down the analysis considerably.
2024-04-16 18:41:53,808 [PID 559] WARNING  `object.__str__` has 81 overrides, this might slow down the analysis considerably.
2024-04-16 18:41:53,808 [PID 559] WARNING  `pika.amqp_object.Method.synchronous` has 66 overrides, this might slow down the analysis considerably.
2024-04-16 18:41:53,809 [PID 559] WARNING  `type.__call__` has 220 overrides, this might slow down the analysis considerably.
2024-04-16 18:41:53,809 [PID 559] WARNING  `type.__init__` has 1674 overrides, this might slow down the analysis considerably.
2024-04-16 18:41:53,809 [PID 559] WARNING  `type.__new__` has 294 overrides, this might slow down the analysis considerably.
2024-04-16 18:41:53,810 [PID 559] WARNING  `type.__or__` has 53 overrides, this might slow down the analysis considerably.
2024-04-16 18:41:53,810 [PID 559] WARNING  `typing.Collection.__len__` has 59 overrides, this might slow down the analysis considerably.
2024-04-16 18:41:53,810 [PID 559] WARNING  `typing.GenericMeta.__getitem__` has 72 overrides, this might slow down the analysis considerably.
2024-04-16 18:41:53,811 [PID 559] WARNING  `typing.Iterable.__iter__` has 57 overrides, this might slow down the analysis considerably.
2024-04-16 18:41:53,811 [PID 559] WARNING  `typing.NamedTuple.__init__` has 232 overrides, this might slow down the analysis considerably.
2024-04-16 18:41:53,811 [PID 559] PERFORMANCE  Overrides computed: 0.858s
2024-04-16 18:41:53,812 [PID 559] INFO  Indexing global constants...
2024-04-16 18:41:54,807 [PID 559] PERFORMANCE  Finished constant propagation analysis: 0.555s
2024-04-16 18:41:54,815 [PID 559] INFO  Building call graph...
2024-04-16 18:41:59,819 [PID 559] PERFORMANCE  Call graph built: 5.809s
2024-04-16 18:41:59,820 [PID 559] INFO  Computing dependencies...
2024-04-16 18:41:59,820 [PID 559] PERFORMANCE  Computed dependencies: 0.097s
2024-04-16 18:41:59,821 [PID 559] INFO  Purging shared memory...
2024-04-16 18:41:59,821 [PID 559] PERFORMANCE  Purged shared memory: 0.015s
2024-04-16 18:41:59,821 [PID 559] INFO  Purging shared memory...
2024-04-16 18:41:59,822 [PID 559] PERFORMANCE  Purged shared memory: 0.010s
2024-04-16 18:41:59,822 [PID 559] INFO  Analysis fixpoint started for 17843 overrides and 8 functions...
2024-04-16 18:42:00,820 [PID 559] PERFORMANCE  Recorded initial models: 0.951s
2024-04-16 18:42:00,821 [PID 559] INFO  Iteration #0. 4 callables [vuln.$toplevel, vuln.my_sink, vuln.my_source, vuln.vulnerable_func]
2024-04-16 18:42:00,822 [PID 559] WARNING  vuln:15:4-15:15: Revealed type for engine.execute: BoundMethod[typing.Callable(sqlalchemy.engine.base.Engine.execute)[..., unknown][[[Named(self, sqlalchemy.engine.base.Engine), Named(statement, typing.Union[sqlalchemy.sql.compiler.Compiled, sqlalchemy.sql.ddl.DDLElement, sqlalchemy.sql.elements.ClauseElement, sqlalchemy.sql.functions.FunctionElement, sqlalchemy.sql.schema.DefaultGenerator]), Variable(typing.Mapping[str, typing.Any]), Keywords(typing.Any)], sqlalchemy.engine.cursor.CursorResult][[Named(self, sqlalchemy.engine.base.Engine), Named(statement, str), Variable(typing.Any), Keywords(typing.Any)], sqlalchemy.engine.cursor.CursorResult]], sqlalchemy.engine.base.Engine]
2024-04-16 18:42:00,822 [PID 559] WARNING  vuln:21:4-21:15: Revealed type for engine.execute: BoundMethod[typing.Callable(sqlalchemy.engine.base.Engine.execute)[..., unknown][[[Named(self, sqlalchemy.engine.base.Engine), Named(statement, typing.Union[sqlalchemy.sql.compiler.Compiled, sqlalchemy.sql.ddl.DDLElement, sqlalchemy.sql.elements.ClauseElement, sqlalchemy.sql.functions.FunctionElement, sqlalchemy.sql.schema.DefaultGenerator]), Variable(typing.Mapping[str, typing.Any]), Keywords(typing.Any)], sqlalchemy.engine.cursor.CursorResult][[Named(self, sqlalchemy.engine.base.Engine), Named(statement, str), Variable(typing.Any), Keywords(typing.Any)], sqlalchemy.engine.cursor.CursorResult]], sqlalchemy.engine.base.Engine]
2024-04-16 18:42:00,823 [PID 559] INFO  Processed 4 of 4 callables
2024-04-16 18:42:00,823 [PID 559] INFO  Iteration #0, 4 callables, heap size 0.211GB took 0.03s
2024-04-16 18:42:00,823 [PID 559] INFO  Post-processing issues for multi-source rules...
[
  {
    "line": 23,
    "column": 12,
    "stop_line": 23,
    "stop_column": 17,
    "path": "vuln.py",
    "code": 5002,
    "name": "Test flow",
    "description": "Test flow [5002]: Data from [Test] source(s) may reach [Test] sink(s)",
    "define": "vuln.vulnerable_func"
  }
]
root@1ce54022191b:/usr/src/app#

Additional context
Add any other context about the problem here. (like dependencies in your venv, third party stub files being used, overall goals, etc.)

reveal_type only seems to work some of time, and I haven't been able to figure out why. #825 In the beginning when it worked, reveal_type(engine.execute) printed out ƛ vuln:21:4-21:15: Revealed type for engine.execute: unknown, which is obviously a problem.

I ran pyre infer and pyre infer -i --annotate-from-existing-stubs to try to fix the types. It added some, and explicitly assigned a type to engine as engine: Engine = create_engine('sqlite:///test.db'). Now reveal_type prints out

ƛ  vuln:21:4-21:15: Revealed type for engine.execute: BoundMethod[typing.Callable(sqlalchemy.engine.base.Engine.execute)[..., unknown][[[Named(self, sqlalchemy.engine.base.Engine), Named(statement, typing.Union[sqlalchemy.sql.compiler.Compiled, sqlalchemy.sql.ddl.DDLElement, sqlalchemy.sql.elements.ClauseElement, sqlalchemy.sql.functions.FunctionElement, sqlalchemy.sql.schema.DefaultGenerator]), Variable(typing.Mapping[str, typing.Any]), Keywords(typing.Any)], sqlalchemy.engine.cursor.CursorResult][[Named(self, sqlalchemy.engine.base.Engine), Named(statement, str), Variable(typing.Any), Keywords(typing.Any)], sqlalchemy.engine.cursor.CursorResult]], sqlalchemy.engine.base.Engine]

That... seems like it should work.

The engine.execute call should match one of these sink rules:

def sqlalchemy.engine.base.Engine.execute(self, __object: TaintSink[SQL], *multiparams, object: TaintSink[SQL], statement: TaintSink[SQL], **params): ...
def sqlalchemy.engine.base.Engine.execute(self, statement: TaintSink[SQL], *multiparams: TaintSink[SQL], **params: TaintSink[SQL]): ...

The second is mine because I was wondering if the one supplied in the repo was somehow wrong. Neither one works.

I have types-SQLAlchemy==1.4.52 to get types for the older SQLAlchemy. I did try a bit with 2.0 and couldn't get it to work.

Any pointers would be appreciated. What version of SQLAlchemy should I be trying? Are the flows here expected to work or no?
@cyounkins
Copy link
Author

I did find this commit that maybe breaks SQLAlchemy 1.4? c93e0a1

@arthaud
Copy link
Contributor

arthaud commented Apr 18, 2024

Hi, thanks for reaching out.
I am able to reproduce, I will need more time to look into this.

@arthaud
Copy link
Contributor

arthaud commented Apr 18, 2024 •
edited

This is a simple mistake.

my_source() returns a source with kind Test.
engine.execute() has a sink with kind SQL.
There is a rule from source Test to sink Test: https://github.com/cyounkins/pysa-testing/blob/63395fc6b3ba826b1ebb9dec1c55abcadf4c7622/stubs/taint.config#L184-L192
but there is no rule for source Test to sink SQL. There is only one for UserControlled to SQL: https://github.com/cyounkins/pysa-testing/blob/63395fc6b3ba826b1ebb9dec1c55abcadf4c7622/stubs/taint.config#L197-L205
If I change that rule to also accept Test as a source, it does find the flows:

[
  {
    "line": 20,
    "column": 28,
    "stop_line": 20,
    "stop_column": 33,
    "path": "vuln.py",
    "code": 5005,
    "name": "SQL injection.",
    "description": "SQL injection. [5005]: Data from [Test] source(s) may reach [SQL] sink(s)",
    "define": "vuln.vulnerable_func"
  },
  {
    "line": 20,
    "column": 28,
    "stop_line": 20,
    "stop_column": 33,
    "path": "vuln.py",
    "code": 5005,
    "name": "SQL injection.",
    "description": "SQL injection. [5005]: Data from [Test] source(s) may reach [SQL] sink(s)",
    "define": "vuln.vulnerable_func"
  },
  {
    "line": 23,
    "column": 12,
    "stop_line": 23,
    "stop_column": 17,
    "path": "vuln.py",
    "code": 5002,
    "name": "Test flow",
    "description": "Test flow [5002]: Data from [Test] source(s) may reach [Test] sink(s)",
    "define": "vuln.vulnerable_func"
  }
]

@ljw1004 ljw1004 closed this as completed Apr 19, 2024
@cyounkins
Copy link
Author

Ahh thank you! I apologize for the waste of time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@cyounkins @arthaud @ljw1004