Beautiful relative imports + Edited README + Fixed Travis CI :D #23

KevinHock · 2017-02-24T01:00:42Z

Also, we should add pylint to pre-commit hooks.

KevinHock · 2017-02-24T17:02:13Z

Current pyt notes:

DONE Fix 'sql/sqli.py' test
Add directory option from JunkHacker/bandit
JSON output file

Thalmann

67d2b1f:
A lot of the stuff pylint does is great.
But as you can see in the github_search module pylint does not track what names are in the globals. This means when going from module.class_name import to from module import class_name .

Stefan and i agreed on that we want PyT to be able to do what pylint does but better. As PyT actually tracks the global names so we can manage the above problem.
More on that in a task i will define.

Thalmann · 2017-02-26T14:36:18Z

pyt/github_search.py

-import requests
-import repo_runner
+from reaching_definitions_taint import ReachingDefinitionsTaintAnalysis
+from repo_runner import add_repo_to_csv, NoEntryPathError, Repo


This one breaks the github_search module as there is a Repo defined in this module aswell.
This means we have two Repo definitions in globals so the last imported overwrites the first.

Thalmann · 2017-02-26T14:37:11Z

pyt/github_search.py

@@ -209,7 +210,7 @@ def scan_github(search_string, start_date, analysis_type, analyse_repo_func, csv
                      Languages.python, repo)
            s = SearchCode(q)
            if s.results:
-                r = repo_runner.Repo(repo.url)


This is the line we want back.

Thalmann · 2017-02-26T14:38:16Z

pyt/github_search.py

@@ -209,7 +210,7 @@ def scan_github(search_string, start_date, analysis_type, analyse_repo_func, csv
                      Languages.python, repo)
            s = SearchCode(q)
            if s.results:
-                r = repo_runner.Repo(repo.url)
+                r = Repo(repo.url)


This Repo is referencing the definition of Repo in this module and not the one in the repo_runner.

Thalmann · 2017-02-26T14:38:36Z

pyt/github_search.py

    scan_github('flask', ReachingDefinitionsTaintAnalysis)
    exit()
    q = Query(SEARCH_REPO_URL, 'flask')
    s = SearchRepo(q)
    for repo in s.results[:3]:
        q = Query(SEARCH_CODE_URL, 'app = Flask(__name__)', Languages.python, repo)
        s = SearchCode(q)
-        r = repo_runner.Repo(repo.url)
+        r = Repo(repo.url)


Same as above.

Good catch, I will change it.

Thalmann · 2017-02-26T14:47:31Z

997796d
The additions to the README is great, we definitely need more of this. And some stuff from our master thesis can probably be used.
Regarding the changes where we run PyT as a module, why do you think this is better?

KevinHock · 2017-02-26T15:40:11Z

It's necessary for the relative imports to work, I don't love having to use -m but the relative imports are worth it IMO and there's no longer any sys ../pyt hacks.

Regarding the switch to __main__.py from pyt.py and test.py, that's so we can do e.g. -m pyt instead of pyt.pyt

I'm on my phone so I don't have the succinct stackoverflow link I had yesterday but David Beazley's awesome talk may convince you, around 41 minutes in https://www.youtube.com/watch?v=0oTh1CXRaQ0&t=2490

[edit]
Here are some other links of relative import problems needing the -m http://stackoverflow.com/questions/11536764/how-to-fix-attempted-relative-import-in-non-package-even-with-init-py http://stackoverflow.com/questions/72852/how-to-do-relative-imports-in-python/73149#73149

Grazfather · 2017-02-27T16:11:37Z

README.md

+
+Clone the project into the directory
+
+`git clone https://github.com/python-security/pyt.git`


This will make a pyt/pyt dir. You can clone right from ~ and it'll create a pyt dir.

I realize pyt/pyt is awkward, but I want python3 -m venv ~/somefolder/ that's not ~ or the Git Repository itself to hold the env files. Do you think I should I change it to mkdir ~/somefolder to make it less confusing?

Naw it works, just wanted to make sure you were accounting for it. You could change the README so that it's clear that the directory for the venv doesn't matter

Good point :)

Grazfather · 2017-02-27T16:12:15Z

README.md

+`pip install -r requirements.txt`
+
+`pip list` sample output
+```


Want an extra blank line here

Grazfather · 2017-02-27T16:12:30Z

README.md

+`pip list` sample output
+```
+gitdb (0.6.4)
+


extra lines shouldn't be needed if the triple backticks work

I didn't know that, thanks.

Grazfather · 2017-02-27T16:13:16Z

example/cfg_example.py

-import os
-import sys
-
-sys.path.insert(0, os.path.abspath('../pyt'))


Great to get rid of this

Grazfather · 2017-02-27T16:14:41Z

pyt/analysis_base.py

@@ -1,5 +1,5 @@
 """Thos module contains a base class for the analysis component used in PyT."""


Grazfather · 2017-02-27T16:15:13Z

pyt/base_cfg.py

@@ -166,12 +166,12 @@ def __init__(self, label, left_hand_side, right_hand_side_variables, ast_node, *
            right_hand_side_variables(list[str]): A list of variables on the right hand side.
            line_number(Optional[int]): The line of the expression the Node represents.
        """
-        super(ReturnNode, self).__init__(label, left_hand_side, ast_node, right_hand_side_variables, line_number=line_number, path=path)    
+        super(ReturnNode, self).__init__(label, left_hand_side, ast_node, right_hand_side_variables, line_number=line_number, path=path)


Why not use python3 super syntax?

TIL :) Thanks Graz

Thalmann · 2017-02-28T17:52:42Z

Great talk, i have watched it before.
But as this is the first time i am packaging a python program i was following the pypi guide to get it there.
I have some local commits with that, but if this method with main.py is possible to get PyT on pypi i will support the changes.
Is this possible? :)

KevinHock · 2017-02-28T21:57:55Z

Okay, that is the last commit (barring a fix from feedback.) That I will make, this PR is not getting any bigger :)

re:"PyT on pypi" I do not know but I will look into it.

Thalmann · 2017-03-02T06:50:55Z

I think it will not be a problem or this is definitely not a step in the wrong direction in regards to getting PyT on pypi :)

KevinHock added 2 commits February 23, 2017 19:57

[style]pretty imports and README

67d2b1f

[tests]Fixed sqli results in results file

480acb7

KevinHock changed the title ~~[style]pretty imports and README~~ [style]pretty imports and README + Fixed Travis CI :D Feb 24, 2017

KevinHock mentioned this pull request Feb 24, 2017

Check why travis is failing #16

Closed

[style]relative imports [readme]added virtualenv guide

997796d

KevinHock mentioned this pull request Feb 25, 2017

Virtual env setup guide #9

Closed

KevinHock changed the title ~~[style]pretty imports and README + Fixed Travis CI :D~~ [style]beautiful relative imports and README + Fixed Travis CI :D Feb 25, 2017

KevinHock changed the title ~~[style]beautiful relative imports and README + Fixed Travis CI :D~~ [style]beautiful relative imports + Edited README + Fixed Travis CI :D Feb 25, 2017

KevinHock changed the title ~~[style]beautiful relative imports + Edited README + Fixed Travis CI :D~~ Beautiful relative imports + Edited README + Fixed Travis CI :D Feb 25, 2017

Thalmann reviewed Feb 26, 2017

View reviewed changes

KevinHock and others added 2 commits February 26, 2017 22:57

Fixed my Repo mistake

a964d2a

Merge branch 'master' into master

7fadc22

Grazfather reviewed Feb 27, 2017

View reviewed changes

KevinHock added 2 commits February 27, 2017 20:05

[style]Graz Feedback

1d20f9e

Merge branch 'master' of https://github.com/kevinhock/pyt

b06f3db

[style]style, grammar, and clarifications

755fd97

Thalmann merged commit 797adbf into python-security:master Mar 2, 2017

KevinHock mentioned this pull request May 4, 2017

sqli.py test fails on non-3.6 versions of python #40

Closed


		Clone the project into the directory

		`git clone https://github.com/python-security/pyt.git`

		@@ -1,5 +1,5 @@
		"""Thos module contains a base class for the analysis component used in PyT."""

Beautiful relative imports + Edited README + Fixed Travis CI :D #23

Beautiful relative imports + Edited README + Fixed Travis CI :D #23

Uh oh!

Conversation

KevinHock commented Feb 24, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KevinHock commented Feb 24, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Thalmann left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Thalmann commented Feb 26, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KevinHock commented Feb 26, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Thalmann commented Feb 28, 2017

Uh oh!

KevinHock commented Feb 28, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Thalmann commented Mar 2, 2017

Uh oh!

Uh oh!

KevinHock commented Feb 24, 2017 •

edited

Loading

KevinHock commented Feb 24, 2017 •

edited

Loading

Thalmann left a comment •

edited

Loading

Thalmann commented Feb 26, 2017 •

edited

Loading

KevinHock commented Feb 26, 2017 •

edited

Loading

KevinHock commented Feb 28, 2017 •

edited

Loading