Skip to content

Conversation

@natiiix
Copy link
Collaborator

@natiiix natiiix commented Aug 2, 2019

  • Fix docstyle (except for Iodine algorithm's files)
  • Add missing code documentation, especially to classes and public functions
  • Make database structure more convenient and detailed
  • Remove global config
  • At least partially prevent accidental parallel analysis of the same repository (this could easily lead to DoS attacks)
  • Give the user more useful messages
  • Improve logging of web app
  • Split origin string into a file path, a line number and a column offset (=> NodeOrigin class) - this will be particularly useful later, when/if a code viewer is implemented
  • Move argument parsing from the engine to CLI
  • Use urllib.parse.urlparse() instead of overly complicated read-only regular expressions (regexps are still used, but not for the whole URL, which is something of an improvement)
  • Add PostgreSQL library to requirements.txt => close Add postgres package to requirements.txt #92
  • Make all errors package imports absolute => close Avoid relative imports by making subdirectory "errors" a module #82
  • Replace engine.utils.list_tools.flatten() with itertools.chain.from_iterable() => close Replace function for list flatenning with standard library equivalent #95

natiiix added 30 commits July 30, 2019 13:23
@natiiix natiiix changed the title [WIP] Restructure database and stabilize back-end logic Restructure database and stabilize back-end logic Aug 5, 2019
Copy link
Collaborator

@tisnik tisnik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice, I've just dropped a couple of comments, but nothing that prevents the merge :)

tuple[string] -- Tuple of local repository paths.
"""
if len(argv) == 1 or (len(argv) == 2 and argv[1] in ['-h', '--help', '--usage']):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now you might start to accept, that the proposed argparse module might be better approach :)

https://docs.python.org/3.5/library/argparse.html

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it most certainly is, especially once we begin to allow the user to specify options for the algorithms. I plan on switching to it as soon as I have some tests for the core/engine.

class DetectionResult:
"""
Represents the final result of a detection query.
Represent the final result of a detection query.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd probably write:

Representation of the ...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It sounds awful, but I wanted to avoid having "of the"/"of a" twice in such a short sentence. The final result of a detection query. might work even better. Also, this change was an unfortunate and unintentional consequence of me using regular expressions to fix >500 docstyle violations.

"""
flat = []

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

chain.from_iterable(listOfLists)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw itertools.chain.from_iterable() on StackOverflow, but I didn't want to add an import just for this, because the performance difference is negligible. Somehow I didn't realize itertools are in the standard library. I will change it within this PR. #95

@natiiix natiiix merged commit 9e8f4bb into iresbaylor:master Aug 6, 2019
@natiiix natiiix deleted the db-repo-cloner-rework branch August 6, 2019 10:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request

Projects

None yet

2 participants