Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WIP] implement match backend #22

Merged
merged 68 commits into from
Nov 8, 2016
Merged

[WIP] implement match backend #22

merged 68 commits into from
Nov 8, 2016

Commits on Nov 8, 2016

  1. [WIP] implement match backend 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    a6db5b2 View commit details
    Browse the repository at this point in the history

  2. add celery and empty tasks file 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    bd3f58c View commit details
    Browse the repository at this point in the history

  3. Try my multi-table inheritence idea 

    See http://stackoverflow.com/questions/39146334 for implementation details
and community scrutiny

Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    a878d5d View commit details
    Browse the repository at this point in the history

  4. Merge Task model with async tasks and create proper views 

    Add basic statuses for Tasks, simple update in match task and serializer exposure
Make Task.project optional, defaulting to file.project when available
make task id a uuid field and make it an additional field instead of the primary key

Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    09db5cd View commit details
    Browse the repository at this point in the history

  5. Create task when function data upload is done 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    ad24f3a View commit details
    Browse the repository at this point in the history

  6. Create base MatchAction class both MatchAll and MatchFunction inherit 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    7eb3470 View commit details
    Browse the repository at this point in the history

  7. call get_functions() in MatchAllAction.get_functions_count 

    plus turn methods into staticmethods and classmethods

Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    ccf5a83 View commit details
    Browse the repository at this point in the history

  8. initialize MatchAction params in __init__ 

    and remove unnecessary parenthesis

Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    34a4280 View commit details
    Browse the repository at this point in the history

  9. add match dialog 

    * match dialog fixes and connections

* Add BaseDialog radio group utility and use that in MatchDialog

* Also make MatchDialog functional

* Merge match all and match function actions to match

Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    36126ae View commit details
    Browse the repository at this point in the history

  10. fix minor codacy issues 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    25243e0 View commit details
    Browse the repository at this point in the history

  11. fix flake8 issues 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    3122721 View commit details
    Browse the repository at this point in the history

  12. Implement simple match sources 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    8d863f8 View commit details
    Browse the repository at this point in the history

  13. Fix single source function match 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    a290903 View commit details
    Browse the repository at this point in the history

  14. Add file_id to vectors and an empty loop that pulls the vectors 

    Signed-off-by: Nir Izraeli <nir@sentinel-labs.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    f2ec53e View commit details
    Browse the repository at this point in the history

  15. Add progress fields to task 

    Signed-off-by: Nir Izraeli <nir@sentinel-labs.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    25ddae6 View commit details
    Browse the repository at this point in the history

  16. fix spaces around '+' in tasks.py 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    1401ac1 View commit details
    Browse the repository at this point in the history

  17. Fix vector file field population 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    e202739 View commit details
    Browse the repository at this point in the history

  18. Add match task exection's progress bar

    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    1d95efb View commit details
    Browse the repository at this point in the history

  19. Fix target_vectors filter 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    35f2a4a View commit details
    Browse the repository at this point in the history

  20. Add task execution progress report and progress bar 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    19cffe4 View commit details
    Browse the repository at this point in the history

  21. use BaseDialog's bottom layout 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    69b48b6 View commit details
    Browse the repository at this point in the history

  22. explicitly catch all exceptions and raise on MatchAction.perform_task 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    d7d17af View commit details
    Browse the repository at this point in the history

  23. Add task to admin panel 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    d7b4100 View commit details
    Browse the repository at this point in the history

  24. fix two base_layout issues 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    eac0bea View commit details
    Browse the repository at this point in the history

  25. accept null project when creating task 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    a7f6de7 View commit details
    Browse the repository at this point in the history

  26. Accept progress bar when task is complete 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    f73894d View commit details
    Browse the repository at this point in the history

  27. Bulk delayed instance requests to speed up the collection phase 

    Which now takes only a few seconds on a 2k functions idb

Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    b02ddd8 View commit details
    Browse the repository at this point in the history

  28. simplify get_functions_count 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    f2244cd View commit details
    Browse the repository at this point in the history

  29. Create serverside vector classes to reduce boilerplate 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    94a7d75 View commit details
    Browse the repository at this point in the history

  30. safer timer use in match action 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    b05bf87 View commit details
    Browse the repository at this point in the history

  31. fix codacy minor issues 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    4df4f20 View commit details
    Browse the repository at this point in the history

  32. add type and score to Match Model 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    4578c45 View commit details
    Browse the repository at this point in the history

  33. implement match for identity functions 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    975b184 View commit details
    Browse the repository at this point in the history

  34. make Vector class inherit and remove VectorHash from list of avaiable… 

    … hashes

Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    4ad58b4 View commit details
    Browse the repository at this point in the history

  35. Add space 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    b8958ee View commit details
    Browse the repository at this point in the history

  36. HistogramVector skeleton 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    d341729 View commit details
    Browse the repository at this point in the history

  37. create Match objects when successfully matching 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    042a105 View commit details
    Browse the repository at this point in the history

  38. Split id to match_type and vector_type, to allow several matches usin… 

    …g the same vector type

Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    8915bfe View commit details
    Browse the repository at this point in the history

  39. Begining of histogram matching 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    3b4a9d8 View commit details
    Browse the repository at this point in the history

  40. Add scipy and sklearn to requirements 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    9c71fc5 View commit details
    Browse the repository at this point in the history

  41. Mark task as failed on exception (and reraise it) 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    85b690e View commit details
    Browse the repository at this point in the history

  42. move Match object creation to tasks.py to avoid cyclic import 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    f3e9aac View commit details
    Browse the repository at this point in the history

  43. Add miniconda install to travis when testing server 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    eb3320e View commit details
    Browse the repository at this point in the history

  44. finish HistogramVector.match 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    5e24543 View commit details
    Browse the repository at this point in the history

  45. Remove python 3.2 from travis, fix codacy issues 

    Conda doesn't support python 3.2

Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    60b8e11 View commit details
    Browse the repository at this point in the history

  46. Better seperation between server matching algos and database vectors 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    afd5b63 View commit details
    Browse the repository at this point in the history

  47. Fix new action/dialog related issues introduced in 6913865 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    144e51a View commit details
    Browse the repository at this point in the history

  48. Ditch my through replacement attempt, make matching flow work 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    547b62f View commit details
    Browse the repository at this point in the history

  49. make task progress bar visible without requiring actual progress 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    f529d39 View commit details
    Browse the repository at this point in the history

  50. more progress on making histogram matches work 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    db2b09b View commit details
    Browse the repository at this point in the history

  51. Time each match type 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    775bd41 View commit details
    Browse the repository at this point in the history

  52. set Task model progress_max nullable and handle optional None value i… 

    …n task progress bar

Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    fbc8078 View commit details
    Browse the repository at this point in the history

  53. more fixes to histogram match backend 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    072ccc9 View commit details
    Browse the repository at this point in the history

  54. Another attempt at speeding up the matches process 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    d8edd71 View commit details
    Browse the repository at this point in the history

  55. more robust status handling in task progress query 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    6633342 View commit details
    Browse the repository at this point in the history

  56. send more parameters in a match task creation 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    e1bb374 View commit details
    Browse the repository at this point in the history

  57. More adjustments to the source/target/methods parameters 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    7718f39 View commit details
    Browse the repository at this point in the history

  58. Make item selection combobox a dialog.Base method 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    13b696f View commit details
    Browse the repository at this point in the history

  59. Add optional choice-specific additional input to the radio group crea… 

    …tion function

Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    e489af6 View commit details
    Browse the repository at this point in the history

  60. Send selected target_project and target_file to server 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    27da9ff View commit details
    Browse the repository at this point in the history

  61. Handle target_project and target_file match parameters on backend 

    plus some small code simplifications

Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    90ceb59 View commit details
    Browse the repository at this point in the history

  62. Allow excuding values in create_item_select and exclude current file … 

    …in target_file selection

Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    2c7bf08 View commit details
    Browse the repository at this point in the history

  63. import qt from idasix 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    1b80518 View commit details
    Browse the repository at this point in the history

  64. Add support for range and single function sources 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    2d4521a View commit details
    Browse the repository at this point in the history

  65. self review fixes part one 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    b8ef12f View commit details
    Browse the repository at this point in the history

  66. Prevent editing target and source info in existing task 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    c31224f View commit details
    Browse the repository at this point in the history

  67. Replace create_item_select and create_radio_select with QItemSelect a… 

    …nd QRadioGroup

Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    6cb948c View commit details
    Browse the repository at this point in the history

  68. Split matches.py to entire directory of match type definitions 

    Signed-off-by: Nir Izraeli <nirizr@gmail.com>
    @nirizr
    nirizr authored and Nir Izraeli committed Nov 8, 2016
    Configuration menu
    Copy the full SHA
    41ccc63 View commit details
    Browse the repository at this point in the history