Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimize deduplication while processing held tasks #2203

Open
wants to merge 39 commits into
base: master
Choose a base branch
from
Open

Optimize deduplication while processing held tasks #2203

wants to merge 39 commits into from

Commits on Jan 18, 2023

  1. Stored groups to avoid recomputation. Brings down held task computati… 

    …on time to 15% of earlier ( 6X speedup )
    @rahul-privado
    rahul-privado committed Jan 18, 2023
    Configuration menu
    Copy the full SHA
    e62f85b View commit details
    Browse the repository at this point in the history

  2. Code betterment

    @rahul-privado
    rahul-privado committed Jan 18, 2023
    Configuration menu
    Copy the full SHA
    31f968e View commit details
    Browse the repository at this point in the history

  3. Removed unused function

    @rahul-privado
    rahul-privado committed Jan 18, 2023
    Configuration menu
    Copy the full SHA
    ecc40e8 View commit details
    Browse the repository at this point in the history

Commits on Jan 19, 2023

  1. Merge branch 'master' into held-tasks-opti

    @rahul-privado
    rahul-privado committed Jan 19, 2023
    Configuration menu
    Copy the full SHA
    9e4042c View commit details
    Browse the repository at this point in the history

Commits on Jan 20, 2023

  1. More compact code for old

    @rahul-privado
    rahul-privado committed Jan 20, 2023
    Configuration menu
    Copy the full SHA
    e6b18a8 View commit details
    Browse the repository at this point in the history

  2. Revert "More compact code for old" 

    This reverts commit e6b18a8.
    @rahul-privado
    rahul-privado committed Jan 20, 2023
    Configuration menu
    Copy the full SHA
    a7b9aea View commit details
    Browse the repository at this point in the history

  3. Cache table entry list

    @rahul-privado
    rahul-privado committed Jan 20, 2023
    Configuration menu
    Copy the full SHA
    0269925 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into held-tasks-opti

    @rahul-privado
    rahul-privado committed Jan 20, 2023
    Configuration menu
    Copy the full SHA
    5378828 View commit details
    Browse the repository at this point in the history

  5. Formatted the code

    @rahul-privado
    rahul-privado committed Jan 20, 2023
    Configuration menu
    Copy the full SHA
    e9b24da View commit details
    Browse the repository at this point in the history

Commits on Jan 24, 2023

  1. Merge branch 'joernio:master' into held-tasks-opti

    @rahul-privado
    rahul-privado committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    1457a60 View commit details
    Browse the repository at this point in the history

Commits on Jan 25, 2023

  1. 1. Properly concatenated the lists to avoid missing TableEntries 

    2. Removed caching for merge list since the new table entries would need a recomputation
    @rahul-privado
    rahul-privado committed Jan 25, 2023
    Configuration menu
    Copy the full SHA
    42daee6 View commit details
    Browse the repository at this point in the history

  2. Removed sorting O(n^2) worst case and made it a linear search O(n) to… 

    … find the max elements
    @rahul-privado
    rahul-privado committed Jan 25, 2023
    Configuration menu
    Copy the full SHA
    214edfc View commit details
    Browse the repository at this point in the history

Commits on Jan 27, 2023

  1. Unique hash for each TableEntry

    @rahul-privado
    rahul-privado committed Jan 27, 2023
    Configuration menu
    Copy the full SHA
    a252feb View commit details
    Browse the repository at this point in the history

Commits on Jan 30, 2023

  1. Lazy computation of Sha1 hash

    @rahul-privado
    rahul-privado committed Jan 30, 2023
    Configuration menu
    Copy the full SHA
    22f0cb1 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into held-tasks-opti

    @rahul-privado
    rahul-privado committed Jan 30, 2023
    Configuration menu
    Copy the full SHA
    bbd65f0 View commit details
    Browse the repository at this point in the history

  3. Reintroduced caching of table entry hashes

    @rahul-privado
    rahul-privado committed Jan 30, 2023
    Configuration menu
    Copy the full SHA
    9dfadd7 View commit details
    Browse the repository at this point in the history

  4. Variable renaming

    @rahul-privado
    rahul-privado committed Jan 30, 2023
    Configuration menu
    Copy the full SHA
    16323fd View commit details
    Browse the repository at this point in the history

  5. Formatted the code

    @rahul-privado
    rahul-privado committed Jan 30, 2023
    Configuration menu
    Copy the full SHA
    34372f4 View commit details
    Browse the repository at this point in the history

  6. Moved hash computation to a different function

    @rahul-privado
    rahul-privado committed Jan 30, 2023
    Configuration menu
    Copy the full SHA
    04cc73d View commit details
    Browse the repository at this point in the history

  7. Merging code improvement

    @rahul-privado
    rahul-privado committed Jan 30, 2023
    Configuration menu
    Copy the full SHA
    f1ef699 View commit details
    Browse the repository at this point in the history

  8. Filter on max length

    @rahul-privado
    rahul-privado committed Jan 30, 2023
    Configuration menu
    Copy the full SHA
    e5afde1 View commit details
    Browse the repository at this point in the history

  9. Filtering while appendin new list

    @rahul-privado
    rahul-privado committed Jan 30, 2023
    Configuration menu
    Copy the full SHA
    ee78f6b View commit details
    Browse the repository at this point in the history

  10. Accounted for new max entries

    @rahul-privado
    rahul-privado committed Jan 30, 2023
    Configuration menu
    Copy the full SHA
    9ebba8f View commit details
    Browse the repository at this point in the history

  11. Removed SHA/MD5 usage since it is expensive

    @rahul-privado
    rahul-privado committed Jan 30, 2023
    Configuration menu
    Copy the full SHA
    83dc73f View commit details
    Browse the repository at this point in the history

  12. Used minBy() instead of sorting

    @rahul-privado
    rahul-privado committed Jan 30, 2023
    Configuration menu
    Copy the full SHA
    d4eba26 View commit details
    Browse the repository at this point in the history

Commits on Jan 31, 2023

  1. Updated group list map that was missed earlier

    @rahul-privado
    rahul-privado committed Jan 31, 2023
    Configuration menu
    Copy the full SHA
    b56b25f View commit details
    Browse the repository at this point in the history

  2. 1. Converted merge list to hash map 

    2. Used parallelization
    @rahul-privado
    rahul-privado committed Jan 31, 2023
    Configuration menu
    Copy the full SHA
    a866227 View commit details
    Browse the repository at this point in the history

  3. Better sync of the critical section

    @rahul-privado
    rahul-privado committed Jan 31, 2023
    Configuration menu
    Copy the full SHA
    125e30b View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into held-tasks-opti

    @rahul-privado
    rahul-privado committed Jan 31, 2023
    Configuration menu
    Copy the full SHA
    901c50e View commit details
    Browse the repository at this point in the history

Commits on Feb 1, 2023

  1. Removed merge list map since it is no longer needed

    @rahul-privado
    rahul-privado committed Feb 1, 2023
    Configuration menu
    Copy the full SHA
    f614366 View commit details
    Browse the repository at this point in the history

  2. Introduced RW lock for proper protection

    @rahul-privado
    rahul-privado committed Feb 1, 2023
    Configuration menu
    Copy the full SHA
    328c287 View commit details
    Browse the repository at this point in the history

  3. Parallization in the outer loop

    @rahul-privado
    rahul-privado committed Feb 1, 2023
    Configuration menu
    Copy the full SHA
    8375599 View commit details
    Browse the repository at this point in the history

  4. Removed outer loop parallelism due to race issues

    @rahul-privado
    rahul-privado committed Feb 1, 2023
    Configuration menu
    Copy the full SHA
    c4ce927 View commit details
    Browse the repository at this point in the history

  5. Parallel group computation

    @rahul-privado
    rahul-privado committed Feb 1, 2023
    Configuration menu
    Copy the full SHA
    febe497 View commit details
    Browse the repository at this point in the history

  6. Line formatting

    @rahul-privado
    rahul-privado committed Feb 1, 2023
    Configuration menu
    Copy the full SHA
    39db644 View commit details
    Browse the repository at this point in the history

Commits on Feb 2, 2023

  1. Made priority computation more accurate

    @rahul-privado
    rahul-privado committed Feb 2, 2023
    Configuration menu
    Copy the full SHA
    5ed522d View commit details
    Browse the repository at this point in the history

Commits on Feb 6, 2023

  1. Merge branch 'master' into held-tasks-opti

    @rahul-privado
    rahul-privado committed Feb 6, 2023
    Configuration menu
    Copy the full SHA
    dba8d1d View commit details
    Browse the repository at this point in the history

Commits on Feb 7, 2023

  1. Merge branch 'joernio:master' into held-tasks-opti

    @rahul-privado
    rahul-privado committed Feb 7, 2023
    Configuration menu
    Copy the full SHA
    70a53b8 View commit details
    Browse the repository at this point in the history

Commits on Feb 10, 2023

  1. Merge branch 'joernio:master' into held-tasks-opti

    @rahul-privado
    rahul-privado committed Feb 10, 2023
    Configuration menu
    Copy the full SHA
    3a01ade View commit details
    Browse the repository at this point in the history