Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove variable/column map redundancy (WIP) #281

Closed
wants to merge 9 commits into from
Closed

Remove variable/column map redundancy (WIP) #281

wants to merge 9 commits into from

Commits on Sep 4, 2019

  1. Don't keep aborted Operations in the cache

    The theory behind caching aborted Operations was that they would fail
    again anyway. However, under memory pressure Operations may fail due to
    failing allocations and will succeed in the future once memory pressure
    drops, if we kept them in the cache we wouldn't try again.
    
    While we are at it, improve wording and use an extra abort() method on
    Operation.
    niklas88 committed Sep 4, 2019
    Configuration menu
    Copy the full SHA
    c440049 View commit details
    Browse the repository at this point in the history
  2. Add basic pinning functionality to LRUCache

    this allows to mark elements as pinned, meaning they will not be dropped
    due to capacity constraints or clear() but only with an explicit erase
    (e.g. as used for aborted queries).
    niklas88 committed Sep 4, 2019
    Configuration menu
    Copy the full SHA
    08fbfd2 View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2019

  1. unique_ptr root Operation in QueryExecutionTree

    The Operation is owned so it should be a unique_ptr not shared_ptr. This
    will hopefully save quite a few memory barriers (for atomic ref
    counting) and is cleaner.
    
    We also prevent the copying of QueryExecutionTrees which was previously
    done in several places and may be expensive but is definitely
    unexpected.
    niklas88 committed Sep 6, 2019
    Configuration menu
    Copy the full SHA
    c795eb9 View commit details
    Browse the repository at this point in the history
  2. Fix wrong variable columns in IndexScan

    They are actually kept in the order of the triple file as seen by the
    setVariableColumn() calls in QueryPlanner.cpp
    niklas88 committed Sep 6, 2019
    Configuration menu
    Copy the full SHA
    bace316 View commit details
    Browse the repository at this point in the history
  3. Turn QueryExecutionContext into a query context

    The name always suggested that it carried context per query while in
    fact it held the SubtreeCache which should really live for many queries.
    Since it is indeed useful to have a per query context we move the
    SubtreeCache outside the QueryExecutionContext and make it truly per
    query.
    niklas88 committed Sep 6, 2019
    Configuration menu
    Copy the full SHA
    02088a8 View commit details
    Browse the repository at this point in the history
  4. Use QueryExecutionContext to track cache pinning

    This allows to pin all subtrees of a query to the cache and have that as
    a per query option. On the QLever server this is exposed as the
    "pinsubtrees" URL param. If set to true all subtrees in the query will
    be cached. This is especially useful for completion queries and the
    like.
    niklas88 committed Sep 6, 2019
    Configuration menu
    Copy the full SHA
    1c85b1f View commit details
    Browse the repository at this point in the history
  5. Only store each key once in the LRUCache

    In the entries we now only store a pointer to the key inside the hash
    map
    niklas88 committed Sep 6, 2019
    Configuration menu
    Copy the full SHA
    4db3815 View commit details
    Browse the repository at this point in the history
  6. Revert "Only store each key once in the LRUCache"

    This reverts commit 9209326.
    niklas88 committed Sep 6, 2019
    Configuration menu
    Copy the full SHA
    605d165 View commit details
    Browse the repository at this point in the history
  7. Reduce redundancy QueryExectuionTree and Operation (WIP)

    Both currently keep separate _variableColumnMap hash maps. This commit
    tries to always init the one in the QueryExecutionTree from the
    _rootOperation we have anyway. Sadly it seems they aren't just redundant
    but diverge somewhere as this commit currently fails to pass the e2e
    tests.
    niklas88 committed Sep 6, 2019
    Configuration menu
    Copy the full SHA
    905de53 View commit details
    Browse the repository at this point in the history