Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Experimental feature to filter duplicate query segments, refs 2099 #2176

Merged
merged 1 commit into from Jan 15, 2017

Conversation

Projects
None yet
2 participants
@mwjames
Copy link
Contributor

commented Jan 15, 2017

This PR is made in reference to: #2099

This PR addresses or contains:

  • Add Description::getFingerprint to compute a signature that allows to compare description instances
  • Add Description::setMembership to track members of a distinct segment within
    the description instance
  • Add smwgQFilterDuplicates as experimental setting to optimize and filter duplicate conditions (Foo OR Foo === Foo, Foo.Bar OR Foo remains) from a query segment

This PR includes:

  • Tests (unit/integration)
  • CI build passed
Experimental feature to filter duplicate query segments, refs 2099
- Add `Description::getFingerprint` to compute a signature that allows to compare description instances
- Add `Description::setMembership` to track members of a distinct segment witin
the description instance
- Add `smwgQFilterDuplicates` as experimental setting to optimize and
filter duplicate conditions (Foo OR Foo === Foo, Foo.Bar OR Foo remains)
from a query segment

@mwjames mwjames added the enhancement label Jan 15, 2017

@mwjames mwjames added this to the SMW 2.5 milestone Jan 15, 2017

@mwjames

This comment has been minimized.

Copy link
Contributor Author

commented Jan 15, 2017

@kghbln In #2099 with the condition stated the query signature would change and in order for smwgQFilterDuplicates to work sufficiently, I had to re-adjust how the signature (aka fingerprint) is computed therefore we should see on the sandbox a small spike on new query ID's given the new model (smwgQueryResultCacheType or smwgQFilterDuplicates would force the usage of the new model, yet if both are disabled then everything remains the same for the user and the system).

The idea behind smwgQFilterDuplicates is to filter some query segments (part that represents individual SQL statements of a query) that can be duplicate and would create an additional temp table which in return would lengthen the time the SQL engine needs to answer the query.

smwgQFilterDuplicates is an experimental feature and if enabled tries to match the fingerprint of a description and replaces the reference if matched before the QueryEngine becomes aware of the segments.

The impact of described optimization is unknown.

@mwjames mwjames merged commit f2d821f into master Jan 15, 2017

3 checks passed

Scrutinizer 15 new issues, 30 updated code elements
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details

@mwjames mwjames deleted the filter-dups branch Jan 15, 2017

@kghbln

This comment has been minimized.

Copy link
Member

commented Mar 26, 2017

@kghbln

This comment has been minimized.

Copy link
Member

commented Mar 26, 2017

@kghbln

This comment has been minimized.

Copy link
Member

commented Oct 18, 2017

Renaming of $smwgQStrictComparators now documented, too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.