Skip to content
This repository has been archived by the owner on Jun 14, 2024. It is now read-only.

[Gold Standard] Initial Code showing Hyperspace Indexes with a sample query #385

Open
wants to merge 36 commits into
base: master
Choose a base branch
from
Open

[Gold Standard] Initial Code showing Hyperspace Indexes with a sample query #385

wants to merge 36 commits into from

Conversation

apoorvedave1
Copy link
Contributor

@apoorvedave1 apoorvedave1 commented Mar 18, 2021 •
edited
Loading

What is the context for this pull request?

Gold Standard: Hyperspace code for creation of index metadata files and its use in Plan Stability test Suite.

What changes were proposed in this pull request?

In this PR:
TPCDSHyperspace suite extends PlanStabilitySuite

  • Ability to define indexes : indexname, tablename, indexed cols(comma-separated), included cols(comma-separated)
  • Create index metadata files (metadata files are enough for generating query plans)
  • A simple query with join between two tables on a key
  • Approved plans (golden files) for the simple query

Does this PR introduce any user-facing change?

No

How was this patch tested?

Unit tests

@apoorvedave1
gold standard initial commit
208ea5e
@apoorvedave1
fix q32
cc14991
@apoorvedave1
Merge branch 'master' of github.com:apoorvedave1/hyperspace-1 into gs
7c8ee78
@apoorvedave1
Merge branch 'master' of github.com:apoorvedave1/hyperspace-1 into gs
fa9bd4a
@apoorvedave1
keep only tpcds v1.4 and remove others
4b007a5
@apoorvedave1
Merge remote-tracking branch 'upstream/master' into gs_initial
59207f6
@apoorvedave1
Trigger Build
5c0cee9
@apoorvedave1
build error: test with sequential run
900539d
@apoorvedave1
revert previous commit
530dfa7
@apoorvedave1
update plans
add01f1
@apoorvedave1
update plan
8b58b6b
@apoorvedave1
added sorting for fixing build pipeline
97e8441
@apoorvedave1
udpated plans with sorting
411450f
@apoorvedave1
fix q49
b70f00b
@apoorvedave1
updated instructions on how to run tests
7880f40
@apoorvedave1
test with updated plans for q47, q49
3dcd2d5
@apoorvedave1
update q47, 49 plans
a7a1149
@apoorvedave1
remove rogue query
dd295a9
@apoorvedave1
remove q49
f47fbd3
@apoorvedave1
fix scalastyle
d15a4e8
@apoorvedave1
add query files for q49.sql
4c511fb
@apoorvedave1
restructuring
e48c713
@apoorvedave1
cleanup before rebase
6c5a2f7
@apoorvedave1
Merge remote-tracking branch 'upstream/master' into gs_initial
25cb361
@apoorvedave1
updated plans based on the plan stability suite
0b40853
@apoorvedave1
normalize location: fix
32f5899
@apoorvedave1
Merge branch 'master' of github.com:apoorvedave1/hyperspace-1 into gs…
352f2f5 
…_initial

# Conflicts:
#	src/test/scala/com/microsoft/hyperspace/goldstandard/PlanStabilitySuite.scala
#	src/test/scala/com/microsoft/hyperspace/goldstandard/TPCDSBase.scala
@apoorvedave1
query plans for all queries
d5fbec8
@apoorvedave1
sample query q0 with sample indexes
3b51533
@apoorvedave1 apoorvedave1 mentioned this pull request Mar 18, 2021
Open
@apoorvedave1 apoorvedave1 changed the title [Gold Standard] Initial Code for showcasing Hyperspace Indexes with a sample query [Gold Standard] Initial Code showing Hyperspace Indexes with a sample query Mar 18, 2021
@apoorvedave1 apoorvedave1 mentioned this pull request Mar 18, 2021
Open
6 tasks
@imback82
Copy link
Contributor

Should we just have code update + one query example?

@apoorvedave1
Copy link
Contributor Author

Should we just have code update + one query example?

@imback82 removed dependencies coming from the dependency PR. now this one contains only relevant files.

@imback82
Copy link
Contributor

Can you check the test failures?

@apoorvedave1
Copy link
Contributor Author

Can you check the test failures?

@imback82 thank you, I also cleaned up some other parts of the code. Please take a look.

@apoorvedave1 apoorvedave1 self-assigned this Mar 19, 2021
imback82
imback82 reviewed Mar 19, 2021
View reviewed changes
Copy link
Contributor

@imback82 imback82 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it somehow possible to create a diff against simplified.txt (or explain.txt) without hyperspace? Otherwise, it's hard to see what has changed after applying hyperspace.

src/test/scala/com/microsoft/hyperspace/goldstandard/IndexLogEntryCreator.scala Outdated Show resolved Hide resolved
src/test/scala/com/microsoft/hyperspace/goldstandard/IndexLogEntryCreator.scala Outdated Show resolved Hide resolved
@imback82
Copy link
Contributor

One way is to diff the file and attach it to the PR?

@apoorvedave1
Copy link
Contributor Author

One way is to diff the file and attach it to the PR?

I added files for without hyperspace enabled and also with the diffs. Please let me know if we intend to keep them in the final version or is it just for the sake of validation in the PR.?

@imback82
Copy link
Contributor

I added files for without hyperspace enabled and also with the diffs. Please let me know if we intend to keep them in the final version or is it just for the sake of validation in the PR.?

This is just for the PR.

@apoorvedave1 apoorvedave1 mentioned this pull request Apr 22, 2021
Open
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@imback82 imback82 Awaiting requested review from imback82

@sezruby sezruby Awaiting requested review from sezruby

At least 1 approving review is required to merge this pull request.

Assignees

@apoorvedave1 apoorvedave1

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@apoorvedave1 @imback82 @sezruby