Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: separation of storage and compute #192

Merged
merged 10 commits into from
Sep 6, 2023

Conversation

HuSen8891
Copy link
Collaborator

We separate the storage and computer, meta data is stored in catalog service, and user data is stored in data store. QD is responsible for catalog modification、query plan and dispatch; QE is responsible for all operations related with user data, executing queries.

core changes:

  1. add macro serverless, configure with --enable-serverless
  2. add hooks to get control in transaction/dispatch management.
  3. add transaction processing framework.
  4. add session state dispatch framework.

Why are the changes needed?

Describe why the changes are necessary.

Does this PR introduce any user-facing change?

If yes, please clarify the previous behavior and the change this PR proposes.

How was this patch tested?

Please detail how the changes were tested, including manual tests and any relevant unit or integration tests.

Contributor's Checklist

Here are some reminders and checklists before/when submitting your pull request, please check them:

  • Make sure your Pull Request has a clear title and commit message. You can take git-commit template as a reference.
  • Sign the Contributor License Agreement as prompted for your first-time contribution(One-time setup).
  • Learn the coding contribution guide, including our code conventions, workflow and more.
  • List your communication in the GitHub Issues or Discussions (if has or needed).
  • Document changes.
  • Add tests for the change
  • Pass make installcheck
  • Pass make -C src/test installcheck-cbdb-parallel
  • Feel free to @cloudberrydb/dev team for review and approval when your PR is ready🥳

husen and others added 10 commits August 10, 2023 19:55
core changes:
1. add macro serverless, configure with --enable-serverless
2. add hooks to get control in transaction/dispatch management
3. add transaction processing framework
4. add session state dispatch framework
…ue in plugin.

2. add SimpleLruReadPage_hook for plugin to read SLRU page.
3. add StartChildProcess_hook for plugin to get control in child process startup.
…n catalog is permitted on single master without warehouse.
Currently, we use randomly distribution for hashdata table, and the
number of segments is set to 0. When we query on hashdata table, the
distribution policy's segment number is set to number of segments of
current warehouse.
@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ HuSen8891
❌ husen


husen seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@my-ship-it my-ship-it merged commit 321bc89 into cloudberrydb:union_store_catalog Sep 6, 2023
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants