Support separation of catalog and compute #138

HuSen8891 · 2023-08-10T12:02:10Z

Change logs

core changes:

add macro serverless, configure with --enable-serverless
add hooks to get control in transaction/dispatch management.
add transaction processing framework.
add session state dispatch framework.

Why are the changes needed?

Describe why the changes are necessary.

Does this PR introduce any user-facing change?

If yes, please clarify the previous behavior and the change this PR proposes.

How was this patch tested?

Please detail how the changes were tested, including manual tests and any relevant unit or integration tests.

Contributor's Checklist

core changes: 1. add macro serverless, configure with --enable-serverless 2. add hooks to get control in transaction/dispatch management 3. add transaction processing framework 4. add session state dispatch framework

CLAassistant · 2023-08-10T12:02:17Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ HuSen8891
❌ husen

husen seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

github-actions

Hiiii, @HuSen8891 welcome!🎊 Thanks for taking the effort to make our project better! 🙌 Keep making such awesome contributions!

my-ship-it · 2023-08-11T01:07:45Z

configure

@@ -1604,6 +1606,7 @@ Optional Features:
  --enable-cassert        enable assertion checks (for debugging)
  --disable-orca          disable ORCA optimizer
  --enable-catalog-ext    enable CloudberryDB catalog extension
+  --enable-serverless     use Cloudberry serverless architecture


CloudberryDB?

I'll fix it later.

src/backend/Makefile

my-ship-it · 2023-08-11T01:11:40Z

src/backend/access/transam/slru.c

@@ -421,6 +427,17 @@ SimpleLruReadPage(SlruCtl ctl, int pageno, bool write_ok,
 				/* Now we must recheck state from the top */
 				continue;
 			}
+#ifdef SERVERLESS


Better to add a hook for LRUPageRead?

add SimpleLruReadPage_hook for plugin to read SLRU page.

src/backend/access/transam/xact.c

my-ship-it · 2023-08-11T01:17:17Z

src/backend/access/transam/xlog.c

@@ -3130,6 +3148,15 @@ XLogBackgroundFlush(void)
 	TimestampTz now;
 	int			flushbytes;

+#ifdef SERVERLESS


remove this later，add whitelist function to avoid launching WalWriter.

my-ship-it · 2023-08-11T01:21:07Z

src/backend/cdb/cdbtm.c

@@ -1608,6 +1608,14 @@ doQEDistributedExplicitBegin()
 static bool
 isDtxQueryDispatcher(void)
 {
+#ifdef SERVERLESS


Add a guc to control?
For example:

return (Gp_role == GP_ROLE_DISPATCH && isDtmStarted && isSharedLocalSnapshotSlotPresent);

add global variable enable_serverless, default to false, set to true in plugin.

my-ship-it · 2023-08-11T01:25:52Z

src/backend/postmaster/autovacuum.c

+/*
+ * Hook for plugins to get control in AutoVacLauncher.
+ */
+AutoVacLauncherMain_hook_type AutoVacLauncherMain_hook = NULL;


Provide a whitelist/blacklist so that we could control what background worker processes start?

Add whitelist function for auxiliary/background process.

my-ship-it · 2023-08-11T01:27:39Z

src/backend/storage/buffer/bufmgr.c

+		 * The page in buffer may be out of date, we need to check the buffer
+		 * and refresh the buffer if the page has been modified.
+		 */ 
+		if (Gp_role == GP_ROLE_EXECUTE && valid)


Need to think again about mechanism on the page consistent between QD and QE

…ue in plugin. 2. add SimpleLruReadPage_hook for plugin to read SLRU page. 3. add StartChildProcess_hook for plugin to get control in child process startup.

… jansson

…n catalog is permitted on single master without warehouse.

Currently, we use randomly distribution for hashdata table, and the number of segments is set to 0. When we query on hashdata table, the distribution policy's segment number is set to number of segments of current warehouse.

yjhjstz · 2023-08-25T03:19:35Z

src/backend/cdb/cdbcat.c

@@ -458,8 +458,16 @@ GpPolicyFetch(Oid tbloid)
 				}

 				/* Create a GpPolicy object. */
-				policy = makeGpPolicy(POLICYTYPE_PARTITIONED,
-									  nattrs, policyform->numsegments);
+				if (policyform->numsegments == 0)


why change this ？

We support to create table on single master without any segments. We create segments through creating warehouse on single master, switch to warehouse and query.

yjhjstz · 2023-08-25T04:16:19Z

src/backend/cdb/dispatcher/cdbdisp_extra.c

+		totalLen += sizeof(int) + strlen(names[i]) + 1 + *(lengths + i);
+		i++;
+	}
+	Assert(i = n);


Yes, let me fix this.

yjhjstz · 2023-08-25T04:46:53Z

src/backend/access/transam/slru.c

+	if (SimpleLruReadPage_hook)
+	{
+		LWLockAcquire(shared->ControlLock, LW_EXCLUSIVE);
+		return (*SimpleLruReadPage_hook) (ctl, pageno, true, xid);


where release lock ?

Lock released by the caller.

tuhaihe · 2023-08-29T03:44:06Z

Hi @HuSen8891 : Thanks for your contribution!

Please use rebase instead of merge for your commits to stay up to date with the main branch.
For a good commit title and message body, please follow our commit conventions and rewrite yours.
Additionally, please remember to squash your commits down to one before your PR is merged.

Thanks again.

…Request

HuSen8891 · 2023-08-29T08:49:40Z

Hi @HuSen8891 : Thanks for your contribution!

Please use rebase instead of merge for your commits to stay up to date with the main branch.

For a good commit title and message body, please follow our commit conventions and rewrite yours.

Additionally, please remember to squash your commits down to one before your PR is merged.

Thanks again.

Thanks. I'll rebase the code and rewrite the commit message when the PR is ready.

husen and others added 2 commits August 10, 2023 19:55

Support separation of catalog and compute.

f6c2ecf

core changes: 1. add macro serverless, configure with --enable-serverless 2. add hooks to get control in transaction/dispatch management 3. add transaction processing framework 4. add session state dispatch framework

Merge branch 'cloudberrydb:main' into main

ce02c5d

github-actions bot reviewed Aug 10, 2023

View reviewed changes

HuSen8891 marked this pull request as draft August 10, 2023 12:03

my-ship-it reviewed Aug 11, 2023

View reviewed changes

src/backend/Makefile Show resolved Hide resolved

my-ship-it reviewed Aug 11, 2023

View reviewed changes

src/backend/access/transam/xact.c Show resolved Hide resolved

my-ship-it reviewed Aug 11, 2023

View reviewed changes

husen and others added 4 commits August 11, 2023 17:44

1. add global variable enable_serverless, default to false, set to tr…

4dbcda7

…ue in plugin. 2. add SimpleLruReadPage_hook for plugin to read SLRU page. 3. add StartChildProcess_hook for plugin to get control in child process startup.

rename json_object to pg_json_object to resolve conflict with library…

e3ddd0f

… jansson

disable WAL-log information required only for Hot Standby in serverless

60e9089

Add support for creating cluster with single master, and only query o…

adc9b32

…n catalog is permitted on single master without warehouse.

roseduan mentioned this pull request Aug 23, 2023

Support create/drop database in serverless architecture #163

Closed

8 tasks

yjhjstz reviewed Aug 25, 2023

View reviewed changes

1. set distributedXid to LocalTransactionId 2. do not send FTS Probe …

b5f5525

…Request

HuSen8891 added 2 commits August 31, 2023 10:16

Feature: support subtransaction and savepoint

d3fa198

Fix: Only master can set transaction status

8e698c1

HuSen8891 closed this Sep 6, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support separation of catalog and compute #138

Support separation of catalog and compute #138

HuSen8891 commented Aug 10, 2023

CLAassistant commented Aug 10, 2023 •

edited

github-actions bot left a comment

my-ship-it Aug 11, 2023

HuSen8891 Aug 11, 2023

my-ship-it Aug 11, 2023

HuSen8891 Aug 11, 2023

my-ship-it Aug 11, 2023

HuSen8891 Aug 11, 2023 •

edited

my-ship-it Aug 11, 2023

HuSen8891 Aug 11, 2023

my-ship-it Aug 11, 2023

HuSen8891 Aug 11, 2023

my-ship-it Aug 11, 2023

yjhjstz Aug 25, 2023

HuSen8891 Aug 25, 2023

yjhjstz Aug 25, 2023

HuSen8891 Aug 25, 2023

yjhjstz Aug 25, 2023

HuSen8891 Aug 25, 2023

tuhaihe commented Aug 29, 2023

HuSen8891 commented Aug 29, 2023

Support separation of catalog and compute #138

Support separation of catalog and compute #138

Conversation

HuSen8891 commented Aug 10, 2023

Change logs

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Contributor's Checklist

CLAassistant commented Aug 10, 2023 • edited

github-actions bot left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

HuSen8891 Aug 11, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tuhaihe commented Aug 29, 2023

HuSen8891 commented Aug 29, 2023

CLAassistant commented Aug 10, 2023 •

edited

HuSen8891 Aug 11, 2023 •

edited