Skip to content
Permalink
Browse files
Notes on memory limit (closes #119)
  • Loading branch information
janebeckman authored and dyozie committed Jun 5, 2017
1 parent 214684a commit 973791b2257656d96708c770d356ea5e172be2aa
Showing 2 changed files with 4 additions and 2 deletions.
@@ -31,7 +31,7 @@ The `pg_resqueue` system catalog table contains information about HAWQ resource
| `rsqname` | name |   | The name of the resource queue. |
| `parentoid` | oid |   | OID of the parent queue of the resource queue. |
| `activestats` | integer |   | The maximum number of parallel active statements allowed for the resource queue. |
| `memorylimit` | text |   | The maximum amount of memory that can be consumed by the resource queue (expressed as a percentage of the cluster's memory.) |
| `memorylimit` | text |   | The maximum amount of memory that can be consumed by the resource queue (expressed as a percentage of the cluster's memory.) Because of how memory is determined, the memory limit is not an upper limit on the memory actually used by a resource queue. |
| `corelimit` | text |   | The maximum amount of cores that can be consumed by the resource queue (expressed as a percentage of the cluster's cores.) |
| `resovercommit` | real |   | The ratio of resource consumption overcommit for the resource queue. |
| `allocpolicy` | text |   | The resource allocation policy name for the resource queue. |
@@ -58,6 +58,8 @@ Resource queues with an `ACTIVE_STATEMENTS` threshold set a maximum limit on the

When creating the resource queue, use the MEMORY\_LIMIT\_CLUSTER and CORE\_LIMIT\_CLUSTER queue attributes to tune the allowed resource usage of the resource queue. MEMORY\_LIMIT\_CLUSTER and CORE\_LIMIT\_CLUSTER must set to the same value for a resource queue. In addition the sum of the percentages of MEMORY\_LIMIT\_CLUSTER (and CORE\_LIMIT\_CLUSTER) for resource queues that share the same parent cannot exceed 100%.

Note: The MEMORY\_LIMIT of a resource queue assigns the quota to operators in a query plan during planning, before actual execution. Actual memory consumption during query might exceed the limit.

You can optionally configure the maximum or minimum number of virtual segments to use when executing a query by setting NVSEG\_UPPER\_LIMIT/NVSEG\_LOWER\_LIMIT or NVSEG\_UPPER\_LIMIT\_PERSEG/NVSEG\_LOWER\_LIMIT\_PERSEG attributes for the resource queue.

After defining a resource queue, you can assign a role to the queue by using the [ALTER ROLE](ALTER-ROLE.html) or [CREATE ROLE](CREATE-ROLE.html) command. You can only assign roles to the leaf-level resource queues (resource queues that do not have any children.)
@@ -75,7 +77,7 @@ See also [Best Practices for Using Resource Queues](../../bestpractices/managing
The parent queue cannot have any roles assigned.</dd>

<dt>MEMORY\_LIMIT\_CLUSTER=\<percentage\> </dt>
<dd>Required. Defines how much memory a resource queue can consume from its parent resource queue and consequently dispatch to the execution of parallel statements. Since a resource queue obtains its memory from its parent, the actual memory limit is based from its parent queue. The valid values are 1% to 100%. The value of MEMORY\_ LIMIT\_CLUSTER must be identical to the value of CORE\_LIMIT\_CLUSTER. The sum of values for MEMORY\_LIMIT\_CLUSTER of this queue plus other queues that share the same parent cannot exceed 100%. The HAWQ resource manager periodically validates this restriction.</dd>
<dd>Required. Defines how much memory a resource queue can consume from its parent resource queue and consequently dispatch to the execution of parallel statements. Since a resource queue obtains its memory from its parent, the actual memory limit is based from its parent queue. The valid values are 1% to 100%. The value of MEMORY\_ LIMIT\_CLUSTER must be identical to the value of CORE\_LIMIT\_CLUSTER. The sum of values for MEMORY\_LIMIT\_CLUSTER of this queue plus other queues that share the same parent cannot exceed 100%. The HAWQ resource manager periodically validates this restriction. Despite its name, the actual memory consumed by a resource queue may exceed a limit inferred from this value.</dd>

<dt>CORE\_LIMIT\_CLUSTER=\<percentage\> </dt>
<dd>Required. The percentage of consumable CPU (virtual core) resources that the resource queue can take from its parent resource queue. The valid values are 1% to 100%. The value of MEMORY\_ LIMIT\_CLUSTER must be identical to the value of CORE\_LIMIT\_CLUSTER. The sum of values for CORE\_LIMIT\_CLUSTER of this queue and queues that share the same parent cannot exceed 100%.</dd>

0 comments on commit 973791b

Please sign in to comment.