Skip to content
Permalink
Browse files
add ambari procedures for all "hawq config" references
  • Loading branch information
lisakowen authored and dyozie committed Nov 15, 2016
1 parent dc98696 commit 740b6ee698461c73ddfa2fd2cfdce889b1c13f42
Showing 7 changed files with 260 additions and 167 deletions.
@@ -80,11 +80,11 @@ This example shows how to edit the `pg_hba.conf` file of the master to allow rem

HAWQ allocates some resources on a per-connection basis, so setting the maximum number of connections allowed is recommended.

To limit the number of active concurrent sessions to your HAWQ system, you can configure the `max_connections` server configuration parameter on master or the `seg_max_connections` server configuration parameter on segments. These parameters are *local* parameters, meaning that you must set them in the `hawq-site.xml` file of all HAWQ instances. The current recommendation is to set the value of `seg_max_connections` to 5-10 the value of `max_connections`.
To limit the number of active concurrent sessions to your HAWQ system, you can configure the `max_connections` server configuration parameter on master or the `seg_max_connections` server configuration parameter on segments. These parameters are *local* parameters, meaning that you must set them in the `hawq-site.xml` file of all HAWQ instances.

When you set `max_connections`, you must also set the dependent parameter `max_prepared_transactions`. This value must be at least as large as the value of `max_connections`, and all HAWQ instances should be set to the same value.

For example, in `$GPHOME/etc/hawq-site.xml`:
Example `$GPHOME/etc/hawq-site.xml` configuration:

``` xml
<property>
@@ -101,51 +101,50 @@ For example, in `$GPHOME/etc/hawq-site.xml`:
</property>
```

The following steps set the parameter values with the HAWQ utility `hawq config`.
**Note:** Raising the values of these parameters may cause HAWQ to request more shared memory. To mitigate this effect, consider decreasing other memory-related server configuration parameters such as [gp\_cached\_segworkers\_threshold](../reference/guc/parameter_definitions.html#gp_cached_segworkers_threshold).

### <a id="ip142411"></a>To change the number of allowed connections

1. Log into the HAWQ master host as the HAWQ administrator and source the file `/usr/local/hawq/greenplum_path.sh`.
2. Set the value of the `max_connections` parameter. This `hawq config` command sets the value to 100 on all HAWQ instances.
### <a id="ip142411"></a>Setting the number of allowed connections

``` bash
$ hawq config -c max_connections -v 100
```
You will perform different procedures to set connection-related server configuration parameters for your HAWQ cluster depending upon whether you manage your cluster from the command line or use Ambari. If you use Ambari to manage your HAWQ cluster, you must ensure that you update server configuration parameters only via the Ambari Web UI. If you manage your HAWQ cluster from the command line, you will use the `hawq config` command line utility to set server configuration parameters.

This configuration command sets the value for the master.
If you use Ambari to manage your cluster:

3. Set the value of the `seg_max_connections` parameter. This `hawq config` command sets the value to 6400 on all HAWQ instances.
1. Set the `max_connections`, `seg_max_connections`, and `max_prepared_transactions` configuration properties via the HAWQ service **Configs > Advanced > Custom hawq-site** drop down.
2. Select **Service Actions > Restart All** to load the updated configuration.

``` bash
$ hawq config -c seg_max_connections -v 6400
```
If you manage your cluster from the command line:

This configuration command sets the value for all segment instances.
1. Log in to the HAWQ master host as a HAWQ administrator and source the file `/usr/local/hawq/greenplum_path.sh`.

4. Set the value of the `max_prepared_transactions` parameter. This `hawq config` command sets the value to 200 for the master and all segments.
``` shell
$ source /usr/local/hawq/greenplum_path.sh
```

2. Use the `hawq config` utility to set the values of the `max_connections`, `seg_max_connections`, and `max_prepared_transactions` parameters to values appropriate for your deployment. For example:

``` bash
$ hawq config -c max_connections -v 100
$ hawq config -c seg_max_connections -v 6400
$ hawq config -c max_prepared_transactions -v 200
```

The value of `max_prepared_transactions` must be greater than or equal to `max_connections`.

5. Stop and restart your HAWQ system.
5. Load the new configuration values by restarting your HAWQ cluster:

``` bash
$ hawq restart
$ hawq restart cluster
```

6. You can check the value of parameters on the master and segments with the `hawq config``-s` option. These `hawq config` commands display the values of the `max_connections` and `seg_max_connections` parameter.
6. Use the `-s` option to `hawq config` to display server configuration parameter values:

``` bash
$ hawq config -s max_connections
$ hawq config -s seg_max_connections
```


**Note:** Raising the values of these parameters may cause HAWQ to request more shared memory. To mitigate this effect, consider decreasing other memory-related server configuration parameters such as [gp\_cached\_segworkers\_threshold](../reference/guc/parameter_definitions.html#gp_cached_segworkers_threshold).

## <a id="topic5"></a>Encrypting Client/Server Connections

Enable SSL for client connections to HAWQ to encrypt the data passed over the network between the client and the database.
@@ -159,65 +159,68 @@ You can set your chosen encryption method system-wide or on a per-session basis.

#### <a id="topic11"></a>System-wide

To set the `password_hash_algorithm` server parameter on a complete HAWQ system \(master and its segments\):
You will perform different procedures to set the encryption method (`password_hash_algorithm` server parameter) system-wide depending upon whether you manage your cluster from the command line or use Ambari. If you use Ambari to manage your HAWQ cluster, you must ensure that you update encryption method configuration parameters only via the Ambari Web UI. If you manage your HAWQ cluster from the command line, you will use the `hawq config` command line utility to set encryption method configuration parameters.

1. Log into your HAWQ instance as a superuser.
2. Execute `hawq config` with the `password_hash_algorithm` set to `SHA-256` \(or `SHA-256-FIPS` to use the FIPS-compliant libraries for SHA-256\)
If you use Ambari to manage your HAWQ cluster:

``` bash
$ hawq config -c password_hash_algorithm -v 'SHA-256'
```
1. Set the `password_hash_algorithm` configuration property via the HAWQ service **Configs > Advanced > Custom hawq-site** drop down. Valid values include `SHA-256` \(or `SHA-256-FIPS` to use the FIPS-compliant libraries for SHA-256\).
2. Select **Service Actions > Restart All** to load the updated configuration.

or:
If you manage your HAWQ cluster from the command line:

``` bash
$ hawq config -c password_hash_algorithm -v 'SHA-256-FIPS'
1. Log in to the HAWQ master host as a HAWQ administrator and source the file `/usr/local/hawq/greenplum_path.sh`.

``` shell
$ source /usr/local/hawq/greenplum_path.sh
```

3. Verify the setting:
1. Use the `hawq config` utility to set `password_hash_algorithm` to `SHA-256` \(or `SHA-256-FIPS` to use the FIPS-compliant libraries for SHA-256\):

``` bash
$ hawq config -s
``` shell
$ hawq config -c password_hash_algorithm -v 'SHA-256'
```

Or:

``` shell
$ hawq config -c password_hash_algorithm -v 'SHA-256-FIPS'
```

You will see:
2. Reload the HAWQ configuration:

```
Master value: SHA-256
Segment value: SHA-256
``` shell
$ hawq stop cluster -u
```

or:
3. Verify the setting:

``` bash
$ hawq config -s password_hash_algorithm
```
Master  value: SHA-256-FIPS
Segment value: SHA-256-FIPS
```


#### <a id="topic12"></a>Individual Session

To set the `password_hash_algorithm` server parameter for an individual session:
To set the `password_hash_algorithm` server parameter for an individual database session:

1. Log into your HAWQ instance as a superuser.
1. Log in to your HAWQ instance as a superuser.
2. Set the `password_hash_algorithm` to `SHA-256` \(or `SHA-256-FIPS` to use the FIPS-compliant libraries for SHA-256\):

``` sql
# SET password_hash_algorithm = 'SHA-256'
=# SET password_hash_algorithm = 'SHA-256'
SET
```

or:

``` sql
# SET password_hash_algorithm = 'SHA-256-FIPS'
=# SET password_hash_algorithm = 'SHA-256-FIPS'
SET
```

3. Verify the setting:

``` sql
# SHOW password_hash_algorithm;
=# SHOW password_hash_algorithm;
password_hash_algorithm
```

@@ -240,7 +243,7 @@ To set the `password_hash_algorithm` server parameter for an individual session:
4. Login in as a super user and verify the password hash algorithm setting:

``` sql
# SHOW password_hash_algorithm
=# SHOW password_hash_algorithm
password_hash_algorithm
-------------------------------
SHA-256-FIPS
@@ -249,7 +252,7 @@ To set the `password_hash_algorithm` server parameter for an individual session:
5. Create a new role with password that has login privileges.

``` sql
CREATE ROLE testdb WITH PASSWORD 'testdb12345#' LOGIN;
=# CREATE ROLE testdb WITH PASSWORD 'testdb12345#' LOGIN;
```

6. Change the client authentication method to allow for storage of SHA-256 encrypted passwords:
@@ -276,7 +279,7 @@ To set the `password_hash_algorithm` server parameter for an individual session:
2. Execute the following:

``` sql
# SELECT rolpassword FROM pg_authid WHERE rolname = 'testdb';
=# SELECT rolpassword FROM pg_authid WHERE rolname = 'testdb';
Rolpassword
-----------
sha256<64 hexidecimal characters>
@@ -142,11 +142,35 @@ There are several options for configuring HAWQ statistics collection.

The statistics target is the size of the `most_common_vals`, `most_common_freqs`, and `histogram_bounds` arrays for an individual column. By default, the target is 25. The default target can be changed by setting a server configuration parameter and the target can be set for any column using the `ALTER TABLE` command. Larger values increase the time needed to do `ANALYZE`, but may improve the quality of the legacy query optimizer (planner) estimates.

Set the system default statistics target to a different value by setting the `default_statistics_target` server configuration parameter. The default value is usually sufficient, and you should only raise or lower it if your tests demonstrate that query plans improve with the new target. For example, to raise the default statistics target from 25 to 50 you can use the `hawq config` utility:
Set the system default statistics target to a different value by setting the `default_statistics_target` server configuration parameter. The default value is usually sufficient, and you should only raise or lower it if your tests demonstrate that query plans improve with the new target.

``` shell
$ hawq config -c default_statistics_target -v 50
```
You will perform different procedures to set server configuration parameters for your whole HAWQ cluster depending upon whether you manage your cluster from the command line or use Ambari. If you use Ambari to manage your HAWQ cluster, you must ensure that you update server configuration parameters via the Ambari Web UI only. If you manage your HAWQ cluster from the command line, you will use the `hawq config` command line utility to set server configuration parameters.

The following examples show how to raise the default statistics target from 25 to 50.

If you use Ambari to manage your HAWQ cluster:

1. Set the `default_statistics_target` configuration property to `50` via the HAWQ service **Configs > Advanced > Custom hawq-site** drop down.
2. Select **Service Actions > Restart All** to load the updated configuration.

If you manage your HAWQ cluster from the command line:

1. Log in to the HAWQ master host as a HAWQ administrator and source the file `/usr/local/hawq/greenplum_path.sh`.

``` shell
$ source /usr/local/hawq/greenplum_path.sh
```

1. Use the `hawq config` utility to set `default_statistics_target`:

``` shell
$ hawq config -c default_statistics_target -v 50
```
2. Reload the HAWQ configuration:

``` shell
$ hawq stop cluster -u
```

The statististics target for individual columns can be set with the `ALTER TABLE` command. For example, some queries can be improved by increasing the target for certain columns, especially columns that have irregular distributions. You can set the target to zero for columns that never contribute to query optimization. When the target is 0, `ANALYZE` ignores the column. For example, the following `ALTER TABLE` command sets the statistics target for the `notes` column in the `emp` table to zero:

@@ -174,12 +198,12 @@ The automatic statistics collection mode is set separately for commands that occ

With the `on_change` mode, `ANALYZE` is triggered only if the number of rows affected exceeds the threshold defined by the `gp_autostats_on_change_threshold` configuration parameter. The default value for this parameter is a very high value, 2147483647, which effectively disables automatic statistics collection; you must set the threshold to a lower number to enable it. The `on_change` mode could trigger large, unexpected analyze operations that could disrupt the system, so it is not recommended to set it globally. It could be useful in a session, for example to automatically analyze a table following a load.

To disable automatic statistics collection outside of functions, set the `gp_autostats_mode` parameter to `none`:
To disable automatic statistics collection outside of functions, set the `gp_autostats_mode` parameter to `none`. For a command-line-managed HAWQ cluster:

``` shell
$ hawq configure -c gp_autostats_mode -v none
```

Set the `log_autostats` system configuration parameter to on if you want to log automatic statistics collection operations.

For an Ambari-managed cluster, set `gp_autostats_mode` via the Ambari Web UI.

Set the `log_autostats` system configuration parameter to `on` if you want to log automatic statistics collection operations.

0 comments on commit 740b6ee

Please sign in to comment.