published
true

Toolkit for Bare Metal Solution: User Guide

Command quick reference for single instance deployments
Command quick reference for RAC deployments
Command quick reference for DR deployments
Overview
- Software Stack
- Requirements and Prerequisites
  - Control node requirements
  - Target server requirements
Installing the toolkit
Downloading and staging the Oracle Software
Prerequisite configuration
Configuring Installations
Post installation tasks

Command quick reference for single instance deployments

Sample commands for a simple quick-start and basic toolkit usage for an Oracle "single instance" database. Refer to the remainder of this document for additional details and comprehensive explanations of the toolkit, scripting, options, and usage scenarios. All commands run from the "control node".

Validate media specifying GCS storage bucket and optionally database:

./check-swlib.sh --ora-swlib-bucket gs://[cloud-storage-bucket-name] \
 --ora-version 19.3.0.0.0

Validate access to target server (optionally include -i and location of private key file):
```
ssh ${INSTANCE_SSH_USER:-`whoami`}@${INSTANCE_IP_ADDR} sudo -u root hostname
```
Review toolkit parameters:
```
./install-oracle.sh --help
```

Run an installation:

./install-oracle.sh \
--ora-swlib-bucket gs://[cloud-storage-bucket-name] \
--backup-dest "+RECO" \
--ora-swlib-path /u02/swlib/ \
--ora-swlib-type gcs \
--instance-ip-addr ${INSTANCE_IP_ADDR}

Command quick reference for RAC deployments

Sample installation for an Oracle Real Application Clusters (RAC) installation. Initial steps similar to those of the Single Instance installation.

Validate media specifying Cloud Storage bucket and optionally database version:

./check-swlib.sh --ora-swlib-bucket gs://[cloud-storage-bucket-name]
--ora-version 19.3.0.0.0

Validate access to target RAC nodes:

ssh ${INSTANCE_SSH_USER:-`whoami`}@${INSTANCE_IP_ADDR_NODE_1} sudo -u root hostname
ssh ${INSTANCE_SSH_USER:-`whoami`}@${INSTANCE_IP_ADDR_NODE_2} sudo -u root hostname

Review optional toolkit parameters:

./install-oracle.sh --help
Create the cluster configuration file by editing the cluster_config.json file template that is provided with the toolkit.

Install the database with the path to the cluster configuration file specified on the --cluster-config property:

./install-oracle.sh \
--ora-swlib-bucket gs://[cloud-storage-bucket-name] \
--backup-dest "+RECO" \
--ora-swlib-path /u02/swlib/ \
--ora-swlib-type gcs \
--cluster-type RAC \
--cluster-config cluster_config.json

Command quick reference for DR deployments

The primary database must exist before you can create a standby database.

When you create the primary database, omit the --cluster-type option or set it to NONE. To create the primary database, see Single Instance Deployments section.

To create a standby database, add the following options to the command options that you used to create the primary database:

--primary-ip-addr ${PRIMARY_IP_ADDR}
--cluster-type DG

Install a standby database:

./install-oracle.sh \
--ora-swlib-bucket gs://[cloud-storage-bucket-name] \
--instance-ip-addr ${INSTANCE_IP_ADDR} \
--ora-swlib-path /u02/swlib/ \
--backup-dest "+RECO" \
--ora-swlib-type gcs \
--primary-ip-addr ${PRIMARY_IP_ADDR} \
--cluster-type DG

Overview

The Implementation Toolkit for Oracle provides an automated (scripted) mechanism to help you install Oracle software and configure an initial Oracle database on the Google Cloud Bare Metal Solution. You can also use the toolkit to provision initial Oracle Database Recovery Manager (RMAN) backups to Google Cloud Storage or another storage system.

This guide is for experienced professional users of Oracle software who are deploying Oracle Database software and preparing initial Oracle databases on Google Cloud Bare Metal Solution. The toolkit defines default values for most options, so you can run the toolkit with only a few specifications. Your configuration options are listed later in this guide.

The toolkit supports the following major releases of Oracle Database and applies the most recent quarterly patches, also known as Oracle Release Updates or RUs:

Oracle 11.2.0.4.0
Oracle 12.1.0.2.0
Oracle 12.2.0.1.0
Oracle 18c
Oracle 19c

The toolkit does not include any Oracle software. You must obtain the appropriate licenses and download the Oracle software on your own. This guide provides information about where to obtain Oracle software solely for your convenience.

After downloading the Oracle software, you stage the software in a Cloud Storage bucket where the toolkit can access it.

Software Stack

The toolkit customizes the software stack for Oracle Database workloads. Any out of a number of Oracle Database software releases can be installed. In addition, the configuration of the software stack includes:

The Oracle Grid Infrastructure (GI) and Automatic Storage Manager (ASM), at the same major release as the database software.
The configuration of Oracle resources, like the database, listener, and ASM resources, via "Oracle Restart" for single-instance deployments and Oracle Clusterware for RAC deployments.
The optional separation of OS roles,"role separation," so you can have different OS users for the GI and database software.
The installation of all of the required OS packages that are necessary for the Oracle software installation, including common packages, such as ntp, bind-utils, unzip, expect, wget, and net-tools.
The configuration of Linux Huge Pages, usually as a percentage of the available memory, and the disabling of Red Hat Transparent Huge Pages (THP), as per the recommended Oracle practices.
The adjustment of Linux kernel settings, as necessary. For more information, see the Oracle documentation.
The disabling of the Linux firewall and SELinux, as recommended for Oracle database servers.
The creation of a swap device, as necessary.

You can further customize the environment and host server(s), as needed.

Requirements and Prerequisites

You need at least two servers to install Oracle software by using the toolkit:

Control node: a virtual or physical machine from which the toolkit is executed.
Database server(s): target where the Oracle software will be installed and configured.

A second database server (or node) is required for RAC deployments.

Control node requirements

The control node can be any server capable of ssh.

The control node must have the following software installed:

Ansible version 2.9 or higher.
If you are using a Cloud Storage bucket to stage your Oracle installation media, the Google Cloud SDK.
Ideally, a mainstream Linux OS.

Depending on the Linux distribution you are using on your control node, you can install Ansible with sudo apt-get install ansible. Your installation command might be different. You can verify your version of Ansible with ansible --version.

You can use the Google Cloud Shell as your control node. Cloud Shell provides command-line access to a virtual machine instance in a terminal window that opens in the web console. The latest version of Cloud SDK is installed for you.

Target server requirements

Prior to running the toolkit, ensure that the control node has SSH access to a Linux user account on the target server. The user account must have elevated security privileges, such as granted by "sudo su -", to install and configure Oracle software. The toolkit creates Oracle software owners, such as oracle and grid.

The target database server(s) must be running a version of Linux that is certified for Oracle Database. The toolkit currently supports the following certified OS versions:

Red Hat Enterprise Linux (RHEL) 7 and 8 (versions 7.3 and up).
Oracle Linux (OL) 7 and 8 (versions 7.3 and up).

For more information about Oracle-supported platforms see the Oracle certification matrix in the "My Oracle Support" (MOS) site (sign in required): https://support.oracle.com.

Installing the toolkit

The latest version of the toolkit can be downloaded from Google Git Repositories: https://github.com/google/bms-toolkit

On the google/bms-toolkit home page in GitHub, download the toolkit to your control node by clicking the Clone or Download button and selecting Download zip.

If you are using the Cloud Shell as your control node, download the tool to your $HOME directory.

Downloading and staging the Oracle Software

You must download and stage the Oracle software yourself, in accordance with the applicable licenses governing such software. The toolkit doesn't contain any Oracle software. You are responsible for procuring the Oracle software that you need and for complying with the applicable licenses.

Downloading the Oracle installation software

Oracle software is divided into two general categories: base software that you download from the Oracle Software Delivery Cloud site (also known as Oracle "eDelivery"), and patches that you download from Oracle's My Oracle Support (MOS) site.

You can also download base software from [Oracle Technology Network][https://www.oracle.com/database/technologies/oracle-database-software-downloads.html#db_ee). In this case, please rename the downloaded files to the software delivery cloud equivalent names, and use --no-patch to skip patching. Note that unpatched software may have known defects and security vulnerabilities.

One key exception: Oracle 11g base software can be downloaded directly from My Oracle Support. Only Oracle 12c or later base software needs to be downloaded from Oracle Software Delivery Cloud. Direct links to MOS downloads are provided below.

Before you download Oracle software and patches, review and acknowledge Oracle's license terms.

Before using the toolkit, download all of the software pieces for your Oracle release, including the base release, patchsets, the OPatch utility, and any additional patches listed by Oracle (unless using --no-patch, at which point only the base release is installed).

Do not unzip the downloaded installation files. The toolkit requires the downloads in their original, compressed-file format.

Downloading Patches from My Oracle Support

For convenience, direct links to My Oracle Support (MOS) for applicable patches are listed in the following section. You need an Oracle Single Sign-on account that is linked to a valid Oracle Customer Support Identifier (CSI) to download patches through My Oracle Support.

Required Oracle Software - Download Summary

Oracle Release	Category - Site	Software Piece	File Name (From "Oracle eDelivery" or "My Oracle Support")
19.3.0.0.0	Base - eDelivery	Oracle Database 19.3.0.0.0 for Linux x86-64	V982063-01.zip
		Oracle Grid Infrastructure 19.3.0.0.0 for Linux x86-64	V982068-01.zip
	Patch - MOS	COMBO OF OJVM RU COMPONENT 19.23.0.0.240416 + GI RU 19.23.0.0.240416	p36209493_190000_Linux-x86-64.zip
	Patch - MOS	COMBO OF OJVM RU COMPONENT 19.22.0.0.240116 + GI RU 19.22.0.0.240116	p36031453_190000_Linux-x86-64.zip
	Patch - MOS	COMBO OF OJVM RU COMPONENT 19.21.0.0.231017 + GI RU 19.21.0.0.231017	p35742441_190000_Linux-x86-64.zip
	Patch - MOS	COMBO OF OJVM RU COMPONENT 19.13.0.0.211019 + GI RU 19.13.0.0.211019	p33248471_190000_Linux-x86-64.zip
	Patch - MOS	COMBO OF OJVM RU COMPONENT 19.11.0.0.210420 + GI RU 19.11.0.0.210420	p32578973_190000_Linux-x86-64.zip
		COMBO OF OJVM RU COMPONENT 19.10.0.0.210119 + GI RU 19.10.0.0.210119	p32126842_190000_Linux-x86-64.zip
		COMBO OF OJVM RU COMPONENT 19.9.0.0.201020 + GI RU 19.9.0.0.201020	p31720429_190000_Linux-x86-64.zip
		COMBO OF OJVM RU COMPONENT 19.8.0.0.200714 + GI RU 19.8.0.0.200714	p31326369_190000_Linux-x86-64.zip
		COMBO OF OJVM RU COMPONENT 19.7.0.0.200414 + GI RU 19.7.0.0.200414	p30783556_190000_Linux-x86-64.zip
		COMBO OF OJVM RU COMPONENT 19.6.0.0.200114 GI RU 19.6.0.0.200114	p30463609_190000_Linux-x86-64.zip
		COMBO OF OJVM RU COMPONENT 19.5.0.0.191015 GI RU 19.5.0.0.191015	p30133178_190000_Linux-x86-64.zip
		COMBO OF OJVM RU COMPONENT 19.4.0.0.190716 + GI RU 19.4.0.0.190716	p29699097_190000_Linux-x86-64.zip
		OPatch Utility	p6880880_190000_Linux-x86-64.zip
18.0.0.0.0	Base - eDelivery	Oracle Database 18.0.0.0.0 for Linux x86-64	V978967-01.zip
		Oracle Grid Infrastructure 18.0.0.0.0 for Linux x86-64	V978971-01.zip
	Patch - MOS	COMBO OF OJVM RU COMPONENT 18.14.0.0.210420 + GI RU 18.14.0.0.210420	p32579024_180000_Linux-x86-64.zip
		COMBO OF OJVM RU COMPONENT 18.13.0.0.210119 + GI RU 18.13.0.0.210119	p32126862_180000_Linux-x86-64.zip
		COMBO OF OJVM RU COMPONENT 18.12.0.0.201020 + GI RU 18.12.0.0.201020	p31720457_180000_Linux-x86-64.zip
		COMBO OF OJVM RU COMPONENT 18.11.0.0.200714 + GI RU 18.11.0.0.200714	p31326376_180000_Linux-x86-64.zip
		COMBO OF OJVM RU COMPONENT 18.10.0.0.200414 GI RU 18.10.0.0.200414	p30783607_180000_Linux-x86-64.zip
		COMBO OF OJVM RU COMPONENT 18.9.0.0.200114 GI RU 18.9.0.0.200114	p30463635_180000_Linux-x86-64.zip
		COMBO OF OJVM RU COMPONENT 18.8.0.0.191015 GI RU 18.8.0.0.191015	p30133246_180000_Linux-x86-64.zip
		COMBO OF OJVM RU COMPONENT 18.7.0.0.190716 + GI RU 18.7.0.0.190716	p29699160_180000_Linux-x86-64.zip
		COMBO OF OJVM RU COMPONENT 18.6.0.0.190416 + GI RU 18.6.0.0.190416	p29251992_180000_Linux-x86-64.zip
		OPatch Utility	p6880880_180000_Linux-x86-64.zip
12.2.0.1.0	Base - eDelivery	Oracle Database 12.2.0.1.0 for Linux x86-64	V839960-01.zip
		Oracle Grid Infrastructure 12.2.0.1.0 for Linux x86-64 for Linux x86-64	V840012-01.zip
	Patch - MOS	COMBO OF OJVM RU COMPONENT 12.2.0.1.211019 + 12.2.0.1.211019 GIOCT2021RU	p333248546_122010_Linux-x86-64.zip
	Patch - MOS	COMBO OF OJVM RU COMPONENT 12.2.0.1.210420 + 12.2.0.1.210420 GIAPR2021RU	p32579057_122010_Linux-x86-64.zip
		COMBO OF OJVM RU COMPONENT 12.2.0.1.210119 + 12.2.0.1.210119 GIJAN2021RU	p32126883_122010_Linux-x86-64.zip
		COMBO OF OJVM RU COMPONENT 12.2.0.1.201020 + 12.2.0.1.201020 GIOCT2020RU	p31720486_122010_Linux-x86-64.zip
		COMBO OF OJVM RU COMPONENT 12.2.0.1.200714 + 12.2.0.1.200714 GIJUL2020RU	p31326390_122010_Linux-x86-64.zip
		COMBO OF OJVM RU COMPONENT 12.2.0.1.200414 12.2.0.1.200414 GIAPR2020RU	p30783652_122010_Linux-x86-64.zip
		COMBO OF OJVM RU COMPONENT 12.2.0.1.200114 12.2.0.1.200114 GIJAN2020RU	p30463673_122010_Linux-x86-64.zip
		COMBO OF OJVM RU COMPONENT 12.2.0.1.191015 12.2.0.1.191015 GIOCT2019RU	p30133386_122010_Linux-x86-64.zip
		COMBO OF OJVM RU COMPONENT 12.2.0.1.190716 + 12.2.0.1.190716 GIJUL2019RU	p29699173_122010_Linux-x86-64.zip
		COMBO OF OJVM RU COMPONENT 12.2.0.1.190416 + 12.2.0.1.190416 GIAPR2019RU	p29252072_122010_Linux-x86-64.zip
		ACFS MODULE ORACLEACFS.KO FAILS TO LOAD ON OL7U3 SERVER WITH RHCK (Patch) patch 25078431 for Linux x86-64	p25078431_122010_Linux-x86-64.zip
		OPatch Utility	p6880880_122010_Linux-x86-64.zip
12.1.0.2.0	Base - eDelivery	Oracle Database 12.1.0.2.0 for Linux x86-64	V46095-01_1of2.zip V46095-01_2of2.zip
		Oracle Database 12c Standard Edition 2 12.1.0.2.0 for Linux x86-64	V77388-01_1of2.zip V77388-01_2of2.zip
		Oracle Grid Infrastructure 12.1.0.2.0 for Linux x86-64	V46096-01_1of2.zip V46096-01_2of2.zip
	Patch - MOS	COMBO OF OJVM COMPONENT 12.1.0.2.211019 DB PSU + GIPSU 12.1.0.2.211019	p33248580_121020_Linux-x86-64.zip
	Patch - MOS	COMBO OF OJVM COMPONENT 12.1.0.2.201020 DB PSU + GIPSU 12.1.0.2.201020	p31720761_121020_Linux-x86-64.zip
		COMBO OF OJVM COMPONENT 12.1.0.2.200714 DB PSU + GIPSU 12.1.0.2.200714	p31326400_121020_Linux-x86-64.zip
		COMBO OF OJVM COMPONENT 12.1.0.2.200414 DB PSU GIPSU 12.1.0.2.200414	p30783882_121020_Linux-x86-64.zip
		COMBO OF OJVM COMPONENT 12.1.0.2.200114 DB PSU GIPSU 12.1.0.2.200114	p30463691_121020_Linux-x86-64.zip
		COMBO OF OJVM COMPONENT 12.1.0.2.191015 DB PSU GIPSU 12.1.0.2.191015	p30133443_121020_Linux-x86-64.zip
		COMBO OF OJVM COMPONENT 12.1.0.2.190716 DB PSU + GIPSU 12.1.0.2.190716	p29699244_121020_Linux-x86-64.zip
		COMBO OF OJVM COMPONENT 12.1.0.2.190416 DB PSU + GIPSU 12.1.0.2.190416	p29252164_121020_Linux-x86-64.zip
		Combo OJVM PSU 12.1.0.2.190416 and Database Proactive BP 12.1.0.2.190416 patch 29252171 for Linux x86-64	p29252171_121020_Linux-x86-64.zip
		GI PSU 12.1.0.2.190416 patch 29176115 for Linux x86-64	p29176115_121020_Linux-x86-64.zip
		OPatch Utility	p6880880_121010_Linux-x86-64.zip
11.2.0.4	Patch - MOS	11.2.0.4.0 PATCH SET FOR ORACLE DATABASE SERVER - Patch 13390677 for Linux x86-64	p13390677_112040_Linux-x86-64_1of7.zip p13390677_112040_Linux-x86-64_2of7.zip
		11.2.0.4.0 PATCH SET FOR ORACLE DATABASE SERVER - Patch 13390677 for Linux x86-64	p13390677_112040_Linux-x86-64_3of7.zip
	Patch - MOS	Combo of OJVM Component 11.2.0.4.201020 DB PSU + GI PSU 11.2.0.4.201020	p31720783_112040_Linux-x86-64.zip
		Combo of OJVM Component 11.2.0.4.200714 DB PSU + GI PSU 11.2.0.4.200714	p31326410_112040_Linux-x86-64.zip
		COMBO OF 11.2.0.4.200414 OJVM PSU GIPSU 11.2.0.4.200414	p30783890_112040_Linux-x86-64.zip
		GRID INFRASTRUCTURE PATCH SET UPDATE 11.2.0.4.200114	p30501155_112040_Linux-x86-64.zip
		GRID INFRASTRUCTURE PATCH SET UPDATE 11.2.0.4.191015	p30070097_112040_Linux-x86-64.zip
		GRID INFRASTRUCTURE PATCH SET UPDATE 11.2.0.4.190716	p29698727_112040_Linux-x86-64.zip
		GRID INFRASTRUCTURE PATCH SET UPDATE 11.2.0.4.190416	p29255947_112040_Linux-x86-64.zip
		Combo OJVM PSU 11.2.0.4.190416 and Database PSU 11.2.0.4.190416 patch 29252186 for Linux x86-64	p29252186_112040_Linux-x86-64.zip
		GI PSU 11.2.0.4.190416 patch 29255947 for Linux x86-64	p29255947_112040_Linux-x86-64.zip
		RC SCRIPTS (/ETC/RC.D/RC.* , /ETC/INIT.D/* ) ON OL7 FOR CLUSTERWARE (Patch) patch 18370031 for Linux x86-64	p18370031_112040_Linux-x86-64.zip
		OPatch Utility	p6880880_112000_Linux-x86-64.zip

If the required software components are not properly downloaded and staged, the toolkit will fail.

Staging the Oracle installation media

You can stage the Oracle installation media in any one of the following repository types:

A Cloud Storage bucket.
Cloud Storage FUSE, an open source FUSE adapter that allows you to mount Cloud Storage buckets as file systems on Linux or macOS systems.
A network NFS share.

Cloud Storage bucket

To use a Cloud Storage bucket to stage your installation media, you need the gsutil tool installed on your control node. The gsutil tool is a Python application that lets you access Cloud Storage from the command line. To get the gsutil tool, install the Cloud SDK.

Cloud Storage FUSE

With Cloud Storage FUSE, you can also pass the location of a Cloud Storage service account json file. This is useful if your control node doesn't have a proper instance service account. You should also confirm that the service account scopes allow you to use Cloud Storage.

To create a new service account:

Navigate to your Google Cloud Console.
Select the IAM & Admin tab > Service accounts.
Click on Create Service Account and chose a relevant name.
Select a role that provides the permission that you need, such as Storage Admin.
Click on the "Create Key" button, and download the file in JSON format.

You can then pass the file as a parameter to the deployment:

--ora-swlib-type gcsfuse --ora-swlib-bucket oracle-swlib --ora-swlib-credentials ~/path_to/service_account.json

The toolkit uploads the service account to the server so that Cloud Storage FUSE can use it.

NFS share

When you use an NFS share, you specify the NFS mount point on the ora-swlib-bucket parameter:

--ora-swlib-type nfs --ora-swlib-bucket 192.168.0.250:/my_nfs_export

Validating Media

You can validate that you have correctly staged all of the required installation files by using the check-swlib.sh script, which validates the files based on name, size, and MD5 message digest. To validate the media, specify the GCS software bucket where the software is staged and the Oracle software version that you are installing. The version default is 19.3.0.0.0.

Example of a successful media validation:

$ ./check-swlib.sh --ora-swlib-bucket gs://oracle-software --ora-version 19.3.0.0.0

Running with parameters from command line or environment variables:

ORA_SWLIB_BUCKET=gs://oracle-software
ORA_VERSION=19.3.0.0.0

Found V982063-01.zip : Oracle Database 19.3.0.0.0 for Linux x86-64
        file size matches (3059705302), md5 matches (1858bd0d281c60f4ddabd87b1c214a4f).

Found V982068-01.zip : Oracle Grid Infrastructure 19.3.0.0.0 for Linux x86-64
        file size matches (2889184573), md5 matches (b7c4c66f801f92d14faa0d791ccda721).

Found p29859737_190000_Linux-x86-64.zip : Oracle 19c DB RU patch 29859737 for Linux x86-64
        file size matches (498214157), md5 matches (3b017f517341df5b35e9fbd90f1f49aa).

Found p29800658_190000_Linux-x86-64.zip : Oracle 19c GI RU patch 29800658 for Linux x86-64
        file size matches (1365811472), md5 matches (13c0041a5ea7eb9fad4725d2136da627).

Found p29699097_190000_Linux-x86-64.zip : COMBO OF OJVM RU COMPONENT 19.4.0.0.190716 + GI RU 19.4.0.0.190716
        file size matches (1986870968), md5 matches (2206c8a2431eb6fa0c4f7dd5aa7a58b2).

Found p30133178_190000_Linux-x86-64.zip : COMBO OF OJVM RU COMPONENT 19.5.0.0.191015 GI RU 19.5.0.0.191015
        file size matches (2004604850), md5 matches (4189caeae850a7c4191fdd3fa4c0af6a).

Found p30463609_190000_Linux-x86-64.zip : COMBO OF OJVM RU COMPONENT 19.6.0.0.200114 GI RU 19.6.0.0.200114
        file size matches (2308492999), md5 matches (0b2f7ae16f623e8d26905ae7ba600b06).

Found p6880880_190000_Linux-x86-64.zip : OPatch Utility
        file size matches (111682884), md5 matches (ad583938cc58d2e0805f3f9c309e7431).

Example of a failed media validation:

$ ./check-swlib.sh --ora-swlib-bucket gs://oracle-software --ora-version 12.2.0.1.0
Running with parameters from command line or environment variables:

ORA_SWLIB_BUCKET=gs://oracle-software
ORA_VERSION=12.2.0.1.0

Found V839960-01.zip : Oracle Database 12.2.0.1.0 for Linux x86-64
        file size matches (3453696911), md5 matches (1841f2ce7709cf909db4c064d80aae79).

Found V840012-01.zip : Oracle Grid Infrastructure 12.2.0.1.0 for Linux x86-64 for Linux x86-64
        file size matches (2994687209), md5 matches (ac1b156334cc5e8f8e5bd7fcdbebff82).

Found p29252035_122010_Linux-x86-64.zip : Combo Of OJVM Update Component 12.2.0.1.190416 + DB Update 12.2.0.1.190416 patch 29252035 for Linux x86-64
        file size matches (514033994), md5 matches (1a645dd57d06795a966a8882cc59243e).

Found p29301687_122010_Linux-x86-64.zip : Grid Infrastructure Release Update 12.2.0.1.190416 patch 29301687 for Linux x86-64
        file size matches (1736326653), md5 matches (1648e66220987dae6ecd757bc9f424ba).

Found p25078431_122010_Linux-x86-64.zip : ACFS MODULE ORACLEACFS.KO FAILS TO LOAD ON OL7U3 SERVER WITH RHCK (Patch) patch 25078431 for Linux x86-64
        file size matches (537299043), md5 matches (84ad563860b583fdd052bca0dcc33939).

Object gs://oracle-software/p29252072_122010_Linux-x86-64.zip COMBO OF OJVM RU COMPONENT 12.2.0.1.190416 + 12.2.0.1.190416 GIAPR2019RU not found: CommandException: One or more URLs matched no objects.

Found p29699173_122010_Linux-x86-64.zip : COMBO OF OJVM RU COMPONENT 12.2.0.1.190716 + 12.2.0.1.190716 GIJUL2019RU
        file size matches (2096740052), md5 matches (d5955b2e975752d3cd164a3e7db9aaaf).

Found p30133386_122010_Linux-x86-64.zip : COMBO OF OJVM RU COMPONENT 12.2.0.1.191015 12.2.0.1.191015 GIOCT2019RU
        file size matches (1925393453), md5 matches (25c30defbcc6e470e574fb3e16abb1d2).

Found p30463673_122010_Linux-x86-64.zip : COMBO OF OJVM RU COMPONENT 12.2.0.1.200114 12.2.0.1.200114 GIJAN2020RU
        file size matches (2135739707), md5 matches (04e26701ecdf04898abe363cdbeaaa40).

Found p6880880_122010_Linux-x86-64.zip : OPatch Utility
        size does not match (remote: 118408624, expected: 111682884), md5 does not match (remote: b8e1367997544ab2790c5bcbe65ca805, expected: ad583938cc58d2e0805f3f9c309e7431).

Prerequisite configuration

Before you run the tool you need to create JSON formatted configuration files for the data mount devices and the ASM disk group.

Data mount configuration file

In the data mount configuration file, you specify disk device attributes for:

Oracle software installation, which is usually mounted at /u01
Oracle diagnostic destination, which is usually mounted at /u02

In the configuration file, specify the block devices (actual devices, not partitions), the mount point names, the file system types, and the mount options in valid JSON format.

When you run the toolkit, specify the path to the configuration file by using either the --ora-data-mounts command line option or the ORA_DATA_MOUNTS environment variable. The file path can be relative or fully qualified. The file name defaults to data_mounts_config.json.

The following example shows a properly formatted JSON data mount configuration file:

[
    {
        "purpose": "software",
        "blk_device": "/dev/mapper/3600a098038314352502b4f782f446138",
        "name": "u01",
        "fstype":"xfs",
        "mount_point":"/u01",
        "mount_opts":"nofail"
    },
    {
        "purpose": "diag",
        "blk_device": "/dev/mapper/3600a098038314352502b4f782f446230",
        "name": "u02",
        "fstype":"xfs",
        "mount_point":"/u02",
        "mount_opts":"nofail"
    }
]

ASM disk group configuration file

In the ASM disk group configuration, specify the disk group names, the ASM disk names, and the associated block devices (the actual devices, not partitions) in valid JSON format.

When you run the toolkit, specify the path to the configuration file by using either the --ora-asm-disks command line option or the ORA_ASM_DISKS environment variable. The file path can be relative or fully qualified. The file name defaults to ask_disk_config.json.

The following example shows a properly formatted JSON ASM disk group configuration file:

[{
  "diskgroup": "DATA",
  "disks": [
    { "name": "DATA_1234567538", "blk_device": "/dev/mapper/3600a098038314344382b4f1234567538" },
    { "name": "DATA_123456752D", "blk_device": "/dev/mapper/3600a098038314344382b4f123456752d" },
    { "name": "DATA_1234567541", "blk_device": "/dev/mapper/3600a098038314344382b4f1234567541" },
    { "name": "DATA_1234567542", "blk_device": "/dev/mapper/3600a098038314344382b4f1234567542" },
    { "name": "DATA_1234567543", "blk_device": "/dev/mapper/3600a098038314344382b4f1234567543" },
    { "name": "DATA_1234567544", "blk_device": "/dev/mapper/3600a098038314344382b4f1234567544" },
  ]},
 {
  "diskgroup": "RECO",
  "disks": [
    { "name": "RECO_1234567546", "blk_device": "/dev/mapper/3600a098038314344382b4f1234567546" },
    { "name": "RECO_1234567547", "blk_device": "/dev/mapper/3600a098038314344382b4f1234567547" },
    { "name": "RECO_1234567548", "blk_device": "/dev/mapper/3600a098038314344382b4f1234567548" },
  ]}
]

Specifying LVM logical volumes

In addition to the raw devices, you can also specify LVM logical volumes by using the following format:

"blk_device": "/dev/mapper/oracle-data"

Configuring Installations

You run the toolkit by using the install-oracle.sh shell script.

IMPORTANT: From the control node, run the toolkit shell scripts by using a Linux user account that has the necessary SSH permissions and privileges on the target database server(s).

You need to specify the Cloud Storage bucket that contains the Oracle software and the backup destination for an initial RMAN backup. Running with the --help argument displays the list of available options.

Although the toolkit provides defaults for just about everything, in most cases, you need to customize your installation to some degree. Your customizations can range from simple items, such as the name of a database or the associated database edition, to less frequently adjusted items, such as ASM disk group configurations. Regardless, the toolkit allows you to specify overrides for most configuration parameters.

As well as creating the initial database, the toolkit implements and schedules a simple RMAN backup script. You can adjust the backup parameters either before or after running the toolkit, as required.

Configuration defaults

Most parameters have default values, so you only need to specify them when you need a different value. The parameter values that the toolkit uses are echoed during execution so you can confirm the configuration.

The complete list of parameters and their values are provided in the Parameters section.

Oracle User Directories

The Oracle convention for naming of file system mounts is /pm, where p is a string constant and m is a unique key, typically a two digit integer. The standard string constant for Oracle user file system mounts is the letter "u".

Following this convention, the toolkit creates the following default file system mounts:

/u01 - For Oracle software. For example, /u01/app/oracle/product.
/u02 - For other Oracle files, including software staging and, optionally, the Oracle Automatic Diagnostic Repository (ADR).

You don't have to use a separate file system, physical device, or logical volume for the software staging and other purposes. You can use the single file system, /u01, or whatever you choose to call it, if you want to.

Database backup configuration

As a part of installation, the toolkit creates an initial RMAN full database backup, an archived redo log backup, and sets the initial backup schedule based on your specifications or the default backup values.

The parameters for configuring your backups are described in Backup configuration parameters. The following list shows the default backup configuration implemented by the toolkit:

Backup scripts are stored in the directory /home/oracle/scripts.
Associated log files are stored in the directory /home/oracle/logs.
Weekly FULL database LEVEL=0 backups are run at 01:00 on Sundays.
Daily FULL database LEVEL=1 cumulative incremental backups are run at 01:00, Monday through Friday.
Hourly archived redo log backups run at 30 minutes past every hour.
RMAN backups are written to the Fast Recovery Area (FRA).

The toolkit schedules the backups by using the Linux cron utility under the Oracle software owner user. You can run the backup scripts as necessary.

After installation is complete, you can adjust any of the attributes of the backup scheme. You can also replace any and all parts of the initial backup scheme or the backup script with your own scripts or backup tools.

Parameters

The following sections document the parameters, organized by installation task and then by the attribute that you use the parameter to specify.

Most attributes can be specified by using either an environment variable or a command-line command. Environment variables are presented in capital letters. Command-line commands are presented in lower case letters and are preceded by two dashes.

You can specify parameters as either command line arguments or as predefined environment variables.

Default values for the parameters are shown in bold letters.

Target environment parameters

Attribute	Parameter options	Range of Values	Notes
Target server IP address	INSTANCE_IP_ADDR --instance-ip-addr	user defined - no default	The IP address of the target server to host the Oracle software and database. Applicable for Oracle "single instance" installations.
Primary server IP address	PRIMARY_IP_ADDR --primary-ip-addr	user defined - no default	The IP address of the primary server to use as source of primary database for Data Guard configuration. Applicable for Oracle "single instance" installations.
Target server host name	INSTANCE_HOSTNAME --instance-hostname	user defined INSTANCE_IP_ADDR	Optional hostname for the target server. Defaults to value of INSTANCE_IP_ADDR. Specifying a hostname adds clarity to log and debug files.
User on target server	INSTANCE_SSH_USER --instance-ssh-user	user defined current user	Remote user with connectivity (including privilege escalation capabilities) on target server.
Private key file for ssh connectivity to target server	INSTANCE_SSH_KEY --instance-ssh-key	user defined ~/.ssh/id_rsa
Ansible inventory file name	No environment variable --inventory-file	user defined toolkit generated	Optional Ansible inventory file name. If not supplied, the toolkit generates a filename.

Software installation parameters

Attribute	Parameters	Parameter Values	Notes
Oracle version	ORA_VERSION --ora-version	19.3.0.0.0 18.0.0.0.0 12.2.0.1.0 12.1.0.2.0 11.2.0.4.0	All mainstream major releases.
Oracle edition	ORA_EDITION --ora-edition	EE SE, for 11.2.0.4.0 only SE2, for 12.1.0.2.0 and above	SE or SE2 depending on the Oracle version chosen.
Software library type	ORA_SWLIB_TYPE --ora-swlib-type	GCS GCSFUSE NFS	Remote storage type acting as a software library where the required installation media is stored.
Software library location	ORA_SWLIB_BUCKET --ora-swlib-bucket	user defined - no default Example: gs://oracle-software	GCS bucket where the required base software and patches have been downloaded and staged. Only used when ORA_SWLIB_TYPE=GCS.
Software library path	ORA_SWLIB_PATH --ora-swlib-path	user defined /u01/swlib	Path where the required base software and patches have been downloaded and staged. Not used when ORA_SWLIB_TYPE=GCS.
Service account key file	ORA_SWLIB_CREDENTIALS --ora-swlib-credentials	user defined - no default	Service account key file name. Only used when ORA_SWLIB_TYPE=GCSFUSE.
Storage configuration	ORA_DATA_MOUNTS --ora-data-mounts	user defined data_mounts_config.json	Properly formatted JSON file providing mount and file system details for local mounts including installation location for the Oracle software and the location for Oracle diagnostic (ADR) directories. See Data mount configuration file.
Software unzip location	ORA_STAGING --ora-staging	user defined ORA_SWLIB_PATH	Working area for unzipping and staging software and installation files. Should have at least 16GB of available free space.
Listener Name	ORA_LISTENER_NAME --ora-listener-name	user defined LISTENER
Listener Port	ORA_LISTENER_PORT --ora-listener-port	user defined 1521
Preferred NTP server	NTP_PREF --ntp-pref	user defined - no default	Preferred NTP server to use in /etc/chrony.conf. Optional: set only if you need to manually define an NTP server, instead of relying on the OS defaults.
Swap device	SWAP_BLK_DEVICE --swap-blk-device	user defined - no default	Swap device to optionally create. Optional: set if you would like a swap partition and swap file created.

Storage configuration parameters

Attribute	Parameters	Parameter Values	Notes
ASM disk management	ORA_DISK_MGMT --ora-disk-mgmt	asmlib udev	ASMlib option is applicable to Oracle Linux as RHEL implementation requires Red Hat support. See MOS Doc ID: 1089399.1
Grid user role separation	ORA_ROLE_SEPARATION --ora-role-separation	true false	Role separation means that the Grid Infrastructure is owned by the OS user "grid" instead of the OS user "oracle".
Data disk group name	ORA_DATA_DISKGROUP --ora-data-diskgroup	user defined DATA	Default disk group for DB files for initial database.
Reco disk group name	ORA_RECO_DISKGROUP --ora-reco-diskgroup	user defined RECO	Default disk group for FRA files for initial database.
ASM disk configuration	ORA_ASM_DISKS --ora-asm-disks	user defined asm_disk_config.json	Name of an ASM configuration file that contains ASM disk definitions in valid JSON format. See ASM disk group configuration file.

Database configuration parameters

Attribute	Parameters	Parameter Values	Notes
Database name	ORA_DB_NAME --ora-db-name	user defined ORCL	Up to 8 characters. Must start with a letter. Other 7 characters can include alphanumeric characters, underscore, number sign, and dollar sign
Database domain	ORA_DB_DOMAIN --ora-db-domain	user defined .world	String of name components up to 128 characters long including periods.
Character set	ORA_DB_CHARSET --ora-db-charset	user defined AL32UTF8
National character set	ORA_DB_NCHARSET --ora-db-ncharset	user defined AL16UTF16
Database compatibility setting	COMPATIBLE_RDBMS --compatible-rdbms	user defined Oracle version	Defaults to the value of ORA_VERSION.
Container database	ORA_DB_CONTAINER --ora-db-container	true false	Not applicable for release 11.2.0.4.
PDB name	ORA_PDB_NAME --ora-pdb-name-prefix	PDB	Not applicable for release 11.2.0.4.
PDB count	ORA_PDB_COUNT --ora-pdb-count	1	If greater than 1, a numeric is appended to each PDB name. The PDB count may have Oracle licensing implications. Not applicable for release 11.2.0.4.
Database type	ORA_DB_TYPE --ora-db-type	MULTIPURPOSE DATA_WAREHOUSING OLTP
Redo log size	ORA_REDO_LOG_SIZE --ora-redo-log-size	user defined 100MB

RAC configuration parameters

Attribute

Parameters

Parameter Values

Notes

Cluster type

CLUSTER_TYPE
--cluster-type

NONE
RAC
DG

Specify "RAC" to install a RAC cluster. Use "DG" for standby installation. Otherwise a "Single Instance" installation is performed.

RAC specific configuration parameters

CLUSTER_CONFIG
--cluster-config

user defined
cluster_config.json

Used to specify the RAC scan listener name, port, IPs, and so forth. Also used to list RAC nodes.

Specifies a file containing properly formed JSON text.

Backup configuration parameters

RMAN backup destination	BACKUP_DEST --backup-dest	user defined - no default Example: +RECO	Disk group name or NFS file share location. Can include formatting options, such as "/u02/db_backups/ORCL_%I_%T_%s_%p.bak", for example. When writing to a non-ASM disk group location, include a valid RMAN format specification to ensure file name uniqueness, such as the example string shown above. If you are writing to a local file system, the directory does not have to exist, but initial backups will fail if the destination is not available or writeable.
RMAN full DB backup redundancy	BACKUP_REDUNDANCY --backup-redundancy	user defined field 2	An integer that specifies the number of full backups to keep.
RMAN archived redo log backup redundancy	ARCHIVE_REDUNDANCY --archive-redundancy	user defined field 2	An integer that specifies the number of times to redundantly backup archived redo logs into an RMAN backup set.
Archived redo logs online retention days	ARCHIVE_ONLINE_DAYS --archive-online-days	user defined field 7	Archived redo logs are only deleted from disk when they are older than this number of days. (And have been backed up with the specified redundancy.)
Day(s) of week for full DB backup (RMAN level=0)	BACKUP_LEVEL0_DAYS --backup-level0-days	user defined 0	Day(s) of week in cron format to be used for cron creation.
Day(s) of the week for incremental full DB backup (RMAN level=1)	BACKUP_LEVEL1_DAYS --backup-level1-days	user defined 1-6	Day(s) of week in cron format to be used for cron creation.
Start hour for RMAN full DB backups	BACKUP_START_HOUR --backup-start-hour	user defined 01	Hour in 24hour format. Used in cron for RMAN full (level=0 and level=1) backups.
Start minute for RMAN DB full backups	BACKUP_START_MIN --backup-start-min	user defined 00	Minute in XX format. Used in cron for RMAN full (level=0 and level=1) backups.
Start minute for archived redo log RMAN backups.	ARCHIVE_BACKUP_MIN --archive-backup-min	user defined 30	Minute in XX format. Used in cron for RMAN full (level=0 and level=1) backups.
Script location	BACKUP_SCRIPT_LOCATION --backup-script-location	user defined /home/oracle/scripts	Location for storing the provided RMAN backup scripts and other provided database scripts. Because only a handful of small text (.sh) files are provided, the freespace requirements for this directory is minimal.
Log file location	BACKUP_LOG_LOCATION --backup-log-location	user defined /home/oracle/logs	Location for storing log and output files from the provided RMAN backup scripts. Backup and log files are relatively small and hence the freespace requirements for this directory is minimal.

Additional operational parameters

Attribute	Parameters	Notes
Command help	--help	Display usage and all possible command line arguments.
Validate parameter definitions	--validate	Validate supplied parameters for such things as conformity to expected input types and exit. Nothing is installed or changed on the target server.
	--check-instance	Run the "check-instance.yml" playbook only.
	--prep-host	Run the "prep-host.yml" playbook only.
	--install-sw	Run the "install-sw.yml" playbook only.
	--no-patch	Install the base release, and do not apply patch set updates. Use in conjunction with patching functionality to apply patches post-installation.
	--config-db	Run the "config-db.yml" playbook only.
	--skip-database-config	Run all other playbooks but skip "config-db.yml", so that no database instance is created.
	--debug	Run with the Ansible debugging flag enabled.

Example Toolkit Execution

In the following example, environment variables are used to specify the following values:

The IP address of the target instance
The Oracle release
The database name

For all other parameters, the default values are accepted.

Note: Unless you specify a hostname on the INSTANCE_HOSTNAME environment variable or the --instance-hostname command line argument, the target hostname defaults to the target IP address.

$ export INSTANCE_IP_ADDR=10.150.0.42
$ export ORA_VERSION=19.3.0.0.0
$ export ORA_DB_NAME=PROD1
$ ./install-oracle.sh --ora-swlib-bucket gs://oracle-software --backup-dest +RECO

Inventory file for this execution: ./inventory_files/inventory_10.150.0.42_PROD1.

Running with parameters from command line or environment variables:

ANSIBLE_LOG_PATH=./logs/log_10.150.0.42_PROD1_20200610_160132.log
ARCHIVE_BACKUP_MIN=30
ARCHIVE_ONLINE_DAYS=7
ARCHIVE_REDUNDANCY=2
BACKUP_DEST=+RECO
BACKUP_LEVEL0_DAYS=0
BACKUP_LEVEL1_DAYS=1-6
BACKUP_LOG_LOCATION=/home/oracle/logs
BACKUP_REDUNDANCY=2
BACKUP_SCRIPT_LOCATION=/home/oracle/scripts
BACKUP_START_HOUR=01
BACKUP_START_MIN=00
CLUSTER_CONFIG=cluster_config.json
CLUSTER_TYPE=NONE
INSTANCE_HOSTGROUP_NAME=dbasm
INSTANCE_HOSTNAME=10.150.0.42
INSTANCE_IP_ADDR=10.150.0.42
INSTANCE_SSH_EXTRA_ARGS=''\''-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o IdentityAgent=no'\'''
INSTANCE_SSH_KEY='~/.ssh/id_rsa'
INSTANCE_SSH_USER=goryunov
ORA_ASM_DISKS=asm_disk_config.json
ORA_DATA_DISKGROUP=DATA
ORA_DATA_MOUNTS=data_mounts_config.json
ORA_DB_CHARSET=AL32UTF8
ORA_DB_CONTAINER=TRUE
ORA_DB_DOMAIN=world
ORA_DB_NAME=PROD1
ORA_DB_NCHARSET=AL16UTF16
ORA_DB_TYPE=MULTIPURPOSE
ORA_DISK_MGMT=UDEV
ORA_EDITION=EE
ORA_LISTENER_NAME=LISTENER
ORA_LISTENER_PORT=1521
ORA_PDB_COUNT=1
ORA_PDB_NAME_PREFIX=PDB
ORA_RECO_DISKGROUP=RECO
ORA_REDO_LOG_SIZE=100MB
ORA_RELEASE=latest
ORA_ROLE_SEPARATION=TRUE
ORA_STAGING=/u01/swlib
ORA_SWLIB_BUCKET=gs://oracle-software
ORA_SWLIB_CREDENTIALS=
ORA_SWLIB_PATH=/u01/swlib
ORA_SWLIB_TYPE='""'
ORA_VERSION=19.3.0.0.0
PB_CHECK_INSTANCE=check-instance.yml
PB_CONFIG_DB=config-db.yml
PB_CONFIG_RAC_DB=config-rac-db.yml
PB_INSTALL_SW=install-sw.yml
PB_LIST='check-instance.yml prep-host.yml install-sw.yml config-db.yml'
PB_PREP_HOST=prep-host.yml
PRIMARY_IP_ADDR=

Ansible params:
Found Ansible at /usr/bin/ansible-playbook

Running Ansible playbook: /usr/bin/ansible-playbook -i ./inventory_files/inventory_10.150.0.42_PROD1 check-instance.yml

PLAY [all] ******************************************************************************************************************************************

TASK [Verify that Ansible on control node meets the version requirements] ***************************************************************************
ok: [10.150.0.42] => {
    "changed": false,
    "msg": "Ansible version is 2.9.9, continuing"
}

TASK [Test connectivity to target instance via ping] ************************************************************************************************
ok: [10.150.0.42]

TASK [Abort if ping module fails] *******************************************************************************************************************
ok: [10.150.0.42] => {
    "changed": false,
    "msg": "The instance has an usable python installation, continuing"
}

TASK [Collect facts from target] ********************************************************************************************************************
ok: [10.150.0.42]

... output truncated for brevity

In the following example, command-line arguments are used to specify the Oracle Standard Edition and to create a non-container database.

$ ./install-oracle.sh --ora-edition SE2 --ora-db-container false \
 --ora-swlib-bucket gs://oracle-software --backup-dest +RECO \
 --instance-ip-addr 10.150.0.42

Inventory file for this execution: ./inventory_files/inventory_dbserver_ORCL.

Running with parameters from command line or environment variables:

ANSIBLE_LOG_PATH=./logs/log_dbserver_ORCL_20200610_161259.log
ARCHIVE_BACKUP_MIN=30
ARCHIVE_ONLINE_DAYS=7
ARCHIVE_REDUNDANCY=2
BACKUP_DEST=+RECO
BACKUP_LEVEL0_DAYS=0
BACKUP_LEVEL1_DAYS=1-6
BACKUP_LOG_LOCATION=/home/oracle/logs
BACKUP_REDUNDANCY=2
BACKUP_SCRIPT_LOCATION=/home/oracle/scripts
BACKUP_START_HOUR=01
BACKUP_START_MIN=00
CLUSTER_CONFIG=cluster_config.json
CLUSTER_TYPE=NONE
INSTANCE_HOSTGROUP_NAME=dbasm
INSTANCE_HOSTNAME=dbserver
INSTANCE_IP_ADDR=10.150.0.42
INSTANCE_SSH_EXTRA_ARGS=''\''-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o IdentityAgent=no'\'''
INSTANCE_SSH_KEY='~/.ssh/id_rsa'
INSTANCE_SSH_USER=goryunov
ORA_ASM_DISKS=asm_disk_config.json
ORA_DATA_DISKGROUP=DATA
ORA_DATA_MOUNTS=data_mounts_config.json
ORA_DB_CHARSET=AL32UTF8
ORA_DB_CONTAINER=false
ORA_DB_DOMAIN=world
ORA_DB_NAME=ORCL
ORA_DB_NCHARSET=AL16UTF16
ORA_DB_TYPE=MULTIPURPOSE
ORA_DISK_MGMT=UDEV
ORA_EDITION=SE2
ORA_LISTENER_NAME=LISTENER
ORA_LISTENER_PORT=1521
ORA_PDB_COUNT=1
ORA_PDB_NAME_PREFIX=PDB
ORA_RECO_DISKGROUP=RECO
ORA_REDO_LOG_SIZE=100MB
ORA_RELEASE=latest
ORA_ROLE_SEPARATION=TRUE
ORA_STAGING=/u01/swlib
ORA_SWLIB_BUCKET=gs://oracle-software
ORA_SWLIB_CREDENTIALS=
ORA_SWLIB_PATH=/u01/swlib
ORA_SWLIB_TYPE='""'
ORA_VERSION=19.3.0.0.0
PB_CHECK_INSTANCE=check-instance.yml
PB_CONFIG_DB=config-db.yml
PB_CONFIG_RAC_DB=config-rac-db.yml
PB_INSTALL_SW=install-sw.yml
PB_LIST='check-instance.yml prep-host.yml install-sw.yml config-db.yml'
PB_PREP_HOST=prep-host.yml
PRIMARY_IP_ADDR=

Ansible params:
Found Ansible at /usr/bin/ansible-playbook

Running Ansible playbook: /usr/bin/ansible-playbook -i ./inventory_files/inventory_dbserver_19.3.0.0.0_ORCL   check-instance.yml

PLAY [all] ******************************************************************************************************************************************

TASK [Verify that Ansible on control node meets the version requirements] ***************************************************************************
ok: [dbserver] => {
    "changed": false,
    "msg": "Ansible version is 2.9.9, continuing"
}

TASK [Test connectivity to target instance via ping] ************************************************************************************************
ok: [dbserver]

TASK [Abort if ping module fails] *******************************************************************************************************************
ok: [dbserver] => {
    "changed": false,
    "msg": "The instance has an usable python installation, continuing"
}

TASK [Collect facts from target] ********************************************************************************************************************
ok: [dbserver]

... <output truncated for brevity>

The following example shows the error message that is received when an invalid parameter is specified.

$ ./install-oracle.sh --ora-version=7.3.4 --ora-swlib-bucket gs://oracle-software --backup-dest +RECO
Incorrect parameter provided for ora-version: 7.3.4

Post installation tasks

Reset passwords

The Oracle toolkit does not use or store any passwords. At runtime, passwords for the Oracle SYS and SYSTEM database users are set with strong, unique, and randomized passwords that are not written to or persisted in any OS files.

Change the passwords immediately after running the toolkit.

To change the passwords, connect to the database by using a SYSDBA administrative connection and change the passwords by using the SQL Plus password command:

sqlplus / as sysdba

SQL> password SYSTEM

Validate the environment

After deployment, you can validate your environment using several scripts that are provided with the toolkit.

Listing Oracle ASM devices

You can list the devices that are used by Oracle ASM using the asm_disks.sh script, as shown in the following example:

$ /home/oracle/scripts/asm_disks.sh
Disk device /dev/sdd1 may be an ASM disk - Disk name: DATA_0000
Disk device /dev/sde1 may be an ASM disk - Disk name: RECO_0000
Disk device /dev/sdf1 may be an ASM disk - Disk name: DEMO_0000

Displaying cluster resource status

You can generate a report that shows the Oracle Restart cluster resources by using the crs_check.sh script, as shown in the following example:

$ /home/oracle/scripts/crs_check.sh

***** CRS STATUS *****

Oracle High Availability Services release version on the local node is [18.0.0.0.0]
Oracle High Availability Services version on the local node is [18.0.0.0.0]
CRS-4638: Oracle High Availability Services is online
CRS-4529: Cluster Synchronization Services is online

NAME=ora.DATA.dg
TYPE=ora.diskgroup.type
TARGET=ONLINE
STATE=ONLINE on db-host-1

... output truncated for brevity

Verify an Oracle cluster

You can verify the integrity of an Oracle cluster by using the cluvfy_checks.sh script to run the Oracle Cluster Verify utility.

Oracle validation utilities

For a more comprehensive validation, use the utilities provided by Oracle Support, which are available for download from My Oracle Support:

Autonomous Health Framework (AHF) - Including TFA and ORAchk/EXAChk (Doc ID 2550798.1)

Oracle's Autonomous Health Framework (AHF) includes utilities such as Trace File Analyzer (TFA). TFA provides options such as tfactl summary, which gives a complete environment overview. AHF also includes a copy of ORAchk. Refer to the Oracle documentation for more information.

The following example shows the use of the Oracle Cluster Verify utility:

$ /home/oracle/scripts/cluvfy_checks.sh

Verifying Oracle Restart Integrity ...PASSED

Verification of Oracle Restart integrity was successful.

CVU operation performed:      Oracle Restart integrity
Date:                         Jul 15, 2019 11:26:15 PM
CVU home:                     /u01/app/18.0.0/grid/
User:                         oracle

Verifying Physical Memory ...
  Node Name     Available                 Required                  Status
  ------------  ------------------------  ------------------------  ----------
  dbserver      14.5286GB (1.523434E7KB)  8GB (8388608.0KB)         passed
Verifying Physical Memory ...PASSED

... output truncated for brevity

Verifying Users With Same UID: 0 ...PASSED
Verifying Root user consistency ...
  Node Name                             Status
  ------------------------------------  ------------------------
  dbserver                              passed
Verifying Root user consistency ...PASSED

Verification of system requirement was successful.

CVU operation performed:      system requirement
Date:                         Jul 15, 2019 11:26:26 PM
CVU home:                     /u01/app/18.0.0/grid/
User:                         oracle

Patching

You can apply Oracle Release Update (RU) or Patch Set Update (PSU) patches to both the Grid Infrastructure and Database homes by using the apply-patch.sh script of the toolkit.

By default, install-oracle.sh updates to the latest available patch. To apply a specific patch instead, use the --no-patch option in install-oracle.sh to skip patching at installation time. After installation is complete, execute apply-patch.sh with the --ora-release option. Specify the full release name including timestamp; a list of release names is available in https://github.com/google/bms-toolkit/tree/master/roles/common/defaults/main.yml under gi-patches and rdbms-patches.

A digest of the required patch files, including checksum hashes is provided in the file roles/common/defaults/main.yml.

To apply patches, you need to specify the location of the software library. You can optionally specify the base database version, the release to patch to, the database name, the staging location, and other optional parameters.

Example:

$ ./apply-patch.sh --help
        Usage: apply-patch.sh
         --ora-swlib-bucket <value>
         --inventory-file <value>
         [ --ora-version <value> ]
         [ --ora-release <value> ]
         [ --ora-swlib-path <value> ]
         [ --ora-staging <value> ]
         [ --ora-db-name <value> ]
         [ --help ]
         [ --validate ]
         -- [parameters sent to ansible]

$ ./apply-patch.sh \
  --ora-swlib-bucket gs://oracle-software \
  --ora-swlib-path /u02/oracle_install \
  --ora-staging /u02/oracle_install \
  --ora-version 19.3.0.0.0 \
  --ora-release 19.6.0.0.200114 \
    --inventory-file inventory_files/inventory_toolkit-db2_ORCL

Running with parameters from command line or environment variables:

ORA_DB_NAME=ORCL
ORA_RELEASE=19.6.0.0.200114
ORA_STAGING=/u02/oracle_install
ORA_SWLIB_BUCKET=gs://oracle-software
ORA_SWLIB_PATH=/u02/oracle_install
ORA_VERSION=19.3.0.0.0

Ansible params: -i inventory_files/inventory_toolkit-db2_ORCL
Found Ansible at /usr/bin/ansible-playbook
Running Ansible playbook: /usr/bin/ansible-playbook -i inventory_files/inventory_toolkit-db2_ORCL   patch.yml

PLAY [OPatch Restart patch] ****************************************************

TASK [Gathering Facts] *********************************************************
ok: [toolkit-db2]

TASK [Verify Ansible meets the version requirements] ********************************************************************************
ok: [toolkit-db2] => {
    "changed": false,
    "msg": "Ansible version is 2.9.6, continuing"
}

TASK [patch : Update OPatch in GRID_HOME]

... output truncated for brevity

A note on patch metadata

See patching-metadata.md for details on updating quarterly Release Update metadata.

Patching RAC databases

The patching process for RAC databases is based on applying the OJVM+GI RU "combo patches", which contain all necessary Release Update (RU) patches for both the GI and RDBMS homes, as well as the Oracle Java Virtual Machine (OJVM) database patches.

All GI patching is implemented using the opatchauto utility run by the root user from the GI home and, therefore, if the GI and RDBMS homes are of the same major release, patches both the GI and RDBMS homes in a single step.

By default, if RAC GI and RDBMS homes are of the same base release, the install-oracle.sh script applies the latest RU/PSU patch to both.

You can skip all RU/PSU patching steps and install only the base software by specifying the command line option --no-patch. You can then apply patches separately later, either manually or by using the toolkit.

Alternatively, you can apply RAC GI and RDBMS patches independently from the base software installations by using the script apply-patch.sh. This script also allows you apply patches with more granularity.

The list of RU/PSU patches to apply are defined by the gi_patches and rdbms_patches variables. By default, both are specified in the roles/common/defaults/main.yml Ansible file. For each major Oracle release, you can specify multiple versions from the various quarterly releases When you install by using install-oracle.sh, the toolkit uses the most recent patch version.

When creating a database, if the RDBMS home software is no longer at the base release because it was patched during installation, the toolkit uses the Oracle datapatch utility to apply patches at the database level, which is known as SQL level patching.

BMS RAC install with latest RU

The following example RAC installation command applies the PSU/RU patches, which is the default behavior:

./install-oracle.sh \
  --ora-swlib-bucket gs://oracle-software \
  --instance-ssh-key '~/.ssh/id_rsa' \
  --instance-ssh-user dba-user \
  --ora-swlib-path /u01/oracle_install \
  --ora-staging /u01/oracle_install \
  --backup-dest "+DATA" \
  --ora-version 19.3.0.0.0 \
  --ora-swlib-type gcs \
  --compatible-rdbms "11.2.0.4.0" \
  --ora-asm-disks bms_asm.json \
  --ora-data-mounts bms_mounts.json \
  --cluster-type RAC \
  --cluster-config bms_cluster.json \
  --ora-reco-diskgroup DATA \
  --ora-db-name ORCL

If you do not specify a value on the --compatible-rdbms parameter, the RDBMS compatibility of the ASM disk group is set to the major version level that is defined on the --ora-version parameter.

To patch RAC databases, the toolkit performs the following actions:

Stops the RAC databases in their homes by using the "stop home" option from the master node.
Stops TFA.
Kills the asmcmd daemon processes.
Executes opatchauto apply, patching both nodes.
Restarts the services, including start home.
On the master node only, runs the datapatch utility over several iterations to resolve any PDB invalid states.

Regardless of which script is used, the specifics about which patch files to use, such as the file names of the source media, patch paths, and the software versions, are taken from the gi_patches and rdbms_patches environment variables. The defaults for these variables are defined in the roles/common/defaults/main.yml Ansible file. You can override the default values or specify patches that are not included in the roles/common/defaults/main.yml file in a properly structured JSON file that you reference on the --extra-vars Ansible argument.

The following example shows a YAML file that contains patch specifications for both the GI and RDBMS software:

gi_patches:
  - { category: "RU", base: "19.3.0.0.0", release: "19.7.0.0.200414", patchnum: "30783556", patchfile: "p30783556_190000_Linux-x86-64.zip", patch_subdir: "/30899722", prereq_check: FALSE, method: "opatchauto apply", ocm: FALSE, upgrade: FALSE }

rdbms_patches:
  - { category: "RU_Combo", base: "19.3.0.0.0", release: "19.7.0.0.200414", patchnum: "30783556", patchfile: "p30783556_190000_Linux-x86-64.zip", patch_subdir: "/30805684", prereq_check: TRUE, method: "opatch apply", ocm: FALSE, upgrade: TRUE }

The following example shows the specification of the YAML file by using the --extra-vars Ansible parameter:

./install-oracle.sh \
  --ora-swlib-bucket gs://oracle-software \
  --instance-ssh-key '~/.ssh/id_rsa' \
  --instance-ssh-user dba-user \
  --ora-swlib-path /u01/oracle_install \
  --ora-staging /u01/oracle_install \
  --backup-dest "+DATA" \
  --ora-version 19.3.0.0.0 \
  --ora-swlib-type gcs \
  --compatible-rdbms "11.2.0.4.0" \
  --ora-asm-disks bms_asm.json
  --ora-data-mounts bms_mounts.json \
  --cluster-type RAC \
  --cluster-config bms_cluster.json \
  --ora-reco-diskgroup DATA \
  --ora-db-name ORCL \
  -- "--extra-vars @patches.yaml"

Note: you can use the --extra-vars Ansible parameter to specify other things besides a JSON file that contains patch details, such as specifying other optional or ad-hoc Ansible run-time parameters.

To apply additional granularity for patches when you use the apply-patch.sh script use the --ora-version and --ora-release parameters to specify a major Oracle version and a specific patch release level, respectively.

For example:

./apply-patch.sh \
  --ora-swlib-bucket gs://oracle-software \
  --inventory-file inventory_files/inventory_ORCL_RAC \
  --ora-version 19.3.0.0.0 \
  --ora-release 19.7.0.0.200414 \
  --ora-swlib-path /u01/oracle_install \
  --ora-staging /u01/oracle_install \
  --ora-db-name ORCL
  -- "--extra-vars @patches.yaml"

Destructive Cleanup

If you need to uninstall the Oracle software, the Oracle software includes uninstallation options, which are the recommended way to uninstall Oracle software.

If necessary, you can use the script cleanup-oracle.sh to perform a destructive or a brute-force clean-up of Oracle databases, services, and software from a specified target database server. A brute-force clean-up takes the following actions:

Kills all running Oracle services.
Deconfigures the Oracle Restart software.
Removes Oracle related directories and files.
Removes Oracle software owner users and groups.
Re-initializes ASM storage devices and uninstalls ASMlib if installed.

Important: a destructive cleanup permanently deletes the databases and any data they contain. Any backups that are stored local to the server are also deleted. Backups stored in Cloud Storage, Cloud Storage FUSE, or NFS devices are not affected by a destructive cleanup.

Recommendation: provide a value for the role separation parameter, which defaults to TRUE. On the --inventory-file parameter, specify the location of the inventory file:

$ ./cleanup-oracle.sh --help
        Usage: cleanup-oracle.sh
          --ora-version <value>
          --inventory-file <value>
          --yes-i-am-sure
          [ --ora-role-separation <value> ]
          [ --ora-disk-mgmt <value> ]
          [ --ora-swlib-path <value> ]
          [ --ora-staging <value> ]
          [ --ora-asm-disks <value> ]
          [ --ora-data-mounts <value> ]
          [ --help ]

Sample usage:

$ ./cleanup-oracle.sh --ora-version 19 \
--inventory-file ./inventory_files/inventory_oracledb1_ORCL \
--yes-i-am-sure \
--ora-disk-mgmt udev \
--ora-swlib-path /u02/oracle_install \
--ora-staging /u02/oracle_install \
--ora-asm-disks asm_disk_config.json \
--ora-data-mounts data_mounts_config.json

Running with parameters from command line or environment variables:

INVENTORY_FILE=./inventory_files/inventory_oracledb1_ORCL
ORA_ASM_DISKS=asm_disk_config.json
ORA_DATA_MOUNTS=data_mounts_config.json
ORA_DISK_MGMT=udev
ORA_ROLE_SEPARATION=TRUE
ORA_STAGING=/u02/oracle_install
ORA_SWLIB_PATH=/u02/oracle_install
ORA_VERSION=19.3.0.0.0

Ansible params:
Found Ansible at /usr/bin/ansible-playbook

Running Ansible playbook: /usr/bin/ansible-playbook -i ./inventory_files/inventory_oracledb1_ORCL  brute-cleanup.yml

PLAY [all] ************************************************************************************
... output truncated for brevity

Files

user-guide.md

Latest commit

History

user-guide.md

File metadata and controls

Toolkit for Bare Metal Solution: User Guide

Table of Contents

Command quick reference for single instance deployments

Command quick reference for RAC deployments

Command quick reference for DR deployments

Overview

Software Stack

Requirements and Prerequisites

Control node requirements

Target server requirements

Installing the toolkit

Downloading and staging the Oracle Software

Downloading the Oracle installation software

Downloading Patches from My Oracle Support

Required Oracle Software - Download Summary

Staging the Oracle installation media

Cloud Storage bucket

Cloud Storage FUSE

NFS share

Validating Media

Prerequisite configuration

Data mount configuration file

ASM disk group configuration file

Specifying LVM logical volumes

Configuring Installations

Configuration defaults

Oracle User Directories

Database backup configuration

Parameters

Target environment parameters

Software installation parameters

Storage configuration parameters

Database configuration parameters

RAC configuration parameters

Backup configuration parameters

Additional operational parameters

Example Toolkit Execution

Post installation tasks

Reset passwords

Validate the environment

Listing Oracle ASM devices

Displaying cluster resource status

Verify an Oracle cluster

Oracle validation utilities

Patching

A note on patch metadata

Patching RAC databases

BMS RAC install with latest RU

Destructive Cleanup