Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error when deploying RAC #3

Closed
kuhafa opened this issue Jan 2, 2019 · 21 comments
Closed

Error when deploying RAC #3

kuhafa opened this issue Jan 2, 2019 · 21 comments

Comments

@kuhafa
Copy link

kuhafa commented Jan 2, 2019

Hello Tim,
I have run this script multiple time. I single instances or dataguard are deployed without any issues. However, I have not been able to create RAC on 12 or 18c. These error below are just driving me crazy.

The dns and node2 work perfectly, but when I start node1, I get the error that the disk is not unique even after I destroy and redeploy the script. I think the rest of the error are just chain reaction of the first failure. Please advice.

default: default: Installed: default: cvuqdisk.x86_64 0:1.0.10-1 default: default: Complete! default: ****************************************************************************** default: Do grid software-only installation. Sun Dec 30 06:21:30 UTC 2018 default: ****************************************************************************** default: Launching Oracle Grid Infrastructure Setup Wizard... **default: [FATAL] [INS-30516] Please specify unique disk groups. default: CAUSE: Installer has detected that the diskgroup name provided already exists on the system. default: ACTION: Specify different disk group. default: [FATAL] [INS-30530] Following specified disks have invalid header status: [/dev/oracleasm/asm-disk1, /dev/oracleasm/asm-disk3, /dev/oracleasm/asm-disk4] default: ACTION: Ensure only Candidate or Provisioned disks are specified.** default: ****************************************************************************** default: Run grid root scripts. Sun Dec 30 06:22:23 UTC 2018 default: ****************************************************************************** **default: sh: /u01/app/oraInventory/orainstRoot.sh: No such file or directory default: sh: /u01/app/oraInventory/orainstRoot.sh: No such file or directory** default: Check /u01/app/18.0.0/grid/install/root_ol7-183-rac1.localdomain_2018-12-30_06-22-23-942200686.log for the output of root script **default: sh: /u01/app/18.0.0/grid/root.sh: No such file or directory** default: ****************************************************************************** default: Do grid configuration. Sun Dec 30 06:22:24 UTC 2018 default: ****************************************************************************** default: Launching Oracle Grid Infrastructure Setup Wizard... **default: [FATAL] [INS-32603] The central inventory was not detected.** default: ACTION: The -executeConfigTools flag can only be used for an Oracle home software that has been already installed using the configure or upgrade options. Ensure that the orainstRoot.sh script, from the inventory location, has been executed. default: ****************************************************************************** default: Check cluster configuration. Sun Dec 30 06:22:27 UTC 2018 default: ****************************************************************************** default: /vagrant/scripts/oracle_grid_software_config.sh: line 45: /u01/app/18.0.0/grid/bin/crsctl: No such file or directory default: ****************************************************************************** default: Unzip database software. Sun Dec 30 06:22:27 UTC 2018 default: ****************************************************************************** default: ****************************************************************************** default: Do database software-only installation. Sun Dec 30 06:24:40 UTC 2018 default: ****************************************************************************** default: Launching Oracle Database Setup Wizard... **default: [FATAL] [INS-35354] The system on which you are attempting to install Oracle RAC is not part of a valid cluster. default: CAUSE: Before you can install Oracle RAC, you must install Oracle Grid Infrastructure (Oracle Clusterware and Oracle ASM) on all servers to create a cluster. default: ACTION: Oracle Grid Infrastructure for Clusterware is not installed.** Install it either from the separate installation media included in your media pack, or install it by downloading it from Electronic Product Delivery (EPD) or the Oracle Technology Network (OTN). Oracle Grid Infrastructure normally is installed by a different operating system user than the one used for Oracle Database. It may need to be installed by your system administrator. See the installation guide for more details. default: ****************************************************************************** default: Run DB root scripts. Sun Dec 30 06:24:47 UTC 2018 default: ****************************************************************************** default: Check /u01/app/oracle/product/18.0.0/dbhome_1/install/root_ol7-183-rac1.localdomain_2018-12-30_06-24-47-612572414.log for the output of root script default: sh: /u01/app/oracle/product/18.0.0/dbhome_1/root.sh: No such file or directory default: ****************************************************************************** default: Create database. Sun Dec 30 06:24:47 UTC 2018 default: ****************************************************************************** default: [FATAL] java.lang.NullPointerException default: ****************************************************************************** default: Check cluster configuration. Sun Dec 30 06:24:50 UTC 2018 default: ****************************************************************************** default: ****************************************************************************** default: Output from crsctl stat res -t Sun Dec 30 06:24:50 UTC 2018 default: ****************************************************************************** default: /vagrant/scripts/oracle_create_database.sh: line 35: /u01/app/18.0.0/grid/bin/crsctl: No such file or directory default: ****************************************************************************** default: Output from srvctl config database -d cdbrac Sun Dec 30 06:24:50 UTC 2018 default: ****************************************************************************** default: /u01/app/oracle/product/18.0.0/dbhome_1/bin/srvctl: line 255: /u01/app/oracle/product/18.0.0/dbhome_1/srvm/admin/getcrshome: No such file or directory default: PRCD-1027 : Failed to retrieve database cdbrac default: PRCR-1070 : Failed to check if resource ora.cdbrac.db is registered default: CRS-0184 : Cannot communicate with the CRS daemon. default: ****************************************************************************** default: Output from srvctl status database -d cdbrac Sun Dec 30 06:24:51 UTC 2018 default: ****************************************************************************** default: /u01/app/oracle/product/18.0.0/dbhome_1/bin/srvctl: line 255: /u01/app/oracle/product/18.0.0/dbhome_1/srvm/admin/getcrshome: No such file or directory default: PRCD-1027 : Failed to retrieve database cdbrac default: PRCR-1070 : Failed to check if resource ora.cdbrac.db is registered default: CRS-0184 : Cannot communicate with the CRS daemon. default: ****************************************************************************** default: Output from v$active_instances Sun Dec 30 06:24:51 UTC 2018 default: ****************************************************************************** default: /vagrant/scripts/oracle_create_database.sh: line 50: /u01/app/oracle/product/18.0.0/dbhome_1/bin/sqlplus: Permission denied The SSH command responded with a non-zero exit status. Vagrant assumes that this means the command failed. The output for this command should be in the log above. Please read the output to determine what went wrong.

Thanks

@oraclebase
Copy link
Owner

I've just destroyed an instance and I'm trying it again now, in case an update to the 'bento/oracle-7.5' box has altered anything. It's going to take a couple of hours to complete, but I should get to this point quite quickly.

Note. When you destroy the current setup, make sure none of the shared disks are left behind. If they are, they will be reused, and that would be a problem. :)

@oraclebase
Copy link
Owner

Just checked. I got this, which was expected.

default: ******************************************************************************
default: Do grid software-only installation. Wed Jan 2 14:36:46 UTC 2019
default: ******************************************************************************
default: Launching Oracle Grid Infrastructure Setup Wizard...
default: [WARNING] [INS-41808] Possible invalid choice for OSASM Group.
default:    CAUSE: The name of the group you selected for the OSASM group is commonly used to grant other system privileges (For example: asmdba, asmoper, dba, oper).
default:    ACTION: Oracle recommends that you designate asmadmin as the OSASM group.
default: [WARNING] [INS-41809] Possible invalid choice for OSDBA Group.
default:    CAUSE: The group name you selected as the OSDBA for ASM group is commonly used for Oracle Database administrator privileges.
default:    ACTION: Oracle recommends that you designate asmdba as the OSDBA for ASM group, and that the group should not be the same group as an Oracle Database OSDBA group.
default: [WARNING] [INS-41812] OSDBA and OSASM are the same OS group.
default:    CAUSE: The chosen values for OSDBA group and the chosen value for OSASM group are the same.
default:    ACTION: Select an OS group that is unique for ASM administrators. The OSASM group should not be the same as the OS groups that grant privileges for Oracle ASM access, or for database administration.
default: [WARNING] [INS-40109] The specified Oracle Base location is not empty on this server.
default:    ACTION: Specify an empty location for Oracle Base.
default: [WARNING] [INS-13014] Target environment does not meet some optional requirements.
default:    CAUSE: Some of the optional prerequisites are not met. See logs for details. gridSetupActions2019-01-02_02-36-46PM.log
default:    ACTION: Identify the list of failed prerequisite checks from the log: gridSetupActions2019-01-02_02-36-46PM.log. Then either from the log file or from installation manual find the appropriate configuration to meet the prerequisites and fix it manually.
default: The response file for this session can be found at:
default:  /u01/app/18.0.0/grid/install/response/grid_2019-01-02_02-36-46PM.rsp
default: You can find the log of this install session at:
default:  /tmp/GridSetupActions2019-01-02_02-36-46PM/gridSetupActions2019-01-02_02-36-46PM.log

Just expected warnings.

It does seem like there is something wrong with your shared disks.

I would suggest:

  1. Destroy all three VMs.
  2. Check to see if the shared disks have been removed properly. If not, remove them.
  3. Check the VirtualBox interface. Make sure it's not complaining about missing disks. If it is, then clean them up and remove them. Maybe some old disk reference is affecting your build?

Also I would check:

  1. You are on the latest version of VirtualBox.
  2. You are on the latest version of Vagrant.

Cheers

Tim...

PS. I'm going to update the box to 'bento/oracle-7.6' and try it. The box only came out recently, so I'm not sure if it works yet.

@kuhafa
Copy link
Author

kuhafa commented Jan 2, 2019

Got, let me double check to make sure the shared disks are destroyed as well.
I'm currently using VirtualBox 6.0.

Thanks for the quick feedback!

@oraclebase
Copy link
Owner

What OS are you using as the host?

@kuhafa
Copy link
Author

kuhafa commented Jan 2, 2019

I'm using a MAC OS. I just found out there indeed some leftover shared disks. I just removed them and will be rerun the script soon.

@oraclebase
Copy link
Owner

OK. Let me know how you get on.

I've check added a clarification to the README.txt about a message at the end of the node2 build and I'll put one in about the disk cleanup.

The build with the 'bento/oracle-7.6' box is at the ASM config stage (post root scripts), but I won't commit that change until I've seen it complete.

@oraclebase
Copy link
Owner

18c build using 'bento/oracle-7.6' completed successfully. Change committed.

Trying now with 12.2.

@kuhafa
Copy link
Author

kuhafa commented Jan 2, 2019

Tim,
This is the second (run) time I'm getting this error. Did your build with 7.6 ran successfully?

default: ****************************************************************************** default: Create database. Wed Jan 2 17:01:34 UTC 2019 default: ****************************************************************************** default: Prepare for db operation default: 7% complete default: Copying database files default: 8% complete default: [WARNING] ORA-12547: TNS:lost contact default: 27% complete default: 100% complete default: [FATAL] Error while cataloging RMAN Backups default: 7% complete default: 0% complete default: Look at the log file "/u01/app/oracle/cfgtoollogs/dbca/cdbrac/cdbrac.log" for further details. default: ****************************************************************************** default: Check cluster configuration. Wed Jan 2 17:14:48 UTC 2019 default: ****************************************************************************** default: ****************************************************************************** default: Output from crsctl stat res -t Wed Jan 2 17:14:48 UTC 2019 default: ****************************************************************************** default: -------------------------------------------------------------------------------- default: Name Target State Server State details default: -------------------------------------------------------------------------------- default: Local Resources default: -------------------------------------------------------------------------------- default: ora.ASMNET1LSNR_ASM.lsnr default: ONLINE ONLINE ol7-183-rac1 STABLE default: ONLINE ONLINE ol7-183-rac2 STABLE default: ora.DATA.GHCHKPT.advm default: OFFLINE OFFLINE ol7-183-rac1 STABLE default: OFFLINE OFFLINE ol7-183-rac2 STABLE default: ora.DATA.dg default: ONLINE ONLINE ol7-183-rac1 STABLE default: ONLINE ONLINE ol7-183-rac2 STABLE default: ora.LISTENER.lsnr default: ONLINE ONLINE ol7-183-rac1 STABLE default: ONLINE ONLINE ol7-183-rac2 STABLE default: ora.data.ghchkpt.acfs default: OFFLINE OFFLINE ol7-183-rac1 volume /opt/oracle/r default: hp_images/chkbase is default: unmounted,STABLE default: OFFLINE OFFLINE ol7-183-rac2 STABLE default: ora.helper default: OFFLINE OFFLINE ol7-183-rac1 STABLE default: OFFLINE OFFLINE ol7-183-rac2 IDLE,STABLE default: ora.net1.network default: ONLINE ONLINE ol7-183-rac1 STABLE default: ONLINE ONLINE ol7-183-rac2 STABLE default: ora.ons default: ONLINE ONLINE ol7-183-rac1 STABLE default: ONLINE ONLINE ol7-183-rac2 STABLE default: ora.proxy_advm default: ONLINE ONLINE ol7-183-rac1 STABLE default: ONLINE ONLINE ol7-183-rac2 STABLE default: -------------------------------------------------------------------------------- default: Cluster Resources default: -------------------------------------------------------------------------------- default: ora.LISTENER_SCAN1.lsnr default: 1 ONLINE ONLINE ol7-183-rac2 STABLE default: ora.LISTENER_SCAN2.lsnr default: 1 ONLINE ONLINE ol7-183-rac1 STABLE default: ora.LISTENER_SCAN3.lsnr default: 1 ONLINE ONLINE ol7-183-rac1 STABLE default: ora.MGMTLSNR default: 1 OFFLINE OFFLINE STABLE default: ora.asm default: 1 ONLINE ONLINE ol7-183-rac1 Started,STABLE default: 2 ONLINE ONLINE ol7-183-rac2 Started,STABLE default: 3 OFFLINE OFFLINE STABLE default: ora.cdbrac.db default: 1 OFFLINE OFFLINE STABLE default: 2 OFFLINE OFFLINE STABLE default: ora.cvu default: 1 ONLINE ONLINE ol7-183-rac1 STABLE default: ora.ol7-183-rac1.vip default: 1 ONLINE ONLINE ol7-183-rac1 STABLE default: ora.ol7-183-rac2.vip default: 1 ONLINE ONLINE ol7-183-rac2 STABLE default: ora.qosmserver default: 1 ONLINE ONLINE ol7-183-rac1 STABLE default: ora.rhpserver default: 1 OFFLINE OFFLINE STABLE default: ora.scan1.vip default: 1 ONLINE ONLINE ol7-183-rac2 STABLE default: ora.scan2.vip default: 1 ONLINE ONLINE ol7-183-rac1 STABLE default: ora.scan3.vip default: 1 ONLINE ONLINE ol7-183-rac1 STABLE default: -------------------------------------------------------------------------------- default: ****************************************************************************** default: Output from srvctl config database -d cdbrac Wed Jan 2 17:14:48 UTC 2019 default: ****************************************************************************** default: Database unique name: cdbrac default: Database name: cdbrac default: Oracle home: /u01/app/oracle/product/18.0.0/dbhome_1 default: Oracle user: oracle default: Spfile: default: Password file: +DATA/CDBRAC/PASSWORD/pwdcdbrac.259.996513287 default: Domain: default: Start options: open default: Stop options: immediate default: Database role: PRIMARY default: Management policy: AUTOMATIC default: Server pools: default: Disk Groups: DATA default: Mount point paths: default: Services: default: Type: RAC default: Start concurrency: default: Stop concurrency: default: OSDBA group: dba default: OSOPER group: oper default: Database instances: cdbrac1,cdbrac2 default: Configured nodes: ol7-183-rac1,ol7-183-rac2 default: CSS critical: no default: CPU count: 0 default: Memory target: 0 default: Maximum memory: 0 default: Default network number for database services: default: Database is administrator managed default: ****************************************************************************** default: Output from srvctl status database -d cdbrac Wed Jan 2 17:14:51 UTC 2019 default: ****************************************************************************** default: Instance cdbrac1 is not running on node ol7-183-rac1 default: Instance cdbrac2 is not running on node ol7-183-rac2 default: ****************************************************************************** default: Output from v$active_instances Wed Jan 2 17:14:52 UTC 2019 default: ****************************************************************************** default: default: SQL*Plus: Release 18.0.0.0.0 - Production on Wed Jan 2 17:14:52 2019 default: Version 18.3.0.0.0 default: default: Copyright (c) 1982, 2018, Oracle. All rights reserved. default: ERROR: default: ORA-12547: TNS:lost contact default: default: default: Enter user-name: default: SP2-0306: default: Invalid option. default: Usage: CONN[ECT] [{logon|/|proxy} [AS {SYSDBA|SYSOPER|SYSASM|SYSBACKUP|SYSDG|SYSKM|SYSRAC}] [edition=value]] default: where <logon> ::= <username>[/<password>][@<connect_identifier>] default: <proxy> ::= <proxyuser>[<username>][/<password>][@<connect_identifier>] default: Enter user-name: Enter password: default: ERROR: default: ORA-12547: TNS:lost contact default: default: default: SP2-0157: default: unable to CONNECT to ORACLE after 3 attempts, exiting SQL*Plus The SSH command responded with a non-zero exit status. Vagrant assumes that this means the command failed. The output for this command should be in the log above. Please read the output to determine what went wrong.

@kuhafa
Copy link
Author

kuhafa commented Jan 2, 2019

I just saw your notes that It completed successfully, but I getting ORA-12547: TNS:lost contact error.
I spent a lot of time trying to fix it a few days ago when I got the error at the same spot.

@oraclebase
Copy link
Owner

How much RAM do you have on your Mac?

I'm wondering of there is a resource problem on you kit. Things get really slow if there is a lack of memory. The build expects 21G just for the 3 VMs, not counting some left over for the host OS, so it's really only possible if you have 32G or RAM. Host is swapping, things aren't going to go well.

I've done this with 32G on my Windows 8 Laptop. 24G on a Linux Server and I'm going to try now on a 16G MBP. I'll reduce the memory size of the VMs for that though.

Cheers

Tim...

@kuhafa
Copy link
Author

kuhafa commented Jan 2, 2019

I have 16G in total, but have allocated 3.2G for each node. Let me give 6G per node and see.

@oraclebase
Copy link
Owner

OK. That's not going to work. I think you should try:

DNS: 1024
Node1: 7168
Node2: 6144

That's 14G and leave 2 for the host. You need a little extra on node 1 as it is running the installation.

I have no idea if this will work. It's not a lot of memory for a RAC installation.

@kuhafa
Copy link
Author

kuhafa commented Jan 2, 2019

Got it. Let me give that a try.

Thanks again!

@oraclebase
Copy link
Owner

The 12.2 build went fine too.

@oraclebase
Copy link
Owner

I'm trying an 18.3 build on a MBP with 16G RAM now, using the settings I suggested to you. Fingers crossed. :)

@oraclebase
Copy link
Owner

Just completed the build on a 2014 MBP running macOS Mojave with 16G RAM. Worked fine. Actually a lot quicker than I expected.

@kuhafa
Copy link
Author

kuhafa commented Jan 2, 2019

My build just completed. I tried as you recommended, but the machine froze as soon as the Grid installation begins.
I was able to create with the following settings:

`dns:
vm_name: ol7_183_dns
mem_size: 624
cpus: 1
public_ip: 192.168.56.100

node1:
vm_name: ol7_183_rac1
mem_size: 6040
cpus: 2
public_ip: 192.168.56.101
private_ip: 192.168.1.101
u01_disk: ./ol7_183_rac1_u01.vdi

node2:
vm_name: ol7_183_rac2
mem_size: 5040
cpus: 2
public_ip: 192.168.56.102
private_ip: 192.168.1.102
u01_disk: ./ol7_183_rac2_u01.vdi`

Output
default: ****************************************************************************** default: Output from srvctl status database -d cdbrac Wed Jan 2 21:27:45 UTC 2019 default: ****************************************************************************** default: Instance cdbrac1 is running on node ol7-183-rac1 default: Instance cdbrac2 is running on node ol7-183-rac2 default: ****************************************************************************** default: Output from v$active_instances Wed Jan 2 21:27:47 UTC 2019 default: ****************************************************************************** default: default: SQL*Plus: Release 18.0.0.0.0 - Production on Wed Jan 2 21:27:47 2019 default: Version 18.3.0.0.0 default: default: Copyright (c) 1982, 2018, Oracle. All rights reserved. default: default: Connected to: default: Oracle Database 18c Enterprise Edition Release 18.0.0.0.0 - Production default: Version 18.3.0.0.0 default: SQL> default: default: INST_NAME default: -------------------------------------------------------------------------------- default: ol7-183-rac1.localdomain:cdbrac1 default: ol7-183-rac2.localdomain:cdbrac2

Thanks a lot Tim.

@oraclebase
Copy link
Owner

OK. Great. I'll close this issue.

@kuhafa
Copy link
Author

kuhafa commented Apr 30, 2020 via email

@oraclebase
Copy link
Owner

oraclebase commented Apr 30, 2020

A single directory is only for deploying a single server, so I would expect you to do something like this.

  1. You need to copy the whole directory to create a new on.
  2. Remember to remove the ".vagrant" directory from the new copy.
  3. Edit the config, making sure there aren't port clashes etc.

You can see the sort of thing I do under the dataguard directory, where I have two nodes.

If you are talking golden gate between two RAC databases, then you will need a duplicate of the whole RAC setup, allowing you to create a second RAC.

@kuhafa
Copy link
Author

kuhafa commented Apr 30, 2020 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants