Skip to content

Install OCI360

Jump to bottom
Rodrigo Jorge edited this page Dec 1, 2020 · 13 revisions

Introduction

This page will give you a walk-through of how to Deploy the OCI360 using Docker on a Compute Instance in your tenancy that will create a new report every X minutes and make it available via web server. This is the easiest and the recommended way to go.

If for some reason you don't want to use docker and want to go through all the manual steps, please refer to:

Index:

Pre-requisites

Deploy a new server with the following specifications in OCI:

  • Oracle Linux 7.X or Oracle Linux 8.X
  • Minimal Shape. VM.Standard.E2.1 - 1 Core OCPU, 8 GB Memory
  • Root Disk = 50GB
  • Attached Block Volume Disk = 100GB
    • This space will be used by OCI360 output and Oracle Database 18c XE.
  • Public IP is not recommended

For security reasons, it's recommended to:

  • KEEP SELinux running in enforcing mode. No need to stop it.
  • KEEP iptables/firewalld service running. No need to stop it.

During installation process, the target compute instance will need temporarily egress rule on:

  • CIDR: 0.0.0.0/0 Port: 443 (to download docker images, oracle database rpm, etc)

After installation is completed, you can change the egress rule just to allow Oracle Services, required by the oci-cli communication.

Installation Steps

1. Mount the Attached Block Volume Disk in the new compute

First, ensure your disk is detect by OS. If not, run ISCSIADM commands.

You can check it with lsblk command:

[ROOT]# lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sdb      8:16   0   50G  0 disk
sda      8:0    0 46.6G  0 disk
├─sda2   8:2    0    8G  0 part [SWAP]
├─sda3   8:3    0 38.4G  0 part /
└─sda1   8:1    0  200M  0 part /boot/efi

As you can see above, a new disk was detected.

Now let's create a lvm volume to manage and format it.

[ROOT]# pvcreate /dev/sdb
  Physical volume "/dev/sdb" successfully created.
[ROOT]# vgcreate vg_oci360 /dev/sdb
  Volume group "vg_oci360" successfully created
[ROOT]# lvcreate -n lv_oci360 -l 100%FREE vg_oci360
  Logical volume "lv_oci360" created.
[ROOT]# mkfs.xfs /dev/vg_oci360/lv_oci360
meta-data=/dev/vg_oci360/lv_oci360  isize=256    agcount=4, agsize=3276544 blks
         =                          sectsz=4096  attr=2, projid32bit=1
         =                          crc=0        finobt=0, sparse=0, rmapbt=0, reflink=0
data     =                          bsize=4096   blocks=13106176, imaxpct=25
         =                          sunit=0      swidth=0 blks
naming   =version 2                 bsize=4096   ascii-ci=0 ftype=1
log      =internal log              bsize=4096   blocks=6399, version=2
         =                          sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                      extsz=4096   blocks=0, rtextents=0

Time to mount the disk:

[ROOT]# mkdir /u01
[ROOT]# mount /dev/vg_oci360/lv_oci360 /u01

Get the UUID and add the volume to fstab for auto-mount on boot.

[ROOT]# blkid /dev/vg_oci360/lv_oci360
/dev/vg_oci360/lv_oci360: UUID="4b3ef26a-1480-4052-942b-66100cbb4eb1" TYPE="xfs"

Now add to fstab:

[ROOT]# echo "UUID=$(blkid -s UUID -o value /dev/vg_oci360/lv_oci360) /u01 xfs defaults,_netdev,_netdev 0 0" >> /etc/fstab

Umount and mount to test:

[ROOT]# umount /u01
[ROOT]# mount /u01

2. Run the OCI360 installation script

Connect as ROOT and run:

[ROOT]# bash -c "$(curl -L https://raw.githubusercontent.com/dbarj/oci360/master/container/setup_docker.sh)"

This step will take about 30-40 minutes. It will download docker images for ol-slim, get Oracle 18cXE edition RPM, setup the database, config the OCI360 tool and finally deploy another container for the apache https access. You can monitor the steps opening another session and checking the OS processes.

Optional (change base directory):

By default, the OCI360 base directory will be "/u01". If you want to use a different folder, run the following export command before calling de setup script above:

[ROOT]# export OCI360_ROOT_DIR='/u02'

3. Setup oci-cli

Now it's time to setup your oci-cli utility so the export phase of oci360 can run without any issues. To do that, you have 2 options:

  • Make oci-cli authenticate to the API using instance_principal method (if this host in inside the OCI tenancy that you are reading from)
  • Create a read-only user in your OCI web console tenancy and authenticate through API Key.

3.1 Instance_principal method (Recommended)

In instance_principal method, the oci-cli connect to the OCI internally and thus don't need to connect though the internet. Only to the Service Gateway.

Modify the /u01/.oci/config file, specifying into it just your tenancy OCID and the default region:

[ROOT]$ cat /u01/.oci/config
[DEFAULT]
tenancy=ocid1.tenancy.oc1..xxx
region=us-ashburn-1
[ROOT]$ vi /u01/.oci/config

That's all you need to set in your compute side. Now you need to tell your tenancy that this compute can make REST API calls using the internal API.

Now, connect on OCI web-console with a admin account. Open the Cloud Shell and run:

[ROOT]# bash -c "$(curl -L https://raw.githubusercontent.com/dbarj/oci360/master/container/oci360_oci_cli_access.sh)"

This script will automatically create the following rules for you:

  1. Dynamic Group, named OCI360_DG with the simple rule include your own machine OCID:
    1. Name: OCI360_DG
    2. Description: Group to handle oci-cli calls from the host of OCI360.
    3. Rule:
      • instance.id = 'ocid1.instance.oc1.iad.xxx' (Your Compute OCID)
  2. Policy, named OCI360_Policy with the simple rule include your own machine OCID:
    1. Name: OCI360_Policy
    2. Description: Policy to handle oci-cli calls from the host of OCI360.
    3. Rule:
      • allow dynamic-group OCI360_DG to read all-resources in tenancy

More info at: https://docs.cloud.oracle.com/iaas/Content/Identity/Tasks/managingdynamicgroups.htm

Now test if oci-cli is working:

[OCI360]$ docker exec -it --user oci360 oci360-tool bash -c 'export OCI_CLI_AUTH=instance_principal; cd /tmp/; /u01/oci360_tool/app/sh/oci_json_export.sh Comp-Instances'

Expected output:

{
  "data": [
...

3.2 API Key method

Skip this section if you configure the oci-cli using section 3.1 (instance principal method).

For this option, check this article that will show how to create a readonly user for your tenancy: https://www.dbarj.com.br/en/2018/09/creating-a-read-only-user-in-oracle-cloud-infrastructure/

After you configure the oci-cli properly with this account in configuration file "/u01/.oci/config", remove the entry "export OCI_CLI_AUTH=instance_principal" from "/u01/oci360_tool/scripts/oci360.cfg" file.

Now test if oci-cli is working:

[OCI360]$ docker exec -it --user oci360 oci360-tool bash -c 'cd /tmp/; /u01/oci360_tool/app/sh/oci_json_export.sh Comp-Instances'

Expected output:

{
  "data": [
...

4. Run OCI360 manually and check logs

Let's do a trial run. We will simple run the crontab line in nohup mode:

[OCI360]$ docker exec -it --user oci360 oci360-tool bash /u01/oci360_tool/scripts/oci360_run.sh
# In another session:
[OCI360]$ tail -f /u01/oci360_tool/log/run.*.log

After it completes, now you can test the output accessing your server at: https://YOURSERVER/oci360/.

Note you will need to add an ingress rule to this compute instance on port 443 in order to access the OCI360 web report.

The output will also be in a zip format and available at /u01/oci360_tool/out/processed/

5. Enable oci360 auto-execution

Before running it for the first time, create an auto-execution entry in you crontab. You may adjust how often it will execute (/6 = every 6 hours. /12 = every 12 hours):

[OCI360]$ crontab -l > mycron
no crontab for oci360
[OCI360]$ echo '00 */6 * * * docker exec --user oci360 oci360-tool bash /u01/oci360_tool/scripts/oci360_run.sh' >> mycron
[OCI360]$ crontab mycron
[OCI360]$ rm -f mycron

Checking..

[OCI360]$ crontab -l
00 */6 * * * docker exec --user oci360 oci360-tool bash /u01/oci360_tool/scripts/oci360_run.sh

Optional Steps

6. Add usage info in OCI360:

To add usage info in OCI360, the created account used by your oci-cli must also be granted privileges to access tenancy usage info. For more details about giving that privilege, check: https://docs.cloud.oracle.com/en-us/iaas/Content/Billing/Tasks/accessingusagereports.htm and https://docs.cloud.oracle.com/en-us/iaas/Content/GSG/Concepts/costs.htm.

7. Add billing info in OCI360:

OCI360 has also a full billing report. In order to enable it, you need some extra steps as billing info is taken with direct REST calls to your tenancy IDCS domain, not using oci-cli.

The most secure way to do it is to create a Trusted Application in the IDCS. Check this article to get the steps to create this account: https://www.dbarj.com.br/en/2019/06/creating-read-only-account-for-rest-billing-access-on-oracle-cloud/.

After application is created, create the oci360.cfg file in the scripts folder with the variables below. Fill them with the corresponding ones for your tenancy (copy and paste all the lines from cat until EOF):

[OCI360]$ cat << 'EOF' >> /u01/oci360_tool/scripts/oci360.cfg
export CLIENT_ID="c18e050ae7c32a4994c26252b8adf703"
export CLIENT_SECRET="e7c32a49-219b-4fda-a404-252b8adf70302"
export CLIENT_DOMAIN="idcs-50ae7c32a4994c2625250ae7c32a4994"
EOF
[OCI360]$ chmod 600 /u01/oci360_tool/scripts/oci360.cfg

To test if it's working:

[OCI360]$ docker exec -it --user oci360 oci360-tool bash -c 'cd /tmp/; source /u01/oci360_tool/scripts/oci360.cfg; /u01/oci360_tool/app/sh/oci_json_billing.sh serviceEntitlements'

Overview

Deploying OCI360 application

Database

Running OCI360 once

OCI360 Manual Deployment

(Only use this if you want to do everything manually.)

Optional OCI360 add-ons

Misc

Clone this wiki locally