New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add role for remote DWH installation #9

Closed
pkubica opened this Issue Jun 20, 2016 · 7 comments

Comments

4 participants
@pkubica
Copy link
Collaborator

pkubica commented Jun 20, 2016

  • propably need different answerfile

@pkubica pkubica self-assigned this Jun 21, 2016

@pkubica pkubica added the enhancement label Jun 22, 2016

@lukas-bednar lukas-bednar added this to the not-scheduled milestone Aug 17, 2016

@StLuke

This comment has been minimized.

Copy link
Collaborator

StLuke commented Feb 10, 2017

This idea is proposed as GSOC please contact me on resolution

@lukas-bednar lukas-bednar added this to TODO in all-in-one Mar 6, 2017

@lukas-bednar lukas-bednar removed this from TODO in all-in-one May 11, 2017

@lukas-bednar lukas-bednar added this to TODO in GSOC May 11, 2017

@StLuke StLuke changed the title Add role for remote DWH installation (ver 3.x and 4.x) Add role for remote DWH installation May 11, 2017

@tasdikrahman

This comment has been minimized.

Copy link
Contributor

tasdikrahman commented Jun 18, 2017

Hey there,

Was wondering how to get started working on this issue. Thanks 😄

cc: @StLuke @pkubica @lukas-bednar

@StLuke

This comment has been minimized.

Copy link
Collaborator

StLuke commented Jun 19, 2017

Hi,

sorry for the late response.
I would consider here 2 scenarios.

  1. Playbook for installation with remote dwh
    this consist of 2 different flows:
    a) Engine with remote dwh and database of dwh on dwh machine
    b) Engine with remote dwh and database of dwh on 3rd machine - this would be the same as engine instalaltion with remote database for the dwh part

For these I would preferably extend engine-setup playbook to have remote-dwh flag set to False by default. If set to true it will require IP of new dwh machine where dwh will be stored. In both cases, local and remote dwh, we should consider that dwh itself can have remote and local database, thus this can follow the flow of engine-setup with remote database of engine.

  1. Migration dwh from local machine of engine to remote machine. [1]
    This new playbook counts with already installed engine with local dwh and moved dwh to this remote server.
    [1] http://www.ovirt.org/develop/release-management/features/engine/migration-of-local-dwh-reports-to-remote/ (Article is outdated, reports does not need to be considered)

For testing of these rolls we can use new containers in travis.

tasdikrahman added a commit to tasdikrahman/ovirt-ansible that referenced this issue Jun 25, 2017

[issue9] modified engine-setup role for remote-dwh
working on rhevm-qe-automation#9
done
====
- modifications done to just install ovirt-engine 4.1 on a fresh centos7
  box

TODO
====
- to add role for integrating remote dwh with a remote engine

tasdikrahman added a commit to tasdikrahman/ovirt-ansible that referenced this issue Jun 25, 2017

[issue9] modified engine-setup role for remote-dwh
working on rhevm-qe-automation#9
done
====
- modifications done to just install ovirt-engine 4.1 on a fresh centos7
  box

TODO
====
- to add role for integrating remote dwh with a remote engine

@StLuke StLuke moved this from TODO to In Progress in GSOC Jun 26, 2017

tasdikrahman added a commit to tasdikrahman/ovirt-ansible that referenced this issue Jun 28, 2017

[issue9] remote dwh playbook for dwh on dwh added
Done
====
- Playbook for installing remote DWH when the database of dwh is on dwh machine
- modified the original engine-setup role(mainly the answerfile for 4.1) to accomodate the changes
  required when we are not installing the dwh on the same host
- modified the original engine-install-packages role to accomodate
  changes for remote dwh package installation on remote dwh host

TODO
====
- cover the case when The database of dwh on 3rd machine

Link to issue
=============
- rhevm-qe-automation#9

tasdikrahman added a commit to tasdikrahman/ovirt-ansible that referenced this issue Jun 28, 2017

[issue9] remote dwh playbook for dwh on dwh added
Done
====
- Playbook for installing remote DWH when the database of dwh is on dwh machine
- modified the original engine-setup role(mainly the answerfile for 4.1) to accomodate the changes
  required when we are not installing the dwh on the same host
- modified the original engine-install-packages role to accomodate
  changes for remote dwh package installation on remote dwh host

TODO
====
- cover the case when The database of dwh on 3rd machine

Link to issue
=============
- rhevm-qe-automation#9

tasdikrahman added a commit to tasdikrahman/ovirt-ansible that referenced this issue Jun 28, 2017

[issue9] remote dwh playbook for dwh on dwh added
Done
====
- Playbook for installing remote DWH when the database of dwh is on dwh machine
- modified the original engine-setup role(mainly the answerfile for 4.1) to accomodate the changes
  required when we are not installing the dwh on the same host
- modified the original engine-install-packages role to accomodate
  changes for remote dwh package installation on remote dwh host

TODO
====
- cover the case when The database of dwh on 3rd machine

Link to issue
=============
- rhevm-qe-automation#9
@tasdikrahman

This comment has been minimized.

Copy link
Contributor

tasdikrahman commented Jul 14, 2017

@StLuke

For the 2nd case where we have three machines, one for the engine, the other for holding the dwh-db and the 3rd for holding the dwh-service

I was having a look at this http://www.ovirt.org/documentation/data-warehouse/Migrating_Data_Warehouse_to_a_Separate_Machine/ .

I am trying to setup a 3 VM setup where one would have ovirt-engine, one with the dwh service and the last one having dwh db in the guide above, looks like its targeted for migrating an exisiting ovirt_engine_history

what if one wanted to set one from scratch, how should I go about it?

I am thinking the flow to be

  1. setup the engine without the dwh and then create the ovirt_engine_history on the 2nd machine (if anyone can point out how?)

  2. Configure the dwhd service to look for these two remote machines on the 3rd VM

Had posted it on the IRC but couldn't get a response

@StLuke

This comment has been minimized.

Copy link
Collaborator

StLuke commented Jul 16, 2017

what if one wanted to set one from scratch, how should I go about it?

I would suggest with fresh installation of engine and dwh together, link you posted is about migration of pre-existing dwh service/db.

  1. setup the engine without the dwh and then create the ovirt_engine_history on the 2nd machine (if anyone can point out how?)

a) On installation of engine on machine1 choose to have dwh on remote machine.
b) On machine where you want dwh service to be installed install package ovirt-engine-dwh-setup, which will install necessary requirements for DWH service. Afterwards run engine-setup and continue with dwh steps to connect this service to preexisting engine on machine1. During this dwh service installation you are able to choose to have databse of dwh service located on the remote service, thus connected to 3rd machine.

This will set up databse in a way:
engine machine <- dwh service machine <- dwh service database machine

  1. Configure the dwhd service to look for these two remote machines on the 3rd VM
    Not sure which two you mean here, I hope they are mentioned in the previous steps.

All these steps are connected to scenario 1. Playbook for installation with remote dwh a) and b). Link you posted for migration is for scenario 2. Migration dwh from local machine of engine to remote machine.
For this you can either use steps on ovirt mages you mentioned, or some steps are available here: https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.1/html-single/data_warehouse_guide/#Migrating_the_Data_Warehouse_Database_to_a_Separate_Machine

@tasdikrahman

This comment has been minimized.

Copy link
Contributor

tasdikrahman commented Jul 27, 2017

@StLuke I was able to set up a 3 box setup where

  • one contains the engine manager and the engine db
  • 2nd contains the dwh service
  • 3rd contains the dwh database holding the ovirt_engine_history db on it

Was stuck on some remote connections issues which made it a hard time for me to debug as they were not showing up in any logs but just giving some arcane connection errors (auth was being successful).

Writing the playbooks for the automation part of this. Should I add it to the same PR/playbook or make a new one?

Thanks

lukas-bednar added a commit that referenced this issue Aug 28, 2017

Remote dwh fresh engine install (#137)
* [issue9] modified engine-setup role for remote-dwh

working on #9
done
====
- modifications done to just install ovirt-engine 4.1 on a fresh centos7
  box

TODO
====
- to add role for integrating remote dwh with a remote engine

* removed the extra variable as a flag for remotedwh

* [issue9] remote dwh playbook for dwh on dwh added

Done
====
- Playbook for installing remote DWH when the database of dwh is on dwh machine
- modified the original engine-setup role(mainly the answerfile for 4.1) to accomodate the changes
  required when we are not installing the dwh on the same host
- modified the original engine-install-packages role to accomodate
  changes for remote dwh package installation on remote dwh host

TODO
====
- cover the case when The database of dwh on 3rd machine

Link to issue
=============
- #9

* [tests] skip firewalld command inside travis build

* [remote-dwh] role added for postgres installation

Done
====
- Commit for 3vm setup
    - engine (VM1)
    - dwhdb (VM2)

TODO
====
- modify existing roles/ovirt-engine-remote-dwh-setup/ to accomodate the
  3 vm installation config

* [3vmsetup] Added dwhservice playbook

DONE
====
- Completed the feature of adding the remaining role for remote dwh
  where there is a 3 VM setup.
- Added docs documenting the process

TODO
====
- work on upgrade tests playbook

* [dwhremote] fix linting errors

* [refactor] renamed `roles/ovirt-engine-remote-dwh-setup` -> `roles/ovirt-engine-remote-dwh`

* [refactor] cleaned up firewalld configuration changes in the playbook and the handler associated with it

* removed firewalld command to open port in `ovirt-engine-setup` playbook

* added link to root README for ovirt-engine-remote-dwh

* making `engine_vm_network_cidr` and `dwhservice_vm_network_cidr` mandatory

* changed `ovirt_engine_host_root_passwd` to password as default

* changed README to reflect `engine_vm_network_cidr` and `dwhservice_vm_network_cidr` generic IP's

* changed retries from 2 -> 12
@tasdikrahman

This comment has been minimized.

Copy link
Contributor

tasdikrahman commented Aug 28, 2017

Was closed by 526f0db

@StLuke StLuke moved this from In Progress to Done in GSOC Sep 8, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment