Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement pg_autoctl do azure commands. #544

Merged
merged 18 commits into from
Dec 9, 2020
Merged

Implement pg_autoctl do azure commands. #544

merged 18 commits into from
Dec 9, 2020

Conversation

DimCitus
Copy link
Collaborator

@DimCitus DimCitus commented Dec 3, 2020

The following commands allow to easily deploy a testing or QA environment
for pg_auto_failover using Azure VMs, which looks more like what typical
users and customers are using than our default make cluster approach where
all the nodes are running on localhost.

In particular we can test system integration and packages based
provisioning, so that's better for end-to-end QA.

Here's the list of new commands provided:

pg_autoctl do azure
+ provision  provision azure resources for a pg_auto_failover demo
+ tmux       Run a tmux session with an Azure setup for QA/testing
+ show       show azure resources for a pg_auto_failover demo
  deploy     Deploy a pg_autoctl VMs, given by name
  create     Create an azure QA environment
  drop       Drop an azure QA environment: resource group, network, VMs
  ls         List resources in a given azure region
  ssh        Runs ssh -l ha-admin <public ip address> for a given VM name
  sync       Rsync pg_auto_failover sources on all the target region VMs

pg_autoctl do azure provision
  region  Provision an azure region: resource group, network, VMs
  nodes   Provision our pre-created VM with pg_autoctl Postgres nodes

pg_autoctl do azure tmux
  session  Create or attach a tmux session for the created Azure VMs
  kill     Kill an existing tmux session for Azure VMs

pg_autoctl do azure show
  ips    Show public and private IP addresses for selected VMs
  state  Connect to the monitor node to show the current state

And there's the new top-level commands:

make azcluster
make azdrop

Contrary to make cluster when exiting the tmux session the resources are not dropped, and it's possible to connect again to the same tmux session with either make azcluster or pg_autoctl do azure tmux session.

@DimCitus DimCitus added enhancement New feature or request user experience labels Dec 3, 2020
@DimCitus DimCitus added this to the pg_auto_failover 1.4.2 milestone Dec 3, 2020
@DimCitus DimCitus requested a review from JelteF December 3, 2020 17:47
@DimCitus DimCitus self-assigned this Dec 3, 2020
@DimCitus DimCitus added Size:L Effort Estimate: Large Size: XL Effort Estimate: eXtra Large and removed Size:L Effort Estimate: Large labels Dec 7, 2020
@JelteF JelteF added the Developer productivity Enhancements to ability to ship quality code label Dec 9, 2020
src/bin/pg_autoctl/azure.c Outdated Show resolved Hide resolved
Copy link
Contributor

@JelteF JelteF left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I skimmed the code in this PR and it looked reasonable. Since this is essentially just dev helpers I think it's fine to merge this without me doing a very thorough review. I ran make azcluster and make azdrop. They worked, but there's some pg_hba lines that we probably want to add. I think it's fine to do that in a later PR though.

The following commands allow to easily deploy a testing or QA environment
for pg_auto_failover using Azure VMs, which looks more like what typical
users and customers are using than our default `make cluster` approach where
all the nodes are running on localhost.

In particular we can test system integration and packages based
provisioning, so that's better for end-to-end QA.

Here's the list of new commands provided:

    pg_autoctl do azure
    + create  create azure resources for a pg_auto_failover demo
    + show    show azure resources for a pg_auto_failover demo
      ls      List resources in a given azure region
      ssh     Runs ssh -l ha-admin <public ip address> for a given VM name
      sync    Rsync pg_auto_failover sources on all the target region VMs

    pg_autoctl do azure create
      region  Create an azure region: resource group, network, VMs
      nodes   Create and provision our VM nodes in an azure region

    pg_autoctl do azure show
      ips    Show public and private IP addresses for selected VMs
      state  Connect to the monitor node to show the current state
Store the --prefix, --region, --location, and derivative information in a
configuration file in XDG_CONFIG/$(pwd)/azure.cfg so that we can avoid
passing options to most commands.

    $ pg_autoctl do azure create region --prefix ha-demo-dim --region paris --location francecentral --monitor
	$ pg_autoctl do azure ls
	$ pg_autoctl do azure show ips
	$ pg_autoctl do azure create nodes
Review commands so that the main entry point for azure QA setup creates all
the VMs, provision them with our selected software, and then create a tmux
session with interactive control over the setup. The tmux session is primed
with commands that deploy pg_autoctl on each node, including integration in
the systemd setup.
We use the 3 following environment variables to control the version of
parget packages installed on Azure:

  - AZ_PG_VERSION defaults to "13"
  - AZ_PGAF_VERSION default to "1.4.1"
  - AZ_PGAF_DEB_VERSION default to "1.4"
It turns out we need the debian revision to be able to select a target
version with precision in debian.

- *   sudo apt-get install -q -y postgresql-13-auto-failover-1.4=1.4.1
+ *   sudo apt-get install -q -y postgresql-13-auto-failover-1.4=1.4.1-1
…sion.

That's needed because tmux sessions are where we call

  pg_autoctl do azure deploy
@DimCitus DimCitus merged commit de02d36 into master Dec 9, 2020
@DimCitus DimCitus deleted the feature/do-azure branch December 9, 2020 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Developer productivity Enhancements to ability to ship quality code enhancement New feature or request Size: XL Effort Estimate: eXtra Large user experience
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants