Add integration tests for windows

[clintoncwolfe]

Description

The integration tests - which use Test Kitchen to spin up actual VMs, then tests train's ability to connect to them - does not test any Windows platform.

Train and Platform Version

1.7.x

Replication Case

rake test:integration

Possible Solutions

Add tests for winrm and possibly windows-via-ssh.

[clintoncwolfe]

refs #426

[StefanScherer]

Here is a Vagrantfile that I used with VMware Fusion to test the Microsoft OpenSSH service with InSpec.

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure("2") do |config|
  config.vm.box = "StefanScherer/windows_10"

  config.vm.provider "vmware_desktop" do |v|
    # Display the VirtualBox GUI when booting the machine
    v.gui = true
  
    # Customize the amount of memory on the VM:
    v.memory = "2048"
  end
  #
  # View the documentation for the provider you are using for more
  # information on available options.

  # Enable provisioning with a shell script. Additional provisioners such as
  # Puppet, Chef, Ansible, Salt, and Docker are also available. Please see the
  # documentation for more information about their specific syntax and use.
  config.vm.provision "shell", inline: <<-SHELL
    $ErrorActionPreference = 'Stop'

    [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12

    Write-Output "Downloading OpenSSH"
    Invoke-WebRequest "https://github.com/PowerShell/Win32-OpenSSH/releases/download/v7.9.0.0p1-Beta/OpenSSH-Win64.zip" -OutFile OpenSSH-Win64.zip -UseBasicParsing

    Write-Output "Expanding OpenSSH"
    Expand-Archive OpenSSH-Win64.zip C:\\
    Remove-Item -Force OpenSSH-Win64.zip

    Write-Output "Disabling password authentication"
    # Add-Content C:\\OpenSSH-Win64\\sshd_config "`nPasswordAuthentication no"
    Add-Content C:\\OpenSSH-Win64\\sshd_config "`nUseDNS no"

    Push-Location C:\\OpenSSH-Win64

    Write-Output "Installing OpenSSH"
    & .\\install-sshd.ps1

    Write-Output "Generating host keys"
    .\\ssh-keygen.exe -A

    Write-Output "Fixing host file permissions"
    & .\\FixHostFilePermissions.ps1 -Confirm:$false

    Write-Output "Fixing user file permissions"
    & .\\FixUserFilePermissions.ps1 -Confirm:$false

    Pop-Location

    $newPath = 'C:\\OpenSSH-Win64;' + [Environment]::GetEnvironmentVariable("PATH", [EnvironmentVariableTarget]::Machine)
    [Environment]::SetEnvironmentVariable("PATH", $newPath, [EnvironmentVariableTarget]::Machine)

    Write-Output "Adding public key to authorized_keys"
    $keyPath = "~\\.ssh\\authorized_keys"
    New-Item -Type Directory ~\\.ssh > $null
    $sshKey | Out-File $keyPath -Encoding Ascii

    Write-Output "Opening firewall port 22"
    New-NetFirewallRule -Protocol TCP -LocalPort 22 -Direction Inbound -Action Allow -DisplayName SSH

    Write-Output "Setting sshd service startup type to 'Automatic'"
    Set-Service sshd -StartupType Automatic
    Set-Service ssh-agent -StartupType Automatic
    Write-Output "Setting sshd service restart behavior"
    sc.exe failure sshd reset= 86400 actions= restart/500
    Start-Service sshd
  SHELL
end

I don't have VirtualBox running, but with a small adjustment this should be work as well.

I figured out the IP address of the VMware VM and then ran

inspec exec my_win10 --backend ssh --password vagrant --host 172.16.208.155 --user vagrant -l debug

from my Mac.

[StefanScherer]

Another integration test would be to run InSpec against a Windows container.

It doesn't work right now as

train/lib/train/transports/docker.rb

Line 96 in e3e4096

'/bin/sh', '-c', cmd

has a hard coded

'/bin/sh', '-c', cmd

that should something like

'cmd', '/c', cmd

if the running container is a Windows container:

$ docker ps
CONTAINER ID        IMAGE               COMMAND                    CREATED             STATUS              PORTS               NAMES
fb610d89bad6        install             "c:\\windows\\system32…"   15 minutes ago      Up 15 minutes                           keen_gagarin
~/code/eli
$ docker inspect fb610d89bad6 | jq .[].Platform
"windows"
~/code/eli
$ inspec exec test/install -t docker://fb610d89bad6 -l debug
[2019-03-25T22:39:29+01:00] DEBUG: Option backend_cache is enabled
[2019-03-25T22:39:29+01:00] DEBUG: Resolve test/install into cache /Users/stefanscherer/.inspec/cache
[2019-03-25T22:39:29+01:00] DEBUG: Dependency does not exist in the cache test/install
[2019-03-25T22:39:29+01:00] DEBUG: Starting run with targets: ["Inspec::Profile<install>"]
[2019-03-25T22:39:29+01:00] DEBUG: Using existing lockfile test/install/inspec.lock
[2019-03-25T22:39:29+01:00] DEBUG: Loading test/install/controls/example.rb into #<Inspec::ProfileContext:0x00007fc44418cad0>
[2019-03-25T22:39:29+01:00] DEBUG: Registering rule (generated from example.rb:7 34bc935165b237f1ed289db0d47f4c84)

Profile: InSpec Profile (install)
Version: 0.1.0
Target:  docker://fb610d89bad6746ca66a8cf3ad26d20b543b95f3f0f5f83cbc76809fa64092af

  File C:/install.ps1
     ×  should be file
     expected `File C:/install.ps1.file?` to return true, got false

Test Summary: 0 successful, 1 failure, 0 skipped

But this is another story/issue/pull request ;-) And it's not that simple, my test still is red although the file exists in the container. Just another idea what we can do with InSpec on Windows.

[StefanScherer]

Just an idea as you are using AppVeyor for Windows tests.
We could create two containers:

1. one with the Microsoft OpenSSH service, let's say running on port 2222
2. another with the initial port of OpenSSH for Windows, eg. running on port 2223

and run some tests against these containers with an SSH connection to localhost:2222 and localhost:2223. Just as normal SSH connection tests and not with the train/lib/train/transports/docker.rb I mentioned above.

[StefanScherer]

Just created a Windows Docker image stefanscherer/openssh-windows that runs the Microsoft OpenSSH as a real Windows service to have the same environment as in a VM. This image comes with a default user User03 and Passw0rd to test logins.

You should be able to run tests in AppVeyor with it. The Docker image has a manifest list for both Windows Server 2016 and Windows Server 2019.

docker run -d -p 2222:22 stefanscherer/openssh-windows

Then try to SSH into localhost:2222