Linux Integration Services Automation (LISA), version 2

June 2020

Overview

LISAv2 is the one-stop automation solution implemented by PowerShell scripts, Linux BASH scripts and Python scripts for verifying Linux image/kernel on below platforms:

Microsoft Azure
Microsoft Azure Stack
Microsoft Hyper-V
WSL
Ready

LISAv2 includes below test suite categories:

Functional tests
Performance tests
Stress tests
Test suites developed by Open Source communities

Prerequisite

You must have a Windows Machine (Host) with PowerShell (v5.0 and above but not 6.x) as test driver. It should be Windows Server for localhost, or any Windows system including Windows 10 for remote host access case. PowerShell 6.x shows run-time error due to missing nugget.
You must be connected to Internet.
You download 3rd party software in Tools folder. If you are using secure blob in Azure Storage Account or UNC path, you can add a tag https://myownsecretlocation.blob.core.windows.net/binarytools in the secret xml file.

7za.exe
dos2unix.exe
gawk
go1.11.4.linux-amd64.tar.gz
golang_benchmark.tar.gz
http_load-12mar2006.tar.gz
jq
plink.exe
pscp.exe
kvp_client32
kvp_client64
nc.exe
nc64.exe

For running Azure tests, you must have a valid Windows Azure Subscription, if you want to enable ssh key authentication on Azure platform, please refer here for generating SSH key pairs.
For running Hyper-V tests, the resource requirements are:

Hyper-V role enabled
At least 8 GB of memory on the Host - Most of LISAv2 tests will create and start Virtual Machines (Guests) with 3.5 GB of memory assigned
1 External vSwitch in Hyper-V Manager/Virtual Switch Manager. This vSwitch will be named 'External' and must have an internet connection. For Hyper-V NETWORK tests you need 2 more vSwitch types created: Internal and Private. These 2 vSwitches will have the naming also 'Internal' and 'Private'.

For running WSL tests, you must enable WSL on the test server

a. Open Powershell as Administrator and run:
```
Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux
```
b. Restart your computer when prompted.

Download Latest Azure PowerShell

Download Web Platform Installer from here
Start Web Platform Installer and select Azure PowerShell (required 6.3.0 or above) and proceed for Azure PowerShell Installation. (Azure PowerShell version is different from PowerShell version)
Install the Azure Powershell Az module here

Authenticate from LISAv2 Orchestrator Machine with Your Azure Subscription

Sign in with Azure PowerShell

Refer to this URL here

Prepare Service Principal and create secret file

Refer to this URL here

Prepare VHD for Your Test

Applicable if you are uploading your own Linux VHD to Azure for test.

A VHD with Linux OS must be made compatible to work in Hyper-V environment. This includes:

Linux Integration Services. If not available, at least KVP daemon must be running. Without KVP daemon running, the framework will not be able to obtain an IP address from the guest.
Windows Azure Linux Agent (for testing in Azure environment only)
SSH, and the SSH daemon configured to start on boot.
Port 22 open in the firewall
A regular user account on the guest OS

"root" user cannot be used in the test configuration

Please follow the steps mentioned at here

Launch Test Suite

Clone this automation code to your test system by:

   git clone https://github.com/LIS/LISAv2.git

Create a new secret xml file, and update required fields from .\XML\GlobalConfigurations.xml:

2.1 Secret File to be used by -XMLSecretFile <Path of Secret File> when Run-LISAv2.ps1

If Run LISAv2 for Azure Platform with authentication from service principal, please update created service principal info in your secrete file, can use this script to create service principal.
If Run-LISAv2.ps1 for Azure Platform from an authenticated PowerShell session (check Get-AzContext from PowerShell), no need prepare Service Principla info in your secrete file.
Storage Accounts Tips:
- If you want LISAv2 to create all potential needed Azure storage accouts from all available Azure regions that supports 'Microsoft.Storage' resource type and keep using those storage accounts in the following automation or ad-hoc testing, give <ARMStorageAccount>Auto_Complet_RG=Xxx</ARMStorageAccount> from .\XML\GlobalConfigurations.xml, or you can specify -StorageAccount "Auto_Complet_RG=Xxx" during Run-LISAv2.ps1.
  - 'Xxx' is the Resource Group Name to host storage accounts which are used for LISAv2 execution. If Azure resource group 'Xxx' does not exist, LISAv2 will create 'Xxx' resource group automatically.
  - LISAv2 will create new storage accounts automatically with name following regular expression lisa[a-z0-9]{15}, and make sure there are two storage accounts (one is Standard_LRS type, another is Premium_LRS type) created (or checked for existence) in Azure resource group 'Xxx' for each Azure region which supports 'Microsoft.Storage' resource type. Any existing storage account that follows the naming format lisa[a-z0-9]{15} from Azure resource group 'Xxx' will be taken as expected storage accounts and LISAv2 will not create duplicate storage accounts with same storage account type (Standard_LRS or Premium_LRS) at the same region.
  - LISAv2 will update <RegionAndStorageAccounts> of user specified secret file with expected storage accounts from Azure resource group 'Xxx' for the following execution, just as user created those storage accounts manually and replaced in secret file as below example.
  - If user changes to another Azure resource group 'Yyy' instead of 'Xxx' used previously when Run-LISAv2 with the same subscription, LISAv2 will create (or check existence of) another set of storage accounts with naming lisa[a-z0-9]{15} in Azure resource group 'Yyy'. So note about this, and think over before changing to another resource group in the following automation or ad-hoc testing.
- If you already prepared standard and premium storage accounts in your test subscription for your test location, e.g., 'eastasia', please replace storage account names in secret file as below. If you may run against other regions, add more tag elements like <other_region></other_region>.

    <secrets>
        <!--Not mandatory-->
        <SubscriptionName>Enter your subscription name</SubscriptionName>
        <!--'SubscriptionID' is manadatory if 'SubscriptionID' is empty/invalid from .\XML\GlobalConfigurations.xml, otherwise, it's optional-->
        <SubscriptionID>Enter your subscription id</SubscriptionID>
        <!--Below three elements are mandatory if testing against Azure platform with service principal, otherwise they are optional (e.g., Signing in with Azure PowerShell before Run-LISAv2.ps1)-->
        <SubscriptionServicePrincipalTenantID>Enter a new Tenant id from CreateServicePrincipal.ps1 result</SubscriptionServicePrincipalTenantID>
        <SubscriptionServicePrincipalClientID>Enter a new Client id from CreateServicePrincipal.ps1 result</SubscriptionServicePrincipalClientID>
        <SubscriptionServicePrincipalKey>Enter a new Principal key from CreateServicePrincipal.ps1 result<SubscriptionServicePrincipalKey>
        <!--Download needed tools from the blob-->
        <blobStorageLocation>Enter your blob storage location if needed</blobStorageLocation>
        <!--VMs Credential-->
        <linuxTestUsername>Enter Linux VM user name</linuxTestUsername>
        <linuxTestPassword>Enter Linux VM user password with high complexity</linuxTestPassword>
        <sshPrivateKey>Downloadable URL or local file - ssh ppk private key</sshPrivateKey>
        <!--Database info for upload results-->
        <DatabaseServer></DatabaseServer>
        <DatabaseUser></DatabaseUser>
        <DatabasePassword></DatabasePassword>
        <DatabaseName></DatabaseName>
        <!--Below <RegionAndStorageAccounts> is optional when <ARMStorageAccount> is set with 'Auto_Complete_RG=Xxx' from .\XML\GlobalConfigurations.xml-->
        <!--Below <RegionAndStorageAccounts> is optional when run LISAv2 in Windows PowerShell with parameter '-StorageAccount "Auto_Complete_RG=Xxx"'-->
        <!-- 'Xxx' is the Resource Group Name to host all the storage accounts, if 'Xxx' is not exist, LISAv2 will create resource group automatically and will create storage accounts in that resource group for all available regions-->
        <RegionAndStorageAccounts>
            <eastasia>
                <StandardStorage>Enter Standard Storage Account name</StandardStorage>
                <PremiumStorage>Enter Premium Storage Account name</PremiumStorage>
            </eastasia>
            <westus>
                <StandardStorage>Enter Standard Storage Account name</StandardStorage>
                <PremiumStorage>Enter Premium Storage Account name</PremiumStorage>
            </westus>
            <!--Other locations sections-->
        </RegionAndStorageAccounts>
    </secrets>

2.2 Update .\XML\GlobalConfigurations.xml file is optional, depending on your authentication options and how you have prepared the secrets file. But in some cases, you may need to update .\XML\GlobalConfigurations.xml with your Azure subscription information or Hyper-V host information:

 Go to Global > Azure/HyperV and update following fields if necessary:

     a. SubscriptionID
     b. SubscriptionName (Optional)
     c. Environment (For Azure PublicCloud, use `AzureCloud`)
     d. ARMStorageAccount
     e. DefaultARMImageName

 Example:

    <Azure>
        <Subscription>
            <!--This 'SubscriptionID' is mandatory if 'SubscriptionID' is empty/invalid from the secret xml file, which is indicated by '-XMLSecretFile' when Run-LISAv2.ps1, otherwise, it's optional -->
            <SubscriptionID>2cd20493-0000-1111-2222-0123456789ab</SubscriptionID>
            <SubscriptionName>YOUR_SUBSCRIPTION_NAME</SubscriptionName>
            <Environment>AzureCloud</Environment>
            <!--This 'ARMStorageAccount' is mandatory if '-StorageAccount' is not specified when Run-LISAv2.ps1, otherwise, it's optional-->
            <ARMStorageAccount>ExistingStorage_Standard</ARMStorageAccount>
        </Subscription>
        <!--This 'DefaultARMImageName' is the default value for '-ARMImageName' from Azure Gallery, when '-OsVHD' and '-ARMImageName' both are not specified form Run-LISAv2 test parameters-->
        <DefaultARMImageName>Canonical UbuntuServer 18.04-LTS Latest</DefaultARMImageName>
    </Azure>
    <HyperV>
        <Hosts>
            <Host>
                <!--ServerName can be localhost or Hyper-V host name-->
                <ServerName>localhost</ServerName>
                <DestinationOsVHDPath>VHDs_Path</DestinationOsVHDPath>
            </Host>
            <Host>
                <!--If run test against 2 hosts, set ServerName as another host computer name-->
                <ServerName>lis-01</ServerName>
                <!--If run test against 2 hosts, DestinationOsVHDPath is mandatory-->
                <DestinationOsVHDPath>D:\vhd</DestinationOsVHDPath>
            </Host>
        </Hosts>
    </HyperV>
    <WSL>
        <Hosts>
            <Host>
                <!--The name of the WSL host, which can be local or remote -->
                <ServerName>localhost</ServerName>
                <!--The destination path to extract the distro package on the WSL host-->
                <DestinationOsVHDPath></DestinationOsVHDPath>
            </Host>
            <Host>
                <ServerName>localhost</ServerName>
                <DestinationOsVHDPath></DestinationOsVHDPath>
            </Host>
        </Hosts>
    </WSL>

There are two ways to run LISAv2 tests:

Note when testing with 'Azure' platform:

a. Provide all parameters to Run-LisaV2.ps1

    .\Run-LisaV2.ps1 -TestPlatform "Azure" [-TestLocation "<Azure Region>(,<Azure Region>)..."] -RGIdentifier "<Identifier of the resource group>" [-ARMImageName "<publisher offer SKU version>(,<publisher offer SKU version>)..." | -OsVHD "<VHD from storage account>" ] [OverrideVMSize "<Azure_Vm_Size>(,<Azure_Vm_Size>)..."] [[-TestCategory "<Test Catogry from Jenkins pipeline>" | -TestArea "<Test Area from Jenkins pipeline>"]* | -TestTag "<A Tag from Jenkins pipeline>" | -TestNames "<Test cases separated by comma>"]"

    Basic Azure platform example:
    .\Run-LisaV2.ps1 -TestPlatform "Azure" -TestLocation "westus" -RGIdentifier "deployment" -ARMImageName "canonical ubuntuserver 18.04-lts Latest" -TestNames "VERIFY-DEPLOYMENT-PROVISION"
    Azure platform using secret file:
    .\Run-LisaV2.ps1 -TestPlatform "Azure" -TestLocation "westus" -RGIdentifier "deployment" -ARMImageName "canonical ubuntuserver 18.04-lts Latest" -TestNames "VERIFY-DEPLOYMENT-PROVISION" -XMLSecretFile "E:\AzureCredential.xml"
    Azure platform using default ARMImageName and auto-selected TestLoacation:
    .\Run-LisaV2.ps1 -TestPlatform "Azure" -RGIdentifier "deployment" -TestNames "VERIFY-DEPLOYMENT-PROVISION" -XMLSecretFile "E:\AzureCredential.xml"
    Azure platform using Comma-Separated ARMImageName, OverrideVMSize, TestLoacation and TestIterations to test multiple times for selected test cases:
    .\Run-LisaV2.ps1 -TestPlatform "Azure" -RGIdentifier "deployment" -TestLocation "westus,eastus2" -ARMImageName "canonical ubuntuserver 18.04-lts Latest,Redhat RHEL 7.7 Latest" -OverrideVMSize "Standard_D2s_v2,Standard_D15s_v2" -TestIterations 2 -TestNames "VERIFY-DEPLOYMENT-PROVISION" -XMLSecretFile "E:\AzureCredential.xml"

    .\Run-LisaV2.ps1 -TestPlatform "HyperV" [-TestLocation "ServerName"] -RGIdentifier "<Identifier of the vm group>" -OsVHD "<local or UNC path or downloadable URL of VHD>" [[-TestCategory "<Test Catogry from Jenkins pipeline>" | -TestArea "<Test Area from Jenkins pipeline>"]* | -TestTag "<A Tag from Jenkins pipeline>" | -TestNames "<Test cases separated by comma>"]
    HyperV platform examples:
    .\Run-LisaV2.ps1 -TestPlatform "HyperV" -RGIdentifier "ntp" -OsVHD 'E:\vhd\ubuntu_18_04.vhd' -TestNames "TIMESYNC-NTP"
    .\Run-LisaV2.ps1 -TestPlatform "HyperV" -RGIdentifier "ntp" -OsVHD 'http://www.somewebsite.com/vhds/ubuntu_18_04.vhd' -TestNames "TIMESYNC-NTP"

    .\Run-LisaV2.ps1 -TestPlatform "WSL" [-TestLocation "<WSL host name>"] -RGIdentifier "<Identifier for the test run>" -OsVHD "<local path or public URL>" [-DestinationOsVHDPath "<destination path on WSL host>"] [[-TestCategory "<Test Catogry from Jenkins pipeline>" | -TestArea "<Test Area from Jenkins pipeline>"]* | -TestTag "<A Tag from Jenkins pipeline>" | -TestNames "<Test cases separated by comma>"]
     WSL platform example:
    .\Run-LisaV2.ps1 -TestPlatform WSL -TestLocation "localhost" -RGIdentifier 'ubuntuwsl' -TestNames "VERIFY-BOOT-ERROR-WARNINGS" -OsVHD 'https://aka.ms/wsl-ubuntu-1804' -DestinationOsVHDPath "D:\test"

    Custom Parameters example:
    .\Run-LisaV2.ps1 -TestPlatform "Azure" -TestLocation "<Region location>" -RGIdentifier "<Identifier of the resource group>" [-ARMImageName "<publisher offer SKU version>" | -OsVHD "<VHD from storage account>" ] [[-TestCategory "<Test Catogry from Jenkins pipeline>" | -TestArea "<Test Area from Jenkins pipeline>"]* | -TestTag "<A Tag from Jenkins pipeline>" | -TestNames "<Test cases separated by comma>"] -CustomParameters "TiPCluster=<cluster name> | TipSessionId=<Test in Production ID> |  DiskType=Managed/Unmanaged | Networking=SRIOV/Synthetic | ImageType=Specialized/Generalized | OSType=Windows/Linux"
    Nested Azure VM example:
    .\Run-LisaV2.ps1 -TestPlatform "Azure" -TestLocation "westus" -RGIdentifier "deployment" -ARMImageName "MicrosoftWindowsServer WindowsServer 2016-Datacenter latest" -TestNames "NESTED-HYPERV-NTTTCP-DIFFERENT-L1-PUBLIC-BRIDGE" -CustomeParameters "OSType=Windows"

    Multiple override virtual machine size example:
    .\Run-LisaV2.ps1 -TestPlatform "Azure" -TestLocation "westus" -RGIdentifier "deployment" -ARMImageName "canonical ubuntuserver 18.04-lts Latest" -TestNames "VERIFY-DEPLOYMENT-PROVISION" -OverrideVMSize "Standard_A2,Standard_DS1_v2"

    'Ready' platform relies on '-TestLocation' to provide the locations of test environment, then LISAv2 will skip deployment step for selected test cases. TestLocation of Read platform should follow this Comma-Separated format: `<public_ip_address_a>:<ssh_port_a>,<public_ip_address_b>:<ssh_port_b>`
    Ready platform example:
    .\Run-LisaV2.ps1 -TestPlatform "Ready" -TestLocation "10.100.100.100:1111,10.100.100.100:1112" -TestNames "<Test cases separated by comma>" -XMLSecretFile "E:\AzureCredential.xml" [-EnableTelemetry]

b. Provide parameters in .\XML\TestParameters.xml.

    .\Run-LisaV2.ps1 -TestParameters .\XML\TestParameters.xml

Note: Please refer .\XML\TestParameters.xml file for more details.

More Information

For more details, please refer to the documents here.

Contact: lisasupport@microsoft.com

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

glima/lisa

ErrorLooks like something went wrong!

About

Resources

Code of conduct

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Error
Looks like something went wrong!

Packages