Skip to content

virt: Rewrite network/nic management #316

Closed
wants to merge 6,057 commits into from
@cevich
Autotest member
cevich commented Apr 23, 2012
  • Integrated new network code with kvm module
  • Added many logging and debugging statements for new code
  • unit tests: New virt_utils_unittests tests
  • Updated shared ping test module
  • Updated kvm nic_bonding test module
  • Updated kvm nic_hotplug test module comment
  • virt_utils Env init now always warns when creating an empty env file
  • virt_utils Env init prints warning traceback when loading env fails
  • Pickling support for all new classes
  • Raise exception when adding nic with clashing name
  • New virt_utils_unittest deletes temporary testing database
  • IP/MAC cache is only updated with new entries
  • VM cleanup resets static addresses properly
  • Enhanced output of needs_restart() VM comparison function
  • Enhanced Env class to output more details on non-fatal load failures
  • Old virt_util functions all raise depreciation warnings to aide bug squashing
  • remove unnecessary nic_mode from subtests parameter, rely on default nettype=bridge instead
  • virt config: New universal virt. networking configuration nettype + netdst parameters determine networking style and destination for virtual NICs. Separate NIC sets can be defined for each VM. Documented examples of other per-nic parameters in config. comments.
  • virt_utils change etails:

New Class: PropCan
Allow container-like access to fixed set of items (slots)
This allows NIC/Net handling code to lock-down a fixed set of
parameters relevant to a vm_type / driver_type combination.
Data is accessible in a dictionary-like manner or as object
properties, however empty or None value items will pretend to
not exist. The special empty/None handling optimises access
and storage by only representing values containing meaningful
data.

New Class: Base_VirtIface
Networking information for single guest interface and host
connection. This is an abstract base-child-class of PropCan
containing the minimal amount of networking keys needed, along
with some basic name and mac address handling methods. Some
amount of effort is made to ensure stored mac address data
is in an expected format - two ASCII character hex-byte
representation, separated by :'s, six or less bytes in total.
Complete mac addresses are generated by filling in missing
trailing bytes from a sequential pool maintained as a class
variable. Numbering starts randomly from 0-255, then proceeds
incrementally for each byte requested. Sequential byte generation
used to ease unit testing.

New Class: Libvirt_VirtIface & KVM_VirtIface
Specializations of Base_VirtIface parent class (and PropCan)
with networking keys specific to Libvirt or KVM respectively.
If a new type or variation is needed, simply subclass Base_VirtIface
and update the VMNet_Style.VMNet_Style_Map as needed. The new or
variant class will be used automatically based on resolving
vm_type, and/or driver_type from the supplied Params dict (see
other classes below)

New Class: VMNet
Abstract base-class collection of networking information, stored
within a particular container class (e.g. a subclass of
Base_VirtIface). This class behaves like a list of PropCans.
It may be initially empty or created from a list-like container
of dictionary-like objects holding key/value pairs of networking
data. Items within a VMNet instance are indexed by number and
the 'nic_name' value of each contained item. Consequently, the
'nic_name' key is required for each contained item, and it's value
must be hashable. Updates to the 'mac' value of each item is handled
specially, in that storage of incomplete byte lists, and/or values
that do not pass the container class's mac_is_valid method,
will not be stored.

New Class: VMNet_Style
Make decisions about needed info from vm_type and driver_type params.
The VMNet_Style_Map class variable is utilized for initial setup of
VMNet derivatives (see below). Essentially it helps provide a
container-class type based on the contents of Params dict-like
values for vm_type and driver_type. It also establishes a mac
address prefix to be used when generating MAC addresses. However,
any number of additional VMNet instance variables may also be set
with default values. The special 'default' keys are utilized when
vm_type and/or driver_type don't match from Params.

New Class: ParamsNet
VMNet sub-class specialized for initialization from a Params
dict-like instance. Essentially it filters out and initializes
all networking info. for a particular VM. Once initialized,
networking information is accessed by NIC index or it's name (e.g.
'nic1', 'nic2', etc.) in a dictionary-like or object property-like
manner. For example, instance[0].mac, instance('nic1').mac,
instance['nic1']['mac'], etc. Each nic object will be stored in
the proper container class as determined by the VMNet_Style class.
A utility generator method is provided for iteration over all
VM's and NIC's mac address params definitions. This makes validating
or checking for mac address conflicts easy.

New Class DbNet:
VMNet sub-class specialized for initialization from a database.
Essentially analogous to ParamsNet, with a set of database
management and locking methods. Database items are hashed by the
key provided at initialization time, so hash-clashes must be managed
by implementation code. Changes to any indexed NIC object
automatically result in atomic database updates. This also follows
for any re-ordering or deletion of contained items. The lock/unlock
methods must be called externally prior to any database activity.
Also provides a utility generator function for iteration over all
MAC addresses contained within the database.

New Class: VirtNet (Re-write former networking code)
Multi-inheritance of the ParamsNet and DbNet classes providing
mediated networking info. management behavior combination of both.
Provides the former generate_mac_address(), free_mac_address,
set_mac_address(), and get_mac_address() functions, which behave
exactly as before. The only exception is the 'instance' parameter
is no-longer required as it is established at initialization time.
Setting and Freeing of arbitrary mac addresses is allowed (as
before), however generation of new mac addresses is a sieve over
cached values. Deleting the database cache or freeing individual
NICs MAC addresses will extend the pool within the confines of any
mac prefixes defined by the responsible VMNet_Style definition. A
utility generator is provided which wrapps the mac_index generators
from ParamsNet and DbNet (in that order). This permits iteration
over all known MAC address values possible.

  • KVM change details:

    Intended CodePath:

    Base Class __init__:
            Load/Parse Cartesian params + address_pool cache
    
    Base Class nic addition:
        BaseVM.add_nic
        BaseVM.activate_nic
        <testing>
        BaseVM.deactivate_nic
        BaseVM.del_nic
    
    KVM VM class __init__:
        Call BaseVM.__init__
    
    KVM VM create:
        vm.create:
            vm.add_nic -> vm.add_netdev
            specialized nic & netdev activation
        <testing>
    
    KVM VM nic addition:
        vm.add_netdev -or- vm.add_nic -> vm.add_netdev
            Uses BaseVM.add_nic
        vm.activate_netdev
        vm.activate_nic
        <testing>
        vm.deactivate_netdev
        vm.del_netdev
        vm.deactivate_nic
        vm.del_nic
    

    Modules Changes summary:

    BaseVM module
    Before:
    Different behavior per sub-class

    After:
        Defined new VMUnknownNetTypeError, raised when validating 'nettype'
        parameter doesn't match supported type.  Supported types left up
        to sub-classes to define.
    

    BaseVM.init
    Before:
    Subclasses and methods deal with params and cache DB ad hoc
    After:
    Common setup/validate networking from parameters and cache DB input

    BaseVM._generate_unique_id
    Before:
    vm.instance unique across thousands of VM instances created per
    second
    After:
    vm.instance unqie across millions of VM instances created per
    second

    BaseVM.get_mac_address
    Before:
    Obtain mac defined in params or cache DB input. Raise
    VMMACAddressMissingError if no mac defined (note: doesn't actually
    check vm object in any way).
    After:
    Loading mac from params and/or cache db moved to init. Check
    instance variables for mac and return or raise
    VMMACAddressMissingError

    BaseVM.get_address
    Before:
    Require sub-classes to impliment.
    After:
    Rely on init to parse options and cache DB. If nic has mac
    defined, and corresponding IP in IP/MAC cache, verify in host
    arp cache then return IP. Return localhost if no
    mac address defined for nic.

    BaseVM.free_mac_address
    Before:
    Different behavior per sub-class
    After:
    Unified behavior to drop mac reference only if they're not
    staticly defined. When static, the original mac from params
    is re-established for the nic.

    BaseVM.add_nic
    Before:
    Left up to subclasses
    After:
    Adds new or updates existing NIC definition on vm instance with
    new or changed options. Generates random nic_name if none specified.
    If IP and MAC are defined, updates IP/MAC cache. Otherwise, generates
    random mac address. Returns standardized instance of dict-like object
    containing NIC new/existing (possibly changed) parameters.

    BaseVM.del_nic
    Before:
    Left up to subclasses
    After:
    Drop references to mac in cache DB. Drop references to IP
    from IP/MAC cache. Ensure NIC parameters removed from vm instance.

    BaseVM.activate_nic
    Before:
    Host resource allocation bundled with sub-class add_nic method.
    After:
    Require sub-classes to implement separate from add_nic.

    BaseVM.deactivate_nic
    Before:
    Host resource deallocation bundled with sub-class add_nic method.
    After:
    Require sub-classes to implement separate from del_nic

    kvm_vm.init
    Before: Track nic-indexed netdev_id, device_id, and tapfd separately.
    After: Moved into BaseVM.init, single virt_utils.virtnet class.

    kvm_vm.clone
    unchanged- rely on init to produce new virtnet instance

    kvm_vm.__make_qemu_command
    add_nic: unchanged
    add_net: unchanged
    Body: Referenced names, and assignment-order changes / updates
    so may be called independently of vm.create().

    kvm_vm.create
    Before:
    For each nic- Parse params, generating random netdev_id, device_id.
    For tap a.k.a. bridge devices, obtain ifname, bridge name, tapfd,
    add to bridge, then raise interface
    For each nic- obtain mac or generate random mac
    if params define ip address,update address cache
    make_qemu_command
    background-run qemu command
    for each open tapfd- close tapfd
    After:
    (Rely on __init
    to parse params and cache db content)
    For each nic, use common add_nic(), but specialize activation:
    if nettype is bridge/tap:
    open tap device node
    register tapfd and tapfd_id to qemu
    add interface to bridge
    raise interface
    if nettype is user:
    log info
    __make_qemu_command
    background-run qemu command
    for each already parsed nic:
    if tapfd open, close tapfd

    kvm_vm.destroy
    Same behavior, updated names

    kvm_vm.get_port
    Same behavior, updated names

    kvm_vm.get_peer
    Unchanged

    kvm_vm.get_address
    Before:
    Parses params for IP, if bridge/TAP mode defined for nic, parses
    params for mac address. Consults IP/MAC cache DB for IP. Returns
    localhost if no IP defined or nic_mode is not 'tap'.
    After:
    Functionality made common by implimentation in BaseVM class.

    kvm_vm.get_ifname
    Before:
    Return the letter 't' followed by the vm's nic index, followed
    by a '-' and the last 11 characters of vm.instance.
    After:
    Retrieve ifname (generated in vm.add_netdev) for NIC or raise
    KeyError.

    kvm_vm.get_mac_address
    Before:
    Duplicated code from old BaseVM method with same name
    After:
    Functionality made common by implimentation in BaseVM class.

    kvm_vm.free_mac_address
    Before:
    Use virt_utils function to drop mac reference from cache DB.
    After:
    Functionality made common by implementation in BaseVM class.

    kvm_vm.add_netdev
    Before:
    Parse params for bridge name, generate random netdev_id, set vlan
    to number of other tap NICs on VM. Generate ifname, open tap
    device node, set tapfd, bring up interface, generate random
    tapfd_id, run 'netdev_add' monitor command. Verify netdev_id shows
    up in network info. Return netdev_id.
    After:
    Rely on BaseVM.init to parse options and cache DB for bridge
    name. Validate supported nettype specified in parameters ('bridge'
    or 'user') Generate random netdev_id, use defined vlan or set based
    on # total of nics on VM., and generate ifname. Open tap device
    node moved to vm.activate_netdev(). Generate random tapfd_id.
    Running 'netdev_add' moved to vm.activate_netdev(). Return
    netdev_id. Moved running of 'netdev_add' functionality into
    vm.activate_netdev.

    kvm_vm.del_netdev
    Before:
    Run 'netdev_del' monitor command. Validate netdev_id no longer
    shows in network info.
    After:
    Moved 'netdev_del' monitor command and validation to
    vm.deactivate_nic(). Remove properties from nic: netdev_id,
    vlan, ifname, tapfd, tapfd_id.

    kvm_vm.add_nic
    Before:
    Take fixed set of nic parameters, generate a name for nic
    (nic_id) if not specified. Call add_netdev() if netdev_id parameter
    not specified. Generate random mac address if mac parameter not
    specified. Setup/Run device_add monitor command with id=nic_id.
    Return dict containing nic parameters. Verify nic_id shows up
    in qtree.
    After:
    Take arbitrary set of key/value nic prameters. Call BaseVM.add_nic
    method with nic parameters. Generate random device_id. Call
    add_netdev with parameters if no netdev_id specified. Return
    standardized dict-like object containing nic parameters. Moved
    call to device_add command into vm.activate_nic method.

    kvm_vm.del_nic
    Before:
    Runs device_del monitor command. Checks for wait timeout for
    nic to not show in qtree.
    After:
    Call BaseVM's del_nic method. Functionality to
    call device_del moved to vm.deactivate_nic

    kvm_vm.activate_netdev
    Before:
    Functionality combined with vm.add_netdev
    After:
    Re-validate nettype is supported, setup netdev_add monitor
    command with parameters from nic based on nettype value. Run
    netdev_add monitor command. Verify netdev_id shows up in network
    info.

    kvm_vm.deactivate_netdev
    Before:
    Functionality combined with vm.del_netdev
    After:
    Call netdev_del monitor command, verify netdev_id no longer shows
    in network info.

    kvm_vm.activate_nic
    Before:
    Functionality combined into vm.add_nic
    After:
    Setup/run device_add monitor command with id=nic_name. Verify
    nic_name shows up in qtree.

    kvm_vm.deactivate_nic
    Before:
    Functionality combined with vm.del_nic
    After:
    Run 'device_del' monitor command. Validate netdev_id unknown to
    monitor. Raise virt_vm.VMDelNicError if qtree shows nic_name after
    wait time.

    kvm_vm.migrate
    Unchanged

Signed-off-by: Chris Evich cevich@redhat.com

lmr and others added some commits Feb 15, 2012
@lmr lmr kvm_vm: Amend b69c24346f912bb00ad25a6f6206b8c3afb2a865
There was yet another place that needed to be replaced
with the variable cd_format.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
15743f0
@LaneWolf LaneWolf virt: Fix using iso9660 module for xen
Commint 3b7fe85 introduced iso9660 module for handling cdrom isos. However, it just replace mounting iso with class handling work with iso. This is required forkvm based libvirt tests. Xen based tests uses special handling as cdrom is not as easily usable as with kvm. Missing mount in specified directory cause test to failed.

This patch add missing handling required for xen. Unfortunately, this is kernel based solution requiring root priviledges.

Signed-off-by: Miroslav Rezanina <mrezanin@redhat.com>
f552a00
@LaneWolf LaneWolf virt: Fix 2.5+ syntax usage in libvirt_vm.py
Commit 5734af00 uses scheme from libvirt uri to identify hypervisor autotest is running on. To get scheme string "urlparse.urlsplit().scheme" expression is used. However, using touple attributes to access values of urlsplit touple is feature introduced in python 2.5 so it breaks run on system with python 2.4 (e.g. RHEL 5).

This patch uses array index access for getting requested value.

Signed-off-by: Miroslav Rezanina <mrezanin@redhat.com>
32eb3c0
Jiří Župka kvm test: Change cpuflags test call structure and add new test
The changes are neccesary because people who use
the test want to use this test another way.

Also, Adds new subtest which tests warning of
<qemu-kvm> -cpu started with parameter "check".

Signed-off-by: Jiří Župka <jzupka@redhat.com>
946a7d7
@vinsonlee vinsonlee ltp: Use runltp skipfile option to skip tests instead of patch file.
Signed-off-by: Vinson Lee <vlee@twitter.com>
4b3970c
@clebergnu clebergnu client/autotest_local.py: split option parser to a separate python mo…
…dule

This will help to create test runners similar to autotest-local but
with other options (think of added functionality not relevant to plain
autotest-local)

Signed-off-by: Cleber Rosa <crosa@redhat.com>
65fac83
@clebergnu clebergnu client/autotest_local.py: turn app code into a class
Turn the application code into a class so that we can inherit from
it and have variations of the autotest-local tool with custom options
or for some specific purporses.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
3b92f14
@lmr lmr contrib/install-autotest-server-f16.sh: Fix a basename problem
Also, add the df -P option (posix output format) to avoid
problems with long partition names, such as LVM.

Signed-off-by: Daniel Veillard <veillard@redhat.com>
f11f8f7
@lmr lmr install_server (cobbler): Don't change profile if no fallback found
If we passed no profile to the install server and we don't
have a fallback profile set, then don't try to set the
machine profile and instead just go with what is currently
set to the machine in the cobbler server.

Thanks to Nishanth Aravamudan <nacc@linux.vnet.ibm.com>
for reporting the problem and pointing out a solution.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
d6c70b8
@mkrizek mkrizek scheduler: Rework entry points
As part of the effort to make autotest more packaging
friendly, refactor some of the scheduler code and
rename:

scheduler/monitor_db.py -> scheduler/autotest-scheduler
scheduler/monitor_db_babysitter.py -> scheduler/autotest-scheduler-watcher

This renaming makes the entry points easier to discover
and figure out what they do. The actual functionality
of the entry points was actually split to libraries,
to ease unit testing.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
Signed-off-by: Martin Krizek <mkrizek@redhat.com>
725c8e3
@mkrizek mkrizek Add logs directory and pid files to .gitignore
Signed-off-by: Martin Krizek <mkrizek@redhat.com>
d8e48e9
Nishanth Aravamudan install_server (cobbler): ensure a profile name gets printed
If no fallback profile is defined, 'profile' remains None and results in
messages like:

Installing machine elm3b202.beaverton.ibm.com with profile  (timeout 3600 s)

Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>
f6129e7
@lmr lmr virt.kvm_vm: Set driver_type attribute for kvm_vm
This way we fix a bug introduced by f552a005, where we
start checking out driver_type to verify if some xen
installation hacks were needed during unattended install
setup.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
b6477ea
@clebergnu clebergnu apache: add conditional aliases to the apache config files
This is intended to support different installation types of
autotest, either from source or from packages.

Signed-off-by: Martin Krizek <mkrizek@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
494181a
@clebergnu clebergnu tko: replace legacy help URL with best current wiki doc on tko
Signed-off-by: Cleber Rosa <crosa@redhat.com>
56fb0d1
@lmr lmr libvirt: Fix tests-shared.cfg.sample
As it was referring 'kvm' as the vm_type rather than
libvirt.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
1407005
guyanhua libvirt: Introducing 'virsh version' test
This patch adds three test cases for "virsh version" command.

Use three cases:(1) virsh version
                (2) virsh version with a unexpected option
                (3) virsh version with libvirtd service stop

Signed-off-by: Gu Yanhua <guyanhua-fnst@cn.fujitsu.com>
703abd4
@lmr lmr libvirt: Add member functions to libvirt_vm
Signed-off-by: Tang Chen <tangchen@cn.fujitsu.com>
9a391b3
@tang-chen tang-chen libvirt: Add virsh_migrate test cases configuration
This patch adds 4 test case for virsh_migrate.
All these 4 cases are uni-direction migration.

Signed-off-by: Tang Chen <tangchen@cn.fujitsu.com>
de7c860
@tang-chen tang-chen libvirt: Improve virsh_migrate to support uni-direction migration
This patch adds a cleanup function to clean up environment
on destination host after migration. So that we can do
uni-direction migration in this test.

Signed-off-by: Tang Chen <tangchen@cn.fujitsu.com>
62caa3f
@lmr lmr virt.libvirt_vm: Add member functions for libvirt_vm
This patch adds 3 member functions for libvirt_vm:
    1) get_id()
    2) get_uuid()
    3) get_xml()

Signed-off-by: Tang Chen <tangchen@cn.fujitsu.com>
4b8e75f
@lmr lmr common_lib.base_utils: Do not hardcode /bin/bash on BgJob class
At some point we hardcoded bash as the BgJob default class,
but that artificially restricts us in embedded environments
with busybox, for example. So let's try to use bash, but
if we can't just default to /bin/sh.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
e9d1f0d
@lmr lmr harness_standalone: rework init level detection
 * Handle embedded systems with busybox
 * Inform the user that in some times, when we
   determine '2' as the runlevel is because we
   really couldn't figure out something better
   and we are in fact, guessing... we did try,
   though.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com
9af00bd
@lmr lmr virt.virt_vm: Omit dot on image name when no image format is specified
This way we can handle image names that are not supposed to have
extensions.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
5bc2d9a
@lmr lmr Update old style (RHEL 5) kvm unittest
We actually forgot to update the testcase type on the
config file.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
a6fcb62
@lmr lmr ltp: Re-enable LTP capability tests
The capability module is built-in into the Linux kernel since 2.6.24.

Tested the re-enabled tests capget01, capget02, capset01, and capset02
from ltp-20110104. They pass on the following distros.
Centos 5     - 2.6.18-274.18.1.e15
Ubuntu 8.04  - 2.6.24-30-generic
Ubuntu 10.04 - 2.6.32-38-generic
Ubuntu 12.04 - 3.2.0-16-generic
Fedora FC16  - 3.2.6-3.fc16.x86_64

Signed-off-by: Vinson Lee <vlee@twitter.com>
21a585b
@cevich cevich virt: fixed spelling mistake
Signed-off-by: Chris Evich <cevich@redhat.com>
8e51f07
@lmr lmr Apache config directives: Fix typo and fix rewriterule regexp
Currently we are using regexp matching on the apache config
files the following way:

foo(.*) alias/bar$1

Which can cause some problems with some url patterns. So
let's make the regexp to take into account an extra leading
/, such as in:

foo/(.*) alias/bar/$1

Also, add a favicon directive, so the favicon set actually
shows, and force the .DEBUG, .INFO, .WARNING and .ERROR
log files to be served as plain text files.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
10533a4
@lmr lmr utils/build_externals.py : Update gwt to 2.4.0
Following the policy of keeping track of the latest
stable versions of each dependency.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
ca1a730
@lmr lmr gwt frontends: Rename css files
Since they are causing problems with the otherwise simple
apache rewrite files. It's a cleaner solution to the
problem, rather than trying to find some regexp that
will avoid the css naming clash.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
5c56da3
@lmr lmr Apache config directives: Remove leading slash on directives
Renaming the conflicting css files turned out to be
a cleaner solution than to prepend the slashes on
the configuration file, therefore we are reverting
part of commit 10533a4 on this change.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
8d2d884
@vinsonlee vinsonlee ltp: Use the LTP source directory as the temporary directory for runltp.
When using the Autotest generated temporary directory for runltp,
several tests fail but otherwise wouldn't if a different temporary
directory is used.

For example, access02 fails with a message like the following.
access02 1 TBROK : tst_tmpdir:
mkdtemp(/usr/local/autotest/tmp/tmpsIEABp_ltp/ltp-sQqKR20197/acc7CjKsN)
failed: errno=EACCES(13): Permission denied

The root cause of why this occurs is still unknown.

This patch switches the temporary directory used from the Autotest
generated one to the LTP source directory to workaround this issue.

Signed-off-by: Vinson Lee <vlee@twitter.com>
87cc668
@lmr lmr Updating references to the new autotest hosting page
We should have done this a while ago, but bear with us.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
5d5a110
@lmr lmr logs dir: Update README
With the recent scheduler name change.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
a2f79df
@clebergnu clebergnu client/common_lib/test.py: remove unused imports
Signed-off-by: Cleber Rosa <crosa@redhat.com>
a4ad75a
@lmr lmr rpc web client: CSS update
Fix the default font settings and color schema.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
23f7ce7
@lmr lmr rpc web client: Fix small css inconsistency
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
3bff51b
@sathnaga sathnaga virt.tests.kdump: Add offset parameter to kdump
Add the offset to the crash kernel parameter.

Signed-off-by: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
6a4c6d3
@veillard veillard Server install script updates for RHEL-6 and Centos-6
Changes in this patch for those distros:
- we need to use EPEL repos on get some of the needed packages
- we use init.d and not systemctl on the older OSes
- the lsb package provide a required support of the init.d script
- Now that it supports other distros, drop the -f16 from the name

A generic fix to skip out 192.168.122.x nextwork addresses usually
associated to libvirt default network (fixes the last output message
not an installation problem)

Signed-off-by: Daniel Veillard <veillard@redhat.com>
ff59cb1
Nishanth Aravamudan conmux: add IVM support
IVM (comes with VIOS) has some differences from HMC, but is overall
pretty similar. Tested on VIOS 2.2.1.3 only.

Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>
8a87b06
Yu Mingfei virt.libvirt_vm: Add virsh_define() function
This patch adds a virsh_define() function to libvirt_vm.

Signed-off-by: yumingfei <yumingfei@cn.fujitsu.com>
ab211ee
@tang-chen tang-chen virt.libvirt_vm: Add attach-device() and detach-device() methods
This patch adds 2 functions to libvirt_vm:

    1) attach-device()
    2) detach-device()

Signed-off-by: Tang Chen <tangchen@cn.fujitsu.com>
5bb8126
guyanhua virt.libvirt_vm: Add virsh_version() function
Signed-off-by: Gu Yanhua <guyanhua-fnst@cn.fujitsu.com>
8d11cbf
@tang-chen tang-chen virt.tests: Improve virsh_migrate to support destination vm state check
If migration failed, VM could be alive on destination,
but not running (paused). So vm.is_alive() cannot make
sure that migration succeeded. VM state on destination
needs to be checked.

Signed-off-by: Tang Chen <tangchen@cn.fujitsu.com>
d25d5bb
guyanhua virt.libvirt: Add three test cases for "virsh hostname"
This patch adds three test cases for "virsh hostname" command.

Use three cases:(1) virsh hostname
                (2) virsh hostname with an unexpected option
                (3) virsh hostname with libvirtd service stop

Signed-off-by: Gu Yanhua <guyanhua-fnst@cn.fujitsu.com>
645d9fe
@lmr lmr virt.libvirt_vm: Fix spacing and redundant get_uuid method
The autotest standard is 2 lines of space between methods
and functions. Also, get rid of an extra get_uuid method
defined on that class.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
8579ae1
@vinsonlee vinsonlee ltp: Fix pass/fail determination
The existing results checking checks for any appearance of the string
'FAILED' from stdout of the test run. runltp logs as information the
location of the failed command file with the string 'FAILED COMMAND
File'. This leads to the test being detected as a failure on every run.

runltp stdout excerpt:
-e FAILED COMMAND File:
/usr/local/autotest/results/default/ltp/debug/failcmdfile

This patch changes the heuristic tokens used to determine a failure. If
the tokens TFAIL, TBROK, and TWARN are observed, report the run as a
failure. This correlates to how LTP reports the status of an individual
test. See the LTP source files include/test.h and lib/tst_res.c for more
information.

This patch also fixes the stdout result parsing by correctly splitting
out lines from stdout. It was incorrectly splitting stdout into words.

Changes from v2:
 * Gather a list of all failed tests rather than just the first failed
   test that came up

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
Signed-off-by: Vinson Lee <vlee@twitter.com>
72444e0
@Antique Antique virt.libvirt_vm: Fix disk cache option
We were not reflecting the present of the cache
option on the virt-install command.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
b6bb711
@ypu ypu Virt tests: Move image_func outside the loop of vms
Update process put image_func outside the loop of vms to make the
image_func can work for cases don't need to boot up vm. Such as some
tests related with qcow2.

And fix the missing docstring for vm_first.

Signed-off-by: Yiqiao Pu <ypu@redhat.com>
0a50baf
@kongove kongove virt-test: add NTttcp subtests
This case will test tcp throughput between 2 windows guests,
or between 1 guest and 1 external Windows host.

When test between guest and external Windows host,
'receiver_address' should be set to external Windows' ip address.

NTttcp is not a freely redistributable binary, so you *must* download it
from microsoft and be in agreement with its EULA. See @see tag for a
complete download link and also documentation on how to integrate it to
your autotest setup.

@see: http://msdn.microsoft.com/en-us/windows/hardware/gg463264
@see: http://download.microsoft.com/download/f/1/e/f1e1ac7f-e632-48ea-83ac-56b016318735/NT%20Testing%20TCP%20Tool.msi
@see: https://github.com/autotest/autotest/wiki/KVMAutotest-Networking

! ntttcp.au3: This script will sign "End-user license agreement"
!   for you, please don't use this script if you don't agree EULA.

This test will generate result files with 'standard' format,
split different items by '|', use one line as the title.
We can analyze them by a general modules.
    raw_output_1.RHS:
      buf(k)| throughput(Mbit/s)
           ...
          64| 2407.548
         128| 2102.254
         256| 4930.362
         512| 4723.035
        1024| 4725.334

Changes from v1:
- pin vcpus/vhost_net threads to numa node
- add autoio script for ntttcp test
- user should put msi and autoit script to iso
- fix threads sync issue
- set test time to 30 seconds
- support to use fixed receiver buf or use same buf as sender
- 30 seconds is not enough, assign buf number to 2000000

Signed-off-by: Qingtang Zhou <qzhou@redhat.com>
Signed-off-by: Amos Kong <akong@redhat.com>
0b0d403
@kongove kongove virt: Add vhost_threads and vcpu_threads to VM object
Record vhost_net threads ID and vcpus threads ID to vm object
after creating VM.

Signed-off-by: Amos Kong <akong@redhat.com>
b7388d2
@kongove kongove virt_test_utils: Add pin_vm_threads
This function is used to pin vhost and vcpu threads of VM to
host cpu (in same numa node).

Signed-off-by: Amos Kong <akong@redhat.com>
2126b76
@kongove kongove virt: Introduce regression testing infrastructure
> regression.py:
Usage: python regression.py $testname $dir1 $dir2 $configfile
 'regression' module is used to compare the test results
 of two jobs, we can use it (regression.compare()) at
 the end of control file, this script can also be used directly.

Example:
| # python regression.py netperf /result1-dir /result2-dir perf.conf

> analyzer.py:
Usage: python analyzer.py "$results list1" "$results list2" $log_file
 It's used to compute average, standard deviation, augment rate, etc,
 and compare two test results (standard format).

 It can be used directly, example:
| # python analyzer.py "result-v1-1.RHS result-v1-2.RHS" \
|   "result-v2-1.RHS result-v2-2.RHS result-v2-3.RHS" log.txt
| Thu Jan  5 10:17:24 2012
|
| == Avg1 SD Augment Rate ==========================
|     TCP_STREAM
|     size|sessions|throughput|   cpu|normalize| ...
|     2048|       2|  14699.17| 31.73|   463.19| ...
| %SD     |     0.0|       0.6|   0.0|      0.8| ...
|     2048|       4|  15935.68| 34.30|   464.66| ...
| %SD     |     0.0|       0.3|   1.7|      1.5| ...
| ...
|
| == AvgS Augment Rate =============================
|     TCP_STREAM
|     size|sessions|throughput|   cpu|normalize| ...
|     2048|       2|   7835.61| 31.66|   247.36| ...
|     2048|       2|   8757.03| 31.94|   274.14| ...
| %       |    +0.0|     +11.8|  +0.9|    +10.8| ...
|     2048|       4|  12000.65| 32.38|   370.62| ...
|     2048|       4|  13641.20| 32.27|   423.29| ...
| %       |    +0.0|     +13.7|  -0.3|    +14.2| ...
|

> perf.conf:
 config test related parameters.

perf regression guide:
https://github.com/autotest/autotest/wiki/KVMAutotest-Networking

Changes from v1:
- refactor analysis code
- add standard deviation percent
- only provide mechanism to user, user can use tools directly or use the lib in scripts

Signed-off-by: Amos Kong <akong@redhat.com>
79ecfd0
@kongove kongove virt-test: Refactor netperf test and add analysis module
Always use a VM as netperf server, we can use
another VM/localhost/external host as the netperf
clients.
We setup env and launch test by executing remote
ssh commands, you need to configure the IP of
local/external host in configure file, VMs' IP
can be got automatically.
Generate a file with 'standard' format at the end of test,
then we can analyze them by general module.

Changes from v1:
- record packet bytes
- enable arp_ignore
- get packet info from ifconfig
- shape functions
- don't change ssh config
- use server.hosts.ssh_host.SSHHost to setup ssh

Signed-off-by: Amos Kong <akong@redhat.com>
629c9a4
@kongove kongove netperf: pin guest vcpus/memory/vhost thread to numa node
Dynamically checking hardware and pin guest cpu threads and
guest memory to last numa node

Changes from v1:
- assign numanode to -1 for netperf test

Signed-off-by: Amos Kong <akong@redhat.com>
46454f4
@ldoktor ldoktor virt.virt_vm: Add option to create raw images with dd
Adds option to create raw images with dd (non-sparse file unlike
qemu-img one).

Signed-off-by: Lukas Doktor <ldoktor@redhat.com>
f2def6c
@ldoktor ldoktor tests.cfg.sample: change import order
Currently subtests.cfg is proceeded and then all other configs. My test
needs to override smp parameter in some variant which is currently
impossible.

Using words current order means: we define subtests variants, than we
specify base and guest and other details. In the end we limit what
we want to execute.

My proposed order enables forcing base/guest params in subtest variants.

By words this means we specify base, guest system, cdkeys, etc. and in
the end we define subtests with various variants. Then we limit what
we actually want to execute but now subtest can force varius base/guest
settings.

Signed-off-by: Lukas Doktor <ldoktor@redhat.com>
e64b17d
@ldoktor ldoktor virt.kvm_vm: Support for bool option in _add_option
Let _add_option know about type of this option. It can verify value
and decide which one it uses. Currently it supports bool type
and default.

Signed-off-by: Lukas Doktor <ldoktor@redhat.com>
c6ecdae
@ldoktor ldoktor tests.cgroup: Rework test execution
Complete rework of cgroup test from subtests to singe-test-execution.
It improves stability of testing and allows better test
customiuations. The speed is similar/faster in single variant
execution and a bit slower in all-variants execution compare to
previous version.

Most tests now supports cgroup_test_time, cgroup_test_limit*
and other cfg params for test tweaking.

Also some enhancements were made:
* blkio_bandwidth - support for multiple speeds
* blkio_throttle - support for multiple VMs
* blkio_throttle_multi - support for multiple VMs
* cpu_cfs_util - per vcpu stats
* cpu_share - support for multiple speeds
* cpuset_cpus - per vcpu stats
* cpuset_cpus_switching - time-driven finish
* devices_access - RW and RO testing and QMP support
* freezer
* memory_limit
* memory_memsw_limit - Newly added
* memory_move

+ a lot of bugfixes.

See descriptions for details.

Signed-off-by: Lukas Doktor <ldoktor@redhat.com>
92eef42
@tang-chen tang-chen Fix the virsh_start() return value.
If VM is already alive, we should also return True in virsh_start().

Signed-off-by: Tang Chen <tangchen@cn.fujitsu.com>
feb62f8
guyanhua Adds service_libvirtd_control API to libvirt_vm.py file
This patch adds service_libvirtd_control API to libvirt_vm.py file.

Signed-off-by: Gu Yanhua <guyanhua-fnst@cn.fujitsu.com>
12d2a25
@lmr lmr virt.libvirt_vm: Fix toplevel dosctring
As it was really outdated.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
b454fe9
@lmr lmr base_utils_unittest - Force use of bash when testing shell scaping
In any box that does not point /bin/sh as /bin/bash, the
unittest tends to fail due to slight differences in behavior
among shell versions. As utils.BgJob automatically selects
bash if it's available in the system, we can reasonably
expect that we won't have trouble. In embedded environments,
it might be a different story, but even then the chances of
utils.sh_escape fail to produce an appropriately escaped string
are low.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
ebb771f
Nishanth Aravamudan client/kernel: fallback to defconfig if no .config is present
due to `mv' not finding the source file. Rather than relying on .config
being present, fallback to defconfig when its not.

Changes from v1:
  * Fix kernel_unittest.py and add a new unittest for
    testing extraversion with no config.

Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>
86a63e5
Nishanth Aravamudan client/kernexpand: handle 3.0-style kernel versions
Passing in 3.0 to autotest results in messages about
being unable to parse the major/minor version. This
is because the code assumes a 3-digit verions number.
Additionally, the 'raise' in this chunk of code throws
its own exception due to a raw string parameter.

Fix both by having an alternate function to check
post-2.x version numbers, that will also consider
this new naming schema. It's forward compatible,
for the (potentially) far away 4.x times (well,
assuming that the naming schema will remain the
same).

Changes from v2:
 * Open the urls and after that close them

Changes from v1:
 * Rewrite of the original patch code
 * Addition of unittests
 * Fixed incorrect logic that printed unavailable
   URLs when it shouldn't do that
 * Used urllib2 instead of external programs to
   verify the state of a given URL
 * Fixed docstrings and comments
 * Fixed use of string exceptions

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>
77671d1
Nishanth Aravamudan client/base: fix cleanup_kernels string manipulation
This code uses the length of a variable, which includes the '/boot'
prefix to trim the length of the kernel name, which in turn does not
have the /boot prefix. So this results in bad kernel version strings --
and when using autotest to build kernels, the default control file
results in all kernels being deleted from the system.

Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>
6e4c769
Nishanth Aravamudan client/boottool: add default entry in set_default if no default exists
If no 'default' entry is in yaboot.conf (the only type of bootloader I
tested on), then boottool --set-default= will fail to change the default
and reboots will go back to the first listed kernel. Fix this by
determining if we have found a default line and if not, inserting one at
the top of the file.

Changes from v1:
 * Fixed perl syntax error

Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>
73bda1c
@lmr lmr virt.subtests.cfg.sample: Fix tcpdump commands
The same mechanism to verify background activity is employed
in several tests, we forgot to apply the correction in several
tests. This commit follows up b94fe88b9b80022c32c1e702a171b6d90c62abf1
and should fix random problems with 'pgrep tcpdump' in
many tests.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
d1f7f3f
@vinsonlee vinsonlee virt: Use pre-python-2.7 TestCase methods in unit tests.
This patch substitutes out TestCase methods that were introduced with
Python 2.7 with TestCase.assertTrue.

Fixes unittest_suite failures on Ubuntu 10.04 (Python 2.6.5).

Signed-off-by: Vinson Lee <vlee@twitter.com>
f37ad03
@lmr lmr Revert "tests.cfg.sample: change import order"
This reverts commit e64b17d7a15602db0cd26ec55ccc902010985d0c,
as it's causing problems with the test execution order.

Signed-off-by: Lucas Meneghel Rodrigues
6e4b5cf
@lmr lmr virt.guest-hw.cfg.sample: Override smp settings for cgroups
It's a bad workaround, but we'll have to do it for now

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
0dfeeeb
@vinsonlee vinsonlee server.monitors_utils_unittest: Used named constant for O_NONBLOCK
Signed-off-by: Vinson Lee <vlee@twitter.com>
0c27cbe
guyanhua Add four test cases for "virsh uri" command
This patch adds four test cases for "virsh uri" command.

Use four cases:(1) Call virsh uri
               (2) Call virsh -c remote_uri uri
               (3) Call virsh uri with an unexpected option
               (4) Call virsh uri with libvirtd service stop

Signed-off-by: Gu Yanhua <guyanhua-fnst@cn.fujitsu.com>
db5b57a
guyanhua Add configuration for "virsh uri" test
This adds configuration for "virsh uri" test.

Signed-off-by: Gu Yanhua <guyanhua-fnst@cn.fujitsu.com>
c218d3f
@vinsonlee vinsonlee utils/run_pylint.py: Fix exception with disable-msg-cat option.
This patch fixes an exception with the disable-msg-cat command line
option on older versions of Pylint. The command line option expects just
one letter and not an entire string.

Tested that it fixes the Pylint 0.19 crash on Ubuntu 10.04. The
disable-msg-cat option is no longer available with Pylint 0.21.1 on
Ubuntu 10.10.

There's a bug report and some history at
https://bugzilla.redhat.com/show_bug.cgi?id=500272.

Signed-off-by: Vinson Lee <vlee@twitter.com>
f3785db
@lmr lmr virt.scripts.ntttcp.au3: Fixing script permission
So pylint will stop complaining.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
e7dff42
@vinsonlee vinsonlee utils/run_pylint.py: Do not stop Pylint after checking first file.
pylint.lint.Run calls sys.exit when the function completes. Starting
with Pylint 0.20.0, Run can take an additional argument to not exit.
This will allow run_pylint.py to continue checking other files.

Signed-off-by: Vinson Lee <vlee@twitter.com>
fc85ad9
@cevich cevich virt: Add RHEL 5.8 definition
Signed-off-by: Chris Evich <cevich@redhat.com>
60517ba
@lmr lmr libvirt.virsh_uri: Fixes to the virsh_uri libvirt test
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
5141410
@vinsonlee vinsonlee common_lib.base_utils: Replace is_alive with isAlive.
threading.is_alive is only available starting with Python 2.6.

Fixes asynctest on Centos 5.7 (Python 2.4.3).

Signed-off-by: Vinson Lee <vlee@twitter.com>
d03dd09
@kongove kongove virt: replace is_alive with isAlive as commit d03dd090
Check the whole autotest repo by grep, fix rest problem.

threading.is_alive is only available starting with Python 2.6.

Signed-off-by: Amos Kong <akong@redhat.com>
4470bea
@zhouqt zhouqt client.fio: Update fio from 1.16.5 to 2.0.5
Since 1.16.5 version of fio can't be compiled on Fedora16 and
RHEL6 boxes, update this fio package to 2.0.5.

Signed-off-by: Qingtang Zhou <qzhou@redhat.com>
c625fb3
@lmr lmr virt.tests.netstress_kill_guest: Workaround udev writing to serial
On some Linux guests such as RHEL 6.2, it's common that when
the succession of rmmods and modprobes happen, udev, SELinux
and other platform software write messages directly to the
serial, confusing aexpect. So let's write some code to
work around this peculiarity, being more robust in handling
these situations.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
1711f14
@lmr lmr virt.tests.ethtool: Improving logging and fixing loop execution
Make sure we go through all tests available and record the failures.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
38c9177
Guannan Ren client tests: Introduce libvirt_tck wrapper
    This is an autotest wrapper based on the released
    TCK version 0.1.0.

     *Sys-Virt-TCK-v0.1.0.tar.gz  libvirt-tck tarball
     *control
     *default.cfg
     *ks.cfg
     *libvirt_tck.py
     *testcase.cfg

Signed-off-by: Guannan Ren <gren@redhat.com>
ba4b748
@ldoktor ldoktor tests.cgroup: Fix cfg parameters
This patch solves the problem with smp parameter.
* cpu_cfs_util - test creates the first VM instead of preprocess
* cpu_share - 'cgroup_use_max_smp' cfg parameter let test calculate smp
* cpuset_cpus - when cpusets is defined it calculates smp from it. Otherwise
                user can force test to set smp by cgroup_use_half_smp.
                When nothing is specified it tries to run on current VM(s).

Signed-off-by: Lukas Doktor <ldoktor@redhat.com>
33e4baf
@lmr lmr client.tests.cpu_hotplug: Fix test execution and parsing
As it was, the cpu_hotplug test was executing the lhcs
suite main entry point and checking its return value.
However, the suite was not well structured, and it will
return 0 even if the subtests failed.

So worked on the suite itself and created a patch
that makes it comply with the rules stablished by
the testsuite author:

http://cpansearch.perl.org/src/BRYCE/Test-Parser-1.3/lib/Test/Parser/LHCS.pm

Also, made the autotest wrapper to parse correctly
the test suite output. Now it should execute and
actually report failures during the test execution.

Some of the testcases were disabled, since they
describe a behavior in the userspace program sar
that I can't reproduce with any sar available in
Fedora or RHEL 5-6 to date, and one of the testcases
does require a linux kernel tree for kernel
compilation.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
320cb4d
@lmr lmr kernbench: Bump kernel version used to 3.2.1
The version 2.6.11 does not compile anymore on older
distros, asking for an update.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
35f621b
@LaneWolf LaneWolf Disable drive_cache for xen hvm installation
Commit b6bb71130 fix using drive_cache option. However, this is not
supported by xen qemu and so breaks xen hvm installation.

This patch unset drive_cache so it is not used for hvm installation.

Signed-off-by: Miroslav Rezanina <mrezanin@redhat.com>
b65d1d7
@vinsonlee vinsonlee common_lib.base_utils: Fix spelling errors.
Signed-off-by: Vinson Lee <vlee@twitter.com>
a857301
@ypu ypu virt test: Make cpus check pattern configurable
The cpus info output from monitor is different between human
monitor and qmp monitor, so make it configurable to fit different
cases.

changes from v1:
  - typo fix
  - Add default value for the pattern. Use the pattern for human
    monitor as default
  - Add comments in base.cfg.sample for the value of qmp monitor

Signed-off-by: Yiqiao Pu <ypu@redhat.com>
2e83e5d
@vinsonlee vinsonlee memory_api: Do not recreate the temporary test directory.
The directory has already been created by base_test.

Fixes OSError exception.

Signed-off-by: Vinson Lee <vlee@twitter.com>
84a5270
@leiyang leiyang dhclient failed in nic_bonding cases V3
Before dhclient, we need to check if it has been running,and then kill it

Signed-off-by: Lei Yang <yanglei.fage@gmail.com>
38306d1
guyanhua Add three test cases for "virsh capabilities" command
This patch adds three test cases for "virsh capabilities" command.

Use three cases:(1) Call virsh capabilities
                 (2) Call virsh capabilities with an unexpected option
                 (3) Call virsh capabilities with libvirtd service stop

Signed-off-by: Gu Yanhua <guyanhua-fnst@cn.fujitsu.com>
5495f55
@lmr lmr Add configuration for "virsh capabilities" test
This adds configuration for "virsh capabilities" test.

Signed-off-by: Gu Yanhua <guyanhua-fnst@cn.fujitsu.com>
51f09b3
@vinsonlee vinsonlee deps: Update stale tarball links.
Signed-off-by: Vinson Lee <vlee@twitter.com>
9b1de21
Nishanth Aravamudan apache: avoid globally modifying /media serving
When attempting to serve a sles11-sp2 install from cobbler on the same
host as autotest, I got 404s for

http://<server>/cblr/links/sles11-sp2-suseboot-ppc64/media.1

with httpd complaining:

File does not exist: x/usr/lib/python2.7/site-packages/django/contrib/admin/media/.1

This is because of the django directives from autotest being global to
apache's configuration, so they affect more than just autotest's
installation. Fix this by pushing the RewriteRules under the Location
tag.

Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>
a9d1f29
Guannan Ren client.tests: Introduce libvirt-test-api wrapper
Introduce a wrapper for the libvirt_test_api. This
testsuite is a functional suite for libvirt, that
makes use of the python bindings available.

Some base config files are provided so the user
can tweak appropriately before running the test.
The tests currently cover:

 * domain: all classical lifetime operations, installation of
various guests OSes, snapshots
 * interfaces: define, create, destroy, undefine, NPIV
 * virtual networks: define, create, destroy, undefine
 * storage: regression tests for most storage types and
configurations dir, disk, netfs, iSCSI, multipath

Signed-off-by: Guannan Ren <gren@redhat.com>
c915fa8
@ypu ypu virt test: Add qemu-io class
Add qemu-io class to provide interface for tests to execute qemu-io
command.
There are two kinds of qemu-io execute way:
  - QemuIOShellSession: Provide a shell session to execute qemu-io
    operates continuously
  - QemuIOSystem: Execute qemu-io command which will return
    immediately. Mainly for qemu-io -c, -h and -V

changes from v1:
  - typo fix
changes from v2:
  - Add function get_cmd_line to reduce the reduplicate code
  - remove useless lines
  - Use qemu_io_binary to get qemu_io from params

Signed-off-by: Yiqiao Pu <ypu@redhat.com>
05e2bdb
@ypu ypu KVM test: Add qemu-io blkdebug test
This test will check if the error output from qemu-io is fit the
blkdebug.cfg. Steps as following:
    1. Create image with given parameters
    2. Write the blkdebug config file
    3. Try to do operate in image with qemu-io and get the error
       message
    4. Get the error message from perror by error number set in
       config file
    5. Compare the error message

If you want to make up your own qemu-io blkdebug test with different
error event. You can modify these parameters:
  err_event: the error event will write in blkdebug.cfg
  err_command: the command running in qemu-io which will trigger
the error event

changes from v1:
  - Use ConfigParser to generate the blkdebug.cfg file from the
changes from v2:
  - Update the interface
  - Remove the useless line and fix and make the variable name
    more clear

Signed-off-by: Yiqiao Pu <ypu@redhat.com>
34583f5
lmr and others added some commits Apr 11, 2012
@lmr lmr tko: Remove empty file create-db-postgres
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
dfd51c8
Nishanth Aravamudan server/install_host: synchronize dhcp after setting netboot_enabled
Recent Cobbler (as of 741faeafc4782ee66d7d1934a29b107602f1ebe9 in the
Cobbler git tree) only writes dhcpd.conf entries for netboot-enabled
systems. When we mark a system as netboot_enabled, therefore, we need to
also ensure dhcpd is synchronized, and there is a new API for this
purpose.

Signed-off-by: Nishanth Aravamudan <nacc@us.ibm.com>
d6d6333
@lmr lmr virt: RHEL-6-series.ks - Add coreutils and usbutils
Those packages are needed for some of the new tests
added to the kvm test set.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
d979748
@lmr lmr client job: Instantiate job.bootloader lazily
Only instantiate job.bootloader to a usable value
if it's really needed (ie, we want to reboot the
machine), since now we depend on a C tool to do
that and the compilation of this tool might fail.

So only throw errors if something wrong happens
there, so we don't have to punish users that only
want to run some tests with the autotest client.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
267bebd
@clebergnu clebergnu contrib/install-autotest-server.sh: make the script more modularized
The encapsulation of the script code into functions will allow one
to chose specific installation actions, such as just installing packages,
and may help the code reuse between the script itself and %pre/%post
scripts on RPM packages.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
1834f7b
@clebergnu clebergnu contrib/install-autotest-server.sh: install packages all packages at …
…once

This should improve the install time a little, as YUM won't have to do all
the checking (including contacting repos, cheking depencies) multiple times.

Still, the packages are still separated into different groups, so it's easy
to select only a subset of them if necessary.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
f2386a6
@clebergnu clebergnu contrib/install-autotest-server.sh: only play with iptables config if…
… it exists

I've observed that minimal installations might lack the iptables configuration
file, so add a check.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
525b6ee
@clebergnu clebergnu contrib/install-autotest-server.sh: better filter libvirt's default n…
…etwork address

If installing a VM with an autotest server using libvirt's default network
address assigment, if the aquired address starts with '1', then it'd not
be shown on the installation status message.

So, let's filter out just the address assigned to the host interface.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
c8a0eaa
@clebergnu clebergnu contrib/install-autotest-server.sh: allow to short-circuit the instal…
…lation

This first commit allows one to install the necessary packages on a base
system. Suitable for generating and keeping a base autotest installation
image, handy for development.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
d438ccd
@clebergnu clebergnu contrib/install-autotest-server.sh: check and skip package installation
We can quickly detect if packages are already installed. If desired and
installed counters match, skip installation.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
eda7525
@lmr lmr boottool: Instantiate job.bootloader lazily take 2
We did find out that the job.boottool attribute was
more pervasively used than we originally thought, so
267bebd93546d43a0900052742767dad941e7ed2 did resolve
the problem for users of autotest local, but broke
use for any code that involved kernel code. So instead
of treating all the places that might use job.bootloader
methods, make boottool.boottool a child class that will
initialize the grubby compilation on demand, only
if one of the methods of it are actually accessed.

With this, we resolve the bug that makes any job that
installs a kernel to fail, since self.job.bootloader
might be None currently.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
f02ff5a
@lmr lmr subtests.cfg: Make boot_with_usb and reboot_with_usb Linux only
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
ef54a7a
@lmr lmr boottool: Fix __getattribute__ acessor
The shorthand __getattr__ can't be used.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
6f2605a
@clebergnu clebergnu boottool: optmize implementation of get_entries()
Now 'grubby --info=<arg>' should be called just once, instead of
once for every entry.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
19b8a3b
@clebergnu clebergnu boottool: fix misleading error message
When running --info without parameters, a misleading error message
was printed out. Replace it with a proper message.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
2261887
@clebergnu clebergnu boottool: made parsing of grubby output (entries) more resilient
Signed-off-by: Cleber Rosa <crosa@redhat.com>
1308a0f
@clebergnu clebergnu boottool: make warnings show up on default logging level
This changes the default logging level, so that warnings also show
up by default.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
7f70135
@clebergnu clebergnu boottool: warn on usage of not ideal version of grubby
Since boottool can use the grubby version that is already installed
if the user explicitly chooses it, we should warn if that version
is not guaranteed to work.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
697b005
@clebergnu clebergnu boottool: make grubby auto installation more robust
This adds more granular checks to the automatic grubby installation,
and also allows grubby to be installed under '/tmp', so that
unpriviledged users can, to the extent that grubby and the boot
configuration files themselves allow, also use boottool.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
3b77709
@clebergnu clebergnu boottool: made info action implementation rely on entries instead of …
…lines

This reuses the get_entry()/get_entries() methods to obtain information
in a better format (dicts instead of lines of text).

Also, by switching to these methods, 'grubby --info=<param>' is called
only once.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
df6876b
@lmr lmr boottool: Fix boot_once unittest
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
259ebdd
@cevich cevich utils: make_clean temporary files and env
Signed-off-by: Chris Evich <cevich@redhat.com>
91e6220
@clebergnu clebergnu database: rebase initial schema to latest version
As planned and discused on github issue #266.

Also, do not manually set the database version,
as the schema file already contains the migrate_info
table and value.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
3e6016e
@clebergnu clebergnu boottool: only print info on grubby install if it's really performed
Signed-off-by: Cleber Rosa <crosa@redhat.com>
94697a1
@clebergnu clebergnu boottool: reduce verbosity level
By removing debug messages on every single command execution. Instead
log messages as errors if they occur.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
89aba82
Lubos Kocman kvm_vm.py: SPICE add_spice() -> set_value() set spice property only i…
…f it's defined
8ff0644
Lubos Kocman kvm_vm.py: add get_spice_var() getter for spice properties
It may be used retrieve spice port,tls-port generated by find_free_port()
99db8f0
@clebergnu clebergnu utils/check_patch.py: remove curl requirement
By using urllib instead.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
9010d7a
@clebergnu clebergnu utils/check_patch.py: run reident without requiring client/util in $PATH
Signed-off-by: Cleber Rosa <crosa@redhat.com>
fb59eea
@clebergnu clebergnu utils/check_patch.py: run pylint without requiring client/util in $PATH
Signed-off-by: Cleber Rosa <crosa@redhat.com>
b32000a
@clebergnu clebergnu utils/check_patch.py: fix typo
Signed-off-by: Cleber Rosa <crosa@redhat.com>
4b7fcf6
Philipp Seiler conmux: Better integration in linux environment
- made makefile more generic to install for example to /usr/lib64
- added search path for conmux to /usr/lib and /usr/lib64
- added lsb header to init script
- add shebang line to expect file in drivers

Signed-off-by: Philipp Seiler <seiler@b1-systems.de>
6b7c9bc
@lmr lmr virt: Restrict dd_test to Linux variants
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
ee8d51a
@lmr lmr virt.trans_hugepage_swapping: fix code cleanup
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
9793d41
root libvirt: Use hvm_or_pv for kvm too
Unlike RHEL6, RHEL5 virt-install uses non-kvm qemu as default option. To use
kvm virtualization, --accelerate option has to be used. Libvirt uses hvm_or_pv
option to add this necessary option. However, it's not set for kvm linux guests
as they're not needed for RHEL6.

This patch adds this option to the linux kvm guest tests so RHEL5 is using
kvm virtualization as expected.

Signed-off-by: Miroslav Rezanina <mrezanin@redhat.com>
ec6278e
root virt: libvirt: Allow to choose method of passing ks into the guest
Old version of virt-install do not allow to assigne two cdroms to a guest.
This prevent unattended_install as we pass ks in secondary cdrom.

This patch add possibility to choose way of passing ks - extra cdrom,
integrated into the install cdrom, using internall http server or using floppy.
All used methods are already implemented but it's not possible to select
which one will be used.

Signed-off-by: Miroslav Rezanina <mrezanin@redhat.com>
7876939
Jiří Župka kvm test cpuflags: Try all supported cpu models.
If cpu model isn't defined then test compare all
guest cpu models with host cpu model and try test
all cpu models which is supported by host.

Signed-off-by: Jiří Župka <jzupka@redhat.com>
2529eef
@kongove kongove virt: Only add machine type parameters when qemu supports this option
qemu-upstream has "-machine" option, "-M" is an alias.
qemu-kvm-rhel6 only has "-M" option.

So add "-M .." parameters when "-machine" or "-M" exists.

Signed-off-by: Amos Kong <akong@redhat.com>
d81707c
@lmr lmr virt: add "--machine ?" parameter in libvirt command line wrappers
--machine MACHINE :set the machine type to emulate.
add libvirt command line parameter "--machine ?" wrapper in
libvirt_vm.py.

Signed-off-by: Qing Lin <qinglbj@linux.vnet.ibm.com>
a96e6ef
@lmr lmr virt: trans_hugepage_swapping: Fix test to use current APIs
Test was using a reference to an old API that no longer
exists in virt_utils. Instead, let's use the current
methods to grab a VM class (methods of the env object).

Also, use up to date methods to get an ssh session to the vm.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
afefc5b
@lmr lmr API: client: Drop bin directory
This patch introduces API changes for test writers

Drop the client/bin directory to make autotest
more palatable for distro packagers, also making
the API more flat. Now all uses of

autotest_lib.client.bin.[library]

become

autotest_lib.client.[library]

The conversion was mostly automated, plus some
tweaks to make autotest-local work, and moving
profilers.py code to the __init__.py of the profilers
directory.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
6471010
@lmr lmr API: Moving autotest_lib to autotest
This patch brings API changes to test writers.

Move autotest_api to a shorter and cleaner
autotest. So all references to

autotest_lib.[lib]

Become

autotest.[lib]

The conversion was made mostly automated by
script.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
71a6a91
@lmr lmr API: Move common_lib to shared
This patch brings API changes to test writers.

Move common_lib to the cleaner and shorter shared,
since that is a shared library space among all
autotest modules. So all references to

autotest.client.common_lib.[lib]

become

autotest.client.shared.[lib]

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
e3ae189
@lmr lmr unittest_suite: Add another file to blacklist
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
1d31f16
@lmr lmr utils.run_pylint: Remove the need for custom import filters
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
b65a151
@lmr lmr client.setup_modules: Handle client stand alone execution
The setup_modules handling for stand alone client wasn't
being considered in the previous patches, it was broken.
With this patches, executing the client stand alone should
work again.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
ba5d712
@lmr lmr server: Fix autotest install
As we had to move client/bin/profilers.py ->
client/profilers/__init__.py, our packaging install code
for autotest was breaking client execution, since profilers
is not really copied to the client. Fix this by copying
only the __init__.py to the appropriate location.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
4f93af0
@lmr lmr virt: Fix autotest execution in guests
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
52646b9
@lmr lmr client.setup.py: Remove inexistent bin pkg
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
fe704df
@lmr lmr virt: Move file_transfer to virt_test_utils
Since 4 different tests were using it, it seemed an
obvious place to move it to. Besides, the new import
dynamics was causing problems to execute it from
inside its source file.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
f484ea0
@lmr lmr server: Fix get_install_dir
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
4285854
@lmr lmr client: Prunning unused imports
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
5927ce0
@lmr lmr server: Prunning unused imports
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
40e2834
@lmr lmr scheduler: Prunning unused imports
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
afe5fa4
@lmr lmr client.base_utils: Silence file_contains_pattern
So we get a little less debug output that is not really
necessary.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
8fe2973
@lmr lmr utils.get_os_vendor: Update Fedora detection
Update ID string from 'Fedora Core' to just 'Fedora',
as the former definition is, um, kinda old.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
867c4af
@lmr lmr client.base_utils.get_os_vendor: Debug print OS vendor
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
fface28
@lmr lmr Fixing setup_modules_unittest
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
31f8137
@lmr lmr client: Fix setup_job_unittest
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
f96a50c
@lmr lmr database: Fix afe_sql db backend
python-sqlite2 was removed from Fedora, now there's the
sqlite3 module that can be used, however it has a slightly
different interface. So handle import errors and object
initialization errors accordingly so one can run db related
unittests again on F17.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
aeae793
@kongove kongove perf: refactor regression.py and add t-test
scipy: http://www.scipy.org/
t-test: http://en.wikipedia.org/wiki/Student's_t-test

scipy and numpy modules are needed.

Use unpaired T-test to compare two samples, user can check
p-value to know if regression bug exists.

Verified by this page: http://graphpad.com/quickcalcs/ttest1.cfm

If the difference of two samples is considered to be not
statistically significant(p <= 0.05), it will add a '+' or '-'
before p-value.
  '+': avg_sample1 < avg_sample2
  '-': avg_sample1 > avg_sample2

The analysis module doesn't care about what's tested,
what's the relationship of each cols/lines.
Currently I describe the sample relationship of cols in perf.conf
This is used to update col results after computing averages.

eg.
| [netperf]
| update_avg = 4,2,3|14,5,12|15,6,13

"normalize"   = $throughput / $cpu
"#tpkt/#exit" = $tx-pkts    / $io_exit
"#rpkt/#irq"  = $rx-pkts    / $irq_inj

Then col 4(normalize),14(#tpkt/#exit),15(#rpkt/#irq) will be re-computed.

Signed-off-by: Amos Kong <akong@redhat.com>
4ed707f
@kongove kongove use paired t-test to compute significance of all averages
The paired t-test is computed for averages of same category.
For netperf, we need to compute three paired t-tests for
TCP_RR, TCP_STREAM, TCP_MAERTS.

Code was verified by online tool:
 http://graphpad.com/quickcalcs/ttest1.cfm

Signed-off-by: Amos Kong <akong@redhat.com>
0143505
@kongove kongove add the raw data that the averages are based on in another file
Users need to reference raw data in analyzing,
so output the raw data and averages to another file.

Signed-off-by: Amos Kong <akong@redhat.com>
07d7587
@kongove kongove display perf result by HTML file
Use HTML syntax to display results by beautiful tables.

Signed-off-by: Amos Kong <akong@redhat.com>
104e3c1
@kongove kongove netperf: write version/description in result
fix tiles:
 - Change cpu to %CPU
 - Change normalize to thr/%CPU

write version/description infomation in result file.

Signed-off-by: Amos Kong <akong@redhat.com>
6dbd6ce
@kongove kongove add version info and case description in result
Collect version info and case description from raw results,
add them in main result files.

Signed-off-by: Amos Kong <akong@redhat.com>
52a68e6
@kongove kongove ntttcp: write version/description in result file
Signed-off-by: Amos Kong <akong@redhat.com>
e2d615a
@lmr lmr tools.regression.py: Add shebang and executable permissions
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
05af063
@clebergnu clebergnu server/kvm.py: remove legacy KVM support
The server side API for KVM has been unmaintaned for ages, and targets
very old versions of KVM. Instead of carrying code that is not relevant
or may not work at all, clean it up.

People interested in KVM testing should look at the better maintained
'kvm' test in client/tests.

If there's demand for pure server side KVM test support, we can consider
creating server side code using the client virt/kvm libraries.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
b9575d3
@lmr lmr cient/virt/virt_utils.py: Introduce KojiDirIndexParser
KojiDirIndexParser is a parser for HTML directory index pages,
specialized to look for RPM links.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
4759580
@clebergnu clebergnu client/virt/virt_utils.py: Introduce RPMFileNameInfo
RPMFileNameInfo is a simple parser for RPM based on information
present on the filename itself.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
2f0ed14
@lmr lmr client/virt/virt_utils.py: Introduce KojiScratchPkgSpec
KojiScratchPkgSpec is a package specification syntax parser for Koji
scratch builds.

This also introduces changes to KojiClient to support downloading
packages from a scratch build.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
7a47f52
@clebergnu clebergnu virt installer: add support for koji scratch builds
Signed-off-by: Cleber Rosa <crosa@redhat.com>
99554e7
@ldoktor ldoktor kvm tests: Update cdrom test
This patch adds the check for tray status and fixes some bugs. Also it
changes the VM handling. The VM restart is now necessarily only when
the test fails.

* tray-open status subtest
* change vm handling
* absolute paths of cdrom locations
* workaround locked cdrom
* workaround eject time

Signed-off-by: Lukas Doktor <ldoktor@redhat.com>
8b2c468
Jiří Župka virt multihost_mig: Repairs bug in starting sequence of tests.
Signed-off-by: Jiří Župka <jzupka@redhat.com>
4447888
Jiří Župka virt: Adds migration over fd for kvm.
Migration over fd:
  source:
    1) Make new descriptor (pipe, socket) and send the descriptor
       to qemu-monitor over unix socket.
    2) Register the descriptor in qemu-monitor with function
          getfd DSC_NAME
       and close the descriptor in main process.
    3) Migrate over descriptor.
          migrate fd:DSC_NAME

  destination:
    1) Start child process with open second side of source descriptor.
    2) Start machine with param -incoming fd:descriptor
    3) Wait for finishing migration.

Signed-off-by: Jiří Župka <jzupka@redhat.com>
056cd55
Jiří Župka kvm-test: Add support for multihost migration over file descriptor.
Test create socket get descriptor from socket and migrate through
the descriptor.

This test allow migration only of one machine at once.

Signed-off-by: Jiří Župka <jzupka@redhat.com>
239e7af
@clebergnu clebergnu client/tools/regression.py: usage message fix
Remove the need for the interpreter, since the script has been made
a proper executable (see 05af063b39f50cc2f5b398365e59bb226b829ad7).

Also, use more standard syntax for denoting mandatory arguments.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
55fc21d
@lmr lmr virt: move image utility functions from virt_vm to virt_utils
This allows us to break the circular dependency that currently exists
in virt_vm.py/virt_utils.py.

Reasoning and longer explanation: while testing a script that uses some
features of virt_utils.py, it was noticed that a gstreamer
initialization
method was called. That seemed to be unnecessary given that the video
generation code lives in a separate module. A bit more investigation
showed that virt_utils.py imported both virt_vm.py and
virt_env_process.py.

This patch removes the circular dependency for virt_vm.py only. The
dependency on virt_env_process.py is a bit more tricky, but will be
done.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
5018fa7
@lmr lmr client: Removing more unused imports
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
4ae4418
@lmr lmr client.shared: Remove barrier_unittest.py
Since we have an identical base_barrier_unittest.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
4dfe244
@lmr lmr rename unittests ending in _test.py
Let's standardize that unittests do end in _unittest.py,
to avoid mistakes with test case files. In this process,
cleaned up the code of utils/unittest_suite, so we
don't have to resort to SKIP hacks and similar stuff.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
c3c7ca4
@cevich cevich commented on an outdated diff Apr 23, 2012
client/virt/base.cfg.sample
-# Less common options
-# image compression opts (auto_glz, auto_lz, quic, glz, lz, off)
-spice_image_compression = auto_glz
-# jpeg wan compression opts (auto, never, always)
-spice_jpeg_wan_compression = auto
-# zlib-glz wan compression opts (auto, never, always)
-spice_zlib_glz_wan_compression = auto
-# streaming-video opts (off, all, filter)
-spice_streaming_video = all
-# agent mouse opts (on, off)
-spice_agent_mouse = on
-# playback compression opts (on, off)
-spice_playback_compression = on
-spice_ipv4 = yes
-spice_ipv6 = no
-
@cevich
Autotest member
cevich added a note Apr 23, 2012

Note: Not sure where the above came in from (maybe a rebase). Need help confirming all the above settings should have been removed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@cevich cevich commented on an outdated diff Apr 23, 2012
client/virt/virt_utils.py
- ('%s req -new -key %s -out %s/server-key.csr -subj "%s"' %
- (ssl_cmd, path + server_key, path, server_subj)),
- ('%s x509 -req -passin pass:%s -days %d -in %sserver-key.csr -CA '
- '%sca-cert.pem -CAkey %sca-key.pem -set_serial 01 -out %sserver-cert.pem' %
- (ssl_cmd, passphrase, days, path, path, path, path))
- ]
-
- if not secure:
- cmd_set.append('%s rsa -in %s -out %sserver-key.pem' %
- (ssl_cmd, path + server_key, path))
-
- for cmd in cmd_set:
- utils.run(cmd)
- logging.info(cmd)
-
-
@cevich
Autotest member
cevich added a note Apr 23, 2012

Note: Not sure where the above came in from (maybe a rebase). Need help confirming all the above code should have been removed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
clebergnu and others added some commits Apr 23, 2012
@clebergnu clebergnu server: remove hypervisor/guest support
These classes were used solely by the now deprecated and removed
kvm.py server module (see #b9575d3f7957b8f1aa85c5ef54c9f3a08d67f99b).

Unless other folks are using this hypervisor/guest interface in a
site specific way, I think they should be removed to avoid confusion.

Again, if there's demand for pure server side virt code, we can and
should leverage client/virt code.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
5c2a911
Lubos Kocman client.virt.aexpect: Added ok_status=[0,] into aexpect.cmd()
* allows user to specify exitcodes that will pass ok except of
throwing ShellCmdException

Signed-off-by: Lubos Kocman <lkocman@redhat.com>
79a5d0e
@zhouqt zhouqt client.shared: iso9660.py: Check extensions in iso image
Some images (eg. RHEL3.9/4.9, Ubuntu11.10) contain
Joliet extension, and others (FreeBSD9.0, etc) has
Rock Ridge, these images need special options to
get the correct file name. This patch add a method
in Iso9660IsoInfo class to check these extensions.

If isoinfo fails to check extentions, another
method will get the file name from isoinfo's
output.

For these extensions, please see:
http://en.wikipedia.org/wiki/ISO_9660#Extensions

Signed-off-by: Qingtang Zhou <qzhou@redhat.com>
6294a6a
@zhouqt zhouqt virt: unattended_install.py: Make sure guest kernel/initrd file is co…
…rrectly copied

Add an assert() to make sure guest kernel/initrd file are dumped
correctly from iso image.

Signed-off-by: Qingtang Zhou <qzhou@redhat.com>
37cf810
@kongove kongove and 1 other commented on an outdated diff Apr 24, 2012
client/virt/base.cfg.sample
@@ -73,18 +98,13 @@ drive_cache = none
monitors = humanmonitor1
# Default monitor type (protocol), if multiple types to be used
monitor_type = human
-# Pattern to get vcpu threads from monitor.
-# human monitor: thread_id=(\d+)
-# qmp monitor: u'thread_id':\s+(\d+)
-vcpu_thread_pattern = "thread_id=(\d+)"
@kongove
Autotest member
kongove added a note Apr 24, 2012

The default value of vcpu_thread_pattern is "thread_id=(\d+)" in code,
this block can help user know where/how to setup this parameter.
Do we need to describe this kind of parameters in document (effective scope, example, etc)

Or we suppose user would read the code when they touch some problem, then update parameters the config file.

@cevich
Autotest member
cevich added a note Apr 24, 2012

Ohh, wait a sec, this may be another section similar to my notes (below) where it got removed as part of my rebase. As below, we need to double-check this stuff is suppose to be removed (i.e. b/c of a prior commit) or if I screwed up my rebase (entirely possible). Thanks for pointing it out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@cevich cevich virt: Rewrite network/nic management
+ Integrated new network code with kvm module
+ Added many logging and debugging statements for new code
+ unit tests: New virt_utils_unittests tests
+ Updated shared ping test module
+ Updated kvm nic_bonding test module
+ Updated kvm nic_hotplug test module comment
+ virt_utils Env init now always warns when creating an empty env file
+ virt_utils Env init prints warning traceback when loading env fails
+ Pickling support for all new classes
+ Raise exception when adding nic with clashing name
+ New virt_utils_unittest deletes temporary testing database
+ IP/MAC cache is only updated with new entries
+ VM cleanup resets static addresses properly
+ Enhanced output of needs_restart() VM comparison function
+ Enhanced Env class to output more details on non-fatal load failures
+ Old virt_util functions all raise depreciation warnings
  to aide bug squashing
+ remove unnecessary nic_mode from subtests parameter, rely on default
    nettype=bridge instead
+ virt config: New universal virt. networking configuration
        nettype + netdst parameters determine networking style and
        destination for virtual NICs.  Separate NIC sets can be defined
        for each VM.  Documented examples of other per-nic parameters
        in config. comments.
+ virt_utils change etails:

New Class: PropCan
    Allow container-like access to fixed set of items (__slots__)
    This allows NIC/Net handling code to lock-down a fixed set of
    parameters relevant to a vm_type / driver_type combination.
    Data is accessible in a dictionary-like manner or as object
    properties, however empty or None value items will pretend to
    not exist.  The special empty/None handling optimises access
    and storage by only representing values containing meaningful
    data.

New Class: Base_VirtIface
    Networking information for single guest interface and host
    connection.  This is an abstract base-child-class of PropCan
    containing the minimal amount of networking keys needed, along
    with some basic name and mac address handling methods.  Some
    amount of effort is made to ensure stored mac address data
    is in an expected format - two ASCII character hex-byte
    representation, separated by :'s, six or less bytes in total.
    Complete mac addresses are generated by filling in missing
    trailing bytes from a sequential pool maintained as a class
    variable.  Numbering starts randomly from 0-255, then proceeds
    incrementally for each byte requested.  Sequential byte generation
    used to ease unit testing.

New Class: Libvirt_VirtIface & KVM_VirtIface
    Specializations of Base_VirtIface parent class (and PropCan)
    with networking keys specific to Libvirt or KVM respectively.
    If a new type or variation is needed, simply subclass Base_VirtIface
    and update the VMNet_Style.VMNet_Style_Map as needed.  The new or
    variant class will be used automatically based on resolving
    vm_type, and/or driver_type from the supplied Params dict (see
    other classes below)

New Class: VMNet
    Abstract base-class collection of networking information, stored
    within a particular container class (e.g. a subclass of
    Base_VirtIface). This class behaves like a list of PropCans.
    It may be initially empty or created from a list-like container
    of dictionary-like objects holding key/value pairs of networking
    data.  Items within a VMNet instance are indexed by number and
    the 'nic_name' value of each contained item.  Consequently, the
    'nic_name' key is required for each contained item, and it's value
    must be hashable. Updates to the 'mac' value of each item is handled
    specially, in that storage of incomplete byte lists, and/or values
    that do not pass the container class's mac_is_valid method,
    will not be stored.

New Class: VMNet_Style
    Make decisions about needed info from vm_type and driver_type params.
    The VMNet_Style_Map class variable is utilized for initial setup of
    VMNet derivatives (see below).  Essentially it helps provide a
    container-class type based on the contents of Params dict-like
    values for vm_type and driver_type.  It also establishes a mac
    address prefix to be used when generating MAC addresses.  However,
    any number of additional VMNet instance variables may also be set
    with default values.  The special 'default' keys are utilized when
    vm_type and/or driver_type don't match from Params.

New Class: ParamsNet
    VMNet sub-class specialized for initialization from a Params
    dict-like instance.  Essentially it filters out and initializes
    all networking info. for a particular VM.  Once initialized,
    networking information is accessed by NIC index or it's name (e.g.
    'nic1', 'nic2', etc.) in a dictionary-like or object property-like
    manner.  For example, instance[0].mac, instance('nic1').mac,
    instance['nic1']['mac'], etc.  Each nic object will be stored in
    the proper container class as determined by the VMNet_Style class.
    A utility generator method is provided for iteration over all
    VM's and NIC's mac address params definitions.  This makes validating
    or checking for mac address conflicts easy.

New Class DbNet:
    VMNet sub-class specialized for initialization from a database.
    Essentially analogous to ParamsNet, with a set of database
    management and locking methods.  Database items are hashed by the
    key provided at initialization time, so hash-clashes must be managed
    by implementation code.  Changes to any indexed NIC object
    automatically result in atomic database updates.  This also follows
    for any re-ordering or deletion of contained items.  The lock/unlock
    methods must be called externally prior to any database activity.
    Also provides a utility generator function for iteration over all
    MAC addresses contained within the database.

New Class: VirtNet (Re-write former networking code)
    Multi-inheritance of the ParamsNet and DbNet classes providing
    mediated networking info. management behavior combination of both.
    Provides the former generate_mac_address(), free_mac_address,
    set_mac_address(), and get_mac_address() functions, which behave
    exactly as before.  The only exception is the 'instance' parameter
    is no-longer required as it is established at initialization time.
    Setting and Freeing of arbitrary mac addresses is allowed (as
    before), however generation of new mac addresses is a sieve over
    cached values.  Deleting the database cache or freeing individual
    NICs MAC addresses will extend the pool within the confines of any
    mac prefixes defined by the responsible VMNet_Style definition.  A
    utility generator is provided which wrapps the mac_index generators
    from ParamsNet and DbNet (in that order).  This permits iteration
    over all known MAC address values possible.

+ KVM change details:

    Intended CodePath:

        Base Class __init__:
                Load/Parse Cartesian params + address_pool cache

        Base Class nic addition:
            BaseVM.add_nic
            BaseVM.activate_nic
            <testing>
            BaseVM.deactivate_nic
            BaseVM.del_nic

        KVM VM class __init__:
            Call BaseVM.__init__

        KVM VM create:
            vm.create:
                vm.add_nic -> vm.add_netdev
                specialized nic & netdev activation
            <testing>

        KVM VM nic addition:
            vm.add_netdev -or- vm.add_nic -> vm.add_netdev
                Uses BaseVM.add_nic
            vm.activate_netdev
            vm.activate_nic
            <testing>
            vm.deactivate_netdev
            vm.del_netdev
            vm.deactivate_nic
            vm.del_nic

   Modules Changes summary:

    BaseVM module
        Before:
            Different behavior per sub-class

        After:
            Defined new VMUnknownNetTypeError, raised when validating 'nettype'
            parameter doesn't match supported type.  Supported types left up
            to sub-classes to define.

    BaseVM.init
        Before:
            Subclasses and methods deal with params and cache DB ad hoc
        After:
            Common setup/validate networking from parameters and cache DB input

    BaseVM._generate_unique_id
        Before:
            vm.instance unique across thousands of VM instances created per
            second
        After:
            vm.instance unqie across millions of VM instances created per
            second

    BaseVM.get_mac_address
        Before:
            Obtain mac defined in params or cache DB input.  Raise
            VMMACAddressMissingError if no mac defined (note: doesn't actually
            check vm object in any way).
        After:
            Loading mac from params and/or cache db moved to __init__.  Check
            instance variables for mac and return or raise
            VMMACAddressMissingError

    BaseVM.get_address
        Before:
            Require sub-classes to impliment.
        After:
            Rely on __init__ to parse options and cache DB. If nic has mac
            defined, and corresponding IP in IP/MAC cache, verify in host
            arp cache then return IP.  Return localhost if no
            mac address defined for nic.

    BaseVM.free_mac_address
        Before:
            Different behavior per sub-class
        After:
            Unified behavior to drop mac reference only if they're not
            staticly defined.  When static, the original mac from params
            is re-established for the nic.

    BaseVM.add_nic
        Before:
            Left up to subclasses
        After:
            Adds new or updates existing NIC definition on vm instance with
        new or changed options.  Generates random nic_name if none specified.
        If IP and MAC are defined, updates IP/MAC cache.  Otherwise, generates
        random mac address.  Returns standardized instance of dict-like object
        containing NIC new/existing (possibly changed) parameters.

    BaseVM.del_nic
        Before:
            Left up to subclasses
        After:
            Drop references to mac in cache DB.  Drop references to IP
        from IP/MAC cache.  Ensure NIC parameters removed from vm instance.

    BaseVM.activate_nic
        Before:
            Host resource allocation bundled with sub-class add_nic method.
        After:
            Require sub-classes to implement separate from add_nic.

    BaseVM.deactivate_nic
        Before:
            Host resource deallocation bundled with sub-class add_nic method.
        After:
            Require sub-classes to implement separate from del_nic

    kvm_vm.init
        Before: Track nic-indexed netdev_id, device_id, and tapfd separately.
        After: Moved into BaseVM.__init__, single virt_utils.virtnet class.

    kvm_vm.clone
        unchanged- rely on __init__ to produce new virtnet instance

    kvm_vm.__make_qemu_command
        add_nic: unchanged
        add_net: unchanged
        Body: Referenced names, and assignment-order changes / updates
              so may be called independently of vm.create().

    kvm_vm.create
        Before:
            For each nic- Parse params, generating random netdev_id, device_id.
            For tap a.k.a. bridge devices, obtain ifname, bridge name, tapfd,
                add to bridge, then raise interface
            For each nic- obtain mac or generate random mac
                if params define ip address,update address cache
            __make_qemu_command
            background-run qemu command
            for each open tapfd- close tapfd
        After:
            (Rely on __init__ to parse params and cache db content)
            For each nic, use common add_nic(), but specialize activation:
                if nettype is bridge/tap:
                    open tap device node
                    register tapfd and tapfd_id to qemu
                    add interface to bridge
                    raise interface
                if nettype is user:
                    log info
            __make_qemu_command
            background-run qemu command
            for each already parsed nic:
                if tapfd open, close tapfd

    kvm_vm.destroy
        Same behavior, updated names

    kvm_vm.get_port
        Same behavior, updated names

    kvm_vm.get_peer
        Unchanged

    kvm_vm.get_address
        Before:
            Parses params for IP, if bridge/TAP mode defined for nic, parses
            params for mac address. Consults IP/MAC cache DB for IP.  Returns
            localhost if no IP defined or nic_mode is not 'tap'.
        After:
            Functionality made common by implimentation in BaseVM class.

    kvm_vm.get_ifname
        Before:
            Return the letter 't' followed by the vm's nic index, followed
            by a '-' and the last 11 characters of vm.instance.
        After:
            Retrieve ifname (generated in vm.add_netdev) for NIC or raise
            KeyError.

    kvm_vm.get_mac_address
        Before:
            Duplicated code from old BaseVM method with same name
        After:
            Functionality made common by implimentation in BaseVM class.

    kvm_vm.free_mac_address
        Before:
            Use virt_utils function to drop mac reference from cache DB.
        After:
            Functionality made common by implementation in BaseVM class.

    kvm_vm.add_netdev
        Before:
            Parse params for bridge name, generate random netdev_id, set vlan
            to number of other tap NICs on VM.  Generate ifname, open tap
            device node, set tapfd, bring up interface, generate random
            tapfd_id, run 'netdev_add' monitor command.  Verify netdev_id shows
            up in network info.  Return netdev_id.
        After:
            Rely on BaseVM.__init__ to parse options and cache DB for bridge
            name. Validate supported nettype specified in parameters ('bridge'
            or 'user') Generate random netdev_id, use defined vlan or set based
            on # total of nics on VM., and generate ifname.  Open tap device
            node moved to vm.activate_netdev().  Generate random tapfd_id.
            Running 'netdev_add' moved to vm.activate_netdev().  Return
            netdev_id.  Moved running of 'netdev_add' functionality into
            vm.activate_netdev.

    kvm_vm.del_netdev
        Before:
            Run 'netdev_del' monitor command.  Validate netdev_id no longer
            shows in network info.
        After:
            Moved 'netdev_del' monitor command and validation to
            vm.deactivate_nic().  Remove properties from nic: netdev_id,
            vlan, ifname, tapfd, tapfd_id.

    kvm_vm.add_nic
        Before:
            Take fixed set of nic parameters, generate a name for nic
            (nic_id) if not specified.  Call add_netdev() if netdev_id parameter
            not specified.  Generate random mac address if mac parameter not
            specified.  Setup/Run device_add monitor command with id=nic_id.
            Return dict containing nic parameters.  Verify nic_id shows up
            in qtree.
        After:
            Take arbitrary set of key/value nic prameters.  Call BaseVM.add_nic
            method with nic parameters.  Generate random device_id.  Call
            add_netdev with parameters if no netdev_id specified. Return
            standardized dict-like object containing nic parameters. Moved
            call to device_add command into vm.activate_nic method.

    kvm_vm.del_nic
        Before:
            Runs device_del monitor command.  Checks for wait timeout for
            nic to not show in qtree.
        After:
            Call BaseVM's del_nic method.  Functionality to
            call device_del moved to vm.deactivate_nic

    kvm_vm.activate_netdev
        Before:
            Functionality combined with vm.add_netdev
        After:
            Re-validate nettype is supported, setup netdev_add monitor
            command with parameters from nic based on nettype value. Run
            netdev_add monitor command.  Verify netdev_id shows up in network
            info.

    kvm_vm.deactivate_netdev
        Before:
            Functionality combined with vm.del_netdev
        After:
            Call netdev_del monitor command, verify netdev_id no longer shows
            in network info.

    kvm_vm.activate_nic
        Before:
            Functionality combined into vm.add_nic
        After:
            Setup/run device_add monitor command with id=nic_name.  Verify
            nic_name shows up in qtree.

    kvm_vm.deactivate_nic
        Before:
            Functionality combined with vm.del_nic
        After:
            Run 'device_del' monitor command. Validate netdev_id unknown to
            monitor. Raise virt_vm.VMDelNicError if qtree shows nic_name after
            wait time.

    kvm_vm.migrate
        Unchanged

Signed-off-by: Chris Evich <cevich@redhat.com>
cb89865
@cevich
Autotest member
cevich commented Apr 24, 2012

Okay, I fixed up the accidentally removed sections and rebased again.

@LaneWolf

When will be libvirt_vm update? Without those changes request can't be committed as it will break libvirt testing (at least on xen hypervisor).

@cevich
Autotest member
cevich commented Apr 25, 2012

Yep, working on libvirt now and will be pushing those commits up when they're ready.

cevich added some commits Apr 25, 2012
@cevich cevich fixup! virt: Rewrite network/nic management
Signed-off-by: Chris Evich <cevich@redhat.com>
7edee96
@cevich cevich squash! virt: Rewrite network/nic management
* Implimented new nic/net management within libvirt_vm
* Added stubbs for nic activation/deactivation in libvirt_vm
* Removed extranious netdev_id/device_id/tap/ifname support from libvirt_vm
* Enhanced logging for vm.needs_restart()

Signed-off-by: Chris Evich <cevich@redhat.com>
f3b4025
@cevich
Autotest member
cevich commented Apr 25, 2012

The default install, reboot, shutdown, undefine tests are now working on RHEL 5 host with libvirt/kvm. Still need to test xen, but it should be fine.

@cevich cevich squash! virt: Rewrite network/nic management
* Changed libvirt vm to only free mac addresses for undefined VMs
* Improved libvirt vm.is_persistent() to handle non-existing VMs
* Defined mac address prefix convensions for libvirt (kvm/xen)

Signed-off-by: Chris Evich <cevich@redhat.com>
79ba699
@lmr lmr referenced this pull request Apr 26, 2012
Closed

Virt utils net rewrite #324

@lmr
Autotest member
lmr commented Apr 26, 2012

Opened #324, pull from that branch and resend the pull request, thanks!

@lmr lmr closed this Apr 26, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.