New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Beaker/Autotest Integtration #629

Closed
wants to merge 5 commits into
base: next
from

Conversation

Projects
None yet
3 participants
@lmr
Copy link
Member

lmr commented Apr 12, 2013

This is a pull request with Don Zickus' patches integrating Beaker and Autotest

Hi Folks,

Throwing this over the wall for a new round of reviewing.

Recently a bunch of autotest and beaker folks have been discussing how to get
an autotest client working under beaker (namely Nick, Dan, Bill from beaker and
Lucas, Cleber, Ademar from autotest).

A long time ago Jan Stancek whipped up some code to do that based on some
discussions we both had. I later modified a bunch of his code and came up with
these patches.

I lasted touched these patches back in August 2012. I never finished polishing
them off or completed my testing due to other Red Hat obligations.

I am presenting these patches as a starting point for discussions as to how Jan
and myself solved various problems. They are very much 'beta' patches (there
has been a lot of testing so they were stable at one point).

Unfortunately, I don't remember how all the pieces worked so instead of
breaking the files into digestable chunks, I am just submitting whole files as
new (well the first patch is a 'patch').

Please be kind. I am cross-posting to various mailing lists to let anyone
interested participate.

The end goal is to have autotest client work with the beaker server smoothly.
This would allow various autotest projects to leverage the
provisioning/inventorying mechanisms provided by beaker for various labs.

Feel free to direct all questions to me and I will do my best to remember what
Jan and myself did.

I started running our internal kernel testsuite to get more live coverage of these
changes. Things work for the most part. Reboot still has a bunch of issues,
namely the 'autotest --continue' needs work done. :-(

I add some initial support to handle legacy beaker commands to talk with the
server. I expect to simplify it later.

v2: re-coded to use beaker's new stable API
thinned things out, added support for debug env
more testing
cleaned up coding style and syntax based on Lucas's feedback

v3: cleanup based on Lucas's feedback
reworked file uploading to deal with reboot state
little fixes from more testing

dzickusrh added some commits Apr 13, 2013

cmdparser: add new command bootstrap
Because beaker uses the 'push' method for running tests, there has always
been a catch-22 on how to start autotest after provisioning the box.

Currently, beaker uses a daemon called 'beah' to do all the initial
bootstrapping.  In order to move away from beah we needed another
alternate approach.

Bill P. suggested to use an env variable.  So that is what I did.
When autotest runs from the /etc/init.d on bootup it will look for
'AUTOTEST_HARNESS'.  If that is set, call 'autotest bootstrap'.  The
env will tell autotest to use the harness beaker, download an initial
control and pass that to the 'run' command.  autotest will proceed as normal.

This is kind of an ugly hack but I could not think of another solution to deal
with bootstrapping after provisioning.

v2:  look for harness args in the environment, empty control files silently
     exit autotest (to handle reservesys case)

Signed-off-by: Don Zickus <dzickus@redhat.com>
harness: support for beaker's xml
Beaker passes its recipes through xml files.  This patch just converts the xml
recipe into a python dictionary.  Only used by beaker's get_recipe command.
Pretty straightforward.

Beaker folks should review this.  Not very interesting to autotest folks other
than style and syntax.

Signed-off-by: Don Zickus <dzickus@redhat.com>
harness: support beaker proxy protocol
This patch contains the pieces to communicate with Beaker's new stable API.

It is a thin layer that takes commands from the harness and translates them into
the appropriate http GET, POST or PUT command.

One thing I did to make debugging easier is to allow the ability to write to
files locally (offline mode).  The harness doesn't care where the data is sent,
so writing locally allow me to quickly test and address problems without complicating
it with broken http requests.

I also implemented chunk uploading to minimize memory usage.

I spent most of my time testing the log uploading and task result reporting
aspect of the API.  Everything seems to work great.  I expect the 'status'
and 'watchdog' parts to just work.

Mostly of interest to the beaker folks.  Autotest folks can review for style,
syntax and entertainment.

v2: python lint fixes

Signed-off-by: Don Zickus <dzickus@redhat.com>
harness: new beaker harness
This is the bulk of the harness file for enabling beaker.

The idea is to bootstrap it by downloading the remote xml file, parse it,
convert it into a control file and pass that to autotest.

The rest of the harness has various hacks to deal with passing info
from autotest to beaker.

Two of the biggest pieces are converting the xml recipe into a control
file.  This will still need tweaking.

The other piece is trying to figure out when recipe/tasks/sub-tasks
start and stop.  Again probably needs tweaking.

Overall, this patch has been tested with some stub recipes and responds
well.  Work still needs to be done on figuring out which files to
upload and real workflow testing.

This patch will need to be reviewed by autotest and beaker folks.

v2: rewrote file uploading to handle reboot case (save state), python
    lint fixes, handle reservesys case, param fixes

Signed-off-by: Don Zickus <dzickus@redhat.com>
Harness: Add beaker hook for quick commands
Beaker tests call a lot of shell scripts to report statuses and logs to the beaker
server.  In order to control where the data goes and how it gets there, redirect
everything to autotest for processing.  This is done by adding a cheap hack to
the bootstrap command and creating stub scripts.

The beaker tests now call the stub scripts, which call the autotest bootstrap
command, which takes the data and reformats it for delivery to the destination.

I attached the scripts I am using as examples of usage cases.  I am trying to work
out a solution where this can be one shell script sourced from an environment.

Its a little ugly but works around some of the beaker quirks.

Signed-off-by: Don Zickus <dzickus@redhat.com>
@lmr

This comment has been minimized.

Copy link
Member

lmr commented Apr 12, 2013

Forgot to mention that this links to issue #609, necessary for completion of the milestone 1.0

@ghost ghost assigned lmr Apr 12, 2013

@lmr

This comment has been minimized.

Copy link
Member

lmr commented May 15, 2013

Update - Ok, so after talking to Don, there are some bugs in the reboot code that have to be ironed out before this is fully useful on a beaker deployment.

@lmr

This comment has been minimized.

Copy link
Member

lmr commented May 15, 2013

Another comment from Beaker developer Dan Callaghan:

https://lists.fedorahosted.org/pipermail/beaker-devel/2013-April/000549.html

@lmr

This comment has been minimized.

Copy link
Member

lmr commented on client/harness_beaker.py in 26b9c0f Jun 18, 2013

Oh well, it'd be more elegant to use os.path.join here...

@lmr

This comment has been minimized.

Copy link
Member

lmr commented on client/harness_beaker.py in 26b9c0f Jun 18, 2013

Seems like this should be:

control.write("job.run_test(url='%s', timeout=%s)\n" % (rpm_name, timeout))

This comment has been minimized.

Copy link
Member

lmr replied Jun 18, 2013

Ok, now I see, in the beginning of the function the timeout is provided as an entire string...

@lmr

This comment has been minimized.

Copy link
Member

lmr commented on client/harness_beaker.py in 26b9c0f Jun 18, 2013

I wonder why make help isn't working...

@lmr

This comment has been minimized.

Copy link
Member

lmr commented on client/harness_beaker.py in 26b9c0f Jun 18, 2013

Oh well, I'm not overly fond of this import rename, since in autotest we always tend to use the full name, and therefore always makes me confused of 'why is this log. thing doing here'...

@lmr

This comment has been minimized.

Copy link
Member

lmr commented on client/harness_beaker.py in 26b9c0f Jun 18, 2013

I think we could start to leverage the new init system API that @rbbratta is working on... But well, this is for a later point in the game.

This comment has been minimized.

Copy link
Member

lmr replied Jun 18, 2013

Anyway, this copy and paste thing is a bit embarassing... I'll see if some refactoring helps here...

@lmr

This comment has been minimized.

Copy link
Member

lmr commented on client/harness_beaker.py in 26b9c0f Jun 18, 2013

Ok, fair enough...

@lmr

This comment has been minimized.

Copy link
Member

lmr commented on client/harness_beaker.py in 26b9c0f Jun 18, 2013

Considering this mapping here on client/shared/base_job.py:

job_statuses = {
    "TEST_NA": False,
    "ABORT": False,
    "ERROR": False,
    "FAIL": False,
    "WARN": False,
    "GOOD": True,
    "START": True,
    "END GOOD": True,
    "ALERT": False,
    "RUNNING": False,
    "NOSTATUS": False
}

We realize that 'TEST_NA' should be in the list of status to get 'Fail' status....

@lmr

This comment has been minimized.

Copy link
Member

lmr commented on client/harness_beaker.py in c0d6b6d Jun 18, 2013

TESTNAME

@lmr

This comment has been minimized.

Copy link
Member

lmr commented on client/harness_beaker.py in c0d6b6d Jun 18, 2013

One could capture the output with system_output, but I guess this shell redirect is OK too...

def harness_args_env():
if 'AUTOTEST_HARNESS_ARGS' in os.environ:
return os.environ['AUTOTEST_HARNESS_ARGS']
return None

This comment has been minimized.

@rbbratta

rbbratta Jun 18, 2013

Contributor

don't do hash lookups twice. Also, assume the happy path and let the exception catch the abnormal path.

def harness_args_env():
    try:
        return os.environ['AUTOTEST_HARNESS_ARGS']
    except KeyError:
        return None 

This comment has been minimized.

@lmr

lmr Jun 18, 2013

Member

Some really don't like this approach (I do, just for the record). As here the KeyError exception is specific and would hardly happen for any other reason than 'AUTOTEST_HARNESS_ARGS' not being present, so yeah, good suggestion.

@lmr

This comment has been minimized.

Copy link
Member

lmr commented Jun 18, 2013

Fixed the small problems and pushed to next. There are some known problems that will be properly documented, thanks!

@lmr lmr closed this Jun 18, 2013

@lmr lmr deleted the beaker branch Jun 18, 2013

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment