Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
tree: 315d471cbe
Fetching contributors…

Cannot retrieve contributors at this time

231 lines (199 sloc) 10.11 kb
\section{Results and Discussion}
\seclabel{results}
% Outline:
% - exploration of parameter space
% - parameterized optimization results, discussion
% - HyperNEAT results, discussion
% \subsection{Exploration of Parameterized Gait Space}
%
% Before optimizing the chosen family of parameterized gaits
% (\secref{motionModel}) with learning methods, we performed an
% experiment to explore the five dimensions of the SineModel5 parameter
% space. Specifically, we selected a random parameter vector that
% resulted in some motion, but not an exceptional gait. We then varied
% each of the five parameters individually and measured performance,
% repeating each measurement twice to get a rough estimate of the
% measurement noise at each point. The results of this exploration,
% shown in \figref{explore_dim_1}, reveal that some dimensions ($\amp$,
% $\tau$, $m_F$) are fairly smooth and exhibit global structure across
% the allowable parameter range, while others ($m_O$, $m_R$) exhibit
% more complex behavior. In addition, it gives a rough indication that
% measurement noise is often significant and is more likely to be large
% for gaits that move more. Of course, this is only a slice in each
% dimension through a single point, and slices through a different point
% could reveal different behavior. The common point at the
% intersection of all slices is shown as a red triangle in
% each plot of \figref{explore_dim_1}.
%\acmFig{explore_dim_1}{1}{Fitness mean and standard deviation
% vs. dimension 1. The circle is a common point in
% \figref{explore_dim_1} through \figref{explore_dim_5}}
%\acmFig{explore_dim_2}{1}{Fitness mean and standard deviation
% vs. dimension 2. The circle is a common point in
% \figref{explore_dim_1} through \figref{explore_dim_5}}
%\acmFig{explore_dim_3}{1}{Fitness mean and standard deviation
% vs. dimension 3. The circle is a common point in
% \figref{explore_dim_1} through \figref{explore_dim_5}}
%\acmFig{explore_dim_4}{1}{Fitness mean and standard deviation
% vs. dimension 4. The circle is a common point in
% \figref{explore_dim_1} through \figref{explore_dim_5}}
%\acmFig{explore_dim_5}{1}{Fitness mean and standard deviation
% vs. dimension 5. The circle is a common point in
% \figref{explore_dim_1} through \figref{explore_dim_5}}
% \acmFiggggg{explore_dim_1}{explore_dim_2}{explore_dim_3}{explore_dim_4}{explore_dim_5}{1}{.53}{Fitness
% mean and standard deviation when each parameter dimension is varied
% independently. The red triangle in each plot represents the same point in the 5-dimensional parameter space.}
\subsection{Learning Methods for Parameterized Gaits}
% Outline
% - beat hand gait
% - no learning strategy outperformed another by that much
% - real world noise yay
The results for the parameterized gaits are shown in
\figref{std_error} and \tabref{results}. A total of 1217 hardware
fitness evaluations were performed during the learning of
parameterized gaits, with the following distribution by learning
method: 200 random, 234 uniform, 284 Gaussian, 174 gradient, 172
simplex, 153 linear regression. The number of runs varies because
each run plateaued at its own pace. The best overall gait for
the parameterized methods was found by linear regression, which also
had the highest average performance. The Nelder-Mead simplex also
performed quite well on average. The other local search methods did not
outperform random search; however, all methods did manage to explore
enough of the parameter space to significantly improve on the previous
hand-coded gait in at least one of the three runs. No single strategy
consistently beat the others: for the first trial Linear Regression
produced the fastest gait at 27.58 body lengths/minute, for the second
a random gait actually won with 17.26, and for the
third trial the Nelder-Mead simplex method attained the fastest gait
with 14.83.
One reason the randomly-generated SineModel5 gaits were so effective
may have been due to the SineModel5's bias toward regular, symmetric
gaits. This may have allowed the random strategy --- focusing on
exploration --- to be competitive with the more directed strategies
that exploit information from past evaluations.
% octave:15> fit = [6.04 17.26 4.90; 11.37 9.44 2.69; 3.10 13.59 13.40; 0.68 14.69 3.60; 8.51 13.62 14.83; 27.58 12.51 1.95; 24.27 36.44 27.07];
% octave:16> mean(fit,2)
% ans =
%
% 9.4000
% 7.8333
% 10.0300
% 6.3233
% 12.3200
% 14.0133
% 29.2600
%
% octave:17> std(fit,0,2)
% ans =
%
% 6.8308
% 4.5576
% 6.0023
% 7.3914
% 3.3546
% 12.8810
% 6.3737
%%%%%% OLD TABLE
% \begin{table*}
% \begin{center}
% \begin{tabular}{|r|c|c|c||c|}
% \hline
% & A & B & C & Average \\
% \hline
% \hline
% Previous hand-coded gait & -- & -- & -- & 5.16 \\
% \hline
% Random search & 6.04 & 17.26 & 4.90 & 9.40 \\
% \hline
% Uniform Random Hill Climbing & 11.37 & 9.44 & 2.69 & 7.83 \\
% \hline
% Gaussian Random Hill Climbing & 3.10 & 13.59 & 13.40 & 10.03 \\
% \hline
% Policy Gradient Descent & 0.68 & 14.69 & 3.60 & 6.32 \\
% \hline
% Nelder-Mead simplex & 8.51 & 13.62 & 14.83 & 12.32 \\
% \hline
% Linear Regression & 27.58 & 12.51 & 1.95 & 14.01 \\
% \hline
% Evolutionary Neural Network (HyperNEAT) & 24.27 & 36.44 & 27.07 & 29.26 \\
% \hline
% \end{tabular}
% \caption{The best gaits found for each starting vector and algorithm,
% in body lengths per minute.}
% \tablabel{results}
% \end{center}
% \end{table*}
%%%%%%% NEW TABLE
\begin{table}
\begin{center}
\begin{tabular}{|r|c|c|c||c|}
\hline
& Average & Std. Dev. \\
\hline
\hline
Previous hand-coded gait & 5.16 & -- \\
\hline
Random search & 9.40 & 6.83 \\
\hline
Uniform Random Hill Climbing & 7.83 & 4.56 \\
\hline
Gaussian Random Hill Climbing & 10.03 & 6.00 \\
\hline
Policy Gradient Descent & 6.32 & 7.39 \\
\hline
Nelder-Mead simplex & 12.32 & 3.35 \\
\hline
Linear Regression & 14.01 & 12.88 \\
\hline
Evolved Neural Network & & \\
(HyperNEAT) & 29.26 & 6.37 \\
\hline
\end{tabular}
\caption{The average and standard deviation of the best gaits found
for each algorithm during each of three runs, in body
lengths/minute.} \tablabel{results}
\end{center}
\end{table}
\acmFig{std_error}{1}{Average results ($\pm $ SE) for the
parameterized learning methods, computed over three separately
initialized runs. Linear regression found the fastest overall gait
and had the highest average, followed by Nelder-Mead simplex. Other
methods did not outperform a random strategy.}
\subsection{HyperNEAT Gaits}
The results for the gaits evolved by HyperNEAT are shown in
\figref{hnResults} and \tabref{results}. A total of 540 evaluations
were performed for HyperNEAT (180 in each of three runs).
Overall the HyperNEAT gaits
were the fastest by far, beating all the parameterized models when
comparing either average or best gaits. We believe that this is
because HyperNEAT was allowed to explore a much richer space of
motions, but did so while still utilizing symmetries when
advantageous. The single best gait found during this study had a
speed of 45.72 body lengths/minute, 66\% better than the best
non-HyperNEAT gait and 8.9 times faster than the hand-coded gait.
\figref{neat_110115_211410_00000_002_filt_zoom} shows a typical
HyperNEAT gait that had high fitness. The pattern of motion is both
complex (containing multiple frequencies and repeating patterns across
time) and regular, in that patterns of multiple motors are
coordinated.
The evaluation of the gaits produced by HyperNEAT was more noisy than
for the parameterized gaits, which made learning difficult. For
example, we tested an example HyperNEAT generation-champion gait 11
times and found that its mean performance was 26 body lengths/minute
($\pm$ 13 SD), but it had a max of 38 and a min of 3. Many
effective HyperNEAT gaits were not preserved across generations
because a single poor-performing trial could prevent their selection. The HyperNEAT learning curve would be smoother if
the noise in the evaluations could be reduced or more than one
evaluation per individual could be afforded.
\acmFig{hnResults}{.8}{Average fitness ($\pm $ SE) of the highest
performing individual in the population for each generation of
HyperNEAT runs. The fitness of many high-performing HyperNEAT gaits
were halved if the gait overly stressed the motors (see text),
meaning that HyperNEAT's true performance without this penalty would
be even higher.}
%\acmFig{neat_110115_211410_00000_002_filt}{1}{Caption here...???}
\acmFig{neat_110115_211410_00000_002_filt_zoom}{1}{Example of one
high-performance gait produced by HyperNEAT showing
commands for each of nine motors. Note the complexity of the motion
pattern. Such patterns were not possible with the parameterized
SineModel5, nor would they likely result from a human designing
a different low-dimensional parameterized motion model.}
Jump to Line
Something went wrong with that request. Please try again.