%\documentclass[10pt]{article}
%\documentclass{sig-alternate}
\documentclass[letterpaper]{sig-alternate}  
\usepackage[vmargin=1in,hmargin=.75in]{geometry} % Need this so that letterpaper (US Letter, 8.5 by 11) works
%\usepackage[textwidth=42pc,textheight=666pt]{geometry}
\usepackage{graphicx}


\title{Developmental Tuning of Neural Networks using Weighted Function Graphs}

\numberofauthors{1}

\author{
\alignauthor Keith L. Downing \\ 
\affaddr{Department of Computer Science} \\
\affaddr{The Norwegian University of Science and Technology (NTNU)} \\
\affaddr{Trondheim, Norway} \\
\email{keithd@idi.ntnu.no} 
}


\begin{document}
\conferenceinfo{GECCO'10,} {July 7--11, 2010, Portland, Oregon, USA.} 
\CopyrightYear{2010}
\crdata{978-1-4503-0072-8/10/07}
\clubpenalty=10000
\widowpenalty = 10000

\bibliographystyle{siam}

\maketitle

\begin{abstract}
The Weighted Function Graph (WFG) is introduced as a form of Compositional Pattern-Producing Network (CPPN) that evolves using a very indirect genotype-phenotype mapping.  
 Although CPPNs and WFGs have broad applicability in evolving neural networks, the former are primarily used as mappings from spatial coordinates to connection weights or learning
 parameters.  Conversely, this work uses WFGs to encode developmental activation waves across adjacent layers of neurons, akin to those observed in developing brains, which provide early patterning of synaptic change based on correlated neural firing.  This WFG-based developmental tuning is then used to illustrate the Baldwin Effect in neural networks that
 evolve, develop and learn.  

\end{abstract}

\textbf{Categories and Subject Descriptors}

I.2.6[\textbf{Learning}]: Connectionism and Neural Nets

\textbf{General Terms}

Algorithms

\section{Introduction}

Stanley's Compositional Pattern-Producing Networks (CPPNs) \cite{stanley:2007} are elegant pattern-generating tools, often employed as mappings from spatial coordinates to output properties, such as pixel color intensity.  When evolved interactively, CPPNs can produce intricate pictures of high artistic quality \cite{stanley:2011a}.  In addition, in their HyperNEAT system \cite{stanley:2009}, Stanley and co-workers have used CPPNs to map the locations of pairs of neurons to the weights of the connections between them, thus enabling a compact CPPN to assign weights to the arcs of arbitrarily large neural networks.  The CPPN thus serves as a developmental encoding for synaptic weights, and one with inherent geometric properties that give rise to functionally significant topological patterns in ANN weight matrices \cite{stanley:2010a, stanley:2010b}.

This work, though highly inspired by the CPPN research, moves in a different direction, one involving higher degrees of both implicit/indirect coding and biological plausibility.

First, the Weighted Function Graph (WFG) is an alternate implementation of the CPPN, one that relies on a very indirect mapping from a bit-string genotype to a network of
interconnected functions (with weights on the arcs between them).  Each gene merely codes for a function node and several tags, which indirectly determine the other functions
to (from) which it will send (receive) weighted outputs.  This exhibits \textit{weak linkage} of modular system components, one of the key enablers of evolvability according to
the popular new evolutionary theory known as \textit{facilitated variation} \cite{kirschner:2005}.  This contrasts sharply with the highly direct encoding of CPPN links and weights
in Stanley's NEAT\cite{stanley:2002}, the evolutionary engine behind CPPNs.

Second, the route from WFG to artificial neural network (ANN) weights is more biologically plausible, and thus more  circuitous, in this research than in contemporary
applications of CPPNs to evolving ANNs (EANNs).  Here, the WFG exploits the fundamental pattern-producing ability of the CPPN to generate simulated waves of neural
stimulation.  These waves spread across a neural layer, yielding complex, often heterogeneous, activation time series for each of the layer's neurons.  When these correlate with
wave-induced activations in neighboring layers, Hebbian-based synaptic tuning of the inter-layer connections occurs.  

For each neural layer in the EANN, genes code for diverse properties, including the components of a WFG.  Thus, each layer includes a wave-generating WFG employed
during development to help produce an initial pattern of synaptic strengths, which can be further tuned by standard ANN learning upon exposure of the network to training situations.

This use of WFGs for neural development is motivated by our long-standing interest in the Baldwin Effect \cite{baldwin:1896,weber:2003} and recent attempts \cite{downing:2004b, downing:2010} to understand
the disruptive and facilitative roles that development plays in the Baldwinian interactions between learning and evolution, particularly when these pertain to neural structures, and thus to the evolution of intelligence.  This paper continues that investigation by using WFG-based neural development in a simple test of the Baldwin Effect involving EANNs.

\section{Weighted Function Graphs}

As shown in Figure \ref{wfg-example}, a WFG resembles a neural network, but with each node housing one of a number of alternative activation functions, as opposed to the
standard sigmoids, step functions and hyperbolic tangents of neural-network nodes.  For example, the WFG may include Gaussians, absolute values, and sine waves (as well as the common ANN activation functions). Each WFG connection includes a weight, and all nodes compute the sum of their weighted inputs, which serves as input to the activation function, whose result becomes the node's output.

WFGs have no explicit layered organization (other than pre-designed input and output nodes), so any node can send outputs to any other node; and all nodes (except the inputs, depicted as boxes in Figure \ref{wfg-example}) can receive weighted outputs.  At each timestep, the nodes undergo asynchronous activation, wherein each node simply sums the weighted outputs in its input buffer and feeds
that sum to its activation function to produce an output value, which is immediately propagated to the input buffers of all post-synaptic neighbors.  After a user-determined number of update rounds, the WFG's outputs are gathered from the output nodes (e.g., the lower node in Figure \ref{wfg-example}).  


\begin{figure}[htbp]
\begin{center}
\includegraphics[scale = 0.3]{figs/wfg-example}
\caption{Simple example of a Weighted Function Graph (WFG), with inputs coming from the 3 boxes on the top, and output read from the lower of the 3 function nodes.}
\label{wfg-example}
\end{center}
\end{figure}

\subsection{Evolving WFGs}

WFGs are evolved via a simple bit-vector chromosome consisting of multiple segments, one for each node in the network, including inputs and outputs.  Each segment
consists of 5 genes that encode:

\begin{itemize}
\item the activation function
\item the afferent connection tag
\item the efferent connection tag
\item the afferent weight tag
\item the efferent weight tag
\end{itemize}

 The first is simply an index into a list of
possible activation functions, while, as shown in  Figure \ref{wfg-evolve}, the afferent tags for node N help determine a) which nodes can send input to N, and b) the weights on those incoming arcs.  Similarly, the efferent tags influence a) the nodes to which N can send its output, and b) the weights on those outgoing arcs.  The two afferent (efferent) tags
constitute the afferent (efferent) \textit{mask} of each node.

More specifically, if the afferent connection tag of node N matches the
efferent connection tag of node M (above a user-defined match threshold), then M will send an excitatory connection to N.  Conversely, if the match is very poor, and thus below
a similar threshold, then M will send an inhibitory connection to N.  For medium-strength tag matches, no connection between M and N is created.  Then, the strength of an excitatory or inhibitory arc is positively correlated with the matching degree of M's efferent-weight- and N's afferent-weight tags.

For example, in Figure \ref{wfg-evolve}, the genotype encodes 4 nodes, labelled A-D.  Each has afferent and efferent connection tags as shown (with weight tags omitted for
clarity).  The match thresholds imply that efferent-to-afferent tags that share 75\% or more bits will produce an excitatory link between their respective nodes, while sharing of 25\% or less
justifies an inhibitory arc; otherwise, no connect is formed.  So node A forms an excitatory link to node C, due to the close match between A's efferent and C's afferent mask.  Similarly,
B links to A in an excitatory manner, as does C to itself.   Conversely, C inhibits both B and D due to very poor matches between C's efferent mask and the afferent masks of B and D; and finally, D inhibits A.  The absolute values of the weights on each arc are then determined by comparisons of the efferent-to-afferent weight tags, which are omitted from the figure.

WFGs have a pre-defined number of input and output nodes, but the efferent and afferent tags, along with the activation functions, of output nodes can evolve, as can the two efferent tags of each input node.  Since recurrent links are permitted in WFGs, both types of tags are relevant for output nodes, while input nodes are restricted to feed-forward signalling. During WFG configuration, once all connections are determined, nodes that either form no connections or, more generally, do not lie along at least one pathway from some input to some output node, are removed.

The indirect coding and tag-controlled configuration process supports a balance between stability and flexibility of interaction between nodes over evolutionary time, as a single tag-bit mutation can lead to major topological changes, although most mutations will cause little or no change.  This embodies the concept of \textit{weak linkage} among modular components, a key precursor to evolvability, as described in Gerhart and Kirschner's theory of \textit{facilitated variation} \cite{kirschner:2005}.  To wit, modules whose interactions involve simple enabling signals (as opposed to complex, instructive signals) whose general structure matters more than its precise details, are both a) easy to reconfigure, since other modules may evolve to produce a similar enabling signal, and b) robust to many genetic mutations, since small changes to the signal often do not affect its triggering ability.

This attempt at increased biological plausibility in line with facilitated variation increases the indirect nature of the genotype-phenotype mapping, thus increasing the difficulty of
finding \textit{needle-in-a-haystack} solution WFGs, in problems where only isolated solutions exist.  However, the general enhancement of evolvability accrued by weak linkage has 
clear advantages in artificial evolution, as shown in the Enzyme Genetic Programming system \cite{lones:2002}.

\begin{figure}[htbp]
\begin{center}
\includegraphics[scale = 0.45]{figs/wfg-evolve}
\caption{WFGs evolve as sets of nodes, each of which contains a function indicator along with two masks, afferent and efferent, each consisting of connection- and weight-influencing bits.  The topology and arc weights are determined by tag matches, with nodes that fail to hook up being discarded.  To simplify the illustration, no inputs nor outputs are included, and only the connection tags of each mask are shown. The match thresholds for inhibitory and excitatory connection formation are 25\% and 75\%, respectively.}
\label{wfg-evolve}
\end{center}
\end{figure}

\section{Developmental Synaptic Tuning}

Neuroscientific studies indicate that spontaneous waves of neural activity, modulated by cyclic-AMP (cAMP) concentrations, lead to early synaptic tuning during
development, prior to the exposure to normal sensory inputs.  This has shown to play an important role in the binocular segregation of connections from
the retina to the lateral geniculate nucleus (LGN) \cite{stellwagen:2002}, while others \cite{mcnaughton:2006} postulate similar wave-induced synaptic tuning in the hippocampus, and a variety of evidence (summarized in \cite{sanes:2006} indicates both a) the presence of these waves throughout the brain during neural development, and b) their instructive role in synaptic formation and tuning.

These waves promote neural firing such that neurons in adjacent regions/layers that happen to fire simultaneously (due to the stimulation from their respective activation waves) will have
their synaptic connections modified, typically by Hebbian means.  Thus, early chemical waves strongly influence the patterning of neuronal connections, prior to the molding effects of normal sensory stimuli.  A comprehensive model of this phenomena would include the chemical and physical bases of reaction-diffusion processes, a reasonably straightforward but computationally-intensive endeavor.

A host of interesting neural development models can be found in \cite{ooyen:2003b}, but this research attempts to incorporate the basic neuroscientific mechanisms into a much more abstract model.  Fortunately, WFGs provide an efficient alternative for compactly representing any number of pattern-producing processes found in nature.  

Just as a WFG can generate
pictures by mapping Cartesian coordinates to pixel intensities, by supplementing these coordinates with an input time step, they can produce a time series of patterns, which can easily depict a dynamic structure such as an activation wave, as shown in Figure \ref{wfg-waves}.  


\begin{figure}[htbp]
\begin{center}
\includegraphics[scale = 0.5]{figs/wfg-waves}
\caption{A WFG, when provided with Cartesian coordinates and time as inputs, produces abstract temporal activation patterns.} 
\label{wfg-waves}
\end{center}
\end{figure}

When neural layers are modeled as 2d surfaces, each neuron (n) has a center coordinate, ($x_n$,$y_n$).  Each layer also houses a WFG, whose mapping abstractly represents a 2d activation pattern that will spread across the layer during development.  Then, to compute the wave-induced activity of n at time t, simply input $x_n$,$y_n$ and t to the WFG and interpret the output value as a local activation.

When adjacent layers in an ANN include a WFG, each can be run to produce activation patterns.  As shown in Figure \ref{wfg-devp-tune}, when neurons j and k  in adjacent layers (J and K) have correlated wave-induced activation, Hebbian-based synaptic tuning on the j-k connection provides an early bias of the network.  When the activation waves reflect some aspect of the sensory world (with its motor requirements) to which the organism will eventually be exposed, then this preliminary synaptic tuning should provide a useful \textit{head start} for the neural network and agent. 

Hence, by including WFG parameters with the other layer-specific genes in an evolving ANN, any pair of interconnected layers with WFG-based development stimulation can achieve an evolving prenatal bias of its synaptic weights.

\begin{figure}[htbp]
\begin{center}
\includegraphics[scale = 0.5]{figs/wfg-devp-tune}
\caption{Two adjacent neural layers (J and K) exposed to different WFG-generated activation waves over 3 timsteps. Co-stimulated neurons in each layer induce correlation-based synaptic changes (dW), only a few of which are shown.  All colored region indicate some level of stimulation, with red (blue) being most (least) intense.} 
\label{wfg-devp-tune}
\end{center}
\end{figure}

\section{A Baldwin-Effect Model}

To assess the viability of weighted function graphs as models of neural activation spreading during development, we use a primitive
test of the Baldwin Effect (B.E.).

As described in \cite{baldwin:1896,weber:2003,downing:2010}, B.E. concerns the ability of learning to accelerate evolution via a
two-stage process.  In phase I, individuals with phenotypic plasticity achieve higher fitness than their innate traits could alone.  This moves the population distribution toward plastic individuals.  In phase II, some of these learned skills become innate
by chance mutations.  Selection pressure favors this assimilation of plastic features into the genome and developmental process when a) the environment is reasonably static across the generations, and b) learning has a fitness cost, and c) the genotype-phenotype mapping is not overly complex.

Although B.E. seems plausible for some phenotypic traits, such as the size of muscles and the efficacy of physical skills, its relationship to the evolution of intelligence is more tenuous, given contemporary understanding of the brain, neural development and synaptic change.  If learning is generally equated with synaptic change, then how can the modification of a few of the (human) brain's 100 trillion synapses be assimilated into DNA consisting of approximately 25 thousand genes? As \cite{downing:2010} argues, an interpretation of neural Baldwinism as a heterochronous shift seems more appropriate: B.E. phase II involves the transfer of a certain degree
of neurogenesis, synaptogenesis and synaptic tuning from post-natal, experience-driven learning to pre-natal development.

In \cite{downing:2010}, this heterochronous neural Baldwinism is illustrated by a shift of the brunt of neurogenesis and synaptogenesis from the post-natal (learning) to the pre-natal (developmental) phase of life, as summarized in Figure \ref{be-heterochrony}.  In this work, we turn to the third key factor, synaptic tuning, and explore the degree to which it can be assimilated into neural development.

\begin{figure}[htbp]
\begin{center}
\includegraphics[scale = 0.5]{figs/nn-devp-learn-2}
\caption{The heterochronous view of the Baldwin Effect (phases I and II) with respect to three key mechanisms of brain formation. Long-term potentiation (LTP) and long-term depression (LTD) are two forms of synaptic tuning.} 
\label{be-heterochrony}
\end{center}
\end{figure}

This investigation utilizes a simple two-layered neural network, with both input and output layers housing evolved WFGs that generate pre-natal (i.e. prior to the presentation of training cases to the network) activation waves, which then facilitate developmental synaptic modification, as depicted in Figure \ref{wfg-devp-tune}.

The task of each network is simply to map binary input patterns to output patterns, with input-output pairs generated randomly at the beginning of each evolutionary run.  The network's fitness is inversely proportional to both a) the total output error across all learning epochs, and b) the learning effort, a simple function of the learning rate and number of epochs, both evolvable parameters.

The general intuition is that networks will originally rely on a high learning effort to reduce output error and thus achieve high fitness, as exemplifies B.E. phase I.  Then, as WFGs evolve for the input and output layers, the resulting developmental activation waves and concomitant prenatal synaptic tuning should eventually bias synapses to reduce output error prior to the presentation of any training
cases.  Thus, learning effort (and its fitness cost) can decrease to achieve B.E. phase II.

\subsection{Modeling Details}

In all of the scenarios discussed below, the neural network consists of an input and output layer, fully connected in a strictly feed-forward manner, with both containing 16 neurons.    Each output neuron has a simple linear activation function: the output equals the sum of the weighted inputs.  

Synaptic weights are randomly initialized to values between -0.05 and +0.05, and after each round of synaptic modification, either during development or learning, the absolute values of all incoming weights to each output neuron are normalized geometrically (i.e. so that the weight vector has length 1). The binary input and output patterns are sparse, consisting of one 1 and many 0's.

Table \ref{ga-genes} summarizes the 6 basic characteristics that are evolved for each neural network.  Each of
the two WFG's consists of 14 genes of the type shown in Figure \ref{wfg-evolve}: 10 for the internal nodes, 1 for the output node, and 3 for the input nodes (coding for x, y and time).  The function set for evolved WFG's consists of: identity, absolute value, sine, gaussian, and the logistic (sigmoid) function, while WFG connection weights are restricted to reals in $[-5, 5 ]$. Each of the 16 input and output nodes has Cartesian coordinates to use as inputs to the respective WFG during development.
 
\begin{table}[htbp]
\begin{center}

\begin{tabular}{|c|c|c|}

\hline
Symbol & Name & Range \\
\hline 
\multicolumn{3}{|c|}{\textit{Development}} \\
\hline 
$D_{r}$ & Developmental tuning rate & [0, 0.5] \\
$D_{s}$ & Activation wave steps & [0, 5] \\
$WFG_{in}$ & WFG for input layer & 0-10 internal nodes \\
$WFG_{out}$ & WFG for output layer & 0-10 internal nodes \\
\hline
\multicolumn{3}{|c|}{\textit{Learning}} \\
\hline
$L_{r}$ & Learning rate & [0, 0.5] \\
$L_{e}$  & Learning epochs & [1, 10] \\
\hline
\end{tabular}
\end{center}
\caption{Genes in the genetic-algorithm chromosome for all scenarios described in this article.}
\label{ga-genes}
\end{table}


During both developmental and training-based synaptic tuning, the standard correlational Hebbian rule of equation \ref{hebb-rule} 
applies:

\begin{equation}
\triangle w_{i,j} = \lambda x_i x_j
\label{hebb-rule}
\end{equation}

where $w_{i,j}$ is the weight on the connection from neuron j to neuron i, $\lambda$ is either $D_r$ or $L_r$ (see Table \ref{ga-genes}), and $x_i$ and $x_j$ are the outputs of neurons i and j, respectively.

Equation \ref{ann-error} expresses the output error of each neural network:
\begin{equation}
E = \frac{1}{L_e\parallel N \parallel \parallel C \parallel} \sum_{e} \sum_{c \in C} \sum_{n \in N}  \mid x_n^{e,c} - t_n^{c} \mid
\label{ann-error}
\end{equation}

where N is the set of output neurons, C is the set of training cases, $x_n^{e,c}$ is the output of the nth neuron on case
c during epoch e, and  $t_n^{c}$ is the target output for neuron n on case c. 

Equation \ref{ann-fitn} provides the fitness function used in all scenarios:

\begin{equation}
f = \frac{1}{\sqrt{E} + \Theta L_eL_r}
\label{ann-fitn}
\end{equation}

where  $L_eL_r$ is the learning effort and $\Theta$ is the tuning tax, typically 0.05.

The key genetic-algorithm parameters include a population size of 20, full-generational replacement
with a rank-based selection mechanism and elitism of one individual, a crossover rate of 0.8 and a mutation rate of
0.05 per bit.




\section{Results}

\subsection{Scenario 1}

This scenario employs 10 training cases: sparse input and target-output vectors containing exactly one 1 and 15 0's.  The 50-generation
run of Figure \ref{wavey-fitn} illustrates a Baldwin Effect supported by WFG-based synaptic tuning.  Early on, the learning rate
rises, thus decreasing output error and increasing fitness, thereby achieving B.E. phase I.  Then, around generation 25, the learning effort drops dramatically, while maximum fitness continues its slow rise, which ushers in B.E. phase II.  The drop in learning effort produces a slight rise in output error, which trades off with learning effort in the fitness function.

What enables learning effort to decrease without incurring a major error increase is the WFG-based developmental process, which evolves into two activation patterns across the input and output layers, as shown in Figures \ref{wavey-seq1} and \ref{wavey-seq2}.  These reduce the output error (on the 10 training/test cases) from 0.25 prior to developmental synaptic tuning to 0.17 afterwards (but prior to any learning).  Since developmental effort has no cost in the model, fitness can increase when any significant amount of synaptic tuning is transfered from learning to development, provided that output error does not increase substantially as a side effect. 

Because developmental tuning has no basis in the training data, evolution must search for WFG-generated activation waves that
fortuitously bias the synapses to learn those patterns.  In examples such as this, the bias is significant: over 30\%, and in some
examples, it reaches as high as 60\%.



\begin{figure}[htbp]
\begin{center}
\includegraphics[scale = 0.4]{figs/wavey-fitn}
\caption{The evolutionary progression of best-of-generation fitness, along with population-average values for learning effort and output error, which is scaled by 10 for ease of viewing.  Note the initial increase in learning effort (B.E. phase I) that eventually gives way to a decrease, while fitness continues to rise (B.E. phase II). } 
\label{wavey-fitn}
\end{center}
\end{figure}

In addition to the WFG's, the four other evolved parameters for this best-of-run individual appear in Table \ref{wavey-ptype}, which
reveals a high tuning rate for development, 0.251, compared to the corresponding rate for learning, 0.1666.

\begin{table}[htbp]
\begin{center}
\begin{tabular}{|c|c|c|c|}
\hline
$D_{r}$ & $D_{s}$ & $L_{r}$ & $L_{e}$ \\
\hline 
0.251 & 5 & 0.1666 & 10 \\
\hline
\end{tabular}
\end{center}
\caption{Four evolved parameters for the most fit individual of scenario 1.}
\label{wavey-ptype}
\end{table}

Despite the use of elitism, a noticeable jitter appears in the fitness plot of Figure \ref{wavey-fitn}.  This stems from the
random initialization of network weights prior to developmental tuning, which cannot always produce identically-biased
synaptic patterns (prior to learning) when given diverse initial-weight distributions.  However, in most cases, a top-fitness
individual in generation k remains very highly fit when cloned into generation k+1.


\begin{figure}[t]
\begin{center}
\begin{tabular}{c c}
\includegraphics[scale = 0.2]{figs/wavey1-0.pdf} & \includegraphics[scale = 0.2]{figs/wavey1-1.pdf} \\
\includegraphics[scale = 0.2]{figs/wavey1-2.pdf} & \includegraphics[scale = 0.2]{figs/wavey1-3.pdf} \\
\end{tabular}
\end{center}
\caption{Four seqential (left-to-right, top-to-bottom) snapshots of a WFG-generated activation sequence for the network's input layer.    Lighter colors indicate higher activation.}
\label{wavey-seq1}
\end{figure}

\begin{figure}[t]
\begin{center}
\begin{tabular}{c c}
\includegraphics[scale = 0.2]{figs/wavey2-0.pdf} & \includegraphics[scale = 0.2]{figs/wavey2-1.pdf} \\
\includegraphics[scale = 0.2]{figs/wavey2-2.pdf} & \includegraphics[scale = 0.2]{figs/wavey2-3.pdf} \\
\end{tabular}
\end{center}
\caption{Four seqential (left-to-right, top-to-bottom) snapshots of a WFG-generated activation sequence for the network's output layer.}
\label{wavey-seq2}
\end{figure}

To test the generality of this result, 20 runs were performed using different sparse data sets of size 10, with exactly one 1 and 15 0's in
each vector.  As shown in Figure \ref{20-runs1}, fitness rises throughout the 50 generations, while error declines.  Learning effort
begins high but gradually decreases as developmental synaptic tuning evolves to help reduce output error.

\begin{figure}[htbp]
\begin{center}
\includegraphics[scale = 0.4]{figs/20-runs1}
\caption{Averages over 20 runs of the best-of-generation network's fitness, learning effort and output error for networks trained on 10 sparse training cases.} 
\label{20-runs1}
\end{center}
\end{figure}

\subsection{Scenario 2}

In this scenario, 20 sparse input-output pairs (again containing exactly one 1 apiece) are given to the network.  The population size (20) and all other parameters from scenario 1 are reused.  The evolution in Figure \ref{wavey3-fitn} clearly shows a gradually increasing fitness while learning effort begins high but gradually declines.  In this scenario, the learning decline is less obvious in the population averages than in the (cascading) best-of-generation learning trace. 

\begin{figure}[htbp]
\begin{center}
\includegraphics[scale = 0.4]{figs/wavey3-fitn}
\caption{The evolutionary progression of fitness and learning effort for the best-of-generation individuals.  Population averages for learning effort and output error (scaled by 10) are also plotted.} 
\label{wavey3-fitn}
\end{center}
\end{figure}

Figure \ref{wavey-seq3} shows results common to many of the (unpublished) runs: a) the evolved WFG's ignore the timestep and produce the same pattern for each timestep, and b) the output layer's WFG produces a monotonic pattern of either high or low activation in all spatial locations.  However, this mundane pair of patterns reduces output error by 44\% prior to any learning.  The remaining
phenotypic traits appear in Table \ref{wavey3-ptype}.  Note that only 3 developmental steps and 5 learning epochs were employed by this individual.

\begin{figure}[t]
\begin{center}
\begin{tabular}{c c}
\includegraphics[scale = 0.2]{figs/wavey3-0.pdf} & \includegraphics[scale = 0.2]{figs/wavey4-0.pdf} \\
\end{tabular}
\end{center}
\caption{Static (i.e. non-varying for all 3 developmental timesteps) WFG-generated activation patterns for the input (left) and
output (right) layers of the neural network for the best-fit individual of scenario 2.  These simple patterns reduced the pre-learning output error from 0.25 to 0.14, i.e. by 44\%.}
\label{wavey-seq3}
\end{figure}

\begin{table}[htbp]
\begin{center}
\begin{tabular}{|c|c|c|c|}
\hline
$D_{r}$ & $D_{s}$ & $L_{r}$ & $L_{e}$ \\
\hline 
0.492 & 3 & 0.148 & 5 \\
\hline
\end{tabular}
\end{center}
\caption{Four evolved parameters for the most fit individual of scenario 2.}
\label{wavey3-ptype}
\end{table}


\bibliography{/keithd/latex/bibtek/all}


\end{document}
\end