Note: This is an older (outdated) page. Please see the updated page, and update your bookmark.
(SFI-93-03-014)
We present results from an experiment similar to one performed by Packard (1988), in which a genetic algorithm is used to evolve cellular automata (CA) to perform a particular computational task. Packard examined the frequency of evolved CA rules as a function of Langton's lambda parameter (Langton, 1990), and interpreted the results of his experiment as giving evidence for the following two hypotheses: (1) CA rules able to perform complex computations are most likely to be found near ``critical'' lambda values, which have been claimed to correlate with a phase transition between ordered and chaotic behavioral regimes for CA; (2) When CA rules are evolved to perform a complex computation, evolution will tend to select rules with lambda values close to the critical values. Our experiment produced very different results, and we suggest that the interpretation of the original results is not correct. We also review and discuss issues related to lambda, dynamical-behavior classes, and computation in CA.
The main constructive results of our study are identifying the emergence and competition of computational strategies and analyzing the central role of symmetries in an evolutionary system. In particular, we demonstrate how symmetry breaking can impede the evolution toward higher computational capability.
(SFI-93-06-040)
In this paper we review previous work and present new work concerning the relationship between dynamical systems theory and computation. In particular, we review work by Langton (1990) and Packard (1988) on the relationship between dynamical behavior and computational capability in cellular automata (CA). We present results from an experiment similar to the one described in Packard (1988), that was cited there as evidence for the hypothesis that rules capable of performing complex computations are most likely to be found at a phase transition between ordered and chaotic behavioral regimes for CA (the "edge of chaos").
Our experiment produced very different results from the original experiment, and we suggest that the interpretation of the original results is not correct. We conclude by discussing general issues related to dynamics, computation, and the "edge of chaos" in cellular automata.
SFI Working Paper 93-11-071
We present results from experiments in which a genetic algorithm was used to evolve cellular automata (CAs) to perform a particular computational task---one-dimensional density classification. We look in detail at the evolutionary mechanisms producing the GA's behavior on this task and the impediments faced by the GA. In particular, we identify four ``epochs of innovation'' in which new CA strategies for solving the problem are discovered by the GA, describe how these strategies are implemented in CA rule tables, and identify the GA mechanisms underlying their discovery. The epochs are characterized by a breaking of the task's symmetries on the part of the GA. The symmetry breaking results in a short-term fitness gain but ultimately prevents the discovery of the most highly fit strategies. We discuss the extent to which symmetry breaking and other impediments are general phenomena in any GA search.
SFI Working Paper 94-03-012
A simple evolutionary process can discover sophisticated methods for emergent information processing in decentralized spatially-extended systems. The mechanisms underlying the resulting emergent computation are explicated by a novel technique for analyzing particle-based logic embedded in pattern-forming systems. Understanding how globally-coordinated computation can emerge in evolution is relevant both for the scientific understanding of natural information processing and for engineering new forms of parallel computing systems.
SFI Working Paper 94-03-015
How does evolution produce sophisticated emergent computation in systems composed of simple components limited to local interactions? To model such a process, we used a genetic algorithm (GA) to evolve cellular automata to perform a computational task requiring globally-coordinated information processing. On most runs a class of relatively unsophisticated strategies was evolved, but on a subset of runs a number of quite sophisticated strategies was discovered. We analyze the emergent logic underlying these strategies in terms of information processing performed by ``particles'' in space-time, and we describe in detail the generational progression of the GA evolution of these strategies. Our analysis is a preliminary step in understanding the general mechanisms by which sophisticated emergent computational capabilities can be automatically produced in decentralized multiprocessor systems.
SFI Working Paper 95-01-005
How does an evolutionary process interact with a decentralized, distributed system in order to produce globally coordinated behavior? Using a genetic algorithm (GA) to evolve cellular automata (CAs), we show that the evolution of spontaneous synchronization, one type of emergent coordination, takes advantage of the underlying medium's potential to form embedded particles. The particles, typically phase defects between synchronous regions, are designed by the evolutionary process to resolve frustrations in the global phase. We describe in detail one typical solution discovered by the GA, delineating the discovered synchronization algorithm in terms of embedded particles and their interactions. We also use the particle-level description to analyze the evolutionary sequence by which this solution was discovered. Our results have implications both for understanding emergent collective behavior in natural systems and for the automatic programming of decentralized spatially extended multiprocessor systems.
We describe an application of genetic algorithms (GAs) to the design of cellular automata (CAs) that can perform computations requiring global coordination. A GA was used to evolve CAs for two computational tasks: density classification and synchronization. In both cases, the GA discovered rules that gave rise to sophisticated emergent computational strategies. These strategies can be analyzed using a ``computational mechanics'' framework in which ``particles'' carry information and interaction between particles effects information processing. This framework can also be used to explain the process by which the strategies are designed by the GA. This work is a first step in employing GAs to engineer useful emergent computation in decentralized multi-processor systems.
We review recent work done by our group on applying genetic algorithms (GAs) to the design of cellular automata (CAs) that can perform computations requiring global coordination. A GA was used to evolve CAs for two computational tasks: density classification and synchronization. In both cases, the GA discovered rules that gave rise to sophisticated emergent computational strategies. These strategies can be analyzed using a ``computational mechanics'' framework in which ``particles'' carry information and interactions between particles effects information processing. This framework can also be used to explain the process by which the strategies were designed by the GA. The work described here is a first step in employing GAs to engineer useful emergent computation in decentralized multi-processor systems. It is also a first step in understanding how an evolutionary process can produce complex systems with sophisticated collective computational abilities.
Physics Letters A , 229 (2), 144-150.
Santa Fe Institute Working Paper 96-08-054
We introduce an analytical model that predicts the dynamics of a simple evolutionary algorithm in terms of the flow in the space of fitness distributions. In the limit of infinite populations the equations of motion are derived in closed form. We show how finite populations induce periods of stasis--"fitness epochs"--and rapid jumps--"innovations". The analysis identifies the epochs with the flow's metastable fixed points and gives exact predictions of epoch fitness level, duration, and population distribution.
In Proceedings of Physics and Computation '96
SFI Working Paper 96-09-073
In previous work, we analyzed the process by which a genetic algorithm designed CAs to perform particular tasks. In this paper we focus on how these CAs implement the emergent computational strategies for performing a task. In particular, we develop a class of embedded-particle models to describe the computational strategies implemented by particular CAs. To do this, we use the computational mechanics framework of Crutchfield and Hanson, in which a CA's information processing is described in terms of regular domains, embedded particles, and their interactions. We then evaluate this class of models by comparing their computational performance to that of the CAs they model. The results demonstrate, via a generally close quantitative agreement between the CAs and the embedded particle models, that this new model class captures the significant functional features in the CAs' space-time behavior that underlie the CAs' computational capability and evolutionary fitness.
Special Issue on Evolutionary Computation, Theoretical Computer Science 229 (1999) 41-102, A. E. Eiben and G. Rudolph, editors.
SFI Working Paper 97-04-035.
Abstract
Metastability is a common phenomenon. Many evolutionary processes, both natural and artificial, alternate between periods of stasis and brief periods of rapid change in their behavior. In this paper an analytical model for the dynamics of a mutation-only genetic algorithm (GA) is introduced that identifies a new and general mechanism causing metastability in evolutionary dynamics. The GA's population dynamics is described in terms of flows in the space of fitness distributions. The trajectories through fitness distribution space are derived in closed form in the limit of infinite populations. We then show how finite populations induce metastability, even in regions where fitness does not exhibit a local optimum. In particular, the model predicts the occurrence of ``fitness epochs''---periods of stasis in population fitness distributions---at finite population size and identifies the locations of these fitness epochs with the flow's hyperbolic fixed points. This enables exact predictions of the metastable fitness distributions during the fitness epochs, as well as giving insight into the nature of the periods of stasis and the innovations between them. All these results are obtained as closed-form expressions in terms of the GA's parameters.
An analysis of the Jacobian matrices in the neighborhood of an epoch's metastable fitness distribution allows for the calculation of its stable and unstable manifold dimensions and so reveals the state space's topological structure. More general quantitative features of the dynamics---fitness fluctuation amplitudes, epoch stability, and speed of the innovations---are also determined from the Jacobian eigenvalues. The analysis shows how quantitative predictions for a range of dynamical behaviors, that are specific to the finite population dynamics, can be derived from the solution of the infinite population dynamics. The theoretical predictions are shown to agree very well with statistics from GA simulations. We also discuss the connections of our results with those from population genetics and molecular evolution theory.
Submitted to PPSN-V
SFI Working Paper 98-04-034
We introduce a class of embedded-particle models for describing the emergent computational strategies observed in cellular automata (CAs) that were evolved for performing certain computational tasks. The models are evaluated by comparing their estimated performances with the actual performances of the CAs they model. The results show, via a close quantitative agreement, that the embedded-particle framework captures the main information processing mechanisms of the emergent computation that arise in these evolved CAs.
To appear in the Special Issue on Evolutionary and Genetic Algorithms in Computational Mechanics and Engineering of the journal Computer Methods in Applied Mechanics and Engineering, D. Goldberg, editor (1999).
SFI Working Paper 98-06-046.
LANL Archive adap-org/9810003.
Abstract
Epochal dynamics, in which long periods of stasis in population fitness are punctuated by sudden innovations, is a common behavior in both natural and artificial evolutionary processes. We use a recent quantitative mathematical analysis of epochal evolution to estimate, as a function of population size and mutation rate, the average number of fitness function evaluations to reach the global optimum. This is then used to derive estimates of and bounds on evolutionary parameters that minimize search effort.
Submitted to the Machine Learning Journal.
SFI Working Paper 98-09-080.
LANL Preprint
http://xxx.lanl.gov/abs/adap-org/9809001.
Abstract
We investigate the ability of a genetic algorithm to design cellular automata that perform computations. The computational strategies of the resulting cellular automata can be understood using a framework in which ``particles'' embedded in space-time configurations carry information and interactions between particles effect information processing. This structural analysis can also be used to explain the evolutionary process by which the strategies were designed by the genetic algorithm. More generally, our goals are to understand how machine-learning processes can design complex decentralized systems with sophisticated collective computational abilities and to develop rigorous frameworks for understanding how the resulting dynamical systems perform computation.
To appear in the Machine Learning Journal (1999).
SFI Working Paper 98-10-090.
LANL Archive adap-org/9810004.
Abstract
Epochal dynamics, in which long periods of stasis in an evolving population are punctuated by a sudden burst of change, is a common behavior in both natural and artificial evolutionary processes. We analyze the population dynamics for a class of fitness functions that exhibit epochal behavior using a mathematical framework developed recently. In the latter the approximations employed led to a population-size independent theory that allowed us to determine optimal mutation rates. Here we extend this approach to include the destabilization of epochs due to finite-population fluctuations and show that this dynamical behavior often occurs around the optimal parameter settings for efficient search. The resulting, more accurate theory predicts the total number of fitness function evaluations to reach the global optimum as a function of mutation rate, population size, and the parameters specifying the fitness function. We further identify a generalized error threshold, smoothly bounding the two-dimensional regime of mutation rates and population sizes for which evolutionary search operates efficiently.
To appear in Evolution as Computation, L. F. Landweber, E. Winfree, R. Lipton, and S. Freeland, editors, Lecture Notes in Computer Science, Springer-Verlag (1999). Proceedings of a DIMACS Workshop, 11-12 January 1999, Princeton University.
SFI Working Paper 99-02-015.
LANL Archive adap-org/9903001.
Abstract
We analyze the population dynamics of a broad class of fitness functions that exhibit epochal evolution---a dynamical behavior, commonly observed in both natural and artificial evolutionary processes, in which long periods of stasis in an evolving population are punctuated by sudden bursts of change. Our approach---{\em statistical dynamics}---combines methods from both statistical mechanics and dynamical systems theory in a way that offers an alternative to current ``landscape'' models of evolutionary optimization. We describe the population dynamics on the macroscopic level of fitness classes or phenotype subbasins, while averaging out the genotypic variation that is consistent with a macroscopic state. Metastability in epochal evolution occurs solely at the macroscopic level of the fitness distribution. While a balance between selection and mutation maintains a quasistationary distribution of fitness, individuals diffuse randomly through selectively neutral subbasins in genotype space. Sudden innovations occur when, through this diffusion, a genotypic portal is discovered that connects to a new subbasin of higher fitness genotypes. In this way, we identify innovations with the unfolding and stabilization of a new dimension in the macroscopic state space. The architectural view of subbasins and portals in genotype space clarifies how frozen accidents and the resulting phenotypic constraints guide the evolution to higher complexity.
Proceedings of the National Academy of Sciences 96 (1999) 9716-9720.
SFI Working Paper 99-03-021.
LANL Archive adap-org/9903006.
Abstract
We introduce and analyze a general model of a population evolving over a network of selectively neutral genotypes. We show that the population's limit distribution on the neutral network is solely determined by the network topology and given by the principal eigenvector of the network's adjacency matrix. Moreover, the average number of neutral mutant neighbors per individual is given by the matrix spectral radius. This quantifies the extent to which populations evolve mutational robustness: the insensitivity of the phenotype to mutations. Since the average neutrality is independent of evolutionary parameters---such as, mutation rate, population size, and selective advantage---one can infer global statistics of neutral network topology using simple population data available from {\it in vitro} or {\it in vivo} evolution. Populations evolving on neutral networks of RNA secondary structures show excellent agreement with our theoretical predictions.
To appear in Bulletin of Mathematical Biology (2000).
SFI Working Paper 99-04-041.
LANL Archive adap-org/9907002.
Abstract
We analytically study the dynamics of evolving populations that exhibit
metastability on the level of phenotype or fitness. In constant
selective environments, such metastable behavior is caused by two
qualitatively different mechanisms. One the one hand, populations may
become pinned at a local fitness optimum, being separated from
higher-fitness genotypes by a fitness barrier of low-fitness
genotypes. On the other hand, the population may only be metastable on
the level of phenotype or fitness while, at the same time, diffusing
over neutral networks of selectively neutral genotypes.
Metastability occurs in this case because the population is separated
from higher-fitness genotypes by an entropy barrier: The
population must explore large portions of these neutral networks
before it discovers a rare connection to fitter phenotypes.
We derive analytical expressions for the barrier crossing times in both the fitness barrier and entropy barrier regime. In contrast with ``landscape'' evolutionary models, we show that the waiting times to reach higher fitness depend strongly on the width of a fitness barrier and much less on its height. The analysis further shows that crossing entropy barriers is faster by orders of magnitude than fitness barrier crossing. Thus, when populations are trapped in a metastable phenotypic state, they are most likely to escape by crossing an entropy barrier, along a neutral path in genotype space. If no such escape route along a neutral path exists, a population is most likely to cross a fitness barrier where the barrier is narrowest, rather than where the barrier is shallowest.
Submitted to IEEE Trans. Evolutionary Computation (1999).
SFI Working Paper 99-07-045.
LANL Archive adap-org/9907007.
Abstract
Evolving one-dimensional cellular automata (CAs) with genetic algorithms has provided insight into how improved performance on a task requiring global coordination emerges when only local interactions are possible. Two approaches that can affect the search efficiency of the genetic algorithm are coevolution, in which a population of {\it problems}---in our case, initial configurations of the CA lattice---evolves along with the population of CAs; and resource sharing, in which a greater proportion of a limited fitness resource is assigned to those CAs which correctly solve problems that fewer other CAs in the population can solve. Here we present evidence that, in contrast to what has been suggested elsewhere, the improvements observed when both techniques are used together depend largely on resource sharing alone.