1The formative process is the supreme process, indeed the only one, alike in nature and art.

2Within fifty to a hundred years, a new class of organisms is likely to emerge. These organisms will be artificial in the sense that they will originally be designed by humans. However, they will reproduce, and will evolve into something other than their initial form; they will be “alive” under any reasonable definition of the word.

3In September 1987, the first workshop on Artificial Life was held at the Los Alamos National Laboratory. Jointly sponsored by the Center for Nonlinear Studies, the Santa Fe Institute, and Apple Computer Inc, the workshop brought together 160 computer scientists, biologists, physicists, anthropologists, and other assorted ‘-ists,’ all of whom shared a common interest in the simulation and synthesis of living systems. During five intense days, we saw a wide variety of models of living systems, including mathematical models for the origin of life, self-reproducing automata, computer programs using the mechanisms of Darwinian evolution to produce co-adapted ecosystems, simulations of flocking birds and schooling fish, the growth and development of artificial plants, and much, much more.

The workshop itself grew out of my frustration with the fragmented nature of the literature on biological modeling and simulation. For years I had prowled around libraries, shifted through computer-search results, and haunted bookstores, trying to get an overview of a field which I sensed existed but which did not seem to have any coherence or unity. Instead, I literally kept stumbling over interesting work almost by accident, often published in obscure journals if published at all.

4Throughout the workshop, there was a growing sense of excitement and camaraderie — even profound relief — as previously isolated research efforts were opened up to one another for the first time. It quickly became apparent that despite the isolation we had all experienced a remarkably similar set of problems, frustrations, successes, doubts, and visions. Even more exciting was that, as the workshop progressed, one could sense an emerging consensus among the participants — a slowly dawning collective realization of the “essence” of Artificial Life.

5Although I think that none of us could have put it into words at the time, I think that many of us went away from that tumultuous interchange of ideas with a very similar vision, strongly based on themes such as bottom-up rather than top-down modeling, local rather than global control, simple rather than complex specifications, emergent rather than prespecified behavior, population rather than individual simulation, and so forth.

Perhaps, however, the most fundamental idea to emerge at the workshop was the following: Artificial systems which exhibit lifelike behaviors are worthy of investigation on their own rights, whether or not we think that the processes that they mimic have played a role in the development or mechanics of life as we know it to be. Such systems can help us expand our understanding of life as it could be.

By allowing us to view the life that has evolved here on Earth in the larger context of possible life, we may begin to derive a truly general theoretical biology capable of making universal statements about life wherever it may be found and whatever it may be made of.

6Thus it was into a rich cauldron of revolutionary ideas that the announcement of a workshop on “Artificial Life” dropped in 1987. Even though I had nothing to present, I managed to scrape together enough funding to attend. The experience was eye-opening. Here was an entire burgeoning field that shared some of my misgivings about classical AI, my increasingly biological perspective on cognition, and my intuition that these ideas could be explored through computer simulation.

7exploring the wild frontier, unencumbered by staid disciplinary boundaries, more Burning Man than Royal Society

8AL attempts to look beyond the collection of naturally occurring life in order to discover things about that set that could not be discovered by studying that set alone. AL isn’t the same thing as computational biology, which primarily restricts itself to computational problems arising in the attempt to analyze biological data, such as algorithms for matching protein sequences to gene sequences, or programs to reconstruct phylogenies from comparisons of gene sequences. Artificial life reaches far beyond computational biology. For example, AL investigates evolution by studying evolving populations of computer programs — entities that aren’t even attempting to be anything like “natural” organisms.

Many biologists wouldn’t agree with that, saying that we’re only simulating evolution. But what’s the difference between the process of evolution in a computer and the process of evolution outside the computer? The entities that are being evolved are made of different stuff, but the process is identical. I’m convinced that such biologists will eventually come around to our point of view, because these abstract computer processes make it possible to pose and answer questions about evolution that are not answerable if all one has to work with is the fossil record and fruit flies.

9Physics has largely been the science of necessity, uncovering the fundamental laws of nature and what must be true given those laws. Biology, on the other hand, is the science of the possible, investigating processes that are possible, given those fundamental laws, but not necessary.

10The demonstration of a purely logical living system, existing only in an abstract mathematical world, is the goal that Chris and others are working toward. If they succeed, then we will have a new and profound understanding of life.

11A reporter once asked me how I would feel about my children living in an era in which there was a lot of artificial life. I answered, “Which children are you referring to? My biological children, or the artifactual children of my mind?” — to use Hans Moravec’s phrase. They would both be my children, in a sense.

12I believe the day must come when the biologist will—without being a mathematician—not hesitate to use mathematical analysis when he requires it.

13Living things are made of the same chemical elements as minerals; a living being is the arena of the same physical forces as those which affect the inorganic world.

14The passage from animate to inanimate is gradual and insensible. The step between a stalagmite and a polyp is less than that between a polyp and a man, and even the trained biologist is often at a loss to determine whether a given borderland form is the result of life, or of the inanimate forces of the mineral world.

15A living being is a transformer of matter and energy, both matter and energy being uncreateable and indestructible, i.e. invariable in quantity. A living being is only a current of matter and of energy, both of which change from moment to moment while passing through the organism.

16The essential character of the living being is its Form. This is the only characteristic which it retains during the whole of its existence, with which it is born, which causes its development, and disappears with its death. The task of synthetic biology is the recognition of those physico-chemical forces and conditions which can produce forms and structures analogous to those of living beings.

17Just as synthetic chemistry began with the artificial formation of the simplest organic products, so biological synthesis must content itself at first with the fabrication of forms resembling those of the lowest organisms.

18The synthesis of life, should it ever occur, will not be the sensational discovery which we usually associate with the idea. If we accept the theory of evolution, then the first dawn of the synthesis of life must consist in the production of forms intermediate between the inorganic and the organic world, forms which possess only some of the rudimentary attributes of life, to which other attributes will be slowly added in the course of development by the evolutionary action of the environment.

19I wrote this book in wartime, and its revision has employed me during another war. It gave me solace and occupation, when service was debarred me by my years.

20Cell and tissue, shell and bone, leaf and flower, are so many portions of matter, and it is in obedience to the laws of physics that their particles have been moved, moulded and conformed. They are no exceptions to the rule that God always geometrizes.

21To treat the living body as a mechanism was repugnant, and seemed even ludicrous, to Pascal; and Goethe, lover of nature as he was, ruled mathematics out of place in natural history. Even now the zoologist has scarce begun to dream of defining in mathematical language even the simplest organic forms. When he meets with a simple geometrical construction, for instance in the honeycomb, he would fain refer it to psychical instinct, or to skill and ingenuity, rather than to the operation of physical forces or mathematical laws; when he sees in snail, or nautilus, or tiny foraminiferal or radiolarian shell a close approach to sphere or spiral, he is prone of old habit to believe that after all it is something more than a spiral or a sphere, and that in this “something more” there lies what neither mathematics nor physics can explain. In short, he is deeply reluctant to compare the living with the dead, or to explain by geometry or by mechanics the things which have their part in the mystery of life.

22In our own day the philosopher neither minimises nor unduly magnifies the mechanical aspect of the Cosmos; nor need the naturalist either exaggerate or belittle the mechanical phenomena which are profoundly associated with Life, and inseparable from our understanding of Growth and Form.

23How far even then mathematics will suffice to describe, and physics to explain, the fabric of the body, no man can foresee. It may be that all the laws of energy, and all the properties of matter, and all the chemistry of all the colloids are as powerless to explain the body as they are impotent to comprehend the soul. For my part, I think it is not so. Of how it is that the soul informs the body, physical science teaches me nothing; and that living matter influences and is influenced by mind is a mystery without a clue. Consciousness is not explained to my comprehension by all the nerve-paths and neurones of the physiologist; nor do I ask of physics how goodness shines in one man’s face, and evil betrays itself in another. But of the construction and growth and working of the body, as of all else that is of the earth earthy, physical science is, in my humble opinion, our only teacher and guide.

24Some physicists declare, as Maxwell did, that atoms or molecules more complicated by far than the chemist’s hypotheses demand, are requisite to explain the phenomena of life. If what is implied be an explanation of psychical phenomena, let the point be granted at once; we may go yet further and decline, with Maxwell, to believe that anything of the nature of physical complexity, however exalted, could ever suffice. Other physicists, like Auerbach, or Larmor, or Joly, assure us that our laws of thermodynamics do not suffice, or are inappropriate, to explain the maintenance, or (in Joly’s phrase) the accelerative absorption, of the bodily energies, the retardation of entropy, and the long battle against the cold and darkness which is death.

25The terms Growth and Form, which make up the title of this book, are to be understood, as I need hardly say, in their relation to the study of organisms. We want to see how, in some cases at least, the forms of living things, and of the parts of living things, can be explained by physical considerations, and to realise that in general no organic forms exist save such as are in conformity with physical and mathematical laws. And while growth is a somewhat vague word for a very complex matter, which may depend on various things, from simple imbibition of water to the complicated results of the chemistry of nutrition, it deserves to be studied in relation to form: whether it proceed by simple increase of size without obvious alteration of form, or whether it so proceed as to bring about a gradual change of form and the slow development of a more or less complicated structure.

26Wherever we have a true cellular complex, an arrangement of cells in actual physical contact by means of their intervening boundary walls, we find these general principles in force; we must only bear in mind that, for their easy and perfect recognition, we must be able to view the object in a plane at right angles to the boundary walls. For instance, in any ordinary plane section of a vegetable parenchyma, we recognise the appearance of a ‘froth,’ precisely resembling that which we can construct by imprisoning a mass of soap-bubbles in a narrow vessel with flat sides of glass; in both cases we see the cell-walls everywhere meeting, by threes, at angles of 120°, irrespective of the size of the individual cells: whose relative size, on the other hand, determines the curvature of the partition-walls.

27The bee makes no claim to skill in mathematics; but the mathematician has much to learn from the bee’s honeycomb.

28The engineer, who had been busy designing a new and powerful crane, saw in a moment that the arrangement of the bony trabeculae was nothing more nor less than a diagram of the lines of stress, or directions of tension and compression, in the loaded structure: in short, that Nature was strengthening the bone in precisely the manner and direction in which strength was required; and he is said to have cried out, ‘That’s my crane!’

29An organism is so complex a thing, and growth so complex a phenomenon, that for growth to be so uniform and constant in all the parts as to keep the whole shape unchanged would indeed be a much harder thing to understand than the most widespread and remote departures from such a condition.

30For the harmony of the world is made manifest in Form and Number, and the heart and soul and all the poetry of Natural Philosophy are embodied in the concept of mathematical beauty.

31From the point of view of philosophy of science the conception associated with entropy must, I think, be ranked as the great contribution of the nineteenth century to scientific thought. It marked a reaction from the view that everything to which science need pay attention is discovered by a microscopic dissection of objects.

32These specialized fields are continually growing and invading new territory. The result is like what occurred when the Oregon country was being invaded simultaneously by the United States settlers, the British, the Mexicans, and the Russians — an inextricable tangle of exploration, nomenclature, and laws.

There are fields of scientific work, as we shall see in the body of this book, which have been explored from the different sides of pure mathematics, statistics, electrical engineering, and neurophysiology; in which every single notion receives a separate name from each group; and in which important work has been triplicated or quadruplicated; while still other important work is delayed by the unavailability in one field of results that may have already become classical in the next field.

It is these boundary regions of science which offer the richest opportunities to the qualified investigator. They are at the same time the most refractory to the accepted techniques of mass attack and the division of labor. If the difficulty of a physiological problem is mathematical in essence, ten physiologists ignorant of mathematics will get precisely as far as one physiologist ignorant of mathematics, and no further. If a physiologist, who knows no mathematics, works together with a mathematician who knows no physiology, the one will be unable to state his problem in terms that the other can manipulate, and the second will be unable to put the answers in any form that the first can understand.

33The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point. Frequently the messages have meaning; that is they refer to or are correlated according to some system with certain physical or conceptual entities. These semantic aspects of communication are irrelevant to the engineering problem.

34Information is a measure of one’s freedom of choice when one selects a message.

35This means that when we write English half of what we write is determined by the structure of the language and half is chosen freely.

36The redundancy of a language is related to the existence of crossword puzzles. If the redundancy is zero any sequence of letters is a reasonable text in the language and any two dimensional array of letters forms a crossword puzzle.

37If a source can produce only one particular message its entropy is zero, and no channel is required. A computing machine set up to calculate the successive digits of π produces a definite sequence with no chance element. No channel is required to ‘transmit’ this to another point. One could construct a second machine to compute the same sequence at the point.

38The concept of information developed in this theory at first seems disappointing and bizarre — disappointing because it has nothing to do with meaning, and bizarre because it deals not with a single message but rather with the statistical character of a whole ensemble of messages. I think, however, that these should be only temporary reactions; and that one should say, at the end, that this analysis has so penetratingly cleared the air that one is now, perhaps for the first time, ready for a real theory of meaning.

39What is the difference between the astronomical and the meteorological situation which brings about all these differences, and in particular the difference between the apparent reversibility of astronomical time and the apparent irreversibility of meteorological time? In the first place, the meteorological system is one involving a vast number of approximately equal particles, some of them very closely coupled to one another, while the astronomical system of the solar universe contains only a relatively small number of particles, greatly diverse in size, and coupled with one another in a sufficiently loose way that the second order coupling effects do not change the general aspect of the picture we observe, and the very high order coupling effects are completely negligible.

40The record of paleontology indicates a definite long-time trend, interrupted and complicated though it might be, from the simple to the complex.

41In tidal evolution as well as in the origin of species, we have a mechanism by means of which a fortuitous variability, that of the random motions of the waves in a tidal sea, and of the molecules of the water, is converted by a dynamical process into a pattern of development which reads in one direction.

42This transition from a Newtonian, reversible time to a Gibbsian, irreversible, time has had its philosophical echoes. Bergson emphasized the difference between the reversible time of physics, in which nothing new happens, and the irreversible time of evolution and biology, in which there is always something new. The realization that the Newtonian physics was not the proper frame for biology was perhaps the central point in the old controversy between vitalism and mechanism; although this was complicated by the desire to conserve in some form or other at least the shadows of the soul and of God against the inroads of materialism.

43At every stage of technique since Daedalus or Hero of Alexandria, the ability of the artificer to produce a working simulacrum of a living organism has always intrigued people. This desire to produce and to study automata has always been expressed in terms of the living technique of the age. In the days of magic, we have the bizarre and sinister concept of the Golem, that figure of clay into which the Rabbi of Prague breathed in life with the blasphemy of the Ineffable Name of God. In the time of Newton, the automaton becomes the clockwork music box, with the little effigies pirouetting stiffly on top. In the nineteenth century, the automaton is a glorified heat engine, burning some combustible fuel instead of the glycogen of the human muscles. Finally, the present automaton opens doors by means of photocells, or points guns to the place at which a radar beam picks up an airplane, or computes the solution of a differential equation.

44The living organism is above all a heat engine, burning glucose or glycogen or starch, fats, and proteins, into carbon dioxide, water, and urea. It is the metabolic balance which is the center of attention; and if the low working temperatures of animal muscle attract attention as opposed to the high working temperatures of a heat engine of similar efficiency, this fact is pushed into a corner and glibly explained by a contrast between the chemical energy of the living organism and the thermal energy of the heat engine.

45In short, the newer study of automata, whether in the metal or in the flesh, is a branch of communication engineering, and its cardinal notions are those of message, amount of disturbance or « noise » — a term taken over from the telephone engineer — quantity of information, coding technique, and so on.

46The machines of which we are now speaking are not the dream of the sensationalist, nor the hope of some future time. They already exist as thermostats, automatic gyro-compass ship-steering systems, self-propelled missiles — especially such as seek their target — anti-aircraft fire-control systems, automatically controlled oil-cracking stills, ultra-rapid computing machines, and the like. They had begun to be used long before the war — indeed, the very old steam-engine governor belongs among them — but the great mechanization of the second world war brought them into their own, and the need of handling the extremely dangerous energy of the atom will probably bring them to a still higher point of development. Scarcely a month passes but a new book appears on these so-called control mechanisms or servo-mechanisms, and the present age is as truly the age of servo-mechanisms as the nineteenth century was the age of the steam engine or the eighteenth century the age of the clock.

47One of the inspirations of the present symposium has been the life work of d’Arcy Wentworth Thompson, that truly remarkable man who occupied the chair of biology at St. Andrews’ University for nearly half a century. His magnum opus “Growth and Form”, which first appeared in 1917, and was revised as a second edition in 1942, remains one of the two or three most brilliant and original books in the life sciences which this century has seen or is likely to see. D’Arcy Thompson’s approach was fundamentally mathematical; he saw in the myriad manifestations of living form a wonderful opportunity for mathematical analysis. I always remember one particular demonstration of his, that the spirals in fir-cones and other plant structures involve the famous Fibonacci series (named after the 13th century Italian mathematician) which corresponds to a continued fraction and has very far-reaching implications.

D’Arcy Thompson studied, in this 1,100-page book of his, all conceivable aspects of the mathematics of living forms the shapes of cells, in isolation or association; the shapes of concretions, spicules, and skeletons; the twists of horns, teeth, and tusks; the characters of eggs and hollow structures. His final chapter deals with the interesting theory of transformations. If certain animal or plant bodies are plotted upon Cartesian coordinates, it is found that when these are systematically distorted in various ways, forms characteristic of species related to the original species plotted appear. While d’Arcy Thompson often examined the engineering problems which living organisms have had to face, his primary interest was the mathematical description of their forms.

48What is this character, which the naturally organic possesses and the artificial usually lacks? It has something, certainly, to do with growth. Organic forms develop. The flow of time is an essential component of their full nature, and the spatial objects we can hold in our hands and examine, with eyes or microscope, during a short few minutes is only a single still out of a continuous sequence of forms which continuously unfolds, sometimes quickly, sometimes more slowly, throughout the life of the organism of which it is a part. All scientific consideration of organic form must start from this point. Science is essentially concerned with causal relations; and causal relations cannot be expressed unless there is change. It is therefore in the changes of form - during individual development, during evolution, or under the influence of function - that the biologist is mainly interested.

49The developing organism is at all times a going concern, at all times a functioning whole, and not merely a set of parts being assembled. It is self-forming, and the ‘information’ which guides its development is not a blueprint external to the process but is embedded in the process itself.

50The study of form in organisms goes by the name of Morphology, a term introduced into biology by Goethe in 1827. He regarded plants as variants of an “archetypical” form, his “Urpflanze” consisting of an axial structure (the stem) bearing lateral appendages, the leaves. Development he regarded as a reiteration of successive parts, thus a “time pattern” constituting the life history of the plant, in the course of which modifications of the fundamental repetitive unit occur - essentially a “metamorphosis” of the leaf culminating in the organs of the flower, which Goethe regarded as modified leaf structures. It is not easy in post-Darwinian days to recapture the notion of the archetype in its essential simplicity; what to us appears artificial is the divorce of a pure form or shape from any functional consideration. And yet the notion of a pure morphology exerted a powerful influence over biological thought in the early part of last century. The morphological categories of leaf, stem, and root were generally accepted, and the notion of metamorphosis was transferred from the purely abstract realm to the realm of reality, in that the functional aspect of the plant organs became the dominant consideration. The categories of leaf, stem, and root of the higher plants are the seats of essential functional activities; thus the leaf synthesizes organic constituents of the plant; the stem tissues conduct the essential materials, water and nutritive elements, to the leaf and translocate the final products to the various parts of the plant, and in addition provide the mechanical support for the effective display of the leaves to light and air; the root absorbs nutrients and water from the soil and provides the necessary anchorage for the plant.

51The principal function of the higher centres, which must pilot a vulnerable animal through a changing and hostile world, a world where chaos and cosmos are interlaced and superimposed, where anything may happen, but nothing happens twice, is to deal with uncertainty.

52There is no gruesome vitalistic fascination about viruses; they have the same gruesome mechanistic fascination as nitro-glycerine or a soufflé.

53A perfect crystal is, from the point of view of growth, a dead crystal.

54There is a great feeling of satisfaction, that no one can deny, in being able to take a living phenomenon and apply the ruler and balance to it.

55Neither development nor crystallization are downhill processes, yet each will occur spontaneously. They are machines, as Aristotle thought of them, but unlike clocks or mechanical devices, they do not decrease their thermodynamic order, but increase it by bleeding the environment.

56Here also order seems to spring from chaos, and this is perhaps even more remarkable as the crystal has no metabolic machinery, and cannot capture and store energy.

57Rarely does it happen in mathematics that a new discipline achieves the character of a mature and developed scientific theory in the first investigation devoted to it.

58Information theory is relevant to statistical inference and should be of basic interest to statisticians. Information theory provides a unification of known results, and leads to natural generalizations and the derivation of new results.

59The great beauty and power of a mathematical theory or model lies in the separation of the relevant from the irrelevant.

60Mathematically, white Gaussian noise is the least predictable. To a human being, however, all white Gaussian noise sounds alike.

61Complication, on its lower levels, is probably degenerative, that is, every automaton that can produce other automata will only be able to produce less complicated ones. There is, however, a certain minimum level where this degenerative characteristic ceases to be universal. At this point automata which can reproduce themselves, or even construct higher entities, become possible.

62It is a fundamental fact of the theory of automata that any automaton which is powerful enough to be a universal constructor must also be powerful enough to include a universal computing machine.

63Ladies and gentlemen, I wish to thank you for your very friendly welcome on my five occasions to talk to you, and I hope that I will be able to offer something for the variety of interests which are represented here. I will talk about automata-the behavior of very complicated automata and the very specific difficulties caused by high complication. I shall discuss briefly the very plausible, very obvious analogies which come to mind between artificial automata and organisms, which within a certain limit of their functioning are natural automata. We must consider the similarities, the dissimilarities, the extent to which the dissimilarities are due to our skill or clumsiness (the latter being the more normal phenomenon), and the extent to which these dissimilarities are really matters of principle.

64Unfortunately, because of his premature death, von Neumann was unable to put in final form any of the research he was doing in automata theory. In his last work on this subject he said that ‘it would be very satisfactory if one could talk about a “theory” of such automata. Regrettably, what at this moment exists … can as yet be described only as an imperfectly articulated and hardly formalized “body of experience”.’ Von Neumann’s accomplishments in this area were nevertheless substantial. He outlined the general nature of automata theory: its structure, its materials, some of its problems, some of its applications, and the form of its mathematics. He began a comparative study of artificial and natural automata. Finally, he formulated and partially answered two basic questions of automata theory: How can reliable systems be constructed from unreliable components? What kind of logical organization is sufficient for an automaton to be able to reproduce itself?

65Natural and Artificial Automata. The scope of the theory of automata and its interdisciplinary character are revealed by a consideration of the two main types of automata: the artificial and the natural. Analog and digital computers are the most important kinds of artificial automata, but other man-made systems for the communication and processing of information are also included, for example, telephone and radio systems. Natural automata include nervous systems, self-reproductive and self-repairing systems, and the evolutionary and adaptive aspects of organisms.

Automata theory clearly overlaps communications and control engineering on the one hand, and biology on the other. In fact, artificial and natural automata are so broadly defined that one can legitimately wonder what keeps automata theory from embracing both these subjects. Von Neumann never discussed this question, but there are limits to automata theory implicit in what he said. Automata theory differs from both subjects in the central role played by mathematical logic and digital computers. Though it has important engineering applications, it itself is a theoretical discipline rather than a practical one. Finally, automata theory differs from the biological sciences in its concentration on problems of organization, structure, language, information, and control.

Automata theory seeks general principles of organization, structure, language, information, and control. Many of these principles are applicable to both natural and artificial systems, and so a comparative study of these two types of automata is a good starting point. Their similarities and differences should be described and explained. Mathematical principles applicable to both types of automata should be developed. Thus truth-functional logic and delay logic apply to both computer components and neurons, as does von Neumann’s probabilistic logic.

66All of this will lead to theories which are much less rigidly of an all-or-none nature than past and present formal logic. They will be of a much less combinatorial, and much more analytical, character. In fact, there are numerous indications to make us believe that this new system of formal logic will move closer to another discipline which has been little linked in the past with logic. This is thermodynamics, primarily in the form it was received from Boltzmann, and is that part of theoretical physics which comes nearest in some of its aspects to manipulating and measuring information. Its techniques are indeed much more analytical than combinatorial, which again illustrates the point that I have been trying to make above.

67I suspect that a deeper mathematical study of the nervous system… will affect our understanding of the aspects of mathematics itself that are involved. In fact, it may alter the way in which we look on mathematics and logics proper.

68Signal detectability is a function of both the signal-to-noise ratio and the observer’s criterion.

69Morphic Processes. Since around 1870 the branch of physics concerned with natural tendencies - i.e. processes going one way towards some characteristic terminus - has been somewhat unbalanced in its emphasis. Much attention has been paid to the tendency towards dynamical disorder (heat processes, entropy), and much less to the extensive and important class of contrary processes leading towards spatial order. Curiously enough this class has not yet a clear scientific name of its own, though it must be responsible for the existence of organisms and of organisms with minds. In my view Schrodinger insulted this pre-eminent class of processes by giving them a negative and, in certain technical respects, misleading name: negative entropy (now structural neg-entropy).”

“To do something to correct this unbalance, to emphasize the positive aspect of these formative processes, and to make clear how extensive and rich with consequences they are, I have given them a scientific name: morphic. This is defined to mean ‘displaying a movement toward greater three-dimensional spatial order, symmetry, or form’.

70The meaning of a message can be defined as its selective function on the range of the recipient’s states of conditional readiness.

71A noise generator can be regarded as a source of completely original but meaningless information. A logical computer per contra provides completely unoriginal but rigorously meaningful information.

72It seems important to rid ourselves of the current heresy that only what can be expressed precisely in words has precise meaning. The selective function of an utterance for an individual may obviously be perfectly well-defined, although he can say nothing well-defined about it.

73It is a fundamental question whether metabolic stability and epigenesis require the genetic regulatory circuits to be precisely constructed.

74There can be a moral element in conceptual blindness.

75The freedom we attribute to each other is not a matter of convention, but a matter of fact.

76It has always been clear that we were not so deeply interested in the theory of any particular biological phenomenon for its own sake, but mainly in so far as it helps to a greater comprehension of the general character of the processes that go on in living as contrasted with non-living systems.

77The existence of self-reproducing machines is only a very special case of a much wider phenomenon, the theory of which might conceivably be applied to situations of quite a different kind from those occurring in biological simulation.

78Thus we have proved that the existence of two indistinguishable configurations is a necessary as well as a sufficient condition for the existence of Garden-of-Eden configurations.

79Unrestricted adaptability requires that the adaptive system be able initially to generate any of the programs of some universal computer.

80Reductionism, roughly speaking, is the view that everything in this world is really something else, and that the something else is always in the end unedifying. So lucidly formulated, one can see that this is a luminously true and certain idea.

81It is worthwhile to notice with Leach that “change is no longer something that is done to us by nature but something we can choose to do to nature and to ourselves.” Clearly, in this perspective one of the major objectives of science is to elucidate the dynamics of change. The methods described in this monograph might be one step in this direction.

Although much work remains to be done, it already clearly appears that self-organization is an emerging paradigm of science (Jantsch, 1975) emphasizing macroscopic coordination processes at many levels, in which nonlinear processes and nonequilibrium conditions play a significant role.

82The spectacular progress of molecular biology has been of utmost importance in the formulation of our approach to self-organization. Indeed, a preliminary condition for the study of the relation between function and structure is a detailed knowledge of the chemical mechanisms involved.

83Spandrels — the tapering triangular spaces formed by the intersection of two rounded arches at right angles (figure 1) — are necessary architectural by-products of mounting a dome on rounded arches. Each spandrel contains a design admirably fitted into its tapering space. An evangelist sits in the upper part flanked by the heavenly cities. Below, a man representing one of the four Biblical rivers (Tigris, Euphrates, Indus and Nile) pours water from a pitcher into the narrowing space below his feet.

The design is so elaborate, harmonious and purposeful that we are tempted to view it as the starting point of any analysis, as the cause in some sense of the surrounding architecture.

84We wish to question a deeply engrained habit of thinking among students of evolution. We call it the adaptationist programme, or the Panglossian paradigm. It is rooted in a notion popularized by A. R. Wallace and A. Weismann (but not, as we shall see, by Darwin) toward the end of the nineteenth century: the near omnipotence of natural selection in forging organic design and fashioning the best among possible worlds.

85Gödel showed that provability is a weaker notion than truth, no matter what axiomatic system is involved.

86It is an inherent property of intelligence that it can jump out of the task which it is performing, and survey what it has done; it is always looking for, and often finding, patterns.

87The fascinating thing is that any such system digs its own hole; the system’s own richness brings about its own downfall.

88The secret of self-reproducing programs — and, as we shall see, of self-reproducing molecules — is that one string functions in two ways: first as program, and second as data.

89I think; therefore I have no access to the level where I sum.

90The self comes into being at the moment it has the power to reflect itself.

91Every room is its own recording studio.

92It is a fairly widespread delusion among physicists that statistical physics is the least well-founded branch of theoretical physics. Reference is generally made to the point that some of its conclusions are not subject to rigorous mathematical proof; and it is overlooked that every other branch of theoretical physics contains just as many non-rigorous proofs, although these are not regarded as indicating an inadequate foundation for such branches.

93In principle, we can obtain complete information concerning the motion of a mechanical system by constructing and integrating the equations of motion of the system, which are equal in number to its degrees of freedom. But if we are concerned with a system which, though it obeys the laws of classical mechanics, has a very large number of degrees of freedom, the actual application of the methods of mechanics involves the necessity of setting up and solving the same number of differential equations, which in general is impracticable. It should be emphasised that, even if we could integrate these equations in a general form, it would be completely impossible to substitute in the general solution the initial conditions for the velocities and coordinates of all the particles.

At first sight we might conclude from this that, as the number of particles increases, so also must the complexity and intricacy of the properties of the mechanical system, and that no trace of regularity can be found in the behaviour of a macroscopic body. This is not so, however, and we shall see below that, when the number of particles is very large, new types of regularity appear.

94A fundamental feature of this approach is the fact that, because of the extreme complexity of the external interactions with the other parts of the system, during a sufficiently long time the subsystem considered will be many times in every possible state.

95The following circumstance is extremely important in statistical physics. The statistical distribution of a given subsystem does not depend on the initial state of any other small part of the same system, since over a sufficiently long time the effect of this initial state will be entirely outweighed by the effect of the much larger remaining parts of the system. It is also independent of the initial state of the particular small part considered, since in time this part passes through all possible states, any of which can be taken as the initial state. Without having to solve the mechanical problem for a system (taking account of initial conditions), we can therefore find the statistical distribution for small parts of the system.

The determination of the statistical distribution for any subsystem is in fact the fundamental problem of statistical physics. In speaking of ‘small parts’ of a closed system, we must bear in mind that the macroscopic bodies with which we have to deal are usually themselves such ‘small parts’ of a large closed system consisting of these bodies together with the external medium which surrounds them.

If this problem is solved and the statistical distribution for a given subsystem is known, we can calculate the probabilities of various values of any physical quantities which depend on the states of the subsystem (i.e. on the values of its coordinates q and momenta p). We can also calculate the mean value of any such quantity f(p, q), which is obtained by multiplying each of its possible values by the corresponding probability and integrating over all states.

96In practice, however, when statistical physics is applied to macroscopic bodies, its probabilistic nature is not usually apparent. The reason is that, if any macroscopic body (in external conditions independent of time) is observed over a sufficiently long period of time, it is found that all physical quantities describing the body are practically constant (and equal to their mean values) and undergo appreciable changes relatively very rarely; we mean, of course, macroscopic quantities describing the body as a whole or macroscopic parts of it, but not individual particles.

97These statistical laws resulting from the very presence of a large number of particles forming the body cannot in any way be reduced to purely mechanical laws.

98According to the results of statistics, the universe ought to be in a state of complete statistical equilibrium. Everyday experience shows us, however, that the properties of Nature bear no resemblance to those of an equilibrium system; and astronomical results show that the same is true throughout the vast region of the Universe accessible to our observation.

99It has already been mentioned in § 83 that the transition between phases of different symmetry (crystal and liquid; different crystal modifications) cannot occur in a continuous manner such as is possible for a liquid and a gas. In every state the body has either one symmetry or the other, and therefore we can always assign it to one of the two phases.

The transition between different crystal modifications is usually effected by means of a phase transition in which there is a sudden rearrangement of the crystal lattice and the state of the body changes discontinuously. As well as such discontinuous transitions, however, another type of transition involving a change of symmetry is also possible.

100It is a striking experience, especially for a nonbiologist, to attend a movie describing the development of, for example, the chicken embryo. We see the progressive organization of a biological space in which every event proceeds at a moment and in a region that make it possible for the process to be coordinated as a whole. This space is functional, not geometrical. The standard geometrical space, the Euclidean space, is invariant with respect to translations or rotations. This is not so in the biological space. In this space the events are processes localized in space and time and not merely trajectories.

101Jacques Monod has called living systems ‘these strange objects,’ and they are very strange indeed compared with the ‘nonliving’ world (Monod 1970). Thus, one of my objectives is to try to disentangle a few general features of these objects. In molecular biology there has been fundamental progress without which this discussion would not have been possible. But I wish to emphasize other aspects: namely, that living organisms are far-from-equilibrium objects separated by instabilities from the world of equilibrium and that living organisms are necessarily ‘large,’ macroscopic objects requiring a coherent state of matter in order to produce the complex biomolecules that make the perpetuation of life possible.

102It is probably not an exaggeration to say that Western civilization is time centered. Is this perhaps related to a basic characteristic of the point of view taken in both the Old and the New Testaments?

103Instead of finding stability and harmony, wherever we look, we discover evolutionary processes leading to diversification and increasing complexity. This shift in our vision of the physical world leads us to investigate branches of mathematics and theoretical physics that are likely to be of interest in the new context.

104The mere fact of adopting a certain type of modelling, e.g., choosing to describe the situation in discrete time, has already a profound influence on the outcome of the analysis.

105The two sets (periodic and aperiodic) are intimately intertwined: between any two values of μ for which the orbit is periodic, there is one for which it is aperiodic, and vice versa.

106A remarkable feature of the LMTR problem, discovered by Shannon and established in great generality by further research, is a phenomenon suggesting the heuristic interpretation that information, like liquids, ‘has volume but no shape’, i.e. the amount of information is measurable by a scalar. Just as the time necessary for conveying the liquid content of a large container through a pipe (at a given flow velocity) is determined by the ratio of the volume of the liquid to the cross-sectional area of the pipe, the LMTR equals the ratio of two numbers, one depending on the source and fidelity criterion, the other depending on the channel.

107The second law of thermodynamics implies that isolated microscopically reversible physical systems tend with time to states of maximal entropy and maximal ‘disorder.’ However, ‘dissipative’ systems involving microscopic irreversibility, or those open to interactions with their environment, may evolve from ‘disordered’ to more ‘ordered’ states. The states attained often exhibit a complicated structure. Examples are outlines of snowflakes, patterns of flow in turbulent fluids, and biological systems.

108Investigations of simple ‘self-organization’ phenomena in physical and chemical systems have often been based on the Boltzmann transport differential equations (or its analogs) for the time development of macroscopic quantities. The equations are obtained by averaging over an ensemble of microscopic states and assuming that successive collisions between molecules are statistically uncorrelated. For closed systems (with reversible or at least unitary microscopic interactions) the equations lead to Boltzmann’s H theorem, which implies monotonic evolution towards the macroscopic state of maximum entropy.

109With ‘random’ initial configurations, the irreversible character of the cellular automaton evolution leads to several self-organization phenomena.

110To discover and analyze the mathematical basis for the generation of complexity, one must identify simple mathematical systems that capture the essence of the process. Cellular automata are a candidate class of such systems. This article surveys their nature and properties, concentrating on fundamental mathematical features. Cellular automata promise to provide mathematical models for a wide variety of complex phenomena, from turbulence in fluids to patterns in biological growth. The general features of their behavior discussed here should form a basis for future detailed studies of such specific systems.

111The notorious solitaire computer game ‘Life’ (qualitatively similar in some respects to the game of ‘Go’) is an example of a two-dimensional cellular automaton.

112As a consequence of their locality, cellular automaton rules define no intrinsic length scale other than the size of a single site (or of a neighborhood of three sites) and no intrinsic time scale other than the duration of a single time step.

113Finite one-dimensional cellular automata are similar to a class of feedback shift registers.

114A particular type-II two-dimensional cellular automaton whose evolution has been studied extensively is the game of ‘Life’. The local rules take a site to ‘die’ (attain value zero) unless two or three of its neighbors are ‘alive’ (have value one). If two neighbors are alive, the value of the site is left unchanged; if three are alive, the site always takes on the value one. Many configurations exhibiting particular properties have been found. The simplest isolated configurations invariant under time evolution are the ‘square’ (or ‘block’) consisting of four adjacent live sites, and the ‘hexagon’ (or ‘beehive’) containing six live sites. ‘Oscillator’ configurations which cycle through a sequence of states are also known. The simplest is the ‘blinker’ consisting of a line of three live sites, which cycles with a period of two time steps. Oscillators with periods 3, 5, and 7 are also known; other periods may be obtained by composition. So long as they are separated by four or more unfilled sites, many of these structures may exist without interference in the configurations of a cellular automaton, and their effects are localized.

115The game of ‘Life’ is an example of a special class of ‘totalistic’ cellular automata, in which the value of a site depends only on the sum of the values of its neighbors at the previous time step, and not on their individual values. Such cellular automata may arise as models of systems involving additive local quantities, such as chemical concentrations.

116A potentially important feature of cellular automata is the capability for ‘self-reproduction’ through which the evolution of a configuration yields several separated identical copies of the configuration.

117Many of the qualitative features found for elementary cellular automata appear to survive in more complicated cellular automata, although several novel phenomena may appear. For example, in one-dimensional cellular automata with three or more possible values at each site, protective membranes may be generated which shield finite regions from the effects of external noise, and allow very regular patterns to grow from small seeds.

118Cellular automata may be viewed as computers, with initial configurations considered as input programs and data processed by cellular automaton time evolution. Sufficiently complicated cellular automata are known to be universal computers, capable of computing any computable function given appropriate input. Such cellular automata may be considered as capable of the most complicated behavior conceivable and are presumably capable of simulating any physical system given a suitable input encoding and a sufficiently long running time. In addition, they may be used to simulate the evolution of any other cellular automaton. If the necessary encoding is sufficiently simple, the statistical properties of the simulated cellular automaton should follow those of the universal cellular automaton. Although not capable of universal simulation, simpler cellular automata may often simulate each other. This capability may well form a basis for the universality found in the statistical properties of various cellular automata.

119The theory of mutually entraining oscillators has a potential importance comparable to the theory of general cooperative phenomena which has been one of the central subjects of statistical and solid-state physics.

120Actually, even if an ingenious experimenter were to devise wholly quantum apparatus, in the end the observer himself and his observational instruments would necessarily still themselves be solid, broken-symmetry objects; otherwise, the pointers wouldn’t point, the recorders wouldn’t record, etc.

121This is an exercise in fictional science, or science fiction, if you like that better. Not for amusement: science fiction in the service of science. Or just science, if you agree that fiction is part of it, always was, and always will be as long as our brains are only minuscule fragments of the universe, much too small to hold all the facts of the world but not too idle to speculate about them.

I have been dealing for many years with certain structures within animal brains that seemed to be interpretable as pieces of computing machinery because of their simplicity and/or regularity. Much of this work is only interesting if you are yourself involved in it. At times, though, in the back of my mind, while I was counting fibers in the visual ganglia of the fly or synapses in the cerebral cortex of the mouse, I felt knots untie, distinctions dissolve, difficulties disappear, difficulties I had experienced much earlier when I still held my first naive philosophical approach to the problem of the mind. This process of purification has been, over the years, a delightful experience. The text I want you to read is designed to convey some of this to you, if you are prepared to follow me not through a world of real brains but through a toy world that we will create together.

122Valentino Braitenberg is a cybernetician, a neuroanatomist, and a musician. He seeks to understand how the beautiful structures of the brain constitute a machine that can enable us to exhibit such skilled behavior as that involved in playing music. Since the early 1960s, I have turned to Valentino for detailed neuroanatomy and for lively essays that cut away the technical details to illuminate the key issues of what we may call cybernetics or artificial intelligence or cognitive science.

123At this point we are ready to make a fundamental discovery. We have gathered evidence for what I would like to call the ‘law of uphill analysis and downhill invention.’ What I mean is this. It is pleasurable and easy to create little machines that do certain tricks. It is also quite easy to observe the full repertoire of behavior of these machines - even if it goes beyond what we had originally planned, as it often does. But it is much more difficult to start from the outside and to try to guess internal structure just from the observation of behavior.

124This story is quite old and goes by the name of Darwinian evolution. Many people don’t like the idea that everything beautiful and marvelous in organic nature should be due to the simple cooperation of reproduction, errors, and selection. This is no problem for us. We have convinced ourselves that beautiful, marvelous, and shrewd machines can be made out of inorganic matter by this simple trick. Moreover, we already know that analysis is much more difficult than synthesis. Where there has been no conscious engineering at all, as in the case of our type 6 vehicles, analysis will necessarily produce the feeling of a mysterious supernatural hand guiding the creation. We can imagine that in most cases our analysis of brains in type 6 vehicles would fail altogether: the wiring that produces their behavior may be so complicated and involved that we will never be able to isolate a simple scheme. And yet it works.

125Now it is different with type 14 vehicles. They move through their world with consistent determination, always clearly after something that very often we cannot guess at the outset - something that may not even be there when the vehicle reaches the place it wants to get to. But it seems to be a good strategy, this running after a dream. Most of the time the chain of optimistic predictions that seems to guide the vehicle’s behavior proves to be correct, and Vehicle 14 achieves goals that Vehicle 13 and its predecessors ‘couldn’t even dream of.’ The point is that while the vehicle goes through its optimistic predictions, the succession of internal states implies movements and actions of the vehicle itself. While dreaming and sleepwalking, the vehicle transforms the world (and its own position in the world) in such a way that ultimately the state of the world is a more favorable one.

We observe at some stage how one of the vehicles of type 14 is waiting for another vehicle to appear. This other vehicle carries a very appealing source which Vehicle 14 intends to tap. It seems to be waiting impatiently, since every now and then it performs the motions that are associated with the tapping, as if by anticipating its own behavior in the presence of the desired event, it could accelerate the event’s occurrence. ‘This is very human,’ we say. ‘Haven’t we all felt an urge to run to the door long before the doorbell rings, when waiting impatiently for a beloved friend?’ Indeed, it is aberrant behavior dictated by a very subjective law of causality, but it does seem to reflect a basic attitude of humankind, this irrational belief in the effectiveness of one’s own actions.

126The more there is of the quality to which the sensor is tuned, the faster the motor goes.

127Kolmogorov complexity and Shannon entropy both increase monotonically with disorder, making them measures of randomness rather than complexity. When we call a random pattern ‘not complex,’ we are implicitly assigning it to an ensemble, and it is the ensemble that is simple.

128The experimental approaches to the question of the anatomical nature of the engram are all marred by a fundamental difficulty. (Engrams are the memory traces postulated many years ago by psychologists, long before there was any hope of ever finding one in the brain.) Suppose we have some idea about the anatomical changes responsible for memory and want to prove it. We present some input to one animal, but not to another animal, in order to use it as a control. It is not that the control animal has experienced nothing while the experimental animal received its input: it has had its own experiences and its own thoughts. In order to compare the traces left in the brains of the two animals by the two different inputs, we would have to know exactly where the information ended up in the two animals. In truth we don’t. Most likely, two inputs that may have entirely different meanings are represented in the brain in quite the same way, as diffuse patterns of activity in an enormous network of neurons.

Suppose the engram were embodied in changes that are very easily visible in the electron microscope, or possibly in the light microscope: a change in the thickness of axonal terminals, a change in the number of synaptic vesicles, or a change in the amount of pre- or postsynaptic thickening. No matter what kind of information we presented to the animal, in the end we would expect to see some synapses of one kind and some of the other, for information must be represented in a pattern of elements in different states, or else it wouldn’t be information at all. But different patterns can only be distinguished if they are understood in every detail. In other words, they cannot be distinguished at the present stage of our knowledge of the brain.

129Vitalism amounted to the assertion that living things do not behave as though they were nothing but mechanisms constructed of mere material components; but this presupposes that one knows what mere material components are and what kind of mechanisms they can be built into.

130Artificial Life is the study of man-made systems that exhibit behaviors characteristic of natural living systems. It complements the traditional biological sciences concerned with the analysis of living organisms by attempting to synthesize life-like behaviors within computers and other artificial media. By extending the empirical foundation upon which biology is based beyond the carbon-chain life that has evolved on Earth, Artificial Life can contribute to theoretical biology by locating life-as-we-know-it within the larger picture of life-as-it-could-be.

131Certainly life, as a dynamic physical process, could ‘haunt’ other physical material: the material just needs to be organized in the right way. Just as certainly, the dynamic processes that constitute life — in whatever material bases they might occur — must share certain universal features — features that will allow us to recognize life by its dynamic form alone, without reference to its matter.

Since it is quite unlikely that organisms based on different physical chemistries will present themselves to us for study in the foreseeable future, our only alternative is to try to synthesize alternative life-forms ourselves: Artificial Life: life made by man rather than by nature.

132Whereas biology has largely concerned itself with the material basis of life, Artificial Life is concerned with the formal basis of life. Biology has traditionally started at the top, viewing a living organism as a complex biochemical machine, and worked analytically downwards from there — through organs, tissues, cells, organelles, membranes, and finally molecules — in its pursuit of the mechanisms of life. Artificial Life starts at the bottom, viewing an organism as a large population of simple machines, and works upwards synthetically from there — constructing large aggregates of simple, rule-governed objects which interact with one another nonlinearly in the support of life-like, global dynamics.

133The ‘key’ concept in AL is emergent behavior. Natural life emerges out of the organized interactions of a great number of nonliving molecules, with no global controller responsible for the behavior of every part. Rather, every part is a behavior itself, and life is the behavior that emerges from out of all of the local interactions among individual behaviors. It is this bottom-up, distributed, local determination of behavior that AL employs in its primary methodological approach to the generation of lifelike behaviors.

134Artificial Life involves the realization of lifelike behavior on the part of man-made systems consisting of populations of semi-autonomous entities whose local interactions with one another are governed by a set of simple rules. Such systems contain no rules for the behavior of the population at the global level, and the often complex, high-level dynamics and structures observed are emergent properties, which develop over time from out of all of the local interactions among low-level primitives by a process highly reminiscent of embryological development, in which local hierarchies of higher-order structures develop and compete with one another for support among the low-level entities. These emergent structures play a vital role in organizing the behavior of the lowest-level entities by establishing the context within which those entities invoke their local rules and, as a consequence, these structures may evolve in time.

135Several people raised the question of establishing a general measure of complexity, adaptability, or progress in evolutionary processes. Such a measure — or measures — would certainly be valuable, and there have been efforts to apply Shannon’s entropy concept and Chaitin’s algorithmic complexity measures to living systems and the process of evolution — with somewhat mixed results! The field awaits an appropriate, natural measure of complexity, as do many other fields.

136There was little discussion of possible practical applications of Artificial Life.

137Nor was there much discussion of the moral or ethical issues involved in the pursuit of Artificial Life. Such issues must be discussed before we go much further down the road to creating life artificially. We are once again at a point where our technical grasp of a problem is far ahead of our moral understanding of the issues involved, or of the possible consequences of mastering the technology.

138It is, perhaps, not coincidental that the first workshop on Artificial Life was held at Los Alamos, site of the mastery of atomic fission and fusion. Both technologies have tremendous potential for benefiting life on earth, but both also have tremendous potential for abuse, whether intentional or accidental. Whereas the technology of death was developed in secret, under government mandate, and with little official attention to social and moral consequences, this first workshop on the technology of life was held in the open, by voluntary participation, and we must see to it that it is pursued with the most careful attention to social and moral consequences.

139A cellular automata machine is a universe synthesizer.

140Cellular automata are stylized, synthetic universes defined by simple rules much like those of a board game. They have their own kind of matter which whirls around in a space and a time of their own. One can think of an astounding variety of them. One can actually construct them, and watch them evolve. As inexperienced creators, we are not likely to get a very interesting universe on our first try.

141In a cellular automaton, there are no ‘particles’ — only bits. Yet when we watch the screen, we irresistibly see objects that move, collide, bounce, and scatter. The question of what constitutes an ‘object’ in a CA, and what it means for that object to ‘move,’ is more subtle than it first appears. Identity is not given by the substrate but by the pattern.

142In principle, evolution, life, and intelligence can take place within a world governed by a very simple cellular-automaton rule.

143In a cellular automaton, objects that may be interpreted as passive data and objects that may be interpreted as computing devices are both assembled out of the same kind of structural elements and subject to the same fine-grained laws; computation and construction are just two possible modes of activity.

144Gleick’s Chaos is not only enthralling and precise, but full of beautifully strange and strangely beautiful ideas.

145The classification of the constituents of a chaos, nothing less here is essayed.

146When Mitchell Feigenbaum began thinking about chaos at Los Alamos, he was one of a handful of scattered scientists, mostly unknown to one another. A mathematician in Berkeley, California, had formed a small group dedicated to creating a new study of ‘dynamical systems.’ A population biologist at Princeton University was about to publish an impassioned plea that all scientists should look at the surprisingly complex behavior lurking in some simple models. A geometer working for IBM was looking for a new word to describe a family of shapes - jagged, tangled, splintered, twisted, fractured - that he considered an organizing principle in nature. A French mathematical physicist had just made the disputatious claim that turbulence in fluids might have something to do with a bizarre, infinitely tangled abstraction that he called a strange attractor.

147Chaos poses problems that defy accepted ways of working in science. It makes strong claims about the universal behavior of complexity. The first chaos theorists, the scientists who set the discipline in motion, shared certain sensibilities. They had an eye for pattern, especially pattern that appeared on different scales at the same time. They had a taste for randomness and complexity, for jagged edges and sudden leaps. Believers in chaos - and they sometimes call themselves believers, or converts, or evangelists - speculate about determinism and free will, about evolution, about the nature of conscious intelligence. They feel that they are turning back a trend in science toward reductionism, the analysis of systems in terms of their constituent parts: quarks, chromosomes, or neurons. They believe that they are looking for the whole.

148Relativity eliminated the Newtonian illusion of absolute space and time; quantum theory eliminated the Newtonian dream of a controllable measurement process; and chaos eliminates the Laplacian fantasy of deterministic predictability.

149The simplest systems are now seen to create extraordinarily difficult problems of predictability. Yet order arises spontaneously in those systems - chaos and order together. Only a new kind of science could begin to cross the great gulf between knowledge of what one thing does - one water molecule, one cell of heart tissue, one neuron - and what millions of them do.

150Sudden cardiac death is a problem in topology.

151The singularity is a state, not merely a stimulus.

152The rotor is a singularity incarnate in a physical medium.

153If we think of complexity as a physical property of an object (such as mass or entropy) then there is a puzzle. Objects can be copied. A bull is a complex object. Are seven bulls seven times as complex as one bull? Can complexity proliferate so cheaply?

154It took billions of years for the earth to evolve one bull; but one bull and a few compliant cows will produce seven bulls relatively speedily.

155The algorithmic definition of complexity is really a definition of randomness (a profound one).

156Take a glass and smash it with a hammer. Keep on smashing, and you will in short order create a mass of shards that contains an amount of mutual information greater than that between all the DNA in the human body. Since a bowl of glass dust ought not to be more complex than human genes, mutual information ought not to be considered a measure of complexity.

157The value of a message appears to reside not in its information (its absolutely unpredictable parts), nor in its obvious redundancy (verbatim repetitions, unequal digit frequencies), but rather in what might be called its buried redundancy — parts predictable only with difficulty, things the receiver could in principle have figured out without being told, but only at considerable cost in money, time, or computation.

158If a computation is encoded as an attractor of a dynamical system, then small perturbations are automatically corrected as the system relaxes back to the attractor. The system is self-repairing.

159A cat following a bird with its eyes need not do floating-point arithmetic to know where to pounce.

160The genetic complexity of an organism is proportional to the amount of genetic information tried out and discarded by the process of natural selection on the ancestors of the organism.

161In general, when sufficient physical insight is lacking and one makes more or less arbitrary probabilistic assumptions about the data and proceeds to make logical deductions, the results must be considered irrelevant to the task at hand, which is to learn from the data.

162Periodic behavior has low information content; purely random behavior has high information content; but both are simple. The interesting behavior lies between these extremes, where a process is an amalgam of regular and stochastic components.

163It is my experience from talks I have given on the subject that a person well indoctrinated in traditional statistical thinking may have greater difficulty in understanding the new concepts than one who is not so blessed.

164A chaotic map possesses three ingredients: unpredictability, indecomposability, and an element of regularity.

165If F has a periodic point of period 3, then F has periodic points of all periods.

166The period-doubling route to chaos is one of the most commonly observed transitions from simple to complicated behavior in parameterized families of maps. It is also universal: the same constants arise for all families.

167Life breaks free. Life expands to new territories. Painfully, perhaps even dangerously. But life finds a way.

168The specter of information is haunting sciences.

169It from bit. Otherwise put, every it — every particle, every field of force, even the spacetime continuum itself — derives its function, its meaning, its very existence entirely — even if in some contexts indirectly — from the apparatus-elicited answers to yes-or-no questions, binary choices, bits.

170Build physics, with its false face of continuity, on bits of information!

171That seminal workshop gave birth to the growing field of Artificial Life. Since that first workshop, a large number of people have become interested in the field and its methodological approaches and have initiated new research projects. Many of these new research projects were reported at the Second Artificial Life Workshop, held in February 1990, in Santa Fe, New Mexico. This volume constitutes the proceedings of that second workshop.

It is difficult to compare the two workshops. We will never recapture the novelty and excitement of the first workshop. It was the first time many of the participants became aware of the true depth and breadth of the questions and techniques they had been toying with until then. For many of us, it was the first confirmation that we were not crazy — that there was a solid basis for the line of inquiry we had all been pursuing in isolation: studying biological phenomena without studying biological things.

172The primary purpose of the first workshop was to find out what kinds of approaches researchers were pursuing in their attempts to simulate or even synthesize life, evolution, ecological dynamics, and so forth. Another goal was to find out what kinds of fundamental biological questions were most appropriately addressed by such techniques.

The proceedings from that first workshop constituted an overview of the possibilities. Although there were no major theoretical breakthroughs reported in any of the papers, and many of the research efforts were clearly in the very earliest stages of development, it was easy to see the power implicit in these early tentative experiments. However, it was not easy to predict the direction in which the field would progress from such a start.

These proceedings of the 1990 Artificial Life Workshop provide a second data point on the direction in which the field of Artificial Life is progressing, and I am very pleased with the trajectory that it appears to be on. Overall, there is more good science and less “gee-whiz” than in the first proceedings. This is not to put down those first Proceedings—they were exactly what they had to be. However, it is clear that the field has matured a good deal in the intervening two years, and this is a sign of more good things to come.

173The essence of the Artificial Life approach was captured by A. W. Burks, in a description of von Neumann’s investigations on the phenomenon of self-reproduction:

‘What kind of logical organization is sufficient for an automaton to reproduce itself? This question is not precise and admits to trivial versions as well as interesting ones. Von Neumann had the familiar natural phenomenon of self-reproduction in mind when he posed it, but he was not trying to simulate the self-reproduction of a natural system at the level of genetics and biochemistry. He wished to abstract from the natural self-reproduction problem its logical form.’ [emphasis added]

174Chris Langton is a cellular-automata engineer. He’s following in the footsteps of John von Neumann, engineering self-reproducing systems with cellular automata. Von Neumann showed that it was possible to hand-engineer a self-replicating pattern — engineer a universe — by writing down a particular cellular automaton with a certain set of rules. He was able to show that there are patterns in that universe that replicate themselves, and are both construction universal and computation-universal. What that means is that the pattern is capable of constructing any pattern, and, secondly, that it’s capable of making any computation that a computer can do.

175The Hungarian mathematician John Von Neumann had the insight that we could learn a lot even if we didn’t try to model some specific existing biological thing. He went after the logical basis, rather than the material basis, of a biological process, by attempting to abstract the logic of self-reproduction without trying to capture the mechanics of self-reproduction (which were not known in the late 1940s, when he started his investigations).

Von Neumann demonstrated that one could have a machine, in the sense of an algorithm, that would reproduce itself. Most biologists weren’t interested, because it wasn’t like any specific instance of biological self-reproduction (it wasn’t a model of chromosomes, for example). Von Neumann was able to derive some general principles for the process of self-reproduction. For instance, he determined that the information in a genetic description, whatever it was, had to be used in two different ways: (1) it had to be interpreted as instructions for constructing itself or its offspring, and (2) it had to be copied passively, without being interpreted. This turned out to be the case for the information stored in DNA when James Watson and Francis Crick determined its structure in 1953. It was a far-reaching and very prescient thing to realize that one could learn something about “real biology” by studying something that was not real biology — by trying to get at the underlying “bio-logic” of life.

176To understand the breadth and scope of the field of Artificial Life, it is only necessary to note that we can replace the references to ‘self-reproduction’ above with references to many other biological phenomena, including: the origin of life, molecular self-assembly, embryogenesis (including growth, development, and differentiation), animal behavior (ethology), insect-colony dynamics, evolution, speciation, ecological dynamics, and even linguistic and socio-cultural evolution.

In addition to providing new ways to study the biological phenomena associated with life here on Earth, life-as-we-know-it, Artificial Life allows us to extend our studies to the larger domain of the ‘bio-logic’ of possible life, life-as-it-could-be, whatever it might be made of and wherever it might be found in the universe.

Thus, Artificial Life is not only about studying existing life, but also about the possibility of synthesizing new life, within computers or other ‘artificial’ media. The life that is realized in these alternative media will force us to broaden our understanding of the proper domain of biology to include self-organizing, evolving, and even ‘living’ machines, regardless of the specific physical stuff of which they are constituted, or whether or not they are based upon the same chemical and physical principles as the life that has evolved here on Earth.

177The talks at the second workshop were arranged to reflect the chronological and hierarchical ordering evident in the natural world. Thus, talks treating the origin of life came first, followed by talks on the evolutionary process, on development and differentiation, learning, computer life, ecological dynamics, and finally to discussions on the future evolution of life. Progressing through the material in this natural sequence continuously reinforces the ‘emergent’ nature of biological structures and functions, as the phenomena under discussion on one day can be seen to have emerged ‘on the shoulders,’ so to speak, of the phenomena under discussion the day before.

178Modern organisms owe their structure to the complex process of biological evolution, and it is very difficult to discern which of their properties are due to chance, and which to necessity. If biologists could ‘re-wind the tape’ of evolution and start it over, again and again, from different initial conditions, or under different regimes of external perturbations along the way, they would have a full ensemble of evolutionary pathways to generalize over. Such an ensemble would allow them to distinguish universal, necessary properties (those which were observed in all the pathways in the ensemble) from accidental, chance properties (those which were unique to individual pathways). However, biologists cannot rewind the tape of evolution, and are stuck with a single, actual evolutionary trace out of a vast, intuited ensemble of possible traces.

179We consider co-evolution occurring among species each of which is itself adapting on a rugged, multipeaked fitness landscape. But the fitness of each genotype, via the phenotype, of each species is affected by the genotypes via the phenotypes of the species with which it is coupled in the ecosystem. Adaptive moves by one co-evolutionary partner, therefore, may change the fitness and the fitness landscapes of the co-evolutionary partners. Anecdotally, development of a sticky tongue by the frog alters the fitness of the fly, and what it ought to do: it should develop slippery feet. In this framework, adaptive moves by any partner may deform the fitness landscapes of other partners.

180One interesting result is that co-evolutionary systems seem to achieve a ‘poised’ state ‘at the edge of chaos,’ lending support to an idea originally due to Norman Packard and Chris Langton, that evolutionary dynamics will bring populations to the vicinity of a phase transition between ordered and disordered behaviors.

181After hearing Kristian Lindgren’s talk at the workshop, many researchers went back to their models, took another look at the ongoing evolutionary dynamics, and discovered a similar pattern of long periods of stasis punctuated by brief periods of rapid evolutionary change. I think a lot of these researchers kicked themselves for not having noted this feature in their data earlier, a consequence of concentrating on the products of the process, rather than on the process itself.

182Evolutionary adaptation can be viewed as a kind of ‘learning’ that takes place on time scales much longer than the lifetimes of individual organisms. Traditional computational learning methods, such as back-propagation applied to neural nets, often require that each action of the learning system be evaluated by a ‘teacher’ or ‘critic,’ who ‘knows’ what the correct action should have been, and provides the learning system with an evaluation of its performance, ranging from a simple ‘yes’ or ‘no’ to providing it with the ‘correct’ answer. However, in many real learning situations, there is no ‘correct’ answer, or even if there is, it may be computationally intractable to determine what it is, so there can be no teacher that provides the system with the correct feedback. In such learning situations, neural networks perform rather poorly.

183The paper by David Ackley and Michael Littman employs an adaptation strategy they call Evolutionary Reinforcement Learning (ERL). ERL ‘combines genetic evolution with neural network learning, and an artificial life “ecosystem” called AL, within which populations of ERL-driven adaptive “agents” struggle for survival.’ During the course of their simulations, they observe that the best performance occurs when both evolutionary learning and short-term learning are employed. More importantly, however, they observe the occurrence of a number of subtle effects at the level of population genetics that appear to enhance overall learning, such as the classic ‘Baldwin effect,’ a mechanism by which changes in behavior learned over the short term can eventually be transferred to the ‘genetics’ of long-term learning.

184Multicellular organisms start life as a single cell. Through repeated cell divisions, cell movements, and cellular differentiation, the mature organism develops via the complex process known as morphogenesis. The mechanics of the process by which the growing body of cells shapes itself and produces all the right kinds of cells in all the right places is still largely a mystery. All the cells contain the same genetic program, and it is not well understood how the different types of cells diverge into different domains of genetic program space.

185Lindenmayer systems (L-systems) have found broad application within the Artificial Life community for analyzing the complex process of morphogenesis. The use of L-systems is in keeping with the overall spirit of the Artificial Life approach. L-systems abstract the ‘logical form’ of the natural problem of morphogenesis. They allow researchers to work and experiment with systems that grow, develop, and differentiate like real organisms, but which are not, themselves, real organisms. They are another instance of the general class of growing, developing, and differentiating things. Therefore, by studying them, and by comparing and contrasting them with the growth, development, and differentiation of living systems, we can learn more about the ‘real’ system by understanding the general class of systems it belongs to.

Sadly, Aristid Lindenmayer, the inventor of L-systems, recently passed away after a battle with cancer. However, he has inspired many researchers to carry on his work.

186It should be clear by this point that representation is a major issue in Artificial Life research. A representation scheme for implementing algorithms that is understandable by human beings (such as a familiar programming language like Pascal or Fortran) may not be amenable to manipulation by genetic algorithms. Thus, programmable representations may not be evolvable, and evolvable representations may not be programmable. Many of the research efforts reported in these proceedings employ representations that were chosen for their evolvability, not for their programmability.

John Koza attempts to effect a compromise between programmability and evolvability. By applying a genetic algorithm to the parse tree of a program (rather than to its linear representation as a bit-string), and by respecting syntactic categories, Koza is able to achieve evolution within the context of a programmable language. In the context of the tree representation of a program, the crossover operator of the genetic algorithm is allowed to swap subtrees between two parent programs when the roots of those subtrees belong to the same syntactic category. In Koza’s system, all subtrees are S-expressions in Lisp. Since all S-expressions belong to the same syntactic category, any subtree can be interchanged with any other. Koza has applied this system to evolve programs to solve a number of hard problems, including a number of Artificial Life problems.

187If we want to understand what makes symbols meaningful (and related phenomena such as intentionality), then AI — at least as currently pursued — will not do. If we want genuine meaning and original intentionality, then communication must have real relevance to the communicators. Furthermore, if we are to understand the pragmatic context of the communication and preserve ecological validity, then it must occur in the communicators’ natural environment, that to which they have become coupled through natural selection. Unfortunately, the natural environments of biological organisms are too complicated for carefully controlled experiments.

188Von Neumann’s studies of self-reproduction involved the construction of structures which were not at all like any known living things. Yet, through their study, he was able to determine principles which are likely to be true of any self-reproducing system, including known biological organisms. For instance, he found that the genetic description of an organism must be used in two different ways: 1) interpreted as instructions to construct its offspring, and 2) uninterpreted as data to be copied to create duplicates of the description to be passed on to its offspring. This was found to be the case for biological organisms once Watson and Crick revealed the workings of the DNA molecule.

Von Neumann’s proof of the possibility of machine self-reproduction is achieved via a book-length constructive proof. The paper by Alvy Ray Smith provides a one-page proof of the possibility of machine self-reproduction. Whereas von Neumann required both computation universality and construction universality of his self-reproducing machines, Smith shows that computational universality alone suffices. Smith’s proof relies on an elegant application of the famous Recursion Theorem of recursive function theory.

189Another class of computer life-forms is already in existence, and is, in fact, becoming quite a problem. Computer viruses have been in existence for approximately ten years now, and since the introduction of the first viruses, not a single species has been eradicated. In fact, more sophisticated computer viruses are being introduced more and more rapidly, and the problem promises to become epidemic in the near future.

Biologists argue over whether or not biological viruses can be considered to be alive. Although they reproduce, they have no metabolism of their own to produce the required parts, and must take over the metabolic and reproductive machinery of a host cell in order to produce offspring. Allowing for the different medium, computer viruses appear to be every bit as much alive as biological viruses. The paper by Eugene Spafford presents an overview of the workings of different classes of computer viruses and argues that the release of computer viruses onto a network is as morally reprehensible as would be the release of biological viruses into a public reservoir.

190The debate about the possibility of machine life will have something in common with the debate about the possibility of machine intelligence. Both AI and AL study their respective subject matter by attempting to realize it within computers. Although AI has not yet achieved anything that even its most ardent supporters would call genuine machine intelligence, AI has completely changed the way in which scientists think about what it is to be ‘intelligent,’ and has, therefore, made a major scientific contribution, even though it hasn’t achieved its overall goal.

Similarly, Artificial Life will force us to rethink what it is to be ‘alive.’ The fact is, we have no commonly agreed upon definition of ‘the living state.’ When asked for a definition, biologists will often point to a long list of characteristic behaviors and features shared by most living things (such as the list collated by Mayr) which includes things like self-reproduction, metabolic activity, mortality, complex organization and behavior, etc. However, as most such lists are constituted of strictly behavioral criteria, it is quite possible that we will soon be able to exhibit computer processes that exhibit all of the behaviors on such a list. When that happens, we will have two choices, either 1) admit that the computer process is alive, or (more likely!) 2) change the list so as to exclude the computer process from being considered alive. Part (but only part) of the failure of AI to achieve ‘intelligence’ is due to the fact that it is chasing a moving target. It used to be thought that playing chess required true intelligence. Now that computers can beat most humans at it, the ability to play chess is no longer considered an indicator of intelligence.

191There is a caution here that we all must attend to. Attempts to create Artificial Life may be pursued for the highest scientific and intellectual goals, but they may have very devastating consequences in the real world, if researchers do not take care to insure that the products of their research cannot ‘escape,’ either into computer networks or into the biosphere itself.

192What will the world be like when the means to produce self-reproducing robots is as widely available as the means to produce self-reproducing computer programs?

193The paper by Peter Cariani addresses the difficult concept of ‘emergence,’ a central, but poorly defined, concept in many fields, including Artificial Life. Cariani identifies the different senses in which the term ‘emergence’ is typically used, and argues that computers cannot exhibit ‘genuinely’ emergent processes. Rather, Cariani argues, only nature can support emergence in the most important sense of the term. Although his conclusions are in opposition to some of the beliefs held by many Artificial Life researchers, he is clearly correct that our usage of the term ‘emergence’ must be more carefully qualified, if it is to have any real meaning or utility in the theories that we ultimately construct.

194These organisms will evolve in a fundamentally different manner than contemporary biological organisms, since their reproduction will be under at least partial conscious control, giving it a Lamarckian component. The pace of evolutionary change consequently will be extremely rapid. The advent of artificial life will be the most significant historical event since the emergence of human beings. The impact on humanity and the biosphere could be enormous, larger than the industrial revolution, nuclear weapons, or environmental pollution. We must take steps now to shape the emergence of artificial organisms; they have potential to be either the ugliest terrestrial disaster, or the most beautiful creation of humanity.

195With the advent of artificial life, we may be the first species to create its own successors. What will these successors be like? If we fail in our task as creators, they may indeed be cold and malevolent. However, if we succeed, they may be glorious, enlightened creatures that far surpass us in their intelligence and wisdom. It is quite possible that, when the conscious beings of the future look back on this era, we will be most noteworthy not in and of ourselves but rather for what we gave rise to. Artificial life is potentially the most beautiful creation of humanity. To shun artificial life without deeper consideration reflects a shallow anthropocentrism.

196A simple and brief proof of the existence of nontrivial self-reproducing machines, as cellular automata (CA) configurations, is presented, which relies only on computation universality. Earlier proofs are book length and rely on “construction universality.” Furthermore, simple CA are shown to support nontrivial self-reproduction—hence, simultaneously simple and nontrivial.

197John von Neumann introduced the concept in a book which also created the field of CA theory (from a suggestion by Stanislaw Ulam). We will use the term ‘machine’ in the same sense as von Neumann — a finite collection of non-quiescent cells in a CA. Both his CA (29-state cells) and his self-reproducing configuration (40,000 or so nonquiescent cells) were complex and inspired others to simplify.

198There has been a revival in interest in CA theory since the combined advent of chaos, fractals, and interactive computer graphics in the 1980s, inspired principally by Wolfram and the other authors. There is a surprising ignorance by much of the post-revival literature of the hundreds of papers written during the preceding 20 years or so. Part of the problem is the lack of knowledge that CA had many other names then: iterative arrays, tessellation automata, cellular spaces, modular arrays, polyautomata, etc. The references herein can be used as pointers into this literature.

199The recent revival of interest in CA has concentrated on the totalistic case, where the transition function is required to be an arithmetic function of the neighborhood states. A result that has been rediscovered numerous times is that trivial self-reproduction — that is, self-replication — is possible in totalistic CA.

200Scaling is the observation; the renormalisation group is the explanation.

201An ice cube floating in a glass of water, at just the melting point, poses a genuine conceptual problem in theoretical physics.

202The phenomenon of critical opalescence is one of the most visually dramatic manifestations of critical phenomena. Normally transparent fluids become milky white near the critical point, because density fluctuations occur on all length scales, including those comparable to the wavelength of visible light.

203Imagine poor Ising’s disappointment on deriving this result!

204Symmetry-breaking is a paradox only if you confuse the symmetry of the underlying laws with the symmetry of the things that those laws govern.

205Symmetry is not a number, not a shape, not a property: it is a transformation — an action — that leaves the thing apparently unchanged.

206The Geometer God constructed the universe using symmetry as a guide: then She broke it.

207Anomalous dimensions are the fingerprints of the microscopic world, persisting at macroscopic scales through the collective action of fluctuations at all intermediate scales.

208Life is an expected, emergent, collective property of complex systems of polymer catalysts.

209Ordered systems are too frozen to coordinate complex behavior; chaotic systems are too wild to coordinate complex behavior; systems near the edge of chaos are most likely to coordinate complex behavior.

210Again, remarkably, coevolving systems may optimize their capacity to coevolve by mutually attaining the edge of chaos.

211MALCOLM

I’m a chaotician. (deprecating smile)
That’s what they call us. Mathematicians
who study chaos theory.

212Our experiment produced very different results, and we suggest that the interpretation of the original results is not correct.

213Thus for this class of computational tasks, the lambda_c values associated with an ‘edge of chaos’ are not correlated with the ability of rules to perform the task.

214The great question of developmental theory is this: Do the self-organizing structures of living organisms paint themselves into three-dimensional pictures by joining together structurally, bit by bit in building-block or jigsaw-puzzle fashion, in sufficient strength to overcome the hurricane, or does development at some stage make use of the hurricane and produce order out of chaos in the Boltzmann-statistical manner?

215Anyone can feel this wind when hearing the remarkably evocative music played. Only a fully trained musician can feel it when looking silently at the printed score. When one contemplates an electron micrograph, and then a diagram of chemical changes in biochemical cycles, one should feel first the storm which raises the question and then the great determined flows through it which shape the organism.

216The terms in equations have functions in the organism, as groups of amino acids do. Here is an alpha-helix; it helps to make the protein span a membrane. Here is a cubic term; it helps the organism to make stripes.

217Scientific theory is an art in which the surface on which to elaborate the details is not available a priori, but is itself a product of the enterprise.

218From a theoretical point of view, since we are not near equilibrium there is no a priori reason to suppose that we have a Gibbs ensemble or a free energy functional whose minima yield the patterns obtained under given external conditions.

219The most interesting aspect of mollusk shell patterns is that they are generated one row at a time, during growth, so that the two-dimensional pattern on the shell surface represents a space-time record of a one-dimensional solution of the model equations.

220If we take selection as the sole source of order, it is because we have come to suppose that without selection there could be only chaos.

221In sufficiently complex systems, selection cannot avoid the order exhibited by most members of the ensemble.

222Thus I suspect that the same principles of self-organization apply to the emergence of a protometabolism. I suggest that the formation of a connected web of metabolic transformations arises almost inevitably in a sufficiently complex system of organic molecules and polymer catalysts. This view implies that, from the outset, life possessed a certain inalienable holism. It also suggests that almost any metabolic web, were life to evolve again, would have a very similar statistical structure.

223The alternative attractors in a genomic regulatory network, for example, can be interpreted as the alternative cell types in the organism.

224Real morphogenesis is due not to the unfolding of any single developmental mechanism but to the beautifully ordered unfolding in time and space of some richly integrated combination of simpler mechanisms such as cell-sorting, sheet-folding, positional discontinuities, and reaction-diffusion mechanisms.

225The usual biologists’ explanations are fundamentally hierarchical. You don’t try to get back to the boss and change his mind. The essence of a Turing model is the existence of feedback loops in which everything interacts back on everything else and there is no boss.

226In short, the capacity to evolve is itself subject to evolution and may have its own lawful properties.

227I believe this to be a genuinely fundamental restraint facing adaptive evolution. As systems with many parts increase both the number of those parts and the richness of interactions among the parts, it is typical that the number of conflicting design constraints among the parts increases rapidly. Those conflicting constraints imply that optimization can attain only ever poorer compromises. No matter how strong selection may be, adaptive processes cannot climb higher peaks than afforded by the fitness landscape. This limitation cannot be overcome by stronger selection.

228As in New England, in rugged landscapes ‘you can’t get there from here.’

229Evolution is not just ‘chance caught on the wing.’ It is not just a tinkering of the ad hoc, of bricolage, of contraption. It is emergent order honored and honed by selection.

230The fundamental order lies deeper, the routes to life are broader.

231To suppose, as I do, that such an intellectual task may one day be achieved is, among other things, to suspect with quiet passion that below the particular teeming molecular traffic in each cell lie fundamental principles of order any life would reexpress.

232Contrary to intuition, morphogenesis may be deeply robust. Organisms, rather than being tinkered-together contraptions, may exhibit a nearly inevitable and stable order.

233Genetic algorithms permit virtual entities to be created without requiring an understanding of the procedures or parameters used to generate them. The measure of success, or fitness, of each individual can be calculated automatically, or it can instead be provided interactively by a user.

234It is convenient to use the biological terms genotype and phenotype when discussing artificial evolution. A genotype is a coded representation of a possible individual or problem solution. In biological systems, a genotype is usually composed of DNA and contains the instructions for the development of an organism. Genetic algorithms typically use populations of genotypes consisting of strings of binary digits or parameters. These are read to produce phenotypes which are then evaluated according to some fitness criteria and selectively reproduced. New genotypes are generated by copying, mutating, and/or combining the genotypes of the most fit individuals, and as the cycle repeats the population should ascend to higher and higher levels of fitness.

235In the spirit of unbounded genetic languages, directed graphs are presented here as an appropriate basis for a grammar that can be used to describe both the morphology and nervous systems of virtual creatures. New features and functions can be added to creatures, or existing ones removed, so the levels of complexity can also evolve.

236A genetic language for representing virtual creatures with directed graphs of nodes and connections allows an unlimited hyperspace of possible creatures to be explored. It is believed that these methods have potential as a powerful tool for the creation of desirable complexity for use in virtual worlds and computer animation.

As computers become more powerful, the creation of virtual actors, whether animal, human, or completely unearthly, may be limited mainly by our ability to design them, rather than our ability to satisfy their computational requirements. A control system that someday actually generates ‘intelligent’ behavior might tend to be a complex mess beyond our understanding. Artificial evolution permits the generation of complicated virtual systems without requiring design, and the use of unbounded genetic languages allows evolving systems to increase in complexity beyond our understanding. Perhaps methods such as those presented here will provide a practical pathway toward the creation of intelligent behavior.

237People have been working with genetic algorithms ever since, but these algorithms haven’t been very useful tools for studying biological evolution. This isn’t because there’s anything wrong with the algorithms per se, but rather because they haven’t been embedded in the proper biological context. As genetic algorithms have been traditionally implemented, they clearly involve artificial selection: some human being provides explicit, algorithmic criteria for which of the entities is to survive to mate and reproduce. The real world, however, makes use of natural selection, in which it is the “nature” of the interactions among all the organisms — both with one another and with the physical environment — that determines which entities will survive to mate and reproduce. It required a bit of experimentation to work out how to bring about natural selection within the artificial worlds we create in computers.

Over the last several years, however, we’ve learned how to do that, through the work of Danny Hillis, Tom Ray, and others. We don’t specify the selective criteria externally. Rather, we let all the “organisms” interact with one another, in the context of a dynamic environment, and the selective criteria simply emerge naturally. To any one of these organisms, “nature,” in the computer, is the collective dynamics of the rest of the computerized organisms there. When we allow this kind of interaction among the organisms — when we allow them to pose their own problems to one another — we see the emergence of a Nature with a capital “N” inside the computer, whose “nature” we can’t predict as it evolves through time.

Typically, a collection of organisms in such artificial worlds will form an ecology, which will be stable for a while but will ultimately collapse. After a chaotic transition, another stable ecology will form, and the process continues. What defines fitness — and what applies the selective pressure — is this constantly changing collective activity of the set of organisms themselves. I argue that such a virtual ecosystem — what I have termed “artificial life” — constitutes a genuine “nature under glass,” and that the study of these virtual natures within computers can be extremely useful for studying the nature of nature outside the computer.

238Stuart Kauffman is a theoretical biologist who studies the origin of life and the origins of molecular organization. Twenty-five years ago, he developed the Kauffman models, which are random networks exhibiting a kind of self-organization that he terms “order for free.” Kauffman is not easy. His models are rigorous, mathematical, and, to many of his colleagues, somewhat difficult to understand. A key to his worldview is the notion that convergent rather than divergent flow plays the deciding role in the evolution of life. With his colleague Christopher G. Langton, he believes that the complex systems best able to adapt are those poised on the border between chaos and disorder.

Kauffman asks a question that goes beyond those asked by other evolutionary theorists: if selection is operating all the time, how do we build a theory that combines self-organization (order for free) and selection? The answer lies in a “new” biology, somewhat similar to that proposed by Brian Goodwin, in which natural selection is married to structuralism.

239Chris Langton is wild, scattered, deeply intuitive, not very critical, very creative, good, good, good intuitions. His thesis about the edge of chaos, phase transitions, was a lovely thing to have done. He developed the ideas of looking at cellular automata and a phase transition, and the idea that you have to ask how complex systems can actually generate, create, and pass information around. The idea that it may happen best as a phase transition may not be correct, but it’s a lovely hypothesis.

240What kinds of complex systems can evolve by accumulation of successive useful variations? Does selection by itself achieve complex systems able to adapt? Are there lawful properties characterizing such complex systems? The overall answer may be that complex systems constructed so that they’re on the boundary between order and chaos are those best able to adapt by mutation and selection.

Chaos is a subset of complexity. It’s an analysis of the behavior of continuous dynamical systems — like hydrodynamic systems, or the weather — or discrete systems that show recurrences of features and high sensitivity to initial conditions, such that very small changes in the initial conditions can lead a system to behave in very different ways. A good example of this is the so called butterfly effect: the idea is that a butterfly in Rio can change the weather in Chicago. An infinitesimal change in initial conditions leads to divergent pathways in the evolution of the system. Those pathways are called trajectories. The enormous puzzle is the following: in order for life to have evolved, it can’t possibly be the case that trajectories are always diverging. Biological systems can’t work if divergence is all that’s going on. You have to ask what kinds of complex systems can accumulate useful variation.

We’ve discovered the fact that in the evolution of life very complex systems can have convergent flow and not divergent flow. Divergent flow is sensitivity to initial conditions. Convergent flow means that even different starting places that are far apart come closer together. That’s the fundamental principle of homeostasis, or stability to perturbation, and it’s a natural feature of many complex systems. We haven’t known that until now. That’s what I found out twenty-five years ago, looking at what are now called Kauffman models — random networks exhibiting what I call “order for free.”

Complex systems have evolved which may have learned to balance divergence and convergence, so that they’re poised between chaos and order. Chris Langton has made this point, too. It’s precisely those systems that can simultaneously perform the most complex tasks and evolve, in the sense that they can accumulate successive useful variations. The very ability to adapt is itself, I believe, the consequence of evolution. You have to be a certain kind of complex system to adapt, and you have to be a certain kind of complex system to coevolve with other complex systems. We have to understand what it means for complex systems to come to know one another — in the sense that when complex systems coevolve, each sets the conditions of success for the others. I suspect that there are emergent laws about how such complex systems work, so that, in a global, Gaia-like way, complex coevolving systems mutually get themselves to the edge of chaos, where they’re poised in a balanced state. It’s a very pretty idea. It may be right, too.

241By spontaneous order, or order for free, I mean this penchant that complex systems have for exhibiting convergent rather than divergent flow, so that they show an inherent homeostasis, and then, too, the possibility that natural selection can mold the structure of systems so that they’re poised between these two flows, poised between order and chaos. It’s precisely systems of this kind that will provide us with a macroscopic law that defines ecosystems, and I suspect it may define economic systems as well.

242The origin of life might be another example of order for free. If you have complex-enough systems of polymers capable of catalytic action, they’ll self-organize into an autocatalytic system and, essentially, simply be alive. Life may not be as hard to come by as we think it is.

243He’s trying to understand what aspects of organic order follow from the physical principles of matter, and the mathematical structure of nature, and need not be seen as Darwinian optimalities produced by natural selection.

He’s following in the structuralist tradition, which should not be seen as contrary to Darwin but as helpful to Darwin. Structural principles set constraints, and natural selection must work within them. His “order for free” is an outcome of sets of constraints; it shows that a great deal of order can be produced just from the physical attributes of matter and the structural principles of organization. You don’t need a special Darwinian argument; that’s what he means by “order for free.” It’s a very good phrase, because a strict Darwinian thinks that all sensible order has to come from natural selection. That’s not true.

244The standard way of looking at evolution is that evolution is a matter of transforming the physical properties of organisms. Stuart’s got models jumping around from adaptive peak to adaptive peak, to explain the early Cambrian explosion.

245I have a little harder time with his last book. The monster, The Origins of Order. Although many of the pieces in there have a flavor of something quite interesting, it doesn’t seem to me that the book hangs together as a whole. There’s too much of “Let’s assume this, and let’s assume that, and if this were right, then…” But the basic idea is that we’re back to the notion of evolution having intrinsic factors, and in this regard it has to be right.

246Francisco Varela is amazingly inventive, freewheeling, and creative. There’s a lot of depth in what he and Humberto Maturana have said. Conversely, from the point of view of a tied-down molecular biologist, this is all airy-fairy, flaky stuff. Thus there’s the mixed response. That part of me that’s tough-minded and critical is questioning, but the other part of me has cottoned on to the recent stuff he’s doing on self-representation in immune networks. I love it.

247Stuart Kauffman and his colleague Brian Goodwin are particularly eager to discredit the powerful image first made popular by the great French biologists Jacques Monod and François Jacob — the image of Mother Nature as a tinkerer engaged in the opportunistic handiwork that the French call bricolage. Kauffman wants to stress that the biological world is much more a world of Newtonian discoveries than of Shakespearean creations. He’s certainly found some excellent demonstrations to back up this claim. Kauffman is a meta-engineer. I fear that his attack on the metaphor of the tinkerer feeds the yearning of those who don’t appreciate Darwin’s dangerous idea. It gives them a false hope that they’re seeing not the forced hand of the tinkerer but the divine hand of God in the workings of nature. Kauffman gets that from Brian Goodwin. John Maynard Smith has been pulling Kauffman in the other direction — very wisely so, in my opinion.

248Homeochaos provides a dynamic stability sustained by high-dimensional weak chaos.

249No matter how impressive the products of an algorithm, the underlying process always consists of nothing but a set of individually mindless steps succeeding each other without the help of any intelligent supervision.

250The actual is, as a matter of brute historical fact, a Vanishingly small subset of the possible.

251Searching through the Library of Mendel for the best genomes is rather like searching for a particular book in the Library of Babel, but with one important difference: there is a non-miraculous, indeed algorithmic, process that has been running for billions of years.

252In the early days of artificial intelligence, researchers assumed that the most important thing about the brain, for the purposes of understanding intelligence, was that it was a universal computer. Its parallel, distributed architecture was thought to be merely a consequence of the bizarre path that nature had to take to evolve a universal computer. Since we know that all universal computers are equivalent in principle in their computational power, it was thought that we could effectively ignore the architecture of the brain and get intelligent software running on our newly engineered universal computers, which had very different architectures. However, I think that the difference in architecture is crucial. Our engineered computers involve a central controller working from a top down set of rules, while the brain has no such central controller and works in a very distributed, parallel manner, from the bottom up. What’s natural and spontaneous for this latter architecture can be achieved by the former only by using our standard serial computers to simulate parallel, distributed systems. There’s something in the dynamics of parallel, distributed, highly nonlinear systems which lies at the roots of intelligence and consciousness — something that nature was able to discover and take advantage of.

253Think of an ant colony — a beautiful example of a massively parallel, distributed system. There’s no one ant that’s calling the shots, picking from among all the other ants which one is going to get to do its thing. Rather, each ant has a very restricted set of behaviors, but all the ants are executing their behaviors all the time, mediated by the behaviors of the ants they interact with and the state of the local environment. When one takes these behaviors on aggregate, the whole collection of ants exhibits a behavior, at the level of the colony itself, which is close to being intelligent. But it’s not because there’s an intelligent individual telling all the others what to do. A collective pattern, a dynamical pattern, takes over the population, endowing the whole with modes of behavior far beyond the simple sum of the behaviors of its constituent individuals. This is almost vitalistic, but not quite, because the collective pattern has its roots firmly in the behavior of the individual ants.

This example shows how one can be both a vitalist and a mechanist at the same time.

254In the late nineteenth century, the Austrian physicist Ludwig Boltzmann showed that one could account for many of the thermodynamic properties of macroscopic systems in terms of the collective activity of their constituent atoms. Boltzmann’s most famous contribution to our understanding of the relationship between the microcosm of atoms and the macroscopic world of our experience was his definition of entropy: S = k log W. In the 1950s, the computer scientist Claude Shannon generalized Boltzmann’s formula, lifting the concept of entropy from the thermodynamic setting in which it was discovered to the more general level of probability theory, providing a precise, quantitative meaning for the term “information."

255As Doyne Farmer has pointed out, our current understanding of complex systems is very much in the same state as our understanding of thermodynamics was in the mid-1800s, when people were screwing around with the basic concepts but didn’t yet know which were the right quantities to measure. Until you know which are the relevant quantities to measure, you can’t come up with quantitative expressions relating those quantities to one another.

256Meanwhile, Chris has gone on to stir up interest in what he calls artificial life. I myself don’t use that way of slicing things. I believe we can learn most by considering natural and artificial complex adaptive systems together — in my view, they form a single subject. Moreover, I don’t myself subscribe to the idea that it’s valuable to separate out those artificial systems that imitate organisms or biological evolution in certain respects and put them in a separate category from those that imitate other natural complex adaptive systems, such as human societies. However, Chris’s category has caught on in a remarkable way, and the term “artificial life” is now widely used. Chris has managed in that way to attract a great deal of attention to the field of plectics and to draw a lot of people into it.

257A physical system undergoes a phase transition when its state changes — for instance, when water freezes into ice. I’ve found that during phase transitions physical systems often exhibit their most complex behavior. I’ve also found that it’s during phase transitions that information processes can appear spontaneously in physical systems and play an important role in the determination of the systems’ behavior. One might even say that systems at phase transitions are caught up in complex computations to determine their own physical state. My belief is that the dynamics of phase transitions are the point at which information processing can gain a foothold in physical systems, gaining the upper hand over energy in the determination of the systems’ behavior. It has long been a goal of science to discover where and how information theory and physics fit into each other; it’s become something of a Holy Grail. I can’t say I’ve found the Grail, but I do think I’ve found the mountain range it’s located in.

258After working for a long time creating these artificial universes, wondering about getting life going in them, and wondering if such life would ever wonder about its own existence and origins, I find myself looking over my shoulder and wondering if there isn’t another level on top of ours, with something wondering about me in the same way. It’s a spooky feeling to be caught in the middle of such an ontological recursion. This is Edward Fredkin’s view: the universe as we know it is an artifact in a computer in a more “real” universe.

259Collapse the distinction between learning and reward by making the reward signal identical to the rate of learning itself.

260If I were to give an award for the single best idea anyone has ever had, I’d give it to Darwin, ahead of Newton and Einstein and everyone else. In a single stroke, the idea of evolution by natural selection unifies the realm of life, meaning, and purpose with the realm of space and time, cause and effect, mechanism and physical law.

261Should it be that much harder for an algorithmic process to write an ode to a nightingale or a poem as lovely as a tree? Surely Orgel’s Second Rule is correct: Evolution is cleverer than you are.

262A scholar is just a library’s way of making another library.

263The mind is the effect, not the cause, of the brain’s being the way it is.

264We watch an ant make his laborious way across a wind- and wave-molded beach. He moves ahead, angles to the right to ease his climb up a steep dunelet, detours around a pebble, stops for a moment to exchange information with a compatriot. So as not to anthropomorphize about his purposes, I sketch the path on a piece of paper. It is a sequence of irregular, angular segments-not quite a random walk, for it has an underlying sense of direction, of aiming towards a goal. Viewed as a geometric figure, the ant’s path is irregular, complex, hard to describe. But its complexity is really a complexity in the surface of the beach, not a complexity in the ant. On that same beach another small creature with a home at the same place as the ant might well follow a very similar path. The ant, viewed as a behaving system, is quite simple. The apparent complexity of its behavior over time is largely a reflection of the complexity of the environment in which it finds itself.

265There is no progress in evolution. The fact of evolutionary change through time doesn’t represent progress as we know it. Progress is not inevitable. Much of evolution is downward in terms of morphological complexity, rather than upward. We’re not marching toward some greater thing. The actual history of life is awfully damn curious in the light of our usual expectation that there’s some predictable drive toward a generally increasing complexity in time. If that’s so, life certainly took its time about it: five-sixths of the history of life is the story of single-celled creatures only.

266I would like to propose that the modal complexity of life has never changed and it never will, that right from the beginning of life’s history it has been what it is; and that our view of complexity is shaped by our warped decision to focus on only one small aspect of life’s history; and that the small bit of the history of life that we can legitimately see as involved in progress arises for an odd structural reason and has nothing to do with any predictable drive toward it.

267Symbiosis is a physical association between organisms, the living together of organisms of different species in the same place at the same time. My work in symbiosis comes out of cytoplasmic genetic systems. We were all taught that the genes were in the nucleus and that the nucleus is the central control of the cell. Early in my study of genetics, I became aware that other genetic systems with different inheritance patterns exist.

From the beginning, I was curious about these unruly genes that weren’t in the nucleus. The most famous of them was a cytoplasmic gene called “killer,” which, in the protist Paramecium aurelia, followed certain rules of inheritance. The killer gene, after twenty years of intense work and shifting paradigmatic ideas, turns out to be in a virus inside a symbiotic bacterium. Nearly all extranuclear genes are derived from bacteria or other sorts of microbes.

In the search for what genes outside the nucleus really are, I became more and more aware that they’re cohabiting entities, live beings. Live small cells reside inside the larger cells. Understanding that led me and others to study modern symbioses.

Symbiosis has nothing to do with cost or benefit. The benefit/cost people have perverted the science with invidious economic analogies. The contention is not over modern symbioses, simply the living together of unlike organisms, but over whether “symbiogenesis” — long-term symbioses that lead to new forms of life — has occurred and is still occurring. The importance of symbiogenesis as a major source of evolutionary change is what is debated.

I contend that symbiogenesis is the result of long-term living together — staying together, especially involving microbes — and that it’s the major evolutionary innovator in all lineages of larger nonbacterial organisms.

268For more than a billion years, the only life on this planet consisted of bacterial cells, which, lacking nuclei, are called prokaryotes, or prokaryotic cells. They looked very much alike, and from the human-centered vantage point seem boring. However, bacteria are the source of reproduction, photosynthesis, movement — indeed, all interesting features of life except perhaps speech! They’re still with us in large diversity and numbers. They still rule Earth.

At some point, a new more complex kind of cell appeared on the scene, the eukaryotic cell, of which plant and animal bodies are composed. These cells contain certain organelles, including nuclei. Eukaryotic cells with an individuated nucleus are the building blocks of all familiar large forms of life. How did that evolution revolution occur? How did the eukaryotic cell appear? Probably it was an invasion of predators, at the outset.

It may have started when one sort of squirming bacterium invaded another — seeking food, of course. But certain invasions evolved into truces; associations once ferocious became benign. When swimming bacterial would-be invaders took up residence inside their sluggish hosts, this joining of forces created a new whole that was, in effect, far greater than the sum of its parts: faster swimmers capable of moving large numbers of genes evolved. Some of these newcomers were uniquely competent in the evolutionary struggle. Further bacterial associations were added on, as the modern cell evolved.

269In the early seventies, I was trying to align bacteria by their metabolic pathways. I noticed that all kinds of bacteria produced gases. Oxygen, hydrogen sulfide, carbon dioxide, nitrogen, ammonia — more than thirty different gases are given off by the bacteria whose evolutionary history I was keen to reconstruct. Why did every scientist I asked believe that atmospheric oxygen was a biological product but the other atmospheric gases — nitrogen, methane, sulfur, and so on — were not? “Go talk to Lovelock,” at least four different scientists suggested. Lovelock believed that the gases in the atmosphere were biological. He had, by this time, a very good idea of which live organisms were probably “breathing out” the gases in question. These gases were far too abundant in the atmosphere to be formed by chemical and physical processes alone. He argued that the atmosphere was a physiological and not just a chemical system.

The Gaia hypothesis states that the temperature of the planet, the oxidation state and other chemistry of all of the gases of the lower atmosphere (except helium, argon, and other nonreactive ones) are produced and maintained by the sum of life. We explored how this could be. How could the temperature of the planet be regulated by living beings? How could the atmospheric gas composition — the 20-percent oxygen and the one to two parts per million methane, for example — be actively maintained by living matter?

270Gaia is a tough bitch — a system that has worked for over three billion years without people. This planet’s surface and its atmosphere and environment will continue to evolve long after people and prejudice are gone.

271Some of the most commonplace items in our world are also paradoxically the most improbable and wonderful: the physical properties of water, the existence of sex, the fact that the millions of animal species have only thirty-five basic body plans, the fact of evolutionary transformation through time. The latter two items, the objects of this book, are related in a seemingly antithetical way. The basic animal body plans are half a billion years old. Arthropods, chordates, mollusks, echinoderms, and others can all be recognized from the first appearance of the animal fossil record. Even so, the striking diversity of living animals shows that dramatic evolutionary transformations have taken place within these ancient patterns. Body form evolves through geologic time, but it also arises in each generation through development. The time frames and apparent processes couldn’t be more different, but explanations of the evolution of form have to consider how form is generated, and they must account for both underlying stability and immense change in design.

My awakening to the scope and significance of the relationship between development and evolution came during my first year as a graduate student at Duke University, when I read de Beer’s book, Embryos and Ancestors. I no longer remember how I found out about the book, but I got hold of a copy and set out to study it. This was not as simple a task as vacation novel reading. I was engaged in my first year of heavy coursework in biochemistry and was expected to read the Journal of Biological Chemistry in my spare time. Consequently, on Saturday mornings I sat with de Beer in the laundromat across the street from an old Durham eatery, the Little Pigs Barbecue, while I added coins to the dryers and waited for the endless drying cycles to finish. At that time I knew nothing about embryology, and so the terminology was simply mind-boggling; germ layers, neural plates, and archenterons swam before my mystified eyes until I got it all straight. But it was a stimulating experience. Until that time, my views of evolutionary change were largely of transformations between features of adult body plans. I had never considered the embryological roots of those transformations. Embryos and Ancestors provided the connection between the construction of an individual animal and the course of its evolutionary history. De Beer also presented a mechanistic explanation by which changes in the timing of developmental events in the descendant as contrasted with the ancestor could yield evolutionary novelties by changing the relations of developmental features. This phenomenon, heterochrony, has motivated most thought on the evolution of development. De Beer offered fascinating scenarios to demonstrate how animal groups, including the vertebrates, could potentially have originated by heterochronic changes. What de Beer didn’t do was consider the roles of genes in development, nor did he discuss any experimental approaches. He provided a paradigm for the evolution of morphology, but his approach lacked any means by which one might investigate the hypotheses presented. On reflection, it’s hard to blame him. It certainly took me a long time before I was able to design for myself an experimental approach to some problems in this still largely theoretical and anecdotal discipline.

272Of necessity, an understanding of the relationship between development and evolution requires synthesizing two great and traditionally separate disciplines in biology, developmental biology and evolutionary biology. The relationship between the two disciplines has always been an enterprise in creative inference making. The recognition that an appreciation of embryonic development would be important in understanding evolution appears in The Origin of Species. Darwin suggested that we could use embryos to trace evolutionary relationships because ‘the embryo is the animal in its less modified state; and in so far it reveals the structure of its progenitor.’ Darwin certainly had a concrete example before him, having earlier carried out a long study of barnacles. Barnacles were long thought to be mollusks, but once discovered, their larvae quickly gave them away as related to shrimp and other crustaceans.

From the 1860s to the end of the nineteenth century, the German evolutionary biologist Ernst Haeckel built a hypothesis as to how ontogeny and phylogeny should interact, and founded a research program that featured the large-scale application of data from embryology to working out phylogenies. Early in the twentieth century, embryologists became interested in how embryos worked rather than how they evolved, abandoned Haeckel’s program, and turned to experimental studies of development itself. The current study of the relationship between evolution and development has grown from a rebirth of interest in the subject as well as from the invention of new tools in genetics and molecular biology.

273Another element unavailable ten years ago is an understanding of the molecular machinery underlying development. During the past decade, developmental genetics has become one of the liveliest disciplines of biology. Its enormous success has come from the realization on the part of developmental biologists that genes underlie developmental processes and even serve as switches that determine the course of differentiation of cells within the embryo. During the same period, powerful tools became available to dissect developmental processes at the molecular level. The most striking part of that success has come from the joint application of genetics and molecular biology to identifying and isolating the genes that control development. The genetic analysis of development was under way a decade ago, and we could point to the roles of homeotic genes in the evolution of segmental features in insects. But there were major limitations. At that time, none of the genes involved had been cloned, and the functions of their products were not known. Without gene sequence information, gene homology could not be established. Once that information became available, it became possible to trace patterns of gene utilization in development across phylogenetic lines. The discovery during the past decade that insects and vertebrates use homologous homeobox-containing genes in their development has revealed a deep genetic relationship underlying the development of the two phyla. More than any other single discovery, this finding of a deep commonality of the genetic controls for establishing components of axial polarity in animal development has helped to fix the focus of developmental biologists on how evolutionary data can contribute to our understanding of developmental mechanisms.

274The fossil record of events around the Cambrian boundary is important in framing our questions about the mechanisms by which development affects patterns of macroevolution.

275In many cases, we can recognize constraining factors in evolution. For instance, why there are no animals with wheels is easily explained. It is not possible to connect nerves and circulatory systems to the tissue of a free-turning wheel.

276The phenotypes we observe in nature should reflect some balance between internal constraints and external selection. The empty phenotypic space houses Platonic shadow phenotypes that might be attainable under other rules than those that have operated in metazoans since the Cambrian radiation.

277It was a sort of investigation where you got a good lead, and certainly you had to pursue that; and before you reached the end of that lead up opened another, and this was if anything even more fascinating. One kept on going in several stages; one beautiful lead opening up after the other, and every one much too good to abandon.

278It took until 1990 for the first conclusive experimental evidence of Turing patterns to appear.

279When two waves in an excitable medium collide, they annihilate one another, because each wave leaves behind it a refractory region in which the system cannot be excited. This behavior is exactly the opposite of what occurs with linear waves such as sound or light, which pass through one another.

280The term ‘complexity’ has been used without qualification by so many authors, both scientific and nonscientific, that it has been almost stripped of its meaning.

281The physics lies in the fluctuations.

282Fractal geometry will make you see everything differently. There is danger in reading further. You risk the loss of your childhood vision of clouds, forests, flowers, galaxies, leaves, feathers, rocks, mountains, torrents of water, carpets, bricks, and much else besides.

283‘Know thyself’ seems to be an impossible command for a computer program. If a model of computation is so weak that it cannot ask questions about programs, then it will skirt the abyss, but at the cost of being too stupid to solve interesting problems.

284Between any two entrained regions, no matter how close together in frequency ratio, there is always another entrained region — a chemical analog of the number-theoretic result that between any two rational numbers there is always another rational.

285The BZ reaction in a Petri dish provides some of the most beautiful and instructive demonstrations in all of chemistry.

286Would it not be extraordinary, however, if these laws could by themselves contrive to generate rich and beautiful patterns?

287Dissipative dynamics, symmetry breaking, phase transitions, bifurcations, and pattern formation, acting over different temporal and spatial scales, at different levels and on different substrates, are presumably responsible for assembling and freezing in the wide diversity of structures observed in the natural world. Each of these processes has its more-or-less well-developed foundations. But where are the principles that define and describe their products? What is structure itself?

288You put in featureless, indiscriminate energy, and the out-of-equilibrium system uses it to organize itself into patterns that can astonish.

289Patterns live on the edge, in a fertile borderland between these extremes, where small changes can have large effects. Pattern appears when competing forces banish uniformity but cannot quite induce chaos. It sounds like a dangerous place to be, but it is where we have always lived.

290The choice of macroscopic states is not a detail — it is the entire problem.

291As on the sea’s wrinkled surface, it is the pattern that remains, not its individual components.

292If life were started from scratch a thousand times over, it would every time alight on these fundamental structures eventually.

293Competition lies at the heart of beauty and complexity in pattern formation. If the competition is too one-sided, all form disappears, and one gets either unstructured, shifting randomness or featureless homogeneity — bland, in either event. Patterns live on the edge, in a fertile borderland between these extremes, where small changes can have large effects.

294The symmetry of a pattern formed by a symmetry-breaking force does not always reflect the symmetry of that force. If you heat a shallow pan of oil, it will develop roughly hexagonal circulation cells. The system was initially uniform, and the symmetry-breaking force was also applied uniformly — yet suddenly this uniformity is lost. Where has this sixfold pattern come from?

295The diverse range of pelt patterns and markings can be explained with the same basic mechanism. William of Ockham would have reminded us that if we can account not only for the leopard’s spots but also for the zebra’s stripes and the giraffe’s dapples with the same theoretical model, that is surely more satisfactory.

296There is no way we could detect the ‘presence’ of this geometric principle by decoding the genetic information in the bee’s DNA. We could see that they make use of it only by watching the organism as a whole go about her job.

297It is as though individuals in a jabbering crowd were able to converse with one another from opposite sides of a room.

298Equilibrium is a dull place to be. Nothing happens there. If the Universe were itself at thermodynamic equilibrium, it would be a lifeless place pervaded by a uniform, dim glow of just a few degrees above absolute zero. Just about every phenomenon that interests us is an out-of-equilibrium process — life, to mention one.

299One is ‘swimming around in Occam’s pool’ of possible state representations, and epsilon-machines sit at the shallowest point.

300Quite apart from the fact that percolation theory has its origin in an honest applied problem, it is a source of fascinating problems of the best kind for which a mathematician can wish: problems which are easy to state with a minimum of preparation, but whose solutions are apparently difficult and require new methods.

301Biological life is in control of its own means of reproduction, and this autonomy of design and manufacture is a key element which has not yet been understood or reproduced artificially.

302The field of Artificial Life examines “life as it could be” based on understanding the principles and simulating the mechanisms of real biological forms. Just as airplanes use the same principles as birds, but have fixed wings, artificial lifeforms may share the same principles, but not the same implementation in chemistry. Every feature of living systems seems wondrous until it is understood: Stored energy, autonomous movement, and even animal communication are no longer miracles, as they are replicated in toys using batteries, motors, and computer chips.

303Our approach is based on use of only elementary building blocks and operators in both the design and fabrication process. As building blocks are more elementary, any inductive bias associated with them is minimized, and at the same time architectural flexibility is maximized. Similarly, use of elementary building blocks in the fabrication process allows it to be more systematic and versatile. As a theoretic extreme, if we could use only atoms as building blocks, laws of physics as constraints and nano-manipulation for fabrication, the versatility of the design space would be maximized. Earlier reported work used higher-level components and limited architectures (like only tree structures) resulted in expedited convergence to acceptable solutions, but at the expense of truncating the design space. Furthermore, these design spaces did not consider manufacturability.

304Although the underlying dynamics describing this system is very simple, and entirely deterministic, there is an enormous variety, and complexity, of emergent particle-particle interactions. Such simple systems are powerful reminders that complex higher-level dynamics need not have a complex underlying origin. Indeed, suppose that we had been shown such a space-time pattern but were told nothing whatsoever about its origin. How would we make sense of its dynamics? … It would take a tremendous leap of intuition to fathom the utter simplicity of the real dynamics.

305What we are thus seeing … is rather dramatic (albeit crude) evidence of the phenomenon of self-organization, in which an initially random state evolves to a state exhibiting long-range correlations and whose overall space-time patterns contain manifestly nonlocal structure.

306Such systems provide evidence that an apparent global ordered collective dynamics need not stem from an intrinsically global cooperative dynamics, but may instead be nothing more than an emergent high-level (phenotype) construct stemming from an essentially non-cooperative low-level (genotype) dynamics.

307Complexity generically increases with randomness up until a phase transition is reached, beyond which further increases in randomness decrease complexity.

308Life — like all computationally universal systems — defines the most efficient simulation of its own behavior.

309Inside any sufficiently large random broth, we expect just by chance, there will be some of these self-replicating creatures.

310We conclude from this fact that undecidability and computational irreducibility are not good measures for physical complexity.

311Diffusion is usually considered a stabilising process which is why this was such a novel concept.

312Perhaps we should turn the pattern formation question around and ask: ‘What patterns cannot be formed by such simple mechanisms?’

313The fundamental importance of pattern and form in biology is self-evident. Whatever pattern we observe in the animal world, it is almost certain that the process that produced it is still unknown. The genes themselves cannot create the pattern.

314Computational mechanics — in its focus on letting the process speak for itself through (possibly impoverished) measurements — follows the spirit that motivated one approach to experimentally testing dynamical systems theory.

315The right strategy is to formalize pattern, then define complexity as the amount of pattern, then define organization as the growth of complexity, and finally define self-organization as self-generated growth of complexity.

316An epsilon-machine reconstruction algorithm takes in data and gives back a representation of causal patterns, suitable for use in prediction or intervention, ‘untouched by human hands.’ Such an algorithm is a phenomenological engine.

317Statistical mechanics handles randomness and disorder well, but it lacks a coherent, principled way to describe, quantify, and detect the many kinds of structure nature exhibits.

318Epsilon-machines are simultaneously maximally predictive, minimally complex, unique, and minimally stochastic.

319Complexity, on this view, must be a function of the process that generates configurations; we need movies, not snapshots. But this should not be distressing to physicists: we, of all people, should be very suspicious if pattern appeared without a causal history to back it up.

320‘Organized’ does not equal ‘ordered’ — order means low entropy, while organization lies between order and randomness.

321To call something emergent is therefore not to say anything about the property at all, but merely to make a confession of scientific and mathematical incompetence.

322Set a child alone with a heap of Legos, and in an hour the Legos are probably a lot more ordered than they were at the start. Their organization has increased, but in a pretty unmysterious and, to us, uninteresting way: the kid organized them. On the other hand, there are some things which will organize without this kind of outside intervention, which self-organize.

323We are not trying to explain everything we can measure; we are trying to find what’s intrinsically important in our measurements. Emergence is anti-regression.

324No unbiased estimator for entropy or mutual information exists.

325Unfortunately, this is a statement that is ‘obviously true’ but turns out to be false.

326The fitness-based approach is no better than random search, demonstrating that the objective function, rather than helping, has actually been misleading search away from the true answer.

327When I first read a biology textbook, it was like reading a thriller. Every page brought a new shock. As a physicist, I was used to studying matter that obeys precise mathematical laws. But cells are matter that dances.

328Although cells evolved to function and did not evolve to be comprehensible, simplifying principles make biological design understandable to us.

329Evolution converges again and again to the same network motifs in transcription networks. This suggests that motifs are selected because they confer an advantage relative to other circuit designs.

330Development is the remarkable process in which a single cell, an egg, becomes a multicellular organism. During development, cells must assume different fates in a spatially organized manner.

331This is a story about dynamics: about change, flow, and rhythm, mostly in things that are alive. The subject matter being dynamics, we are embarked upon a study of temporal morphology, of shapes not in space so much as in time.

332The phase singularity itself is not an instability, but just an alternative — and fatal — mode of operation in normal tissue.

333A lot of behavioral physiology is temporally organized in periodic patterns. If I had to decide what impresses me as the single most conspicuous feature of natural ecosystems, I would say that it is the daily and seasonal periodism.

334Apparent randomness can arise from very simple deterministic systems.

335The presence of chaos in a system implies that perfect prediction is impossible not only in practice but also in principle, since we can never know x(0) to infinitely many decimal places.

336The creation of this book and the science it describes has been a vast personal undertaking, spanning the better part of half my life so far. And for it to all have been even remotely possible has required a series of particular personal circumstances. Foremost among them is that I have lived at the moment in history when technology has first made it possible to do the kinds of things I have done. But also crucial has been that my early successes in science and business have for more than twenty years allowed me to be free to pursue the personal intellectual objectives I have chosen.

337I did what is in a sense one of the most elementary imaginable computer experiments: I took a sequence of simple programs and then systematically ran them to see how they behaved. And what I found — to my great surprise — was that despite the simplicity of their rules, the behavior of the programs was often far from simple. Indeed, even some of the very simplest programs that I looked at had behavior that was as complex as anything I had ever seen.

338Griffeath and Hickerson construct a finite Game of Life initial seed — exactly 2,392 cells — whose growing crystal has an asymptotic density of (3 − √5)/90, an irrational number.

339The word complex comes from the Latin root plectere: to weave, entwine.

340It is conceivable that there are processes which organize themselves into conditions so complex that no human being can grasp them. They would be so organized, in other words, that they would look very like noise.

341Curiosity reward should be proportional to predictor improvement, not predictor error.

342Animals prove that this kind of learning is possible, and set a lower bound on how well it can be achieved: anything a sea slug, a lorikeet, or a tenured professor can do, a learning algorithm can do.

343I am happy to still have a sense of wonder when reading the chapters. The wonder comes because networks of thousands of interacting components are generally incomprehensible, yet simplifying principles can still be found.

344Why does nature appear to use only a few fundamental forms in so many different contexts? Why does the branching of trees resemble that of arteries and rivers? Why do crystal grains look like soap bubbles and the plates of a tortoise shell?

345Instability is not merely the breakdown of order — it is the mechanism by which nature spontaneously generates spatial structure.

346No river, regardless of size, runs straight for more than about ten times its average width.

347The beauty of life is, therefore, geometrical beauty of a type that Plato would have much appreciated.

348Of all regular tessellations that tile the plane, hexagons enclose the most area for a given perimeter — a fact bees have exploited for millions of years, and that Thomas Hales proved rigorously only in 1999.

349The deep ideas of computation are intimately related to the deep ideas of life and intelligence.

350One can visualize the necessity of some such irregularity by trying to construct a smooth surface of data points over the square, subject to the constraint that along the boundary it must rise up one floor like a parking garage rampway. Think how this is arranged in the garage: It isn’t.

351Good problems and mushrooms of certain kinds have something in common: they grow in clusters. Having found one, you should look around; there is a good chance that there are some more quite near.

352Any system is a channel that communicates its past to its future through its present.

353A completely ordered universe, however, would be dead. Chaos is necessary for life.

354Determinism and randomness are not opposites but essential and unavoidable twins.

355The crucial lesson: maximal randomness (fair coin) has zero structural complexity, and perfect periodicity has zero randomness but nonzero structural complexity. The two quantities are genuinely independent.

356The most characteristic feature of a critical point turns out to be the divergence of the correlation length that renders microscopic details oblivious.

357The line between an explanation and a description of SOC is blurred; an explanation usually contains a description but whether it should contain a prescription is a contentious epistemological question.

358The most important lesson SOC has taught is that it exists as a robust phenomenon. The two models just mentioned are perfect candidates to put to the test any future theory of emergence, which complexity is waiting for. No complete theory of SOC exists.

359We provide the first quantitative evidence, via the local information dynamics framework, for the long-held conjecture that in cellular automata, domains are information storage, particles are information transfer, and particle collisions are information modification.

360Classification schemes are the holy grail in the theory of cellular automata.

361There is no general method to directly link the definition of a cellular automaton with its global, long-term behavior.

362A central problem in the subject is computation of the entropy rate (sometimes known simply as entropy) of an HMP.

363There is a very simple closed-form formula for the entropy rate of a finite-state Markov chain, but there is no such simple formula for entropy rate of HMPs except in very special cases. However, more than fifty years ago Blackwell discovered an expression for the entropy rate of an HMP as the integral of a very simple integrand with respect to a typically very complicated measure on a simplex.

364Ockham’s razor is a consequence, not an assumption.

365The beautiful fractal patterns of rules like 60, 90, and 150 are infinitely fragile. They exist only at a measure-zero point in parameter space and have no robustness whatsoever to stochastic perturbations.

366Rather than measuring novelty explicitly, nature is guided by a single, fundamental constraint: survive long enough to reproduce. Surprisingly, this simple constraint produces both complexity and diversity in a continual process unparalleled by any algorithm to date.

367Lenia: a continuous generalization of cellular automata in which space, time, and state are all made continuous. A handful of parameters — a kernel peaks vector, a growth center, a growth width — generate over 400 species organized into 18 families. The parameter space maps exactly to Wolfram’s four classes: desert, savannah, forest, and river — where the river is where the lifeforms swim.

368The critical state is a global attractor of the dynamics.

369The ‘law of series’ refers to a popular, intuitively felt tendency of rare events to occur in clusters rather than being spread uniformly in time. While it has been commonly dismissed as a purely psychological effect of selective attention, in many cases it has a sound mathematical basis rooted in the ergodic theory of dynamical systems.

370Universality lurks just below the surface of almost any system complex enough to be interesting.

371Whatever policy learned inside of this generated environment will achieve a perfect score most of the time, but will obviously fail when unleashed into the harsh reality of the actual world, underperforming even a random policy.

372We believe that the boundary between easy and hard problems, between the computable and the uncomputable, is not an artifact of a particular mathematical formalism. We believe it is carved into the fabric of the universe.

373If P = NP, then the world would be a profoundly different place than we usually assume it to be. There would be no special value in ‘creative leaps,’ no fundamental gap between solving a problem and recognizing the solution once it’s found. Everyone who could appreciate a symphony would be Mozart; everyone who could follow a step-by-step argument would be Gauss.

374The Church-Turing thesis is one of the most remarkable convergences in the history of ideas. A dozen different formalisms were proposed for capturing the notion of ‘effective computation.’ Every single one turned out to define exactly the same class of computable functions.

375The universality of NP-completeness is one of the most remarkable phenomena in all of mathematics. Thousands of problems, arising from the most diverse corners of science, engineering, and mathematics, turn out to be equivalent to each other.

376Zero-knowledge proofs are among the most paradoxical objects in all of mathematics. They allow you to convince someone that a statement is true without conveying any information beyond the truth of the statement itself.

377IP = PSPACE tells us that conversation is extraordinarily powerful: a single conversation with an untrusted genius is as powerful as an exponential amount of trusted computation.

378Randomness and hardness are two sides of the same coin: the existence of hard problems lets us generate pseudorandomness, and pseudorandomness lets us remove the need for true randomness.

379The geometry of the solution space of a random formula undergoes a series of remarkable transitions as the density of clauses increases. At low densities, the solutions form a single giant connected cluster. At higher densities, this cluster shatters into an exponential number of smaller clusters.

380In the frozen phase, the landscape of solutions is like an archipelago of tiny islands in an exponentially vast ocean, and no local algorithm can island-hop its way to satisfaction.

381The theory of computation is not just about computers — it is about the nature of the physical universe, and what it allows us to know.

382The Nagel-Schreckenberg traffic model: the addition of a single stochastic step to deterministic acceleration and deceleration rules produces spontaneous traffic jam formation from homogeneous initial conditions.

383The year 2024 marks the 30th anniversary of the formation of the Artificial Life journal by Chris Langton in 1993. It is also the 37th anniversary of the very first Artificial Life conference organized by Chris Langton at Los Alamos National Labs in 1987.

384In 1985, a philosophy professor, David Helman, gave me a preprint of a penetrating critique of classical AI by Terry Winograd and Fernando Flores. Chapter 4 of that book introduced the ideas of the Chilean biologists Humberto Maturana and Francisco Varela, which provided a road map for an entirely different way of thinking about cognition that was inextricably grounded in its biological underpinnings. This chapter was to have a profound influence on the subsequent direction of my research. I immediately set about reading every book and paper of Maturana and Varela’s that I could find and organizing a small reading group in which I tried to come to grips with the dense, unfamiliar ideas that went against almost everything I had been taught in computer science and AI.

A number of other very important things were also happening in the influential years 1985–1987. The roboticist Rodney Brooks (1986) began a line of work that emphasized the fundamental importance of physical embodiment to intelligent behavior. In parallel, and from the unlikely direction of anthropology and human–computer interaction, came a line of work emphasizing the fundamental situatedness of intelligent systems and the importance of that environmental context to their actions. Additionally, these years saw the beginning of the second neural network revolution, following the original excitement around perceptrons in the 1950s and preceding the current enthusiasm for deep learning.

385Within a typical ALife conference proceedings or volume of Artificial Life, one might find work on oil droplets, large language models, chemical and DNA computing, neural network architectures and learning algorithms, evolutionary algorithms, education, robotics, economics, affective computing, cellular automata, reinforcement learning, ethics, reservoir computing, ecology, evolvable hardware, cooperative behavior, artificial chemistry, neuroscience, sociology, computer music and art, cognitive science, information theory, and culture. The point is not that some aspects of these many topics do not belong in Artificial Life. But Artificial Life cannot just be about neural networks, or evolutionary algorithms, or reinforcement learning, or dynamical systems theory, or information theory. These are merely methods, many of which already have their own communities, conferences, and journals. Likewise, Artificial Life cannot be AI, or robotics, or neuroscience, or chemistry, or biology, or ecology, or sociology, or economics; these already exist as independent scientific fields, with histories much longer than Artificial Life’s. And while the intersection of all these activities threatens to be empty, their union is nothing less than the totality of science and engineering.

386I have been reviewing papers for 35 years, in fields ranging from neuroscience to software engineering, from robotics to cognitive science, from philosophy to mathematics, and the inconsistency of reviews in Artificial Life is among the worst I have seen.

387The vast majority of research projects in ALife are one-offs. They tackle a particular problem of interest to the authors, with an idiosyncratic combination of methods and at best a shallow analysis of results, and then they are dropped. This has the consequence that the literature is filled with many minor but incommensurable variations on a theme.

388We prove a strict hierarchy: globally universal CA form a strict subset of universally self-replicating CA, which form a strict subset of locally universal CA.

389Quantitative theories of physics are possible because macroscale phenomena are often independent of microscopic details.

390A shockingly wide variety of physical systems — from ordinary physical objects to biological cells to brains to the universe as a whole — have come to be regarded by researchers and scholars in multiple disciplines as ‘computational.’ The notion of computation may seem to have lost its distinctiveness.

391For any given sequence of physical microstates visited in the arbitrary time evolution of any sufficiently complex system (e.g., a rock), there exists an assignment of physical microstates to computational states such that the time evolution of the microstates is consistent with the system having performed the computation f. This is not very impressive.

392Shift spaces are to symbolic dynamics what shapes like polygons and curves are to geometry.

393Promptbreeder is not just improving task-prompts, but it is also improving the mutation-prompts that improve these task-prompts.

394Even without any background mutation, this state transition occurs with roughly the same frequency. It is not simply random bit-flipping that causes self-replicators to arise.

395The pansculpturalist would declare that a block of marble is every sculpture that could be chiseled from it because they can interpret the block that way, denying any real distinction between an imagined surface within the block of marble and a real physical surface rendered from the chiseling of the block to expose it.

396It would be absurd to maintain that all there is to sunshine, rain, snow, or hail is that it is the output of a computation. Computational outputs may be taken to represent sunshine, rain, snow, or hail, but they make nothing illuminated, wet, or cold.

397what
SPLENDOUR
IT ALL COHERES