Summary of suggestions and modifications

Summary of suggestions and modifications

Thanks to referees for concerns and critics, they were very useful for me.

Reviewer 1

Suggestion	Reply
* The term "fitness" is used several times in a confusing and ambiguous manner. The usual meaning of fitness is some quantity that is monotonic with the probability of survival under natural selection. It is also usually a property of an organism. Hence I believe it is only a meaningful concept when used in the context of evolutionary algorithms, when an explicit fitness function is applied. I'm aware that occasionally "replication rate" is associated with fitness - whilst in some circumstances this may be valid (Avida experiments come to mind), under most circumstances there is no clear monotonic relationship between replication and survival under selection. This point was even made by the author in his ECAL'03 paper. Similar comments also apply to population sizes, which tend to have a negative correlation with survivability near ecosystem carrying capacity. This simplest way for the author get around this is to avoid using the fitness altogether, except when referring to evolutionary algorithms, and instead just talk about (evolutionary) measures.	I agree with reviewer that today we can find strictly defined notion of fitness only in the field of evolutionary algorithms. The notion of fitness is very controversial, but in spite of this it is widely used in the ALife research. I think that in every particular circumstance it should be clearly stated what the fitness is and why it is expected to reflect survivability. And I tried to modify manuscript to make this point a little bit more clear. From my point of view it will be better to avoid using term fitness in ALife, and study evolutionary stability and transitions.
* The paper introduces a simple model to illustrate the measures proposed. It is a simple model (similar in some ways to Echo) that is probably not interesting in its own right, but is sufficient for the illustrative purposes used in this paper. - The control system used by agents in the model is a simple linear system, seemingly equivalent a feedforward neural network with no hidden layer. Its probably worth mentioning this as an aid to the reader's understanding. - In the genetic algorithm, how are the floating point numbers generated by the mutation procedure converted into the integral values of the weights W, and kinship markers M? - The term "predator" is used without introduction on page 3. Later, it becomes clear that it means the agent initiating a "fight". - The description of the model is fairly clear, but it might be useful for the author to supply the source code on his website to alleviate any other misunderstandings.	All suggestions accepted. Standard JAVA method .nextInt(range) used for generation of random integers Source code for the model and results processing application available at http://www.keldysh.ru/pages/mrbur-web/tte/
* One should be careful using the term "evolutionary transition" - this term tends to be reserved for the major transitions such as Szathmary and Maynard-Smith talk about (eg transition to self-replication, or to multicellularity). The phenomenon discussed here is nothing like that - more likely it corresponds to an extinction event followed by a radiation.	The following sentence added "(It should be noticed that the term "evolutionary transition" means here transition between metastable states in evolving system, and should not be confused with widely known term introduced in evolutionary biology by Maynard-Smith and Szathmary[13].)"
* A fairly significant weakness of the paper is that virtually every measure of the system will pick up the distinct change of behaviour between epoch's 1, 2 and 3. It therefore does not demonstrate any advantages accruing from the suggested measures and visualization techniques.	It is not proposed in the paper that a goal is only to catch transitions between epochs, but also to see how the dynamics of a system differs from epoch to epoch.
* The change of behaviour is described in dynamical systems terms as being like a bifurcation as a dynamical system changes through variation of a control parameter. I believe this analogy is incorrect, and invalid for all evolutionary systems, as it is not a control parameter which is varying, but rather which finite dimensional subspace of an infinite dimensional phase space is occupied at any time (open dimensional system). This attractors wink in and out of existence during the evolutionary process. As the system approaches self-organised criticality (which I expect this system would if simulated long enough), the whole notion of attractors, with stability properties becomes entirely meangingless, as the attractors do not survive long enough to influence the system's dynamics. This abrupt change behaviour observed in this model corresponds instead to the notion of an "avalanche" in self-organised criticality.	I agree that for the simulation presented we couldn't say about bifurcation, and there was no any statement about bifurcation in the manuscript. I think that we can find something like bifurcation for the series of simulations with changing parameter (fig. 7). It looks like that switching during particular run is analogous to avalanche as you suggested. I added this to the discussion of figure 7.
* How do you characterise subpopulations on page 8? Are you using some algorithm such as k-means to compute these, or are they sufficiently obvious you just hand pick them?	Added to the text "Frequencies of strategies in the population at the given moment of time were calculated by picking every agent and calculating its actions for every situation."
* Since the agents are grouped by behaviour, in general the "cloud of points" corresponding to a subpopulation will not be contiguous in genotype space. It may be the case in you model because of the simple linear controller employed.	Yes, of course. And I think that it might be omitted in my case study.
More minor points: * The English expression employed is not too bad, but the author does make a number of article errors (not surprising as Russian does not have articles). Getting a native English speaker to correct these would certainly help the paper's readability. * Page 2: "discreet" should be "discrete" * Page 3: the sentence expressing that population and system, agent and point are synonyms is a bit confusing - perhaps a better way of putting it is "population and system are synonyms, as are agent and point.". * Figure 1: y axis of 1c should read "speed of centroid", since it is the time derivative of displacement. * Fig 5: caption should be "time axis", not "time axe". * Throughout the references, the book ALife VIII workshops, edited by Bilotta et al, is published by UNSW, not MIT Press. It may be useful to include an ISBN, or a URL pointing to the online copy of the proceedings. (http://alife8.alife.org/workshops.html). * Yaeger's paper is about "PolyWorld", not "PolyWord".	Everything is fixed except fig. 1c axis. The first version was "speed of centroid", but speed is derivative and I have not derivative and integral displacements for the certain time step. It is like first difference. So, I changed my mind to displacements. Actually, I don't think this is principal question, and I am ready to change name of axis if my explanations look unsatisfactory.

Reviewer 2

Suggestion

Reply

Probably the most significant weakness of the proposed techniques (or at least those demonstrated in Figure 3 onwards), is that they rely on the experimenter to classify different evolved behaviours according to some kind of categorizaton of strategies. Furthermore, this categorization needs to be done on the population throughout the evolutionary run, so it really requires some *automated* technique by which these behavioural strategies can be identified and recorded.

While this is relatively easy to do in the example given, it may be much harder (or even impossible) to do this in other systems, particularly where there is no direct mapping between genotype and phenotype. This certainly limits the applicability of these techniques. It may be that there are still a wide variety of systems in which the techniques could be useful - my point is that the author should recognize this and other potential drawbacks, and *discuss* them in the paper.

I agree that to find proper categorization is a hard problem. It has no general solution and should be resolved for every particular case by experimenter.

I don't think that calculation of strategies depends on genotype to phenotype mapping. If one has an agent which acts in simulation, it is no problem to pick up the agent and calculate its actions for selected circumstances in simulated world. Also, there is no problem with tracking genotype for agents of interest.

Discussion of these questions added to the manuscript.

Page 3:

In the section describing the Model, I suggest you say very briefly what inputs are possible, and then refer the reader to Appendix 1 for more details (just to save them having to flick backwards and forwards in the paper). Similarly, a brief explanation of kinship markers in this section would help.

I tried to align description with recommendation.

"The rate of growth of average generation in population determines upper bound of speed of the system's movement" You might add that this statement only applies if there is a maximum bound on the size of mutations (which there is in this case).

Similar statement added.

Commenting on Fig 3b, you say "trajectory of centroid persists in the same region of weights psace at the epoch I as at the epoch III".

However, looking at Fig 2 it looks like the centroid is fairly different. It therefore looks like these (significant) differences are being drowned out by the process of averaging to calculate the centroid. Can you discuss this, and say whether you think it is a problem?

Text was modified as follows

"Other interesting thing about particular run is that relative to the jumps during epoch II trajectory of centroid persists in the same region of weights space at the epoch I as at the epoch III. It is not exactly the same region, if epoch I will be compared to the epoch III (see figure 2). Probably, this region of persistence might be divided on smaller sub-regions as it follows from Figure 2."

Page 7, Fig 3a

Please add a scale (or text in the caption) to explain the distance plots (i.e. dark is far, light is near).

Added

Page 8, Fig 8a

You have plotted 100 strategies, having just said that there are 729 possibilities. What about the other 629?

Caption modified as follows

"Strategies bitmap (a) [color scale as on the Fig. 2, but the range starts from 0], strategies graphs (b). (In the simulation discussed here only 142 out of 729 strategies were observed, and frequencies of about one third of them were much smaller than of others, so the bitmap (a) contains only 100 most frequent strategies.)"

Pages 9-10, Figs 5a and 6a.

These are the same plots, yet for Fig 5a you say there is a weak correlation, and for 6a you say there is no correlation. Choose one or the other interpretation!

I am sorry; I meant no correlation inside epoch II and III on fig 6a. I suggest – " Also, one can see that characteristics on the Fig. 6a and 6b not correlate during every distinct epoch (except fig.6a epoch I, where one can find weak correlation)"

Page 14:

Some details of the model are missing. For example, if there are more than one agent at a cell and an agent fights, does the fight occur with just one other agent, or with all of them? From what you say for the description of I13, it sounds like just one is chosen. Is this choice random? Also, is the information in sensory input I13 available to an agent before it interacts, or only afterwards?

Description of the model modified to meet suggestions