7 Manipulative stage. Having traced the evolution of the animal system of representation from somatosensory through subjective animals with spatial imagination, the manipulative animal system of representation can be predicted, in principle, by the function of structural imagination. Just as spatial imagination gave the mammal an intuitive understanding of spatial causation, structural imagination  gives primates an intuitive understanding of structural causation. That is, subjective animals were more powerful than telesensory animals because they could understand how the structure of space imposes regularities on the object’s change of location, and in a similar way, manipulative animals are more powerful than subjective animals because they can understand how the geometrical structures of material objects in space imposes regularities on their interactions. Though there must be some new kind of behavior for it to guide, structural imagination  makes it possible to guide such behavior in using structural causes to control relevant conditions. It would be such a basic addition to the power of animal behavior to act on other objects in space that it could begin a stage

In order to predict a stage of evolution during which manipulative animals change in the direction of maximum power in using structural causes, however, it must be possible for subjective animals to try out a random variation that gives animals structural imagination. The functional description of the structures required for structural imagination shows why such a “manipulative animal system of representation” would require a higher level of neurological organization than subjective animals. Thus, only its possibility remains to be shown, in order to infer its inevitability as a necessary truth of ontological philosophy. But the higher level of neurological complexity required for structural imagination does not require such a radical change in brain structure that it is problematic. Moreover, what it does require explains all the ways that actual primate brains are different from mammalian brains.

After giving a functional description of the manipulative animal system of representation, I will describe the structures in the primate brain that realize it and, finally, confirm that it causes a stage of evolution by considering the empirical evidence for a radiation of higher primates.

The function of the manipulative level of neurological organization. The reason that a less radical random variation is needed to try out the manipulative level of neurological organization is that it happens in animals that already have imagination. It is just a higher level of part-whole complexity in the faculty of imagination.

Subjective animals have a faculty of spatial imagination that enables them to understand the structure of space. As we have seen, it comes from a memory that can link local images in sequences as a map of the territory and a behavior generator that generates covert locomotion that can call them up. This gives them subjective animals capacity to anticipate the consequences of locomotion, and motion generally, on the relations of objects in space. It is a conception of space, for it gives the subjective animal an understanding of spatial causation. They can perceive objects as having locations in space. It even enables them to see their own bodies as objects alongside other objects in space, and so the world in which subjective animals act is seen as singular and whole.

Besides the effects of motion on spatial relations, however, there is another spatial (and spatio-temporal) aspect of the world that is nearly as basic, which subjective animals cannot understand. That is structural causation, or the effects of geometrical structures on the interactions of objects that have them. Though mammals can guide locomotion in relation to large geometrical structures in their territory, they merely inhabit such structures and do not see them as the structures of material objects. Mammals have no need to understand structural causation, for they lack the ability to generate animal behavior of a kind that would use structural causes to control relevant conditions, that is, the capacity to manipulate material objects.

Structural imagination. What would make mammals more powerful, therefore, is an ability to manipulate material objects that is guided by an understanding of the consequences of manipulating them on how their geometrical structures appear, how the objects interact, and how their structures change. Understanding requires a faculty of imagination, but since mammals already have one form of imagination, the addition of another kind of behavior and another way recording images in memory is all that is required for what I will call “structural imagination.” It would enable manipulative mammals, for example, to see that round pegs won’t fit in square holes and that cups have to be oriented in a certain direction in order to contain water. More generally, it would enable them to see, in the geometrical structures of material objects, the structural global regularities they would cause, and with the capacity to manipulate objects, they would be able to use their geometrical structures as machines, or tools, in attaining their goals.

 A faculty of imagination enables the animal subject to see the actual against the background of what is possible (where “the possible” is basically various sets of sequences of images over time representing the kinds of events that can occur). In the case of manipulative animals, that is to see the actual geometrical structures of objects in the local scene against the background of what is possible by manipulating them.

If its mechanisms paralleled spatial imagination in subjective animals, what is possible by manipulation could be presented to the animal subject (via the projection of the local image to the caudate nucleus) as sequences of images along with perception (or the images derived from current sensory input).

Using the same basic memory system for recording memory groups in sequences (the cingulate gyrus), the detailed structure of the brain mechanisms for generating such sequences of images could be learned by experience. We have seen how the faculty of imagination can contain a form of reproductive causation in which behavior schemata evolve by reinforcement selection. The detailed brain mechanisms for structural imagination can be internalized from the world itself, because that is where structural causes actually generate global regularities, and those regularities are evident in now sensory input depends on manipulative behavior.

Since covert behavior would call up images from memory, thought about geometrical structures would involve appearances to the animal. Given our ontological explanation of the nature of consciousness, those appearances would be the kinds of ideas of imagination and memory to which the modern philosophers were referring. In manipulative animals, therefore, the representation of geometrical structures would be explicit.

None of these consequences of behavior is even conceivable by subjective animals, because the geometrical structures of objects are merely implicit in their local images (the combined telesensory images of objects located in the current scene). When the subjective animal focuses on an object, its location in space is represented explicitly, because its telesensory effects are registered along with other telesensory images as parts of a local image (according to input from the bodily condition), and that image contains information that can guide locomotion in relation to objects located in the local scene. The local image is the perceptual appearance of the local scene to the subjective animal, and it is conscious perception, because it has a phenomenal appearance.

But even though the object’s geometrical structure may correspond to a structure in the local image, or telesensory image in the complex idea representing the local scene, its geometrical structure is not represented as a geometrical structure, because the object’s geometrical structure is not used to guide manipulation in relation to it, but, at most, to identify the object or to recognize its kind. The geometrical structure may enable the mammal to see that it is an object of some kind that is located in space. But the mammal does not see it as having a kind of shape that can be changed or used in some way. It is as if subjective animals moved around in a world of shapeless (yet identifiable) objects with locations in space, and with the evolution of structural imagination, those objects acquired shapes that could be used as means in attaining animal goals.

The goals pursued by manipulative animals would be somewhat different from goals of subjective animals. Goals would be selected in the same way, by attaching desires to objects, where desires are basically dispositions to behave in a certain way toward an object. But manipulative animals would have new ways of behaving. For example, apes can pick gnats from one another’s fur and build beds, while cats can only lick themselves (or their kittens) curl up somewhere comfortable. New desires would evolve, because with structural imagination, manipulative animals would be controlling relevant conditions that were out of reach for subjective animals.

In short, whereas subjective animals have a conception of space, manipulative animals would have a conception of geometrical structures in space. And whereas subjective animals perceive objects as being located in space, manipulative animals would perceive objects in space as having geometrical structures. 

The various functions and sub-functions of the manipulative animal system of representation are depicted in the accompanying diagram of the manipulative animal behavior guidance system. Though neural mechanisms serving the function of selecting the kind of behavior are essential to the animal behavior guidance system, only the mechanisms serving the sensory input and behavioral output sub-functions are involved in the animal system of representation (the parts with the gray background). The diagram of the subjective animal behavior guidance system is also included here, because the contrast brings out what kind of higher level of neurological organization is required by the manipulative animal system of representation.

The functions of the systems making up the subjective animal behavior guidance system will be assumed, and only the additional systems required to supplement spatial imagination with the capacity to understand structural causation will be described. (The new neural mechanisms required to serve these new functions are identified below in the section, which uses the structure of the primate brain to show the possibility of the manipulative level of neurological organization.)

The brown squares represent the sensory input and behavioral output systems, and the black lines represent their basic connections to one another and the body (including telesensory input, somatosensory input, input about the current bodily condition, their combination as the perception of the object, and how that is responsible for overt behavior when some kind of behavior is selected). They are the basic structure of the telesensory animal system of representation. They are all depicted in the subjective animal behavior guidance system, but for simplicity in the manipulative animal behavior guidance system, the internal connections between the sensory input and behavioral output system (perception and input about the current bodily condition) are suppressed.

The red lines and red circles in the sensory input system and behavioral output system are what is added to the brain to give the subjective animal spatial imagination. Their complete circuits indicate the higher level of neurological organization in subjective animals (though as we have seen, the behavior generator is actually another complete circuit in itself, which is not represented in these diagrams).

The lavender lines and circles in the local image and body image of the subjective animal system of representation only suggest the kind of higher level of neurological organization required to give manipulative animals structural imagination, because there are actually four sets of new sub-systems, one for each hand and one for the object each hand can manipulate (which are not all represented for the sake of simplicity). The lavender lines represent the causal connections between the new sub-systems by which they function as the manipulative animal’s structural imagination.

Behavioral output system. Overt behavior for the whole body is caused by motor commands from the body image, as in other subjective animals. Those commands are issued by the behavior generator for the whole body using one of its behavior schemata. The behavior of hands in manipulative animals is caused in the same way, but within the context of the body. Hence, the behavioral output system for the hands is depicted as being located within the body image (though it is just a distinct part of the complete circuit through the frontal neocortex, corpus striatum and ventral thalamus for the whole body).

Overt manipulation is caused by motor output to muscles in each of the hands (and limbs) from each of the hand images, and the motor commands are generated by behavior generators for the hands using their behavioral schemata for manipulation. But the entire behavioral output system for the hands is depicted as being part of the body image, because, functionally, manipulation is a kind of behavior that is generated in just certain parts of the body.

Both are behavioral output systems, each with its own behavioral schemata, and thus, they are basically independent structural causes of behavior. In other words, there is a part-whole relationship between the behavioral output systems for the hands and for the body that corresponds to the part-whole relationship between the hands and body themselves. The behavior of the body as a whole is the larger context in which manipulative behavior is generated, as if manipulative behavior were simply appended to the behavior of the whole body. But they are distinct circuits, which can operate independently (and as we shall see in the linguistic brain, body and hands can both be operated by higher level linguistic schemata as if they were on a par).

The behavioral output system for the hand is labeled “hand image” because motor output to the hands is organized somatotopically as representations of the hands. The hand images are just parts of the body image, for the body image is organized somatotopically and includes the hands. And just as the body image receives somatosensory input from all parts of the body, the hand images receive somatosensory input from all parts of the hands.

But since tactile input, which is a kind of somatosensory input to the hands, is used to perceive the geometrical structures (and locations) of objects in the local scene, the diagram also represents tactile images as sensory input to the object image. (Thus, it might be called “exteroception,” to distinguish it from proprioception and interoception.)

The behavior generator for each hand contains schemata for all the various ways that that hand can manipulate objects, and depending on input from the goal selection system, the behavior generator uses one of its manipulative schemata together with the perception of the object in the local scene (that is, the object image, or the input from the sensory input system) to generate precise motor commands that are adapted to the geometrical structure of the object. Thus, hand-eye coordination is possible.

The behavior generator itself for both the body and the hands is also actually a complete circuit (through the ventral anterior nucleus of the thalamus, the frontal neocortex, and the corpus striatum, though it is not represented in the functional diagram). The behavioral schemata are structures recorded in the frontal neocortex, but it is a different region of the neocortex from the body image and hand images from which motor commands are issued (that is, the motor area).

The behavior generator can, with the appropriate behavioral schemata, generate a wide variety of manipulations. Since motor output sends commands separately to each part of each hand, they can be combined in many different ways. And manipulative schemata may involve many steps, because the behavior generator for each hand has feedback from the motor output to its hands by which the completion of one motor command can trigger the next in a sequence, generating temporally complex patterns of manipulation.

Sensory input system. As in subjective animals, the manipulative animal’s local image is constructed from the telesensory input of objects in the local scene, using input from the bodily condition to register telesensory images according to the stance of the body, head, eyes and ears that controls them in the local scene. But there is a new level of part-whole complexity in the local image, because telesensory (and tactile) images from objects are also registered as parts of an “object image” according to input about the bodily condition indicating the current motor commands and sensory input to the hands.

What makes an object image possible is that the local image also contains a new memory system by which telesensory and tactile images of objects can be recorded in sequences according to overtly generated manipulation and by which those images can be recalled in sequences according to covertly generated manipulation. The entire sensory input system for objects, both the object image and its memory, are contained within the local image, because, functionally, the object image is framed by the local image. It must be in order to represent effects of the geometrical structures of objects, because they occur within the local scene.

The object image functions more like the subjective animal’s entire map of local images than like their local image, because structural imagination gives the object image a meaning that resembles the meaning that spatial imagination gives to the local image.

As we have seen, the subjective animal’s capacity to see the telesensory images combined in the local image as spatial relations between objects that can change as they move comes from spatial imagination and its capacity to call up sequences of local images by covert locomotion. It gives the local image an “outside” relationship to other local images, and that makes the spatial relations among objects represented within the local image meaningful in a new way, because their locations can be seen as places where the animal might go and turn, calling up a new local image.

With the evolution of structural imagination, the local image has a comparable relationship to object images, except that object images are “inside” the local image, rather than “outside.” The object image is made up of sequences of telesensory and tactile images, representing the consequences of manipulation. But as they change, the local image remains the same (except for those changes). When a rock is rotated, for example, one side after another comes into view, but the background remains the same. However, the changes in the rock do not just happen, but are caused to happen by turning it in a certain way, and thus, the telesensory and tactile images that are part of an object image are meaningful. The geometrical structures that appear to the manipulative animal in those images have a meaning that comes from understanding geometrical structures in terms of the consequences of manipulating them, in much the same way as the meaning of the spatial relations among objects in the local scene comes from understanding them in terms of the consequences of locomotion and turning in relation to the objects. The manipulative animal can see the geometrical structure of the object as something that would change in a certain way, if it were rotated or twisted appropriately, just as the subjective animal can see the spatial relations among objects in the local scene as some that would change in a certain way, if some object were to move appropriately.

In other words, the object image and the territorial map have opposite relations to the local image and the telesensory and tactile images of which it is composed. The object image involves sequences of telesensory and tactile images within the local image, whereas the territorial map involves sequences of local images themselves, that is, outside the given local image.

In both cases, sequences of images are recorded in memory in conjunction with the kind of behavior being generated overtly at the time, and so they can be called up in the same order when behavior of the same kind is generated covertly. The difference is that the behavior that is relevant to changes in object images is manipulation, whereas the behavior that is relevant to changes in the local images is locomotion (and turning).

This parallel between spatio-temporal and structural imagination means that memory can function in basically the same way in both cases. In both cases, there must be a system that can record images in sequences. It must be able to record them according to the kind of overt behavior being generated at the time the images are received. And it must be able to recall images from memory according to covert forms of such behavior as labels, so that they appear as imagination to the subject. The same system that serves these functions in spatial imagination can be used for structural imagination, if (1) manipulative animals have a new form of behavior and (2) the images called up from memory can be handled somehow independently of the local image in which they occur.

(1)   The foregoing description of the behavior generator for the hands explains how manipulative animals can have a new form of behavior. It comes from having hands as four new parts of the animal body. Thus, in addition to locomotion, turning, and the orientation of the body, head, eyes, etc., the manipulative animal has ways of behaving that depends on the hands and how the arms put the hands in certain locations in the local scene.

There are many different ways of manipulating objects. Objects can be turned, for example, in three independent ways in three dimensional space. The fingers can trace their outlines in different ways. And there are various ways of acting on objects to change their shape, as in folding paper.

Assuming that overt and covert behavior have the same kinds of connections to the memory system (projections to the anterior cingulate gyrus and its projection in turn to the posterior cingulate gyrus), each kind of manipulation could call up a different sequence of sensory images.

(2)   However, since the memory system is already being used for spatial imagination, the new memory system must be able to have a different kind effect on the local image than locomotion and turning. In order to serve its function, instead of calling up new local images, it must be able to call up new telesensory (and tactile) images within a given local image. That is what is labeled as the “object image” within the local image of the functional diagram.

Though at any moment, the object may be just one of the telesensory images that are combined as the local image representing the current local scene, the object image is different, because it can be changed by covert manipulation independently of the rest local image, at least in imagination. That is the object image’s connection to the memory system within the local image in the functional diagram.

The world appears differently to animals with structural imagination. Not only do they see themselves as a body located in a local scene along with other object with spatial relations to one another that can be changed by locomotion, but they also see the objects in the local scene as having structures.

As the manipulative animal plays with an object, say turning it, it receives a series of telesensory and tactile images, and sensory images are registered according to what its hands are doing to it. This can record an object image for that particular object in memory. Thereafter, even though only one aspect of the object may be causing sensory input at the moment, the object image can represent other aspects of the object as well, for images of the other sides can be called up from memory. Thus, manipulation involving the backside, for example, can be rehearsed in imagination before it is generated overtly. The manipulative animal perceives the object as having a certain back side. Or objects with moving parts, such as a box with a lid, can be imagined to have configurations that are not seen.

But the capacity to record and recall sensory images according to manipulation is not just a memory for particular objects, any more than the capacity to record and recall local images according to locomotion is just a map of a particular territory. As we have seen, learning to construct and use memory maps gives the subjective animal a general conception of space, by which they can understand any relations in space by imagining the kinds of motion that would be involved in traversing or changing them. Similarly, the manipulative animal acquires the concept of geometrical structure.

The subjective animal’s general understanding comes from the structure of the memory system on which imagination is based, because the same kinds of behavior generated in similar situations have the same kinds of effects. That is what the spatial structure of the world contributes, and thus, as behavioral schemata evolve by reinforcement selection, the connections in the memory system internalize those general relations. Their internalization is the subjective animal’s conception of space.

The memory system in manipulative animals internalizes a further spatial aspect of the world in the same way. The same kinds of manipulation have the same kinds of effect on objects with the same kinds of geometrical structures. Rotating them in any of the three independent planes possible leads to similar sequences of telesensory images, for example, and likewise for bringing the object closer or moving it to the side. The appearance of hollow objects from the outside can be used to infer how they would appear from the inside. Similar shapes also cause similar tactile images when the fingers move relative to the object in certain ways.

Thus, by recording objects with similar geometrical structures together as the same generalized object images, there would be a “map” of the consequences of manipulation that would work for all objects of that kind involving that kind of manipulation. And the different sequences of images involved in manipulating them in different ways could be interconnected like the sequences of local images representing different directions of locomotion from the same scene, because changing the direction of rotation would have effects that parallel changing direction at some point in the territorial map.

The contained form of reproductive causation in which behavioral schemata evolve by reinforcement selection, which has been described in subjective animals, would enable manipulative animals to acquire the ability to manipulate objects in all ways that are physically possible for the kinds of hands they have, as long as they enabled the animal to control some relevant condition and were reinforced by the memory circuit (in alliance with the goal selection system). Thus, they would internalize structural global regularities about change in the world as part of the structure of their structural imagination. And manipulative schemata would tend to evolve in the direction of more abstract representations of the geometrical structures of objects, though there may be a limit to how useful such abstract understanding is in ecological niches where they do not help satisfy desires.

Structural imagination  would also enable manipulative animals to learn complicated tasks involving hand-eye coordination, if that were useful in satisfying desires. The motor output is sent to the hands from an image of the hands that is part of the body image, and by sending separate motor commands from different parts of those images of the hands to different parts of the hands, it is possible for the behavior generator to generate very complex bodily movements in relation to objects in the object image. A set of motor commands is sent each moment and it can generate long sequences of such sets of such motor commands.

To be sure, the behavior generator for the hands needs a “schema” for each kind of behavior, but such manipulative schemata can also be learned from experience with the world, that is, by operant conditioning and the evolution of behavior schemata in the behavior generator, just as subjective animals learn complex routes through their territories.

Manipulative animals may even learn a behavioral schemata that enables them to recognize how other manipulative animals are manipulating objects. That is, when they learn to manipulate objects in certain ways, they also learn to recognize when other manipulative animals are manipulating objects in the same way. Manipulative animals with that skill could not only recognize kinds of manipulation in other animals, but could also understand the manipulation in the sense of knowing how to act the same way. Since manipulation can be generated covertly, manipulative animals would be able to call up telesensory images from memory of another body behaving in a certain way, so that the behavior it is rehearsing could be seen from the outside, where it may be easier to imagine how it would fit into the current scene.

However, far from being a mere recording devise for learning complex manipulative tasks or for viewing aspects of objects in imagination, the most general new power is the capacity to think about objects as having geometrical structures. With abstract concepts of geometrical structure, particular objects can be represented in structural imagination by adding whatever details about the particular object may be relevant to a general concept. Cups hold water in the same way, and that general conception of cups becomes a formula for memories of particular cups. And their intuitive understanding of geometrical structure would enable manipulative animals to think about objects in terms of how they would interact with other objects in the local scene because of their geometrical structures, that is, as an understanding of structural causation. Thus, an ape can see branches from trees as something to be assembled as a bed to sleep comfortably, or, as in the case of Kohler’s ape, see how a box and a stick would enable it to knock down a banana which was otherwise out of reach. It is, in other words, the capacity to see objects as tools or machines. That is the sense which objects in space are seen as having geometrical structures. 

The goal selection system serves the function in the manipulative animal behavior guidance system as the subjective. It attaches desires to objects. Such desires are actually dispositions to use certain behavioral schemata in relation to the object identified, and thus, not only can manipulative animals have new or refined kinds of desires to attach to objects, but new kinds of problems can be solved in satisfying them. Though it may not be possible to remove an obstacle that is frustrating the satisfaction of a desire by locomotion, it may be possible by manipulation. For example, primates can figure out how to unlatch gates, though dogs cannot. 

The possibility of the manipulative level of neurological organization. The functional description of the manipulative animal system of representation shows how a higher level of neurological organization would make animals more powerful in a basic way, by giving them the use of structural causation. Since the manipulative level is functional in a profound way, all that is required to infer its inevitability is sufficient reason to believe that such a neurological structure can be tried out as a random variation on the brains of subjective animals.

The possibility of the higher level of part-whole complexity is not very problematic in this case. Since subjective animals already have the kind of memory that can record and recall images in sequence according to the kind of behavior that causes them, it requires only two kinds of changes, one each in the behavioral output and sensory input systems. One change is the addition of four new subjective level behavioral output systems, one for each hand, installed as parts of the behavior generator for the body as a whole. The other change is some way of handling as many as four object images within the local image of the sensory input system.

These changes are less radical than what was needed to cause earlier stages of chordate evolution, because they do not involve a further centralization of the whole nervous system.

In order to use telesensory input to guide locomotion, the functions of all the local reflexes of the somatosensory chordate's nervous system had to be centralized in the non-mammalian vertebrate brain where the interaction between sensory input and behavioral output could be separated from the goal selection system.

And in the evolution of subjective animals, the three systems of the animal behavior guidance system (behavioral output, sensory input, and goal selection), which were located in the hindbrain, midbrain and forebrain in non-mammals, had to be centralized in the mammalian forebrain, where the behavioral output and sensory input systems could each be composed of several telesensory level brain mechanisms and interconnected so that behavior could be generated both covertly and overtly.

Centralization is not, however, the only way a higher level of part-whole complexity can be introduced in neurological mechanisms. Instead of housing multiple lower-level systems alongside one another as part a whole new structure, it is possible to introduce a higher level of part-whole complexity by multiplying parts within the main systems of an existing animal system of representation. Though this is a less radical way of organizing nervous mechanism on a higher level of part-whole complexity, it is just as much the cause of a new stage in animal evolution, because it is the fountain of a whole new range of powers that can evolve gradually over a long period of time.

Since a relatively minor change of this kind is surely within the range of random variations that can be tried out by the biological behavior guidance system, we could take the possibility of mammals with structural imagination for granted and conclude that manipulative animals are inevitable. But there is further confirmation, because the kind of manipulative mammal we are looking for does exist on earth, and its brain does have just the kinds of structures we expect.

The mammals that most obviously have structural imagination  are primates. But they may not be the only manipulative animals. It is possible that elephants also have a kind of structural imagination for guiding the behavior of their trunks.

In the primate brain, at least, the is good evidence of both kinds of changes in the mammalian brain.

In the behavioral output system of primates, there is clear evidence of multiple behavioral output systems within the behavior generator for the body as a whole.

And there are just the kinds of changes in the sensory input system we should expect for accommodating object images within the frame of the local image.

Behavioral output system. Since the evolution of hands for manipulating objects is the clearest evidence of a higher level of neurological organization, let us begin with the behavioral output system.

The body image for issuing overt motor commands to all parts of the body includes the primary and supplementary motor cortex as well as the somatosensory cortex, and as we might expect, the areas devoted to the hands in the primate brain are much larger than in non-primates. They are the hand images for overt behavior in the functional diagram.

There are presumably also larger areas for the hands in the covert body image (that is, in the premotor neocortex and in the anterior geniculate neocortex) for generating covert manipulation in parts of the body as a whole.

The new hand images of the behavioral output system are connected to the behavior generator in the same way as the rest of the body, for the hand images are just part of the body image. That is, manipulation in each hand is generated as a separate part of the circuit from the anterior neocortex through the corpus striatum, the ventral lateral and ventral anterior nuclei of the thalamus, back to the anterior neocortex. Yet these hand behavior generators are just part of the body behavior generator. That is how we expect the higher level of neurological organization to be accomplished from the functional diagram.

In mammals, behavioral schemata are contained in the frontal neocortex, and in primates, there is such a great overall increase in the size of the frontal neocortex that primates are often said to have a new "prefrontal" lobe. (The prefrontal neocortex is just anterior to the frontal eye field, including areas 9 and 10 as well as 45 and 46, and all these areas granular, having all six layers of cortex.) These new areas of neocortex, like the frontal neocortex of non-primates, are available to the behavior generator, because they are parts of its circuit for generating behavior.

They all project to the putamen of the corpus striatum, and because the corpus striatum projects to the ventral anterior nucleus of the thalamus, the circuit is completed by the projection from the ventral anterior nucleus back to the frontal neocortex. This circuit lines up 2-D arrays of neurons in each structure it passes through, and thus, as one cycle of interactions through the circuit follows another, the spatial complexity of the behavioral schema in the frontal neocortex can be translated into a temporal sequence of motor commands. But since the projections to the putamen tend to consolidate different areas of the neocortex that are involved in generating each part of the body, there are distinct circuits within the circuit for the body as a whole for generating the behavior of each hand separately.

The corpus striatum is the central organ of the behavior generator, and there are indications of special provisions for generating manipulative behavior. The corpus striatum, as we have seen, uses the active behavioral schema to combine the local image containing (as we shall see) the object image with a similar projection from the hand images and body image to issue precise motor commands to relevant parts of the body.

Since the object images are contained in the local image as their frame, they are supplied to the corpus striatum via their projection to the caudate nucleus, that is, as what the manipulative animal perceives.

However, in order to issue motor commands to the hands that coordinate their movements with visual input from objects in the scene, the behavior generator must be able to identify the parts of the visual input that are coming from the hands so that it can guide the hands in relation to the objects.

No such identification of telesensory input is required for locomotion, since the animal does not need to be able to see its body to move around in space relative to there objects.

But manipulation generally involves hand-eye coordination, and thus, in order to facilitate it, we should expect that the projection from the local image to the behavior generator circuit (via the caudate nucleus) would enable the animal to identify which part of the local image is coming from the hands.

This would account for the primate's new and highly unusual projection from area 9 of the prefrontal cortex, just anterior to the frontal eye fields, to the caudate nucleus. This is the only projection from the frontal neocortex to the caudate nucleus (where the corpus striatum receives the representation of the object).^[1] It could provide markers in the local image indicating where the hands should be located. (There is, of course, no comparable problem using tactile sensations to control manipulation, because they have a fixed location relative to the hand and body images.)

Structural imagination requires the capacity to generate manipulation both overtly and covertly, and their effects on the brain must be parallel (despite being so different in their effects on the body), because in order for memory groups to have labels by which covert manipulation can recall them, those memories must have been recorded according to the overt manipulation that was causing them when they were received. That is how spatial imagination works, and now we are assuming that the same brain structure can also be used for as the memory system for structural imagination, that is, with manipulation and not just locomotion.

The crucial brain structure for imagination is the cingulate gyrus, which is part of the memory circuit (starting from the anterior temporal neocortex, proceeding though the hippocampus, fornix, and anterior nucleus, and then back to the cingulate gyrus, which has rich interconnections with adjacent areas of neocortex). Since the behavioral output system contains four hand images within its body image, the hand images have the same connections to the cingulate gyrus as the body image used in spatial imagination, and they have similar effects.

Both hand and body images for overt behavior project from the frontal neocortex to the anterior cingulate gyrus (by way of association fibers from the motor and supplementary areas), and both the hand and body images for covert behavior also project from frontal neocortex to the anterior cingulate gyrus (by way of the association fibers from the premotor neocortex and frontal neocortex where behavioral schemata are stored and activated).  

The anterior cingulate gyrus is (agranular) motor-type neocortex, like the rest of the anterior neocortex, and it projects to the posterior cingulate gyrus, which has rich interconnections with the sensory and parietal areas of neocortex in the posterior cerebrum. Since both anterior and posterior regions of the cingulate gyrus are part of the memory circuit, it is likely that the cingulate is where images are recorded in sequences with behavioral labels for both spatio-temporal and structural imagination.

In the case of structural imagination, the only difference (apart from effects on images in the sensory input system) is that the behavior is manipulative, rather than locomotor. That is, when object images are recorded in memory, behavior is overt and object images are recorded with labels indicating the kind of manipulative behavior, and then, when behavior is covert (and the projection to the anterior cingulate gyrus from the hand and body images for overt behavior is not active), there is still input to the anterior cingulate indicating the kind of behavior (via the projection from the premotor and frontal neocortex). Thus, covert behavior gives the anterior cingulate gyrus sufficient information of the right kind to call up memory groups by their behavioral labels.

It should be emphasized, again, that most of the detailed neural connections required for the memory labels can be acquired from experience with the world, since the late phase of neurological development, called “learning,” is a contained form of reproductive causation in which behavioral schemata evolve gradually in the direction of greater power to satisfy desires by a form of natural selection through reinforcement.

That is, when a behavioral schema is successful in generating behavior that satisfies some desire, the memory circuit (using criteria built into it with its close and ancient connections with the seat of desire) ties the active neurons in every area of the neocortex together in a memory group (by the growth of synapses among them), so that when some significant portion of them is activated again, the whole group of neurons tends to fire. This occurs not only in the anterior and posterior neocortex, but also in the cingulate area and its connections to them.

Thus, when behavioral schemata that are responsible for successful actions on objects are naturally selected, connections are established through the cingulate gyrus between the kinds of behavior involved and the images of the sensory input system that are involved so that those images are labeled by the relevant kind of behavior. The labels are simply part of the behavioral schema that is naturally selected by reinforcement.

Such labels can, as we have seen, become generalized as abstract, intuitive concepts about the structure of space and the geometrical structures of object in space whose meaning is spelled out by how sequences of images represent the effects of locomotion and manipulation.

What makes structural imagination different from spatial imagination is that, instead of covert locomotion and turning calling up local images in sequences to represent their effect, covert manipulation (such as rotating or bending the object) is calling up telesensory and tactile images of object in sequences to represent their effects. But that difference means that they must have different effects on images in the sensory input system, and thus, it remains only to see how the object image is installed. 

The sensory input system. In the sensory input system, the higher level of neurological organization should show up as a new capacity for representing objects within the local image that keeps track of how the geometrical structure of each is affected by rotation, translation, and other forms of manipulation. There is evidence for such object images in the memory circuit, but in order to see what is different, let us begin by recalling how the local image is constructed in the subjective animal system of representation and consider what is added in the manipulative animal.

The telesensory images registered in the sensory neocortex (the striate neocortex for vision and the superior temporal neocortex for hearing) are assembled as parts of a local image, as we have seen, in the inferior parietal neocortex, using projections from the body image as input about the current bodily condition.

In primates, however, there is additional input about the current bodily condition, because the body image now contain four hand images, each with its own somatosensory input. This information is used in a new way to construct the local image, because it reveals the shapes (and certain other properties) of objects in the local scene as well as their precise locations. Touch is a new (extroceptive) sensory modality, along with the other modalities, and its tactile images must be assembled as part of the local image as well. Somatosensory input from the hands is transformed into tactile images in the superior parietal neocortex, posterior to the somatosensory neocortex, and its connections to the inferior parietal neocortex make it possible to include them as another part of the local image, along with visual and auditory images.

The inferior parietal area also receives information from the frontal eye fields about the direction of fixations of the eyes, so that visual images from the different fixations of the eyes can be assembled into a visual appearance of the whole scene. There is evidence from primates that fixations of the eyes mark targets for manipulation in the representation of the current scene before it is projected to the behavior generator (that is, to the corpus striatum by way of the caudate nucleus).^[2]

Furthermore, there is abundant evidence that local image is required in order to guide the behavior of the hands in relation to objects. The object image is contained by the local image, just as the object itself is located in the local scene, and the local image is assembled in the inferior parietal neocortex. Lesions in the inferior parietal lobe of one hemisphere may cause subjects to neglect entirely the limbs on the side of the body it controls. Even when the hands are not paralyzed by the lesion, patients may be unable to locate objects on that side of their bodies or to identify them by touch (or visually). Lesions may show up, however, only in the inability to put one dimensional units together as a two dimensional configuration, for example, when drawing.^[3]

The local image is, however, just the frame for the object image. Structural imagination calls up sequences of telesensory and tactile images of objects within the local image according to the kind of covert manipulation. (Imagine turning a cup of water upside down.) Thus, the object images themselves must be handled in a different way from local images by the sensory input system. They must be contained in the local images like objects contained in the kind of space that mammals can understand. There are four pieces of evidence that the posterior cerebrum contains the new way of handling images that we expect.

(1) There are two changes within the memory circuit itself that suggests that images are being recorded as groups in a new way. The object image needs a new pathway, because its telesensory and tactile images must be recorded as groups separately from the local image if the later is to serve as its frame. The hippocampus, which is the staging area for forming neurons into memory groups, projects by way of the fornix to the anterior nucleus of the thalamus, which is basically a relay back to the cingulate neocortex. But this projection divides into two pathways, one that passes through the mammillary body and thence to the anterior nucleus, and the other that proceeds directly to the anterior nucleus. In primates, the later, direct pathway is relatively larger than in other mammals. The former, indirect pathway must be responsible for the local image, because the mammillary bodies are known to be essential to spatial orientation. If the images being processed through it were all oriented as part of the local image, then the images passing through the direct pathway could be processed independently of the local image, as if the local image were its frame.

(2) There is further evidence that the expanded, direct projection of the fornix to the thalamus is for processing the object image, because it sends fibers not only the anterior nucleus of the thalamus, but also to a new (or expanded) nucleus of the primate thalamus, the lateral dorsal nucleus. The lateral dorsal nucleus is only indistinctly separated from the anterior nucleus and appears to be a posterior extension of it. The lateral dorsal nucleus has two-way connections with the cingulate gyrus, parietal lobe and, perhaps, even the occipital cortex. Such direct connections with the parietal and occipital cortex are not found in other mammals, but they would make it possible to connect the separate object image to the local image as its frame. The result would be that the object represented by the object image would appear to be located somewhere in the local scene represented by the local image, that is, as located in the space of which subjective animals have a conception.

(3) Just posterior to the lateral dorsal nucleus of the thalamus is another thalamic nucleus newly prominent in primates, the lateral posterior nucleus. It projects to the superior parietal neocortex, where tactile images are analyzed, and that suggests that its function is to help integrate tactile images as parts of the object image, rather than the local image. The lateral posterior nucleus has connections within the thalamus to the pulvinar, which projects to the inferior parietal (and occipital) neocortex, on one side, and with the ventral posterior nucleus, where somatosensory input is relayed to the hand images, on the other side. Thus, its function seems to be connected with manipulation, just as the object image would require.

(4) Finally, there is a new, direct projection found in primates from the anterior temporal neocortex to the pulvinar.^[4] The anterior neocortex is the staging area for all input to the hippocampus and the only area of neocortex that does not receive a projection from the thalamus, and thus, it contains information about both the local image and the object image. The pulvinar, to which it projects, is the nucleus of the thalamus (just beneath the lateral dorsal nucleus) which projects to the parietal and occipital cortex, where the local image and the object image are processed. It is, therefore, a new circuit through the neocortex and thalamus (from the parietal/occipital neocortex to the anterior temporal neocortex to the pulvinar and back to the parietal/occipital area), and its function could well be to integrate the object image into the local image.

There are various ways in which these circuits might work in detail, but taken together, these four prominent changes in the posterior primate cerebrum are just what the sensory input system would need, if it were able to processes telesensory and tactile images as parts of an object image independently of the local image. They can be taken as evidence, therefore, of the new circuit between the object image and memory depicted as contained by the local image in the functional diagram of the manipulative animal behavior guidance system.

Nor is there any reason to doubt that the object image can handle as many objects as there are handles, which would put it on a higher level of neurological organization in the same way as the behavioral output system.

These differences between (higher) primate and non-primate brains suggest just the kind of higher level of part-whole complexity we would expect of animals with a structural imagination that have evolved from subjective animals. It seems that there are multiple channels through the behavioral output systems of the subjective animal system of representation which could be handling each of the hands separately as parts of the body as a whole. And there is a new way of representing the objects they manipulate as part of the local image. Thus, the cingulate neocortex has connections to both systems of just the right kinds needed to record and recall sequence of telesensory and tactile images as object images according to covert manipulation. Since this way for random variations to try out a higher level of neurological organization is not problematic, given that chordates sets up nervous system using 2-D arrays of neurons as units, we may take this as evidence that such a level of neurological organization is possible. Since it is possible, we infer that the evolution of animals like higher primates is inevitable.

The gradual evolution of primates. That primates are more powerful than other mammals in the way that is implied by having structural imagination is, perhaps, patent in the vast superiority of their hand-eye coordination, but there is not much data about their cognitive capacities.

The most famous evidence of structural imagination is Kohler's ape, which put one box on top of another in order to be able to use a stick to reach a banana. Moreover, when monkeys or apes are provided with paints and a canvas on which to paint, they enthusiastically construct geometrical structures of bright colors, whereas other mammals do not. It has been obvious for so long that they like to play with unchanging geometrical structures of matter that such behavior is often called "monkeying around" with them.

But more systematic evidence is conceivable. Primates should be able to count (or keep track of) more objects in the current scene than other mammals. And a properly trained primate should be able, for example, to tell how an object is oriented after a series of rotations. That is, if a primate were conditioned to pick out a distinctive face of a die and the die were rolled, then it should be able to learn to point to the side of die that the distinctive face is on, even when it cannot be seen. But other mammals should not be able to do so. Or primates should be able to pick out from a group of similar objects the one that is the folded up version of a pattern laid out flat, as in standard tests of spatial imagination, while other mammals cannot. Experiments like these would show that primates can make the inferences implied by the mechanism described in the last two chapters as structural imagination.

Evidence that primate evolution is due to the greater power afforded by structural imagination can also be found in the history of evolution, although it may not be obvious at first, because it seems possible to explain primates a mere adaptation to a special habitat. That is, it might be argued that the evolution of primates from mammals can be explained on the model of the evolution of amphibians from fish, of reptiles from amphibians, or of birds from early dinosaurs. In these cases, as we have seen, there is no change in the part-whole complexity of the nervous system, but merely an adaptation of the same kind of animal behavior guidance system to new habitats — to life on land and in the air. In other words, the significant changes occurred in the body, not the brain.

Living in trees is certainly a special habitat, and there is no doubt that primates have bodily characteristics that are well adapted to it. All primates have hands that are capable of grasping structures on trees. They all have acute binocular vision, with both large eyes directed forward. More generally, they are active and restless animals which react quickly to stimuli.

Living in trees is not, however, just a specialized environment. It is also a more demanding environment in which the power that comes from structural imagination would be richly rewarded. Arboreal life is surely possible without having a structural imagination. Indeed, the earliest ancestors of primates probably did not have structural imagination. They stem from which all the species of mammals radiated were insectivores, and primates are more closely related to those insectivores than any of the other classes of mammalian species (except possibly bats). The species from which primates are thought to have evolved has a living remnant, the small, primitive tree shrew (Tupia), and it is highly unlikely that tree shrews have a structural imagination. But given the demands of their arboreal habitat, this lineage of mammals must have had a "need" for structural imagination, if the required random variation was within the range of those being tried out, for it would have made them more powerful animals. And the need would account for the evolution of structural imagination, given that evolutionary change is how reproductive cycles add up in space over time. That means, however, that there must be some juncture in the evolution of primates at which it was added.

There is, indeed, fossil evidence of such a development, for there have been two radiations of primates in forests. The first radiation was that of the prosimians, represented by lemurs and tarsiers, about 65 million years ago during the radiation of mammals. Then there was a second radiation, the anthropoids — represented by the New World monkeys, the Old World monkeys, the great apes and humans — roughly 40 million years ago. As a result of the first great primate radiation, there were lemurs as large as chimpanzees, and anthropoids are thought to have evolved from lemurs of some kind.

Although prosimians typically have elongated flexible limbs with grasping hands and adhesive pads for grasping branches firmly as well as binocular vision, there are marked changes in anthropoid primates. Anthropoid eyes are directed forward more completely. Most anthropoids are able to sit in an upright position, which frees their hands for manipulating objects. The thumb is set apart from the other digits more extremely. Arm joint, elbow and wrist are capable of more rotation, so that the hand can be rotated in all directions possible in space. These are indications of a new capacity for manipulation, beyond a mere grasping reflex, which is probably all that prosimians have.

Moreover, only a general increase in power can explain why the radiation of anthropoid primates replaced prosimians in all but the lowest-energy ecological niches. It cannot have been tapping a new source of energy, because primates of all kinds eat almost anything edible in their habitat.

The actual history of the evolution of higher primates confirms, therefore, that they are a later stage in the evolution of multicellular animals, with a higher level of neurological organization. They are not merely better adapted to their environment, but inherently more powerful as animals. Thus, we would expect a primate radiation out of the forest. Much as mammals took over the high-energy ecological niches once occupied by dinosaurs, anthropoid primates should invade ecological niches outside the forest, displacing less powerful animals from those sources of free energy.

The almost complete absence of anthropoid apes from ecologies other than dense rain forests does not show, however, that there has not been a radiation of primates. Human beings are a kind of anthropoid primate that are adapted to tapping energy in this new environment. Although they have occupied all the other ecologies on the planet, they are not a mere radiation of primates, because they are so powerful that their radiation is now even displacing the great apes from their original environment. The primate radiation into the grasslands occurred long ago and has already led to another stage of evolution, which has already displaced anthropoid apes from this new environment. Indeed, as we shall see, the evolution of human beings involves such a revolutionary change that it starts yet another ladder of evolutionary stages in a different direction — as different from the stages of neurological evolution as those stages are from the stages of biological evolution.