Speculative Web Space

Critical Digital Humanities Bibliography

About Critical DH Bibliography

Search all rows: reset

Home > Bibliography > Bibliography: Principles of neurodynamics : perceptrons and the theory of brain mechanisms

"The theory to be presented here is concerned with a class of 'brain models' called <i>perceptrons</i>. By 'brain model' we shall mean any theoretical system which attempts to explain the psychological functioning of a brain in terms of known laws of physics and mathematics, and known facts of neuroanatomy and physiology. A brain model may actually be constructed in physical form, as an aid to determining its logical potentialities and performance; this, however, is not an essential feauture of the model approach. The essence of a theoretical model is that it is a system with known properties, readily amenable to analysis, which is hypothesized to embody the essential features of a system with unknown or ambiguous properties - in the present case, the biological brain... Perceptrons are of interest because their study appears to throw light upon the biophysics of cognitive systems: they illustrate, in rudimentary form, some of the processes by which organisms, or other suitably organized entities, may come to possess 'knowledge' of the physical world in which they exist, and by which the knowledge that they possess can be represented or reported when occassion demands. The theory of the perceptron shows how such knowledge depends upon the organization of the environment, as well as on the perceiving system." p3

"A perceptron consists of a set of signal generating units (or 'neurons') connected together to form a network. Each of these units, upon receiving a suitable output signal (either from other units in the network or from the environment) responds by generating an output signal, which may be transmitted, through connections, to a selected set of receiving units. Each perceptron includes a sensory input (i.e., a set of units capable of responding to signals emanating from the environment) and one or more output units, which generate signals which can be directly observed by an experimenter, or by an automatic control mechanism. The logical properties of a perceptron are defined by: 1. Its <i>topological organization</i> (ie, the connections among signal units); 2. A set of <i>signal propagation functions</i>, or rules governing the generation and transmission of signals; 3. A set of <i>memory functions</i> or rules for modification of the network properties as a consequence of activity." p4

"At the time that the first perceptron model was proposed, the writer was primarily concerned with the problem of memory storage in biological systems, and particularly with finding a mechanism which would account for the 'distributed memory' and 'equipotentiality' phenomena found by Lashley and others (Refs. 48, 49, 95). It soon became clear that the problem of memory mechanisms could not be divorced from a consideration fo what it is that is remembered, and as a consequence the perceptrong became a model of a more general cognitive system, concerned with both memory and perception." p4

"There are at least two basic points, which are fundamental to a theory of brain functioning, on which most of the present-day theorists seem to be in agreement. First is the assumption that the essential properties of the brain are the topology and the dynamics of impulse-propogation in a network of nerve cells, or neurons. This has been contested by a few theorists who hold that the individual cells and their properties are less important than the bulk properties and electrical currents in the cortical medium as a whole (c.f. Kohler, Ref 45)... It will be assumed that the essential features of the brain can be derived in principle from a knowledge of the connections and states of the neurons which comprise it. Secondly, there is general agreement that the information-handling capabilities of biological networks do not depend upon any specifically vitalistic powers which could not be duplicated by man-made devices... Nonetheless, all currently known properties of a nerve cell can be simulated electronically with readily available devices. It is significant that the indivicual elements, or cells, of a nerve network have never been demonstrated to possess any specifically psychological functions, such as 'memory', 'awareness', or 'intelligence'. Such properties, therefore, presumably reside in the organization and functioning of the network as a whole, rather than in its elementary parts." p9

"It is hoped that network which consists of neruon-like elements, and is capable of computing the required functions, will be found to resemble a biological nerve-net in its organization and the computational principles employed." p12

"The fundamental thesis of the McCulloch-Pitts theory is that all psychological phenomena can be analyzed and understood in terms of activity in a network of two-state (all-or-nothing) logical devices... Despite teh apparent adherence to an outdated atomistic psychological approach, there is an important contribution in the recognition that the proposed axiomatic representation of neural elements and their properties permits strict logical analysis of arbitrarily complicated networks of such elements, and that such networks are capable of representing any logical proposition whatever." p13

"Hebb (Ref. 33) and Hayek (Ref. 32), following the tradition of James Stuart Mill and Helmholtz, have attempted to show how an organism can acquire perceptual capabilities through a maturational process... Hebb's model is more detailed in its biological description, and suggests a process by which neurons which are frequently activated together become linked into functional organizations called 'cell assemblies' and 'phase sequences' which, when stimulated, correspond to the evocation of an elementary idea or percept." p22

"The use of a digital compuer by Rochester and associates was mentioned above in connection with Hebb's model. Simulation of a statistically connected network to investigate possible learning capabilities was first carried out successfully by Farley and Clark in 1954 (Ref. 10)." p24

"A contribution of considerable methodological significance was Ashby's 'Design for a Brain', in 1952 (Ref. 3). While Ashby's work (despite its title) does not specify an actual brain model in our present sense, it develops the rationale for an analysis of closed systems which must include the environment as well as the responding organism and rules of interaction as the object of study." p25

"Perceptrons are not intended to serve as detailed copies of any actual nervous system. They are simplified networks, designed to permit the study of lawful relationships between the organization of a nerve net, the organization of its environment, and the 'psychological' performances of which the network is capable... they represent extreme simplifications of the central nervous system, in which some properties are exaggerated, others suppressed... The main strength of this approach is that it permits meaningful questions to be asked and answered about particular types of organization, hypothetical memory mechanisms, and neuron models." p28

"In the last chapter a methodological doctrine was proposed, which undertakes to evaluate classes of brainlike systems by comparing their performance with that of biological subjects in behavioral experiments; by gradually increasing the sophistication and varying the axiomatic constraints which define the experimental systems, it is hoped that models which closely resemble the biological prototype can ultimately be acheived... What are the parametric constraints, functional properties, and performance criteria which must be met, in order to acheive a model which is a plausible representation of the brain?" p29

"Two main types of synapses are recognized: excitatory and inhibitory. It is generally assumed, although it has not been proven, that a single neuron is either all excitatory or all inhibitory, in its effect upon post-synaptic cells." p32

"Essentially, the currently accepted concept is that the dendritic structure and cell body jointly act as an integrating system, in which a series of incoming signals interact to establish a pre-firing state in a region at the base of the axon, from which impulses originate. If this pre-firing state reaches a threshold level (presumably measured by membrance depolarization) at a point within the critical region, a spike potential is initiated, and spreads without decrement along the axon... Successions of impulses arriving at the same synapse can sometimes cause an increase in the sensistivity of the receiving membrane (faciliation) and can sometimes cause a progressive diminution in sensitivity (Ref. 11). There is evidence to suggest that different local patches of surface membrane are differently specialized, and respond in different ways to impulses received, even within the same neuron. Some of these regions appear to act as sources of internally generated signals, which may lead to spontaneous activity of the neuron, and the emission of spike impulses without any input signals from outside the cell." p32

"The arrival of a single (excitatory) impulse gives rise to a partial depolarization of the post-synaptic membrane surface, which spreads over an appreciable area, and decays exponentially with time. This is called a local excitatory state (l.e.s). The l.e.s. due to successive impulses is (approximately) additive. Several impulses arriving in sufficiently close succession may thus combine to touch off an impulse in the receiving neuron if the local excitatory state at the base of the axon achieves the threshold level. This phenomenon is called <i>temporal summation</i>. Similarly, impulses which arrive at different points on the cell body or on the dendrites may combine by <i>spatial summation</i> to trigger an impulse if the l.e.s induced at the base of the axon is strong enough." p33-34

"The human brain consists of some 10<sup>10</sup> neurons of all types. These are arranged in a network which receives inputs from receptor neurons at one end, and conveys signals to the effector neurons at the output end." p35

"The general picture of the nervous system, then, is one of a large set of signal generators, each having one or more outputs, on which nerve impulses may appear. These impulses may vary in frequency, and to some extent in amplitude, but seem to carry information mainly in pulse-coded form." p35

"A typical cell in the cerebral cortex receives input connections from some hundreds of other cells, which may be located in widely scattered regions, but its output is more likely to be transmitted to a relatively localized region. Cells which receive sensory input signals are likely to have a restricted field of origins in a sensory surface, such as the retina or the skin." p38

"In contrast to the highly specific regional organization in the gross anatomy of the sensory projection areas of the cortex, the detailed microstructure of the network appears to be essentially random, governed only by directional gradients and preferences, and statistical distributions of fiber lengths for variou types of cells (see Sholl, Ref.93). In the human nervous system, it appears that the most specific and constrained topological organizations are to be found in the sensory and motor systems, while the intervening association network of the CNS is less tightly controlled in its organization, presumably depending more on learning and adaptive modification to establish the required pathways and linkages." p39

"One feature which is of particular importance for brain models is the apparent plasticity of localization in the 'association areas' (or 'intrinsic systems', to use the terminology advocated by Primbram) in contrast to the relatively fixed and irreplaceable character of the sensory and motor tracts." p41

"There is no doubt that mechanisms of considerable complexity, sufficient for perceptual tasks and the control of organized behaviour, can be created by genetic control of growth and maturation. This is most dramatically evident in the instinctual patterns of insects... The stimulus analyzing mechanisms discovered by Lettvin and associates for frog vision have already been mentioned. In these studies, it is found that certain ganglion cells in the frog retina respond only to contours or strong contrast gradients within their sensory field; others respond only to convex images; others to moving boundaries; and still others to a general dimming of illumination over the entire field. Each of these four cell types transmits it's information to a distinct layer of the frog's tectum, where it's position is mapped topographically. Thus, one layer represents a contour map, or outline drawing of the stimulus field, another represents a location map for small convex objects or corners, a third represents movement vectors, and a fourth indicates regions of dimming illumination." p44

"It is worth noting that most of the specific computing mechanisms used in muscular control appear to be of an analog variety, rather than digital; they make use of instensities and frequencies of activity for the direct control of servo-systems, rather than computing a control formula from encoded data and then generating the control signal required. The stimulus analyzing mechanism found by Lettvin, however, constitute a sort of digital code, in with stimuls properties are represented by presence or absence of signals from particular neurons. It seems likely, as von Neumann has observed (Ref. 105) that the brain makes extensive use of both digital and analog principles in its operation, and it appears that both types of devices may be genetically determined." p44

"It is generally agreed, simply on the basis of definition, that whatever we call 'memory' involves a modification of neural activity in the central nervous system or its output signals, as a function of exposure to previous events or 'experience'." p53

"While it is not implausible to assume that the surrounding medium participates in the memory trace structure, it seems likely that such interaction between medium and neurons would be highly localized, probably influencing only a single neuron or synaptic junction, rather than forming a widespread organized structure independent of the neurons themselves." p55

"Systems which represent information internally, in such a way that it can be utilized for the control of certain kinds of responses (such as running, thinking, or talking) will be called <i>cognitive</i> with respect to the realm of information which is represented and the class of responses which this information controls... Thus the representation of information in the form of an image on the retina is not sufficient to permit us to say whether or not the organism is cognitive with respect to its visual environment; we must also demonstrate that this information is accessible to the organism for the control of some specified set of responses." p66

"From an operational point of view, the fact of 'consciousness' is closely connected with the accessibility of information and its ability to influence overt behavior; it is, in fact, meaningless to say that an individual is 'conscious' unless there is something that he is conscious <i>of</i>... All we can say, in the last analysis, is that the system acts <i>as if</i> it were conscious, leaving the question of the actual <i>existence</i> of consciousness in the system for metaphysicists to consider." p66

"As indicated above, a pure generalization experiment is one in which the brain model, or perceptron, is required to transfer a selective response from one stimulus (say, a square on the left side of the retina) to a 'similar' stimulus which activates none of the same sensory points (a square on the right side of the retina)... As in the case of discrimination experiments, it is possible to study either <i>spontaneous generalization</i>, in which the criteria for similarity are not suppliee by an outside agency or experimenter, or <i>forced generalization</i>, in which the experimenter's concept of similarity is 'taught' by means of a suitable training procedure. Some of the most significant problems in brain mechanisms concern generalization phenomena, and particularly the meaning of 'similarity' for a particular kind of system." p69

"The ultimate test for a brain model, from the standpoint of pyschological validity, is an experiment of this type, in which the model correctly predicts phenomena which have yet to be discovered in biological systems." p78

"The problem of motivation for perceptrons, considered as models for biological nervous systems, has hardly been treated adequately up to this time. The reinforcement control system, which forms part of the experimental system, plays the role of a sort of <i>deus ex machina</i>, which not only has knowledge of right and wrong responses, but can control the distribution of reinforcement to individual R-units in the perceptron, as required. A more 'natural' system with only a slight reduction of efficiency does seem to be possible, however, although at present the model proposed is a heuristic one, on which no quantitative analysis has been completed. The proposed model for biological reinforcement mechanisms is illustrated in Figure 72. In this system, the r.c.s. is no longer external to the system, but is essentially part of the perceptron. It is assumed that the perceptron system includes a sensing device for a physiological condition which has been arbitrarily called the 'discomfort level', measured by the variable D. This might be compared to Ashby's concept of 'essential variables'. In addition to continuously measuring the variable D, which is assumed for simplicity to be some function of the current stimulus pattern, a second mechanism (readily represented by a neuron with inhibitory input connections with a short time delay and excitatory connections with a longer time delay, both originating from the 'D-detector') responds to a negative dD/dt. The corrections to this system are random perturbations applied either to active connections, or to all connections of the perceptrong; the increments, however, take the form of 'elastic perturbations', so that the connections tend to decay back to their previous values unless a 'positive reinforcement' occurs to 'fix' the new values. Thus negative reinforcement applies a slight random perturbation, which tends to disappear unless it actually proves helpful, in which case it is stabilized by a positive reinforcement." p571

"[Diagram] For this system to function efficiently, it is again necessary to assume some degree of temporal continuity in the environment, so that the change in D indicates a true improvement in the response of the system, rather than an irrelevant change due to a sudden alternation of the environment. Preliminary simulation experiments to evaluate this scheme are now in progress, employing the Burroughs 220 computer, and indicate that the system shoudl work with a reasonable degree of efficiency, as compared to a system employing a more deterministic error correction procedure. The results of these experiments will be reported as soon as the data are complete." p572

"Stated in simplest terms, our objective has been to discover a physical system, or abstract model, which will be capable of "perceiving" its environment, and learning to recognize those objects or events which i shas perceived in the past. However, since it is our purpose to understand the actual mechanisms employed by the brain, rather than simply to construct a new type of comptuing device, the perceptron models are constrained in their organization and dynamic properties by what is known of the biological nervous system. Rather than attempting to 'invent' or 'construct' a machine which will calculate such things as similarities or geometrical properties of stimuli, the approach has been to begin with a hypothetical network of idealized neurons, or nerve cells, resembling the brain in its general organization, and then analyze the system mathematically to determine whether or not it possesses 'psychological' properties of interest. Where the model is found to deviate markedly from the behaviour of biological systems, modifications are suggested, and the new model that results is subjected to the same sort of analysis. In this fashion, it is hoped that the necessary conditions for a system to 'perceive' in the same manner as the brain can be abstracted." p574

"It has been shown that as the topological organization of the perceptron increases in complexity, new psychological properties emerge. The principle results can be summarized as follows: (1) A network consisting of less than three layers of signal transmission units, or a network consisting exclusively of linear elements connected in series, is incapable of learning to discriminate classes of patterns in an isotropic environment (where any pattern can occur in all possible retinal locations, without boundary effects). (2) A three-layer series-coupled perceptron is a minimal system capable of learning to discriminate arbitrary classes of stimulus patterns or stimulus sequences. Any discrimination problem can, in principle, be solved by such a system, and any arbitrary response function can be assigned to the stimuli of a given universe. (3) ..." p575

"The most important technological development which may be inherent in the future development of brain models, would be the provision of 'eyes and ears' for conventional computers and automata, giving them a common universe of discourse with their operators. Currecnt attempts at heuristic problem-solving programs (such as Newell and Simon's programs) and at automatic language translation, are hampered by a lack of common referents for symbols, which can be no more than code-numbers for the computer, but which have a wealth of associated meanings for the operator. The development of a system which, by virtue of a shared sensory experience, can 'comprehend' the nature of the physical referents in a descriptive statement, is probably a necessary first step to the creation of a truly useful problem-solving computer. Linguistic capability, related to perceptual experience, is of the essence for an "intelligent" system, artificial or otherwise." p583-584

"The theoretical approach presented in this volume is clearly a long way from an adequate "explanation" of the foundations of human experience. The work will have fulfilled an important purpose, however, if it has succeeded in conveying a recognition of the potential power of mathematical study of neurodynamic systems, not only for understanding the physical mechanisms of the brain itself, but for comprehending the relationship of the cognitive process in man to the nature of the environment in which it occurs." p583

Notice: Undefined index: alphapagefrom in /home/hri/Website/criticaldh/admin/PublicCredify.php on line 233 Notice: Undefined index: alphapagefrom in /home/hri/Website/criticaldh/admin/PublicCredify.php on line 233

"For this writer, the perceptron program is <i>not</i> primarily concerned with the invention of devices for 'artificial intelligence', but rather with investigating the physical structures and neurodynamic principles which underlie 'natural intelligence'. A perceptron is first and foremost a brain model, not an invention for pattern recognition. As a brain model, its utility is in enabling us to determine the physical conditions for the emergence of various psychological properties."

Next 1000 Records