Dynamic symbolism, chaos, and perception

Dynamic Symbolism, Chaos, and Perception

David DeMaris
University of Texas at Austin
demaris@ece.utexas.edu
demaris@well.com

An earlier version of this paper is in the proceedings of  Einstein Meets Magritte:   An interdiciplinary  reflection on science, nature, human action, and society,  1995 Brussels Belgium.  The Invisible Fountain #1  (1994) work received its premier public performance at the conference.

The early 20th century symbolist poet and essayist Paul Valery, in his meditation on natural and created form "Man and the Seashell", expresses the desire to experience a poetry of mind: to view the workings of a mind as it comprehends a beautiful natural form. The recent developments in understanding of biological perception and the availablilty of powerful visualization and computer music environments have made Valery's dream close to reality. Work is presented on the perception of form, ambiguous percepts, and a theory of involuntary attention and eye movements. While engaged in this research, I have drawn inspiration both from recent findings in perception and nonlinear dynamics and insights from the art and theory of such introspective and verbal artists as Paul Klee, Valery, and from philosophers concerned with perception and art. A virtual fountain with an embedded chaotic neural network is presented. The Invisible Fountain is my first work conforming to a conception I term dynamic symbolism, in which the transformational forms of perceptual processes are articulated as dynamic musical or visual forms. Situated on the border of scientific visualization / sonification and art, the work attempts to realize Valery's dream, and to approach the deep and multiple meanings of symbols in mind and culture.

Introduction

In his essay "Man and the Seashell", Paul Valery invokes "a poetry of the marvels and emotions of the intellect".  He imagines that the perfect subject of such a poetry would be the portrayal of a mind responding to a remarkable natural form of mysterious mathematical integrity and patterned beauty. Valery certainly deserves to be read seriously for his hard won intuition and reasoning on the nature of mind and of the creative process. A contemporary (and reader) of dynamicist Poincare, he was working toward "an algebra of consciousness" based on group theory, topology, and thermodynamic phases; he is said to have conceived of perception as a kind of diffraction process [1]. His choice of the shell as the symbol for this meditation may turn out to be surprisingly prescient; the mathematics and conceptual ground of reaction-diffusion[2], pioneered by Turing to explain biological pattern formation, has been combined with the theory of nonlinear oscillations in contemporary accounts of biological neural networks engaged in perceptual processes. Today, the growing sophistication of biologically based models of perceptual and cognitive processes and the availability of powerful visualization and computing technologies have enabled us to bring Valery's vision closer to fruition.

Launched from the image of the shell, this essay will treat several related themes. First, the definition of a symbol is treated from the perspectives of contemporary cognitive science and from the philosophy of aesthetic perception and artistic communication. The writings of Valery and Klee on the creation and communication of expression through form, and the expressive natural symbolism of Cassirer and Langer are related to a constructive biophysical approach to perceptual modeling. In this approach, the form mediated perceptual process leads to dynamical reorganizations and bifurcations in oscillating neural networks. These reorganizations correspond to the subjective phenomena of emotional states during symbol formation. The author's research program in perception is outlined along with the theories of nonlinear oscillating neural networks on which the work builds. Finally, a computer music and virtual sculpture project on the twin themes of shell and mirror, a self-observing spatially extended dynamical system, is described. The Invisible Fountain is a setting of Valery's poem Narcissus Parle (Narcissus Speaks), and is offered as a prototypical artwork based on this idea of dynamic symbolism. Dynamic symbolism envisions the creation of visual and audio enabled neural network simulations which allow us to develop insight into expression and perceptual processes, while having poetic value in their own right. Throughout, it is hoped that the reader gains a sense of how the statements of artists on their own experience of art is suggestive to students of the mind struggling to transcend the inherited information processing and computational metaphors.

Symbols: grounding, expression, and dynamics

Reason, sometimes, seems to me to be the faculty our soul possesses of understanding nothing about our body. Valery, Dance and The Soul

The definition of symbol and the binding of meaning to symbols are central to the study of mind. In the recent era of cognitive psychology, the prevailing computational metaphor is often simply described as symbolic artificial intelligence. The arbitrary relation between the physical token in a symbol system and real world referents has been critiqued by Searle and refined as the symbol grounding problem by Harnad [3]. Harnad suggests that connectionist artificial neural networks, coupled to sensory transducers and feature detectors, are able to produce symbols whose "shape" and relations to other symbols are nonarbitrary, having been formed through a bottom up process of iconic and categorical relations. The supervised learning connectionist model has in turn been criticized as a model of perception, given that organisms must survive without a teacher [4], and for not conforming to the complexities of perceptual phenomena as known behaviorally since the Gestalt psychology era and now subject to detailed measurements at the neural signal level [5,6].

The symbolic cognitive model has also been criticized for its remoteness from social embedding [7 ] and from the lack of any connection to motor behavior and embodiment [8]. Both critiques are aligned with the philosophical school of phenomenology, and begin to address the problem of the location and origin of meaning. To date however, there have been few attempts to extend these sophisticated treatments of linguistic and perceptual phenomena to the problems of expression and communication through artistic form. The phenomenology of artistic production and perception naturally recognizes the importance of dynamics in symbolism as related to expression. We say that art or music "moves" us; in poetic, visual, or musical production, one gesture seems to naturally call forth another. When viewing a static visual form, our attention as manifested in eye movements and dwelling initiates a dynamic flow of the gaze over a sequence of expressive regions that is largely repeatable by single individuals and shows some evidence of predictability in a population [9]. Investigators of the psychology of art such as Arnheim [10] made a compelling case for the importance of dynamics in the perception of static visual form, but failed to forge the links with biology or pursue cross-cultural historical studies to achieve a complete description of such dynamics.

An important philosophical current, which might be designated the philosophy of expression, is the work of Ernst Cassirer [11,12] and Suzanne Langer [13]. Langer's early work focuses exclusively on aesthetics, while Cassirer examines the perceptual origins of fundamental categories in language (space, time, and number) and the roots of myth and scientific- rational world views. By examining the historic roots of languages, particularly in light of then emerging cross-cultural studies of diverse languages, Cassirer arrived at a conception of symbolism as emergent in language and coevolving with the fundamental subject object distinction in perception. In this view, expression (or expressive perception) is prior to the segmentation or distinction of objects and their attributes. He argues that expression emerges with perception from the organism's living in a world with a basic orientation toward action, anticipating the contemporary enactive cognition approach of Varela and colleagues. Expression enters into the formation (more dynamic than grounding) of symbols when mimesis, in imitative hand or vocal gestures, is replaced by vocal utterances expressing aspects of feeling, and as body-derived spatial designators begin to indicate an awareness of space and objects . Cross-cultural phonetic patterns are identified with certain expressive effects; for example the association of vowels a, o, and u, with more distant objects, e and i with nearer. Eventually, Cassirer says:

Inhibition of the direct outbreak into gestures and inarticulate cries gives rise to an inner measure, a movement within the sphere of sensory appetition and representation. The road leads upwards, more and more clearly, from the mere reflex to the various stages of "reflection". Cassirer, The Philosophy of Symbolic Forms

Such an embodied view of symbol formation allows Cassirer to distinguish between natural (not simply grounded but expressive, imbued with non-discursive feeling in Langer's terms) and the artificial (arbitrary, ungrounded) symbols that arise in later stages of language formation. Similar intuitions concerning the process of expression arise in the introspections of Valery on the affective power of poetic language:

Think of a pendulum oscillating between symmetrical points ... form and content ... at each line, the meaning produced within you, far from destroying the musical form communicated to you, recalls it. Our poetic pendulum travels from sensation toward some ideas or sentiment, and returns towards some memory of the sensation and toward some potential act which could represent the sensation. Valery, Poetry and Abstract Thought

Valery is credited with being the end of the line of the Symbolist poetry movement [14], and in some sense his procedures invert the typical practice of the genre. His descriptions of method often involve beginning with abstract expressions - rhythms, vowel sequences, and sonorities - as pure expression which give rise to the poem's subject, rather than the reverse. With this approach, he recapitulates the archaic formation of language's original symbolic power. This philosophy of symbolic form has been extended to account for more contemporary biology and anthropology by Wallin [15].  His extensive survey and synthesis of biomusicology brings together the concepts of expressive form, animal and human vocal and musical expression, and an understanding of the relationships between various emotional and behavioral states, physiology, nonlinear dynamics, and synchronization tendencies observable in EEG signals. The provocative work of Clynes on the recognition of emotions and their expression in motor gestures [16] is described, and is explictly related to the Langer's theory of symbolic form.

While communication through symbolic form arises and is most effective within a culture, the issue of universal cross-cultural communication is addressed, strongly influenced by research on trance inducing rituals in various cultures. The ethos of a particular culture, consisting of the combination of genetic, ecological, and cultural traditions such as the values placed on expression of particular emotions, constrains the dynamics. In the computational framework described below, an ethos might be translated into ranges of allowable parameters or couplings, or couplings of particular dynamics to value systems which modulate overall thresholds (arousal levels) leading to different bifurcation pathways.

Chaos and Perception of Form

Even if we measure the footsteps of the goddess, note their frequency and average length, we are still far from the secret of her instantaneous grace - Valery, Problems of Poetry

In recent years a movement within neuroscience and physics has emerged, focused on nonlinear oscillations and pattern formation in large scale neural assemblies as the major carriers of perceptual and cognitive information, in contrast to the more computational and signal processing approaches which have predominated in the last fifty years. Typically the older tradition emphasized feature detectors in primary sensation and assumed that modulations of the average spike frequencies of single neurons are the primary sensory processing roles, while learning and higher level formations are handled by symbolic procedures or connectionist networks with fixed point attractor memories.

While the complexity of measuring and interpreting signals in organisms responding and behaving are daunting, another approach to theory building has emerged which has been called constructive [17]. In this approach, the formal characteristics, particularly the larger scale statistics of fluctuations of psychophysical phenomena in space and time are templates to be matched through computational modeling. Modeling is often conducted at a high level of abstraction, employing formulations such as spatially extended coupled map systems, in contrast to models of the low-level electrochemical details of individual cells. Further, the correlations of evolution statistics with changes in the forms presented as input to the dynamical systems can give the constructive modeler confidence in their model. [18,2]. The following description is largely qualitative; a few mathematical details are included in the last section below.

The first task undertaken by the author in the application of locally coupled map lattices (spatially extended, discrete nonlinear dynamical systems) to the computational perception of forms [19] was to show that the evolution of spatial patterns used as initial conditions in the evolution of chaotic reaction diffusion system is useful in characterizing their shape. Initial oscillatory patterns, interacting in weakly chaotic regimes for a few cycles before being quenched to the periodic regime, evolve toward a point in the collapse toward a bipolar distribution to produce a unique intermediate distribution of oscillations which can characterize similarity of patterns via a Euclidean distance metric without explicit feature analysis or representation as is typical in machine vision studies.

As part of the investigations of modulated nonlinear dynamics in perception, some study has also been given to the problem of the formation of attentional foci on visual imagery, particularly as indicated by eye fixation points. It was found that by using a similar procedure as that employed for shape recognition, but changing the time scale at which dynamics change from weakly chaotic to periodic, that at the end of a pattern evolution certain points derived from an original image would be oscillating counterphase to neighboring spatial points. Using one of the test images in the previously cited study of Noton and Stark (based on Klee's  "Old Man" sketch), some correlation was found between the large set of counterphase points produced from the test image and the eye fixation points measured for their subjects.

Conjectured attention foci generated by a chaotic-quenched sequence. The lines indicate points on the image which are indicated as eye fixation points of three different subjects in the referenced study by Noton and Stark.

Considering early psychological theories on the interaction of eye fixations and memory formation [20] and recent mathematical arguments for a mechanism of feature binding via chaotic intermittency [21], I argue that this mechanism or some variation might identify significant spatial regions which containing a rich population of oscillations generated from some feature (an inflection point of a curve, for example). Such a region would be selected for a subsequent dynamical changes in another coupled, retinotopically mapped layer controlling the binding (figural synthesis) of the local regions into a composite intermittent oscillation.

The interior is infinite, all the way to the mystery of the inmost, the charged point, a kind of sum total of the infinite.    Paul Klee, retrospect 1926

There is a resonance inside the particles, immanent within them. Their oscillations range from the very simplest to the composite modes. Inexorable law must express itself throughout. The bow can have no pity. Every expression or function must be cogently justified. Only then will that which is in the beginning, that which mediates, and that which is at the end belong together intimately. And nowhere will the dubious be allowed to obtrude, for each part fits ineluctably into the next.    Paul Klee, Notebooks

Sensitivity to the effects of forms on space and the flow of attention is a hallmark of powerful art. In discriptions of his abstract working method, Klee seems to indicate that an intuitive sense of the perceptual workings of forms serves as a guide in the compositional process; in his forms and statements one sees a kind of reaction in which larger scale forms are then highlighted with such characteristic gestures as the (eye-like) fermata appearing in his work, particularly in the more "hieroglyphic" works produced in the 1930s [22,23].

A third aspect of form perception under investigation is depth gestalt formation, and in particular the way depth-articulating forms sometimes produce reversible or ambiguous percepts. Ambiguous percepts have often been explained by simple satiation or fatiguing of neural circuitry coding the percept. Haken and colleagues have formulated an explanation based on nonlinear dynamics [24], but not addressing all the influences of form on the dynamics. The Necker cube literature indicates that satiation is unlikely in early stages of the visual system, that the formation of depth gestalts is a local process, and that there are correlations between attentional aspects and the parameters of the forms.

These attentional aspects are manifested in eye movement fixation times, switching time distributions, and residency times in the reversible states. It is conjectured that bifurcation parameter gradients in spatially distributed chaotic network interact with motor and somatosensory systems to embody depth gestalts, and to modify the dynamics relating to the formation of attention foci in accordance with the emerging action centered parsing of space. The gradients may be organized by intrinsic spatial flows mediated by forms which imply a perspective. Studies of a coupled map lattice iterating on Necker cube initial conditions with an evolving, anisotropic bifurcation field [25] has shown that the temporal distributions of locally averaged bifurcation field minima around the corners of large and small cubes have a shape similar to human psychophysical data for switching times found by Borsellino et al. [26]. These bifurcation field minima correspond to transient periodic oscillations that I argue are the signature of attention in an emergent saliency map.

Having sketched a research program of "constructivist" computational neuroscience based on intrinsic, carefully timed changes of dynamics, and on local changes in the dynamical flow derived from a form undergoing perception, we return to consider expression - how it functions in symbol formation, and how it may be related to these macro-level dynamics changes. Biologist Freeman argues that meaning is captured in the perceptual encoding based on studies of oscillations in the olfactory bulb of motivated, behaving rabbits[5]. In his view, the particular bifurcation pathway and hence the spatial patterns and complex attractor are determined by the affective state of the organism during the dynamic process, and the affective state is thus part of the memory formed and can be recovered when the original stimulus is encountered again. This demonstration of feeling bound up in the form of a mammal's perceptual encoding of smell must be contrasted with Cassirer's emphasis on meaning arise from language - it appears deeper and more intimately bound into percepts not visibly expressed by the organism.

The invisible fountain

Both Cassirer's philosophy and Valery's poetic subject matter ultimately revolve around the symbolism of myth and its relationship to the mind and body. In the course of the studies outlined here, I realized that particular dynamical architectures might be developed that were particularly appropriate for the encoding or productuion of the form of a particular myth, and that such dynamic architectures might be the original source of the symbolic form of the myth, which visionary or gnostic individuals then translate into the terms of their culture; a similar view is developed by Powell [27]. Contemplating correspondences between the reaction diffusion models of Kaneko [28,29,30], and those used for modeling the pattern formation of the shell, I determined to construct a virtual fountain echoing the shell form and to give it a n ural network interior dynamic fusing aspects of both the reaction -diffusion process and a mirror archetype, a self-observing process to depict the Narcissus myth as outlined in Valery's poem Narcissus Parle«. A coupled map algorithm adapted from Per«z [31], in which a coupled chaotic map layer is joined with a spin observation layer which controls the bifurcations, of each node. The network was implemented in MAX, a visual object oriented language for computer music, and was produced in the facilities of the architecture department Advanced Design Research Group.

The activity in the fountain's neural network was to be realized as drops emanating from the ends of the spiral forms suspended just above the water. A musical rendition of the neural network evolution was produced as a preview of a more elaborate animated virtual world proposal. (Excerpt here).


Fig 1 A rendering of the virtual fountain

The procedure for generating the temporal material, rendered as drops or music,  is rather autonomous, requiring only an initial seed.  This required rethinking the normal process of setting a poetic text.    Two rhyming lines from a stanza of the poem were selected for use as seeds generating aspects of the spatial design and of the accompanying musical process:

Voici me bras d'argent don les gests sont purs!      ( Here are my silvery arms of purest gestures!)        
Et je crie aux echos le noms des dieux obscurs!    ( And I shout the names of unknown gods to the echoes! )
         Valery,  Narcissus Parle'

A series of eight numbers was derived from my reading of the stress patterns on the original French version of the chosen lines. The series determined the control points of spline curves defining the upper and lower surface of the arms of the sculpture.  The resulting curves evoke the arms of the Narcissus flower emerging from a spiral.

The same series was converted to a ratio by dividing the number of each stressed syllable by the mean number of syllables per line; this series was used as the initial state of a ring of eight locally coupled chaotic maps which evolve according to the formulas below.

A coupled map lattice is a dynamical system with discrete time, discrete space, and real valued state. The lattice consists of field variables representing macroscopic (distributed) qualities, such as fluid velocity, local concentration of a chemical substance, or in the present case neuron pulse density in a local assembly with diffusion (entrainment) to neighboring assemblies. The process is decomposed into simple parallel dynamics at each lattice point, with each process carried out successively. In the fountain perceptual simulation , a diffusion step is performed, followed by a reaction step, at each iteration. The entire diffusion step can be expressed as:

Sd(x) =(1-c)St(x)+c/2[St(x+1)+St(x-1)]

where Sd is the intermediate diffusion array, t is the current time step, x the spatial indices of the oscillating ensemble S at the center of a nearest- neighbor diffusion field, S is the state variable at each site of the ring, and c is the coupling constant restricted to the range [0.0 to 1.0]. The second computational unit applied in each time step is the logistic map:

St+1=1-bSd(x)^2

where S, t, x, and y are as above and where b is the bifurcation parameter, restricted to the range (0.0 < b < 2.0). S is restricted to the range (-1.0 < S < 1.0).

Finally, a variable Spin, is maintained at each lattice site which observes the previous and last state and keeps track of the direction of evolution; comparison of spin values in the local neighborhood, which ultimately stabilizes the dynamics:

Spint(x) = 1 if St(x)>St-1(x) , 0 otherwise

SpinHarmonyt(x)= -1 if Spint if Spint(x) + Spint(x+1) + Spint(x-1) = 0 or 3, 1 otherwise

bt+1 = bt + 0.05 * SpinHarmony

Each site is initialized with the bifurcation parameter in the periodic regime and is modulated gradually toward a chaotic but stable configuration in which each senses that it is in harmony with the spin direction of its neighbors.  The network evolves from relatively pure (periodic) gestures to a chaotic regime in which "endless names are shouted to the echoes", as the poem has it. Similar to the development of a shell with its autonomously determined end, the form of the music, governed by its terminal parameters, reaches a stasis though in principle the piece could be performed forever.  When the output of the map is transformed in straightforward fashion to a midi note stream, the resulting music is surprisingly pleasant and tonally coherent while the maps remain in the periodic or intermittent regimes near bifurcation points; one hears the Markov structure underlying modes change dramatically at certain points, a rarity in algorithmic music without explicit directions from the composer.  To map the state values to notes, a lowest bass note was chosen and the real valued output in the range -1.0 to + 1.0 was scaled proportionally to a three octave range.  This value is added to the base after truncating to an integral note value; t he same output stream is also scaled to determine the note duration and onset time, transforming each cycle of the map iteration to a melodic phrase of eight notes,  with harmonies generated from the overlapping of long and short notes.

The use of single nonlinear maps has previously been explored as a generator of musical material by Pressing [33]; while he remarks that the resulting melodies in particular regimes are relatively pleasant and reminiscent of folk melodies, he is quick to discount any psychological relevance. While this seems reasonable for the extremely simple structure of a single map, more complex systems such as those explored in the author's more recent perceptual modeling may begin to approach a psychological realism. The additional complexity of attentional flows evolving along with the forms themselves may be the crucial step in producing more intrinsic formal variation, allowing the grammatical elements commonly used in algorithmic composition in the symbolic artificial intelligence tradition [34] to emerge naturally. In Langer's words:

Non-discursive form in art has a different office, namely to articulate knowledge that cannot be rendered discursively because it concerns experiences that are not formally amenable to the discursive projection. Such experiences are the rhythms of life, organic, emotional and mental (the rhythm of attention is an interesting link among them all), which are not simply periodic, but endlessly complex, and sensitive to every sort of influence. All together they compose the dynamic pattern of feeling. It is this pattern that only non-discursive symbolic forms can present, and that is the point and purpose of artistic construction.

Conclusion

The sources and work in progress of my research program in perception, and an allied artistic program have been outlined. In the domains of form perception and expression, we hope to go further by showing that oscillations in a spatially extended, non-stationary network can correspond to the psychophysical phenomena, and that some of the emerging parameters - the temporal structures of bifurcations, the base values or gradients in a fluctuating bifurcation parameter field, changes in the timing of structured modulations between chaotic and periodic behavior over a network, etc. may be the carriers of expressive form. In organisms, these are embodied as distributions of neuromodulators, or cooperative effects in oscillating subnetworks which condition relations between sense, somatic and motor preparatory behavior. At this stage in the investigation, it seems premature to assign a specific dynamic envelope as the carrier of expression or symbolic form. Cooperative experiments with psychologists to gather data on form perception with modern measurement techniques [35,6] may be required to effectively guide the research.

In the absence of correlating psychophysical data and electrophysiological measurements of oscillations exploring the nature of expressive response, the constructive approach suggests that we might be guided by a kind of "expressive Turing test". In Turing's famous test for a convincing artificial intelligence, a machine passes the test by engaging in conversation over a teletype so effectively that the human cannot distinguish the machine's response from a human. In an expressive Turing test, if a human interpreting a sonification of the response of the "perceiving machine" to art evocative of joy, grief, fear, etc. feels that the machine's responses are appropriate expressive interpretation, the machine passes. Here, then, a situation occurs that calls into question the notion of simulation and the criticisms in the Searlian tradition that are sure to be directed as such systems begin to blur the boundaries of life and artifice. Could we sympathize with the emotional expression of a machine that feels as we do? What would be the philosophical and cultural outcome of living with such machines?

My own hope is that as a mature biophysical understanding of perception diffuses into world cultures, it provokes a transcendental recognition experience, a gaze into the mirror of our being, that could be called religious. The etymology of the word (re-ligere, to bind together again) conveys a pan-theistic sense in which all being in some sense perceives, conspires to break up and bind its constituents, tracing an expressive form as it does so. The great Sufi mystic and pyschologist of the medieval era, Ibn `Arabi, anticipates the philosophy of symbolic form [36]. Corbin summarizes his thought on symbolism:

Because there is Imagination, there is ta'wil (interpretation); because of ta'wil there is symbolism; and because there is symbolism, beings have two dimensions. Hence opposites, and all pairs require a union, unio sympathetica.

Everything we call the universe, is related to the divine being as the shadow (or his reflection in the mirror) is to the person. The world is God's shadow. Ibn 'Arabi

Ibn `Arabi's two dimensions might be mapped to the visible oscillating states of our waking perception and imagination, and the invisible movements in feeling, in bifurcation and coupling. I interpret Arabi's mirror in the universe,  as the collision of perception and imagination:  Valery's pendulum, or the mirrored fountain of Narcissus, transmuting form to feeling through a 'unio sympatica' of order to (coupled) chaos, and of chaos to form.

As Cassirer found aspects of culture (language, art, science) to have unique symbolic forms, Ibn `Arabi recognized that each culture and each individual produces a unique "divine Name", and stated that "each being has as his own God only his particular Lord (Rab) , he cannot possibly have the whole". Perhaps an expressive art, arising not from a single mind or culture as from form, number, and the transformed poetic artifacts of many cultures, will help overcome tendencies toward absolutism, and to honor the diverse and beautiful forms hidden in expression.

References