Iconic gestures the grammatical categories of lexical…

Iconic gestures the grammatical categories of lexical… ICONIC GESTURES: THE GRAMMATICAL CATEGORIES OF LEXICAL AFFILIATES Uri Hadar Department of Psychology Tel Aviv University Ramat Aviv Israel and Robert K Krauss Department of Psychology Columbia University New York USA Short title: Lexical affiliates Published as: Hadar, U. & Krauss, R. (1999) Iconic gestures: the grammatical categories of lexical affiliates. Journal of Neurolinguistics, 12, 1-12. Abstract Iconic gestures occur during continuous speech and show in their form a meaning related to the meaning articulated in speech. In most cases the related speech unit is a word, called the 'lexical affiliate' of the gesture. Grammatical analysis of lexical affiliates may enhance the understanding of the processes that mediate between speech and gesture production. The present investigation offers a grammatical analysis of 408 lexical affiliates produced during continuous speech by healthy and brain damaged subjects. Lexical affiliates divided into seven categories: concrete nouns, manual verbs, other verbs, prepositions, adjectives, abstract nouns and quantifiers. The first four categories were the largest, accounting for 87% of all affiliates. This implies that diverse systems are involved in gesture production. The lexical composition of lexical affiliates was then compared to the lexical composition of the speech sample generally. Results showed that manual verbs tended to affiliate to gesture beyond their share in continuous speech. This suggests that motor schemata are more strongly linked to processes of gesture production. Analysis of the brain damaged corpus suggests that gesture shaping is probably processed in systems which are localized to the right hemisphere, since these patients showed the greatest dis-similarity with the healthy subjects. Introduction Several different kinds of hand and arm gestures normally accompany continuous speech, among them batonic, emblematic, deictic and iconic gestures (Ekman & Friesen, 1969). Of these, iconic gestures have attracted particular attention because they most clearly integrate propositional and non-propositional knowledge in the representation of meaning (McNeill, 1992; Hadar & Butterworth, in press). The precise nature of this integration and its implications for semantic processing are controversial issues which the present article purports to address. Iconic gestures indicate in their form and configuration a meaning which also appears in the accompanying speech as a word, phrase or idea. For example, a circular arm movement in the horizontal plane may accompany the phrase "We entered through the revolving door", suggesting the dynamics of the revolving door. Such a gesture may start prior to the beginning of the accompanying phrase, anticipating the idea of a revolving door, or it may start just prior to the word "revolving", exhibiting the meaning of this word (Hadar & Butterworth, in press). In either case, the gesture usually will not terminate before the word "revolving" is articulated (Krauss et al., 1996). The two kinds of onset timing of iconic gestures may have different processing implications, the former originating early in processing the communicative intention, when the related idea is crudely worked out, and the latter originating late in semantic processing, when the related word is retrieved from the mental lexicon (Hadar & Butterworth, in press). When the gesture relates to a specific word in the accompanying speech, this word is called the 'lexical affiliate' of the gesture (Schegloff, 1983). Most iconic gestures have lexical affiliates, even when they do not start immediately prior to the articulation of their lexical affiliate. The semantic interpretation of the iconic gesture depends critically on the identification of a lexical affiliate. Indeed, in the absence of speech, the interpretation of an iconic gesture may span a wide range of possible meanings (Feyereisen et al., 1988; Krauss et al., 1991). It is usually assumed that the semantic features which shape the iconic gesture are the same as those that serve the retrieval of the lexical affiliate (McNeill, 1992). By inference, the same set -Lexical Affiliates- of semantic features drives both lexical retrieval and iconic gesture. Just as understanding how such a set drives lexical selection is essential for the understanding of speech production (e.g., Levelt, 1989), understanding how it drives an iconic gesture is essential for the understanding of the gestural formulation of meaning. Consider the concept of the revolving door. It comprises of the features of the circular motion, the plane of this motion (horizontal), the orientation of this motion (clockwise or anti-clockwise), the function of this motion (entering and exiting a building), etc. These features can be represented in different formats which, for present purposes, we divide into propositional and non-propositional. A propositional representation carries with it a truth value (correct or incorrect); it is relatively explicit and accessible to consciousness but is uneconomical in processing resources. Cognitive theories usually call the propositional system 'conceptual' (McNeill, 1992; Kintsch, 1994; Hadar & Butterworth, in press; Krauss et al., 1996). Because of the large processing demands of the propositional system, it is assumed that most spatio-temporal information is represented in non-propositional formats (Kosslyn, 1990), which do not require verification procedures and do not presume consciousness. It is also assumed that speech production requires that the intended message be first represented in a propositional format (Levelt, 1989). The preparation of the cognitive material in a propositional form is usually referred to as 'preverbal message construction' (Garrett, 1984). Now, because not all spatio-temporal information is readily available in a propositional format, message construction requires the interaction between propositional and non-propositional representations. This interaction involves activating the non-propositional representation which, in turn, involves gesture production. The question is which non-propositional systems are implicated in gesture production. Rime (1983) and Morrel-Samuels and Krauss (1992) have suggested that iconic gesture production engages abstract motor representations. This hypothetical system mediates between the conceptual system and the action system for all purposes of motor action and involves what has been referred to as 'schemata' in 4 -Lexical Affiliates- motor control literature (Schmidt, 1988). A Schema (or its equivalent) represents a spatio-temporal object (a dynamic form) in terms of the variety of motor configurations by which this object can be attained, but without specifying its metric properties. It abstractly connects dynamic geometrical forms with possible anatomical realizations. The motor program is obtained when the metric properties of the schema are specified. In Rime's (1983) view, abstract motor representations are capable of generating novel schemata by geometric manipulation of existing schemata. This generative capacity amounts to a 'motor cognition', since its products are avaiable to the conceptual system for the purpose of creating propositional representations. McNeill (1992) and Hadar & Butterworth (in press) have suggested visual imagery as the non-propositional system connecting propositional processes with iconic gestures. This system represents spatio-temporal objects irrespective of whether or not they are realizable by the action system. Objects may be represented either with or without information connecting them to action. This allows the representation of objects of greater geometrical complexity than in motor representation, as well as specification of perspective, color and relative size. The system is capable of performing various operations on visual images such as geometrical transformations (topological, projective and affine mappings), union and intersection of representations, etc. It forms the basis of all knowledge of the appearance of objects and actions, and therefore of visual recognition and identification (Kosslyn, 1990). In addition, this system continuously interacts with the conceptual, propositional system in order to facilitate diverse processes such as lexical retrieval and the planning of action (Hadar & Butterworth, in press). Recently, Rauscher et al. (1996) and Krauss et al. (1996) have suggested that the non-propositional system which mediates between conceptual and gestural processes concerns spatial representation. This system is usually considered as concerned with the representation and long-term storage of various spatial relations such as directions (relative to either egocentric or objective coordinates), and the various relations captured 5 -Lexical Affiliates- in prepositions and spatial adjectives (in, out, above, under, near, far, etc.). Indeed, some of Rauscher at al.'s (1996) arguments in favor of spatial mediation concern evidence that phrases which include words of direction, prepositions and spatial adjectives are more likely to be accompanied by ideational gestures than other word categories. Rauscher et al.'s (1996) research makes use of the idea that the non-propositional processes involved in gesture production manifest in the content of the affiliated phrases. If iconic gestures accompany primarily spatial terms (and convey a spatial content), then the mediating system is likely to be spatial representations. This principle may be extended to both of the other systems proposed above: motor schema and visual imagery. Thus, if the non-propositional system involved is primarily that of motor representation, then phrases affiliated with gestures will tend to contain verbs of action, primarily manual action (throw, grasp, etc.). Similarly, if the non-propositional system involved is primarily visual imagery, then the affiliated phrases will tend to contain concrete objects (by virtue of the share of object representation in visual imagery). The problem is that phrases may contain words of more than one of the above categories. However, it is possible to focus upon iconic gestures only and investigate their lexical affiliates. Since iconic gestures are defined by affiliation with one lexical item (the affiliate), it is possible to infer the underlying non-propositional system by analysis of the category of lexical affiliates, in accordance with the principle that spatial mediation will manifest in affiliation with words of direction, prepositions and spatial adjectives; motor mediation will manifest in affiliation with action verbs (especially manual verbs) and imagistic mediation will manifest in affiliation with concrete nouns. This idea is taken up in the present study. In addition to the kinds of words and concepts which act to generate gesture, evidence concerning non-propositional mediation may be gleaned from the study of brain-damaged subjects (Butterworth & Hadar, 1989; Feyereisen & Lannoy, 1991; McNeill, 1992). If visual imagery is involved in gesture generation, then brain damaged subjects who have imagery deficits would also have gestural deficits, and these will affect the 6 -Lexical Affiliates- categorial distribution of their lexical affiliates. Similar considerations hold with regard to motor and spatial deficits. Of course, if the subject has a limb apraxia, that is, a problem in programming motor action generally, then he or she will have a problem producing coverbal gestures as well (Feyereisen & Lannoy, 1991). It is therefore necessary to ensure that the investigation will not include brain damaged subjects who have a limb apraxia. Recent evidence from studies with brain damaged subjects shows that subjects with a left-hemisphere brain damage may not have any problem with gesture production, even if they are seriously aphasic (Feyereisen, 1988; Hadar, 1991; Hadar & Yadlin Gedassy, 1994). Indeed, relative to their verbal production, aphasic subjects who have circumscribed problems in word retrieval have higher rates of iconic gestures than healthy controls (Hadar et al., 1996a; Hadar et al., 1996b). This has been ascribed to their attempts to facilitate word retrieval (Hadar & Butterworth, in press). It is, however, unclear whether some classes of words are more involved here than others, which could offer some idea as to the nature of the non-propositional processes involved in the facilitation of lexical retrieval. In contrast to subjects with a left hemisphere damage, subjects with right hemisphere brain damage have rates of iconic gestures (relative to verbal production) which are markedly lower than those of healthy controls (McNeill & Pedelty, 1995; Hadar et al., in press-a). These subjects have both spatial and imagery deficits, although they do not have any differential disorder in the construction and use of motor schema. Their pattern of gesture production, then, clearly suggests involvement in the shaping of gestures of either imagery or spatial representation (or both), but does not differentiate between the two. The present investigation attempted to look into this further by analyzing the grammatical classification of lexical affiliates for differential effects of brain damage. 7 -Lexical Affiliates- Method Subjects: Subjects were of two main groups, one of brain-damaged patients recruited from the Lowenstein Rehabilitation Hospital (n=17, mean age = 56.3) and one of healthy subjects, matched (except for 2 subjects) with the brain damaged group for age, gender, education and years of Hebrew use (n=15, mean age = 53.8). The former group had 10 males and 7 females and the latter group had 9 males and 6 females. The brain-damaged subjects were CVA patients whose lesion size was limited and whose cognitive deficits were mild to medium. They were subdivided into three groups. The first group ('anomic') consisted of 8 anomic aphasics who had good continuous speech with mild circumlocution, and who performed normally on all language tests except for naming, where they succeeded in only 65.7% of items. The second group ('conceptual') was of 5 patients with left hemisphere brain damage who showed primarily conceptual problems reflected in the processing of language on discourse level, both in production (semantic incoherence) and in comprehension (of both complex sentences and short stories), but relatively preserved naming. These patients were also impaired on non-verbal conceptual tests such as creating a 'story' by ordering a few (2-4) pictures in the correct sequence. The third group ('spatio-temporal') was of 4 patients whose brain damage localized to posterior (parieto-occipito-temporal) regions of the right cerebral hemisphere. Their language performance was unimpaired, but they had problems performing spatio-temporal tasks such as mental rotation and the assessment of relative height of common objects. Their object recognition and their picture arrangement were good. A full description of the neurological and cognitive profiles of the brain damaged subjects appears in Hadar et al. (in press-a) and Hadar et al. (in press-b). Experimental procedure: In an experimental session subjects were video filmed while describing in detail the 8 -Lexical Affiliates- contents of a complex picture. Subjects sat on a straightbacked chair with no arm support, facing the experimenter who sat about 2m away. A video camcorder was placed about 1m behind the experimenter, displaced slightly (less than 30 degrees) to the side, which afforded a frontal view of the subject. At the beginning of the session, a short (10-15 min), friendly conversation was conducted on issues of daily life (work, family, etc.) to enhance habituation to the experimental setup. Subjects were then asked to view a picture placed in front of them until they felt they remembered its contents well. The picture was then taken away, and the subjects were asked to describe it in as much detail as they could to somebody who has not seen the picture. After their spontaneous description, subjects were asked specific questions on the content of the picture. Questions related to whatever details missing from the subject's spontaneous description. Their speech was recorded through the microphone of the camcorder. Four complex pictures were shown to the subjects. Each picture showed a story, either in one or in a few frames. The stories were simple and easy to understand. Data analysis: Gesture was monitored from the right arm and hand only, because the visuo-spatial patients had left hemiparesis. This was not expected to seriously bias results, as previous research showed that right-handed subjects perform most gestures by the right arm and hand, even when suffering a mild right-arm weakness (Hadar, 1991; Hadar & Yadlin-Gedassy, 1994). However, left arm and hand movement was analysed in those cases when it was pertinent for identifying an iconic relation between gesture form and speech content. A gesture was defined as ideational if it comprised of more than 2 independent vectorial components (Hadar, 1989). A vectorial component was defined as a movement between two points of zero velocity of any of 4 segments of the arm and hand: fingers (considered as a single segment for the present purposes), hand, 9 -Lexical Affiliates- lower arm and upper arm. Since every segment is governed by those segments that lie in a more proximal position (fingers by hand and above, upper arm by trunk, etc.), a vectorial component was independent if the direction of its movement was different from the direction of the immediately governing segment. A point of reversal of movement direction was considered of zero velocity, so a to-and-fro movement was considered as consisting of two components. A circular movement has two points of direction reversal relative to any Cartesian coordinate system, so a full circle was considered as consisting of two components. Since ideational gestures are usually wide, and since reversed vectorial components can not be thought of as properly independent (Hadar, 1989), a relatively narrow movement of recurring reversals was not considered ideational even if it consisted of more than two components. Picture descriptions were graded for the number of ideational gestures which they produced. For each subject, the most productive description was selected and 20 successive ideational gestures were recorded from it. If the selected description showed fewer than 20 ideational gestures, the next most productive description was selected and the procedure repeated until 20 successive ideational gestures were recorded. Verbatim transcripts of speech were made for the selected descriptions. The location and relative length of pauses, and the location of the beginnings of ideational gestures, were marked on the transcripts. Ideational gestures were classed by three judges into the following categories. 'Emblematic' gestures were those having a conventional meaning in Israeli culture; 'deictic' gestures were those which pointed to a specific object or spatial location; 'iconic' gestures were those whose form, dynamics or configuration showed a clear semantic or pragmatic relation to a word ('lexical affiliate') or an idea in the accompanying speech; 'indefinite' were gestures which did not fall in any of these categories. Full agreement among judges was achieved in more than 80% of cases. Disagreements were decided by discussion among the judges. Analysis of results focused on iconic gestures only. 10 -Lexical Affiliates- Results Analysis of all lexical affiliates in all groups revealed that they fell into seven different categories: concrete nouns, abstract nouns, manual verbs, other verbs, adjectives and adverbs, quantifiers and prepositions. These categories appeared in all our groups with the exception of abstract nouns, which never affiliated to an iconic gesture in the conceptual group (Table 1). The four groups differed in the mean proportion of iconic gestures (and therefore, number of lexical affiliates) which they produced. The aphasic group showed the highest mean (14.5/20), next highest were healthy subjects ('normals' in Table 1) with a mean of 13.6/20, then the conceptual subjects (mean=10.4/20) and lowest were the visuo-spatial subjects (mean=9.0/20). This finding has been discussed extensively elsewhere (Hadar et al., 1996a; 1996b). Since the four groups differed in the size of their sample of lexical affiliates (N in Table 1), we computed the proportion of the various grammatical categories in each group. In all groups, the category with the highest proportion of lexical affiliates was verbs (mean=48.5%), next highest was the category of concrete nouns (mean=24.0%) and next was the category of prepositions (mean=14.5%). Just under a third of the verbs were manual verbs (13% percent of the total). The other three grammatical categories had a share of 2 test (against the hypothesis of equal affiliates which ranged between means of 3.7% and 5.6%. A X 2distribution) of the differential distribution of affiliates among categories gave a significant value (X(6)=51.2, p<0.001). It is, however, possible that the proportion of lexical affiliates simply reflects the lexical composition of the subjects' picture description. To examine this possibility, we computed the distribution of words among the above categories in the total of the subjects' verbal production (Table 2). 11 -Lexical Affiliates- The greatest proportions of lexical items in the subjects' speech were in the same categories as in lexical affiliates, most words being verbs (33.6%) then concrete nouns (33.2%) and then prepositions (13.5%). Yet, 2 test still gave a significant when the proportions in the speech samples were taken as expected values, a X 2value (X(6)=13.7, p<0.05). The main sources of difference between lexical distribution generally and the distribution of lexical affiliates were the categories of concrete nouns and manual verbs, the former having a lower proportion of lexical affiliates than the respective general proportion, and the latter having a higher proportion of lexical affiliates (Figure 1). The category of 'other verbs' also showed a higher proportion of lexical affiliates. Other categories contributed very little to the observed differences (Figure 1). Generally speaking, the various groups were very similar with regard to lexical composition, both in the affiliates corpus and in the speech corpus. To assess the differences among groups, a difference index was computed in the following way. The compositional difference index between two groups, A and B, was the sum of the proportional differences in each grammatical category. Formally, DI(A,B)=SUM(DCi), where DI is the difference index and DCi is the difference of proportions between A and B in the grammatical category i. First, we wanted to see the differences from the normal group in the speech corpus. In this corpus, DI was greatest for the Conceptual group (32.3), smallest for the Visuo-spatial group (16.5) and intermediate for the Anomic group (18.5). This means that the lexical composition of spontaneous speech in the Conceptual group diverged from that of the normal group more than the lexical composition of the two other groups. Since the composition of lexical affiliates largely reflects the composition of speech, the same picture was seen in the lexical affiliates corpus: difference of composition from the normal group was greatest in the Conceptual group (32.6), smallest in the Visuo-spatial group (25.1) and intermediate in the anomic group (26.0). We also wanted to see whether a cognitive deficit affects lexical affiliation beyond its effects on the lexical composition of speech. To examine this, we computed the difference of differences (from the composition of 12 -Lexical Affiliates- the normal group) between the affiliates corpus and the speech corpus. Now the picture was reversed: the greatest value was seen in the visuo-spatial group (14.0), the smallest in the Conceptual group (0.3) and an intermediate value in the Anomic group (7.5). This indicates that, given a grammatical composition of spontaneous speech, the mechanism which generates iconic gestures in the Conceptual group is most similar to that of the normal group, while the mechanism which generates iconic gestures in the visuo-spatial group is markedly different from that of the normal group. Examination of the differences between the affiliate and the speech corpus in the visuo-spatial group showed that the main source of divergence from the normal group dwelt in an increase in the affiliation rate of manual verbs (relative to spontaneous incidence) and a decrease in the rate of affiliation of other verbs (Tables 1,2). Discussion Looking at a corpus of data, classifying the observed phenomena and identifying regularities has served psycholinguistic research fairly well in the past. Thus, the derivation of some of the most influential models of speech production have been based on this methodology (Garrett, 1984; Butterworth, 1989; Levelt, 1989). This strategy, it was hoped, could help us clarify the issue of the non-propositional mediation between conceptual processes and gesture production. While this would have been particularly instructive if one category proved dominant in the distribution of lexical affiliates, this has not been the case. The three main categories- concrete nouns, verbs and prepositions- all contributed substantial proportions of lexical affiliates. On the present line of reasoning, this implies involvement of all candidate systems, namely, visual imagery, motor schemata and spatial representations. The model which suggests itself is therefore as follows. Communicative intentions are initially processed so as to allow the selection of a set of content words and a syntactic construction (Levelt, 1989). The selection of content words is often enhanced by activating a non- 13 -Lexical Affiliates- propositional representation which helps specify the related concept. This specification involves reductive processes, either in the form of deducing a small set of propositions from larger sets of propositions (Sperber & Wilson, 1986), or in the form of intersecting large sets of semantic features in order to obtain smaller sets of features, largely down to word size (Bierwisch & Schreuder, 1992). When exactly the system will try to boost specification processes by recourse to non-propositional activation is still unclear. On one account, spatial concepts activate non-propositional representations as an inherent part of conceptual processing (Krauss et al., 1996). If a related word is not selected within a critical time lag, a gesture will be produced. Hence the tendency to produce ideational gestures either in association with a hesitation pause, or considerably prior to the articulation of the related word. Alternatively, it is possible that concepts associated with a retrieval difficulty enhance the activation of non-propositional representations (Hadar & Butterworth, in press). Retrieval difficulties may occur early in the processing of conceptual representations without producing a hesitation pause. Gesture onset will be considerably prior, rather than adjacent, to the utterance of the lexical affiliate. Difficulties may also occur relatively late, in the course of lexical retrieval, producing gestures which are associated with hesitation pauses and are adjacent to their lexical affiliates. Both of the above models are currently underconstrained by the available data. While all three systems of non-propositional representation seem to be involved in linking conceptual processes with gesture production, they are not equally engaged. The most productive category (relative to its spontaneous distribution) was that of manual verbs, while the least productive was concrete nouns, with prepositions and other verbs showing intermediate productivity. This may have a number of possible explanations. Firstly, it may be that there are more retrieval difficulties in the production of manual verbs then there are in the production of concrete nouns. There is some circumstantial evidence in this direction. For example, some reading disorders result in over reliance on lexical reading strategies (e.g., deep 14 -Lexical Affiliates- dyslexia). In these, the reading of concrete nouns is better than the reading of verbs (McCarthy & Warrington, 1990). Secondly, in the present model, manual verbs activate motor schemata, while concrete nouns activate visual imagery. The structural similarity between motor schemata and the motor programs of gesture may enhance gesture production, so that equal rates of non-propositional activation may generate different gesture rates. In our samples, the lexico-grammatical composition of the conceptual subjects was most dissimilar from that of healthy controls, while that of visuo-spatial subjects was most similar, with the anomics displaying intermediate similarity. This implies that the conceptual deficits of the patients affect message construction and, through it, the lexical composition of continuous speech, which is consistent with current theories (Garrett, 1984; Levelt, 1989). In the same vein, visuo-spatial deficits do not seem to critically affect the lexical composition of speech but, at best, the efficacy of retrieval processes. Note, however, that the conceptual subjects were most similar to healthy controls in their pattern of gestural affiliation, given the composition of spontaneous speech. This suggests that the mechanisms which generate iconic gestures in the conceptual subjects are similar to those of healthy controls, despite the fact that they produce less iconic gestures (Hadar et al., in press-b). These mechanisms may be affected more by processes which shape gesture and speech after their conceptual processing, as in the other two groups. Indeed, the similarity gap between the spontaneous and the affiliates corpus was greatest in the visuo-spatial group, who were impaired precisely in the related non-propositional representations. What may have happened in these subjects is that their spatial and imagistic systems were more impaired than their motor schema system. They therefore continued to produce iconic gestures which were mediated by motor schemata, but not those mediated by imagistic and spatial representations. As a result, in the visuo-spatial subjects, the relative affiliation of manual verbs was higher than that of concrete nouns or prepositions. 15 -Lexical Affiliates- References Bierwisch, M. & Schreuder, R. (1992) From concepts to lexical items. Cognition, 42, 23-60 Butterworth, B.L. (1989) Lexical access in speech production. In W. Marslen- Wilson (ed.) Lexical Representation and Process, pp 108-135. Cambridge (Mass): MIT Press. Butterworth, B. & Hadar, U. (1989) Gesture, speech and computational stages. Psychological Review, 96, 168-174. Ekman, P. & Friesen, W.V. (1969) Nonverbal leakage and clues to deception. Psychiatry, 3, 88-105. Feyereisen, P. (1988) Nonverbal communication. In F. Clifford Rose, R. Whurr & M. Wyke (eds) Aphasia. London: Whurr Publishers. Feyereisen, P. & de Lannoy, J.-D. (1991) Gestures and Speech: Psychological Investigations. Cambridge: Cambridge University Press. Feyereisen, P., Van de Wiele, M. & Dubois, F. (1988) The meaning of gestures: What can be understood without speech? European Journal of Cognitive Psychology, 8, 3-25. Garrett, M.F. (1984) The organization of processing structure for language production: Applications to aphasic speech. In D. Caplan, A.R. Lecours & A. Smith (eds.), Biological Perspectives on Language, pp 172-193. Cambridge (Mass): MIT Press. 16 -Lexical Affiliates- Hadar, U. (1989) Two types of gesture and their role in speech production. Journal Of Language and Social Psychology, 8, 221-228. Hadar, U. (1991) Speech-related body movement in aphasia: Period analysis of upper arm and head movement. Brain and Language, 41, 339-366. Hadar, U. & Yadlin-Gedassy, S. (1994) Conceptual and lexical aspects of gesture: evidence from aphasia. Journal of Neurolinguistics, 8, 57-65. Hadar, U., Burstein, A., Krauss, R. & Soroker, N. (in press-a) Ideational gesture and speech: A neuropsychological investigation. Language and Cognitive Processes. Hadar, U., Venkert-Olenik, D., Krauss, R. & Soroker, N. (in press-b) Gesture and the processing of speech: Neuropsychological evidence. Brain and Language. Hadar, U. & Butterworth, B. (in press) Iconic gesture, imagery and word retrieval in speech. Semiotica Kintsch, W. (1994) The psychology of discourse processing. In M.A. Gernsbacher (ed) Handbook of Psycholinguistics. New York: Academic Press. Kosslyn, S.M. (1990) Mental imagery. In D. Osherson, S.M. Kosslyn & J.M. Hollerbach (eds.) An Invitation to Cognitive Science, Vol 2: Visual Cognition and Action, pp 73-97. Cambridge (Mass): MIT Press. Krauss, R.M., Chen, Y. & Chawla, P. (1996) Nonverbal behavior and nonverbal communication: What do 17 -Lexical Affiliates- conversational hand gestures tell us? in M. Zanna (ed.) Advances in Experimental Social Psychology, Volume 26. New York: Academic Press. Krauss, R.M., Morrel-Samuels, P. & Colasante, C. (1991) Do conversational hand gestures communicate? Journal of Personality and Social Psychology, 61, 743-754. Levelt, W.J.M. (1989) Speaking: From Intention to Articulation. Cambridge (Mass): MIT Press. McCarthy, R. & Warrington, E. (1990) Cognitive Neuropsychology: A Clinical Introduction. New York: Academic Press. McNeill, D. (1992) Hand and Mind: What Gestures Reveal about Thoughts. Chicago: University of Chicago Press. McNeill, D. & Pedelty, L. (1995) Right brain and gesture. In K. Emmorey & J. Reilly (eds) Language, Gesture and Space, pp 63-86. Hillsdale (NJ): Lawrence Erlbaum. Morrel-Samuels, P. & Krauss, R.M. (1992) Word familiarity predicts the temporal asynchrony of hand gestures and speech. Journal of Experimental Psychology: Learning Memory and Cognition, 18, 615-622 Rauscher, F.H., Krauss, R.M. & Chen, Y. (1996) Gesture, speech and lexical access: The role of lexical movements in speech production. Psychological Science, 7, 226-231. Rime, B. (1983) Nonverbal communication nonverbal behavior? Towards a cognitive-motor theory of nonverbal behavior. In W. Doise & S. Moscovici (eds.), Current Issues in European Social Psychology, pp 18 -Lexical Affiliates- 85-141. Cambridge: Cambridge University Press. Schegloff, E.A. (1984) On some gestures' relation to talk. In J.M. Atkinson & J. Heritage (eds.), Structures of Sound Action: Studies in Conversational Analysis, pp 266-296. Cambridge: Cambridge University Press. Schmidt, R.A. (1988) Motor Control and Learning: A Behavioral Emphasis. Champaign (Il): Human Kinetics. Sperber, D. & Wilson, D. (1986) Relevance: Communication and Cognition. Cambridge: Cambridge University Press. 19 -Lexical Affiliates- Table 1: The grammatical distribution of lexical affiliates. Visuo-spatial Anomics Conceptual Normals Mean Concrete nouns 25.0 20.7 23.1 25.0 24.0 Abstract nouns 8.3 6.1 0.0 6.4 5.6 Manual verbs 16.7 20.7 11.5 8.3 13.0 Other verbs 25.0 35.3 44.2 35.3 35.5 Adjectives 2.8 2.6 5.8 4.0 3.7 Quantifiers 2.8 4.3 5.8 3.0 3.7 Prepositions 19.5 10.3 9.6 17.2 14.5 N 36 116 52 204 The figures shown for each group are proportions (%) of each grammatical category in the total of lexical affiliates. N represents the actual total of lexical affiliates in each group of subjects. 20 -Lexical Affiliates- Table 2: The grammatical distribution of words in the total production of subjects' picture descriptions. Visuo-spatial Anomics Conceptual Normals Mean Concrete nouns 35.4 27.7 27.9 35.8 33.2 Abstract nouns 10.5 10.0 7.6 7.7 8.4 Manual verbs 5.3 6.4 6.4 3.5 4.7 Other verbs 26.5 27.2 35.5 27.4 28.9 Adjectives 5.8 6.4 2.8 4.6 4.6 Quantifiers 3.7 7.2 11.0 5.8 6.7 Prepositions 12.8 15.1 8.8 15.2 13.5 N 430 390 635 1617 The figures shown for each group are proportions (%) of each grammatical category in the total of lexical production in the speech sample in which the iconic gestures occurred. N represents for each group of subjects the actual total of words (belonging to the selected categories). 21 -Lexical Affiliates- Legend to Figure 1 The difference in the proportion of incidence of each grammatical category between the lexical affiliates and the spontaneous corpus. Acknowledgement We are grateful to the US-Israel Binational Science Fund for supporting the research reported here. 22