| dc.description.abstract | - Speaker gestures are semantically co-expressive with speech and serve different pragmatic
functions to accompany oral modality. Therefore, gestures are an inseparable part of the language
system: they may add clarity to discourse, can be employed to facilitate lexical retrieval and retain a
turn in conversations, assist in verbalizing semantic content and facilitate speakers in coming up with
the words they intend to say. This aspect is particularly relevant in political discourse, where speakers
try to apply communication strategies that are both clear and persuasive using verbal and non-verbal
cues. This dissertation aims to analyze the co-speech gestures of several Italian politicians during
face-to-face interviews using a multimodal linguistic approach. The work first introduces the corpus
created: PoliModal corpus (Trotta et al., 2019, 2020), containing the transcripts of 56 TV face-to-face
interviews of 14 hours, taken from the Italian political talk show “In mezz’ora in più” (for a total of
100,870 tokens) that has been manually annotated with information about metadata (i.e. tools used
for the transcription, link to the interview etc.), pauses (used to mark a pause either between or within
utterances), vocal expressions (marking non-lexical expressions such as coughs and semi-lexical
expressions such as primary interjections), deletions (false starts, repetitions and truncated words),
overlaps and facial displays, hand gestures and body posture. Then, the annotation scheme and the
results of a series of statistical analyses aimed at understanding the relationship between annotated
multimodal traits and language complexity are described in detail and testing the validity of existing
studies on political orientation and language use. Finally, after the presentation of an additional
semantic annotation layer related to the function assumed by hand movements, the relationship
between them and other information layers such as a political party or non-lexical and semi-lexical
tags is investigated. Concerning gesture speech relationship, the results obtained suggest that hand
movements are mainly used with integrative and complementary functions. So, the information
provided by such gestures adds precision and emphasis to spoken information. Its, also show that
party affiliation does not significantly influence the gesture-speech relationship. Furthermore - testing
the lexical retrieval hypothesis by calculating the association between the hand movements produced
by each respondent and discourse disfluencies using weighted mutual information - it is shown that
hand movements tend to co-occur with full pauses (i.e., repetition) and empty pauses (i.e., pause) and
more frequently with interjections (semi-lexical tags), suggesting that gesticulation may represent an
attempt at lexical retrieval. [edited by Author] | it_IT |
