A multimodal discourse analysis of TED Talks

This study investigates how presenters make use of multimodal communication to disseminate scientific ideas in TED Talks. It focuses on investigating the presenters’ choices in the most frequently used semiotic modes, that is, speech, images, and gestures, in the delivery of the talks, as well as their co-deployment as a multimodal ensemble to achieve the talks’ communicative purposes. To fulfill the research aims, Systemic Functional Theory is adopted as the theoretical framework.

Based on the popularity (number of views), four talks are sampled and analyzed as illustrative cases to examine the presenters’ orchestration of speech, images on the slides, and hand gestures. Speech is analyzed through the lens of Systemic Functional Grammar. Images are examined with Kress and van Leeuwen’s (2021) visual grammar, and the gestures are studied using the frameworks from Martinec (2000, 2001), Lim (2019), and Hood (2011). The transcripts of these talks are annotated clause by clause, images slide by slide, and gestures action by action.

From the analysis of the data, it is observed that the presenters made specific choices in the three modes to achieve the communicative purposes of transmission, persuasion, and promotion. Different Transitivity types are used to facilitate understanding, highlight the speakers’ expertise, and increase the credibility of the ideas. Similarly, images with different Representational Processes are used to provide a visual illustration to clarify ideas and enhance understanding. These choices are also supported by specific selections of Presenting and Representing gestures. Interpersonally, the three modes are used strategically to increase the credibility of the ideas and, at the same time, achieve audience involvement.

Drawing from existing studies and based on the analysis of the meanings and functions of the three modes from the data, the study proposes two frameworks to describe the semantic interrelations, as well as the interplay in the intersemiosis across the three modes. The intersemiosis across the three modes are theorized as Ideational Concurrence and Ideational Complementarity in Meaning Convergence. Interpersonally, they are theorized as Interpersonal Resonance in Meaning Convergence and Interpersonal Dissonance in Meaning Divergence.

This study can shed light on TED Talks as a new genre of science communication. It reveals a deeper understanding of the power of speech, images, and gestures as communicative tools in presentations and how these semiotic modes work in combination to disseminate scientific ideas in a clear, persuasive, and engaging manner. It also offers insights into the hybrid nature of TED Talks and reveals the features of the scientist’s persona in a neoliberal era. The study contributes to the limited literature on multimodal meaning-making in science communication. The findings have the potential to inform pedagogical practices to develop the students’ semiotic awareness.