Drawing multimodality’s bigger picture: Metalanguages and corpora for multimodal analyses

Drawing multimodality’s bigger picture: Metalanguages and corpora for multimodal analyses
Author: Janina Wildfeuer
Publisher: Frontiers Media SA
Total Pages: 203
Release: 2024-07-30
Genre: Language Arts & Disciplines
ISBN: 2832551963

Download Drawing multimodality’s bigger picture: Metalanguages and corpora for multimodal analyses Book in PDF, Epub and Kindle

Multimodality has most recently been described no longer as a research field or discipline on its own, but rather as a “stage of development within a field” (Bateman 2022a, 49). The realization that (1) many different fields and disciplines now enter their own multimodal phase with new interest in multimodal phenomena and that (2) these disciplines all commit to the development of multimodality research with their own theoretical principles and methodological tools, brings with it not only an immense breadth of potential analytical objects, but also many new meta-methodological issues. “We need to find ways of ‘combining’ insights from the variously imported theoretical and methodological backgrounds brought along by previous non-multimodal stages of any contributing disciplines” (Bateman 2022a, 49). At the same time, the search for a meta-methodology for multimodal analyses is pushed further by the recent trend towards more empirical approaches to multimodal phenomena and the development and use of larger multimodal corpora that just as well require theoretical and methodological refinements. “We need to develop ways of strengthening claims with robustly applicable methods which nevertheless remain firmly anchored theoretically” (Bateman 2022b, 64). For a productive handling of these issues, disciplinary triangulation and finding a ‘common language’ or metalanguage (Maton & Chen 2016) for an ‘integrationist interdisciplinarity’ (van Leeuwen 2005) are the greatest challenges in contemporary multimodality research (Bateman 2022a). Also, there is a need for reconceptualizing the practice of analysis by making available large-scale corpora and broader and more complex empirical setups to fully process the ‘move from theory to data,’ and to substantiate long-lasting theoretical and methodological hypotheses (Pflaeging et al. 2021). For this project, we see these challenges productively as “a multimodal task from the ground up,” as John Bateman (2022b, 64) has phrased it in one of his most recent papers. This Research Topic will address this task by convening the most recent theoretical, methodological, practical, and empirical developments within contemporary multimodality research. The aim is to gain new insights in • the metalanguages or external languages that are currently being developed for multimodal analysis in many different research fields and disciplines, e.g., in pedagogy, literary theory, cultural studies, design, argumentation theory, computer science, and (experimental) psychology; • newest results from data collection methods and multimodal corpus analyses that expand the current quantitative work by, e.g., applying existing theories and methods to larger datasets, or exploring the newest communication technologies. We are particularly interested in seeing how works addressing these aspects contribute to finding ways of productive triangulation and integration for and within a meta-methodology for multimodality research. This Research Topic aims to bring together scholars from a variety of disciplines interested in multimodality research to review, explore, and advance the contributions that John Bateman, as one of the key figures in multimodality research, has made to both theory- and method-building as well as to the driving forward of multimodal empirical and corpus analyses. We welcome contributions that, for example, • critically address the theoretical and methodological advancements that John Bateman has made with regard to the notions of semiotic mode, discourse semantics, genre, textuality, etc.; • apply one of the many approaches that John Bateman has developed for the empirical analysis of multimodal artefacts (e.g., the GeM model for page-based documents, his work on multimodal film and audio-visual analysis, and the discourse semantics and/or annotation approach to visual narratives) to larger corpora or currently newly developing communicative situations; • expand on one of the abovementioned aspects with new ideas and insights from disciplines that have not yet been included in multimodality research.

Multimodality and Genre

Multimodality and Genre
Author: J. Bateman
Publisher: Springer
Total Pages: 331
Release: 2008-04-17
Genre: Language Arts & Disciplines
ISBN: 023058232X

Download Multimodality and Genre Book in PDF, Epub and Kindle

The first systematic, corpus-based and theoretically rigorous approach to the description and analysis of multimodal documents. Drawing on academic research and the experience of designers and production teams, Bateman uses linguistically-based analysis to show how different modes of expression together make up a document with a recognisable genre.

The Structure of Multimodal Documents

The Structure of Multimodal Documents
Author: Tuomo Hiippala
Publisher: Routledge
Total Pages: 250
Release: 2015-06-05
Genre: Language Arts & Disciplines
ISBN: 1317580133

Download The Structure of Multimodal Documents Book in PDF, Epub and Kindle

This book develops a new framework for describing the structure of multimodal documents: how language, image, layout and other modes of communication work together to convey meaning. Building on recent research in multimodal analysis, functional linguistics and information design, the book examines the textual, visual, and spatial aspects of page-based multimodal documents and employs an analytical model to describe and interpret their structure using the concepts of semiotic modes, medium and genre. To demonstrate and test this approach, the study performs a systematic, longitudinal analysis of a corpus of multimodal documents within a single genre: an extensively annotated corpus of tourist brochures produced between 1967-2008. The book provides multimodal discourse analysts with methodological tools to draw empirically-based conclusions about multimodal documents, and will be a valuable resource for researchers planning to develop and study multimodal corpora.

Audiovisual Translation

Audiovisual Translation
Author: Luis Perez-Gonzalez
Publisher: Routledge
Total Pages: 377
Release: 2014-08-27
Genre: Language Arts & Disciplines
ISBN: 1317649907

Download Audiovisual Translation Book in PDF, Epub and Kindle

Audiovisual translation is the fastest growing strand within translation studies. This book addresses the need for more robust theoretical frameworks to investigate emerging text- types, address new methodological challenges (including the compilation, analysis and reproduction of audiovisual data), and understand new discourse communities bound together by the production and consumption of audiovisual texts. In this clear, user- friendly book, Luis Pérez-González introduces and explores the field, presenting and critiquing key concepts, research models and methodological approaches. Features include: • introductory overviews at the beginning of each chapter, outlining aims and relevant connections with other chapters • breakout boxes showcasing key concepts, research case studies or other relevant links to the wider field of translation studies • examples of audiovisual texts in a range of languages with back translation support when required • summaries reinforcing key issues dealt with in each chapter • follow- up questions for further study • core references and suggestions for further reading. • additional online resources on an extensive companion website This will be an essential text for all students studying audiovisual or screen translation at postgraduate or advanced undergraduate level and key reading for all researchers working in the area.

Empirical Multimodality Research

Empirical Multimodality Research
Author: Jana Pflaeging
Publisher: Walter de Gruyter GmbH & Co KG
Total Pages: 213
Release: 2021-11-08
Genre: Language Arts & Disciplines
ISBN: 3110725150

Download Empirical Multimodality Research Book in PDF, Epub and Kindle

This volume advances the data-based study of multimodal artefacts and performances by showcasing methods and results from the latest endeavors in empirical multimodal research, representing a vibrant international and interdisciplinary research community. The collated chapters identify and seek to inspire novel, mixed-method approaches to investigate meaning-making mechanisms in current communicative artifacts, designs, and contexts; while attending to their immersive, aesthetic, and ideological dimensions. Each contribution details innovative aspects of empirical multimodality research, offering insights into challenges evolving from quantitative approaches, particular corpus work, results from eye-tracking and psychological experiments, and analyses of dynamic interactive experiences. The approaches and results presented foreground the inherent multidisciplinary nature and implications of multimodality, renegotiating concepts across linguistics, media studies, (social) semiotics, game studies, and design. With this, the volume will inform both current and future developments in theory, methods, and transdisciplinary contexts and become a landmark reference for anyone interested in the empirical study of multimodality.

Multimodality and Aesthetics

Multimodality and Aesthetics
Author: Elise Seip Tønnessen
Publisher: Routledge
Total Pages: 324
Release: 2018-09-21
Genre: Language Arts & Disciplines
ISBN: 1351592750

Download Multimodality and Aesthetics Book in PDF, Epub and Kindle

This volume explores the relationship between aesthetics and traditional multimodal communication to show how all semiotic resources, not just those situated within fine arts, have an aesthetic function. Bringing together contributions from an interdisciplinary group of researchers, the book meditates on the role of aesthetics in a broader range of semiotic resources, including urban spaces, blogs, digital scrapbooks, children’s literature, music, and online learning environments. The result is a comprehensive collection of new perspectives on how communication and aesthetics enrich and complement one another when meaning is made with semiotic resources, making this key reading for students and scholars in multimodality, fine arts, education studies, and visual culture.

Multimodal Studies

Multimodal Studies
Author: Kay O'Halloran
Publisher: Routledge
Total Pages: 291
Release: 2012-05-22
Genre: Foreign Language Study
ISBN: 1136811176

Download Multimodal Studies Book in PDF, Epub and Kindle

The present volume presents a range of works by an impressive international roster of contributors who both explore issues arising from the study of multimodality and explore the scope of this emerging field within specific domains of multimodal phenomena. Contributors show that each individual work and works in general within multimodal studies represent a dialectic or complementarity between the exploration of issues of general significance to multimodal studies and the exploration of specific domains of multimodality.

Film Discourse Interpretation

Film Discourse Interpretation
Author: Janina Wildfeuer
Publisher: Routledge
Total Pages: 284
Release: 2014-06-03
Genre: Language Arts & Disciplines
ISBN: 1135020868

Download Film Discourse Interpretation Book in PDF, Epub and Kindle

This book contributes to the analysis of film from a multimodal and textual perspective by extending formal semantics into the realm of multimodal discourse analysis. It accounts for both the inferential as well as intersemiotic meaning making processes in filmic discourse and therefore addresses one of the main questions that have been asked within film theory and multimodal analysis: How do we understand film and multimodal texts? The book offers an analytical answer to this question by providing a systematic tool for the description of this comprehension process. It aims to advance knowledge of the various resources in filmic texts, the ways the resources work together in constructing meaning and the ways people understand this meaning construction. This new approach to film interpretation is thus able to remodel and improve the classical paradigm of film text analysis.

Multimodality, Learning and Communication

Multimodality, Learning and Communication
Author: Jeff Bezemer
Publisher: Routledge
Total Pages: 170
Release: 2015-11-19
Genre: Language Arts & Disciplines
ISBN: 1317418433

Download Multimodality, Learning and Communication Book in PDF, Epub and Kindle

This state-of-the-art account of research and theorizing brings together multimodality, learning and communication through detailed analyses of signmakers and their meaning-making in museums, hospitals, schools and the home environment. By analyzing video recordings, photographs, screenshots and print materials, Jeff Bezemer and Gunther Kress go well beyond the comfortable domains of traditional sites of (social) semiotic and multimodal research. They steer away from spurious invention and naming of ever more new and exciting domains, focusing instead on fundamentals in assembling a set of tools for current tasks: namely, describing and analyzing learning and communication in the contemporary world as one integrated field. The theory outlined in the book is grounded in the findings of the authors’ wide-ranging empirical investigations. Each chapter evaluates the work that is being done and has been done, challenging accepted wisdom and standing much of it on its head. With extensive illustrations and many examples presented to show the reach and applicability of the theory, this book is essential reading for all those working in multimodality, semiotics, applied linguistics and related areas. Images from the book are also available to view online at www.routledge.com/9780415709620/

Eyetracking and Applied Linguistics

Eyetracking and Applied Linguistics
Author: Silvia Hansen-Schirra
Publisher: Language Science Press
Total Pages: 202
Release: 2016-11-25
Genre: Applied linguistics
ISBN: 3944675983

Download Eyetracking and Applied Linguistics Book in PDF, Epub and Kindle

Eyetracking has become a powerful tool in scientific research and has finally found its way into disciplines such as applied linguistics and translation studies, paving the way for new insights and challenges in these fields. The aim of the first International Conference on Eyetracking and Applied Linguistics (ICEAL) was to bring together researchers who use eyetracking to empirically answer their research questions. It was intended to bridge the gaps between applied linguistics, translation studies, cognitive science and computational linguistics on the one hand and to further encourage innovative research methodologies and data triangulation on the other hand. These challenges are also addressed in this proceedings volume: While the studies described in the volume deal with a wide range of topics, they all agree on eyetracking as an appropriate methodology in empirical research.