Multimodality in Socio-cultural Contexts: To Talk with Pictures, to Understand Pictures
Valeria Chernyavskaya, Ludmila KulikovaThe present paper is inspired by the challenges posed by multimodal/multi-semiotic resources in the investigation of communication. We focus on the visual resource as a socially and culturally shaped semiotic resource of making meaning and discuss how images work in different socio-cultural contexts. The interpretative complexity of the notions of ‘culture’ and ‘socio-cultural context’ is explained by looking at images as they represent information from a socio-cultural environment. The theoretical framework of the paper was rooted in Halliday, Kress and Leeuwen’s theory of social semiotics, in the concepts of multimodal communication and text linguistic view of the language-image-link according to Stöckl, Schmiz, Sandig, and in Kuße’s discourse sensitive cultural linguistics. The questions under discussion are how we communicate through visual images, and how communicative actors produce and interpret meanings through pictures. We use the multimodal texts from modern German and Russian discourse and discuss the role of the visual image in shaping our socio-cultural experience and knowledge. The paper argues for more sensitivity in respect of the expansion of visual images with regard to multimodal competence. The latter plays a crucial role in exploring the ideological and culturally specific implicatures behind the visual images.
Multimodalität im sozio-kulturellen Kontext: mit Bildern reden, Bilder verstehen
Valeria Chernyavskaya, Ludmila KulikovaDer vorliegende Beitrag will zur Diskussion über die Multimodalität beitragen mit dem Ziel, auf die visuelle Ressource als notwendige Voraussetzung für das Verstehen hinzuweisen. Er fokussiert die Fragen, wie ein Bild im Diskurs wirksam wird, wie Kommunikationsteilnehmer in einem bestimmten Kulturraum Bedeutungen mithilfe einer visuellen Ressource ausdrücken und interpretieren, welche gesellschaftlich sinnvollen Bedeutungen mit dem im Text enthaltenen Bild verbunden sind. Die Auseinandersetzung mit dem Visuellen bietet einen ‚Access Point‘ zur Kontextualisierung und zugleich eine Evidenzbasis in der theoretischen Begründung dessen, wie das gemeinsame, in der kulturellen Praxis geteilte Wissen entsteht und weiter funktioniert. Die methodologischen Grundlagen werden durch die soziale Semiotik von M. A. K. Halliday, G. Kress, van Leeuwen festgelegt, beruhen auf den theoretischen Vorstellungen von Multimodalität von H. Stöckl, H. Schmiz, B. Sandig, folgen der diskurssensitiven kulturwissenschaftlichen Linguistik nach H. Kuße. Fallbeispiele aus der deutschen und russischen kommunikativen Praxis werden analysiert. Die Expansion des Bildes in sozialen Praktiken erfordert von den Teilnehmern der Kommunikation reflektierte visuelle Erfahrung und eine multimodale Kompetenz. Eine solche Kompetenz beinhaltet die Fähigkeit, ideologische, politische, kulturspezifische Implikationen von Bildern abzuleiten und zu interpretieren, die Werteinstellungen zu verstehen, die hinter der visuellen Gestaltung der Bedeutungen stehen.
In the present paper, we will engage with the important challenges posed by multimodal/multi-semiotic resources in communication. Multimodality is considered as a cultural technique and a communicative competence that guarantees mutual transparence and explicitness for interactors. We focus on the visual resource as a socially and culturally shaped semiotic resource of making meaning. In this framework the paper addresses the way knowledge and values are constructed through images as well as language usages. The paper attempts to point out a socio-cultural perspective on the understanding of multimodal texts. The paper discusses how images, as part of language-image-links, work in different socio-cultural contexts and shape our sociocultural experience and knowledge. Thus, the goal is to look at images as they represent information from a sociocultural environment. The interpretative complexity of the notions of ‘culture’ and ‘sociocultural context’ is explained. The analysis follows the position of discourse oriented cultural linguistics that presumes the situational and socio-cultural framework both for the production and the reception of utterances/texts. Methodology. Several approaches to multimodal communication have been developed. The theoretical framework of this paper was found in the theory of social semiotics according to Halliday, Kress, van Leeuwen, and in the concepts of multimodal communication and text linguistic view of the language-image-links according to Stöckl, Schmiz, Sandig and others. The present analysis is also in line with Kuße’s discourse sensitive cultural linguistics (diskurssensitive kulturwissenschaftliche Linguistik). From this theoretical perspective we show that there is a shift from the “linguist’s pride in language” (in Stöckl’s words) towards multimodality and visual methodology in meaning construction. We follow social semiotics in looking at how verbal and nonverbal signs (semiotics resources) are used in certain social practices. It has been emphasized in social semiotics how semiotic practices are driven by socio-cultural environments and how they are embedded in them and emerge from them. The paper looks at multimodal texts and images as they are context sensitive. Findings and Results. The theoretical reflections and text analytical observations show the explanatory charge of multimodal studies. In the provided analysis we tackle the multimodal texts and cases in point from modern German and Russian socio-cultural practice. Different types of image-language-links are considered, following the basic observations in text linguistics, namely complement and discrepancy or contrast. Multimodal metaphors and metonymy are also discussed. Visual images are iconic and also indexical (i.e. metonymic) in terms of their representation of abstract concepts and objects/events in their socio-cultural embeddedness. The authors presume that pictures should be understood in context and in combination with language. Linguistic and visual resources cooperate as they interrelate concepts. From this point of view, images are seen as an additional access point to explain meaning making process within its cognitive dimension. Next, multimodal texts, and images as a part of these texts, allow us to observe how socio-cultural work is performed. This, in turn, leads to the conclusion, that images are context embedded. In other words, they present information from the social environment. A visual image is not just an information vehicle. It is a constituent element of dynamic sense-making process. The cases under discussion show that multimodal texts could act as wrong indexicals and may cause wrong contextualization, if they are misplaced in contexts, i.e. if the context of text reception does not fit the context of text production. We can make sense of imagelanguage-texts if they fit the context. The present paper argues for more sensitivity in respect of the expansion of visual images and with regard to multimodal competence. The multimodal competence plays a crucial role in exploring the ideological and culturally specific implicatures behind the visual images. A potentially wide perspective integrating verbal text and picture within different social practices should also be considered as helpful in this framework.