In this course you will look at ways in which visual aspects add further layers of meaning to printed literary texts and may be considered to contribute to their poetic function.
As you read, look out for:
how multimodal elements can be seen to contribute to the literariness of texts by foregrounding certain aspects through deviation and defamiliarisation;
the application of the theory of semiotics as conceived by Saussure and developed by Peirce (see Reading A by Bignell), and how the interpretation of signs and codes depends upon the social and cultural knowledge of individual readers;
the use of visual images with verbal texts in the creation of fictional reality in both children’s and adult narratives. (Note that the term ‘high modality’ in this context refers to the feeling of truth and reality conveyed and should not be confused with ‘modes’ of communication or ‘multimodality’);
the way words and images are used in some children’s picturebooks to create ‘hybrid’ texts that challenge and subvert established ideological notions of history. In Reading B, Bradford analyses two children’s texts to show how they ideologically disrupt dominant postcolonial perceptions;
how children’s picturebooks may combine words and images in distinctly postmodern ways to create strange and uncertain meanings. See Lewis’ discussion of the features of postmodernity in Reading C, and the analysis of examples in his reading and in the main chapter;
arguments about how and why we make aesthetic judgements about multimodal texts especially in relation to their material form and context.
The use of different forms of communication in a single text is often known as multimodality. I am primarily concerned here with texts utilising image and words, from the genres of poetry and narrative. Texts from other genres, such as advertisements, and texts using other modes of communication, such as movement, are also included where they seem particularly salient and can be seen as literary, in the Formalist sense. It is important to look both at how individual modes can communicate meaning, and at their interaction in the text – how they can be combined to open up new possibilities for interpretation. Such combinations and juxtapositions of modes can be seen to enhance, reinforce or contradict each other, making meaning unstable and challenging the reader's attempts to make sense of the text.
Multimodality is researched in a range of academic disciplines, among them linguistics, art history, information technology, sociology, cultural studies and media studies. Multimodal texts tell us something about linguistic and artistic creativity, but also about the social and cultural spheres in which we live, and about ourselves as producers and consumers of texts. Close analysis opens up questions about how we make judgements about visual texts, about how we value or downplay their status. Creative multimodality also tells us something about how language works – it may force us to focus on a metaphor or a pun, for example. Or it may reveal something about how relationships between people and institutions are represented, or just cause us to reflect on something we had forgotten to notice for a while.
New technology has brought about a surge in multimodal creativity, as anyone with a reasonably up-to-date computer can now produce texts filled with images, sound and movement. But texts using visuals to make meaning are far from new.
Activity 1: The Mouse's Tale
Start by considering the well-known example of ‘The Mouse's Tale’ from Lewis Carroll's Alice's Adventures in Wonderland (1865), shown in Figure 1. First, what is this? A poem, a narrative, a picture?
Think about how the physical shape of the printed text influences your interpretation. It might be helpful to imagine it laid out differently. If the text were set out in the usual, left-aligned layout of prose or some other poems, would you ‘read’ it differently? Why?
‘The Mouse's Tale’ is an example of concrete poetry, where words are arranged on the page in a significant way. When read aloud, its rhyme and metre are easier to access, but the layout initially challenges our attempts to read it as a poem. We consider concrete poetry in more detail in the next section.
‘The Mouse's Tale’ can be considered as literary not only because it comes from a highly valued and canonical work of literature, but because it can be analysed in terms of defamiliarisation and creative deviation in the discussion of Russian Formalism. The poem is eye-catching (‘made strange’) due to its unusual shape and can thus be seen as deviating from ‘normal’ layout conventions of poetry – even though readers of poetry are familiar, of course, with a range of such conventions. The text is also poetic in Formalist terms because it contains an obvious pun – in this case between semiotic modes. The verbal ‘tale’ being told by the mouse is represented by the visual ‘tail’.
Multimodality forces us as readers to focus on ‘the message for its own sake’ (Jakobson, 1960, p. 356) and pay fresh attention to departures from routinised conventions of literature or poetry. Victor Shklovsky believed that the purpose of art was exactly this – to counter our habitualised perceptions, to force us to notice:
Habitualisation devours works, clothes, furniture, one's wife, and the fear of war. ‘If the whole complex lives of many people go on unconsciously, then such lives are as if they had never been.’ And art exists that one may recover the sensation of life; it exists to make one feel things, to make the stone stony.
(Shklovsky, 1965, p. 12)
In an analogous way, then, Carroll's poem can be seen as poetic because it makes the mouse mousy. I will revisit ‘The Mouse's Tale’ later in the course and consider its value as a poem, as well as looking at other creative texts to draw out their multimodal nature. First however, it's important to set out a definition of ‘text’ that is broad enough for the purposes of this course.
A note about text
It will already be clear that to accept non-linguistic textual elements – such as layout and shape in ‘The Mouse's Tale’ – as meaningful and communicative, we will need to work with a definition of ‘text’ which will admit these as a valid focus for analysis. A simple and useful definition of ‘text’ for this purpose is the one provided by August Rubrecht:
A text is …
produced or modified
to communicate meaning.
This definition allows us to consider how meaning can be conveyed via a range of textual elements, as long as these elements are meaningful. It is important to differentiate things we perceive or see, from things we take meaning from:
A piece of driftwood on the beach is not an artifact, just a random object shaped and placed by natural forces. If a beachcomber takes it home, paints a face on it, and hangs it on a wall, it turns into a text communicating the beachcomber's ideas about what is interesting and beautiful. […] A text is purposeful. A line of footprints taking the left fork at a junction on a snowy trail is not a text. An arrow drawn in the snow and pointing left is.
A beautiful sunset is not a text. A painting or photograph of one is.
Later in the course I will return to problematise this definition of text, as not all texts which make a claim to be meaningful, or ‘art’, are necessarily interpreted as such. But I turn now to the field of semiotics and consider what it can offer us when we start to analyse multimodal texts.