One of the advantages of doing a doctorate at a good university is that you end up reading major works from all big names in your field. If you are interested in communication studies, culture, education, journalism or media, you can’t escape Gunther Kress who died on June 20 at the age of 79.

When I was doing my doctorate in education at the University of Birmingham, UK, in the early 2000s, his three earlier books Reading Images: the grammar of graphic design (1996), Before Writing: Rethinking the path to literacy (1997), and Social Semiotics (1998) had already attracted the attentions of academics and students alike. When I was still there, his three more books came in quick succession: Multimodal Discourse: the modes and media of contemporary communication (2002), Multimodal Literacies (2003), and Literacy in the new media age (2003).

The last position Kress held was a professor of Education and English at the Institute of Education, University of London. He was also the Head of the School of Culture, Language and Communication.

Before moving to London, Gunther Kress had lived in Australia where he served as Dean of Communication and Cultural Studies at South Australia College of Advanced Education, and Assistant Vice-Chancellor at the University of Technology, Sydney. Throughout his professional life he moved from one discipline to another, and in the process became a truly multidimensional academic and scholar. Discourse analysis and learning theory were his primary concerns which later led to his explorations in multimodality and social semiotics.

So, what is discourse analysis and what does it have to do with us? But first we need to begin with discourse. It is a term used in a variety of different ways -- it can be a serious talk or a piece of writing to explain something. It can also be a verb, meaning to talk in an authoritative way. Sometimes discourse refers to speech and text refers to writing. In some other contexts, we understand discourse as a process and text as a product. At the most basic level, ‘discourse analysis’ attempts to identify and describe linguistic regularities and irregularities. That means utterances which cannot be accounted for at a grammatical level.

The analysis of discourse is the analysis of language in use, and creates a descriptive framework for how texts are organised. It is concerned with the study and interpretation of the relationship between language and the context in which it is used.

What Kress did was to go beyond normal discourse analysis to different modes of communication -- image, music, speech, writing -- and their effects on forms of learning and knowing. He was more interested in the changes and their effects and consequences -- brought by the shift in the major media of communication from the page to the screen.

Kress was not the first to claim that speech, visual and writing have different functions. Canadian philosopher, Marshall McLuhan (1910-1980) in his famous dictum -- the medium is the message -- had done it much earlier. What was new in Kress’s argument was that in the age of multimedia, due to the advanced technology, writing is no more the most important means of communication, especially in the internet where images and sounds are combined with written texts. Kress explored the effects of colours, images and words on the process of learning and knowing by adopting his theories to education.

The Italian literary critic, professor and writer Umberto Eco (1932-2016) had also taken keen interest in this subject. However, Kress went a couple of steps further from Eco and McLuhan, to focus on visual modality to analyse the grammatical rules of images. Though Kress had started as a linguist in the late 1970s, the shift of emphasis in his research from linguistic to visual communication was a deliberate one. He argued that in the age of multimedia, language is "no longer the carrier of all meaning" -- therefore one must look at other modes of communication.

Perhaps his most seminal work was Multimodal Discourse (2002) that he wrote with his colleague Leeuwen to present the multimodality approach. They explained this approach as dealing "with all the means we have for making meanings -- the modes of representation -- and consider their specific way of configuring the world". They focused on a variety of texts including videogames, CD ROMs, biology books and drawings. Through their research they showed that images, graphics, and even sounds are as important as language to convey a certain message.

The uses and constraints of different modes of communication in education was another area of their investigation in which they ranged from e-learning experiences to the changes in representation in the school textbooks within a multimodal perspective. Kress argued that one should understand the effects of all modes of communication to understand language and its uses such as images, speech and writing have different features and functions. To him "image is spatial and non-sequential whereas writing and speech are temporal and sequential", and understanding this difference is crucial for communication.

How do you apply this to the present-day world -- and to Pakistan? We know that in semiotics -- the study of signs -- a modality is a particular way in which information is to be encoded for presentation to humans. In making sense of a text, its interpreters make ‘modality judgement’ about it. They assess its accuracy or credibility within a given genre. So, let’s ask how Pakistani audience, readership or viewers make judgement about the world presented to them. If we understand this, we may comprehend why in our society intolerance is increasing, or why we become so jingoistic at the drop of a hat.

You may argue that this is not new. The genre of film has been around for over a century now which used multimodal communication including acting, colours, music, songs, sound, speech and visuals. But the propaganda value of films was limited. Now the new media has crossed all boundaries. For films you had to pay for what you watched, and people didn’t pay to watch sheer propaganda. Now the new media which is in your hands and constantly prompts you to watch and respond is much more pervasive and effective. You simply can’t escape from it.

In Pakistan you flip channels and you have to begin with the ones that preach just one religion. There are at least a dozen channels to arouse your religious feelings. The social media are also replete with intolerance and overly nationalistic and patriotic sentiments. The print media also have weekly editions that cater to the imagined or real religious requirements. Even Sunday editions are now increasingly becoming patriotic and religious. You will hardly find a multimodal representation of communication that promotes intolerance or plurality of religions and denominations in society, apart from some books or reports whose readership is pretty low.

Forces of intolerance are much more in number and are adept at using multimodal communication thanks to their abundance of resources; whereas the voices of democracy, human rights, toleration and harmony are lesser in number and devoid of any organised mechanism to propagate their views. Though the right to practice and preach religion is enshrined in most constitutions in the world, in countries such as Pakistan this right is enjoyed by the majority creed who is vocal and at times violent. In comparison, the minority denominations and sects are unable to use multimodal communication.

For example, the use of certain words is prohibited for places of worship; even the use of certain signs and emblems, or even certain architectural features are not allowed to be constructed, displayed or used. This is all semiotics or rather social semiotics as described by Gunther Kress.

Kress tells us that this ‘common cognitive system’ treats almost all conveyed meanings in the same way. If all signs are objects of perception, we may believe that auditory (listening), gustatory (taste), kinesthetic (muscular), olfactory (smell), tactile (touch), and visual (looking) all become part of that multimodal communication. Applied to Pakistan, if you listen to a weekly sermon by the likes of Tariq Jameel or to a talk delivered by Pervez Hoodbhoy, the availability and frequency of such auditory experience will determine the common cognitive system of society.

In gustatory matters, the sherbet you drink in Muharram; in kinesthetic, you may include the inordinate stress on discipline and physical exercise in state and non-state actors, which are mostly aimed at targeting ‘the enemy’; in olfactory, the use of fragrance (attar) and in the realm of touch you may include child abuse. All these are multimodal communications affecting the common cognitive system. The same applies to the use of visuals, such as the release of a patriotic song.

In all of the above mentioned multimodal communication where do you find the voices of harmony and toleration? Following the lead from Gunther Kress, if the democratic, liberal and progressive voices in Pakistan or in any other country for that matter are able to develop and implement a multimodal strategy of communication there may be some success.

For the time being the battle is one-sided.