I sometimes wonder if the chatter that came out of the Linguistics Department in the bowels of Dwinelle Hall on the UC Berkeley campus, where cognitive linguistics gestated and flowered, touched often on comics and comic books. The distinguished linguists there were, after all, North Americans who came of age in the early and mid-20th century, thinking hard about how language made meaning in our minds. I wonder if, when Charles Fillmore and George Lakoff and company came up with stuff like “Frame Semantics” and “Conceptual Metaphor,” they didn’t have comics tumbling around their heads, and no surprise that as they studied the languages and minds of others in an increasingly visual culture, found the influence of comics on minds and conceptions of mind.
Neil Cohn’s The Visual Language of Comics is subtitled, “Introduction to the Structure and Cognition of Sequential Images.” Neil Cohn, a cartoonist, found something valuable as an undergraduate at Berkeley in the fields of cognitive science and linguistics, something valuable about how we read not just comics (which he calls a particular sociocultural phenomenon, like any manifestation of certain media within a time and place) but what he terms visual language. And in the manner of linguists, The Visual Language of Comics sets out to understand how comics reading works as a system, specifically here in the mind of the reader. The book is the linguist’s comics theory book, an attempt to fulfill the potential that any student of linguistics must have imaged when reading McCloud or even Groensteen.
What makes visual language a language, even though Cohn emphasizes that comics “are not a language?” Sequential images, like language, operate with three characteristics that match language: modalities (ie sound, gesture, images created with intention), meanings (using reference), and grammaticality (ways and constraints for how they’re put together) are all involved to form a system of communication. Other forms of communication involve some of these elements, but not all three combined, as traditional language and visual language do. To me, the big insight here is that comics are grammatical. The fact that comics use visual modalities and reference meanings is fairly obvious, though by no means insignificant, and you can plumb the depths of those two aspects of language forever. But it’s the grammar of comics that interests me the most, and no surprise, demonstrating that systematicity is where Cohn turns next.
Cohn demonstrates the structures of visual language using a page from Kibuishi’s Copper and lays out terms like “navigational structure,” “visual morphology,” and “event structure.” Introducing these constructs takes Cohn far in making the parallels between language/linguistic categories (phonology, morphology, syntax, discourse, etc.) and visual language.
Cohn anticipates that the first section of the book will examine this. The second part of the book looks at cognitive research in order to detail how we comprehend visual languages. Then, in a third section, he turns towards the sociocultural distinctions, with chapters on American Visual Language (comics), Japanese Visual Language (manga), and an Aboriginal Australian Visual Language (sand drawings) that presents a striking comparison and contrast. Ultimately, Cohn marks out an agenda for visual language research, one he’s in the process of fulfilling.
So far, so good, and just what I expected as a comics reader and a (sort of) linguist. The evidence of Cohn’s training in Berkeley and at Tufts under Jackendoff shows. There’s some classic linguistics concepts applied (studying language as a system, with structures at various nested levels), but high value on cognition research (empirical evidence of how minds work to make meaning), and the necessary exploration of the (visual) language in use and how it is culturally contingent to some extent. Necessary, that is, because even though a non-linguist would suppose that “cognitive linguistics” is a kind of linguistics fixated on the brain, in fact that brand of linguistics is more interested in language-in-use than the linguistics influenced by Chomsky and others, more focused internally on the system of language itself.
One thing I wondered as I read, which I have to give more thought to, is how nested structures of visual language might involve different proficiencies, different sets of conventions and constraints, and different cognitive tasks. Cohn explains that comics involve two systems, the graphic/visual and the written/verbal. Of course, the former is understudied (thus, this book) and the latter is widely researched.
But for a moment, I wondered if there were more than two systems involved. Putting aside the written/visual, just within the visuals, I wondered if there was a system or structure for the drawings or images, relying on iconic representations (Cohn talks about CS Peirce’s semiotics in the next chapter) within each drawing, where a picture has to look like what it’s representing, say a drawing of a dog in a field within one panel. And then, was there a separate system or structure for the sequentiality and organization of images? And here, the representation would utilize Peirce’s indexical representation, so that the sign has some visual connection but not a one-to-one relationship with what it points to. For instance, a series of panels where each one gets more narrow, suggesting time crunching or speeding up (using the spatial-temporal analogy), or balloon placements that suggest the ordering (or maybe disorderliness) of chatter by a crowd of characters. Were there actually two visual systems at work in comics, one in the images and one in their organization and shaping into panels and pages?
Then I realized that, no, those are nested systems within one visual language, just like words (lexicon, morphology) have their own systems of signification, but these are nested within/beside syntax (organization of elements in an utterance, like a sentence or grammar), another system/structure. Complex sequences of images and various systems, working together, all at the same time. In plainer words, we can study vocabulary separately from we study grammar, and of course, vocabulary and grammar work differently, but we wouldn’t therefore say that grammar and vocabulary don’t belong together in a system of language. They’re both necessary components, and interlaced.
Those thoughts made me excited to read the rest of the book. I’m not very familiar with the research Cohn mentions, particularly the cognition research on people reading visuals. I look forward to delving into those chapters most of all. I’ll continue posting summaries and thoughts here as I read.