These are further reflections on thoughts from a previous post to help me prepare my thoughts for a talk that I’m giving at the Small Press Expo in Bethesda, Maryland.

I look at comics a lot. But I don’t look at them the way I think you think I do.

Deep down I’m a linguist trying to understand language, particularly interested in visual language. I’m convinced that comics are attractive to us as a mode of expression because they allow our human eyes to read images that line up with the mental models that we create for how we “see” the world. You can think of cartoons as diagrams of our existing, procedural abstractions of the world and comics sequences as visual abstractions of how we model the passage of time in our minds.

Why do we cartoon the way we do? Is there a biological reason for it? I believe so.

I recently read an article on the first anatomically accurate mathematical model that explains how vision is possible. It was interesting. It’s a model that recognizes how we can create such seemingly rich visual depictions of the world in our mind despite there only being just 10 or so nerve cells that connect the retina to the visual cortex. The part that really got to me was the idea that these 10 Lateral Genticulate Nucleus cells are scarce and can’t do much. They send a pulse to the visual cortex when they detect a change from dark to light, or vice versa, in their tiny section of the visual field. And that’s it. On. Off.

Information from the eye passes through a bottleneck before it gets to the brain’s visual cortex, which heavily processes the sparse signal. - DVDP for Quanta Magazine

From blips of information, we build the world in our minds. This struck me.

In essence, the world that we “see” is a reconstruction that your mind makes up. Therefore, the neural cortex must process these simple signals in a way that builds + creates representational models of the world. I imagine that there might be a procedural grammar to vision. What does the process of this reconstruction look like? What abstractions are used to model it in our “mind’s eye”? How can we understand what this might look like?

Figure 3.7 Reading in the Brain - Stanislas Dehaene Figure 3.7

I believe that comics offer us a mirror to see an expression of the tendencies and limits of our biology. That we can glean the beginnings of what such a grammar may look like by analyzing the cross-cultural visual syntaxes of comics from around the world. It can offer theories that can guide further research. In a sense, comics are especially equipped to help us see how we see. I wish to clarify my use of the word equipped. Comics don’t exist outside of us. We created them. As such, it is our mental apparatus that has equipped them with this capacity.

What is especially interesting to me with comics as a lens through which to observe human thought and perception is that it’s a universally accessible and naturally occurring petri dish. The reason for this is that comics, in my mind, happens to be an egalitarian stage of expression that allows for us as individuals and also as a society to make high-level abstractions with accessible, immediate tools and technologies. This mode of creation cuts across race and class on Earth. All you need is a mark-making tool and a surface to make marks on.

Comics help us see how we are collectively and individually “seeing”, most notably how we experience and structurally breakdown our experience of space and time. Through comics, we break down the visible world into simple marks, into images that encode reality. As I mentioned earlier, my understanding of cartooning is that it diagrams our existing, procedural abstractions of the world. And comics sequences visually abstract how we model the passage of time in our minds. We break it all down on the page and then a reader reconstructs a model of what we were trying to express.

I believe there is usually a conscious or intuitive understanding of this process when a creator is making a comic. It is when this process is understood as a communicative act that successful comics emerge.

This is important in that it provides an analogous process to how we process sound to understand oral language and music. We don’t hear words, or notes, we decode them from complex signals. This means that we have a way of decoding. And in this decoding, there is an order, a grammar. That is perhaps innate or developed over time through socialization.

Ray Jackendoff’s research is especially interesting in this front - He has always straddled the boundary between generative linguistics and cognitive linguistics, committed to both the existence of an innate universal grammar (an important thesis of generative linguistics) and to giving an account of language that is consistent with the current understanding of the human mind and cognition (the main purpose of cognitive linguistics).

Ray Jackendoff - Foundations of Language: Structure of a simple sentence. Fig. 1.1 Ray Jackendoff - Foundations of Language: Structure of a simple sentence. Fig. 1.1

So what? How does affect us on a level of readers and makers of comics?

I find that an immediate use for this idea is that it helps us see comics as a cognitive bridge. It is a cultural tool that allows us to cross the gulf and better understand others. Particularly, it helps us understand how others experience and perceive the world. If it is true that comics allow us to see the ways that others encode their perception of reality into the two-dimensional substrate, we can better understand how their minds genuinely see and construct their experiences of the world.

When we look at someone’s comics, or even our own, we can ask: What is there? What is not? Why might this be?

Through comics, we can “step into someone else’s shoes” at a level that is unavailable through other media, including writing. That is something that I find fascinating.